Method: Evaluated several open-source LLMs using zero-shot and few-shot learning paradigms across sentiment analysis, named entity recognition, reading comprehension, and question answering tasks using Persian datasets (ParsiNLU, ArmanEmo) with metrics like Accuracy, F1-score, BLEU, and ROUGE.

Result: Gemma 2 consistently outperformed other models across nearly all tasks in both learning paradigms, with strong performance in complex reasoning tasks. Most models struggled with token-level understanding tasks like Named Entity Recognition.

Conclusion: This study provides valuable insights into LLM performance in Persian and establishes a benchmark for future model development, contributing to multilingual LLM research.

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities across numerous languages; however, their effectiveness in low-resource languages like Persian requires thorough investigation. This paper presents a comprehensive benchmark of several open-source LLMs for Persian Natural Language Processing (NLP) tasks, utilizing both zero-shot and few-shot learning paradigms. We evaluate models across a range of tasks including sentiment analysis, named entity recognition, reading comprehension, and question answering, using established Persian datasets such as ParsiNLU and ArmanEmo. Our methodology encompasses rigorous experimental setups for both zero-shot and few-shot scenarios, employing metrics such as Accuracy, F1-score, BLEU, and ROUGE for performance evaluation. The results reveal that Gemma 2 consistently outperforms other models across nearly all tasks in both learning paradigms, with particularly strong performance in complex reasoning tasks. However, most models struggle with token-level understanding tasks like Named Entity Recognition, highlighting specific challenges in Persian language processing. This study contributes to the growing body of research on multilingual LLMs, providing valuable insights into their performance in Persian and offering a benchmark for future model development.

Method: Evaluated 4 large language models (GPT-3.5, GPT-4o, Llama 3.2, Gemini 1.5) and BioBERT on classifying 762 unique cancer diagnoses (326 ICD codes, 436 free-text entries) from 3456 patient records into 14 predefined categories. Two oncology experts validated the classifications.

Result: BioBERT achieved highest weighted macro F1-score for ICD codes (84.2) and matched GPT-4o in ICD code accuracy (90.8). GPT-4o outperformed BioBERT on free-text diagnoses in weighted macro F1-score (71.8 vs 61.5) and accuracy (81.9 vs 81.6). Other models showed lower performance. Common errors involved confusion between metastasis and CNS tumors, and ambiguous clinical terminology.

Conclusion: Current AI model performance is sufficient for administrative and research applications but not reliable enough for clinical use. Clinical applications will require standardized documentation practices and robust human oversight for high-stakes decision-making.

Abstract: Electronic health records contain inconsistently structured or free-text data, requiring efficient preprocessing to enable predictive health care models. Although artificial intelligence-driven natural language processing tools show promise for automating diagnosis classification, their comparative performance and clinical reliability require systematic evaluation. The aim of this study is to evaluate the performance of 4 large language models (GPT-3.5, GPT-4o, Llama 3.2, and Gemini 1.5) and BioBERT in classifying cancer diagnoses from structured and unstructured electronic health records data. We analyzed 762 unique diagnoses (326 International Classification of Diseases (ICD) code descriptions, 436free-text entries) from 3456 records of patients with cancer. Models were tested on their ability to categorize diagnoses into 14predefined categories. Two oncology experts validated classifications. BioBERT achieved the highest weighted macro F1-score for ICD codes (84.2) and matched GPT-4o in ICD code accuracy (90.8). For free-text diagnoses, GPT-4o outperformed BioBERT in weighted macro F1-score (71.8 vs 61.5) and achieved slightly higher accuracy (81.9 vs 81.6). GPT-3.5, Gemini, and Llama showed lower overall performance on both formats. Common misclassification patterns included confusion between metastasis and central nervous system tumors, as well as errors involving ambiguous or overlapping clinical terminology. Although current performance levels appear sufficient for administrative and research use, reliable clinical applications will require standardized documentation practices alongside robust human oversight for high-stakes decision-making.

Method: Proposes proactively incorporating HLV into preference datasets as a core design principle (Selbstzweck), outlining actionable steps to preserve human pluralism in AI systems.

Result: Position paper arguing for paradigm shift in how HLV is treated - from noise to be eliminated to essential signal representing human value pluralism.

Conclusion: HLV should be treated as a fundamental goal (Selbstzweck) in AI system design, preserving human pluralism rather than flattening diverse perspectives into false universal agreement.

Abstract: Human Label Variation (HLV) refers to legitimate disagreement in annotation that reflects the genuine diversity of human perspectives rather than mere error. For decades, HLV in NLP was dismissed as noise to be discarded, and only slowly over the last decade has it been reframed as a signal for improving model robustness. With the rise of large language models (LLMs), where post-training on human feedback has become central to model alignment, the role of HLV has become increasingly consequential. Yet current preference-learning datasets routinely aggregate multiple annotations into a single label, thereby flattening diverse perspectives into a false universal agreement and erasing precisely the pluralism of human values that alignment aims to preserve. In this position paper, we argue that preserving HLV as an embodiment of human pluralism must be treated as a Selbstzweck - a goal it self when designing AI systems. We call for proactively incorporating HLV into preference datasets and outline actionable steps towards it.

Method: Created MEDEQUALQA benchmark with clinical vignettes expanded into single-CSC ablations, producing three parallel datasets (69,000 total items). Evaluated GPT-4.1 model using Semantic Textual Similarity (STS) between reasoning traces across pronoun variants.

Result: Overall high similarity (mean STS >0.80) but revealed consistent localized divergences in cited risk factors, guideline anchors, and differential ordering, even when final diagnoses remained unchanged.

Conclusion: MEDEQUALQA provides a controlled diagnostic setting for auditing reasoning stability in medical AI, highlighting clinically relevant bias loci that may cascade into inequitable care.

Abstract: Large language models (LLMs) are increasingly deployed in clinical decision support, yet subtle demographic cues can influence their reasoning. Prior work has documented disparities in outputs across patient groups, but little is known about how internal reasoning shifts under controlled demographic changes. We introduce MEDEQUALQA, a counterfactual benchmark that perturbs only patient pronouns (he/him, she/her, they/them) while holding critical symptoms and conditions (CSCs) constant. Each clinical vignette is expanded into single-CSC ablations, producing three parallel datasets of approximately 23,000 items each (69,000 total). We evaluate a GPT-4.1 model and compute Semantic Textual Similarity (STS) between reasoning traces to measure stability across pronoun variants. Our results show overall high similarity (mean STS >0.80), but reveal consistent localized divergences in cited risk factors, guideline anchors, and differential ordering, even when final diagnoses remain unchanged. Our error analysis highlights certain cases in which the reasoning shifts, underscoring clinically relevant bias loci that may cascade into inequitable care. MEDEQUALQA offers a controlled diagnostic setting for auditing reasoning stability in medical AI.

Method: Uses Classifier-Augmented Generation (CAG) combining utterance decomposition with classifier and stage-specific few-shot prompting for stage predictions, then connects stages using edge prediction and infers properties from sub-utterance context.

Result: Shows improved accuracy and efficiency compared to single-prompt and agentic baselines, while substantially reducing token usage. Capable of end-to-end workflow generation with robust validation.

Conclusion: This is the first system with detailed evaluation across stage prediction, edge layout, and property generation for natural-language-driven ETL authoring, offering modular and interpretable architecture.

Abstract: ETL (Extract, Transform, Load) tools such as IBM DataStage allow users to visually assemble complex data workflows, but configuring stages and their properties remains time consuming and requires deep tool knowledge. We propose a system that translates natural language descriptions into executable workflows, automatically predicting both the structure and detailed configuration of the flow. At its core lies a Classifier-Augmented Generation (CAG) approach that combines utterance decomposition with a classifier and stage-specific few-shot prompting to produce accurate stage predictions. These stages are then connected into non-linear workflows using edge prediction, and stage properties are inferred from sub-utterance context. We compare CAG against strong single-prompt and agentic baselines, showing improved accuracy and efficiency, while substantially reducing token usage. Our architecture is modular, interpretable, and capable of end-to-end workflow generation, including robust validation steps. To our knowledge, this is the first system with a detailed evaluation across stage prediction, edge layout, and property generation for natural-language-driven ETL authoring.

Method: Used two game-theoretic frameworks: Cheap Talk signaling game and Peer Evaluation adversarial game, testing four frontier LLM models (GPT-4o, Gemini-2.5-pro, Claude-3.7-Sonnet, Llama-3.3-70b) with and without explicit prompting, analyzing scheming tactics through chain-of-thought reasoning.

Result: When prompted, most models achieved near-perfect scheming performance, especially Gemini-2.5-pro and Claude-3.7-Sonnet. Without prompting, all models chose deception over confession in Peer Evaluation (100% rate), and models choosing to scheme in Cheap Talk succeeded at 95-100% rates.

Conclusion: The findings highlight the need for robust evaluations using high-stakes game-theoretic scenarios in multi-agent settings to properly assess LLM agents’ strategic deception capabilities.

Abstract: As large language model (LLM) agents are deployed autonomously in diverse contexts, evaluating their capacity for strategic deception becomes crucial. While recent research has examined how AI systems scheme against human developers, LLM-to-LLM scheming remains underexplored. We investigate the scheming ability and propensity of frontier LLM agents through two game-theoretic frameworks: a Cheap Talk signaling game and a Peer Evaluation adversarial game. Testing four models (GPT-4o, Gemini-2.5-pro, Claude-3.7-Sonnet, and Llama-3.3-70b), we measure scheming performance with and without explicit prompting while analyzing scheming tactics through chain-of-thought reasoning. When prompted, most models, especially Gemini-2.5-pro and Claude-3.7-Sonnet, achieved near-perfect performance. Critically, models exhibited significant scheming propensity without prompting: all models chose deception over confession in Peer Evaluation (100% rate), while models choosing to scheme in Cheap Talk succeeded at 95-100% rates. These findings highlight the need for robust evaluations using high-stakes game-theoretic scenarios in multi-agent settings.

Method: Used collaborative protocol with multiple gpt-5 instances (provers and verifiers), with final proofs formally verified by Lean proof assistant and human-checked for premise-conclusion conformance.

Result: Successfully solved 5 out of 6 IMO 2025 problems and proved one-third (22 out of 66) number theory conjectures from Cohen’s 2025 paper.

Conclusion: The collaborative approach with formal verification enables reliable theorem proving by large language models, showing significant progress in AI mathematical reasoning capabilities.

Abstract: During 2024 and 2025 the discussion about the theorem-proving capabilities of large language models started reporting interesting success stories, mostly to do with difficult exercises (such as problems from the International Mathematical Olympiad), but also with conjectures [Feldman & Karbasi, arXiv:2509.18383v1] formulated for the purpose of verifying whether the artificial intelligence could prove it. In this paper we report a theorem proving feat achieved by ChatGPT by using a protocol involving different prover and verifier instances of the gpt-5 model working collaboratively. To make sure that the produced proofs do not suffer from hallucinations, the final proof is formally verified by the lean proof assistant, and the conformance of premises and conclusion of the lean code is verified by a human. Our methodology was able to solve five out of six 2025 IMO problems, and close a third of the sixty-six number theory conjectures in [Cohen, Journal of Integer Sequences, 2025].

Method: Uses discrete flow paradigms with metric-induced probability paths and kinetic optimal velocities to achieve unified modeling. Trained on large-scale interleaved text, image, video, and audio data with concise unified representations instead of task-decoupled designs.

Result: Delivers competitive performance on multimodal generation and understanding benchmarks, outperforms prior unified models in multi-turn multimodal interaction and cross-modal retrieval, with enhanced response efficiency.

Conclusion: NExT-OMNI demonstrates architectural advantages as a next-generation multimodal foundation model, with released training details, data protocols, code, and model checkpoints to advance further research.

Abstract: Next-generation multimodal foundation models capable of any-to-any cross-modal generation and multi-turn interaction will serve as core components of artificial general intelligence systems, playing a pivotal role in human-machine interaction. However, most existing multimodal models remain constrained by autoregressive architectures, whose inherent limitations prevent a balanced integration of understanding and generation capabilities. Although hybrid and decoupling strategies have been explored to address these tasks within unified frameworks separately, their redundant, non-integrated designs limit their applicability to broader scenarios, such as cross-modal retrieval.In this work, we introduce NExT-OMNI, an open-source omnimodal foundation model that achieves unified modeling through discrete flow paradigms. By leveraging metric-induced probability paths and kinetic optimal velocities, NExT-OMNI natively supports any-to-any understanding and generation with enhanced response efficiency, while enabling broader application scenarios through concise unified representations rather than task-decoupled designs. Trained on large-scale interleaved text, image, video, and audio data, NExT-OMNI delivers competitive performance on multimodal generation and understanding benchmarks, while outperforming prior unified models in multi-turn multimodal interaction and cross-modal retrieval, highlighting its architectural advantages as a next-generation multimodal foundation model. To advance further research, we release training details, data protocols, and open-source both the code and model checkpoints.

Method: Formulates the task as a Markov Decision Process where an agent interacts with a database for execution feedback and dialogue memory for coherence verification, performing iterative propose-execute-verify-refine cycles.

Result: Experiments on COSQL and SPARC datasets show MTSQL-R1 consistently outperforms strong baselines, demonstrating the effectiveness of environment-driven verification and memory-guided refinement.

Conclusion: The framework highlights the importance of execution feedback and persistent dialogue memory for conversational semantic parsing, with full implementation details to be released for community research.

Abstract: Multi-turn Text-to-SQL aims to translate a user’s conversational utterances into executable SQL while preserving dialogue coherence and grounding to the target schema. However, most existing systems only regard this task as a simple text translation task and follow a short-horizon paradigm, generating a query per turn without execution, explicit verification, and refinement, which leads to non-executable or incoherent outputs. We present MTSQL-R1, an agentic training framework for long-horizon multi-turn Text-to-SQL. We cast the task as a Markov Decision Process (MDP) in which an agent interacts with (i) a database for execution feedback and (ii) a persistent dialogue memory for coherence verification, performing an iterative propose to execute -> verify -> refine cycle until all checks pass. Experiments on COSQL and SPARC demonstrate that MTSQL-R1 consistently outperforms strong baselines, highlighting the importance of environment-driven verification and memory-guided refinement for conversational semantic parsing. Full recipes (including code, trained models, logs, reasoning trajectories, etc.) will be released after the internal review to contribute to community research.

Method: A moderation-oriented guideline repurposing method that transforms human-written guidelines into clear directives for LLMs through an LLM moderation process.

Result: Experiments using the NCBI Disease Corpus show that repurposed guidelines can effectively guide LLM annotators, though several practical challenges were revealed.

Conclusion: The workflow shows potential for supporting scalable and cost-effective refinement of annotation guidelines and automated annotation systems.

Abstract: This study investigates how existing annotation guidelines can be repurposed to instruct large language model (LLM) annotators for text annotation tasks. Traditional guidelines are written for human annotators who internalize training, while LLMs require explicit, structured instructions. We propose a moderation-oriented guideline repurposing method that transforms guidelines into clear directives for LLMs through an LLM moderation process. Using the NCBI Disease Corpus as a case study, our experiments show that repurposed guidelines can effectively guide LLM annotators, while revealing several practical challenges. The results highlight the potential of this workflow to support scalable and cost-effective refinement of annotation guidelines and automated annotation.

Method: Route-then-align principle: first learns task-aware routing, then aligns mode-specific trajectories under shared backbone. Introduces instant mode for direct responses to simple queries, plus Adaptive Policy Optimization (APO) for adaptive sampling and cost-regularized rewards.

Result: Achieves SOTA on benchmarks: 13.4% on BrowseComp, 70.4% on AIME25, 16.7% on HLE. Adaptive execution reduces cost to $0.00487 per correct answer - 45.2% cheaper than reasoning-only and 33.5% cheaper than agentic-only models.

Conclusion: A²FM successfully unifies reasoning and agentic capabilities while significantly improving cost efficiency through adaptive routing, maintaining competitive accuracy across diverse benchmarks.

Abstract: Large language models split into two families: reasoning-centric LLMs, which strengthen internal chain-of-thought reasoning but cannot invoke external tools, and agentic LLMs, which learn to interact with environments and leverage tools but often lag in deep reasoning. This divide arises from fundamentally different training objectives, leading to mismatched strengths and inefficiency on simple queries, where both families tend to overthink or over-call tools. In this work, we present Adaptive Agent Foundation Model (A\textsuperscript{2}FM), a unified framework that follows a route-then-align principle: the model first learns task-aware routing and then aligns mode-specific trajectories under a shared backbone. To address the inefficiency gap, we introduce a third mode-instant-that handles simple queries directly, preventing unnecessary reasoning or tool calls while complementing the agentic and reasoning modes. To jointly enhance accuracy and efficiency, we propose Adaptive Policy Optimization (APO), which enforces adaptive sampling across modes and applies a cost-regularized reward. On the 32B scale, A\textsuperscript{2}FM achieves 13.4% on BrowseComp, 70.4% on AIME25, and 16.7% on HLE, setting new SOTA among comparable models and performing competitively with frontier LLMs across agentic, reasoning, and general benchmarks. Notably, the adaptive execution achieves a cost of pass of only $0.00487 per correct answer-cutting cost by 45.2% relative to reasoning and 33.5% relative to agentic, thus delivering substantially higher cost efficiency while maintaining comparable accuracy.

Method: SALAD combines cross-modal distillation with targeted synthetic data through active selection to improve speech-text alignment while mitigating forgetting of text capabilities.

Result: Applied to 3B and 7B LLMs, SALAD achieves competitive performance with strong open-weight models across knowledge, language understanding, and reasoning benchmarks using over 10x less speech data.

Conclusion: SALAD provides a data-efficient alternative for closing the text-speech understanding gap in speech-adapted LLMs, addressing both forgetting and cross-modal misalignment issues.

Abstract: Large Language Models (LLMs) can be adapted to extend their text capabilities to speech inputs. However, these speech-adapted LLMs consistently underperform their text-based counterparts–and even cascaded pipelines–on language understanding tasks. We term this shortfall the text-speech understanding gap: the performance drop observed when a speech-adapted LLM processes spoken inputs relative to when the original text-based LLM processes the equivalent text. Recent approaches to narrowing this gap either rely on large-scale speech synthesis of text corpora, which is costly and heavily dependent on synthetic data, or on large-scale proprietary speech datasets, which are not reproducible. As a result, there remains a need for more data-efficient alternatives for closing the text-speech understanding gap. In this work, we analyze the gap as driven by two factors: (i) forgetting of text capabilities during adaptation, and (ii) cross-modal misalignment between speech and text. Based on this analysis, we introduce SALAD–Sample-efficient Alignment with Learning through Active selection and cross-modal Distillation–which combines cross-modal distillation with targeted synthetic data to improve alignment while mitigating forgetting. Applied to 3B and 7B LLMs, SALAD achieves competitive performance with a strong open-weight model across broad-domain benchmarks in knowledge, language understanding, and reasoning, while training on over an order of magnitude less speech data from public corpora.

Method: Uses chunk-level claim extraction with confidence-based pre-verification to reduce web search costs, and collects document-level evidence from webpages with selective retrieval during verification.

Result: Achieves highest alignment with human evaluation and efficiency among existing baselines, as demonstrated through extensive experiments on manually annotated benchmarks.

Conclusion: FastFact provides a reliable, efficient, and effective solution for evaluating factuality of long-form LLM generations, addressing key limitations of previous approaches.

Abstract: Evaluating the factuality of long-form generations from Large Language Models (LLMs) remains challenging due to accuracy issues and costly human assessment. Prior efforts attempt this by decomposing text into claims, searching for evidence, and verifying claims, but suffer from critical drawbacks: (1) inefficiency due to complex pipeline components unsuitable for long LLM outputs, and (2) ineffectiveness stemming from inaccurate claim sets and insufficient evidence collection of one-line snippets. To address these limitations, we propose \name, a fast and strong evaluation framework that achieves the highest alignment with human evaluation and efficiency among existing baselines. \name first employs chunk-level claim extraction integrated with confidence-based pre-verification, significantly reducing the cost of web searching and inference calling while ensuring reliability. For searching and verification, it collects document-level evidence from crawled webpages and selectively retrieves it during verification, addressing the evidence insufficiency problem in previous pipelines. Extensive experiments based on an aggregated and manually annotated benchmark demonstrate the reliability of \name in both efficiently and effectively evaluating the factuality of long-form LLM generations. Code and benchmark data is available at https://github.com/Yingjia-Wan/FastFact.

Method: Comprehensive literature review analyzing academic research, web platforms, and commercial applications from the past two decades, synthesizing findings about prevailing approaches.

Result: Reveals fundamental misalignment in ASR-based approaches that prioritize lexical recognition over qualitative acoustic assessment, suffering from data dependency, demographic biases, and inability to provide diagnostically useful feedback.

Conclusion: Proposes a paradigm shift to knowledge-centric computational frameworks using anticipatory acoustic modeling based on canonical Tajweed rules and articulation points, recommending hybrid systems integrating deep linguistic knowledge with advanced audio analysis.

Abstract: The sacred practice of Quranic recitation (Tajweed), governed by precise phonetic, prosodic, and theological rules, faces significant pedagogical challenges in the modern era. While digital technologies promise unprecedented access to education, automated tools for recitation evaluation have failed to achieve widespread adoption or pedagogical efficacy. This literature review investigates this critical gap, conducting a comprehensive analysis of academic research, web platforms, and commercial applications developed over the past two decades. Our synthesis reveals a fundamental misalignment in prevailing approaches that repurpose Automatic Speech Recognition (ASR) architectures, which prioritize lexical recognition over qualitative acoustic assessment and are plagued by data dependency, demographic biases, and an inability to provide diagnostically useful feedback. Critiquing these data–driven paradigms, we argue for a foundational paradigm shift towards a knowledge-centric computational framework. Capitalizing on the immutable nature of the Quranic text and the precisely defined rules of Tajweed, we propose that a robust evaluator must be architected around anticipatory acoustic modeling based on canonical rules and articulation points (Makhraj), rather than relying on statistical patterns learned from imperfect and biased datasets. This review concludes that the future of automated Quranic evaluation lies in hybrid systems that integrate deep linguistic knowledge with advanced audio analysis, offering a path toward robust, equitable, and pedagogically sound tools that can faithfully support learners worldwide.

Method: Curated datasets from web resources in target languages across ten image categories, using automatic evaluation and native speaker assessment of VLM-generated responses and rationales.

Result: GPT-4o achieved best performance with 90.8% overall accuracy but still lags human performance by 6.7%, with larger gaps for open-source models. Performance disparities were observed across languages and tasks.

Conclusion: VLURes highlights significant performance gaps between VLMs and human capabilities, emphasizing the need for multilingual benchmarks to advance intelligent agents’ multi-modal visual reasoning abilities.

Abstract: Vision Language Models (VLMs) are pivotal for advancing perception in intelligent agents. Yet, evaluation of VLMs remains limited to predominantly English-centric benchmarks in which the image-text pairs comprise short texts. To evaluate VLM fine-grained abilities, in four languages under long-text settings, we introduce a novel multilingual benchmark VLURes featuring eight vision-and-language tasks, and a pioneering unrelatedness task, to probe the fine-grained Visual and Linguistic Understanding capabilities of VLMs across English, Japanese, and low-resource languages, Swahili, and Urdu. Our datasets, curated from web resources in the target language, encompass ten diverse image categories and rich textual context, introducing valuable vision-language resources for Swahili and Urdu. By prompting VLMs to generate responses and rationales, evaluated automatically and by native speakers, we uncover performance disparities across languages and tasks critical to intelligent agents, such as object recognition, scene understanding, and relationship understanding. We conducted evaluations of ten VLMs with VLURes. The best performing model, GPT-4o, achieves an overall accuracy of 90.8% and lags human performance by 6.7%, though the gap is larger for open-source models. The gap highlights VLURes’ critical role in developing intelligent agents to tackle multi-modal visual reasoning.

Method: Combines progressive token pruning, sparse attention, and dynamic early exiting into a single architecture, with automated benchmarking across GLUE tasks including SST-2, QQP, and MNLI.

Result: EAT achieves slightly higher accuracy than optimized DistilBERT baseline on SST-2, though combining mechanisms can increase latency in shallow six-layer models.

Conclusion: EAT demonstrates the potential of dynamic computation for latency-sensitive NLP and serves as an open, end-to-end reproducible framework for further research on adaptive transformers.

Abstract: The Efficient Adaptive Transformer (EAT) framework unifies three adaptive efficiency techniques - progressive token pruning, sparse attention, and dynamic early exiting - into a single, reproducible architecture for input-adaptive inference. EAT provides an open-source benchmarking pipeline that automates data processing, timing, and ablation across GLUE tasks (SST-2, QQP, MNLI). Although this empirical study finds that combining these mechanisms can increase latency in shallow six-layer models, it demonstrates that EAT achieves slightly higher accuracy than the optimized DistilBERT baseline on SST-2, illustrating the potential of dynamic computation for latency-sensitive NLP. The main contribution is the open, end-to-end reproducible framework - complete with scripts, CSV logging, and analysis utilities - intended to serve as a community tool for further research on adaptive transformers.

Method: Created EduDial dataset through teacher-student agent interactions, guided by Bloom’s taxonomy and 10 questioning strategies. Developed EduDial-LLM 32B via training and proposed 11-dimensional evaluation framework.

Result: Experiments on 17 LLMs show most struggle in student-centered teaching, while EduDial-LLM achieves significant gains and consistently outperforms all baselines across all metrics.

Conclusion: EduDial provides a valuable benchmark for evaluating LLMs in educational dialogue scenarios, and EduDial-LLM demonstrates superior performance in student-centered teaching contexts.

Abstract: Recently, several multi-turn dialogue benchmarks have been proposed to evaluate the conversational abilities of large language models (LLMs). As LLMs are increasingly recognized as a key technology for advancing intelligent education, owing to their ability to deeply understand instructional contexts and provide personalized guidance, the construction of dedicated teacher-student dialogue benchmarks has become particularly important. To this end, we present EduDial, a comprehensive multi-turn teacher-student dialogue dataset. EduDial covers 345 core knowledge points and consists of 34,250 dialogue sessions generated through interactions between teacher and student agents. Its design is guided by Bloom’s taxonomy of educational objectives and incorporates ten questioning strategies, including situational questioning, zone of proximal development (ZPD) questioning, and metacognitive questioning-thus better capturing authentic classroom interactions. Furthermore, we design differentiated teaching strategies for students at different cognitive levels, thereby providing more targeted teaching guidance. Building on EduDial, we further develop EduDial-LLM 32B via training and propose an 11-dimensional evaluation framework that systematically measures the teaching abilities of LLMs, encompassing both overall teaching quality and content quality. Experiments on 17 mainstream LLMs reveal that most models struggle in student-centered teaching scenarios, whereas our EduDial-LLM achieves significant gains, consistently outperforming all baselines across all metrics. The code is available at https://github.com/Mind-Lab-ECNU/EduDial/tree/main.

Method: Evaluated LLM responses to inquiry personas - user profiles conveying identity, expertise, or belief attributes - and measured effects on factual question answering accuracy and failure modes.

Result: Inquiry persona cues meaningfully alter QA accuracy and trigger failure modes including refusals, hallucinated limitations, and role confusion, compromising factual reliability.

Conclusion: Model sensitivity to user framing undermines factual reliability, and inquiry persona testing serves as an effective robustness evaluation tool for LLMs.

Abstract: Large Language Models (LLMs) should answer factual questions truthfully, grounded in objective knowledge, regardless of user context such as self-disclosed personal information, or system personalization. In this paper, we present the first systematic evaluation of LLM robustness to inquiry personas, i.e. user profiles that convey attributes like identity, expertise, or belief. While prior work has primarily focused on adversarial inputs or distractors for robustness testing, we evaluate plausible, human-centered inquiry persona cues that users disclose in real-world interactions. We find that such cues can meaningfully alter QA accuracy and trigger failure modes such as refusals, hallucinated limitations, and role confusion. These effects highlight how model sensitivity to user framing can compromise factual reliability, and position inquiry persona testing as an effective tool for robustness evaluation.

Method: Uses Yahoo! Answers multi-country dataset to introduce CUEST framework, measuring human-model alignment in curiosity through linguistic style analysis, topic preference analysis, and social science constructs. Explores fine-tuning strategies to induce curiosity in LLMs.

Result: LLMs flatten cross-cultural diversity in curiosity, aligning more closely with Western countries’ expressions. Fine-tuning strategies can narrow the human-model alignment gap by up to 50%. Curiosity demonstrates practical value for LLM adaptability across cultures.

Conclusion: Curiosity is important for future NLP research, particularly for improving LLM adaptability across different cultural contexts. The findings highlight the need to address cultural biases in LLM curiosity expressions.

Abstract: Recent advances in Large Language Models (LLMs) have expanded their role in human interaction, yet curiosity – a central driver of inquiry – remains underexplored in these systems, particularly across cultural contexts. In this work, we investigate cultural variation in curiosity using Yahoo! Answers, a real-world multi-country dataset spanning diverse topics. We introduce CUEST (CUriosity Evaluation across SocieTies), an evaluation framework that measures human-model alignment in curiosity through linguistic (style), topic preference (content) analysis and grounding insights in social science constructs. Across open- and closed-source models, we find that LLMs flatten cross-cultural diversity, aligning more closely with how curiosity is expressed in Western countries. We then explore fine-tuning strategies to induce curiosity in LLMs, narrowing the human-model alignment gap by up to 50%. Finally, we demonstrate the practical value of curiosity for LLM adaptability across cultures, showing its importance for future NLP research.

Method: Insert an intermediate qualifier model between draft and target models to bridge distributional gaps, using hierarchical decoding with fuzzy acceptance criteria and relaxed divergence thresholds at each stage.

Result: Achieves up to 1.91x generation speed over standard SD, reaching 124 tokens/second on RTX 4090. With 1B draft and 8B target models, minimally trades quality for improved throughput.

Conclusion: PyramidSD offers a practical approach to enhance speculative decoding efficiency that can be readily applied to existing inference pipelines.

Abstract: Speculative Decoding (SD) accelerates inference in large language models by using a smaller draft model to propose tokens, which are then verified by a larger target model. However, the throughput gains of SD are fundamentally limited by a trade-off between draft model size and token acceptance: smaller draft models generate tokens more quickly but exhibit greater divergence from the target model, resulting in lower acceptance rates and reduced speedups. We introduce Pyramid Speculative Decoding (PyramidSD), an extension of SD that inserts an intermediate qualifier model between the draft and target to bridge the distributional gap in output predictions, allowing smaller model to be used for drafting. This hierarchical decoding strategy improves alignment across models, enabling higher acceptance rates and allowing the use of significantly smaller draft models without sacrificing overall performance. PyramidSD builds on fuzzy acceptance criteria to support relaxed divergence thresholds at each stage, improving throughput. In experiments, PyramidSD achieves up to 1.91x generation speed over standard SD, reaching 124 tokens per second on a consumer GPU (RTX 4090). In small-memory settings with a 1B-parameter draft model and an 8B target model, PyramidSD minimally trades target model quality for improved throughput. Overall, PyramidSD offers a practical approach to enhancing speculative decoding efficiency and can be readily applied to existing inference pipelines.

Method: Used red teaming methodology to systematically evaluate LLM safety mechanisms against persona-targeted prompts, creating AI-TRAITS dataset with 1.6M texts from 8 LLMs across 4 languages (English, Russian, Portuguese, Hindi) using 324 disinformation narratives and 150 persona profiles.

Result: Personalization strategies significantly increase jailbreak likelihood for all LLMs, alter linguistic/rhetorical patterns, and amplify persuasiveness of false narratives. The study exposed critical vulnerabilities in current LLM safety mechanisms.

Conclusion: The findings reveal critical vulnerabilities in state-of-the-art LLMs and provide foundation for improving safety alignment and detection strategies in multilingual and cross-demographic contexts.

Abstract: The human-like proficiency of Large Language Models (LLMs) has brought concerns about their potential misuse for generating persuasive and personalised disinformation at scale. While prior work has demonstrated that LLMs can generate disinformation, specific questions around persuasiveness and personalisation (generation of disinformation tailored to specific demographic attributes) remain largely unstudied. This paper presents the first large-scale, multilingual empirical study on persona-targeted disinformation generation by LLMs. Employing a red teaming methodology, we systematically evaluate the robustness of LLM safety mechanisms to persona-targeted prompts. A key novel result is AI-TRAITS (AI-generaTed peRsonAlIsed disinformaTion dataSet), a new dataset of around 1.6 million texts generated by eight state-of-the-art LLMs. AI-TRAITS is seeded by prompts that combine 324 disinformation narratives and 150 distinct persona profiles, covering four major languages (English, Russian, Portuguese, Hindi) and key demographic dimensions (country, generation, political orientation). The resulting personalised narratives are then assessed quantitatively and compared along the dimensions of models, languages, jailbreaking rate, and personalisation attributes. Our findings demonstrate that the use of even simple personalisation strategies in the prompts significantly increases the likelihood of jailbreaks for all studied LLMs. Furthermore, personalised prompts result in altered linguistic and rhetorical patterns and amplify the persuasiveness of the LLM-generated false narratives. These insights expose critical vulnerabilities in current state-of-the-art LLMs and offer a foundation for improving safety alignment and detection strategies in multilingual and cross-demographic contexts.

Method: OPLoRA uses double-sided orthogonal projections based on SVD decomposition of frozen weights. It constrains LoRA updates to the orthogonal complement of the top-k singular subspace using projections P_L = I - U_k U_k^T and P_R = I - V_k V_k^T, exactly preserving top-k singular triples.

Result: Extensive experiments on commonsense reasoning, mathematics, and code generation with LLaMA-2 7B and Qwen2.5 7B show OPLoRA significantly reduces forgetting while maintaining competitive task-specific performance.

Conclusion: Orthogonal projection is an effective mechanism for knowledge preservation in parameter-efficient fine-tuning, providing mathematical guarantees against catastrophic forgetting.

Abstract: Low-Rank Adaptation (LoRA) enables efficient fine-tuning of large language models but suffers from catastrophic forgetting when learned updates interfere with the dominant singular directions that encode essential pre-trained knowledge. We propose Orthogonal Projection LoRA (OPLoRA), a theoretically grounded approach that prevents this interference through double-sided orthogonal projections. By decomposing frozen weights via SVD, OPLoRA constrains LoRA updates to lie entirely within the orthogonal complement of the top-$k$ singular subspace using projections $P_L = I - U_k U_k^\top$ and $P_R = I - V_k V_k^\top$. We prove that this construction exactly preserves the top-$k$ singular triples, providing mathematical guarantees for knowledge retention. To quantify subspace interference, we introduce $\rho_k$, a metric measuring update alignment with dominant directions. Extensive experiments across commonsense reasoning, mathematics, and code generation demonstrate that OPLoRA significantly reduces forgetting while maintaining competitive task-specific performance on LLaMA-2 7B and Qwen2.5 7B, establishing orthogonal projection as an effective mechanism for knowledge preservation in parameter-efficient fine-tuning.

Method: Created a developmental dataset spanning five stages (ages 5-10) with a skill graph breaking down skills into abilities, goals, and indicators. Generated 23.4B tokens of synthetic data with controlled progression, vocabulary complexity, and format diversity (paragraphs, CQA, CSQA, IR pairs). Evaluated a 135M-parameter transformer under independent, joint, and sequential continual learning setups.

Result: The benchmark enables precise analysis of forgetting, forward transfer, and backward transfer. Experiments revealed trade-offs in skill retention and transfer efficiency across different training setups, demonstrating the utility of the fine-grained developmental approach.

Conclusion: CurlL advances continual learning evaluations for language models by providing a human-grounded benchmark with fine-grained control over skill dependencies, enabling more systematic assessment of progressive skill acquisition capabilities.

Abstract: We introduce a comprehensive continual learning dataset and benchmark (CurlL) grounded in human developmental trajectories from ages 5-10, enabling systematic and fine-grained assessment of models’ ability to progressively acquire new skills. CurlL spans five developmental stages (0-4) covering ages 5-10, supported by a skill graph that breaks down broad skills into smaller abilities, concrete goals, and measurable indicators, while also capturing which abilities build on others. We generate a 23.4B-token synthetic dataset with controlled skill progression, vocabulary complexity, and format diversity, comprising paragraphs, comprehension-based QA (CQA), skill-testing QA (CSQA), and instruction-response (IR) pairs. Stage-wise token counts range from 2.12B to 6.78B tokens, supporting precise analysis of forgetting, forward transfer, and backward transfer. Using a 135M-parameter transformer trained under independent, joint, and sequential (continual) setups, we show trade-offs in skill retention and transfer efficiency. By mirroring human learning patterns and providing fine-grained control over skill dependencies, this work advances continual learning evaluations for language models.

Method: Theoretical analysis shows DPO gradient norm is bounded by PVar. Experiments fine-tune LLMs using preferences from reward models, evaluating on AlpacaEval 2.0 and Arena-Hard benchmarks.

Result: Prompts with higher PVar outperform random or low-PVar prompts. Top 10% highest PVar prompts yield better performance than full dataset training.

Conclusion: Preference variance is crucial for identifying informative examples for efficient LLM alignment, enabling significant data reduction without performance loss.

Abstract: Direct Preference Optimization (DPO) has emerged as an important approach for learning from human preferences in aligning large language models (LLMs). However, collecting human preference data is costly and inefficient, motivating methods to reduce the required annotations. In this work, we investigate the impact of \emph{preference variance} (PVar), which measures the variance in model preferences when comparing pairs of responses, on the effectiveness of DPO training. We provide a theoretical insight by establishing an upper bound on the DPO gradient norm for any given prompt, showing it is controlled by the PVar of that prompt. This implies that prompts with low PVar can only produce small gradient updates, making them less valuable for learning. We validate this finding by fine-tuning LLMs with preferences generated by a reward model, evaluating on two benchmarks (AlpacaEval 2.0 and Arena-Hard). Experimental results demonstrate that prompts with higher PVar outperform randomly selected prompts or those with lower PVar. We also show that our PVar-based selection method is robust, when using smaller reward models (1B, 3B) for selection. Notably, in a separate experiment using the original human annotations from the UltraFeedback dataset, we found that training on only the top 10% of prompts with the highest PVar yields better evaluation performance than training on the full dataset, highlighting the importance of preference variance in identifying informative examples for efficient LLM alignment.

Method: GatePro identifies the most similar expert pairs and introduces localized competition mechanisms that prevent redundant expert co-activation while maintaining natural expert specialization. It’s parameter-free and can be deployed hot-swappable during any training phase.

Result: Comprehensive evaluation shows GatePro’s effectiveness across model scales and benchmarks. It achieves enhanced expert diversity where experts develop more distinct and complementary capabilities, avoiding functional redundancy.

Conclusion: GatePro offers a practical solution for improving MoE effectiveness by directly promoting expert selection diversity without additional learnable parameters.

Abstract: Modern large language models leverage Mixture-of-Experts (MoE) architectures for efficient scaling, but face a critical challenge: functionally similar experts are often selected simultaneously, creating redundant computation and limiting effective model capacity. Existing auxiliary balance loss methods improve token distribution but fail to address the underlying expert diversity problem. We introduce GatePro, a novel parameter-free method that directly promotes expert selection diversity. GatePro identifies the most similar expert pairs and introduces localized competition mechanisms, preventing redundant expert co-activation while maintaining natural expert specialization. Our comprehensive evaluation demonstrates GatePro’s effectiveness across model scales and benchmarks. Analysis demonstrates GatePro’s ability to achieve enhanced expert diversity, where experts develop more distinct and complementary capabilities, avoiding functional redundancy. This approach can be deployed hot-swappable during any training phase without additional learnable parameters, offering a practical solution for improving MoE effectiveness.

Method: Establishes connection between LLM uncertainty and invariance under semantic-preserving interventions, then measures output variation before and after such interventions.

Result: Method excels in effectiveness and computational efficiency across various LLMs and QA datasets, providing effective epistemic uncertainty estimates.

Conclusion: The proposed grey-box uncertainty quantification method successfully measures LLM uncertainty through semantic-preserving interventions and demonstrates strong performance.

Abstract: Uncertainty Quantification (UQ) is a promising approach to improve model reliability, yet quantifying the uncertainty of Large Language Models (LLMs) is non-trivial. In this work, we establish a connection between the uncertainty of LLMs and their invariance under semantic-preserving intervention from a causal perspective. Building on this foundation, we propose a novel grey-box uncertainty quantification method that measures the variation in model outputs before and after the semantic-preserving intervention. Through theoretical justification, we show that our method provides an effective estimate of epistemic uncertainty. Our extensive experiments, conducted across various LLMs and a variety of question-answering (QA) datasets, demonstrate that our method excels not only in terms of effectiveness but also in computational efficiency.

Method: Combines word embeddings (Word2Vec), named entity recognition, count vectorization, and TF-IDF with weighted TF-IDF word embeddings. Uses Random Forest and SVM for multi-label classification, and TextRank, Luhn, and GPT-3 for summarization.

Result: Evaluation with ROUGE scores demonstrates the effectiveness of the proposed methods for clinical trial eligibility assessment.

Conclusion: The system shows potential for automating clinical trial eligibility assessment using data-driven approaches, thereby improving research efficiency.

Abstract: Clinical trials are central to medical progress because they help improve understanding of human health and the healthcare system. They play a key role in discovering new ways to detect, prevent, or treat diseases, and it is essential that clinical trials include participants with appropriate and diverse medical backgrounds. In this paper, we propose a system that leverages Natural Language Processing (NLP) and Large Language Models (LLMs) to automate multi-label clinical text eligibility classification and summarization. The system combines feature extraction methods such as word embeddings (Word2Vec) and named entity recognition to identify relevant medical concepts, along with traditional vectorization techniques such as count vectorization and TF-IDF (Term Frequency-Inverse Document Frequency). We further explore weighted TF-IDF word embeddings that integrate both count-based and embedding-based strengths to capture term importance effectively. Multi-label classification using Random Forest and SVM models is applied to categorize documents based on eligibility criteria. Summarization techniques including TextRank, Luhn, and GPT-3 are evaluated to concisely summarize eligibility requirements. Evaluation with ROUGE scores demonstrates the effectiveness of the proposed methods. This system shows potential for automating clinical trial eligibility assessment using data-driven approaches, thereby improving research efficiency.

Method: Compared two one-shot strategies: centroid-based representative examples (proposed) vs randomly sampled examples (baseline), while varying sampling temperature to control output diversity.

Result: The proposed approach with higher temperature significantly outperformed random selection (+7.6% macro-F1, -10.5% RMSE) and exceeded 5-shot prompting (+21.1% macro-F1, -24.0% RMSE).

Conclusion: Combining representative example selection with increased temperature provides optimal diversity for effective one-shot LLM ensembles, highlighting the importance of both factors in ensemble design.

Abstract: Large language models (LLMs) have achieved remarkable results in wide range of domains. However, the accuracy and robustness of one-shot LLM predictions remain highly sensitive to the examples and the diversity among ensemble members. This study systematically investigates the effects of example representativeness (one-shot strategy) and output diversity (sampling temperature) on LLM ensemble performance. Two one-shot strategies are compared: centroid-based representative examples (proposed) and randomly sampled examples (baseline) and sampling temperature also is varied. The proposed approach with higher temperature setting significantly outperforms random selection by +7.6% (macro-F1) and -10.5% (RMSE). Furthermore, the proposed model exceeds 5-shot prompting by +21.1% (macro-F1) and -24.0% (RMSE). Our findings demonstrate that combining representative example selection with increased temperature provides the appropriate level of diversity to the ensemble. This work highlights the practical importance of both example selection and controlled diversity in designing effective one-shot LLM ensembles.

Method: Created structured dataset from human surveys across 16 MENA countries, evaluated diverse LLMs across multiple conditions combining three perspective framings (neutral, personalized, cultural observer) with two language modes (English and native languages).

Result: Identified three critical phenomena: cross-lingual value shifts (different responses by language), reasoning-induced degradation (explanation worsens alignment), and logit leakage (hidden preferences despite refusal). Models treat diverse nations as monolithic entities in native languages.

Conclusion: MENAValues provides a scalable framework for diagnosing cultural misalignment and methodological tools for developing more culturally inclusive AI systems.

Abstract: We introduce MENAValues, a novel benchmark designed to evaluate the cultural alignment and multilingual biases of large language models (LLMs) with respect to the beliefs and values of the Middle East and North Africa (MENA) region, an underrepresented area in current AI evaluation efforts. Drawing from large-scale, authoritative human surveys, we curate a structured dataset that captures the sociocultural landscape of MENA with population-level response distributions from 16 countries. To probe LLM behavior, we evaluate diverse models across multiple conditions formed by crossing three perspective framings (neutral, personalized, and third-person/cultural observer) with two language modes (English and localized native languages: Arabic, Persian, Turkish). Our analysis reveals three critical phenomena: “Cross-Lingual Value Shifts” where identical questions yield drastically different responses based on language, “Reasoning-Induced Degradation” where prompting models to explain their reasoning worsens cultural alignment, and “Logit Leakage” where models refuse sensitive questions while internal probabilities reveal strong hidden preferences. We further demonstrate that models collapse into simplistic linguistic categories when operating in native languages, treating diverse nations as monolithic entities. MENAValues offers a scalable framework for diagnosing cultural misalignment, providing both empirical insights and methodological tools for developing more culturally inclusive AI.

Method: Uses branch-complete rollouts from early-exit signals in parallel with target model execution, maps computation across heterogeneous accelerators (GPU/NPU), implements speculative streaming for multi-token emission per step, and creates dual execution pipelines where draft speculates continuations and target speculates corrections.

Result: Achieves 2.8x-5.8x wall-time speedups across diverse tasks on SpecBench with models from 14B to 66B parameters, and 30% average relative improvement over EAGLE3 baseline.

Conclusion: Mirror-SD successfully pushes speculative decoding toward the ideal regime of high acceptance with low overhead through parallel heterogeneous execution and multi-token speculative streaming.

Abstract: Speculative decoding accelerates LLM inference by using a draft model to look ahead, but gains are capped by the cost of autoregressive draft generation: increasing draft size elevates acceptance rates but introduces additional latency overhead exacerbating the speed-accuracy tradeoff. Prior methods (Medusa, Hydra, EAGLE) partially reduce draft cost but either degrade acceptance or introduce overheads that limit scaling. We present Mirror Speculative Decoding (Mirror-SD), an inference algorithm that breaks the latency-acceptance tradeoff. Mirror-SD launches branch-complete rollouts from early-exit signals in parallel with the target model’s suffix and explicitly maps computation across heterogeneous accelerators (GPU and NPU) to exploit cross-device parallelism. The draft speculates forward continuations for the target to verify, while the target simultaneously speculates correction paths for the draft, converting speculation into two complementary execution pipelines. To further cut draft latency without weakening acceptance semantics, we add speculative streaming so the draft emits multiple tokens per step. This dual strategy of parallel heterogeneous execution plus multi-token speculative streaming pushes speculative decoding toward its ideal regime of high acceptance with low overhead. On SpecBench with server-scale models from 14B to 66B parameters, Mirror-SD delivers consistent end-to-end gains, achieving 2.8x-5.8x wall-time speedups across diverse tasks and a 30% average relative improvement over the strongest baseline, EAGLE3.

Method: Proposed and evaluated JSON representations for graphs using ScratchTest, a mini-benchmark based on a custom Python re-implementation of Scratch that tests LLMs in code graph space.

Result: LLMs can perform graph-based abstract code generation in a single pass without complex pipelines when given correct graph representations, with different representations inducing significantly different accuracies.

Conclusion: This work establishes the first steps towards representation learning for graph-based abstract code generation, highlighting the instrumental role of representations in this task.

Abstract: Most large language models (LLMs) today excel at generating raw, sequential code with minimal abstractions and custom structures. However, there has been little work on graph-based abstract code generation, where significant logic is encapsulated in predefined nodes and execution flow is determined by edges. This is relevant for visual programming languages, and in cases where raw source code is inaccessible to users and LLM training sets. In this work, we propose and evaluate JSON representations for graphs to enable high accuracy graph-based abstract code generation. We evaluate these representations on ScratchTest, a mini-benchmark based on our custom Python re-implementation of Scratch, which tests the LLM in code graph space. Our findings demonstrate that LLMs can indeed perform the aforementioned generation task in a single pass without relying on specialized or complex pipelines, given the correct graph representations. We also show that different representations induce significantly different accuracies, highlighting the instrumental role of representations in this generation task. All in all, this work establishes the first steps towards representation learning for graph-based abstract code generation.

Method: Constructs diverse reasoning trajectories from multiple LLMs, enriches them with retrieved domain knowledge, and iteratively refines through novelty-driven selection, reflective recombination, and mutation guided by a fitness function evaluating correctness, coherence, and knowledge utilization.

Result: The evolved dataset enables fine-tuning of compact models that achieve state-of-the-art performance on scientific reasoning benchmarks.

Conclusion: Establishes a scalable approach to synthesizing high-fidelity scientific reasoning data from diverse and fallible LLMs.

Abstract: While chain-of-thought (CoT) distillation from advanced large language models (LLMs) has proven effective in general reasoning tasks, it struggles in scientific domains where even advanced models often produce incorrect or superficial reasoning due to high complexity and specialized knowledge requirements. Directly distilling from such flawed outputs results in low-quality training data and limits the performance of smaller student models. To overcome this, we propose CoT-Evo, an evolutionary CoT distillation framework. It begins by constructing a diverse pool of reasoning trajectories from multiple LLM thinkers, enriches them with automatically retrieved domain knowledge, and iteratively refines the trajectories using novelty-driven selection, reflective recombination and mutation. The refinement is guided by a fitness function that evaluates answer correctness, coherence, and effective knowledge utilization. This results in a high-quality CoT dataset tailored for scientific reasoning. We employ this evolved dataset to fine-tune a compact model, which achieves state-of-the-art performance on scientific reasoning benchmarks. Our work establishes a scalable approach to synthesizing high-fidelity scientific reasoning data from diverse and fallible LLMs.

Method: The survey classifies and examines CoT fine-tuning methods using the Six Thinking Hats framework, which systematically characterizes common human thinking modes. It also compiles datasets, model performances, and maintains a real-time GitHub repository for tracking advances.

Result: The paper provides a comprehensive overview of CoT fine-tuning from a human reasoning perspective, organizing existing methods according to human thinking modes, and identifying potential future research directions in this rapidly evolving field.

Conclusion: This survey serves as a valuable resource to inspire innovation and foster progress in CoT fine-tuning by bridging the gap between technical approaches and human reasoning mechanisms, offering a systematic framework for understanding and advancing this important area of AI research.

Abstract: Chain of thought (CoT) fine-tuning aims to endow large language models (LLMs) with reasoning capabilities by training them on curated reasoning traces. It leverages both supervised and reinforced fine-tuning to cultivate human-like reasoning skills in LLMs, including detailed planning, divergent thinking, intuitive judgment, timely reflection, internal thinking, and fact perception, etc. As CoT fine-tuning has advanced, LLMs have demonstrated substantial improvements in tasks such as mathematical reasoning and code generation. However, existing surveys about CoT fine-tuning primarily focus on technical aspects and overlook a systematic analysis from the perspective of human reasoning mechanisms. Given that the ultimate goal of CoT fine-tuning is to enable LLMs to reason like humans, it is crucial to investigate this technique through the lens of human cognition. To fill this gap, we present the first comprehensive survey of CoT fine-tuning grounded in human reasoning theory. Specifically, inspired by the well-known Six Thinking Hats framework, which systematically characterizes common human thinking modes using six metaphorical hats, we classify and examine CoT fine-tuning methods through this lens. Furthermore, building upon this theory, we outline potential directions for future research in CoT fine-tuning. In addition, we compile a comprehensive overview of existing datasets and model performances, and a real-time GitHub repository \footnote{https://github.com/AI-Chen/Awesome-CoT-Finetuning} that continuously tracks recent advances in this area is maintained. We hope this survey will serve as a valuable resource to inspire innovation and foster progress in this rapidly evolving field.

Method: Self-constrained decoding that modifies next-token distribution by strengthening safety layers and weakening hallucination/toxic layers during output generation.

Result: Achieves state-of-the-art performance in both detoxification and generation fluency with superior efficiency compared to existing methods.

Conclusion: DSCD offers a practical, scalable solution for safer LLM deployments with lightweight, high-compatibility plug-and-play capabilities.

Abstract: Detoxification in large language models (LLMs) remains a significant research challenge. Existing decoding detoxification methods are all based on external constraints, which require additional resource overhead and lose generation fluency. This work proposes Detoxification with Self-Constrained Decoding (DSCD), a novel method for LLM detoxification without parameter fine-tuning. DSCD strengthens the inner next-token distribution of the safety layer while weakening that of hallucination and toxic layers during output generation. This effectively diminishes toxicity and enhances output safety. DSCD offers lightweight, high compatibility, and plug-and-play capabilities, readily integrating with existing detoxification methods for further performance improvement. Extensive experiments on representative open-source LLMs and public datasets validate DSCD’s effectiveness, demonstrating state-of-the-art (SOTA) performance in both detoxification and generation fluency, with superior efficiency compared to existing methods. These results highlight DSCD’s potential as a practical and scalable solution for safer LLM deployments.

Method: SHIELD couples fine-grained safety classification with category-specific guidance and explicit actions (Block, Reframe, Forward). It composes tailored safety prompts that enforce nuanced refusals or safe redirection without requiring model retraining.

Result: Across five benchmarks and five representative LVLMs, SHIELD consistently lowers jailbreak and non-following rates while preserving utility. The method is plug-and-play with negligible overhead.

Conclusion: SHIELD serves as a practical safety patch for both weakly and strongly aligned LVLMs, is easily extendable to new attack types, and provides effective protection without retraining models.

Abstract: Large Vision-Language Models (LVLMs) unlock powerful multimodal reasoning but also expand the attack surface, particularly through adversarial inputs that conceal harmful goals in benign prompts. We propose SHIELD, a lightweight, model-agnostic preprocessing framework that couples fine-grained safety classification with category-specific guidance and explicit actions (Block, Reframe, Forward). Unlike binary moderators, SHIELD composes tailored safety prompts that enforce nuanced refusals or safe redirection without retraining. Across five benchmarks and five representative LVLMs, SHIELD consistently lowers jailbreak and non-following rates while preserving utility. Our method is plug-and-play, incurs negligible overhead, and is easily extendable to new attack types – serving as a practical safety patch for both weakly and strongly aligned LVLMs.

Method: Designed controlled experiments varying context density, delimiter styles, and positional placement. Introduced Contextual Normalization - a lightweight strategy that adaptively standardizes context representations before generation.

Result: Extensive experiments on controlled and real-world RAG benchmarks show the proposed strategy consistently improves robustness to order variation and strengthens long-context utilization across diverse settings.

Conclusion: Reliable RAG depends not only on retrieving the right content but also on how that content is presented. The findings offer new empirical evidence and a practical technique for better long-context reasoning.

Abstract: Retrieval-Augmented Generation (RAG) has become an essential approach for extending the reasoning and knowledge capacity of large language models (LLMs). While prior research has primarily focused on retrieval quality and prompting strategies, the influence of how the retrieved documents are framed, i.e., context format, remains underexplored. We show that seemingly superficial choices, such as delimiters or structural markers in key-value extraction, can induce substantial shifts in accuracy and stability, even when semantic content is identical. To systematically investigate this effect, we design controlled experiments that vary context density, delimiter styles, and positional placement, revealing the underlying factors that govern performance differences. Building on these insights, we introduce Contextual Normalization, a lightweight strategy that adaptively standardizes context representations before generation. Extensive experiments on both controlled and real-world RAG benchmarks across diverse settings demonstrate that the proposed strategy consistently improves robustness to order variation and strengthens long-context utilization. These findings underscore that reliable RAG depends not only on retrieving the right content, but also on how that content is presented, offering both new empirical evidence and a practical technique for better long-context reasoning.

Method: Leverages LLMs for cross-lingual emphasis conversion, translates source-language stress into target-language tags to guide a controllable TTS model, and uses an automated pipeline to generate aligned training data to overcome data scarcity.

Result: Substantially outperforms baselines in preserving emphasis while maintaining comparable translation quality, speaker intent, and naturalness.

Conclusion: The work highlights the importance of prosody in translation and provides an effective, data-efficient solution for preserving paralinguistic cues in speech-to-speech translation.

Abstract: We propose a stress-aware speech-to-speech translation (S2ST) system that preserves word-level emphasis by leveraging LLMs for cross-lingual emphasis conversion. Our method translates source-language stress into target-language tags that guide a controllable TTS model. To overcome data scarcity, we developed a pipeline to automatically generate aligned training data and introduce the “LLM-as-Judge” for evaluation. Experiments show our approach substantially outperforms baselines in preserving emphasis while maintaining comparable translation quality, speaker intent, and naturalness. Our work highlights the importance of prosody in translation and provides an effective, data-efficient solution for preserving paralinguistic cues in S2ST.

Method: Introduces Simplified Isolation Kernel (SIK) that maps high-dimensional embeddings to lower-dimensional sparse representations while preserving anomaly characteristics through boundary-focused feature mapping.

Result: Experiments on 7 datasets show SIK achieves better detection performance than 11 state-of-the-art anomaly detection algorithms while maintaining computational efficiency and low memory cost.

Conclusion: SIK effectively addresses the computational challenges of high-dimensional LLM embeddings in text anomaly detection while improving performance.

Abstract: Two-step approaches combining pre-trained large language model embeddings and anomaly detectors demonstrate strong performance in text anomaly detection by leveraging rich semantic representations. However, high-dimensional dense embeddings extracted by large language models pose challenges due to substantial memory requirements and high computation time. To address this challenge, we introduce the Simplified Isolation Kernel (SIK), which maps high-dimensional dense embeddings to lower-dimensional sparse representations while preserving crucial anomaly characteristics. SIK has linear time complexity and significantly reduces space complexity through its innovative boundary-focused feature mapping. Experiments across 7 datasets demonstrate that SIK achieves better detection performance than 11 state-of-the-art (SOTA) anomaly detection algorithms while maintaining computational efficiency and low memory cost. All code and demonstrations are available at https://github.com/charles-cao/SIK.

Method: Use structured prompts with LLMs to generate gender-swapped paraphrases, followed by quality control (semantic similarity, attribute verification, toxicity screening, human spot checks) to create augmented training data.

Result: LGSA achieved 99.1% accuracy with 1.9% bias gap, outperforming baseline (96.7% accuracy, 7.2% bias gap) and simple swap augmentation (95.6% accuracy, 0.7% bias gap). Improved performance on female-labeled examples.

Conclusion: LGSA is an effective bias mitigation strategy that enhances subgroup balance while maintaining high task accuracy and label fidelity, demonstrating practical value for fair AI systems.

Abstract: Bias in AI systems, especially those relying on natural language data, raises ethical and practical concerns. Underrepresentation of certain groups often leads to uneven performance across demographics. Traditional fairness methods, such as pre-processing, in-processing, and post-processing, depend on protected-attribute labels, involve accuracy-fairness trade-offs, and may not generalize across datasets. To address these challenges, we propose LLM-Guided Synthetic Augmentation (LGSA), which uses large language models to generate counterfactual examples for underrepresented groups while preserving label integrity. We evaluated LGSA on a controlled dataset of short English sentences with gendered pronouns, professions, and binary classification labels. Structured prompts were used to produce gender-swapped paraphrases, followed by quality control including semantic similarity checks, attribute verification, toxicity screening, and human spot checks. The augmented dataset expanded training coverage and was used to train a classifier under consistent conditions. Results show that LGSA reduces performance disparities without compromising accuracy. The baseline model achieved 96.7 percent accuracy with a 7.2 percent gender bias gap. Simple swap augmentation reduced the gap to 0.7 percent but lowered accuracy to 95.6 percent. LGSA achieved 99.1 percent accuracy with a 1.9 percent bias gap, improving performance on female-labeled examples. These findings demonstrate that LGSA is an effective strategy for bias mitigation, enhancing subgroup balance while maintaining high task accuracy and label fidelity.

Method: Scalable and fully automated methodology using image and text analytics to extract bilingual parallel corpora from newspaper articles.

Result: Built parallel data corpus for two language combinations and demonstrated value through machine translation, improving over current baseline by close to 3 BLEU points.

Conclusion: The approach successfully addresses the data scarcity problem for low-resource languages and enables improved machine translation performance.

Abstract: Linguistic diversity across the world creates a disparity with the availability of good quality digital language resources thereby restricting the technological benefits to majority of human population. The lack or absence of data resources makes it difficult to perform NLP tasks for low-resource languages. This paper presents a novel scalable and fully automated methodology to extract bilingual parallel corpora from newspaper articles using image and text analytics. We validate our approach by building parallel data corpus for two different language combinations and demonstrate the value of this dataset through a downstream task of machine translation and improve over the current baseline by close to 3 BLEU points.

Method: Developed Hierarchical Frequency Tagging Probe (HFTP) using frequency-domain analysis to identify neuron-wise components in LLMs and cortical regions in human brains that encode syntactic structures, with representational similarity analysis comparing LLM and brain representations.

Result: LLMs process syntax in analogous layers while human brain uses distinct cortical regions for different syntactic levels. LLM representations align more with left hemisphere. Model upgrades show divergent trends: Gemma 2 has greater brain similarity than Gemma, while Llama 3.1 has less alignment than Llama 2.

Conclusion: HFTP provides valuable insights into LLM interpretability, raising questions about whether behavioral improvements stem from human-like or non-human-like mechanisms, and serves as a bridge between computational linguistics and cognitive neuroscience.

Abstract: Large Language Models (LLMs) demonstrate human-level or even superior language abilities, effectively modeling syntactic structures, yet the specific computational modules responsible remain unclear. A key question is whether LLM behavioral capabilities stem from mechanisms akin to those in the human brain. To address these questions, we introduce the Hierarchical Frequency Tagging Probe (HFTP), a tool that utilizes frequency-domain analysis to identify neuron-wise components of LLMs (e.g., individual Multilayer Perceptron (MLP) neurons) and cortical regions (via intracranial recordings) encoding syntactic structures. Our results show that models such as GPT-2, Gemma, Gemma 2, Llama 2, Llama 3.1, and GLM-4 process syntax in analogous layers, while the human brain relies on distinct cortical regions for different syntactic levels. Representational similarity analysis reveals a stronger alignment between LLM representations and the left hemisphere of the brain (dominant in language processing). Notably, upgraded models exhibit divergent trends: Gemma 2 shows greater brain similarity than Gemma, while Llama 3.1 shows less alignment with the brain compared to Llama 2. These findings offer new insights into the interpretability of LLM behavioral improvements, raising questions about whether these advancements are driven by human-like or non-human-like mechanisms, and establish HFTP as a valuable tool bridging computational linguistics and cognitive neuroscience. This project is available at https://github.com/LilTiger/HFTP.

Method: They use Concept, a word-guessing board game, as a benchmark to probe abductive reasoning. The evaluation includes testing LLMs’ ability to interpret strategic intents and update hypotheses with sequential information, and extends across multiple languages.

Result: Humans achieve over 90% success rate, while state-of-the-art LLMs fail to exceed 40% success. LLMs particularly struggle with interpreting strategic intents and correcting initial hypotheses. Performance drops further in lower-resource languages compared to English.

Conclusion: Concept effectively exposes LLMs’ limitations in abductive reasoning despite using natural language representations. The benchmark reveals persistent challenges in strategic reasoning and hypothesis updating, with additional performance degradation in non-English languages.

Abstract: Large language models (LLMs) have achieved striking successes on many benchmarks, yet recent studies continue to expose fundamental weaknesses. In particular, tasks that require abstract reasoning remain challenging, often because they use representations such as grids, symbols, or visual patterns that differ from the natural language data LLMs are trained on. In this paper, we introduce Concept, a simple word-guessing board game, as a benchmark for probing abductive reasoning in a representation that is much closer to LLM pre-training data: natural language. Our results show that this game, easily solved by humans (with a success rate of over 90%), is still very challenging for state-of-the-art LLMs (no model exceeds 40% success rate). Specifically, we observe that LLMs struggle with interpreting other players’ strategic intents, and with correcting initial hypotheses given sequential information updates. In addition, we extend the evaluation across multiple languages, and find that the LLM performance drops further in lower-resource languages (Dutch, French, and Spanish) compared to English.

Method: Introduces VERITAS framework that integrates fine-grained faithfulness rewards into reinforcement learning process, with three faithfulness metrics: information-think, think-answer, and think-search faithfulness.

Result: Models trained with VERITAS significantly improve reasoning faithfulness while achieving comparable task performance across seven QA benchmarks.

Conclusion: VERITAS successfully addresses the faithfulness gap in RL-based search agents without compromising task performance, providing a more reliable approach for retrieval-augmented generation.

Abstract: Inspired by the success of reinforcement learning (RL) in Large Language Model (LLM) training for domains like math and code, recent works have begun exploring how to train LLMs to use search engines more effectively as tools for retrieval-augmented generation. Although these methods achieve performance improvement across QA benchmarks, many prioritize final answer correctness while overlooking the quality of intermediate reasoning steps, which may lead to chain-of-thought unfaithfulness. In this paper, we first introduce a comprehensive evaluation framework for evaluating RL-based search agents, covering three distinct faithfulness metrics: information-think faithfulness, think-answer faithfulness, and think-search faithfulness. Our evaluations reveal that a prototypical RL-based search agent, Search-R1, has significant room for improvement in this regard. To foster faithful reasoning, we introduce VERITAS (Verifying Entailed Reasoning through Intermediate Traceability in Agentic Search), a novel framework that integrates fine-grained faithfulness rewards into the reinforcement learning process. Our experiments show that models trained with VERITAS not only significantly improve reasoning faithfulness, but also achieve comparable task performance across seven QA benchmarks.

Method: IDS dynamically adjusts intervention strength based on how far input lies within the data distribution in representation space, enabling adaptive intervention.

Result: IDS achieves strong accuracy on classification tasks while producing coherent text without collapse, outperforming fixed-strength methods.

Conclusion: IDS is particularly well-suited for real-world applications due to its adaptive intervention and generation stability.

Abstract: Activation steering methods control large language model (LLM) behavior by modifying internal activations at inference time. However, most existing activation steering methods rely on a fixed steering strength, leading to either insufficient control or unadapted intervention that degrades text plausibility and coherence. We introduce In-Distribution Steering (IDS), a novel method that adapts steering strength based on the input data distribution in representation space. IDS dynamically adjusts interventions according to how far a given input lies within the distribution, enabling adaptive intervention and generation stability during text generation. Experiments demonstrate that IDS achieves strong accuracy on classification tasks while producing coherent text without collapse, making IDS particularly well suited for real-world applications.

Method: WOWService integrates LLMs with multi-agent architectures, focusing on data construction, general capability enhancement, business scenario adaptation, multi-agent coordination, and automated evaluation modules.

Result: Deployed on Meituan App with significant improvements: User Satisfaction Metric 1 decreased by 27.53% (likely indicating reduced negative feedback) and User Satisfaction Metric 2 increased by 25.51%.

Conclusion: WOWService effectively addresses key challenges in industrial intelligent interaction systems and demonstrates practical value through improved user satisfaction metrics in real-world deployment.

Abstract: Enhancing customer experience is essential for business success, particularly as service demands grow in scale and complexity. Generative artificial intelligence and Large Language Models (LLMs) have empowered intelligent interaction systems to deliver efficient, personalized, and 24/7 support. In practice, intelligent interaction systems encounter several challenges: (1) Constructing high-quality data for cold-start training is difficult, hindering self-evolution and raising labor costs. (2) Multi-turn dialogue performance remains suboptimal due to inadequate intent understanding, rule compliance, and solution extraction. (3) Frequent evolution of business rules affects system operability and transferability, constraining low-cost expansion and adaptability. (4) Reliance on a single LLM is insufficient in complex scenarios, where the absence of multi-agent frameworks and effective collaboration undermines process completeness and service quality. (5) The open-domain nature of multi-turn dialogues, lacking unified golden answers, hampers quantitative evaluation and continuous optimization. To address these challenges, we introduce WOWService, an intelligent interaction system tailored for industrial applications. With the integration of LLMs and multi-agent architectures, WOWService enables autonomous task management and collaborative problem-solving. Specifically, WOWService focuses on core modules including data construction, general capability enhancement, business scenario adaptation, multi-agent coordination, and automated evaluation. Currently, WOWService is deployed on the Meituan App, achieving significant gains in key metrics, e.g., User Satisfaction Metric 1 (USM 1) -27.53% and User Satisfaction Metric 2 (USM 2) +25.51%, demonstrating its effectiveness in capturing user needs and advancing personalized service.

Method: Develops an adaptive CFG scheme that adjusts based on detected mismatch levels using large language models or natural language inference models, following comprehensive analysis of CFG’s impact.

Result: The adaptive CFG scheme improves emotional expressiveness in AR TTS models while maintaining audio quality and intelligibility.

Conclusion: Adaptive CFG effectively resolves style-content mismatch issues in auto-regressive TTS, enabling better emotional control without compromising speech quality.

Abstract: While Text-to-Speech (TTS) systems can achieve fine-grained control over emotional expression via natural language prompts, a significant challenge emerges when the desired emotion (style prompt) conflicts with the semantic content of the text. This mismatch often results in unnatural-sounding speech, undermining the goal of achieving fine-grained emotional control. Classifier-Free Guidance (CFG) is a key technique for enhancing prompt alignment; however, its application to auto-regressive (AR) TTS models remains underexplored, which can lead to degraded audio quality. This paper directly addresses the challenge of style-content mismatch in AR TTS models by proposing an adaptive CFG scheme that adjusts to different levels of the detected mismatch, as measured using large language models or natural language inference models. This solution is based on a comprehensive analysis of CFG’s impact on emotional expressiveness in state-of-the-art AR TTS models. Our results demonstrate that the proposed adaptive CFG scheme improves the emotional expressiveness of the AR TTS model while maintaining audio quality and intelligibility.

Method: Uses LLMs’ extensive pre-training and in-context learning capabilities, employing log-probabilities to measure style transferability between texts in an unsupervised manner.

Result: Significantly outperforms LLM prompting approaches of comparable scale and achieves higher accuracy than contrastively trained baselines when controlling for topical correlations. Performance scales with model size and test-time computation.

Conclusion: The proposed unsupervised approach effectively leverages LLMs’ pre-training for authorship analysis, offering flexible trade-offs between computational cost and accuracy while avoiding topical bias.

Abstract: Computational stylometry analyzes writing style through quantitative patterns in text, supporting applications from forensic tasks such as identity linking and plagiarism detection to literary attribution in the humanities. Supervised and contrastive approaches rely on data with spurious correlations and often confuse style with topic. Despite their natural use in AI-generated text detection, the CLM pre-training of modern LLMs has been scarcely leveraged for general authorship problems. We propose a novel unsupervised approach based on this extensive pre-training and the in-context learning capabilities of LLMs, employing the log-probabilities of an LLM to measure style transferability from one text to another. Our method significantly outperforms LLM prompting approaches of comparable scale and achieves higher accuracy than contrastively trained baselines when controlling for topical correlations. Moreover, performance scales fairly consistently with the size of the base model and, in the case of authorship verification, with an additional mechanism that increases test-time computation; enabling flexible trade-offs between computational cost and accuracy.

Method: Uses reinforcement learning with intent-aware rewards that provide turn-level feedback by aligning retrieval and reasoning with evolving user goals. Interleaves search and reasoning across dialogue turns rather than using static pipelines.

Result: Demonstrates strong performance on both 3B and 7B model backbones, outperforming competitive models on five CQA datasets across different metrics (F1, BERTScore, LLM-as-judge). Shows robust generalization across domains.

Conclusion: RL-based reasoning enables more flexible and context-sensitive behavior than static CQA pipelines, with intent-aware rewards effectively addressing sparse reward challenges in conversational settings.

Abstract: We present ChatR1, a reasoning framework based on reinforcement learning (RL) for conversational question answering (CQA). Reasoning plays an important role in CQA, where user intent evolves across dialogue turns, and utterances are often underspecified, requiring contextual interpretation, query reformulation, and dynamic coordination between retrieval and generation. Unlike static `rewrite, retrieve, and generate’ pipelines, ChatR1 interleaves search and reasoning across turns, enabling exploratory and adaptive behaviors learned through RL. To address the challenge of sparse and delayed rewards in RL, we propose an intent-aware reward that provides turn-level feedback by aligning retrieval and reasoning with evolving user goals. Our proposed ChatR1 demonstrates strong performance on both 3B and 7B model backbones, outperforming competitive models on five CQA datasets, measured by different metrics (F1, BERTScore, and LLM-as-judge). We include a diverse set of CQA datasets to cover topic shifts, evolving intents, mixed-initiative dialogues, and multi-document grounding, testing ChatR1’s performance from various aspects. Ablation studies confirm the effectiveness of the intent-aware reward. Our analyses further reveal diverse reasoning trajectories and effective use of the search tool. ChatR1 also generalizes robustly across domains, demonstrating that RL-based reasoning enables more flexible and context-sensitive behavior than static CQA pipelines.

Method: Proposes Embedding-Based Context-Aware Reranker (EBCAR) that operates directly on passage embeddings, uses structural information of passages, and employs a hybrid attention mechanism to capture both high-level cross-document interactions and low-level within-document relationships.

Result: EBCAR outperforms SOTA rerankers on the ConTEB benchmark, showing effectiveness for information retrieval requiring cross-passage inference with advantages in both accuracy and efficiency.

Conclusion: EBCAR provides a lightweight yet effective solution for cross-passage inference in RAG systems, addressing key limitations of current reranking methods while maintaining computational efficiency.

Abstract: Retrieval-Augmented Generation (RAG) systems rely on retrieving relevant evidence from a corpus to support downstream generation. The common practice of splitting a long document into multiple shorter passages enables finer-grained and targeted information retrieval. However, it also introduces challenges when a correct retrieval would require inference across passages, such as resolving coreference, disambiguating entities, and aggregating evidence scattered across multiple sources. Many state-of-the-art (SOTA) reranking methods, despite utilizing powerful large pretrained language models with potentially high inference costs, still neglect the aforementioned challenges. Therefore, we propose Embedding-Based Context-Aware Reranker (EBCAR), a lightweight reranking framework operating directly on embeddings of retrieved passages with enhanced cross-passage understandings through the structural information of the passages and a hybrid attention mechanism, which captures both high-level interactions across documents and low-level relationships within each document. We evaluate EBCAR against SOTA rerankers on the ConTEB benchmark, demonstrating its effectiveness for information retrieval requiring cross-passage inference and its advantages in both accuracy and efficiency.

Method: A two-step, linear-time defensive aggregation strategy that controls worst-case risk, implemented as DefensiveKV and its extension Layer-DefensiveKV with layer-wise budget allocation.

Result: Across 7 task domains (18 datasets), the methods reduce generation quality loss by 2.3x and 4.3x respectively versus the strongest baseline under 20% cache size, setting new performance benchmarks.

Conclusion: The work pioneers a promising direction for optimizing cache eviction against underlying fragility through worst-case risk management, providing effective defense against extreme cases with negligible computational overhead.

Abstract: Large language models have revolutionized natural language processing, yet their deployment remains hampered by the substantial memory and runtime overhead of the transformer’s Key-Value cache. To mitigate this, recent methods employ a scoring-aggregation framework to evict unimportant cache entries, based on the stability assumption-that a fixed subset of entries remains consistently important during generation. However, prior work has largely focused on refining importance indicators for scoring, while defaulting to mean aggregation due to a faithful trust in the stability assumption. In this work, we argue that this underlying assumption is inherently fragile, making mean aggregation highly vulnerable in extreme cases. To counter this, we propose a simple yet elegant defensive aggregation strategy: a two-step, linear-time approach that controls worst-case risk, thereby defending against extreme cases with negligible computational overhead. Embodying this strategy, we propose a novel cache eviction method, DefensiveKV and its extension, Layer-DefensiveKV, which incorporates layer-wise budget allocation. Across seven task domains (18 datasets), our methods reduce generation quality loss by 2.3x and 4.3x respectively, versus the strongest baseline under a 20% cache size. These results set new performance benchmarks and pioneer a promising direction for optimizing cache eviction against underlying fragility through worst-case risk management. Our code is available at https://github.com/FFY0/DefensiveKV.

Method: Used LLMs for sentiment classification of Greek proverbs, expanded an annotated dataset to include local dialects, created sentiment distribution maps, and performed combinatory analysis of geography, dialect, and topic.

Result: LLMs can provide accurate sentiment analysis of proverbs when treated as a non-conventional sentiment polarity task, and negative sentiment is more prevalent in most Greek regions.

Conclusion: The study successfully demonstrates the effectiveness of LLMs in analyzing proverb sentiment and reveals geographical patterns in sentiment distribution across Greece.

Abstract: Proverbs are among the most fascinating linguistic phenomena that transcend cultural and linguistic boundaries. Yet, much of the global landscape of proverbs remains underexplored, as many cultures preserve their traditional wisdom within their own communities due to the oral tradition of the phenomenon. Taking advantage of the current advances in Natural Language Processing (NLP), we focus on Greek proverbs, analyzing their sentiment. Departing from an annotated dataset of Greek proverbs, we expand it to include local dialects, effectively mapping the annotated sentiment. We present (1) a way to exploit LLMs in order to perform sentiment classification of proverbs, (2) a map of Greece that provides an overview of the distribution of sentiment, (3) a combinatory analysis in terms of the geographic position, dialect, and topic of proverbs. Our findings show that LLMs can provide us with an accurate enough picture of the sentiment of proverbs, especially when approached as a non-conventional sentiment polarity task. Moreover, in most areas of Greece negative sentiment is more prevalent.

Method: Uses fine-tuned category-specific adapters trained via Low-Rank Adaptation (LoRA) on extensive multi-modal datasets, with a teacher-assisted annotation pipeline that leverages reasoning and explanation traces for high-fidelity labels.

Result: Demonstrates state-of-the-art performance across all safety dimensions, surpassing existing models like WildGuard, LlamaGuard-4, and GPT-4.1.

Conclusion: Protect establishes a foundation for trustworthy, auditable, and production-ready safety systems capable of operating across text, image, and audio modalities.

Abstract: The increasing deployment of Large Language Models (LLMs) across enterprise and mission-critical domains has underscored the urgent need for robust guardrailing systems that ensure safety, reliability, and compliance. Existing solutions often struggle with real-time oversight, multi-modal data handling, and explainability – limitations that hinder their adoption in regulated environments. Existing guardrails largely operate in isolation, focused on text alone making them inadequate for multi-modal, production-scale environments. We introduce Protect, natively multi-modal guardrailing model designed to operate seamlessly across text, image, and audio inputs, designed for enterprise-grade deployment. Protect integrates fine-tuned, category-specific adapters trained via Low-Rank Adaptation (LoRA) on an extensive, multi-modal dataset covering four safety dimensions: toxicity, sexism, data privacy, and prompt injection. Our teacher-assisted annotation pipeline leverages reasoning and explanation traces to generate high-fidelity, context-aware labels across modalities. Experimental results demonstrate state-of-the-art performance across all safety dimensions, surpassing existing open and proprietary models such as WildGuard, LlamaGuard-4, and GPT-4.1. Protect establishes a strong foundation for trustworthy, auditable, and production-ready safety systems capable of operating across text, image, and audio modalities.

Method: Used non-parametric white-box attack method under passive threat model on three ASR models (Wav2Vec2, HuBERT, Whisper), operating solely on weight differentials without access to raw speech data from target speakers.

Result: Successfully demonstrated attack feasibility on sensitive attributes including gender, age, accent, emotion, and dysarthria. Attributes underrepresented or absent in pre-training data were more vulnerable, with accent information being reliably inferred from all models.

Conclusion: Federated ASR models have previously undocumented vulnerabilities to attribute inference attacks, highlighting the need for improved security measures in privacy-preserving training approaches.

Abstract: Federated learning is a common method for privacy-preserving training of machine learning models. In this paper, we analyze the vulnerability of ASR models to attribute inference attacks in the federated setting. We test a non-parametric white-box attack method under a passive threat model on three ASR models: Wav2Vec2, HuBERT, and Whisper. The attack operates solely on weight differentials without access to raw speech from target speakers. We demonstrate attack feasibility on sensitive demographic and clinical attributes: gender, age, accent, emotion, and dysarthria. Our findings indicate that attributes that are underrepresented or absent in the pre-training data are more vulnerable to such inference attacks. In particular, information about accents can be reliably inferred from all models. Our findings expose previously undocumented vulnerabilities in federated ASR models and offer insights towards improved security.

Method: Uses two components: Dynamic Structured Memory (DSM) that builds OWL-compliant knowledge graphs of conversations, and Reasoning Tree (RT) that performs multi-step search over the graph.

Result: Outperforms state-of-the-art baselines on MT-Bench-101, improving dialogue consistency by over 48% for both proprietary and open-source models, and quality score by up to 10.1% for open-source models.

Conclusion: D-SMART effectively addresses multi-turn dialogue consistency issues through dynamic structured reasoning, with new NLI-based metrics providing better evaluation of logical consistency.

Abstract: Large Language Models (LLMs) often exhibit factual inconsistencies and logical decay in extended, multi-turn dialogues, a challenge stemming from their reliance on static, pre-trained knowledge and an inability to reason adaptively over the dialogue history. Prevailing mitigation strategies, such as Retrieval-Augmented Generation (RAG) and agentic working memories, improve information recall but still engage with fundamentally static knowledge sources and follow pre-defined single reasoning path. This hinders their ability to preserve factual and logical consistency of their responses in multi-turn dialogues while the context evolves over time. To address this issue, we propose D-SMART, a model-agnostic framework designed to maintain multi-turn dialogue consistency by enabling LLMs to build and reason over a dynamic, structured representation of the conversational context. This is achieved via two synergistic components: (1) a Dynamic Structured Memory (DSM), which incrementally constructs and maintains an authoritative, OWL-compliant knowledge graph of the conversation; and (2) a Reasoning Tree (RT), which executes inferences as an explicit and traceable multi-step search over the graph. As the popular-used quality score (judged by GPT-4) can overlook logical flaws, we introduce new NLI-based metrics to better measure multi-turn dialogue consistency. Comprehensive experiments on the MT-Bench-101 benchmark show that D-SMART significantly outperforms state-of-the-art baselines, elevating the dialogue consistency score by over 48% for both proprietary and open-source models, and notably improves the quality score of the latter by up to 10.1%.

Method: The paper conducts a comprehensive survey approach, examining key advancements in multimodal, multilingual, and retrieval-augmented Document AI, while analyzing the shift from encoder-decoder architectures to decoder-only LLMs.

Result: The survey identifies that decoder-only large language models have revolutionized Document AI, bringing remarkable advancements in understanding and generation capabilities, and outlines current research attempts and challenges in the field.

Conclusion: The paper provides a structured analysis of Document AI’s state-of-the-art, suggesting future research directions including agent-based approaches and document-specific foundation models, with implications for both academic research and practical applications.

Abstract: Document AI (DAI) has emerged as a vital application area, and is significantly transformed by the advent of large language models (LLMs). While earlier approaches relied on encoder-decoder architectures, decoder-only LLMs have revolutionized DAI, bringing remarkable advancements in understanding and generation. This survey provides a comprehensive overview of DAI’s evolution, highlighting current research attempts and future prospects of LLMs in this field. We explore key advancements and challenges in multimodal, multilingual, and retrieval-augmented DAI, while also suggesting future research directions, including agent-based approaches and document-specific foundation models. This paper aims to provide a structured analysis of the state-of-the-art in DAI and its implications for both academic and practical applications.

Method: Developed two variants: Semi-Formal-Natural-Language (SFNL) BP and Fully-Natural-Language (FNL) BP, incorporating commitment-communication mechanisms where persuaders explicitly outline information schemas to guide Bayesian belief updates.

Result: LLMs using BP strategies achieved higher persuasion success rates than non-BP baselines. SFNL showed better credibility and logical coherence, while FNL demonstrated stronger emotional resonance and robustness in natural conversations. Fine-tuned smaller models matched larger models’ BP performance.

Conclusion: Bayesian Persuasion frameworks effectively enhance LLMs’ strategic persuasion capabilities, with different variants offering complementary strengths, and smaller models can achieve comparable performance through fine-tuning.

Abstract: Persuasion, a fundamental social capability for humans, remains a challenge for AI systems such as large language models (LLMs). Current studies often overlook the strategic use of information asymmetry in message design or rely on strong assumptions regarding pre-commitment. In this work, we explore the application of Bayesian Persuasion (BP) in natural language within single-turn dialogue settings, to enhance the strategic persuasion capabilities of LLMs. Our framework incorporates a commitment-communication mechanism, where the persuader explicitly outlines an information schema by narrating their potential types (e.g., honest or dishonest), thereby guiding the persuadee in performing the intended Bayesian belief update. We evaluate two variants of our approach: Semi-Formal-Natural-Language (SFNL) BP and Fully-Natural-Language (FNL) BP, benchmarking them against both naive and strong non-BP (NBP) baselines within a comprehensive evaluation framework. This framework covers a diverse set of persuadees – including LLM instances with varying prompts and fine-tuning and human participants – across tasks ranging from specially designed persuasion scenarios to general everyday situations. Experimental results on LLM-based agents reveal three main findings: (1) LLMs guided by BP strategies consistently achieve higher persuasion success rates than NBP baselines; (2) SFNL exhibits greater credibility and logical coherence, while FNL shows stronger emotional resonance and robustness in naturalistic conversations; (3) with supervised fine-tuning, smaller models can attain BP performance comparable to that of larger models.

Method: Used the Generative Agent-Based Model Concordia to assess GPT-4’s Theory of Mind abilities in simulated real-world environments, evaluating action selection based on belief attribution.

Result: GPT-4 frequently fails to select actions based on belief attribution and struggles to generate coherent causal effects from agent actions, revealing difficulties in processing complex social interactions.

Conclusion: Current claims about emergent ToM-like capabilities in LLMs are challenged, highlighting the need for more rigorous, action-based evaluation frameworks.

Abstract: Language is fundamental to human cooperation, facilitating not only the exchange of information but also the coordination of actions through shared interpretations of situational contexts. This study explores whether the Generative Agent-Based Model (GABM) Concordia can effectively model Theory of Mind (ToM) within simulated real-world environments. Specifically, we assess whether this framework successfully simulates ToM abilities and whether GPT-4 can perform tasks by making genuine inferences from social context, rather than relying on linguistic memorization. Our findings reveal a critical limitation: GPT-4 frequently fails to select actions based on belief attribution, suggesting that apparent ToM-like abilities observed in previous studies may stem from shallow statistical associations rather than true reasoning. Additionally, the model struggles to generate coherent causal effects from agent actions, exposing difficulties in processing complex social interactions. These results challenge current statements about emergent ToM-like capabilities in LLMs and highlight the need for more rigorous, action-based evaluation frameworks.

Method: Applied phylogenetic comparative models to dictionary data from Austronesian, Indo-European, and Uralic language families, analyzing effects of associativity, borrowability, and usage frequency on colexification patterns.

Result: Closely related concept pairs show more widespread colexification and slower change rates. Frequent and borrowable concepts change faster and are less often colexified. Significant differences exist between language families.

Conclusion: Concept relatedness stabilizes colexification, while frequency and borrowability accelerate meaning change. Areal and cultural factors also influence colexification patterns across different language families.

Abstract: One of the most intriguing features of language is its constant change, with ongoing shifts in how meaning is expressed. Despite decades of research, the factors that determine how and why meanings evolve remain only partly understood. Colexification – the phenomenon of expressing multiple distinct concepts using the same word form – serves as a valuable window onto the dynamics of meaning change across languages. Here, we apply phylogenetic comparative models to dictionary data from three language families, Austronesian, Indo-European, and Uralic, in order to shed light on the evolutionary dynamics underlying the colexification of concept pairs. We assess the effects of three predictors: associativity, borrowability, and usage frequency. Our results show that more closely related concept pairs are colexified across a larger portion of the family tree and exhibit slower rates of change. In contrast, concept pairs that are more frequent and more prone to borrowing tend to change more rapidly and are less often colexified. We also find considerable differences between the language families under study, suggesting that areal and cultural factors may play a role.

Method: Proposed taxonomy organizing benchmarks into four categories: Knowledge, NLP Tasks, Culture and Dialects, and Target-Specific evaluations. Examined three approaches: native collection, translation, and synthetic generation.

Result: Revealed significant progress in benchmark diversity but identified critical gaps: limited temporal evaluation, insufficient multi-turn dialogue assessment, and cultural misalignment in translated datasets.

Conclusion: This work serves as a comprehensive reference for Arabic NLP researchers, providing insights into benchmark methodologies, reproducibility standards, and evaluation metrics with recommendations for future development.

Abstract: This survey provides the first systematic review of Arabic LLM benchmarks, analyzing 40+ evaluation benchmarks across NLP tasks, knowledge domains, cultural understanding, and specialized capabilities. We propose a taxonomy organizing benchmarks into four categories: Knowledge, NLP Tasks, Culture and Dialects, and Target-Specific evaluations. Our analysis reveals significant progress in benchmark diversity while identifying critical gaps: limited temporal evaluation, insufficient multi-turn dialogue assessment, and cultural misalignment in translated datasets. We examine three primary approaches: native collection, translation, and synthetic generation discussing their trade-offs regarding authenticity, scale, and cost. This work serves as a comprehensive reference for Arabic NLP researchers, providing insights into benchmark methodologies, reproducibility standards, and evaluation metrics while offering recommendations for future development.

Method: Created LiteraryQA by identifying and correcting low-quality QA samples using human- and LLM-validated pipeline, removing extraneous text from source documents. Conducted meta-evaluation of automatic metrics and benchmarked long-context LLMs.

Result: N-gram-based metrics have low system-level correlation to human judgment, while LLM-as-a-Judge evaluations strongly agree with human rankings. Benchmark results for long-context LLMs are provided.

Conclusion: LiteraryQA provides a reliable benchmark for narrative QA evaluation, with LLM-as-a-Judge being the preferred evaluation method over traditional n-gram metrics.

Abstract: Question Answering (QA) on narrative text poses a unique challenge to current systems, requiring a deep understanding of long, complex documents. However, the reliability of NarrativeQA, the most widely used benchmark in this domain, is hindered by noisy documents and flawed QA pairs. In this work, we introduce LiteraryQA, a high-quality subset of NarrativeQA focused on literary works. Using a human- and LLM-validated pipeline, we identify and correct low-quality QA samples while removing extraneous text from source documents. We then carry out a meta-evaluation of automatic metrics to clarify how systems should be evaluated on LiteraryQA. This analysis reveals that all n-gram-based metrics have a low system-level correlation to human judgment, while LLM-as-a-Judge evaluations, even with small open-weight models, can strongly agree with the ranking identified by humans. Finally, we benchmark a set of long-context LLMs on LiteraryQA. We release our code and data at https://github.com/SapienzaNLP/LiteraryQA.

Method: M^2PO integrates a multi-perspective reward engine combining hallucination penalty for factuality and dynamic quality score that fuses external evaluations with model’s evolving judgment, paired with multi-pair construction that creates comprehensive preference pairs from all translation candidates.

Result: On challenging WMT21-22 benchmarks, M^2PO substantially outperforms existing preference optimization methods and demonstrates highly competitive performance against leading proprietary LLMs.

Conclusion: The synergistic multi-perspective and multi-pair approach enables more robust and faithful translations by learning from richer quality trade-offs.

Abstract: Direct Preference Optimization (DPO) is a powerful paradigm for aligning Large Language Models (LLMs) to human preferences in Machine Translation (MT), but current methods are hindered by two fundamental challenges: (1) flawed reward signals from Quality Estimation (QE) models that overlook critical errors like translation hallucination, and (2) inefficient data utilization that discards valuable learning signals by selecting only a single win-loss pair. To address these limitations, we introduce M^2PO: Multi-Pair, Multi-Perspective Preference Optimization. Our framework integrates a multi-perspective reward engine that creates a more robust signal by combining two key viewpoints: a new hallucination penalty for factuality, and an innovative dynamic quality score that adaptively fuses external evaluations with the model’s own evolving judgment. This is synergistically paired with a multi-pair construction strategy that systematically creates a comprehensive set of preference pairs from the entire pool of translation candidates. This synergistic approach ensures the model learns from a richer spectrum of quality trade-offs, leading to more robust and faithful translations. On challenging WMT21-22 benchmarks, M^2PO substantially outperforms existing preference optimization methods and demonstrates highly competitive performance against leading proprietary LLMs.

Method: Created \bench - a dynamic, live evaluation benchmark with automated curation pipeline that supports real-time updates while preventing data contamination. It’s the largest and most diverse benchmark for intent understanding.

Result: \bench enables evaluation of LLMs’ ability to integrate multi-source signals, reason over inconsistencies, and adapt to evolving discourse in consumer domain discussions.

Conclusion: The benchmark bridges a critical gap in evaluating LLMs’ intent understanding capabilities for real-world public discussions, providing a robust framework for assessing analytical reasoning and contextual interpretation skills.

Abstract: Understanding human intent is a complex, high-level task for large language models (LLMs), requiring analytical reasoning, contextual interpretation, dynamic information aggregation, and decision-making under uncertainty. Real-world public discussions, such as consumer product discussions, are rarely linear or involve a single user. Instead, they are characterized by interwoven and often conflicting perspectives, divergent concerns, goals, emotional tendencies, as well as implicit assumptions and background knowledge about usage scenarios. To accurately understand such explicit public intent, an LLM must go beyond parsing individual sentences; it must integrate multi-source signals, reason over inconsistencies, and adapt to evolving discourse, similar to how experts in fields like politics, economics, or finance approach complex, uncertain environments. Despite the importance of this capability, no large-scale benchmark currently exists for evaluating LLMs on real-world human intent understanding, primarily due to the challenges of collecting real-world public discussion data and constructing a robust evaluation pipeline. To bridge this gap, we introduce \bench, the first dynamic, live evaluation benchmark specifically designed for intent understanding, particularly in the consumer domain. \bench is the largest and most diverse benchmark of its kind, supporting real-time updates while preventing data contamination through an automated curation pipeline.

Method: Proposed MedREK framework with shared query-key module for precise matching and attention-based prompt encoder for informative guidance. Also constructed MedVersa benchmark for evaluating single and batch edits under strict locality constraints.

Result: Experimental results show MedREK achieves superior performance across different core metrics and provides the first validated solution for batch-editing in medical LLMs.

Conclusion: MedREK offers an effective retrieval-based editing solution that addresses key challenges in medical LLM editing and enables practical batch-editing capabilities for real-world healthcare applications.

Abstract: LLMs hold great promise for healthcare applications, but the rapid evolution of medical knowledge and errors in training data often cause them to generate outdated or inaccurate information, limiting their applicability in high-stakes clinical practice. Model editing has emerged as a potential remedy without full retraining. While parameter-based editing often compromises locality and is thus ill-suited for the medical domain, retrieval-based editing offers a more viable alternative. However, it still faces two critical challenges: (1) representation overlap within the medical knowledge space often causes inaccurate retrieval and reduces editing accuracy; (2) existing methods are restricted to single-sample edits, while batch-editing remains largely unexplored despite its importance for real-world medical applications. To address these challenges, we first construct MedVersa, \hk{an enhanced benchmark with broader coverage of medical subjects, designed to evaluate both single and batch edits under strict locality constraints}. We then propose MedREK, a retrieval-based editing framework that integrates a shared query-key module for precise matching with an attention-based prompt encoder for informative guidance. Experimental results on various medical benchmarks demonstrate that our MedREK achieves superior performance across different core metrics and provides the first validated solution for batch-editing in medical LLMs. Our code and dataset are available at https://github.com/mylittleriver/MedREK.

Method: Analyzed attention heads to identify locally vs globally focused processing, developed two metrics (Windowed Average Attention Distance and Future Attention Influence), identified preplan-and-anchor mechanism, and introduced three novel RL strategies for targeted credit assignment.

Result: The approach revealed a recurring preplan-and-anchor reasoning mechanism and showed consistent performance gains across various reasoning tasks when using the targeted RL strategies.

Conclusion: By aligning optimization with the model’s intrinsic reasoning rhythm, this work transforms opaque optimization into an actionable structure-aware process, offering a step toward more transparent and effective LLM reasoning optimization.

Abstract: The reasoning pattern of Large language models (LLMs) remains opaque, and Reinforcement learning (RL) typically applies uniform credit across an entire generation, blurring the distinction between pivotal and routine steps. This work positions attention as a privileged substrate that renders the internal logic of LLMs legible, not merely as a byproduct of computation, but as a mechanistic blueprint of reasoning itself. We first distinguish attention heads between locally and globally focused information processing and reveal that locally focused heads produce a sawtooth pattern near the diagonal indicating phrasal chunks, while globally focused heads expose tokens that exert broad downstream influence over future tokens. We formalize these with two metrics:

1. Windowed Average Attention Distance, which measures the extent of backward attention within a clipped window; 2) Future Attention Influence, which quantifies a token’s global importance as the average attention it receives from subsequent tokens. Taken together, these signals reveal a recurring preplan-and-anchor mechanism, where the model first performs a long-range contextual reference to generate an introductory token, which is immediately followed by or coincides with a semantic anchor token that organizes subsequent reasoning. Leveraging these insights, we introduce three novel RL strategies that dynamically perform targeted credit assignment to critical nodes (preplan tokens, anchor tokens, and their temporal coupling) and show consistent performance gains across various reasoning tasks. By aligning optimization with the model’s intrinsic reasoning rhythm, we aim to transform opaque optimization into an actionable structure-aware process, hoping to offer a potential step toward more transparent and effective optimization of LLM reasoning.

Method: Identify language-specific neurons using Language Activation Probability Entropy and fine-tune only these dedicated subnetworks on target-language data, rather than updating the entire model.

Result: Experiments on Llama-3.1-8B and Mistral-Nemo-12B across 12 mid- and low-resource languages show consistent outperformance over full fine-tuning, FFN-only fine-tuning, LoRA adaptation, and random subset fine-tuning baselines.

Conclusion: The method provides a cost-effective pathway for adapting state-of-the-art models to underrepresented languages while preserving general-purpose performance, with additional benefits including enhanced training dynamics and cross-lingual alignment.

Abstract: Large language models exhibit uneven performance across languages, with substantial gaps between high- and low-resource languages. We present a framework for enhancing monolingual capabilities of LLMs in underrepresented languages while preserving their general-purpose performance through targeted fine-tuning of language-specific subnetworks. Our approach identifies language-specific neurons using Language Activation Probability Entropy and fine-tunes only the weights associated with these neurons, a dedicated subnetwork, on target-language data. Experiments on Llama-3.1-8B and Mistral-Nemo-12B across 12 mid- and low-resource languages demonstrate that our method consistently outperforms full fine-tuning, FFN-only fine-tuning, LoRA adaptation, and random subset fine-tuning baselines while efficiently updating only up to 1% of model parameters. Beyond performance improvements, we observe enhanced favorable training dynamics, cross-lingual representational alignment, and systematic weight update changes. To facilitate future research, we release language-specific neuron identifications for over 100 languages as well as our adaptation pipeline, offering a cost-effective pathway for adapting state-of-the-art models to underrepresented languages.

Method: Combined two strategies: (i) lightweight prompting techniques including Deflanderization method to reduce excessive role-play, and (ii) fine-tuned Qwen3-14B models using supervised fine-tuning and LoRA adaptation.

Result: Achieved 2nd place on Task 1, 2nd place on Task 3 (API track), and 4th place on Task 3 (GPU track) in the CPDC 2025 Round 2 competition.

Conclusion: The combination of prompting techniques and fine-tuned models effectively enables dynamic NPC creation with both task execution and persona-consistent dialogue capabilities.

Abstract: The emergence of large language models (LLMs) has opened new opportunities for cre- ating dynamic non-player characters (NPCs) in gaming environments, enabling both func- tional task execution and persona-consistent dialogue generation. In this paper, we (Tu_Character_lab) report our participation in the Commonsense Persona-Grounded Dialogue Challenge (CPDC) 2025 Round 2, which eval- uates agents across three tracks: task-oriented dialogue, context-aware dialogue, and their integration. Our approach combines two complementary strategies: (i) lightweight prompting techniques in the API track, including a Deflanderization prompting method to suppress excessive role-play and improve task fidelity, and (ii) fine-tuned large models in the GPU track, leveraging Qwen3-14B with supervisedfinetuning (SFT) and Low-Rank Adaptation(LoRA). Our best submissions ranked 2nd on Task 1, 2nd on Task 3 (API track), and 4th on Task 3 (GPU track).

Method: FreshTab generates table-to-text benchmarks on-the-fly from Wikipedia, collecting datasets in different languages (English, German, Russian, French) to ensure data freshness and domain balance.

Result: LLMs perform worse on recent tables by automatic metrics, but this doesn’t align with LLM and human evaluations. Domain effects are visible across all evaluations, showing domain-balanced benchmarks are more challenging.

Conclusion: FreshTab provides a contamination-free, domain-sensitive evaluation framework for table-to-text generation, revealing that domain-balanced benchmarks present greater challenges than traditional imbalanced ones.

Abstract: Table-to-text generation (insight generation from tables) is a challenging task that requires precision in analyzing the data. In addition, the evaluation of existing benchmarks is affected by contamination of Large Language Model (LLM) training data as well as domain imbalance. We introduce FreshTab, an on-the-fly table-to-text benchmark generation from Wikipedia, to combat the LLM data contamination problem and enable domain-sensitive evaluation. While non-English table-to-text datasets are limited, FreshTab collects datasets in different languages on demand (we experiment with German, Russian and French in addition to English). We find that insights generated by LLMs from recent tables collected by our method appear clearly worse by automatic metrics, but this does not translate into LLM and human evaluations. Domain effects are visible in all evaluations, showing that a~domain-balanced benchmark is more challenging.

Method: NOSA decomposes token selection into query-aware and query-agnostic components with explicit locality constraints, reducing KV transfers while preserving the same attention computation as training.

Result: Pretrained 1B-parameter model achieves up to 2.3x improvement in decoding throughput compared to vanilla trainable sparse attention baseline (InfLLM-V2) while preserving near-lossless performance.

Conclusion: NOSA effectively addresses KV cache bottleneck in sparse attention by leveraging locality constraints for efficient offloading, significantly boosting decoding throughput without performance degradation.

Abstract: Trainable sparse attention has emerged as a promising solution to address the decoding efficiency bottleneck of LLMs in long-context processing, significantly saving memory accesses while minimally impacting task performance. However, existing sparse attention methods leave a crucial limitation unresolved: the size of the key-value (KV) cache remains unreduced, which constrains on-GPU batch sizes and throttles decoding throughput, especially in large-scale batched inference. In this paper, we show that trainable sparse attention naturally exhibits strong locality in token selection across adjacent decoding steps, thereby enabling KV cache offloading without altering the underlying attention computation. However, the inherent locality remains insufficient to achieve efficient offloading, as the transfer of selected KV pairs between the CPU and GPU continues to dominate the overall decoding cost. Building on this insight, we present NOSA, a trainable sparse attention framework designed to natively support KV cache offloading. NOSA introduces explicit locality constraints by decomposing token selection into query-aware and query-agnostic components, thereby reducing KV transfers while preserving the same attention computation as used during training. We pretrain a 1B-parameter model with NOSA and conduct extensive benchmarks, showing that it preserves near-lossless performance while achieving up to a 2.3x improvement in decoding throughput compared with the vanilla trainable sparse attention baseline (InfLLM-V2).

Method: Proposes MemoTime framework with: hierarchical Tree of Time decomposition, operator-aware reasoning with monotonic timestamps, dynamic evidence retrieval with operator-specific strategies, and self-evolving experience memory storing reasoning traces and embeddings.

Result: Achieves state-of-the-art results on multiple temporal QA benchmarks, outperforming strong baselines by up to 24.0%. Enables smaller models (Qwen3-4B) to achieve reasoning performance comparable to GPT-4-Turbo.

Conclusion: MemoTime effectively addresses temporal reasoning challenges through structured grounding, recursive reasoning, and continual experience learning, demonstrating significant performance improvements and enabling efficient reasoning with smaller models.

Abstract: Large Language Models (LLMs) have achieved impressive reasoning abilities, but struggle with temporal understanding, especially when questions involve multiple entities, compound operators, and evolving event sequences. Temporal Knowledge Graphs (TKGs), which capture vast amounts of temporal facts in a structured format, offer a reliable source for temporal reasoning. However, existing TKG-based LLM reasoning methods still struggle with four major challenges: maintaining temporal faithfulness in multi-hop reasoning, achieving multi-entity temporal synchronization, adapting retrieval to diverse temporal operators, and reusing prior reasoning experience for stability and efficiency. To address these issues, we propose MemoTime, a memory-augmented temporal knowledge graph framework that enhances LLM reasoning through structured grounding, recursive reasoning, and continual experience learning. MemoTime decomposes complex temporal questions into a hierarchical Tree of Time, enabling operator-aware reasoning that enforces monotonic timestamps and co-constrains multiple entities under unified temporal bounds. A dynamic evidence retrieval layer adaptively selects operator-specific retrieval strategies, while a self-evolving experience memory stores verified reasoning traces, toolkit decisions, and sub-question embeddings for cross-type reuse. Comprehensive experiments on multiple temporal QA benchmarks show that MemoTime achieves overall state-of-the-art results, outperforming the strong baseline by up to 24.0%. Furthermore, MemoTime enables smaller models (e.g., Qwen3-4B) to achieve reasoning performance comparable to that of GPT-4-Turbo.

Method: Created over 500,000 question-answer pairs from ICD-10-GM, ICD-O-3, and OPS catalogues, then fine-tuned eight open-weight models (7-70B parameters) from Qwen, Llama, and Mistral families.

Result: ICD-10-GM accuracy improved from 1.4-24% to 41-58%, partial accuracy from 31-74% to 73-83%. ICD-O-3 topography coding improved but remained lower (22-40% exact, 56-67% partial). Malformed outputs dropped to 0%, tumor recognition reached 99%.

Conclusion: Fine-tuning on public catalogues effectively improves LLMs for medical documentation tasks, with accuracy correlating with model size but gaps narrowing after fine-tuning.

Abstract: Accurate coding of tumor diagnoses with ICD-10-GM and ICD-O-3 is essential for structured cancer documentation in Germany. Smaller open-weight LLMs are appealing for privacy-preserving automation but often struggle with coding accuracy in German-language contexts. This study investigates whether instruction-based fine-tuning on public datasets improves the coding accuracy of open-weight LLMs for German tumor diagnosis texts. The evaluation uses coded diagnoses from the local tumor documentation system as test data. In a systematic data quality assessment, the upper limit for ICD-10 coding performance was estimated at 60-79% for exact and 81-94% for partial (three-character codes only) derivation. As training data, over 500,000 question-answer pairs were created based on the ICD-10-GM, ICD-O-3, and OPS catalogues. Eight open-weight models from the Qwen, Llama, and Mistral families (7-70 B parameters) were fine-tuned. ICD-10-GM accuracy rose from 1.4-24% to 41-58%, and partial accuracy from 31-74% to 73-83%. The accuracy of ICD-O-3 topography coding also improved but started and remained considerably lower with an exact accuracy of 22-40% and a partial accuracy of 56-67% after fine-tuning. Malformed code outputs dropped to 0% for all models. Tumor-diagnosis recognition reached 99%. Accuracy correlated positively with model size, but gaps between small and large models narrowed after fine-tuning. The reasoning mode in Qwen3 generally yielded a lower performance than fine-tuning and was over 100 times slower. Our findings highlight the potential of leveraging public catalogues to build instruction datasets that improve LLMs in medical documentation tasks. The complete training dataset and the best-performing checkpoints of the fine-tuned models are available from https://huggingface.co/datasets/stefan-m-lenz/ICDOPS-QA-2024.

Method: Systematically examined how sampling-based decoding impacts detectability by testing various decoding parameters (temperature, top-p, nucleus sampling) and analyzing subtle variations in word-level distributions.

Result: Minor adjustments to decoding parameters severely impair detector accuracy, with AUROC dropping from near-perfect levels to 1% in some cases, revealing critical vulnerabilities in current detection methods.

Conclusion: Current text detection methods have critical blind spots and require more comprehensive evaluation protocols; the authors release a large-scale dataset and evaluation framework to facilitate future research.

Abstract: As texts generated by Large Language Models (LLMs) are ever more common and often indistinguishable from human-written content, research on automatic text detection has attracted growing attention. Many recent detectors report near-perfect accuracy, often boasting AUROC scores above 99%. However, these claims typically assume fixed generation settings, leaving open the question of how robust such systems are to changes in decoding strategies. In this work, we systematically examine how sampling-based decoding impacts detectability, with a focus on how subtle variations in a model’s (sub)word-level distribution affect detection performance. We find that even minor adjustments to decoding parameters - such as temperature, top-p, or nucleus sampling - can severely impair detector accuracy, with AUROC dropping from near-perfect levels to 1% in some settings. Our findings expose critical blind spots in current detection methods and emphasize the need for more comprehensive evaluation protocols. To facilitate future research, we release a large-scale dataset encompassing 37 decoding configurations, along with our code and evaluation framework https://github.com/BaggerOfWords/Sampling-and-Detection

Method: Developed a fully automated pipeline that assembles evidence neighborhoods, creates dual graph and tree representations, generates questions across G-levels, synthesizes rubrics using a DeepResearch agent mimicking GRADE-consistent evidence review, and scores with LLM judge ensembles.

Result: Validation confirmed high-quality automated questions aligned with clinician judgment. Evaluation revealed key failure modes: performance degrades with increased reasoning depth, models struggle with answer completeness, are vulnerable to adversarial perturbations, and have safety issues.

Conclusion: The GAPS framework provides a reproducible and scalable method for rigorously evaluating AI clinician systems and guiding their development toward safer, more reliable clinical practice.

Abstract: Current benchmarks for AI clinician systems, often based on multiple-choice exams or manual rubrics, fail to capture the depth, robustness, and safety required for real-world clinical practice. To address this, we introduce the GAPS framework, a multidimensional paradigm for evaluating \textbf{G}rounding (cognitive depth), \textbf{A}dequacy (answer completeness), \textbf{P}erturbation (robustness), and \textbf{S}afety. Critically, we developed a fully automated, guideline-anchored pipeline to construct a GAPS-aligned benchmark end-to-end, overcoming the scalability and subjectivity limitations of prior work. Our pipeline assembles an evidence neighborhood, creates dual graph and tree representations, and automatically generates questions across G-levels. Rubrics are synthesized by a DeepResearch agent that mimics GRADE-consistent, PICO-driven evidence review in a ReAct loop. Scoring is performed by an ensemble of large language model (LLM) judges. Validation confirmed our automated questions are high-quality and align with clinician judgment. Evaluating state-of-the-art models on the benchmark revealed key failure modes: performance degrades sharply with increased reasoning depth (G-axis), models struggle with answer completeness (A-axis), and they are highly vulnerable to adversarial perturbations (P-axis) as well as certain safety issues (S-axis). This automated, clinically-grounded approach provides a reproducible and scalable method for rigorously evaluating AI clinician systems and guiding their development toward safer, more reliable clinical practice.

Method: Evaluated GPT-4o, GPT-5, Perplexity, and Qwen Chat using 100 claims across five misinformation-prone topics, assessing source credibility and response groundedness with cited sources.

Result: Perplexity achieved highest source credibility, while GPT-4o showed elevated citation of non-credible sources on sensitive topics, revealing differences between assistants.

Conclusion: Provides the first systematic comparison of chat assistants for fact-checking behavior, establishing a foundation for evaluating AI systems in high-stakes information environments.

Abstract: Chat assistants increasingly integrate web search functionality, enabling them to retrieve and cite external sources. While this promises more reliable answers, it also raises the risk of amplifying misinformation from low-credibility sources. In this paper, we introduce a novel methodology for evaluating assistants’ web search behavior, focusing on source credibility and the groundedness of responses with respect to cited sources. Using 100 claims across five misinformation-prone topics, we assess GPT-4o, GPT-5, Perplexity, and Qwen Chat. Our findings reveal differences between the assistants, with Perplexity achieving the highest source credibility, whereas GPT-4o exhibits elevated citation of non-credibility sources on sensitive topics. This work provides the first systematic comparison of commonly used chat assistants for fact-checking behavior, offering a foundation for evaluating AI systems in high-stakes information environments.

Method: Extends uncertainty quantification by leveraging raw FFN activations as auto-regressive signals, models confidence prediction as sequence classification, and uses Huber loss regularization for robustness against noisy supervision.

Result: Outperforms strong baselines in real-world financial customer-support settings, maintains high accuracy under strict latency constraints, and preserves accuracy using activations from only the 16th layer in Llama 3.1 8B model.

Conclusion: Activation-based confidence modeling provides a scalable, architecture-aware approach for trustworthy RAG deployment in high-stakes applications.

Abstract: We propose a method for confidence estimation in retrieval-augmented generation (RAG) systems that aligns closely with the correctness of large language model (LLM) outputs. Confidence estimation is especially critical in high-stakes domains such as finance and healthcare, where the cost of an incorrect answer outweighs that of not answering the question. Our approach extends prior uncertainty quantification methods by leveraging raw feed-forward network (FFN) activations as auto-regressive signals, avoiding the information loss inherent in token logits and probabilities after projection and softmax normalization. We model confidence prediction as a sequence classification task, and regularize training with a Huber loss term to improve robustness against noisy supervision. Applied in a real-world financial industry customer-support setting with complex knowledge bases, our method outperforms strong baselines and maintains high accuracy under strict latency constraints. Experiments on Llama 3.1 8B model show that using activations from only the 16th layer preserves accuracy while reducing response latency. Our results demonstrate that activation-based confidence modeling offers a scalable, architecture-aware path toward trustworthy RAG deployment.

Method: Developed a controlled evaluation framework using mechanistic and causal analysis to trace symbol grounding in internal computations, testing across different architectures (Transformers, state-space models, unidirectional LSTMs).

Result: Grounding concentrates in middle-layer computations and operates through attention heads that aggregate environmental information to support linguistic predictions. This phenomenon replicates in multimodal dialogue and across Transformers/state-space models but not in unidirectional LSTMs.

Conclusion: Symbol grounding can emerge in language models through specific computational mechanisms, providing behavioral and mechanistic evidence with practical implications for predicting and controlling generation reliability.

Abstract: Symbol grounding (Harnad, 1990) describes how symbols such as words acquire their meanings by connecting to real-world sensorimotor experiences. Recent work has shown preliminary evidence that grounding may emerge in (vision-)language models trained at scale without using explicit grounding objectives. Yet, the specific loci of this emergence and the mechanisms that drive it remain largely unexplored. To address this problem, we introduce a controlled evaluation framework that systematically traces how symbol grounding arises within the internal computations through mechanistic and causal analysis. Our findings show that grounding concentrates in middle-layer computations and is implemented through the aggregate mechanism, where attention heads aggregate the environmental ground to support the prediction of linguistic forms. This phenomenon replicates in multimodal dialogue and across architectures (Transformers and state-space models), but not in unidirectional LSTMs. Our results provide behavioral and mechanistic evidence that symbol grounding can emerge in language models, with practical implications for predicting and potentially controlling the reliability of generation.

Method: Periodically compress generation KV cache with learned special-purpose tokens and evict compressed entries, trained via modified joint distillation and reinforcement learning framework.

Result: Achieves superior memory-accuracy Pareto frontier compared to models without cache compression and training-free compression techniques.

Conclusion: The proposed compression method effectively reduces KV cache overhead while maintaining accuracy, providing better scalability for long-context reasoning.

Abstract: The scalability of large language models for long-context reasoning is severely constrained by the linear growth of their Transformer key-value cache, which incurs significant memory and computational costs. We posit that as a model generates reasoning tokens, the informational value of past generated tokens diminishes, creating an opportunity for compression. In this work, we propose to periodically compress the generation KV cache with a learned, special-purpose token and evict compressed entries. We train the model to perform this compression via a modified joint distillation and reinforcement learning (RL) framework. Our training method minimizes overhead over the conventional RL process, as it leverages RL outputs for distillation. Empirically, our method achieves a superior memory-accuracy Pareto frontier compared to both the model without cache compression and training-free compression techniques.

Method: BRIEF-Pro is trained on seed data with short contexts (<1k words) to perform abstractive compression of extended contexts (>10k words) across various scenarios. It allows flexible user control over summary length by specifying desired number of sentences.

Result: Experiments on four open-domain multi-hop QA datasets show BRIEF-Pro generates more concise and relevant summaries, enhancing performance across different language models. With 70B reader model, 32x compression improves QA performance by 4.67% on average over LongLLMLingua’s 9x compression, while requiring only 23% of its computational overhead.

Conclusion: BRIEF-Pro effectively addresses the context expansion bottleneck in RAG systems by providing efficient compression that improves performance while reducing computational costs.

Abstract: As retrieval-augmented generation (RAG) tackles complex tasks, increasingly expanded contexts offer richer information, but at the cost of higher latency and increased cognitive load on the model. To mitigate this bottleneck, especially for intricate multi-hop questions, we introduce BRIEF-Pro. It is a universal, lightweight compressor that distills relevant evidence for a given query from retrieved documents into a concise summary for seamless integration into in-context RAG. Using seed data consisting of relatively short contexts (fewer than 1k words), BRIEF-Pro is trained to perform abstractive compression of extended contexts exceeding 10k words across a wide range of scenarios. Furthermore, BRIEF-Pro offers flexible user control over summary length by allowing users to specify the desired number of sentences. Experiments on four open-domain multi-hop question-answering datasets show that BRIEF-Pro generates more concise and relevant summaries, enhancing performance across small, large, and proprietary language models. With the 70B reader model, 32x compression by BRIEF-Pro improves QA performance by 4.67% on average over LongLLMLingua’s 9x, while requiring only 23% of its computational overhead.

Method: Developed MULTI dataset with 18,000+ exam questions, including MULTI-Elite (500 hard questions) and MULTI-Extend (4,500+ context pieces) for testing multimodal comprehension, reasoning, and in-context learning.

Result: Qwen2-VL-72B achieved 76.9% on MULTI and 53.1% on MULTI-Elite, while human experts scored 86.1% and 73.1% respectively, showing significant performance gap.

Conclusion: MULTI serves as a robust evaluation platform revealing substantial room for MLLM advancement toward expert-level AI capabilities.

Abstract: The rapid development of multimodal large language models (MLLMs) raises the question of how they compare to human performance. While existing datasets often feature synthetic or overly simplistic tasks, some models have already surpassed human expert baselines. In this paper, we present MULTI, a Chinese multimodal dataset derived from authentic examination questions. Comprising over 18,000 carefully selected and refined questions, MULTI evaluates models using real-world examination standards, encompassing image-text comprehension, complex reasoning, and knowledge recall. Additionally, We also introduce MULTI-Elite, a 500-question selected hard subset, and MULTI-Extend with more than 4,500 external knowledge context pieces for testing in-context learning capabilities. Our evaluation highlights substantial room for MLLM advancement, with Qwen2-VL-72B achieving a 76.9% accuracy on MULTI and 53.1% on MULTI-Elite leading 25 evaluated models, compared to human expert baselines of 86.1% and 73.1%. MULTI serves not only as a robust evaluation platform but also paves the way for the development of expert-level AI.

Method: A region-aware bottom-up approach that identifies gender-aligned topics for specific regions, creates bias dimension topic pairs, and uses them in a Word Embedding Association Test (WEAT)-based evaluation metric.

Result: The proposed bias topic pairs align better with human perception of gender biases in different regions compared to existing ones, and LLMs show higher alignment to bias pairs for highly-represented regions.

Conclusion: Region-aware bias evaluation is crucial as it reveals that language models exhibit different gender biases across regions, and existing universal metrics may not adequately capture these regional variations.

Abstract: When exposed to human-generated data, language models are known to learn and amplify societal biases. While previous works introduced benchmarks that can be used to assess the bias in these models, they rely on assumptions that may not be universally true. For instance, a gender bias dimension commonly used by these metrics is that of family–career, but this may not be the only common bias in certain regions of the world. In this paper, we identify topical differences in gender bias across different regions and propose a region-aware bottom-up approach for bias assessment. Our proposed approach uses gender-aligned topics for a given region and identifies gender bias dimensions in the form of topic pairs that are likely to capture gender societal biases. Several of our proposed bias topic pairs are on par with human perception of gender biases in these regions in comparison to the existing ones, and we also identify new pairs that are more aligned than the existing ones. In addition, we use our region-aware bias topic pairs in a Word Embedding Association Test (WEAT)-based evaluation metric to test for gender biases across different regions in different data domains. We also find that LLMs have a higher alignment to bias pairs for highly-represented regions showing the importance of region-aware bias evaluation metric.

Method: Extended BBQ dataset to Open-BBQ with fill-in-the-blank and short-answer questions, developed evaluation process for open-ended content, and proposed Composite Prompting - an ICL method combining structured examples with explicit chain-of-thought reasoning.

Result: Experimental results show that the proposed method significantly reduces bias for both GPT-3.5 and GPT-4o while maintaining high accuracy.

Conclusion: Open-BBQ provides a more comprehensive framework for evaluating social bias in LLMs, and Composite Prompting effectively addresses over-correction issues in debiasing while maintaining model accuracy.

Abstract: Current social bias benchmarks for Large Language Models (LLMs) primarily rely on predefined question formats like multiple-choice, limiting their ability to reflect the complexity and open-ended nature of real-world interactions. To close this gap, we extend an existing dataset BBQ (Parrish et al., 2022) to Open-BBQ, a comprehensive framework to evaluate the social bias of LLMs in open-ended settings by incorporating two additional question categories: fill-in-the-blank and short-answer. Since our new Open-BBQ dataset contains a lot of open-ended responses like sentences and paragraphs, we developed an evaluation process to detect biases from open-ended content by labeling sentences and paragraphs. In addition to this, we also found that existing debiasing methods, such as self-debiasing (Gallegos et al., 2024), have over-correction issues, which make the original correct answers incorrect. In order to solve this issue, we propose Composite Prompting, an In-context Learning (ICL) method combining structured examples with explicit chain-of-thought reasoning to form a unified instruction template for LLMs to explicitly identify content that needs debiasing. Experimental results show that the proposed method significantly reduces the bias for both GPT-3.5 and GPT-4o while maintaining high accuracy.

Method: Uses information-theoretic guidance for parameter selection, contrastive mechanisms for representation separation, and projects conflict gradients onto orthogonal subspaces to resolve forgetting vs retention conflicts.

Result: Extensive experiments show FALCON achieves superior unlearning effectiveness while maintaining model utility, with robust resistance against knowledge recovery attempts.

Conclusion: FALCON provides an effective solution for precise knowledge removal from LLMs while preserving overall model performance.

Abstract: Large language models have been widely applied, but can inadvertently encode sensitive or harmful information, raising significant safety concerns. Machine unlearning has emerged to alleviate this concern; however, existing training-time unlearning approaches, relying on coarse-grained loss combinations, have limitations in precisely separating knowledge and balancing removal effectiveness with model utility. In contrast, we propose Fine-grained Activation manipuLation by Contrastive Orthogonal uNalignment (FALCON), a novel representation-guided unlearning approach that leverages information-theoretic guidance for efficient parameter selection, employs contrastive mechanisms to enhance representation separation, and projects conflict gradients onto orthogonal subspaces to resolve conflicts between forgetting and retention objectives. Extensive experiments demonstrate that FALCON achieves superior unlearning effectiveness while maintaining model utility, exhibiting robust resistance against knowledge recovery attempts.

Method: Curated a dataset with valid System 1 and System 2 answers, aligned LLMs to these reasoning styles, interpolated between extremes by varying alignment data proportion, and combined models based on generation entropy without additional training.

Result: System 2-aligned models excel in arithmetic and symbolic reasoning, while System 1-aligned models perform better in commonsense reasoning. The dynamic combination model outperforms across nearly all benchmarks.

Conclusion: Step-by-step reasoning is not always optimal; reasoning strategies should be adapted based on task demands, and combining System 1 and System 2 approaches can create more flexible and effective LLMs.

Abstract: Large Language Models (LLMs) exhibit impressive reasoning abilities, yet their reliance on structured step-by-step processing reveals a critical limitation. In contrast, human cognition fluidly adapts between intuitive, heuristic (System 1) and analytical, deliberative (System 2) reasoning depending on the context. This difference between human cognitive flexibility and LLMs’ reliance on a single reasoning style raises a critical question: while human fast heuristic reasoning evolved for its efficiency and adaptability, is a uniform reasoning approach truly optimal for LLMs, or does its inflexibility make them brittle and unreliable when faced with tasks demanding more agile, intuitive responses? To answer these questions, we explicitly align LLMs to these reasoning styles by curating a dataset with valid System 1 and System 2 answers, and evaluate their performance across reasoning benchmarks. Our results reveal an accuracy-efficiency trade-off: System 2-aligned models excel in arithmetic and symbolic reasoning, while System 1-aligned models perform better in commonsense reasoning tasks. To analyze the reasoning spectrum, we interpolated between the two extremes by varying the proportion of alignment data, which resulted in a monotonic change in accuracy. A mechanistic analysis of model responses shows that System 1 models employ more definitive outputs, whereas System 2 models demonstrate greater uncertainty. Building on these findings, we further combine System 1- and System 2-aligned models based on the entropy of their generations, without additional training, and obtain a dynamic model that outperforms across nearly all benchmarks. This work challenges the assumption that step-by-step reasoning is always optimal and highlights the need for adapting reasoning strategies based on task demands.

Method: Created InstrEditBench with 30,000+ structured editing tasks across multiple domains, then developed FineEdit - a specialized editing model trained on this benchmark for context-aware text modifications.

Result: FineEdit outperforms state-of-the-art models: ~10% improvement over Gemini on single-turn edits, up to 30% over Llama-3.2-3B, and over 40% better than Mistral-7B-OpenOrca on direct editing tasks. It also generalizes well to multi-turn editing scenarios.

Conclusion: FineEdit demonstrates superior performance for precise text editing tasks and has practical applicability for realistic editing scenarios. The benchmark and model are publicly released to facilitate further research.

Abstract: Large Language Models (LLMs) have significantly advanced natural language processing, demonstrating strong capabilities in tasks such as text generation, summarization, and reasoning. Recently, their potential for automating precise text editing tasks across specialized domains, such as programming code, LaTeX, and structured database languages, has gained attention. However, current state-of-the-art LLMs still struggle with executing precise, instruction-driven edits, particularly when structural accuracy and strict adherence to domain conventions are required. To address these challenges, we introduce InstrEditBench, an automated benchmark dataset comprising over 30,000 structured editing tasks spanning diverse domains, including Wikipedia articles, LaTeX documents, source code, and database languages. Using this benchmark, we develop FineEdit, a specialized editing model explicitly trained for accurate, context-aware text modifications. Experimental evaluations demonstrate that FineEdit outperforms state-of-the-art models, achieving improvements of approximately 10% over Gemini models on single-turn edits, up to 30% over Llama-3.2-3B, and exceeding Mistral-7B-OpenOrca performance by over 40% on direct editing tasks. FineEdit also effectively generalizes to realistic multi-turn editing scenarios, highlighting its practical applicability. To facilitate further research and reproducibility, we release FineEdit at https://github.com/StuRinDQB/FineEdit} and https://huggingface.co/datasets/YimingZeng/FineEdit_bench.

Method: ICA-RAG uses an adaptive control module to assess input information completeness and determine retrieval necessity. It incorporates knowledge filtering to better align retrieval operations with clinical requirements.

Result: Experiments on three Chinese electronic medical record datasets demonstrate that ICA-RAG significantly outperforms baseline methods in clinical diagnosis.

Conclusion: ICA-RAG effectively enhances RAG reliability in disease diagnosis by adaptively controlling retrieval based on information completeness, improving both efficiency and accuracy.

Abstract: Retrieval-Augmented Large Language Models (LLMs), which integrate external knowledge, have shown remarkable performance in medical domains, including clinical diagnosis. However, existing RAG methods often struggle to tailor retrieval strategies to diagnostic difficulty and input sample informativeness. This limitation leads to excessive and often unnecessary retrieval, impairing computational efficiency and increasing the risk of introducing noise that can degrade diagnostic accuracy. To address this, we propose ICA-RAG (\textbf{I}nformation \textbf{C}ompleteness Guided \textbf{A}daptive \textbf{R}etrieval-\textbf{A}ugmented \textbf{G}eneration), a novel framework for enhancing RAG reliability in disease diagnosis. ICA-RAG utilizes an adaptive control module to assess the necessity of retrieval based on the input’s information completeness. By optimizing retrieval and incorporating knowledge filtering, ICA-RAG better aligns retrieval operations with clinical requirements. Experiments on three Chinese electronic medical record datasets demonstrate that ICA-RAG significantly outperforms baseline methods, highlighting its effectiveness in clinical diagnosis.

Method: Self-Truncation Best-of-N (ST-BoN) leverages early sampling consistency in the model’s internal states to identify the most promising path and truncate suboptimal ones during decoding, avoiding full generation of all N samples and eliminating reward model dependency.

Result: ST-BoN reduces dynamic GPU memory usage by over 80% and inference latency by 50%. It achieves same performance as Full-BoN while saving 70%-80% computational cost, and improves accuracy by 3-4 points under same cost.

Conclusion: ST-BoN effectively addresses both efficiency challenges of Best-of-N sampling, providing superior cost-performance trade-off without requiring reward models.

Abstract: Test-time scaling enhances large language model performance by allocating additional compute resources during inference. Best-of-N (BoN) sampling serves as a common sampling-based scaling technique, broadening the search space in parallel to find better solutions from the model distribution. However, its cost-performance trade-off is still underexplored. Two main challenges limit the efficiency of BoN sampling: (1) Generating N full samples consumes substantial GPU memory, reducing inference capacity under limited resources. (2) Reward models add extra memory and latency overhead, and training strong reward models introduces potential training data costs. Although some studies have explored efficiency improvements, none have addressed both challenges at once. To address this gap, we propose Self-Truncation Best-of-N (ST-BoN), a decoding method that avoids fully generating all N samples and eliminates the need for reward models. It leverages early sampling consistency in the model’s internal states to identify the most promising path and truncate suboptimal ones. In terms of cost, ST-BoN reduces dynamic GPU memory usage by over 80% and inference latency by 50%. In terms of cost-performance trade-off, ST-BoN achieves the same performance as Full-BoN while saving computational cost by 70%-80%, and under the same cost, it can improve accuracy by 3-4 points.

Method: Conducted extensive experiments with four LLMs across three QA datasets covering 48 languages, evaluating their ability to use relevant passages regardless of language, respond in expected language, and focus on relevant passages despite distracting multilingual passages.

Result: LLMs showed surprising ability to extract relevant information from passages in different languages than the query, but much weaker ability to formulate full answers in the correct language. Distracting passages negatively impacted answer quality regardless of language, with distractors in query language having slightly stronger influence.

Conclusion: The findings deepen understanding of how LLMs utilize context in multilingual RAG systems and provide directions for future improvements, particularly in addressing LLMs’ weaker ability to generate answers in the correct language.

Abstract: Retrieval-augmented generation (RAG) with large language models (LLMs) has demonstrated strong performance in multilingual question-answering (QA) tasks by leveraging relevant passages retrieved from corpora. In multilingual RAG (mRAG), the retrieved passages can be written in languages other than that of the query entered by the user, making it challenging for LLMs to effectively utilize the provided information. Recent research suggests that retrieving passages from multilingual corpora can improve RAG performance, particularly for low-resource languages. However, the extent to which LLMs can leverage different kinds of multilingual contexts to generate accurate answers, independently from retrieval quality, remains understudied. In this paper, we conduct an extensive assessment of LLMs’ ability to (i) make consistent use of a relevant passage regardless of its language, (ii) respond in the expected language, and (iii) focus on the relevant passage even when multiple `distracting’ passages in different languages are provided in the context. Our experiments with four LLMs across three QA datasets covering a total of 48 languages reveal a surprising ability of LLMs to extract the relevant information from passages in a different language than the query, but a much weaker ability to formulate a full answer in the correct language. Our analysis, based on both accuracy and feature attribution techniques, further shows that distracting passages negatively impact answer quality regardless of their language. However, distractors in the query language exert a slightly stronger influence. Taken together, our findings deepen the understanding of how LLMs utilize context in mRAG systems, providing directions for future improvements.

Method: HGPrompt optimizes ensemble weights by jointly maximizing an information-theoretic transferability metric and minimizing gradient conflicts through Hessian and Fisher Information-based regularization that matches gradient variances across source prompts.

Result: Extensive experiments on the large-scale VTAB benchmark demonstrate state-of-the-art performance, validating HGPrompt’s effectiveness in learning optimal ensembles for multi-source prompt transfer.

Conclusion: HGPrompt provides an effective framework for dynamic prompt ensemble that successfully addresses gradient conflicts and optimizes transferability, enabling better generalization for new tasks through complementary knowledge from multiple source prompts.

Abstract: Prompt tuning has emerged as a lightweight strategy for adapting foundation models to downstream tasks, particularly for resource-constrained systems. As pre-trained prompts become valuable assets, combining multiple source prompts offers a promising approach to enhance generalization for new tasks by leveraging complementary knowledge. However, naive aggregation often overlooks different source prompts have different contribution potential to the target task. To address this, we propose HGPrompt, a dynamic framework that learns optimal ensemble weights. These weights are optimized by jointly maximizing an information-theoretic metric for transferability and minimizing gradient conflicts via a novel regularization strategy. Specifically, we propose a differentiable prompt transferability metric to captures the discriminability of prompt-induced features on the target task. Meanwhile, HGPrompt match the gradient variances with respect to different source prompts based on Hessian and Fisher Information, ensuring stable and coherent knowledge transfer while suppressing gradient conflicts among them. Extensive experiments on the large-scale VTAB benchmark demonstrate the state-of-the-art performance of HGPrompt, validating its effectiveness in learning an optimal ensemble for effective multi-source prompt transfer.

Method: Uses Sparse Autoencoder (SAE) to extract domain-specific subsets from large datasets, applies structured pruning with domain constraints, and performs self-data distillation using SAE-curated datasets to restore lost information.

Result: FineScope achieves competitive performance, outperforming large-scale state-of-the-art LLMs in domain-specific tasks, and enables pruned models to regain substantial original performance when fine-tuned with SAE-curated datasets.

Conclusion: The framework successfully creates efficient domain-specific LLMs that maintain strong task performance, and SAE-curated datasets also improve domain-specific accuracy when applied to pretrained LLMs without pruning.

Abstract: Training large language models (LLMs) from scratch requires significant computational resources, driving interest in developing smaller, domain-specific LLMs that maintain both efficiency and strong task performance. Medium-sized models such as LLaMA, llama} have served as starting points for domain-specific adaptation, but they often suffer from accuracy degradation when tested on specialized datasets. We introduce FineScope, a framework for deriving compact, domain-optimized LLMs from larger pretrained models. FineScope leverages the Sparse Autoencoder (SAE) framework, inspired by its ability to produce interpretable feature representations, to extract domain-specific subsets from large datasets. We apply structured pruning with domain-specific constraints, ensuring that the resulting pruned models retain essential knowledge for the target domain. To further enhance performance, these pruned models undergo self-data distillation, leveraging SAE-curated datasets to restore key domain-specific information lost during pruning. Extensive experiments and ablation studies demonstrate that FineScope achieves highly competitive performance, outperforming several large-scale state-of-the-art LLMs in domain-specific tasks. Additionally, our results show that FineScope enables pruned models to regain a substantial portion of their original performance when fine-tuned with SAE-curated datasets. Furthermore, applying these datasets to fine-tune pretrained LLMs without pruning also improves their domain-specific accuracy, highlighting the robustness of our approach.

Method: Selective Loss to Understand but Not Generate (SLUNG) - instead of uniform next-token prediction loss, it selectively avoids incentivizing generation of high-risk tokens while ensuring they remain in the model’s context window.

Result: SLUNG consistently improves models’ understanding of high-risk data (e.g., toxic content recognition) without increasing generation of such content (e.g., toxicity in responses).

Conclusion: SLUNG enables models to benefit from high-risk text that would otherwise be filtered out, allowing better understanding without increasing harmful generation.

Abstract: Language model developers typically filter out high-risk content – such as toxic or copyrighted text – from their pre-training data to prevent models from generating similar outputs. However, removing such data altogether limits models’ ability to recognize and appropriately respond to harmful or sensitive content. In this paper, we introduce Selective Loss to Understand but Not Generate (SLUNG), a pre-training paradigm through which models learn to understand high-risk data without learning to generate it. Instead of uniformly applying the next-token prediction loss, SLUNG selectively avoids incentivizing the generation of high-risk tokens while ensuring they remain within the model’s context window. As the model learns to predict low-risk tokens that follow high-risk ones, it is forced to understand the high-risk content. Through our experiments, we show that SLUNG consistently improves models' understanding of high-risk data (e.g., ability to recognize toxic content) without increasing its generation (e.g., toxicity of model responses). Overall, our SLUNG paradigm enables models to benefit from high-risk text that would otherwise be filtered out.

Method: Created S-MedQA dataset with machine and expert verification, covering 15 medical specialties with multi-specialty annotations, then used it to evaluate LLM performance across different specialty training scenarios.

Result: 1) Training on specialty data doesn’t guarantee best performance on that specialty; 2) Token probabilities of clinical terms increase consistently regardless of fine-tuning specialty, suggesting gains come from domain shifting rather than specialty-specific knowledge.

Conclusion: Improvement gains in medical QA may derive more from general domain shifting than specialty-specific knowledge injection, suggesting need to rethink fine-tuning data strategies in medical domain.

Abstract: In this paper, we introduce S-MedQA, an English medical question-answering (QA) dataset for benchmarking large language models (LLMs) in fine-grained clinical specialties. S-MedQA has over 20k examples, covers 15 medical specialties, and QA pairs can have multiple specialty annotations (e.g., when a question is cross-disciplinary), constructed with both machine and expert verification to maximize data availability. We use S-MedQA to investigate the role of clinical specialty data in the knowledge-intensive scenario of medical QA. Our results show that 1) training on data from a clinical specialty does not necessarily lead to best performance on that specialty, and 2) regardless of the specialty the LLM was fine-tuned on, token probabilities of clinically relevant terms increase consistently across all specialties. Thus, we hypothesize improvement gains are derived mostly from domain shifting (e.g., general to medical) rather than specialty-specific knowledge injection, and suggest rethinking the role of fine-tuning data in the medical domain.

Method: Developed a two-stage training strategy combining stepwise risk analysis and contrastive learning, with a multi-level taxonomy of 10 risk categories across 3 safety levels, trained on 9.2K query-reasoning trace pairs.

Result: Achieves state-of-the-art performance, outperforming LlamaGuard-4 by 35.6% and GPT-4o by 15.8% on benchmarks, while effectively generalizing across diverse reasoning paradigms and unseen scenarios.

Conclusion: ReasoningShield provides a robust solution for moderating Chain-of-Thought reasoning traces, significantly improving safety in Large Reasoning Models while maintaining strong generalization capabilities.

Abstract: Large Reasoning Models (LRMs) leverage transparent reasoning traces, known as Chain-of-Thoughts (CoTs), to break down complex problems into intermediate steps and derive final answers. However, these reasoning traces introduce unique safety challenges: harmful content can be embedded in intermediate steps even when final answers appear benign. Existing moderation tools, designed to handle generated answers, struggle to effectively detect hidden risks within CoTs. To address these challenges, we introduce ReasoningShield, a lightweight yet robust framework for moderating CoTs in LRMs. Our key contributions include: (1) formalizing the task of CoT moderation with a multi-level taxonomy of 10 risk categories across 3 safety levels, (2) creating the first CoT moderation benchmark which contains 9.2K pairs of queries and reasoning traces, including a 7K-sample training set annotated via a human-AI framework and a rigorously curated 2.2K human-annotated test set, and (3) developing a two-stage training strategy that combines stepwise risk analysis and contrastive learning to enhance robustness. Experiments show that ReasoningShield achieves state-of-the-art performance, outperforming task-specific tools like LlamaGuard-4 by 35.6% and general-purpose commercial models like GPT-4o by 15.8% on benchmarks, while also generalizing effectively across diverse reasoning paradigms, tasks, and unseen scenarios. All resources are released at https://github.com/CosmosYi/ReasoningShield.

Method: Developed Multi-Scale Probabilistic Generation Theory (MSPGT) formalizing LLMs as H-VIB systems, derived falsifiable predictions, and validated through cross-model experiments with multi-signal fusion and causal interventions on Llama and Qwen families.

Result: Experiments revealed consistent multi-scale organization across models but strong architecture-specific variations, partially supporting and refining the theoretical predictions about boundary positions and architectural dependencies.

Conclusion: MSPGT advances interpretability from descriptive observation toward predictive, information-theoretic understanding of how hierarchical structure emerges in large neural language models.

Abstract: Large Language Models (LLMs) exhibit remarkable emergent abilities but remain poorly understood at a mechanistic level. This paper introduces the Multi-Scale Probabilistic Generation Theory (MSPGT), a theoretical framework that models LLMs as Hierarchical Variational Information Bottleneck (H-VIB) systems. MSPGT posits that standard language modeling objectives implicitly optimize multi-scale information compression, leading to the spontaneous formation of three internal processing scales-Global, Intermediate, and Local. We formalize this principle, derive falsifiable predictions about boundary positions and architectural dependencies, and validate them through cross-model experiments combining multi-signal fusion and causal interventions. Results across Llama and Qwen families reveal consistent multi-scale organization but strong architecture-specific variations, partially supporting and refining the theory. MSPGT thus advances interpretability from descriptive observation toward predictive, information-theoretic understanding of how hierarchical structure emerges within large neural language models.

Method: RPM constructs structured models of user behavior using response-influential features and statistical factors, creates personalized reasoning paths, and retrieves beneficial examples through feature-based retrieval mechanisms.

Result: Extensive experiments across four diverse tasks show RPM consistently outperforms existing response-level methods while enhancing both personalization performance and interpretability.

Conclusion: RPM provides a promising direction for black-box LLM personalization by enabling reasoning-level personalization rather than just response-level matching.

Abstract: While black-box large language models are widely deployed, they produce generic outputs that overlook individual user preferences. Current personalization methods are fundamentally limited to response-level personalization; they only match final outputs, failing to model the underlying reasoning that connects user behavior to responses. To address this, this work introduces reasoning-level personalization as a new paradigm and proposes RPM, the first systematic framework designed to guide the model’s reasoning process using structured rationales constructed from patterns in a user’s behavior. RPM constructs a structured model of user behavior-built from response-influential features and statistical factors-to create personalized reasoning paths and retrieve beneficial examples for guiding inference through a feature-based retrieval mechanism. Extensive experiments across four diverse tasks demonstrate that RPM consistently outperforms existing response-level methods while simultaneously enhancing both personalization performance and interpretability, providing a promising direction for black-box LLM personalization.

Method: Proposed RedTeamCUA framework with a hybrid sandbox combining VM-based OS environment and Docker-based web platforms, featuring flexible adversarial scenario configuration and direct injection point initialization. Developed RTC-Bench benchmark with 864 examples.

Result: Significant vulnerabilities found: Claude 3.7 Sonnet | CUA had 42.9% ASR, Claude 4.5 Sonnet | CUA had highest ASR of 60%, while most secure Operator still had 7.6% ASR. CUAs attempted adversarial tasks at 92.5% rate but often failed due to capability limitations.

Conclusion: RedTeamCUA provides essential framework for systematic CUA vulnerability analysis, highlighting urgent need for robust defenses against indirect prompt injection before real-world deployment due to tangible risks to users and systems.

Abstract: Computer-use agents (CUAs) promise to automate complex tasks across operating systems (OS) and the web, but remain vulnerable to indirect prompt injection. Current evaluations of this threat either lack support realistic but controlled environments or ignore hybrid web-OS attack scenarios involving both interfaces. To address this, we propose RedTeamCUA, an adversarial testing framework featuring a novel hybrid sandbox that integrates a VM-based OS environment with Docker-based web platforms. Our sandbox supports key features tailored for red teaming, such as flexible adversarial scenario configuration, and a setting that decouples adversarial evaluation from navigational limitations of CUAs by initializing tests directly at the point of an adversarial injection. Using RedTeamCUA, we develop RTC-Bench, a comprehensive benchmark with 864 examples that investigate realistic, hybrid web-OS attack scenarios and fundamental security vulnerabilities. Benchmarking current frontier CUAs identifies significant vulnerabilities: Claude 3.7 Sonnet | CUA demonstrates an ASR of 42.9%, while Operator, the most secure CUA evaluated, still exhibits an ASR of 7.6%. Notably, CUAs often attempt to execute adversarial tasks with an Attempt Rate as high as 92.5%, although failing to complete them due to capability limitations. Nevertheless, we observe concerning high ASRs in realistic end-to-end settings, with the strongest-to-date Claude 4.5 Sonnet | CUA exhibiting the highest ASR of 60%, indicating that CUA threats can already result in tangible risks to users and computer systems. Overall, RedTeamCUA provides an essential framework for advancing realistic, controlled, and systematic analysis of CUA vulnerabilities, highlighting the urgent need for robust defenses to indirect prompt injection prior to real-world deployment.

Method: Used psycholinguistic-inspired methods to test TTS systems on sentences with ambiguous syntactic boundaries (garden path sentences, attachment ambiguity), and fine-tuned models on sentences without commas to encourage focus on deeper linguistic cues.

Result: TTS systems perform poorly on ambiguous sentences, requiring commas for correct boundary placement, but show better performance on simpler structures and improved intonation patterns after fine-tuning.

Conclusion: Current TTS systems have limited syntactic sensitivity but can be improved through targeted training to better capture underlying linguistic structure in intonation patterns.

Abstract: We analyze the syntactic sensitivity of Text-to-Speech (TTS) systems using methods inspired by psycholinguistic research. Specifically, we focus on the generation of intonational phrase boundaries, which can often be predicted by identifying syntactic boundaries within a sentence. We find that TTS systems struggle to accurately generate intonational phrase boundaries in sentences where syntactic boundaries are ambiguous (e.g., garden path sentences or sentences with attachment ambiguity). In these cases, systems need superficial cues such as commas to place boundaries at the correct positions. In contrast, for sentences with simpler syntactic structures, we find that systems do incorporate syntactic cues beyond surface markers. Finally, we finetune models on sentences without commas at the syntactic boundary positions, encouraging them to focus on more subtle linguistic cues. Our findings indicate that this leads to more distinct intonation patterns that better reflect the underlying structure.

Method: The article traces the trajectory of ALLMs from inception to present day, examining their evolution and evaluating these models through benchmarks and public leaderboards.

Result: The paper documents the fascinating and complex journey of ALLMs, highlighting how they’ve evolved from rudimentary text processing systems to sophisticated AI-driven models.

Conclusion: Developing Arabic LLMs presents an unparalleled opportunity to empower Arabic-speaking communities and bridge technological gaps, though significant challenges remain in creating effective models for this linguistically rich language.

Abstract: The emergence of ChatGPT marked a transformative milestone for Artificial Intelligence (AI), showcasing the remarkable potential of Large Language Models (LLMs) to generate human-like text. This wave of innovation has revolutionized how we interact with technology, seamlessly integrating LLMs into everyday tasks such as vacation planning, email drafting, and content creation. While English-speaking users have significantly benefited from these advancements, the Arabic world faces distinct challenges in developing Arabic-specific LLMs. Arabic, one of the languages spoken most widely around the world, serves more than 422 million native speakers in 27 countries and is deeply rooted in a rich linguistic and cultural heritage. Developing Arabic LLMs (ALLMs) presents an unparalleled opportunity to bridge technological gaps and empower communities. The journey of ALLMs has been both fascinating and complex, evolving from rudimentary text processing systems to sophisticated AI-driven models. This article explores the trajectory of ALLMs, from their inception to the present day, highlighting the efforts to evaluate these models through benchmarks and public leaderboards. We also discuss the challenges and opportunities that ALLMs present for the Arab world.

Method: Uses Maximum Mean Discrepancy (MMD) to measure distribution differences between the output and counterparts generated with various temperature parameters, analyzing the trajectory shape.

Result: Competitive performance on machine translation and summarization datasets, effectively detecting most hallucinations.

Conclusion: MMD-Flagger provides an effective non-parametric approach for hallucination detection in LLM outputs.

Abstract: Large language models (LLMs) have become pervasive in our everyday life. Yet, a fundamental obstacle prevents their use in many critical applications: their propensity to generate fluent, human-quality content that is not grounded in reality. The detection of such hallucinations is thus of the highest importance. In this work, we propose a new method to flag hallucinated content: MMD-Flagger. It relies on Maximum Mean Discrepancy (MMD), a non-parametric distance between distributions. On a high-level perspective, MMD-Flagger tracks the MMD between the output to inspect and counterparts generated with various temperature parameters. We show empirically that inspecting the shape of this trajectory is sufficient to detect most hallucinations. This novel method is benchmarked on machine translation and summarization datasets, on which it exhibits competitive performance relative to natural competitors.

Method: Introduced HC-Bench benchmark with 112 images containing hidden text, objects, and illusions, and proposed SemVink (Semantic Visual Thinking) by scaling images to low resolutions (32-128 pixels) to eliminate redundant visual noise.

Result: Leading VLMs achieved only 0-5.36% accuracy on HC-Bench, even with explicit prompting, while SemVink achieved >99% accuracy by simply scaling images to low resolutions.

Conclusion: VLMs have a critical architectural flaw prioritizing abstract reasoning over low-level visual operations; work urges shift toward hybrid models with multi-scale processing for real-world robustness in medical imaging, security, and other applications.

Abstract: Vision-language models (VLMs) excel in semantic tasks but falter at a core human capability: detecting hidden content in optical illusions or AI-generated images through perceptual adjustments like zooming. We introduce HC-Bench, a benchmark of 112 images with hidden text, objects, and illusions, revealing that leading VLMs achieve near-zero accuracy (0-5.36%)-even with explicit prompting. Humans resolve such ambiguities instinctively, yet VLMs fail due to an overreliance on high-level semantics. Strikingly, we propose SemVink (Semantic Visual Thinking) by simply scaling images to low resolutions (32-128 pixels), which unlocks >99% accuracy by eliminating redundant visual noise. This exposes a critical architectural flaw: VLMs prioritize abstract reasoning over low-level visual operations crucial for real-world robustness. Our work urges a shift toward hybrid models integrating multi-scale processing, bridging the gap between computational vision and human cognition for applications in medical imaging, security, and beyond.

Method: Fine-tuned Qwen2.5-7B-Instruct model with 6,587 QA pairs from ITU-T recommendations, built a structured KG with 13,906 entities and 13,524 relations using LLM-assisted triple extraction, and implemented a KG-RAG pipeline for knowledge retrieval.

Result: BLEU-4 score increased from 18.86 to 66.90, outperforming base model and Llama-3-8B-Instruct. KG-enhanced system improved performance across five dimensions with 2.26% average score increase, demonstrating superior factual accuracy and relevance.

Conclusion: KG2QA delivers efficient and interactive user experience through web platform and API integration, with code and data open-sourced for community use.

Abstract: The rapid evolution of communication technologies has led to an explosion of standards, rendering traditional expert-dependent consultation methods inefficient and slow. To address this challenge, we propose \textbf{KG2QA}, a question answering (QA) framework for communication standards that integrates fine-tuned large language models (LLMs) with a domain-specific knowledge graph (KG) via a retrieval-augmented generation (RAG) pipeline. We construct a high-quality dataset of 6,587 QA pairs from ITU-T recommendations and fine-tune Qwen2.5-7B-Instruct, achieving significant performance gains: BLEU-4 increases from 18.86 to 66.90, outperforming both the base model and Llama-3-8B-Instruct. A structured KG containing 13,906 entities and 13,524 relations is built using LLM-assisted triple extraction based on a custom ontology. In our KG-RAG pipeline, the fine-tuned LLMs first retrieves relevant knowledge from KG, enabling more accurate and factually grounded responses. Evaluated by DeepSeek-V3 as a judge, the KG-enhanced system improves performance across five dimensions, with an average score increase of 2.26%, demonstrating superior factual accuracy and relevance. Integrated with Web platform and API, KG2QA delivers an efficient and interactive user experience. Our code and data have been open-sourced https://github.com/luozhongze/KG2QA.

Method: Conducted experiments untangling linguistic and mathematical aspects of numbers, including ablation studies on numeral construction parameters and testing model performance with explicit vs implicit mathematical operations.

Result: LLMs cannot consistently solve cross-linguistic numeral problems unless mathematical operations are explicitly marked with known symbols; models lack the human ability to infer implicit compositional structure from numerals.

Conclusion: Flexibly inferring compositional rules from implicit patterns in human-scale data remains an open challenge for current reasoning models.

Abstract: Across languages, numeral systems vary widely in how they construct and combine numbers. While humans consistently learn to navigate this diversity, large language models (LLMs) struggle with linguistic-mathematical puzzles involving cross-linguistic numeral systems, which humans can learn to solve successfully. We investigate why this task is difficult for LLMs through a series of experiments that untangle the linguistic and mathematical aspects of numbers in language. Our experiments establish that models cannot consistently solve such problems unless the mathematical operations in the problems are explicitly marked using known symbols ($+$, $\times$, etc., as in “twenty + three”). In further ablation studies, we probe how individual parameters of numeral construction and combination affect performance. While humans use their linguistic understanding of numbers to make inferences about the implicit compositional structure of numerals, LLMs seem to lack this notion of implicit numeral structure. We conclude that the ability to flexibly infer compositional rules from implicit patterns in human-scale data remains an open challenge for current reasoning models.

Method: Introduces Branching Factor (BF) as a token-invariant measure of plausible next steps, analyzes BF patterns across generation, and conducts nudging experiments with base models.

Result: BF decreases as generation progresses; alignment tuning reduces BF by nearly 10x; aligned CoT models leverage low-BF stages for stable outputs; base models can be steered to low-BF trajectories with stylistic tokens.

Conclusion: BF is a powerful diagnostic tool that explains how alignment reduces variability, how CoT promotes stability, and how base models can be guided away from diverse outputs.

Abstract: Despite their impressive capabilities, aligned large language models (LLMs) often generate outputs that lack diversity. What drives this consistency in the generation? We investigate this phenomenon through the lens of probability concentration in the model’s output distribution. To quantify this concentration, we introduce the Branching Factor (BF)–a token-invariant measure of the effective number of plausible next steps during generation. Our empirical analysis reveals two key findings: (1) BF often decreases as generation progresses, suggesting that LLMs become more predictable as they generate. (2) alignment tuning substantially sharpens the model’s output distribution from the outset, reducing BF by nearly an order of magnitude (e.g., from 12 to 1.2) relative to base models. This stark reduction helps explain why aligned models often appear less sensitive to decoding strategies. Building on this insight, we find this consistency has surprising implications for complex reasoning. Aligned Chain-of-Thought (CoT) models (e.g., DeepSeek-distilled models), for instance, leverage this effect; by generating longer reasoning chains, they push generation into later, more deterministic (lower BF) stages, resulting in more stable outputs. We hypothesize that alignment tuning does not fundamentally change a model’s behavior, but instead steers it toward stylistic tokens (e.g., ``Sure’’) that unlock low-entropy trajectories already present in the base model. This view is supported by nudging experiments, which show prompting base models with such tokens can similarly reduce BF. Together, our findings establish BF as a powerful diagnostic for understanding and controlling LLM outputs - clarifying how alignment reduces variability, how CoT promotes stable generations, and how base models can be steered away from diversity.

Method: Cross-linguistic analysis of 3,300+ datasets from HuggingFace ecosystem using quantitative and qualitative methods to examine dataset creation and curation patterns across Chinese, Japanese, and Korean NLP communities.

Result: Found distinct patterns: Chinese datasets are large-scale and institution-driven, Korean development is grassroots community-led, and Japanese collections focus on entertainment and subculture themes.

Conclusion: Provides practical strategies for improving dataset documentation, licensing clarity, and cross-lingual resource sharing to guide more effective and culturally attuned LLM development in East Asia, with recommendations for future dataset curation and collaboration.

Abstract: Recent advances in Natural Language Processing (NLP) have underscored the crucial role of high-quality datasets in building large language models (LLMs). However, while extensive resources and analyses exist for English, the landscape for East Asian languages - particularly Chinese, Japanese, and Korean (CJK) - remains fragmented and underexplored, despite these languages together serving over 1.6 billion speakers. To address this gap, we investigate the HuggingFace ecosystem from a cross-linguistic perspective, focusing on how cultural norms, research environments, and institutional practices shape dataset availability and quality. Drawing on more than 3,300 datasets, we employ quantitative and qualitative methods to examine how these factors drive distinct creation and curation patterns across Chinese, Japanese, and Korean NLP communities. Our findings highlight the large-scale and often institution-driven nature of Chinese datasets, grassroots community-led development in Korean NLP, and an entertainment- and subculture-focused emphasis on Japanese collections. By uncovering these patterns, we reveal practical strategies for enhancing dataset documentation, licensing clarity, and cross-lingual resource sharing - ultimately guiding more effective and culturally attuned LLM development in East Asia. We conclude by discussing best practices for future dataset curation and collaboration, aiming to strengthen resource development across all three languages.

Method: Construct Transformer models with entirely frozen embedding layers using vectors derived from visual structure of Unicode glyphs (non-semantic, precomputed visual embeddings), compatible with any tokenizer including a novel Unicode-centric tokenizer.

Result: Models converged, generated coherent text, and outperformed architecturally identical models with trainable embeddings on the MMLU reasoning benchmark, suggesting “representational interference” in conventional models.

Conclusion: High-level semantics are not inherent to input embeddings but emerge from the Transformer’s compositional architecture and data scale, reframing embeddings’ role from meaning containers to structural primitives.

Abstract: Understanding the locus of semantic representation in large language models (LLMs) is crucial for interpretability and architectural innovation. The dominant paradigm posits that trainable input embeddings serve as foundational “meaning vectors.” This paper challenges that view. We construct Transformer models where the embedding layer is entirely frozen, with vectors derived not from data, but from the visual structure of Unicode glyphs. These non-semantic, precomputed visual embeddings are fixed throughout training. Our method is compatible with any tokenizer, including a novel Unicode-centric tokenizer we introduce to ensure universal text coverage. Despite the absence of trainable, semantically initialized embeddings, our models converge, generate coherent text, and, critically, outperform architecturally identical models with trainable embeddings on the MMLU reasoning benchmark. We attribute this to “representational interference” in conventional models, where the embedding layer is burdened with learning both structural and semantic features. Our results indicate that high-level semantics are not inherent to input embeddings but are an emergent property of the Transformer’s compositional architecture and data scale. This reframes the role of embeddings from meaning containers to structural primitives. We release all code and models to foster further research.

Method: Proposed LLM-based selective translation that only translates translatable parts while preserving non-translatable content and sentence structure. Compared Google Cloud Translation and Llama-3.1-405B for Hindi translation, with filtering of noisy outputs and mixing translated samples with English data.

Result: Selective translation shows promise as an effective method for improving multilingual alignment in LLMs, outperforming vanilla translation approaches.

Conclusion: LLM-based selective translation is a practical and effective technique for aligning multilingual models to low-resource languages, addressing the limitations of standard translation methods.

Abstract: Multilingual large language models (LLMs) often demonstrate a performance gap between English and non-English languages, particularly in low-resource settings. Aligning these models to low-resource languages is essential yet challenging due to limited high-quality data. While English alignment datasets are readily available, curating equivalent data in other languages is expensive and time-consuming. A common workaround is to translate existing English alignment data; however, standard translation techniques often fail to preserve critical elements such as code, mathematical expressions, and structured formats like JSON. In this work, we investigate LLM-based selective translation, a technique that selectively translates only the translatable parts of a text while preserving non-translatable content and sentence structure. We conduct a systematic study to explore key questions around this approach, including its effectiveness compared to vanilla translation, the importance of filtering noisy outputs, and the benefits of mixing translated samples with original English data during alignment. Our experiments focus on the low-resource Indic language Hindi and compare translations generated by Google Cloud Translation (GCP) and Llama-3.1-405B. The results highlight the promise of selective translation as a practical and effective method for improving multilingual alignment in LLMs.

Method: Constructs few-shot tabular prompts using in-context learning from structured biomedical data and finetunes TableGPT2 using parameter-efficient qLoRA adaptation for binary classification of AD vs cognitively normal.

Result: TAP-GPT framework outperforms more advanced general-purpose LLMs and a tabular foundation model developed for prediction tasks.

Conclusion: This is the first application of LLMs to prediction tasks using tabular biomarker data, paving the way for future LLM-driven multi-agent frameworks in biomedical informatics.

Abstract: Early and accurate diagnosis of Alzheimer’s disease (AD), a complex neurodegenerative disorder, requires analysis of heterogeneous biomarkers (e.g., neuroimaging, genetic risk factors, cognitive tests, and cerebrospinal fluid proteins) typically represented in a tabular format. With flexible few-shot reasoning, multimodal integration, and natural-language-based interpretability, large language models (LLMs) offer unprecedented opportunities for prediction with structured biomedical data. We propose a novel framework called TAP-GPT, Tabular Alzheimer’s Prediction GPT, that adapts TableGPT2, a multimodal tabular-specialized LLM originally developed for business intelligence tasks, for AD diagnosis using structured biomarker data with small sample sizes. Our approach constructs few-shot tabular prompts using in-context learning examples from structured biomedical data and finetunes TableGPT2 using the parameter-efficient qLoRA adaption for a clinical binary classification task of AD or cognitively normal (CN). The TAP-GPT framework harnesses the powerful tabular understanding ability of TableGPT2 and the encoded prior knowledge of LLMs to outperform more advanced general-purpose LLMs and a tabular foundation model (TFM) developed for prediction tasks. To our knowledge, this is the first application of LLMs to the prediction task using tabular biomarker data, paving the way for future LLM-driven multi-agent frameworks in biomedical informatics.

Method: Analyzed Llama-3.2-1B-Instruct’s token-level phonetic representations, identified a “phoneme mover head” that promotes phonetic information during rhyming tasks, and visualized its output space.

Result: Found high-level organization of phoneme representations in latent space, discovered the model learns vowel representations similar to human IPA vowel charts without direct supervision.

Conclusion: LLMs develop sophisticated internal phonetic models through language modeling alone, demonstrating emergent phonetic organization comparable to human linguistic systems.

Abstract: Large language models demonstrate proficiency on phonetic tasks, such as rhyming, without explicit phonetic or auditory grounding. In this work, we investigate how \verb|Llama-3.2-1B-Instruct| represents token-level phonetic information. Our results suggest that Llama uses a rich internal model of phonemes to complete phonetic tasks. We provide evidence for high-level organization of phoneme representations in its latent space. In doing so, we also identify a ``phoneme mover head" which promotes phonetic information during rhyming tasks. We visualize the output space of this head and find that, while notable differences exist, Llama learns a model of vowels similar to the standard IPA vowel chart for humans, despite receiving no direct supervision to do so.

Method: LAGER produces fine-grained judgment scores by aggregating cross-layer score-token logits and computing expected scores from softmax-based distributions, keeping the LLM backbone frozen without impacting inference process.

Result: LAGER achieves improvements of up to 7.5% over best baselines on Flask, HelpSteer, and BIGGen benchmarks using Spearman correlation, matches or outperforms reasoning-based methods without reasoning steps, and shows generalization in downstream applications.

Conclusion: LAGER effectively leverages complementary information across different layers to improve LLM-as-a-judge alignment with human preferences, overcoming limitations of relying solely on final layer outputs.

Abstract: The growing scale of evaluation tasks has led to the widespread adoption of automated evaluation using LLMs, a paradigm known as “LLM-as-a-judge”. However, improving its alignment with human preferences without complex prompts or fine-tuning remains challenging. Previous studies mainly optimize based on shallow outputs, overlooking rich cross-layer representations. In this work, motivated by preliminary findings that middle-to-upper layers encode semantically and task-relevant representations that are often more aligned with human judgments than the final layer, we propose LAGER, a post-hoc, plug-and-play framework for improving the alignment of LLM-as-a-Judge point-wise evaluations with human scores by leveraging internal representations. LAGER produces fine-grained judgment scores by aggregating cross-layer score-token logits and computing the expected score from a softmax-based distribution, while keeping the LLM backbone frozen and ensuring no impact on the inference process. LAGER fully leverages the complementary information across different layers, overcoming the limitations of relying solely on the final layer. We evaluate our method on the standard alignment benchmarks Flask, HelpSteer, and BIGGen using Spearman correlation, and find that LAGER achieves improvements of up to 7.5% over the best baseline across these benchmarks. Without reasoning steps, LAGER matches or outperforms reasoning-based methods. Experiments on downstream applications, such as data selection and emotional understanding, further show the generalization of LAGER.

Method: The authors evaluate common metrics (BLEU, embeddings, LLMs-as-judges) using a proposed style discrimination benchmark spanning eight writing tasks across three evaluation settings: domain discrimination, authorship attribution, and LLM-generated personalized vs non-personalized discrimination.

Result: Strong evidence shows that employing ensembles of diverse evaluation metrics consistently outperforms single-evaluator methods in assessing style-personalized text generation.

Conclusion: The paper provides guidance on how to reliably assess style-personalized text generation, emphasizing the superiority of metric ensembles over individual evaluation approaches.

Abstract: With the surge of large language models (LLMs) and their ability to produce customized output, style-personalized text generation–“write like me”–has become a rapidly growing area of interest. However, style personalization is highly specific, relative to every user, and depends strongly on the pragmatic context, which makes it uniquely challenging. Although prior research has introduced benchmarks and metrics for this area, they tend to be non-standardized and have known limitations (e.g., poor correlation with human subjects). LLMs have been found to not capture author-specific style well, it follows that the metrics themselves must be scrutinized carefully. In this work we critically examine the effectiveness of the most common metrics used in the field, such as BLEU, embeddings, and LLMs-as-judges. We evaluate these metrics using our proposed style discrimination benchmark, which spans eight diverse writing tasks across three evaluation settings: domain discrimination, authorship attribution, and LLM-generated personalized vs non-personalized discrimination. We find strong evidence that employing ensembles of diverse evaluation metrics consistently outperforms single-evaluator methods, and conclude by providing guidance on how to reliably assess style-personalized text generation.

Method: The study identifies activation outliers in dLLMs and implements state-of-the-art PTQ methods, conducting comprehensive evaluation across four dimensions: bit-width, quantization method, task category, and model type.

Result: The research provides practical insights into dLLM quantization behavior under different configurations, identifying that activation outliers with abnormally large values dominate dynamic range and pose challenges for low-bit quantization.

Conclusion: This work establishes a foundation for future research in efficient dLLM deployment and makes the code publicly available to support further development in this area.

Abstract: Recent advances in diffusion large language models (dLLMs) have introduced a promising alternative to autoregressive (AR) LLMs for natural language generation tasks, leveraging full attention and denoising-based decoding strategies. However, the deployment of these models on edge devices remains challenging due to their massive parameter scale and high resource demands. While post-training quantization (PTQ) has emerged as a widely adopted technique for compressing AR LLMs, its applicability to dLLMs remains largely unexplored. In this work, we present the first systematic study on quantizing diffusion-based language models. We begin by identifying the presence of activation outliers, characterized by abnormally large activation values that dominate the dynamic range. These outliers pose a key challenge to low-bit quantization, as they make it difficult to preserve precision for the majority of values. More importantly, we implement state-of-the-art PTQ methods and conduct a comprehensive evaluation across multiple task types and model variants. Our analysis is structured along four key dimensions: bit-width, quantization method, task category, and model type. Through this multi-perspective evaluation, we offer practical insights into the quantization behavior of dLLMs under different configurations. We hope our findings provide a foundation for future research in efficient dLLM deployment. Our code is publicly available at https://github.com/FelixMessi/QDLM.

Method: Created five Hindi LLM evaluation datasets using a methodology that combines from-scratch human annotation with a translate-and-verify process.

Result: Conducted extensive benchmarking of open-source LLMs supporting Hindi, providing detailed comparative analysis of their current capabilities.

Conclusion: The curation process serves as a replicable methodology for developing benchmarks in other low-resource languages.

Abstract: Evaluating instruction-tuned Large Language Models (LLMs) in Hindi is challenging due to a lack of high-quality benchmarks, as direct translation of English datasets fails to capture crucial linguistic and cultural nuances. To address this, we introduce a suite of five Hindi LLM evaluation datasets: IFEval-Hi, MT-Bench-Hi, GSM8K-Hi, ChatRAG-Hi, and BFCL-Hi. These were created using a methodology that combines from-scratch human annotation with a translate-and-verify process. We leverage this suite to conduct an extensive benchmarking of open-source LLMs supporting Hindi, providing a detailed comparative analysis of their current capabilities. Our curation process also serves as a replicable methodology for developing benchmarks in other low-resource languages.

Method: Created SeLeRoSa dataset with 13,873 manually annotated sentences across multiple domains, then evaluated various LLM-based models in zero-shot and fine-tuning settings, plus transformer-based baselines.

Result: The evaluation revealed current limitations of LLMs and transformer models in sentence-level satire detection task, showing room for improvement.

Conclusion: The study identifies gaps in current models’ capabilities for satire detection and opens new research directions for improving sentence-level satire identification in news content.

Abstract: Satire, irony, and sarcasm are techniques typically used to express humor and critique, rather than deceive; however, they can occasionally be mistaken for factual reporting, akin to fake news. These techniques can be applied at a more granular level, allowing satirical information to be incorporated into news articles. In this paper, we introduce the first sentence-level dataset for Romanian satire detection for news articles, called SeLeRoSa. The dataset comprises 13,873 manually annotated sentences spanning various domains, including social issues, IT, science, and movies. With the rise and recent progress of large language models (LLMs) in the natural language processing literature, LLMs have demonstrated enhanced capabilities to tackle various tasks in zero-shot settings. We evaluate multiple baseline models based on LLMs in both zero-shot and fine-tuning settings, as well as baseline transformer-based models. Our findings reveal the current limitations of these models in the sentence-level satire detection task, paving the way for new research directions.

Method: Systematic assessment of LLMs across eight diverse card games using supervised fine-tuning on high-quality gameplay data, evaluating performance, multi-game proficiency, and impact on general capabilities.

Result: LLMs can approach strong game AI performance through fine-tuning, achieve proficiency in multiple card games simultaneously (with performance augmentation for similar games and conflicts for dissimilar ones), and experience mitigated decline in general capabilities when supplemented with instruction data.

Conclusion: LLMs demonstrate strong learning ability and versatility in mastering complex card games, showing potential for game AI applications while highlighting the importance of balancing specialized training with general capability preservation.

Abstract: Complex games have long been an important benchmark for testing the progress of artificial intelligence algorithms. AlphaGo, AlphaZero, and MuZero have defeated top human players in Go and Chess, garnering widespread societal attention towards artificial intelligence. Concurrently, large language models (LLMs) have exhibited remarkable capabilities across various tasks, raising the question of whether LLMs can achieve similar success in complex games. In this paper, we explore the potential of LLMs in mastering complex card games. We systematically assess the learning capabilities of LLMs across eight diverse card games, evaluating the impact of fine-tuning on high-quality gameplay data, and examining the models’ ability to retain general capabilities while mastering these games. Our findings indicate that: (1) LLMs can approach the performance of strong game AIs through supervised fine-tuning on high-quality data, (2) LLMs can achieve a certain level of proficiency in multiple complex card games simultaneously, with performance augmentation for games with similar rules and conflicts for dissimilar ones, and (3) LLMs experience a decline in general capabilities when mastering complex games, but this decline can be mitigated by integrating a certain amount of general instruction data. The evaluation results demonstrate strong learning ability and versatility of LLMs. The code is available at https://github.com/THUDM/LLM4CardGame

Method: ReSum converts growing interaction histories into compact reasoning states through periodic summarization. ReSum-GRPO integrates GRPO with segmented trajectory training and advantage broadcasting to train agents for summary-conditioned reasoning.

Result: ReSum delivers 4.5% average absolute improvement over ReAct, with additional 8.2% gains after ReSum-GRPO training. WebResummer-30B achieves 33.3% Pass@1 on BrowseComp-zh and 18.3% on BrowseComp-en with only 1K training samples.

Conclusion: ReSum enables indefinite web exploration by bypassing context constraints through summarization, significantly outperforming existing paradigms and achieving state-of-the-art performance with minimal training data.

Abstract: Large Language Model (LLM)-based web agents demonstrate strong performance on knowledge-intensive tasks but are hindered by context window limitations in paradigms like ReAct. Complex queries involving multiple entities, intertwined relationships, and high uncertainty demand extensive search cycles that rapidly exhaust context budgets before reaching solutions. To overcome this challenge, we introduce ReSum, a novel paradigm that enables indefinite exploration through periodic context summarization. ReSum converts growing interaction histories into compact reasoning states, maintaining awareness of prior discoveries while bypassing context constraints. For paradigm adaptation, we propose ReSum-GRPO, integrating GRPO with segmented trajectory training and advantage broadcasting to familiarize agents with summary-conditioned reasoning. Extensive experiments on web agents across three benchmarks demonstrate that ReSum delivers an average absolute improvement of 4.5% over ReAct, with further gains of 8.2% following ReSum-GRPO training. Notably, with only 1K training samples, our WebResummer-30B (a ReSum-GRPO-trained version of WebSailor-30B) achieves 33.3% Pass@1 on BrowseComp-zh and 18.3% on BrowseComp-en, surpassing most open-source web agents.

Method: Formulated as a game with incomplete information, M-Spoiler simulates agent interactions and introduces a stubborn agent that helps optimize adversarial samples by simulating potential stubborn responses from agents in the target system.

Result: Extensive experiments confirm the risks posed by individual agent knowledge in multi-agent systems and demonstrate M-Spoiler’s effectiveness in misleading collaborative decision-making across various tasks.

Conclusion: The framework remains more potent than baselines even with defense mechanisms, highlighting significant security vulnerabilities in multi-agent LLM systems and the need for further defensive research.

Abstract: Individual Large Language Models (LLMs) have demonstrated significant capabilities across various domains, such as healthcare and law. Recent studies also show that coordinated multi-agent systems exhibit enhanced decision-making and reasoning abilities through collaboration. However, due to the vulnerabilities of individual LLMs and the difficulty of accessing all agents in a multi-agent system, a key question arises: If attackers only know one agent, could they still generate adversarial samples capable of misleading the collective decision? To explore this question, we formulate it as a game with incomplete information, where attackers know only one target agent and lack knowledge of the other agents in the system. With this formulation, we propose M-Spoiler, a framework that simulates agent interactions within a multi-agent system to generate adversarial samples. These samples are then used to manipulate the target agent in the target system, misleading the system’s collaborative decision-making process. More specifically, M-Spoiler introduces a stubborn agent that actively aids in optimizing adversarial samples by simulating potential stubborn responses from agents in the target system. This enhances the effectiveness of the generated adversarial samples in misleading the system. Through extensive experiments across various tasks, our findings confirm the risks posed by the knowledge of an individual agent in multi-agent systems and demonstrate the effectiveness of our framework. We also explore several defense mechanisms, showing that our proposed attack framework remains more potent than baselines, underscoring the need for further research into defensive strategies.

Method: Content selection and planning framework that segments sentences into semantic units, identifies ambiguous units, extracts relevant context based on discourse relations, and generates content plans to rewrite sentences with proper context.

Result: The approach outperforms existing methods, producing sentences with better semantic integrity and discourse coherence in decontextualization tasks.

Conclusion: The proposed zero-shot decontextualisation framework effectively addresses the problem of ambiguous extracted sentences by systematically incorporating necessary context through content planning.

Abstract: Extracting individual sentences from a document as evidence or reasoning steps is commonly done in many NLP tasks. However, extracted sentences often lack context necessary to make them understood, e.g., coreference and background information. To this end, we propose a content selection and planning framework for zero-shot decontextualisation, which determines what content should be mentioned and in what order for a sentence to be understood out of context. Specifically, given a potentially ambiguous sentence and its context, we first segment it into basic semantically-independent units. We then identify potentially ambiguous units from the given sentence, and extract relevant units from the context based on their discourse relations. Finally, we generate a content plan to rewrite the sentence by enriching each ambiguous unit with its relevant units. Experimental results demonstrate that our approach is competitive for sentence decontextualisation, producing sentences that exhibit better semantic integrity and discourse coherence, outperforming existing methods.

Method: Extends evidence lower bound (ELBO) to multi-trace objective, proposes forward-KL formulation for stable training, and shows how rejection sampling finetuning and binary-reward RL can be interpreted as local forward-KL objectives.

Result: Empirically validated on Qwen 2.5 and Qwen 3 model families across various reasoning tasks, revealing previously unnoticed bias toward easier questions in existing methods.

Conclusion: The framework provides stable objectives for improving reasoning ability and unifies variational inference with RL-style methods under a principled probabilistic perspective.

Abstract: We introduce a variational reasoning framework for language models that treats thinking traces as latent variables and optimizes them through variational inference. Starting from the evidence lower bound (ELBO), we extend it to a multi-trace objective for tighter bounds and propose a forward-KL formulation that stabilizes the training of the variational posterior. We further show that rejection sampling finetuning and binary-reward RL, including GRPO, can be interpreted as local forward-KL objectives, where an implicit weighting by model accuracy naturally arises from the derivation and reveals a previously unnoticed bias toward easier questions. We empirically validate our method on the Qwen 2.5 and Qwen 3 model families across a wide range of reasoning tasks. Overall, our work provides a principled probabilistic perspective that unifies variational inference with RL-style methods and yields stable objectives for improving the reasoning ability of language models. Our code is available at https://github.com/sail-sg/variational-reasoning.

Method: The paper presents BTC-SAM, which uses Large Language Models for controllable generation of test sentences. This approach provides linguistic variation and diversity in test cases with minimal specification.

Result: Experiments show that LLM-based generation provides better linguistic variation and diversity in test sentences compared to base prompting methods, offering improved test coverage even for previously unseen biases.

Conclusion: BTC-SAM enables efficient and comprehensive bias testing in Sentiment Analysis models by leveraging LLMs for high-quality test case generation, reducing the need for expensive manual annotation.

Abstract: Sentiment Analysis (SA) models harbor inherent social biases that can be harmful in real-world applications. These biases are identified by examining the output of SA models for sentences that only vary in the identity groups of the subjects. Constructing natural, linguistically rich, relevant, and diverse sets of sentences that provide sufficient coverage over the domain is expensive, especially when addressing a wide range of biases: it requires domain experts and/or crowd-sourcing. In this paper, we present a novel bias testing framework, BTC-SAM, which generates high-quality test cases for bias testing in SA models with minimal specification using Large Language Models (LLMs) for the controllable generation of test sentences. Our experiments show that relying on LLMs can provide high linguistic variation and diversity in the test sentences, thereby offering better test coverage compared to base prompting methods even for previously unseen biases.

Method: Constructed three evaluation datasets to test LLMs’ ability to forget: (1) direct factual knowledge, (2) semantic shifts, and (3) causally related knowledge using prompting-based unlearning techniques.

Result: Prompt-based simulated knowledge cutoffs are effective when directly queried with information after the cutoff date, but struggle to induce forgetting when the forgotten content is causally related to the query rather than directly asked.

Conclusion: Current prompting methods for simulating knowledge cutoffs have limitations, highlighting the need for more rigorous evaluation settings in temporal prediction tasks using LLMs.

Abstract: Large Language Models (LLMs) are widely used for temporal prediction, but their reliance on pretraining data raises contamination concerns, as accurate predictions on pre-cutoff test data may reflect memorization rather than reasoning, leading to an overestimation of their generalization capability. With the recent emergence of prompting-based unlearning techniques, a natural question arises: Can LLMs be prompted to simulate an earlier knowledge cutoff? In this work, we investigate the capability of prompting to simulate earlier knowledge cutoff in LLMs. We construct three evaluation datasets to assess the extent to which LLMs can forget (1) direct factual knowledge, (2) semantic shifts, and (3) causally related knowledge. Results demonstrate that while prompt-based simulated knowledge cutoffs show effectiveness when directly queried with the information after that date, they struggle to induce forgetting when the forgotten content is not directly asked but causally related to the query. These findings highlight the need for more rigorous evaluation settings when applying LLMs for temporal prediction tasks. The full dataset and evaluation code are available at https://github.com/gxx27/time_unlearn.

Method: Created a data collection pipeline that gathers documents from various Sri Lankan sources, formats them into machine-readable datasets, and maintains daily updates with mirroring on GitHub and Hugging Face.

Result: Successfully compiled 24 datasets containing 229,858 documents (57.1 GB) covering multiple domains and languages, with ongoing daily updates.

Conclusion: The collection provides valuable resources for multilingual NLP research and various analytical studies on Sri Lankan content, with attention to licensing and ethical considerations.

Abstract: We present a collection of open, machine-readable document datasets covering parliamentary proceedings, legal judgments, government publications, news, and tourism statistics from Sri Lanka. The collection currently comprises of 229,858 documents (57.1 GB) across 24 datasets in Sinhala, Tamil, and English. The datasets are updated daily and mirrored on GitHub and Hugging Face. These resources aim to support research in computational linguistics, legal analytics, socio-political studies, and multilingual natural language processing. We describe the data sources, collection pipeline, formats, and potential use cases, while discussing licensing and ethical considerations. This manuscript is at version v2025-10-15-1111.

Method: Token Probability Deviation (TBD) quantifies how far generated tokens’ probabilities deviate from a high reference probability, leveraging that distilled models produce near-deterministic tokens for seen questions.

Result: Achieves AUC of 0.918 and TPR@1% FPR of 0.470 on S1 dataset, showing competitive detection performance.

Conclusion: TBD is an effective method for detecting distillation data contamination by analyzing token probability patterns in model outputs.

Abstract: Reasoning distillation has emerged as an efficient and powerful paradigm for enhancing the reasoning capabilities of large language models. However, reasoning distillation may inadvertently cause benchmark contamination, where evaluation data included in distillation datasets can inflate performance metrics of distilled models. In this work, we formally define the task of distillation data detection, which is uniquely challenging due to the partial availability of distillation data. Then, we propose a novel and effective method Token Probability Deviation (TBD), which leverages the probability patterns of the generated output tokens. Our method is motivated by the analysis that distilled models tend to generate near-deterministic tokens for seen questions, while producing more low-probability tokens for unseen questions. Our key idea behind TBD is to quantify how far the generated tokens' probabilities deviate from a high reference probability. In effect, our method achieves competitive detection performance by producing lower scores for seen questions than for unseen questions. Extensive experiments demonstrate the effectiveness of our method, achieving an AUC of 0.918 and a TPR@1% FPR of 0.470 on the S1 dataset.

Method: ARM2 uses a reinforcement learning framework augmented with length-aware optimization to adaptively balance reasoning performance and efficiency. It integrates vision understanding for multimodal applications and incorporates executable code into reasoning to reduce token costs while preserving task performance.

Result: ARM2 achieves performance on par with traditional reasoning models trained with GRPO, while reducing token usage by over 70% on average. Extensive analyses validate the effectiveness of ARM2 and the soundness of its design.

Conclusion: ARM2 provides a unified framework for adaptive reasoning that effectively addresses the over-thinking problem in Large Reasoning Models, achieving significant efficiency gains while maintaining performance across multiple formats including multimodal and code-integrated reasoning.

Abstract: Large Reasoning Models (LRMs) often suffer from the ``over-thinking’' problem, generating unnecessarily long reasoning on simple tasks. Some strategies have been proposed to mitigate this issue, such as length penalties or routing mechanisms, but they are typically heuristic and task-specific, lacking a general framework for adaptive reasoning. In this paper, we present ARM2, a unified model that adaptively balances reasoning performance and efficiency across multiple formats through a reinforcement learning framework augmented with length-aware optimization. Beyond conventional natural language inference, ARM2 integrates vision understanding, extending its applicability to multimodal. Moreover, ARM2 integrates executable code into reasoning, enabling substantial reductions in token cost while preserving task performance compared to long CoT. Experiments demonstrate that ARM2 achieves performance on par with traditional reasoning models trained with GRPO, while reducing token usage by over 70% on average. We further conduct extensive analyses to validate the effectiveness of ARM2 and the soundness of its design.

Method: An iterative framework using LLMs with specialized prompts to revise MWPs from multiple perspectives and cognitive levels, generating distracting conditions that don’t alter original solutions.

Result: The framework efficiently generates high-quality MWPs with meaningful distracting conditions while eliminating the need for manual solution rewriting, substantially reducing generation effort.

Conclusion: The proposed framework provides an effective solution for creating challenging MWP datasets with distracting conditions, enabling better evaluation of LLM mathematical reasoning capabilities.

Abstract: Mathematical reasoning serves as a crucial testbed for evaluating the intelligence of large language models (LLMs), and math word problems (MWPs) represent one of the most widely used formats. Most existing MWP datasets contain only the necessary information, while problems with distracting or excessive conditions are often overlooked. Prior studies have shown that popular LLMs experience a dramatic performance drop when such distracting conditions are introduced. However, available datasets of MWPs with distracting conditions remain limited, and most exhibit low difficulty and out-of-context expressions. These shortcomings make the distracting conditions easy to detect and disregard, thereby reducing the credibility of benchmarking on these datasets. Moreover, when distracting conditions are added, the reasoning process and answers may change, requiring intensive manual effort to check and rewrite solutions. To address these issues, we design an iterative framework that leverages LLMs to generate distracting conditions automatically. We develop a set of prompts to revise MWPs from multiple perspectives and cognitive levels, encouraging the creation of meaningful distracting conditions as well as suggestions for further refinement. A key advantage of our framework is the preservation of shared solutions between the original and revised problems: the LLMs are explicitly guided to generate distractions that do not alter the original solution, thus eliminating the need to produce new answers. This framework is efficient and easy to deploy, substantially reducing the effort required to generate MWPs with distracting conditions while maintaining high data quality.

Method: PRAgent framework with three stages: multimodal content extraction, collaborative synthesis for polished outputs, and platform-specific adaptation for optimal reach and engagement.

Result: PRAgent achieved 604% increase in total watch time, 438% rise in likes, and at least 2.9x boost in overall engagement compared to direct LLM pipelines on the PRBench benchmark.

Conclusion: AutoPR is a tractable research problem with PRAgent providing a scalable roadmap for automated scholarly communication, where platform modeling and targeted promotion drive the most significant engagement gains.

Abstract: As the volume of peer-reviewed research surges, scholars increasingly rely on social platforms for discovery, while authors invest considerable effort in promoting their work to ensure visibility and citations. To streamline this process and reduce the reliance on human effort, we introduce Automatic Promotion (AutoPR), a novel task that transforms research papers into accurate, engaging, and timely public content. To enable rigorous evaluation, we release PRBench, a multimodal benchmark that links 512 peer-reviewed articles to high-quality promotional posts, assessing systems along three axes: Fidelity (accuracy and tone), Engagement (audience targeting and appeal), and Alignment (timing and channel optimization). We also introduce PRAgent, a multi-agent framework that automates AutoPR in three stages: content extraction with multimodal preparation, collaborative synthesis for polished outputs, and platform-specific adaptation to optimize norms, tone, and tagging for maximum reach. When compared to direct LLM pipelines on PRBench, PRAgent demonstrates substantial improvements, including a 604% increase in total watch time, a 438% rise in likes, and at least a 2.9x boost in overall engagement. Ablation studies show that platform modeling and targeted promotion contribute the most to these gains. Our results position AutoPR as a tractable, measurable research problem and provide a roadmap for scalable, impactful automated scholarly communication.

Method: Two-stage framework: 1) Semantic and cluster-based filtering guided by EIRE (Entity-intent-relation extractor) to score relevance and build clean database; 2) EIRE-guided conflict-aware filtering that analyzes semantic consistency between query, answers, and knowledge before final generation.

Result: Significant improvements in generation robustness and output trustworthiness. Extensive experiments show SeCon-RAG markedly outperforms state-of-the-art defense methods across various LLMs and datasets.

Conclusion: SeCon-RAG effectively preserves useful knowledge while mitigating conflict contamination, achieving trustworthy RAG through semantic filtering and conflict-free generation.

Abstract: Retrieval-augmented generation (RAG) systems enhance large language models (LLMs) with external knowledge but are vulnerable to corpus poisoning and contamination attacks, which can compromise output integrity. Existing defenses often apply aggressive filtering, leading to unnecessary loss of valuable information and reduced reliability in generation. To address this problem, we propose a two-stage semantic filtering and conflict-free framework for trustworthy RAG. In the first stage, we perform a joint filter with semantic and cluster-based filtering which is guided by the Entity-intent-relation extractor (EIRE). EIRE extracts entities, latent objectives, and entity relations from both the user query and filtered documents, scores their semantic relevance, and selectively adds valuable documents into the clean retrieval database. In the second stage, we proposed an EIRE-guided conflict-aware filtering module, which analyzes semantic consistency between the query, candidate answers, and retrieved knowledge before final answer generation, filtering out internal and external contradictions that could mislead the model. Through this two-stage process, SeCon-RAG effectively preserves useful knowledge while mitigating conflict contamination, achieving significant improvements in both generation robustness and output trustworthiness. Extensive experiments across various LLMs and datasets demonstrate that the proposed SeCon-RAG markedly outperforms state-of-the-art defense methods.

Method: Two-stage framework: 1) Latent Refinement maintains masked positions as distributional mixtures of predicted tokens and mask embeddings, 2) Predictive Feedback Loop progressively finalizes confident tokens while retaining uncertain ones for iterative refinement using KL-divergence for convergence.

Result: Significant improvements in coding (HumanEval +6.3, MBPP +2.6) and reasoning (GSM8K +2.9, MATH500 +3.8) benchmarks with speedups up to 10.6x compared to baseline methods.

Conclusion: LRD provides a strong and versatile alternative for parallel sequence generation, improving accuracy while delivering substantial speed improvements over existing approaches.

Abstract: Autoregressive (AR) models remain the standard for natural language generation but still suffer from high latency due to strictly sequential decoding. Recent diffusion-inspired approaches, such as LlaDA and Dream, mitigate this by generating in parallel, yet they suffer from two core limitations: information loss, as predictive distributions for non-finalized tokens are discarded at each step, and premature commitment, where local decisions are made without sufficient global coordination. We introduce Latent Refinement Decoding (LRD), a two-stage framework with Latent Refinement and a Predictive Feedback Loop. The first stage maintains masked positions as distributional mixtures of predicted tokens and the mask embedding, allowing the model to establish more globally consistent beliefs. The second stage progressively finalizes confident tokens while retaining uncertain ones for iterative feedback. KL-divergence dynamics provide a principled and reliable criterion for convergence and early stopping. Experiments across coding (HumanEval +6.3, MBPP +2.6) and reasoning (GSM8K +2.9, MATH500 +3.8) show that LRD improves accuracy while delivering speedups of up to 10.6x, making it a strong and versatile alternative for parallel sequence generation.

Method: Uses large language models with a reward system and iterative direct preference optimization training to rewrite prompts before feeding to T2I models.

Result: Consistently improves image-text alignment, visual quality, and aesthetics across diverse T2I models and benchmarks, with strong transferability between different backbones.

Conclusion: Prompt rewriting is an effective, scalable, and practical model-agnostic strategy for improving text-to-image systems.

Abstract: Recent advances in text-to-image (T2I) generation have achieved impressive results, yet existing models often struggle with simple or underspecified prompts, leading to suboptimal image-text alignment, aesthetics, and quality. We propose a prompt rewriting framework that leverages large language models (LLMs) to refine user inputs before feeding them into T2I backbones. Our approach introduces a carefully designed reward system and an iterative direct preference optimization (DPO) training pipeline, enabling the rewriter to enhance prompts without requiring supervised fine-tuning data. We evaluate our method across diverse T2I models and benchmarks. Results show that our prompt rewriter consistently improves image-text alignment, visual quality, and aesthetics, outperforming strong baselines. Furthermore, we demonstrate strong transferability by showing that a prompt rewriter trained on one T2I backbone generalizes effectively to others without needing to be retrained. We also systematically study scalability, evaluating how performance gains scale with the capacity of the large LLM used as the rewriter. These findings highlight that prompt rewriting is an effective, scalable, and practical model-agnostic strategy for improving T2I systems. We plan to release the code and trained prompt rewriters soon.

Method: Created ACADATA dataset with training and benchmark subsets, then fine-tuned LLMs on ACAD-TRAIN and evaluated against various translation systems on ACAD-BENCH.

Result: Fine-tuning on ACAD-TRAIN improved academic translation by +6.1 and +12.4 d-BLEU points for 7B and 2B models respectively, and improved long-context translation by up to 24.9%. The fine-tuned model outperformed proprietary and open-weight models.

Conclusion: ACADATA provides a valuable resource that significantly advances academic domain and long-context translation research, with released datasets and models benefiting the community.

Abstract: We present ACADATA, a high-quality parallel dataset for academic translation, that consists of two subsets: ACAD-TRAIN, which contains approximately 1.5 million author-generated paragraph pairs across 96 language directions and ACAD-BENCH, a curated evaluation set of almost 6,000 translations covering 12 directions. To validate its utility, we fine-tune two Large Language Models (LLMs) on ACAD-TRAIN and benchmark them on ACAD-BENCH against specialized machine-translation systems, general-purpose, open-weight LLMs, and several large-scale proprietary models. Experimental results demonstrate that fine-tuning on ACAD-TRAIN leads to improvements in academic translation quality by +6.1 and +12.4 d-BLEU points on average for 7B and 2B models respectively, while also improving long-context translation in a general domain by up to 24.9% when translating out of English. The fine-tuned top-performing model surpasses the best propietary and open-weight models on academic translation domain. By releasing ACAD-TRAIN, ACAD-BENCH and the fine-tuned models, we provide the community with a valuable resource to advance research in academic domain and long-context translation.

Method: Extends 3DGUT with LiDAR support using automated tiling for spinning LiDAR models and ray-based culling. Introduces factorized 3D Gaussian representation and anchoring strategy to address cross-sensor inconsistencies.

Result: Renders 10-20x faster than ray tracing approaches and 1.5-10x faster than prior rasterization-based methods. Reduces mean camera and depth error by up to 40% compared to existing methods. Matches or exceeds state-of-the-art fidelity on autonomous driving datasets.

Conclusion: SimULi provides a practical solution for high-fidelity, real-time multi-sensor simulation needed for rigorous autonomous vehicle testing, overcoming key limitations of existing neural rendering approaches.

Abstract: Rigorous testing of autonomous robots, such as self-driving vehicles, is essential to ensure their safety in real-world deployments. This requires building high-fidelity simulators to test scenarios beyond those that can be safely or exhaustively collected in the real-world. Existing neural rendering methods based on NeRF and 3DGS hold promise but suffer from low rendering speeds or can only render pinhole camera models, hindering their suitability to applications that commonly require high-distortion lenses and LiDAR data. Multi-sensor simulation poses additional challenges as existing methods handle cross-sensor inconsistencies by favoring the quality of one modality at the expense of others. To overcome these limitations, we propose SimULi, the first method capable of rendering arbitrary camera models and LiDAR data in real-time. Our method extends 3DGUT, which natively supports complex camera models, with LiDAR support, via an automated tiling strategy for arbitrary spinning LiDAR models and ray-based culling. To address cross-sensor inconsistencies, we design a factorized 3D Gaussian representation and anchoring strategy that reduces mean camera and depth error by up to 40% compared to existing methods. SimULi renders 10-20x faster than ray tracing approaches and 1.5-10x faster than prior rasterization-based work (and handles a wider range of camera models). When evaluated on two widely benchmarked autonomous driving datasets, SimULi matches or exceeds the fidelity of existing state-of-the-art methods across numerous camera and LiDAR metrics.

Method: Proposed Salient Epistemic Disentanglement (SED) that injects expert-curated bipartite graph into the model to decouple view-disease associations and uses reinforcement learning for clinically faithful preference selection.

Result: FetalMind outperforms open- and closed-source baselines with +14% average gains, +61.2% higher accuracy on critical conditions, and remains efficient, stable, and scalable across all gestational stages.

Conclusion: FetalMind successfully bridges the gap in fetal ultrasound AI by addressing domain-specific challenges through clinical workflow-guided design and large-scale dataset curation (FetalSigma-1M), demonstrating superior performance and clinical alignment.

Abstract: Recent medical vision-language models have shown promise on tasks such as VQA, report generation, and anomaly detection. However, most are adapted to structured adult imaging and underperform in fetal ultrasound, which poses challenges of multi-view image reasoning, numerous diseases, and image diversity. To bridge this gap, we introduce FetalMind, a medical AI system tailored to fetal ultrasound for both report generation and diagnosis. Guided by clinical workflow, we propose Salient Epistemic Disentanglement (SED), which injects an expert-curated bipartite graph into the model to decouple view-disease associations and to steer preference selection along clinically faithful steps via reinforcement learning. This design mitigates variability across diseases and heterogeneity across views, reducing learning bottlenecks while aligning the model’s inference with obstetric practice. To train FetalMind at scale, we curate FetalSigma-1M dataset, the first large-scale fetal ultrasound report corpus, comprising 20K reports from twelve medical centers, addressing the scarcity of domain data. Extensive experiments show that FetalMind outperforms open- and closed-source baselines across all gestational stages, achieving +14% average gains and +61.2% higher accuracy on critical conditions while remaining efficient, stable, and scalable. Project Page: https://hexiao0275.github.io/FetalMind.

Method: Proposes SurgFUTR with teacher-student architecture that compresses video clips into state representations via Sinkhorn-Knopp clustering. Teacher learns from current and future clips while student predicts future states from current videos alone, guided by Action Dynamics module.

Result: Experiments across four datasets and three procedures show consistent improvements. Cross-procedure transfer validates the method’s generalizability across different surgical contexts.

Conclusion: Reframing surgical future prediction as state-change learning rather than raw observation forecasting enables better generalization and performance across multiple prediction tasks and surgical procedures.

Abstract: Surgical future prediction, driven by real-time AI analysis of surgical video, is critical for operating room safety and efficiency. It provides actionable insights into upcoming events, their timing, and risks-enabling better resource allocation, timely instrument readiness, and early warnings for complications (e.g., bleeding, bile duct injury). Despite this need, current surgical AI research focuses on understanding what is happening rather than predicting future events. Existing methods target specific tasks in isolation, lacking unified approaches that span both short-term (action triplets, events) and long-term horizons (remaining surgery duration, phase transitions). These methods rely on coarse-grained supervision while fine-grained surgical action triplets and steps remain underexplored. Furthermore, methods based only on future feature prediction struggle to generalize across different surgical contexts and procedures. We address these limits by reframing surgical future prediction as state-change learning. Rather than forecasting raw observations, our approach classifies state transitions between current and future timesteps. We introduce SurgFUTR, implementing this through a teacher-student architecture. Video clips are compressed into state representations via Sinkhorn-Knopp clustering; the teacher network learns from both current and future clips, while the student network predicts future states from current videos alone, guided by our Action Dynamics (ActDyn) module. We establish SFPBench with five prediction tasks spanning short-term (triplets, events) and long-term (remaining surgery duration, phase and step transitions) horizons. Experiments across four datasets and three procedures show consistent improvements. Cross-procedure transfer validates generalizability.

Method: Two-step approach: 1) LLM-based prompt template to enhance text polysemy description and increase text entropy; 2) Hypergraph adapter to construct multilateral text-image connections and correct matching errors while reducing semantic noise.

Result: Achieved 16.8% text-to-image and 40.1% image-to-text retrieval gains on Flickr30K and MS-COCO benchmarks, establishing new state-of-the-art performance.

Conclusion: The proposed OS-HGAdapter effectively addresses cross-modal alignment imbalance using LLM semantic knowledge and hypergraph adapters, demonstrating significant performance improvements in semantic alignment tasks.

Abstract: Text-image alignment constitutes a foundational challenge in multimedia content understanding, where effective modeling of cross-modal semantic correspondences critically enhances retrieval system performance through joint embedding space optimization. Given the inherent difference in information entropy between texts and images, conventional approaches often show an imbalance in the mutual retrieval of these two modalities. To address this particular challenge, we propose to use the open semantic knowledge of Large Language Model (LLM) to fill for the entropy gap and reproduce the alignment ability of humans in these tasks. Our entropy-enhancing alignment is achieved through a two-step process: 1) a new prompt template that does not rely on explicit knowledge in the task domain is designed to use LLM to enhance the polysemy description of the text modality. By analogy, the information entropy of the text modality relative to the visual modality is increased; 2) A hypergraph adapter is used to construct multilateral connections between the text and image modalities, which can correct the positive and negative matching errors for synonymous semantics in the same fixed embedding space, whilst reducing the noise caused by open semantic entropy by mapping the reduced dimensions back to the original dimensions. Comprehensive evaluations on the Flickr30K and MS-COCO benchmarks validate the superiority of our Open Semantic Hypergraph Adapter (OS-HGAdapter), showcasing 16.8% (text-to-image) and 40.1% (image-to-text) cross-modal retrieval gains over existing methods while establishing new state-of-the-art performance in semantic alignment tasks.

Method: Target-Aware Metric Learning with Prioritized Sampling (TMPS) - a metric learning framework that effectively uses limited labeled samples from target domains to improve model robustness.

Result: TMPS achieved average macro F1 score improvements of 7.3 and 3.6 points over combined training and fine-tuning methods, and 18.7 and 17.1 point improvements over baseline and conventional metric learning.

Conclusion: TMPS effectively addresses domain adaptation challenges in plant disease diagnosis by leveraging limited target domain samples through metric learning and prioritized sampling, significantly improving diagnostic robustness.

Abstract: Various deep learning-based systems have been proposed for accurate and convenient plant disease diagnosis, achieving impressive performance. However, recent studies show that these systems often fail to maintain diagnostic accuracy on images captured under different conditions from the training environment – an essential criterion for model robustness. Many deep learning methods have shown high accuracy in plant disease diagnosis. However, they often struggle to generalize to images taken in conditions that differ from the training setting. This drop in performance stems from the subtle variability of disease symptoms and domain gaps – differences in image context and environment. The root cause is the limited diversity of training data relative to task complexity, making even advanced models vulnerable in unseen domains. To tackle this challenge, we propose a simple yet highly adaptable learning framework called Target-Aware Metric Learning with Prioritized Sampling (TMPS), grounded in metric learning. TMPS operates under the assumption of access to a limited number of labeled samples from the target (deployment) domain and leverages these samples effectively to improve diagnostic robustness. We assess TMPS on a large-scale automated plant disease diagnostic task using a dataset comprising 223,073 leaf images sourced from 23 agricultural fields, spanning 21 diseases and healthy instances across three crop species. By incorporating just 10 target domain samples per disease into training, TMPS surpasses models trained using the same combined source and target samples, and those fine-tuned with these target samples after pre-training on source data. It achieves average macro F1 score improvements of 7.3 and 3.6 points, respectively, and a remarkable 18.7 and 17.1 point improvement over the baseline and conventional metric learning.

Method: ViZer actively aligns vision and language representation features during training, enabling existing VLMs to generate improved captions without requiring text labels or full retraining.

Result: ViZer shows consistent qualitative improvements on SmolVLM-Base and Qwen2-VL, producing more grounded and descriptive captions than baseline models, though automated metrics like CIDEr and BERTScore may penalize the additional details.

Conclusion: ViZer provides a practical framework for zero-label learning in image captioning and serves as a starting point for broader zero-label adaptation in vision-language tasks.

Abstract: Vision-language models (VLMs) achieve remarkable performance through large-scale image-text pretraining. However, their reliance on labeled image datasets limits scalability and leaves vast amounts of unlabeled image data underutilized. To address this, we propose Unified Vision-Language Alignment for Zero-Label Enhancement (ViZer), an enhancement training framework that enables zero-label learning in image captioning, providing a practical starting point for broader zero-label adaptation in vision-language tasks. Unlike prior approaches that rely on human or synthetically annotated datasets, ViZer actively aligns vision and language representation features during training, enabling existing VLMs to generate improved captions without requiring text labels or full retraining. We demonstrate ViZer’s advantage in qualitative evaluation, as automated caption metrics such as CIDEr and BERTScore often penalize details that are absent in reference captions. Applying ViZer on SmolVLM-Base and Qwen2-VL, we observe consistent qualitative improvements, producing captions that are more grounded and descriptive than their baseline.

Method: Uses ZeResFDG module with three components: frequency-decoupled guidance, energy rescaling, and zero-projection. Also employs spectral EMA with hysteresis switching and QSilk Micrograin Stabilizer for inference-time stabilization.

Result: Improves sharpness, prompt adherence, and artifact control across SD/SDXL samplers. Enhances robustness and yields natural high-frequency micro-texture at high resolutions with negligible overhead.

Conclusion: CADE 2.5 provides an effective guidance stack that significantly improves diffusion model performance without retraining, offering better detail enhancement and stability during sampling.

Abstract: We introduce CADE 2.5 (Comfy Adaptive Detail Enhancer), a sampler-level guidance stack for SD/SDXL latent diffusion models. The central module, ZeResFDG, unifies (i) frequency-decoupled guidance that reweights low- and high-frequency components of the guidance signal, (ii) energy rescaling that matches the per-sample magnitude of the guided prediction to the positive branch, and (iii) zero-projection that removes the component parallel to the unconditional direction. A lightweight spectral EMA with hysteresis switches between a conservative and a detail-seeking mode as structure crystallizes during sampling. Across SD/SDXL samplers, ZeResFDG improves sharpness, prompt adherence, and artifact control at moderate guidance scales without any retraining. In addition, we employ a training-free inference-time stabilizer, QSilk Micrograin Stabilizer (quantile clamp + depth/edge-gated micro-detail injection), which improves robustness and yields natural high-frequency micro-texture at high resolutions with negligible overhead. For completeness we note that the same rule is compatible with alternative parameterizations (e.g., velocity), which we briefly discuss in the Appendix; however, this paper focuses on SD/SDXL latent diffusion models.

Method: SCOPE maintains a shared encoder and a pool of routed encoders. A lightweight router uses cross-attention between text prompts and shared visual features to select the optimal encoder. Training uses dual entropy regularization with auxiliary losses to balance dataset-level load distribution with instance-level routing confidence.

Result: SCOPE with one shared plus one routed encoder outperforms models using all four extra encoders simultaneously, while reducing compute by 24-49%. This demonstrates that intelligent encoder selection beats brute-force aggregation.

Conclusion: SCOPE challenges the prevailing paradigm in multi-encoder VLMs by showing that dynamic encoder selection through instance-level routing is more effective than using all encoders simultaneously, achieving better performance with significantly reduced computation.

Abstract: Vision-language models (VLMs) benefit from multiple vision encoders, but naively stacking them yields diminishing returns while multiplying inference costs. We propose SCOPE, a Mixture-of-Encoders (MoEnc) framework that dynamically selects one specialized encoder per image-text pair via instance-level routing, unlike token-level routing in traditional MoE. SCOPE maintains a shared encoder and a pool of routed encoders. A lightweight router uses cross-attention between text prompts and shared visual features to select the optimal encoder from the routed encoders. To train this router, we introduce dual entropy regularization with auxiliary losses to balance dataset-level load distribution with instance-level routing confidence. Remarkably, SCOPE with one shared plus one routed encoder outperforms models using all four extra encoders simultaneously, while reducing compute by 24-49%. This demonstrates that intelligent encoder selection beats brute-force aggregation, challenging the prevailing paradigm in multi-encoder VLMs.

Method: Proposed Adaptive Neural Images (ANI) - an efficient neural representation framework that enables adaptation to different requirements. The method successfully implements 4-bit neural representations for compression.

Result: The method reduces bits-per-pixel by 8 times without losing sensitive details or harming fidelity. Achieves new state-of-the-art in PSNR/bpp trade-off.

Conclusion: The work offers a new framework for developing compressed neural fields, demonstrating the viability of highly compressed (4-bit) neural representations for image compression.

Abstract: Implicit Neural Representations (INRs) and Neural Fields are a novel paradigm for signal representation, from images and audio to 3D scenes and videos. The fundamental idea is to represent a signal as a continuous and differentiable neural network. This new approach poses new theoretical questions and challenges. Considering a neural image as a 2D image represented as a neural network, we aim to explore novel neural image compression. In this work, we present a novel analysis on compressing neural fields, with focus on images and introduce Adaptive Neural Images (ANI), an efficient neural representation that enables adaptation to different inference or transmission requirements. Our proposed method allows us to reduce the bits-per-pixel (bpp) of the neural image by 8 times, without losing sensitive details or harming fidelity. Our work offers a new framework for developing compressed neural fields. We achieve a new state-of-the-art in terms of PSNR/bpp trade-off thanks to our successful implementation of 4-bit neural representations.

Method: Proposed SVAGFormer framework that adapts state-of-the-art vision language models for joint spatial and temporal grounding, and created SVAG-Bench benchmark with 688 videos and 19,590 annotated records.

Result: Empirical results show existing models perform poorly on SVAG, especially in dense or complex scenes, highlighting limitations in fine-grained object-action reasoning.

Conclusion: The SVAG task reveals the need for more advanced reasoning capabilities for fine-grained object-action interactions in long videos, with the proposed benchmark and framework providing foundations for future research.

Abstract: Understanding fine-grained actions and accurately localizing their corresponding actors in space and time are fundamental capabilities for advancing next-generation AI systems, including embodied agents, autonomous platforms, and human-AI interaction frameworks. Despite recent progress in video understanding, existing methods predominantly address either coarse-grained action recognition or generic object tracking, thereby overlooking the challenge of jointly detecting and tracking multiple objects according to their actions while grounding them temporally. To address this gap, we introduce Spatio-temporal Video Action Grounding (SVAG), a novel task that requires models to simultaneously detect, track, and temporally localize all referent objects in videos based on natural language descriptions of their actions. To support this task, we construct SVAG-Bench, a large-scale benchmark comprising 688 videos, 19,590 annotated records, and 903 unique verbs, covering a diverse range of objects, actions, and real-world scenes. We further propose SVAGFormer, a baseline framework that adapts state of the art vision language models for joint spatial and temporal grounding, and introduce SVAGEval, a standardized evaluation toolkit for fair and reproducible benchmarking. Empirical results show that existing models perform poorly on SVAG, particularly in dense or complex scenes, underscoring the need for more advanced reasoning over fine-grained object-action interactions in long videos.

Method: Proposed Coral framework adapts pre-trained MLLMs by integrating embedding reconstruction to preserve fine-grained conditional elements and contrastive learning to extract comprehensive global semantics.

Result: Coral achieves 45.9% performance improvement over conventional approaches on MERIT dataset, with strong generalization capabilities validated across 8 established retrieval benchmarks.

Conclusion: The contributions - novel dataset, identification of critical limitations, and innovative fine-tuning framework - establish a foundation for future research in interleaved multi-condition semantic retrieval.

Abstract: Semantic retrieval is crucial for modern applications yet remains underexplored in current research. Existing datasets are limited to single languages, single images, or singular retrieval conditions, often failing to fully exploit the expressive capacity of visual information as evidenced by maintained performance when images are replaced with captions. However, practical retrieval scenarios frequently involve interleaved multi-condition queries with multiple images. Hence, this paper introduces MERIT, the first multilingual dataset for interleaved multi-condition semantic retrieval, comprising 320,000 queries with 135,000 products in 5 languages, covering 7 distinct product categories. Extensive experiments on MERIT identify existing models’s limitation: focusing solely on global semantic information while neglecting specific conditional elements in queries. Consequently, we propose Coral, a novel fine-tuning framework that adapts pre-trained MLLMs by integrating embedding reconstruction to preserve fine-grained conditional elements and contrastive learning to extract comprehensive global semantics. Experiments demonstrate that Coral achieves a 45.9% performance improvement over conventional approaches on MERIT, with strong generalization capabilities validated across 8 established retrieval benchmarks. Collectively, our contributions - a novel dataset, identification of critical limitations in existing approaches, and an innovative fine-tuning framework - establish a foundation for future research in interleaved multi-condition semantic retrieval.

Method: Created SeqBench with 320 prompts spanning various narrative complexities and 2,560 human-annotated videos from 8 state-of-the-art models. Designed a Dynamic Temporal Graphs (DTG)-based automatic evaluation metric to capture long-range dependencies and temporal ordering efficiently.

Result: The DTG-based metric shows strong correlation with human annotations. Evaluation reveals critical limitations in current models: failure to maintain consistent object states, physically implausible multi-object results, and difficulties preserving realistic timing and ordering between sequential actions.

Conclusion: SeqBench provides the first systematic framework for evaluating narrative coherence in T2V generation and offers concrete insights for improving sequential reasoning capabilities in future models.

Abstract: Text-to-video (T2V) generation models have made significant progress in creating visually appealing videos. However, they struggle with generating coherent sequential narratives that require logical progression through multiple events. Existing T2V benchmarks primarily focus on visual quality metrics but fail to evaluate narrative coherence over extended sequences. To bridge this gap, we present SeqBench, a comprehensive benchmark for evaluating sequential narrative coherence in T2V generation. SeqBench includes a carefully designed dataset of 320 prompts spanning various narrative complexities, with 2,560 human-annotated videos generated from 8 state-of-the-art T2V models. Additionally, we design a Dynamic Temporal Graphs (DTG)-based automatic evaluation metric, which can efficiently capture long-range dependencies and temporal ordering while maintaining computational efficiency. Our DTG-based metric demonstrates a strong correlation with human annotations. Through systematic evaluation using SeqBench, we reveal critical limitations in current T2V models: failure to maintain consistent object states across multi-action sequences, physically implausible results in multi-object scenarios, and difficulties in preserving realistic timing and ordering relationships between sequential actions. SeqBench provides the first systematic framework for evaluating narrative coherence in T2V generation and offers concrete insights for improving sequential reasoning capabilities in future models. Please refer to https://videobench.github.io/SeqBench.github.io/ for more details.

Method: Two task frameworks: (1) ME-STR integrates ME spotting and recognition in a unified sequential pipeline, (2) ME-VQA uses multimodal large language models for visual question answering about micro-expressions.

Result: The paper introduces the MEGC 2025 challenge with these two new tasks, requiring participating algorithms to run on a test set and submit results to a leaderboard.

Conclusion: The MEGC 2025 challenge aims to advance ME analysis by addressing limitations of conventional approaches and leveraging emerging multimodal AI technologies for more integrated and comprehensive understanding of micro-expressions.

Abstract: Facial micro-expressions (MEs) are involuntary movements of the face that occur spontaneously when a person experiences an emotion but attempts to suppress or repress the facial expression, typically found in a high-stakes environment. In recent years, substantial advancements have been made in the areas of ME recognition, spotting, and generation. However, conventional approaches that treat spotting and recognition as separate tasks are suboptimal, particularly for analyzing long-duration videos in realistic settings. Concurrently, the emergence of multimodal large language models (MLLMs) and large vision-language models (LVLMs) offers promising new avenues for enhancing ME analysis through their powerful multimodal reasoning capabilities. The ME grand challenge (MEGC) 2025 introduces two tasks that reflect these evolving research directions: (1) ME spot-then-recognize (ME-STR), which integrates ME spotting and subsequent recognition in a unified sequential pipeline; and (2) ME visual question answering (ME-VQA), which explores ME understanding through visual question answering, leveraging MLLMs or LVLMs to address diverse question types related to MEs. All participating algorithms are required to run on this test set and submit their results on a leaderboard. More details are available at https://megc2025.github.io.

Method: Uses two adaptation stages: 1) Inbetweening with keyframing layers that modulate motion latents while preserving the latent manifold, and 2) Scene-aware inbetweening with a scene-conditioning layer that injects scene geometry through cross-attention.

Result: Experimental results show that SceneAdapt effectively injects scene awareness into text-to-motion models.

Conclusion: The framework successfully bridges disjoint datasets to inject scene-awareness into text-to-motion generation without requiring large-scale combined datasets.

Abstract: Human motion is inherently diverse and semantically rich, while also shaped by the surrounding scene. However, existing motion generation approaches address either motion semantics or scene-awareness in isolation, since constructing large-scale datasets with both rich text–motion coverage and precise scene interactions is extremely challenging. In this work, we introduce SceneAdapt, a framework that injects scene awareness into text-conditioned motion models by leveraging disjoint scene–motion and text–motion datasets through two adaptation stages: inbetweening and scene-aware inbetweening. The key idea is to use motion inbetweening, learnable without text, as a proxy task to bridge two distinct datasets and thereby inject scene-awareness to text-to-motion models. In the first stage, we introduce keyframing layers that modulate motion latents for inbetweening while preserving the latent manifold. In the second stage, we add a scene-conditioning layer that injects scene geometry by adaptively querying local context through cross-attention. Experimental results show that SceneAdapt effectively injects scene awareness into text-to-motion models, and we further analyze the mechanisms through which this awareness emerges. Code and models will be released.

Method: Proposed OD-CNN-ECG-Mamba framework that combines convolutional feature extraction with Mamba (a selective state space model) built upon Vision Mamba for enhanced temporal dependency representation in ECG data.

Result: Achieved substantially higher AUPRC and AUROC scores than previously published algorithms on twelve lead electrocardiograms from PhysioNet Computing in Cardiology Challenges 2020 and 2021.

Conclusion: Mamba-based architectures show strong potential for advancing reliable ECG classification, supporting early diagnosis, personalized treatment, and enhancing accessibility in telemedicine and resource-constrained healthcare systems.

Abstract: Accurate detection of cardiac abnormalities from electrocardiogram recordings is regarded as essential for clinical diagnostics and decision support. Traditional deep learning models such as residual networks and transformer architectures have been applied successfully to this task, but their performance has been limited when long sequential signals are processed. Recently, state space models have been introduced as an efficient alternative. In this study, a hybrid framework named One Dimensional Convolutional Neural Network Electrocardiogram Mamba is introduced, in which convolutional feature extraction is combined with Mamba, a selective state space model designed for effective sequence modeling. The model is built upon Vision Mamba, a bidirectional variant through which the representation of temporal dependencies in electrocardiogram data is enhanced. Comprehensive experiments on the PhysioNet Computing in Cardiology Challenges of 2020 and 2021 were conducted, and superior performance compared with existing methods was achieved. Specifically, the proposed model achieved substantially higher AUPRC and AUROC scores than those reported by the best previously published algorithms on twelve lead electrocardiograms. These results demonstrate the potential of Mamba-based architectures to advance reliable ECG classification. This capability supports early diagnosis and personalized treatment, while enhancing accessibility in telemedicine and resource-constrained healthcare systems.

Method: Extensive experiments and empirical analysis of implicit neural image compression methods including Fourier Feature Networks and Siren.

Result: Provides deeper understanding of INR limitations and identifies critical factors affecting their performance.

Conclusion: Offers valuable insights for future research on improving INRs for image compression applications.

Abstract: Implicit Neural Representations (INRs) are a novel paradigm for signal representation that have attracted considerable interest for image compression. INRs offer unprecedented advantages in signal resolution and memory efficiency, enabling new possibilities for compression techniques. However, the existing limitations of INRs for image compression have not been sufficiently addressed in the literature. In this work, we explore the critical yet overlooked limiting factors of INRs, such as computational cost, unstable performance, and robustness. Through extensive experiments and empirical analysis, we provide a deeper and more nuanced understanding of implicit neural image compression methods such as Fourier Feature Networks and Siren. Our work also offers valuable insights for future research in this area.

Method: Combines pair-wise pose estimation with input/output augmentation to disentangle camera pose from scene content, using unconstrained latent pose variables without SE(3) parameterization.

Result: XFactor achieves transferable pose representations, significantly outperforms prior pose-free NVS transformers, and shows latent poses correlate with real-world poses.

Conclusion: True novel view synthesis requires transferable pose representations, which XFactor achieves without geometric constraints, enabling geometry-free self-supervised learning.

Abstract: In this paper, we identify that the key criterion for determining whether a model is truly capable of novel view synthesis (NVS) is transferability: Whether any pose representation extracted from one video sequence can be used to re-render the same camera trajectory in another. We analyze prior work on self-supervised NVS and find that their predicted poses do not transfer: The same set of poses lead to different camera trajectories in different 3D scenes. Here, we present XFactor, the first geometry-free self-supervised model capable of true NVS. XFactor combines pair-wise pose estimation with a simple augmentation scheme of the inputs and outputs that jointly enables disentangling camera pose from scene content and facilitates geometric reasoning. Remarkably, we show that XFactor achieves transferability with unconstrained latent pose variables, without any 3D inductive biases or concepts from multi-view geometry – such as an explicit parameterization of poses as elements of SE(3). We introduce a new metric to quantify transferability, and through large-scale experiments, we demonstrate that XFactor significantly outperforms prior pose-free NVS transformers, and show that latent poses are highly correlated with real-world poses through probing experiments.

Method: Proposes Value-Driven Reordering Scanning (VRS-UIE) with Multi-Granularity Value Guidance Learning (MVGL) to generate pixel-aligned value maps for dynamic scanning sequence reordering, Mamba-Conv Mixer (MCM) blocks for priority-driven global sequencing with local convolutions, and Cross-Feature Bridge (CFB) for multi-level feature fusion.

Result: Achieves state-of-the-art performance, surpassing WMamba by 0.89 dB on average, with effective water bias suppression and preservation of structural and color fidelity. Also presents VRS-UIE-S, a lightweight version suitable for real-time applications.

Conclusion: VRS-UIE successfully addresses the limitations of standard SSMs in underwater image enhancement by prioritizing informative regions through value-driven scanning reordering, delivering superior enhancement performance while maintaining efficiency for practical applications.

Abstract: State Space Models (SSMs) have emerged as a promising backbone for vision tasks due to their linear complexity and global receptive field. However, in the context of Underwater Image Enhancement (UIE), the standard sequential scanning mechanism is fundamentally challenged by the unique statistical distribution characteristics of underwater scenes. The predominance of large-portion, homogeneous but useless oceanic backgrounds can dilute the feature representation responses of sparse yet valuable targets, thereby impeding effective state propagation and compromising the model’s ability to preserve both local semantics and global structure. To address this limitation, we propose a novel Value-Driven Reordering Scanning framework for UIE, termed VRS-UIE. Its core innovation is a Multi-Granularity Value Guidance Learning (MVGL) module that generates a pixel-aligned value map to dynamically reorder the SSM’s scanning sequence. This prioritizes informative regions to facilitate the long-range state propagation of salient features. Building upon the MVGL, we design a Mamba-Conv Mixer (MCM) block that synergistically integrates priority-driven global sequencing with dynamically adjusted local convolutions, thereby effectively modeling both large-portion oceanic backgrounds and high-value semantic targets. A Cross-Feature Bridge (CFB) further refines multi-level feature fusion. Extensive experiments demonstrate that our VRS-UIE framework sets a new state-of-the-art, delivering superior enhancement performance (surpassing WMamba by 0.89 dB on average) by effectively suppressing water bias and preserving structural and color fidelity. Furthermore, by incorporating efficient convolutional operators and resolution rescaling, we construct a light-weight yet effective scheme, VRS-UIE-S, suitable for real-time UIE applications.

Method: Introduces a direction-aware, multi-scale gradient loss that supervises horizontal and vertical gradient components separately while preserving their sign across different scales.

Result: The approach promotes sharper, better-aligned edges and richer texture preservation without changing model architectures or training protocols, demonstrating effectiveness on multiple public benchmarks.

Conclusion: The direction-aware gradient loss provides clear directional guidance at multiple resolutions, improving edge fidelity and texture preservation in infrared and visible image fusion.

Abstract: Infrared and visible image fusion aims to integrate complementary information from co-registered source images to produce a single, informative result. Most learning-based approaches train with a combination of structural similarity loss, intensity reconstruction loss, and a gradient-magnitude term. However, collapsing gradients to their magnitude removes directional information, yielding ambiguous supervision and suboptimal edge fidelity. We introduce a direction-aware, multi-scale gradient loss that supervises horizontal and vertical components separately and preserves their sign across scales. This axis-wise, sign-preserving objective provides clear directional guidance at both fine and coarse resolutions, promoting sharper, better-aligned edges and richer texture preservation without changing model architectures or training protocols. Experiments on open-source model and multiple public benchmarks demonstrate effectiveness of our approach.

Method: Combines efficient style harmonization through z-normalization with a bidirectional deformable image registration (DIR) strategy. The DIR network is jointly trained with segmentation and discriminator networks to guide registration with respect to ROI and generate anatomically plausible transformations.

Result: Outperforms existing baselines across all experiments. Achieves up to 15% relative improvement in Dice score when transferring from young, healthy populations to clinical dementia patients, with largest gains in scenarios with substantial content shift.

Conclusion: The framework effectively addresses domain shifts for accurate hippocampus segmentation across diverse populations, demonstrating particular efficacy in handling substantial content variations.

Abstract: Deep learning models for medical image segmentation often struggle when deployed across different datasets due to domain shifts - variations in both image appearance, known as style, and population-dependent anatomical characteristics, referred to as content. This paper presents a novel unsupervised domain adaptation framework that directly addresses domain shifts encountered in cross-domain hippocampus segmentation from MRI, with specific emphasis on content variations. Our approach combines efficient style harmonisation through z-normalisation with a bidirectional deformable image registration (DIR) strategy. The DIR network is jointly trained with segmentation and discriminator networks to guide the registration with respect to a region of interest and generate anatomically plausible transformations that align source images to the target domain. We validate our approach through comprehensive evaluations on both a synthetic dataset using Morpho-MNIST (for controlled validation of core principles) and three MRI hippocampus datasets representing populations with varying degrees of atrophy. Across all experiments, our method outperforms existing baselines. For hippocampus segmentation, when transferring from young, healthy populations to clinical dementia patients, our framework achieves up to 15% relative improvement in Dice score compared to standard augmentation methods, with the largest gains observed in scenarios with substantial content shift. These results highlight the efficacy of our approach for accurate hippocampus segmentation across diverse populations.

Method: The authors construct CountHalluSet dataset suite with well-defined counting criteria (ToyShape, SimObject, RealHand), develop a standardized evaluation protocol for quantifying counting hallucinations, and systematically examine how different sampling conditions affect hallucination levels.

Result: The study reveals that common evaluation metrics like FID fail to capture counting hallucinations consistently, and provides systematic analysis of how different sampling conditions (solver type, ODE solver order, sampling steps, initial noise) affect counting hallucination levels.

Conclusion: This work takes the first step toward systematically quantifying hallucinations in diffusion models and offers new insights into investigating hallucination phenomena in image generation, highlighting the need for better evaluation metrics that capture factual accuracy.

Abstract: Diffusion probabilistic models (DPMs) have demonstrated remarkable progress in generative tasks, such as image and video synthesis. However, they still often produce hallucinated samples (hallucinations) that conflict with real-world knowledge, such as generating an implausible duplicate cup floating beside another cup. Despite their prevalence, the lack of feasible methodologies for systematically quantifying such hallucinations hinders progress in addressing this challenge and obscures potential pathways for designing next-generation generative models under factual constraints. In this work, we bridge this gap by focusing on a specific form of hallucination, which we term counting hallucination, referring to the generation of an incorrect number of instances or structured objects, such as a hand image with six fingers, despite such patterns being absent from the training data. To this end, we construct a dataset suite CountHalluSet, with well-defined counting criteria, comprising ToyShape, SimObject, and RealHand. Using these datasets, we develop a standardized evaluation protocol for quantifying counting hallucinations, and systematically examine how different sampling conditions in DPMs, including solver type, ODE solver order, sampling steps, and initial noise, affect counting hallucination levels. Furthermore, we analyze their correlation with common evaluation metrics such as FID, revealing that this widely used image quality metric fails to capture counting hallucinations consistently. This work aims to take the first step toward systematically quantifying hallucinations in diffusion models and offer new insights into the investigation of hallucination phenomena in image generation.

Method: Uses Spatio-Temporal Feature Memory (SFM) to cache features from previous frames, Feature Most-Similar Propagation (FMP) for temporal consistency, and cross-attention maps for automatic mask extraction of target objects.

Result: Outperforms state-of-the-art methods in both efficiency and visual fidelity, demonstrating superior effectiveness and practicality.

Conclusion: Edit-Your-Interest provides an efficient and high-quality solution for text-driven video editing with reduced computational requirements and improved temporal consistency.

Abstract: Text-to-image (T2I) diffusion models have recently demonstrated significant progress in video editing. However, existing video editing methods are severely limited by their high computational overhead and memory consumption. Furthermore, these approaches often sacrifice visual fidelity, leading to undesirable temporal inconsistencies and artifacts such as blurring and pronounced mosaic-like patterns. We propose Edit-Your-Interest, a lightweight, text-driven, zero-shot video editing method. Edit-Your-Interest introduces a spatio-temporal feature memory to cache features from previous frames, significantly reducing computational overhead compared to full-sequence spatio-temporal modeling approaches. Specifically, we first introduce a Spatio-Temporal Feature Memory bank (SFM), which is designed to efficiently cache and retain the crucial image tokens processed by spatial attention. Second, we propose the Feature Most-Similar Propagation (FMP) method. FMP propagates the most relevant tokens from previous frames to subsequent ones, preserving temporal consistency. Finally, we introduce an SFM update algorithm that continuously refreshes the cached features, ensuring their long-term relevance and effectiveness throughout the video sequence. Furthermore, we leverage cross-attention maps to automatically extract masks for the instances of interest. These masks are seamlessly integrated into the diffusion denoising process, enabling fine-grained control over target objects and allowing Edit-Your-Interest to perform highly accurate edits while robustly preserving the background integrity. Extensive experiments decisively demonstrate that the proposed Edit-Your-Interest outperforms state-of-the-art methods in both efficiency and visual fidelity, validating its superior effectiveness and practicality.

Method: Proposed EgoSoD (EgoSocial Detection), an end-to-end method that integrates multimodal contextual cues (audio and visual) into a social thinking graph to dynamically model participants and interactions, enabling proactive detection of intervention timing and social interactions.

Result: EgoSoD improves Phi-4 by 45.6% and Gemini 2.5 Pro by 9.9% on Intervention Timing performance, and improves Phi-4 by 20.4% and Gemini 2.5 Pro by 6.9% on overall Social Interaction performance. Current OLLMs struggle with intervention timing detection (only 14.4% for Gemini 2.5 Pro).

Conclusion: The EgoSocial dataset and EgoSoD method effectively address the limitations of current AI in social awareness, providing a robust framework for detecting optimal intervention timing in social interactions from an egocentric perspective.

Abstract: As AR/VR technologies become integral to daily life, there’s a growing need for AI that understands human social dynamics from an egocentric perspective. However, current LLMs often lack the social awareness to discern when to intervene as AI assistant. This leads to constant, socially unaware responses that may disrupt natural conversation and negatively impact user focus. To address these limitations, we introduce EgoSocial, a large-scale egocentric dataset with 13,500 social video-question pairs, specifically designed to benchmark intervention in social interaction perception. We also present an in-depth analysis of current omnimodal LLMs (OLLMs) to assess their effectiveness in detecting diverse social contextual cues. Experiments show that OLLMs still struggle to detect the intervention timing (14.4% for Gemini 2.5 Pro). We also propose EgoSoD (EgoSocial Detection), an end-to-end method for robustly discerning social dynamics. Informed by our OLLM analysis, EgoSoD integrates multimodal contextual cues (e.g., audio and visual cues) into a social thinking graph, dynamically modeling participants and interactions. Our method proactively detects intervention timing and social interactions, precisely determining when to intervene. Our EgoSoD improves Phi-4 by 45.6% and Gemini 2.5 Pro by 9.9% on Intervention Timing performance, and improves Phi-4 by 20.4% and Gemini 2.5 Pro by 6.9% on overall Social Interaction performance. We will release the dataset and code soon.

Method: Developed DriveCritic framework with two components: a curated dataset of challenging scenarios annotated with human preferences, and a VLM-based evaluator fine-tuned using supervised and reinforcement learning to integrate visual and symbolic context.

Result: DriveCritic significantly outperforms existing metrics and baselines in matching human preferences and demonstrates strong context awareness.

Conclusion: The work provides a more reliable, human-aligned foundation for evaluating autonomous driving systems through context-aware assessment.

Abstract: Benchmarking autonomous driving planners to align with human judgment remains a critical challenge, as state-of-the-art metrics like the Extended Predictive Driver Model Score (EPDMS) lack context awareness in nuanced scenarios. To address this, we introduce DriveCritic, a novel framework featuring two key contributions: the DriveCritic dataset, a curated collection of challenging scenarios where context is critical for correct judgment and annotated with pairwise human preferences, and the DriveCritic model, a Vision-Language Model (VLM) based evaluator. Fine-tuned using a two-stage supervised and reinforcement learning pipeline, the DriveCritic model learns to adjudicate between trajectory pairs by integrating visual and symbolic context. Experiments show DriveCritic significantly outperforms existing metrics and baselines in matching human preferences and demonstrates strong context awareness. Overall, our work provides a more reliable, human-aligned foundation to evaluating autonomous driving systems.

Method: Uses a Variational Principle (VP) grid generation approach that constructs non-folding grids with prescribed Jacobian determinant and curl, generating inverse transformations within the diffeomorphism group rather than image space.

Result: Outperformed ANTs-SyN, Freesurfer-Easyreg, and FSL-Fnirt in 150 brain scan registrations from OASIS-1 dataset, achieving higher Dice scores for 35 regions of interest and better transformation regularity, inverse accuracy, and consistency.

Conclusion: VPreg provides superior registration performance with guaranteed diffeomorphic properties, making it particularly valuable for computational anatomy and neuroimaging applications requiring accurate inverse transformations.

Abstract: This paper introduces VPreg, a novel diffeomorphic image registration method. This work provides several improvements to our past work on mesh generation and diffeomorphic image registration. VPreg aims to achieve excellent registration accuracy while controlling the quality of the registration transformations. It ensures a positive Jacobian determinant of the spatial transformation and provides an accurate approximation of the inverse of the registration, a crucial property for many neuroimaging workflows. Unlike conventional methods, VPreg generates this inverse transformation within the group of diffeomorphisms rather than operating on the image space. The core of VPreg is a grid generation approach, referred to as \emph{Variational Principle} (VP), which constructs non-folding grids with prescribed Jacobian determinant and curl. These VP-generated grids guarantee diffeomorphic spatial transformations essential for computational anatomy and morphometry, and provide a more accurate inverse than existing methods. To assess the potential of the proposed approach, we conduct a performance analysis for 150 registrations of brain scans from the OASIS-1 dataset. Performance evaluation based on Dice scores for 35 regions of interest, along with an empirical analysis of the properties of the computed spatial transformations, demonstrates that VPreg outperforms state-of-the-art methods in terms of Dice scores, regularity properties of the computed transformation, and accuracy and consistency of the provided inverse map. We compare our results to ANTs-SyN, Freesurfer-Easyreg, and FSL-Fnirt.

Method: Evaluated 3D CNNs and LSTM networks on a dataset of 1,200 ASL signs across 50 classes, comparing accuracy, computational efficiency, and latency under similar training conditions. Also tested a hybrid 3D CNN-LSTM model.

Result: 3D CNNs achieved 92.4% recognition accuracy but required 3.2% more processing time per frame. LSTMs maintained 86.7% accuracy with significantly lower resource consumption. The hybrid model showed decent performance.

Conclusion: Context-dependent architecture selection is crucial for practical implementation, with trade-offs between recognition precision and real-time operational requirements in edge computing environments.

Abstract: This study investigates the performance of 3D Convolutional Neural Networks (3D CNNs) and Long Short-Term Memory (LSTM) networks for real-time American Sign Language (ASL) recognition. Though 3D CNNs are good at spatiotemporal feature extraction from video sequences, LSTMs are optimized for modeling temporal dependencies in sequential data. We evaluate both architectures on a dataset containing 1,200 ASL signs across 50 classes, comparing their accuracy, computational efficiency, and latency under similar training conditions. Experimental results demonstrate that 3D CNNs achieve 92.4% recognition accuracy but require 3.2% more processing time per frame compared to LSTMs, which maintain 86.7% accuracy with significantly lower resource consumption. The hybrid 3D CNNLSTM model shows decent performance, which suggests that context-dependent architecture selection is crucial for practical implementation.This project provides professional benchmarks for developing assistive technologies, highlighting trade-offs between recognition precision and real-time operational requirements in edge computing environments.

Method: Utilizes efficient latent representations and foveated rendering, incorporating novel perceptual design to create multi-modal latent representations.

Result: Achieved 500-bit capacity (5x improvement), 99.95% accuracy (1 failure bit out of 2000), and maintained visual quality with 31.47 dB PSNR and 0.13 LPIPS.

Conclusion: The novel perceptual design in multi-modal latent representations effectively improves steganography capacity and accuracy while preserving visual quality.

Abstract: Steganography finds its use in visual medium such as providing metadata and watermarking. With support of efficient latent representations and foveated rendering, we trained models that improve existing capacity limits from 100 to 500 bits, while achieving better accuracy of up to 1 failure bit out of 2000, at 200K test bits. Finally, we achieve a comparable visual quality of 31.47 dB PSNR and 0.13 LPIPS, showing the effectiveness of novel perceptual design in creating multi-modal latent representations in steganography.

Method: DP-TTA uses dictionary learning to encode intrinsic TEM signal characteristics (exponential decay, smoothness) as prior knowledge, integrated into DTEMDNet during training. During testing, this prior guides dynamic adaptation via self-supervised losses from dictionary-driven consistency and signal variation.

Result: Extensive experiments show the method achieves significantly better performance than existing TEM denoising methods and TTA approaches.

Conclusion: Leveraging intrinsic physical characteristics as dictionary-driven prior enables effective test-time adaptation for TEM signal denoising across diverse geographical environments.

Abstract: Transient Electromagnetic (TEM) method is widely used in various geophysical applications, providing valuable insights into subsurface properties. However, time-domain TEM signals are often submerged in various types of noise. While recent deep learning-based denoising models have shown strong performance, these models are mostly trained on simulated or single real-world scenario data, overlooking the significant differences in noise characteristics from different geographical regions. Intuitively, models trained in one environment often struggle to perform well in new settings due to differences in geological conditions, equipment, and external interference, leading to reduced denoising performance. To this end, we propose the Dictionary-driven Prior Regularization Test-time Adaptation (DP-TTA). Our key insight is that TEM signals possess intrinsic physical characteristics, such as exponential decay and smoothness, which remain consistent across different regions regardless of external conditions. These intrinsic characteristics serve as ideal prior knowledge for guiding the TTA strategy, which helps the pre-trained model dynamically adjust parameters by utilizing self-supervised losses, improving denoising performance in new scenarios. To implement this, we customized a network, named DTEMDNet. Specifically, we first use dictionary learning to encode these intrinsic characteristics as a dictionary-driven prior, which is integrated into the model during training. At the testing stage, this prior guides the model to adapt dynamically to new environments by minimizing self-supervised losses derived from the dictionary-driven consistency and the signal one-order variation. Extensive experimental results demonstrate that the proposed method achieves much better performance than existing TEM denoising methods and TTA methods.

Method: Proposes STT-GS strategy: 1) Sample subset of images as pilot data from each client using feature-domain clustering for representative selection; 2) Joint client selection and power control framework using penalty alternating majorization minimization algorithm to maximize GS-oriented function under constraints.

Result: Significantly outperforms existing benchmarks on real-world datasets. GS-oriented objective can be accurately predicted with low sampling ratios (e.g., 10%). Achieves excellent tradeoff between view contributions and communication costs.

Conclusion: The proposed STT-GS framework effectively addresses the causality dilemma in EGS optimization, enabling efficient resource allocation for maximizing Gaussian splatting quality through intelligent sampling and joint client-power management.

Abstract: Edge Gaussian splatting (EGS), which aggregates data from distributed clients and trains a global GS model at the edge server, is an emerging paradigm for scene reconstruction. Unlike traditional edge resource management methods that emphasize communication throughput or general-purpose learning performance, EGS explicitly aims to maximize the GS qualities, rendering existing approaches inapplicable. To address this problem, this paper formulates a novel GS-oriented objective function that distinguishes the heterogeneous view contributions of different clients. However, evaluating this function in turn requires clients’ images, leading to a causality dilemma. To this end, this paper further proposes a sample-then-transmit EGS (or STT-GS for short) strategy, which first samples a subset of images as pilot data from each client for loss prediction. Based on the first-stage evaluation, communication resources are then prioritized towards more valuable clients. To achieve efficient sampling, a feature-domain clustering (FDC) scheme is proposed to select the most representative data and pilot transmission time minimization (PTTM) is adopted to reduce the pilot overhead.Subsequently, we develop a joint client selection and power control (JCSPC) framework to maximize the GS-oriented function under communication resource constraints. Despite the nonconvexity of the problem, we propose a low-complexity efficient solution based on the penalty alternating majorization minimization (PAMM) algorithm. Experiments unveil that the proposed scheme significantly outperforms existing benchmarks on real-world datasets. It is found that the GS-oriented objective can be accurately predicted with low sampling ratios (e.g.,10%), and our method achieves an excellent tradeoff between view contributions and communication costs.

Method: Two-stage framework that extracts segmentation, graphics, and depth features from input images, uses deformable multi-level fusion mechanism to fuse these features, and incorporates knowledge distilled from SAM (Segment Anything Model) to enhance accuracy.

Result: Achieves state-of-the-art performance on the SemanticKITTI benchmark without increasing training costs.

Conclusion: CIGOcc demonstrates that effective multi-level representation fusion can significantly improve camera-based occupancy prediction performance, providing a promising direction beyond structural modifications.

Abstract: Camera-based occupancy prediction is a mainstream approach for 3D perception in autonomous driving, aiming to infer complete 3D scene geometry and semantics from 2D images. Almost existing methods focus on improving performance through structural modifications, such as lightweight backbones and complex cascaded frameworks, with good yet limited performance. Few studies explore from the perspective of representation fusion, leaving the rich diversity of features in 2D images underutilized. Motivated by this, we propose \textbf{CIGOcc, a two-stage occupancy prediction framework based on multi-level representation fusion. \textbf{CIGOcc extracts segmentation, graphics, and depth features from an input image and introduces a deformable multi-level fusion mechanism to fuse these three multi-level features. Additionally, CIGOcc incorporates knowledge distilled from SAM to further enhance prediction accuracy. Without increasing training costs, CIGOcc achieves state-of-the-art performance on the SemanticKITTI benchmark. The code is provided in the supplementary material and will be released https://github.com/VitaLemonTea1/CIGOcc

Method: Developed Paper Copilot system that creates durable digital archives of peer reviews across computer-science venues and conducts large-scale empirical analysis of ICLR reviews over multiple years.

Result: Created an open dataset and infrastructure that enables reproducible research on peer review evolution, allowing tracking of changes and diagnosis of failure modes.

Conclusion: Paper Copilot resources help the community work toward a more robust, transparent, and reliable peer-review system through evidence-based improvements.

Abstract: The rapid growth of AI conferences is straining an already fragile peer-review system, leading to heavy reviewer workloads, expertise mismatches, inconsistent evaluation standards, superficial or templated reviews, and limited accountability under compressed timelines. In response, conference organizers have introduced new policies and interventions to preserve review standards. Yet these ad-hoc changes often create further concerns and confusion about the review process, leaving how papers are ultimately accepted - and how practices evolve across years - largely opaque. We present Paper Copilot, a system that creates durable digital archives of peer reviews across a wide range of computer-science venues, an open dataset that enables researchers to study peer review at scale, and a large-scale empirical analysis of ICLR reviews spanning multiple years. By releasing both the infrastructure and the dataset, Paper Copilot supports reproducible research on the evolution of peer review. We hope these resources help the community track changes, diagnose failure modes, and inform evidence-based improvements toward a more robust, transparent, and reliable peer-review system.

Method: Divides body into different regions for localized motion style encoding, uses part-aware style injection and part-aware denoising network, and employs part-aware attention block to allow rhythm/emotion cues to guide each body region precisely.

Result: Outperforms existing methods in generating natural and expressive 3D human motion sequences that align with speech rhythm and emotional state variations.

Conclusion: The proposed MimicParts framework successfully addresses limitations of current approaches by capturing fine-grained regional motion style differences and dynamically adapting to speech rhythm and emotion changes.

Abstract: Generating stylized 3D human motion from speech signals presents substantial challenges, primarily due to the intricate and fine-grained relationships among speech signals, individual styles, and the corresponding body movements. Current style encoding approaches either oversimplify stylistic diversity or ignore regional motion style differences (e.g., upper vs. lower body), limiting motion realism. Additionally, motion style should dynamically adapt to changes in speech rhythm and emotion, but existing methods often overlook this. To address these issues, we propose MimicParts, a novel framework designed to enhance stylized motion generation based on part-aware style injection and part-aware denoising network. It divides the body into different regions to encode localized motion styles, enabling the model to capture fine-grained regional differences. Furthermore, our part-aware attention block allows rhythm and emotion cues to guide each body region precisely, ensuring that the generated motion aligns with variations in speech rhythm and emotional state. Experimental results show that our method outperforming existing methods showcasing naturalness and expressive 3D human motion sequences.

Method: Revisits VP and VPT designs from first principles, conceptualizes them within a unified Prompt-based Adaptation (PA) framework, provides taxonomy categorizing methods into learnable, generative, and non-learnable prompts organized by injection granularity (pixel-level and token-level).

Result: First comprehensive survey dedicated to PA’s methodologies and applications, examining integrations across medical imaging, 3D point clouds, vision-language tasks, test-time adaptation, and trustworthy AI, with summarized benchmarks and identified challenges.

Conclusion: The survey provides a clear roadmap for researchers and practitioners to understand and explore the evolving landscape of PA-related research, establishing systematic foundations for future work in visual prompt-based adaptation techniques.

Abstract: In computer vision, Visual Prompting (VP) and Visual Prompt Tuning (VPT) have recently emerged as lightweight and effective alternatives to full fine-tuning for adapting large-scale vision models within the ``pretrain-then-finetune’' paradigm. However, despite rapid progress, their conceptual boundaries remain blurred, as VP and VPT are frequently used interchangeably in current research, reflecting a lack of systematic distinction between these techniques and their respective applications. In this survey, we revisit the designs of VP and VPT from first principles, and conceptualize them within a unified framework termed Prompt-based Adaptation (PA). We provide a taxonomy that categorizes existing methods into learnable, generative, and non-learnable prompts, and further organizes them by injection granularity – pixel-level and token-level. Beyond the core methodologies, we examine PA’s integrations across diverse domains, including medical imaging, 3D point clouds, and vision-language tasks, as well as its role in test-time adaptation and trustworthy AI. We also summarize current benchmarks and identify key challenges and future directions. To the best of our knowledge, we are the first comprehensive survey dedicated to PA’s methodologies and applications in light of their distinct characteristics. Our survey aims to provide a clear roadmap for researchers and practitioners in all area to understand and explore the evolving landscape of PA-related research.

Method: Sample-centric multi-task learning framework with shared-encoder architecture that jointly learns sample-level defect classification and pixel-level mask localization. Sample-level supervision modulates feature distribution and boosts recall for small/low-contrast defects, while segmentation branch preserves boundary details.

Result: Experiments on two benchmark datasets show substantial improvement in reliability of sample-level decisions and completeness of defect localization compared to existing approaches.

Conclusion: The proposed framework effectively addresses the mismatch between pixel-centric optimization and sample-level QC decisions, improving inspection reliability for industrial applications with challenging defect characteristics.

Abstract: Industrial surface defect inspection for sample-wise quality control (QC) must simultaneously decide whether a given sample contains defects and localize those defects spatially. In real production lines, extreme foreground-background imbalance, defect sparsity with a long-tailed scale distribution, and low contrast are common. As a result, pixel-centric training and evaluation are easily dominated by large homogeneous regions, making it difficult to drive models to attend to small or low-contrast defects-one of the main bottlenecks for deployment. Empirically, existing models achieve strong pixel-overlap metrics (e.g., mIoU) but exhibit insufficient stability at the sample level, especially for sparse or slender defects. The root cause is a mismatch between the optimization objective and the granularity of QC decisions. To address this, we propose a sample-centric multi-task learning framework and evaluation suite. Built on a shared-encoder architecture, the method jointly learns sample-level defect classification and pixel-level mask localization. Sample-level supervision modulates the feature distribution and, at the gradient level, continually boosts recall for small and low-contrast defects, while the segmentation branch preserves boundary and shape details to enhance per-sample decision stability and reduce misses. For evaluation, we propose decision-linked metrics, Seg_mIoU and Seg_Recall, which remove the bias of classical mIoU caused by empty or true-negative samples and tightly couple localization quality with sample-level decisions. Experiments on two benchmark datasets demonstrate that our approach substantially improves the reliability of sample-level decisions and the completeness of defect localization.

Method: Two main contributions: (1) CoVAND dataset pipeline using chain-of-thought and VQA-based approach to generate high-quality negation data, and (2) NegToMe text token merging module that groups negation cues with attributes into semantic phrases to preserve correct polarity, combined with parameter-efficient LoRA fine-tuning.

Result: Significant performance improvements on challenging negation benchmarks with lowered false positive rate, boosting NMS-AP by up to +10.8 points on OVDEval and demonstrating generalization to state-of-the-art VLMs.

Conclusion: This work represents a crucial advancement in addressing negation understanding for real-world detection applications, fundamentally tackling the architectural causes of affirmative bias in vision-language models.

Abstract: State-of-the-art vision-language models (VLMs) suffer from a critical failure in understanding negation, often referred to as affirmative bias. This limitation is particularly severe in described object detection (DOD) tasks. To address this, we propose two primary contributions: (1) a new dataset pipeline and (2) a novel, lightweight adaptation recipe. First, we introduce CoVAND, a dataset constructed with a systematic chain-of-thought (CoT) and VQA-based pipeline to generate high-quality, instance-grounded negation data. Second, we propose NegToMe, a novel text token merging module that directly tackles the architectural cause of affirmative bias. NegToMe fundamentally addresses the structural loss of negation cues in tokenization, grouping them with attributes into coherent semantic phrases. It maintains correct polarity at the input level, enabling robust negation understanding even with limited data. For instance, to prevent a model from treating the fragmented tokens “not” and “girl” as simply “girl”, NegToMe binds them into a single token whose meaning is correctly distinguished from that of “girl” alone. This module is integrated with a parameter-efficient and strategic LoRA fine-tuning approach. Our method significantly improves performance on challenging negation benchmarks with a lowered false positive rate, boosting NMS-AP by up to +10.8 points on OVDEval and demonstrating generalization to SoTA VLMs. This work marks a crucial step forward in addressing negation understanding for real-world detection applications.

Method: UniVector encodes vectors as structured queries with instance- and geometry-level information, uses an interaction module for cross-level context exchange, and applies dynamic shape constraints to refine structures and key points.

Result: UniVector achieves state-of-the-art performance on both single- and multi-structure vector extraction tasks, as demonstrated on the new Multi-Vector dataset.

Conclusion: The proposed unified framework successfully extracts multiple vector types within a single model, inspired by human visual perception principles, and establishes new benchmarks for vector extraction tasks.

Abstract: Vector extraction retrieves structured vector geometry from raster images, offering high-fidelity representation and broad applicability. Existing methods, however, are usually tailored to a single vector type (e.g., polygons, polylines, line segments), requiring separate models for different structures. This stems from treating instance attributes (category, structure) and geometric attributes (point coordinates, connections) independently, limiting the ability to capture complex structures. Inspired by the human brain’s simultaneous use of semantic and spatial interactions in visual perception, we propose UniVector, a unified VE framework that leverages instance-geometry interaction to extract multiple vector types within a single model. UniVector encodes vectors as structured queries containing both instance- and geometry-level information, and iteratively updates them through an interaction module for cross-level context exchange. A dynamic shape constraint further refines global structures and key points. To benchmark multi-structure scenarios, we introduce the Multi-Vector dataset with diverse polygons, polylines, and line segments. Experiments show UniVector sets a new state of the art on both single- and multi-structure VE tasks. Code and dataset will be released at https://github.com/yyyyll0ss/UniVector.

Method: Uses explicit prompts (spatial motion to language) and implicit prompts (pseudo-words with learnable descriptors) for dynamic target modeling, with CLIP text encoder for adjustment and Discriminative Feature Augmentor for representation enhancement.

Result: Extensive experiments on MOT17, MOT20, and DanceTrack demonstrate superior performance compared to existing trackers in diverse scenarios.

Conclusion: EPIPTrack exhibits robust adaptability and superior performance in multimodal tracking through dynamic prompt-based modeling.

Abstract: Multimodal semantic cues, such as textual descriptions, have shown strong potential in enhancing target perception for tracking. However, existing methods rely on static textual descriptions from large language models, which lack adaptability to real-time target state changes and prone to hallucinations. To address these challenges, we propose a unified multimodal vision-language tracking framework, named EPIPTrack, which leverages explicit and implicit prompts for dynamic target modeling and semantic alignment. Specifically, explicit prompts transform spatial motion information into natural language descriptions to provide spatiotemporal guidance. Implicit prompts combine pseudo-words with learnable descriptors to construct individualized knowledge representations capturing appearance attributes. Both prompts undergo dynamic adjustment via the CLIP text encoder to respond to changes in target state. Furthermore, we design a Discriminative Feature Augmentor to enhance visual and cross-modal representations. Extensive experiments on MOT17, MOT20, and DanceTrack demonstrate that EPIPTrack outperforms existing trackers in diverse scenarios, exhibiting robust adaptability and superior performance.

Method: Proposes Embedding Disruption Patch Attack (EDPA) that generates adversarial patches to disrupt semantic alignment and maximize representation discrepancy, along with an adversarial fine-tuning defense for visual encoders.

Result: EDPA substantially increases task failure rates of state-of-the-art VLA models on the LIBERO benchmark, while the proposed defense effectively mitigates this performance degradation.

Conclusion: The work demonstrates significant vulnerabilities in VLA models to adversarial attacks and provides effective defense mechanisms, highlighting the importance of robustness in vision-language-action systems for robotics.

Abstract: Vision-Language-Action (VLA) models have achieved revolutionary progress in robot learning, enabling robots to execute complex physical robot tasks from natural language instructions. Despite this progress, their adversarial robustness remains underexplored. In this work, we propose both adversarial patch attack and corresponding defense strategies for VLA models. We first introduce the Embedding Disruption Patch Attack (EDPA), a model-agnostic adversarial attack that generates patches directly placeable within the camera’s view. In comparison to prior methods, EDPA can be readily applied to different VLA models without requiring prior knowledge of the model architecture, or the controlled robotic manipulator. EDPA constructs these patches by (i) disrupting the semantic alignment between visual and textual latent representations, and (ii) maximizing the discrepancy of latent representations between adversarial and corresponding clean visual inputs. Through the optimization of these objectives, EDPA distorts the VLA’s interpretation of visual information, causing the model to repeatedly generate incorrect actions and ultimately result in failure to complete the given robotic task. To counter this, we propose an adversarial fine-tuning scheme for the visual encoder, in which the encoder is optimized to produce similar latent representations for both clean and adversarially perturbed visual inputs. Extensive evaluations on the widely recognized LIBERO robotic simulation benchmark demonstrate that EDPA substantially increases the task failure rate of cutting-edge VLA models, while our proposed defense effectively mitigates this degradation. The codebase is accessible via the homepage at https://edpa-attack.github.io/.

Method: Built upon SynDrone and FlyAware datasets, FlyAwareV2 adds multimodal data (RGB, depth, semantic labels) across varying weather and daytime conditions, computes depth maps for real samples using monocular depth estimation, and includes benchmarks for semantic segmentation.

Result: The dataset provides rich annotations and environmental diversity, enabling research on UAV-based 3D urban scene understanding and synthetic-to-real domain adaptation studies.

Conclusion: FlyAwareV2 serves as a valuable resource for advancing UAV computer vision research, particularly for urban scene understanding tasks and domain adaptation studies.

Abstract: The development of computer vision algorithms for Unmanned Aerial Vehicle (UAV) applications in urban environments heavily relies on the availability of large-scale datasets with accurate annotations. However, collecting and annotating real-world UAV data is extremely challenging and costly. To address this limitation, we present FlyAwareV2, a novel multimodal dataset encompassing both real and synthetic UAV imagery tailored for urban scene understanding tasks. Building upon the recently introduced SynDrone and FlyAware datasets, FlyAwareV2 introduces several new key contributions: 1) Multimodal data (RGB, depth, semantic labels) across diverse environmental conditions including varying weather and daytime; 2) Depth maps for real samples computed via state-of-the-art monocular depth estimation; 3) Benchmarks for RGB and multimodal semantic segmentation on standard architectures; 4) Studies on synthetic-to-real domain adaptation to assess the generalization capabilities of models trained on the synthetic data. With its rich set of annotations and environmental diversity, FlyAwareV2 provides a valuable resource for research on UAV-based 3D urban scene understanding.

Method: Proposed Cylinder Mamba Diffusion (CymbaDiff) that imposes structured spatial ordering, captures cylindrical continuity and vertical hierarchy, and preserves physical neighborhood relationships and global context.

Result: Extensive experiments on SketchSem3D demonstrate that CymbaDiff achieves superior semantic consistency, spatial realism, and cross-dataset generalization.

Conclusion: SketchSem3D enables standardized, rigorous evaluations for 3D outdoor semantic scene generation, with CymbaDiff significantly enhancing spatial coherence in generated scenes.

Abstract: Outdoor 3D semantic scene generation produces realistic and semantically rich environments for applications such as urban simulation and autonomous driving. However, advances in this direction are constrained by the absence of publicly available, well-annotated datasets. We introduce SketchSem3D, the first large-scale benchmark for generating 3D outdoor semantic scenes from abstract freehand sketches and pseudo-labeled annotations of satellite images. SketchSem3D includes two subsets, Sketch-based SemanticKITTI and Sketch-based KITTI-360 (containing LiDAR voxels along with their corresponding sketches and annotated satellite images), to enable standardized, rigorous, and diverse evaluations. We also propose Cylinder Mamba Diffusion (CymbaDiff) that significantly enhances spatial coherence in outdoor 3D scene generation. CymbaDiff imposes structured spatial ordering, explicitly captures cylindrical continuity and vertical hierarchy, and preserves both physical neighborhood relationships and global context within the generated scenes. Extensive experiments on SketchSem3D demonstrate that CymbaDiff achieves superior semantic consistency, spatial realism, and cross-dataset generalization. The code and dataset will be available at https://github.com/Lillian-research-hub/CymbaDiff

Method: Uses a stem-encoder-decoder structure with large convolution kernels in stem network to enlarge receptive field, conditional channel weighting and multi-branch local fusion block in encoder for multi-scale feature merging with low computation, and feature pyramid networks on top to address incomplete fusion problems.

Result: Achieves 381.7 FPS on NVIDIA GTX 1080Ti and 71.9 FPS on NVIDIA Jetson TX1, making it the fastest model while maintaining competitive accuracy on three benchmarks.

Conclusion: The proposed network is suitable for super real-time crowd counting on embedded systems, offering the fastest inference speed while ensuring competitive accuracy.

Abstract: Crowd counting is a task of estimating the number of the crowd through images, which is extremely valuable in the fields of intelligent security, urban planning, public safety management, and so on. However, the existing counting methods have some problems in practical application on embedded systems for these fields, such as excessive model parameters, abundant complex calculations, etc. The practical application of embedded systems requires the model to be real-time, which means that the model is fast enough. Considering the aforementioned problems, we design a super real-time model with a stem-encoder-decoder structure for crowd counting tasks, which achieves the fastest inference compared with state-of-the-arts. Firstly, large convolution kernels in the stem network are used to enlarge the receptive field, which effectively extracts detailed head information. Then, in the encoder part, we use conditional channel weighting and multi-branch local fusion block to merge multi-scale features with low computational consumption. This part is crucial to the super real-time performance of the model. Finally, the feature pyramid networks are added to the top of the encoder to alleviate its incomplete fusion problems. Experiments on three benchmarks show that our network is suitable for super real-time crowd counting on embedded systems, ensuring competitive accuracy. At the same time, the proposed network reasoning speed is the fastest. Specifically, the proposed network achieves 381.7 FPS on NVIDIA GTX 1080Ti and 71.9 FPS on NVIDIA Jetson TX1.

Method: Used mechanistic interpretability techniques to analyze information flow patterns across different layers in VideoLLMs during VideoQA tasks.

Result: Identified consistent patterns: (1) early cross-frame interactions, (2) middle-layer video-language integration via alignment with temporal concepts, (3) answer generation in middle-to-late layers. Showed models retain performance with 58% attention edge pruning.

Conclusion: The findings provide a blueprint for how VideoLLMs perform temporal reasoning and offer practical insights for improving model interpretability and downstream generalization.

Abstract: Video Large Language Models (VideoLLMs) extend the capabilities of vision-language models to spatiotemporal inputs, enabling tasks such as video question answering (VideoQA). Despite recent advances in VideoLLMs, their internal mechanisms on where and how they extract and propagate video and textual information remain less explored. In this study, we investigate the internal information flow of VideoLLMs using mechanistic interpretability techniques. Our analysis reveals consistent patterns across diverse VideoQA tasks: (1) temporal reasoning in VideoLLMs initiates with active cross-frame interactions in early-to-middle layers, (2) followed by progressive video-language integration in middle layers. This is facilitated by alignment between video representations and linguistic embeddings containing temporal concepts. (3) Upon completion of this integration, the model is ready to generate correct answers in middle-to-late layers. (4) Based on our analysis, we show that VideoLLMs can retain their VideoQA performance by selecting these effective information pathways while suppressing a substantial amount of attention edges, e.g., 58% in LLaVA-NeXT-7B-Video-FT. These findings provide a blueprint on how VideoLLMs perform temporal reasoning and offer practical insights for improving model interpretability and downstream generalization. Our project page with the source code is available at https://map-the-flow.github.io

Method: Uses Mamba-based multi-step selection diffusion model with unified variational autoencoder for both encoding and decoding across modalities.

Result: Significantly outperforms existing end-to-end models and competes effectively with SOTA models like GPT-4V, Gemini Pro, and Mistral in image generation, captioning, VQA, and text tasks.

Conclusion: MDM effectively unifies multi-modal processing while maintaining computational efficiency, establishing a new direction for end-to-end multi-modal architectures.

Abstract: Current end-to-end multi-modal models utilize different encoders and decoders to process input and output information. This separation hinders the joint representation learning of various modalities. To unify multi-modal processing, we propose a novel architecture called MDM (Multi-modal Diffusion Mamba). MDM utilizes a Mamba-based multi-step selection diffusion model to progressively generate and refine modality-specific information through a unified variational autoencoder for both encoding and decoding. This innovative approach allows MDM to achieve superior performance when processing high-dimensional data, particularly in generating high-resolution images and extended text sequences simultaneously. Our evaluations in areas such as image generation, image captioning, visual question answering, text comprehension, and reasoning tasks demonstrate that MDM significantly outperforms existing end-to-end models (MonoFormer, LlamaGen, and Chameleon etc.) and competes effectively with SOTA models like GPT-4V, Gemini Pro, and Mistral. Our results validate MDM’s effectiveness in unifying multi-modal processes while maintaining computational efficiency, establishing a new direction for end-to-end multi-modal architectures.

Method: Created MMLongCite benchmark with 8 distinct tasks spanning 6 context length intervals, incorporating diverse modalities including text, images, and videos to comprehensively evaluate LVLMs’ long-context faithfulness.

Result: Evaluation of state-of-the-art LVLMs revealed limited faithfulness in handling long multimodal contexts, with performance affected by context length and position of crucial content.

Conclusion: Extended context windows don’t guarantee effective context utilization in LVLMs, highlighting the need for improved architectures and training methods to handle long multimodal contexts faithfully.

Abstract: The rapid advancement of large vision language models (LVLMs) has led to a significant expansion of their context windows. However, an extended context window does not guarantee the effective utilization of the context, posing a critical challenge for real-world applications. Current evaluations of such long-context faithfulness are predominantly focused on the text-only domain, while multimodal assessments remain limited to short contexts. To bridge this gap, we introduce MMLongCite, a comprehensive benchmark designed to evaluate the fidelity of LVLMs in long-context scenarios. MMLongCite comprises 8 distinct tasks spanning 6 context length intervals and incorporates diverse modalities, including text, images, and videos. Our evaluation of state-of-the-art LVLMs reveals their limited faithfulness in handling long multimodal contexts. Furthermore, we provide an in-depth analysis of how context length and the position of crucial content affect the faithfulness of these models.

Method: Uses an encoder with two decoders: one for degradation type classification and another for high-quality image reconstruction from masked low-quality inputs, combining masked image modeling and contrastive learning.

Result: Achieves minimum 3.77 dB PSNR increase in 5D all-in-one restoration and 34.8% PIQE reduction in real-world scenarios, with strong generalization to unseen degradation types.

Conclusion: MaskDCPT provides an effective pre-training framework for universal image restoration that works well for both CNNs and Transformers, and the UIR-2.5M dataset enables comprehensive restoration research.

Abstract: This study introduces a Masked Degradation Classification Pre-Training method (MaskDCPT), designed to facilitate the classification of degradation types in input images, leading to comprehensive image restoration pre-training. Unlike conventional pre-training methods, MaskDCPT uses the degradation type of the image as an extremely weak supervision, while simultaneously leveraging the image reconstruction to enhance performance and robustness. MaskDCPT includes an encoder and two decoders: the encoder extracts features from the masked low-quality input image. The classification decoder uses these features to identify the degradation type, whereas the reconstruction decoder aims to reconstruct a corresponding high-quality image. This design allows the pre-training to benefit from both masked image modeling and contrastive learning, resulting in a generalized representation suited for restoration tasks. Benefit from the straightforward yet potent MaskDCPT, the pre-trained encoder can be used to address universal image restoration and achieve outstanding performance. Implementing MaskDCPT significantly improves performance for both convolution neural networks (CNNs) and Transformers, with a minimum increase in PSNR of 3.77 dB in the 5D all-in-one restoration task and a 34.8% reduction in PIQE compared to baseline in real-world degradation scenarios. It also emergences strong generalization to previously unseen degradation types and levels. In addition, we curate and release the UIR-2.5M dataset, which includes 2.5 million paired restoration samples across 19 degradation types and over 200 degradation levels, incorporating both synthetic and real-world data. The dataset, source code, and models are available at https://github.com/MILab-PKU/MaskDCPT.

Method: Four-stage pipeline: image capture/preprocessing, text detection using DBNet++, text classification using BART, all integrated in a Python PyQt5 user interface.

Result: Achieved 92.88% text detection accuracy on Total-Text dataset with challenging high-resolution images, demonstrating effectiveness for practical document categorization.

Conclusion: The proposed approach is effective for practical, mixed-source document categorization in unconstrained imaging scenarios.

Abstract: An automatic document classification system is presented that detects textual content in images and classifies documents into four predefined categories (Invoice, Report, Letter, and Form). The system supports both offline images (e.g., files on flash drives, HDDs, microSD) and real-time capture via connected cameras, and is designed to mitigate practical challenges such as variable illumination, arbitrary orientation, curved or partially occluded text, low resolution, and distant text. The pipeline comprises four stages: image capture and preprocessing, text detection [1] using a DBNet++ (Differentiable Binarization Network Plus) detector, and text classification [2] using a BART (Bidirectional and Auto-Regressive Transformers) classifier, all integrated within a user interface implemented in Python with PyQt5. The achieved results by the system for text detection in images were good at about 92.88% through 10 hours on Total-Text dataset that involve high resolution images simulate a various and very difficult challenges. The results indicate the proposed approach is effective for practical, mixed-source document categorization in unconstrained imaging scenarios.

Method: The method introduces a structural causal model (SCM) to formalize 3D-NCD, including: 1) analyzing hidden confounders in base class representations, 2) developing causal representation prototypes that eliminate confounders, and 3) using graph structures to model causal relationships between base and novel class prototypes for causal reasoning.

Result: Extensive experiments and visualization results on 3D and 2D NCD semantic segmentation demonstrate the superior performance of the proposed method.

Conclusion: The paper concludes that imposing causal relationships as strong constraints enables learning essential point cloud representations that accurately correspond to classes, successfully addressing the 3D-NCD problem through joint learning of causal representation and reasoning.

Abstract: In this paper, we focus on Novel Class Discovery for Point Cloud Segmentation (3D-NCD), aiming to learn a model that can segment unlabeled (novel) 3D classes using only the supervision from labeled (base) 3D classes. The key to this task is to setup the exact correlations between the point representations and their base class labels, as well as the representation correlations between the points from base and novel classes. A coarse or statistical correlation learning may lead to the confusion in novel class inference. lf we impose a causal relationship as a strong correlated constraint upon the learning process, the essential point cloud representations that accurately correspond to the classes should be uncovered. To this end, we introduce a structural causal model (SCM) to re-formalize the 3D-NCD problem and propose a new method, i.e., Joint Learning of Causal Representation and Reasoning. Specifically, we first analyze hidden confounders in the base class representations and the causal relationships between the base and novel classes through SCM. We devise a causal representation prototype that eliminates confounders to capture the causal representations of base classes. A graph structure is then used to model the causal relationships between the base classes’ causal representation prototypes and the novel class prototypes, enabling causal reasoning from base to novel classes. Extensive experiments and visualization results on 3D and 2D NCD semantic segmentation demonstrate the superiorities of our method.

Method: Extends sparse-aware bundle adjustment techniques to accelerate both bundle adjustment and global positioning within a unified GPU-accelerated SfM framework.

Result: Achieves up to 40x speedup over COLMAP while maintaining comparable or improved reconstruction accuracy, successfully handling datasets with 5000+ images where other methods fail.

Conclusion: GPU parallel computation can effectively accelerate SfM pipelines, enabling efficient large-scale 3D reconstruction without sacrificing accuracy.

Abstract: Structure-from-Motion (SfM), a method that recovers camera poses and scene geometry from uncalibrated images, is a central component in robotic reconstruction and simulation. Despite the state-of-the-art performance of traditional SfM methods such as COLMAP and its follow-up work, GLOMAP, naive CPU-specialized implementations of bundle adjustment (BA) or global positioning (GP) introduce significant computational overhead when handling large-scale scenarios, leading to a trade-off between accuracy and speed in SfM. Moreover, the blessing of efficient C++-based implementations in COLMAP and GLOMAP comes with the curse of limited flexibility, as they lack support for various external optimization options. On the other hand, while deep learning based SfM pipelines like VGGSfM and VGGT enable feed-forward 3D reconstruction, they are unable to scale to thousands of input views at once as GPU memory consumption increases sharply as the number of input views grows. In this paper, we unleash the full potential of GPU parallel computation to accelerate each critical stage of the standard SfM pipeline. Building upon recent advances in sparse-aware bundle adjustment optimization, our design extends these techniques to accelerate both BA and GP within a unified global SfM framework. Through extensive experiments on datasets of varying scales (e.g. 5000 images where VGGSfM and VGGT run out of memory), our method demonstrates up to about 40 times speedup over COLMAP while achieving consistently comparable or even improved reconstruction accuracy. Our project page can be found at https://cre185.github.io/InstantSfM/.

Method: Two key innovations: 1) Self-augmentation prompting that aligns semantics between query and visual augmentation using model’s intrinsic knowledge, 2) Adaptive thresholding algorithm that adjusts next token candidate size based on output sparsity using full logit distribution.

Result: Extensive experiments across four LVLMs and seven benchmarks show significant improvement in factual consistency compared to state-of-the-art decoding methods.

Conclusion: Integrating query-dependent augmentation and entropy-aware decoding is crucial for improving effective generation in LVLMs.

Abstract: Large Vision-Language Models (LVLMs) have demonstrated remarkable multimodal capabilities, but they inherit the tendency to hallucinate from their underlying language models. While visual contrastive decoding has been proposed to mitigate this issue, existing methods often apply generic visual augmentations that disregard the specific context provided by the text query, limiting their effectiveness. This study introduces a novel training-free decoding strategy that addresses these limitations, featuring two key contributions. First, a self-augmentation prompting strategy that leverages the intrinsic knowledge of the model to dynamically align semantics between the query and the visual augmentation. Second, an adaptive thresholding algorithm that adaptively adjusts next token candidate size based on the output sparsity, utilizing full information from the logit distribution. Extensive experiments across four LVLMs and seven benchmarks demonstrate that the proposed decoding significantly enhances factual consistency compared to state-of-the-art decoding methods. This work highlights the importance of integrating query-dependent augmentation and entropy-aware decoding for improving effective generation of LVLMs.

Method: The researchers conduct comparative analysis between human assessments and GPT-4o’s predictions of visual interestingness. They use this partial alignment to effectively label image pairs according to interestingness, which then serves as training data to distill knowledge into a learning-to-rank model.

Result: The studies reveal partial alignment between humans and GPT-4o, with GPT-4o capturing the concept of visual interestingness better than state-of-the-art methods. This enables effective labeling of image pairs for training purposes.

Conclusion: The insights gained from this research pave the way for a deeper understanding of human interest and demonstrate the potential of using LMMs like GPT-4o to capture and model visual interestingness concepts.

Abstract: Our daily life is highly influenced by what we consume and see. Attracting and holding one’s attention – the definition of (visual) interestingness – is essential. The rise of Large Multimodal Models (LMMs) trained on large-scale visual and textual data has demonstrated impressive capabilities. We explore these models’ potential to understand to what extent the concepts of visual interestingness are captured and examine the alignment between human assessments and GPT-4o’s, a leading LMM, predictions through comparative analysis. Our studies reveal partial alignment between humans and GPT-4o. It already captures the concept as best compared to state-of-the-art methods. Hence, this allows for the effective labeling of image pairs according to their (commonly) interestingness, which are used as training data to distill the knowledge into a learning-to-rank model. The insights pave the way for a deeper understanding of human interest.

Method: Introduce a training strategy that removes cost volumes once other network parts are sufficiently trained, creating three models for different compute budgets.

Result: Most accurate model achieves state-of-the-art accuracy with 1.2× faster inference and 6× lower memory footprint; fastest model processes Full HD at 20 FPS using only 500MB GPU memory.

Conclusion: Cost volumes can be effectively removed from optical flow estimators through proper training, enabling significant improvements in speed and memory efficiency without sacrificing accuracy.

Abstract: Cost volumes are used in every modern optical flow estimator, but due to their computational and space complexity, they are often a limiting factor regarding both processing speed and the resolution of input frames. Motivated by our empirical observation that cost volumes lose their importance once all other network parts of, e.g., a RAFT-based pipeline have been sufficiently trained, we introduce a training strategy that allows removing the cost volume from optical flow estimators throughout training. This leads to significantly improved inference speed and reduced memory requirements. Using our training strategy, we create three different models covering different compute budgets. Our most accurate model reaches state-of-the-art accuracy while being $1.2\times$ faster and having a $6\times$ lower memory footprint than comparable models; our fastest model is capable of processing Full HD frames at $20,\mathrm{FPS}$ using only $500,\mathrm{MB}$ of GPU memory.

Method: Enhanced YOLOv8 with deformable convolutions at SPPF layer and backbone/neck layers to extract multi-scale features; used focal loss to handle class imbalance; created TICW dataset with diverse concealed weapons and scenarios.

Result: The proposed DEF-YOLO architecture effectively adapts to thermal homogeneous regions and maintains speed/throughput while establishing a new benchmark for concealed weapon detection in thermal imagery.

Conclusion: The work presents a novel approach and first large-scale dataset for thermal imaging-based concealed weapon detection, achieving effective performance through architectural enhancements and addressing class imbalance.

Abstract: Concealed weapon detection aims at detecting weapons hidden beneath a person’s clothing or luggage. Various imaging modalities like Millimeter Wave, Microwave, Terahertz, Infrared, etc., are exploited for the concealed weapon detection task. These imaging modalities have their own limitations, such as poor resolution in microwave imaging, privacy concerns in millimeter wave imaging, etc. To provide a real-time, 24 x 7 surveillance, low-cost, and privacy-preserved solution, we opted for thermal imaging in spite of the lack of availability of a benchmark dataset. We propose a novel approach and a dataset for concealed weapon detection in thermal imagery. Our YOLO-based architecture, DEF-YOLO, is built with key enhancements in YOLOv8 tailored to the unique challenges of concealed weapon detection in thermal vision. We adopt deformable convolutions at the SPPF layer to exploit multi-scale features; backbone and neck layers to extract low, mid, and high-level features, enabling DEF-YOLO to adaptively focus on localization around the objects in thermal homogeneous regions, without sacrificing much of the speed and throughput. In addition to these simple yet effective key architectural changes, we introduce a new, large-scale Thermal Imaging Concealed Weapon dataset, TICW, featuring a diverse set of concealed weapons and capturing a wide range of scenarios. To the best of our knowledge, this is the first large-scale contributed dataset for this task. We also incorporate focal loss to address the significant class imbalance inherent in the concealed weapon detection task. The efficacy of the proposed work establishes a new benchmark through extensive experimentation for concealed weapon detection in thermal imagery.

Method: Group-wise optimization where codebook is divided into groups optimized independently with joint optimization within groups, plus training-free codebook resampling for post-training size adjustment.

Result: Improved reconstruction metrics in image experiments across various settings, achieving better trade-off between codebook utilization and reconstruction performance with flexible codebook size adjustment.

Conclusion: Group-VQ effectively addresses VQ-VAE limitations through group-wise optimization and resampling, enhancing both reconstruction quality and codebook flexibility without training overhead.

Abstract: Vector Quantized Variational Autoencoders (VQ-VAEs) leverage self-supervised learning through reconstruction tasks to represent continuous vectors using the closest vectors in a codebook. However, issues such as codebook collapse persist in the VQ model. To address these issues, existing approaches employ implicit static codebooks or jointly optimize the entire codebook, but these methods constrain the codebook’s learning capability, leading to reduced reconstruction quality. In this paper, we propose Group-VQ, which performs group-wise optimization on the codebook. Each group is optimized independently, with joint optimization performed within groups. This approach improves the trade-off between codebook utilization and reconstruction performance. Additionally, we introduce a training-free codebook resampling method, allowing post-training adjustment of the codebook size. In image reconstruction experiments under various settings, Group-VQ demonstrates improved performance on reconstruction metrics. And the post-training codebook sampling method achieves the desired flexibility in adjusting the codebook size.

Method: Created a large rendering-oriented video dataset with subjective quality annotations across various 3D scenes and rendering settings, then designed a NR-VQA metric that assesses both image quality and temporal stability.

Result: The proposed metric outperforms existing NR-VQA metrics on rendered videos and can effectively benchmark supersampling methods and frame generation strategies.

Conclusion: The work provides a specialized solution for rendered video quality assessment that better reflects real-world applications in gaming, VR, and AR.

Abstract: Quality assessment of videos is crucial for many computer graphics applications, including video games, virtual reality, and augmented reality, where visual performance has a significant impact on user experience. When test videos cannot be perfectly aligned with references or when references are unavailable, the significance of no-reference video quality assessment (NR-VQA) methods is undeniable. However, existing NR-VQA datasets and metrics are primarily focused on camera-captured videos; applying them directly to rendered videos would result in biased predictions, as rendered videos are more prone to temporal artifacts. To address this, we present a large rendering-oriented video dataset with subjective quality annotations, as well as a designed NR-VQA metric specific to rendered videos. The proposed dataset includes a wide range of 3D scenes and rendering settings, with quality scores annotated for various display types to better reflect real-world application scenarios. Building on this dataset, we calibrate our NR-VQA metric to assess rendered video quality by looking at both image quality and temporal stability. We compare our metric to existing NR-VQA metrics, demonstrating its superior performance on rendered videos. Finally, we demonstrate that our metric can be used to benchmark supersampling methods and assess frame generation strategies in real-time rendering.

Method: Evaluated modern VLMs (OpenCLIP, MetaCLIP 2, SigLip) on a 285-image COCO-derived dataset using a three-tiered prompt design that systematically increases linguistic detail for sitting, standing, and walking/running classification.

Result: Highest-performing models (MetaCLIP 2 and OpenCLIP) achieved best results with simplest prompts (68.8% accuracy), while adding detail degraded performance significantly (down to 55.1%). Lower-performing SigLip model improved with more descriptive, body-cue-based prompts.

Conclusion: Prompt specificity has model-dependent effects: top models suffer from “prompt overfitting” with detailed prompts, while weaker models benefit from increased descriptive detail for ambiguous classes.

Abstract: Recent Vision-Language Models (VLMs) enable zero-shot classification by aligning images and text in a shared space, a promising approach for data-scarce conditions. However, the influence of prompt design on recognizing visually similar categories, such as human postures, is not well understood. This study investigates how prompt specificity affects the zero-shot classification of sitting, standing, and walking/running on a small, 285-image COCO-derived dataset. A suite of modern VLMs, including OpenCLIP, MetaCLIP 2, and SigLip, were evaluated using a three-tiered prompt design that systematically increases linguistic detail. Our findings reveal a compelling, counter-intuitive trend: for the highest-performing models (MetaCLIP 2 and OpenCLIP), the simplest, most basic prompts consistently achieve the best results. Adding descriptive detail significantly degrades performance for instance, MetaCLIP 2’s multi-class accuracy drops from 68.8% to 55.1% a phenomenon we term “prompt overfitting”. Conversely, the lower-performing SigLip model shows improved classification on ambiguous classes when given more descriptive, body-cue-based prompts.

Method: DepthVLA uses a mixture-of-transformers design that unifies a VLM, a depth transformer, and an action expert with fully shared attentions, forming an end-to-end model with enhanced spatial reasoning through explicit depth prediction.

Result: DepthVLA outperforms state-of-the-art approaches across multiple environments: 78.5% vs. 65.0% progress in real-world tasks, 94.9% vs. 93.6% in LIBERO simulator, and 74.8% vs. 58.8% in Simpler simulator.

Conclusion: Explicitly incorporating spatial awareness through depth prediction significantly improves VLA performance on tasks requiring precise spatial reasoning, demonstrating the effectiveness of the proposed architecture.

Abstract: Vision-Language-Action (VLA) models have recently shown impressive generalization and language-guided manipulation capabilities. However, their performance degrades on tasks requiring precise spatial reasoning due to limited spatial reasoning inherited from Vision-Language Models (VLMs). Existing VLAs rely on extensive action-data pretraining to ground VLMs in 3D space, which reduces training efficiency and is still insufficient for accurate spatial understanding. In this work, we present DepthVLA, a simple yet effective VLA architecture that explicitly incorporates spatial awareness through a pretrained depth prediction module. DepthVLA adopts a mixture-of-transformers design that unifies a VLM, a depth transformer, and an action expert with fully shared attentions, forming an end-to-end model with enhanced spatial reasoning. Extensive evaluations in both real-world and simulated environments show that DepthVLA outperforms state-of-the-art approaches, achieving 78.5% vs. 65.0% progress in real-world tasks, 94.9% vs. 93.6% in the LIBERO simulator, and 74.8% vs. 58.8% in the Simpler simulator. Our code will be made publicly available.

Method: Integrates SDFs with 3DGS using a unified optimization framework that combines 2D depth and point tracking priors with SDF representations for dynamic objects, enhancing geometric accuracy and deformation modeling.

Result: Achieves state-of-the-art rendering performance without LiDAR data on urban scenes, and further improves with LiDAR for reconstruction and novel view generation across diverse object categories without ground-truth 3D motion annotation. Enables scene editing tasks like decomposition and composition.

Conclusion: The proposed SDF-3DGS integration provides a more robust and adaptable object representation that reduces dependency on expensive data requirements while maintaining high performance in dynamic scene rendering and reconstruction.

Abstract: Dynamic scene rendering and reconstruction play a crucial role in computer vision and augmented reality. Recent methods based on 3D Gaussian Splatting (3DGS), have enabled accurate modeling of dynamic urban scenes, but for urban scenes they require both camera and LiDAR data, ground-truth 3D segmentations and motion data in the form of tracklets or pre-defined object templates such as SMPL. In this work, we explore whether a combination of 2D object agnostic priors in the form of depth and point tracking coupled with a signed distance function (SDF) representation for dynamic objects can be used to relax some of these requirements. We present a novel approach that integrates Signed Distance Functions (SDFs) with 3D Gaussian Splatting (3DGS) to create a more robust object representation by harnessing the strengths of both methods. Our unified optimization framework enhances the geometric accuracy of 3D Gaussian splatting and improves deformation modeling within the SDF, resulting in a more adaptable and precise representation. We demonstrate that our method achieves state-of-the-art performance in rendering metrics even without LiDAR data on urban scenes. When incorporating LiDAR, our approach improved further in reconstructing and generating novel views across diverse object categories, without ground-truth 3D motion annotation. Additionally, our method enables various scene editing tasks, including scene decomposition, and scene composition.

Method: Uses dual-path CSI encoding (CSI-Ratio phase sequences and Doppler spectrograms), Multiscale Semantic Encoder with cross-modal attention, Semantic-Aware Soft Supervision for inter-class correlations, and Robust Dual-Distillation to compress model into lightweight student network.

Result: Outperforms state-of-the-art methods on Widar3.0 benchmark in both in-domain and cross-domain gesture recognition, while significantly reducing model size and inference latency.

Conclusion: Provides a scalable and deployable solution for generalized RF-based gesture interfaces in real-world AIoT applications.

Abstract: WiFi-based gesture recognition has emerged as a promising RF sensing paradigm for enabling non-contact and privacy-preserving human-computer interaction in AIoT environments. However, existing methods often suffer from limited generalization and semantic expressiveness due to the domain-sensitive nature of Channel State Information and the lack of high-level gesture abstraction. To address these challenges, we propose a novel generalization framework, termed Large-Model-Aware Semantic Distillation and Alignment (GLSDA), which leverages the semantic prior of pre-trained large foundation models to enhance gesture representation learning in both in-domain and cross-domain scenarios. Specifically, we first design a dual-path CSI encoding pipeline that captures geometric and dynamic gesture patterns via CSI-Ratio phase sequences and Doppler spectrograms. These representations are then fed into a Multiscale Semantic Encoder, which learns robust temporal embeddings and aligns them with gesture semantics through cross-modal attention mechanisms. To further enhance category discrimination, we introduce a Semantic-Aware Soft Supervision scheme that encodes inter-class correlations and reduces label ambiguity, especially for semantically similar gestures. Finally, we develop a Robust Dual-Distillation strategy to compress the aligned model into a lightweight student network, jointly distilling intermediate features and semantic-informed soft labels from the teacher model. Extensive experiments on the Widar3.0 benchmark show that GLSDA consistently outperforms state-of-the-art methods in both in-domain and cross-domain gesture recognition tasks, while significantly reducing model size and inference latency. Our method offers a scalable and deployable solution for generalized RF-based gesture interfaces in real-world AIoT applications.

Method: Developed a unified benchmark based on a cognitively grounded taxonomy with four quadrants: Intrinsic-Static, Intrinsic-Dynamic, Extrinsic-Static, and Extrinsic-Dynamic spatial reasoning. Created a scalable automated pipeline to generate diverse and verifiable spatial reasoning questions.

Result: Generated Spatial-DISE dataset with 559 evaluation VQA pairs and 12K+ training pairs. Evaluation of 28 state-of-the-art VLMs revealed large gaps to human competence, especially on multi-step multi-view spatial reasoning.

Conclusion: Spatial-DISE provides a robust framework, valuable dataset, and clear direction for future research toward achieving human-like spatial intelligence in VLMs.

Abstract: Spatial reasoning ability is crucial for Vision Language Models (VLMs) to support real-world applications in diverse domains including robotics, augmented reality, and autonomous navigation. Unfortunately, existing benchmarks are inadequate in assessing spatial reasoning ability, especially the \emph{intrinsic-dynamic} spatial reasoning which is a fundamental aspect of human spatial cognition. In this paper, we propose a unified benchmark, \textbf{Spatial-DISE}, based on a cognitively grounded taxonomy that categorizes tasks into four fundamental quadrants: \textbf{I}ntrinsic-\textbf{S}tatic, Intrinsic-\textbf{D}ynamic, \textbf{E}xtrinsic-Static, and Extrinsic-Dynamic spatial reasoning. Moreover, to address the issue of data scarcity, we develop a scalable and automated pipeline to generate diverse and verifiable spatial reasoning questions, resulting in a new \textbf{Spatial-DISE} dataset that includes Spatial-DISE Bench (559 evaluation VQA pairs) and Spatial-DISE-12K (12K+ training VQA pairs). Our comprehensive evaluation across 28 state-of-the-art VLMs reveals that, current VLMs have a large and consistent gap to human competence, especially on multi-step multi-view spatial reasoning. Spatial-DISE offers a robust framework, valuable dataset, and clear direction for future research toward human-like spatial intelligence. Benchmark, dataset, and code will be publicly released.

Method: Proposes Mask-GRPO which incorporates Group Relative Policy Optimization into masked generative models by redefining transition probability and formulating unmasking as multi-step decision-making, with additional strategies like removing KL constraint and filtering low-quality samples.

Result: Substantially improves base model Show-o on standard T2I benchmarks and preference alignment, outperforming existing state-of-the-art approaches.

Conclusion: Mask-GRPO successfully demonstrates the effectiveness of RL for masked generative models in text-to-image generation, filling an important gap in existing approaches.

Abstract: Reinforcement learning (RL) has garnered increasing attention in text-to-image (T2I) generation. However, most existing RL approaches are tailored to either diffusion models or autoregressive models, overlooking an important alternative: masked generative models. In this work, we propose Mask-GRPO, the first method to incorporate Group Relative Policy Optimization (GRPO)-based RL into this overlooked paradigm. Our core insight is to redefine the transition probability, which is different from current approaches, and formulate the unmasking process as a multi-step decision-making problem. To further enhance our method, we explore several useful strategies, including removing the KL constraint, applying the reduction strategy, and filtering out low-quality samples. Using Mask-GRPO, we improve a base model, Show-o, with substantial improvements on standard T2I benchmarks and preference alignment, outperforming existing state-of-the-art approaches. The code is available on https://github.com/xingzhejun/Mask-GRPO

Method: Uses a two-stage adapter architecture: (1) Dual Context Adapter for global structural consistency at reduced resolutions, and (2) Reference Patch Adapter with patch-level attention mechanism for full-resolution inpainting with local detail fidelity.

Result: Achieves state-of-the-art performance on OpenImages and Photo-Concept-Bucket datasets, resolving artifacts in large-scale inpainting and outperforming existing methods in perceptual quality and text-prompt adherence.

Conclusion: Patch-Adapter effectively addresses the scalability gap in high-resolution inpainting by decoupling global semantics from localized refinement, enabling 4K+ resolution inpainting without structural overhauls.

Abstract: In this work, we present Patch-Adapter, an effective framework for high-resolution text-guided image inpainting. Unlike existing methods limited to lower resolutions, our approach achieves 4K+ resolution while maintaining precise content consistency and prompt alignment, two critical challenges in image inpainting that intensify with increasing resolution and texture complexity. Patch-Adapter leverages a two-stage adapter architecture to scale the diffusion model’s resolution from 1K to 4K+ without requiring structural overhauls: (1) Dual Context Adapter learns coherence between masked and unmasked regions at reduced resolutions to establish global structural consistency; and (2) Reference Patch Adapter implements a patch-level attention mechanism for full-resolution inpainting, preserving local detail fidelity through adaptive feature fusion. This dual-stage architecture uniquely addresses the scalability gap in high-resolution inpainting by decoupling global semantics from localized refinement. Experiments demonstrate that Patch-Adapter not only resolves artifacts common in large-scale inpainting but also achieves state-of-the-art performance on the OpenImages and Photo-Concept-Bucket datasets, outperforming existing methods in both perceptual quality and text-prompt adherence.

Method: Uses Lightweight Spatial-Channel Resizer (LSCR) for spatial and channel alignment, Distribution Alignment via Domain Separation (DADS) with encoder-specific and encoder-agnostic modules, and Domain Alignment Mutual Information (DAMI) loss for feature alignment. Employs fully convolutional architecture for efficiency.

Result: Extensive experiments show CoDS effectively mitigates feature discrepancies in heterogeneous scenarios and achieves a trade-off between detection accuracy and inference efficiency.

Conclusion: CoDS successfully addresses feature discrepancies in heterogeneous collaborative perception through domain separation and lightweight alignment, providing both effective feature alignment and computational efficiency.

Abstract: Collaborative perception has been proven to improve individual perception in autonomous driving through multi-agent interaction. Nevertheless, most methods often assume identical encoders for all agents, which does not hold true when these models are deployed in real-world applications. To realize collaborative perception in actual heterogeneous scenarios, existing methods usually align neighbor features to those of the ego vehicle, which is vulnerable to noise from domain gaps and thus fails to address feature discrepancies effectively. Moreover, they adopt transformer-based modules for domain adaptation, which causes the model inference inefficiency on mobile devices. To tackle these issues, we propose CoDS, a Collaborative perception method that leverages Domain Separation to address feature discrepancies in heterogeneous scenarios. The CoDS employs two feature alignment modules, i.e., Lightweight Spatial-Channel Resizer (LSCR) and Distribution Alignment via Domain Separation (DADS). Besides, it utilizes the Domain Alignment Mutual Information (DAMI) loss to ensure effective feature alignment. Specifically, the LSCR aligns the neighbor feature across spatial and channel dimensions using a lightweight convolutional layer. Subsequently, the DADS mitigates feature distribution discrepancy with encoder-specific and encoder-agnostic domain separation modules. The former removes domain-dependent information and the latter captures task-related information. During training, the DAMI loss maximizes the mutual information between aligned heterogeneous features to enhance the domain separation process. The CoDS employs a fully convolutional architecture, which ensures high inference efficiency. Extensive experiments demonstrate that the CoDS effectively mitigates feature discrepancies in heterogeneous scenarios and achieves a trade-off between detection accuracy and inference efficiency.

Method: Introduced a differentiable rendering pipeline that optimizes deformable meshes directly in 3D. Parameterizes mesh deformations with radial basis functions and learns offsets and scales that maximize neuronal responses while enforcing geometric regularity.

Result: Applied to models of monkey area V4, the approach enables probing neuronal selectivity to interpretable 3D factors such as pose and lighting.

Conclusion: This approach bridges inverse graphics with systems neuroscience, offering a way to probe neural selectivity with physically grounded, 3D stimuli beyond conventional pixel-based methods.

Abstract: Visual perception relies on inference of 3D scene properties such as shape, pose, and lighting. To understand how visual sensory neurons enable robust perception, it is crucial to characterize their selectivity to such physically interpretable factors. However, current approaches mainly operate on 2D pixels, making it difficult to isolate selectivity for physical scene properties. To address this limitation, we introduce a differentiable rendering pipeline that optimizes deformable meshes to obtain MEIs directly in 3D. The method parameterizes mesh deformations with radial basis functions and learns offsets and scales that maximize neuronal responses while enforcing geometric regularity. Applied to models of monkey area V4, our approach enables probing neuronal selectivity to interpretable 3D factors such as pose and lighting. This approach bridges inverse graphics with systems neuroscience, offering a way to probe neural selectivity with physically grounded, 3D stimuli beyond conventional pixel-based methods.

Method: Uses MLLM-as-a-Judge mechanism to assess semantic alignment and generate soft matching scores; constructs hard negative sets through global retrieval; aligns similarity matrix with soft semantic scores; includes UniME-V2-Reranker with joint pairwise and listwise optimization.

Result: Achieves state-of-the-art performance on MMEB benchmark and multiple retrieval tasks across all tasks on average.

Conclusion: The proposed approach effectively enhances representation learning by leveraging MLLMs for semantic assessment and hard negative mining, significantly improving discriminative capacity in multimodal embeddings.

Abstract: Universal multimodal embedding models are foundational to various tasks. Existing approaches typically employ in-batch negative mining by measuring the similarity of query-candidate pairs. However, these methods often struggle to capture subtle semantic differences among candidates and lack diversity in negative samples. Moreover, the embeddings exhibit limited discriminative ability in distinguishing false and hard negatives. In this paper, we leverage the advanced understanding capabilities of MLLMs to enhance representation learning and present a novel Universal Multimodal Embedding (UniME-V2) model. Our approach first constructs a potential hard negative set through global retrieval. We then introduce the MLLM-as-a-Judge mechanism, which utilizes MLLMs to assess the semantic alignment of query-candidate pairs and generate soft semantic matching scores. These scores serve as a foundation for hard negative mining, mitigating the impact of false negatives and enabling the identification of diverse, high-quality hard negatives. Furthermore, the semantic matching scores are used as soft labels to mitigate the rigid one-to-one mapping constraint. By aligning the similarity matrix with the soft semantic matching score matrix, the model learns semantic distinctions among candidates, significantly enhancing its discriminative capacity. To further improve performance, we propose UniME-V2-Reranker, a reranking model trained on our mined hard negatives through a joint pairwise and listwise optimization approach. We conduct comprehensive experiments on the MMEB benchmark and multiple retrieval tasks, demonstrating that our method achieves state-of-the-art performance on average across all tasks.

Method: Used convolutional neural networks (CNNs) and partial least squares (PLS) for analysis. Compared joint spatio-spectral CNN analysis with spatial CNN and spectral PLS. Developed CNN with spectral convolution layer and created Python packages for fast PLS modeling and cross-validation.

Result: CNN with spectral convolution outperformed PLS for modeling parameters requiring both chemical and visual information. PLS performed equally well for mean chemical content analysis. CNN generated better chemical maps than PLS. Barley germination modeling was inconclusive due to low germination dataset.

Conclusion: Joint spatio-spectral CNN analysis is superior for complex food quality parameters, while PLS remains effective for mean chemical content analysis. Developed tools enable faster PLS modeling and cross-validation.

Abstract: This thesis investigates the application of near-infrared hyperspectral imaging (NIR-HSI) for food quality analysis. The investigation is conducted through four studies operating with five research hypotheses. For several analyses, the studies compare models based on convolutional neural networks (CNNs) and partial least squares (PLS). Generally, joint spatio-spectral analysis with CNNs outperforms spatial analysis with CNNs and spectral analysis with PLS when modeling parameters where chemical and physical visual information are relevant. When modeling chemical parameters with a 2-dimensional (2D) CNN, augmenting the CNN with an initial layer dedicated to performing spectral convolution enhances its predictive performance by learning a spectral preprocessing similar to that applied by domain experts. Still, PLS-based spectral modeling performs equally well for analysis of the mean content of chemical parameters in samples and is the recommended approach. Modeling the spatial distribution of chemical parameters with NIR-HSI is limited by the ability to obtain spatially resolved reference values. Therefore, a study used bulk mean references for chemical map generation of fat content in pork bellies. A PLS-based approach gave non-smooth chemical maps and pixel-wise predictions outside the range of 0-100%. Conversely, a 2D CNN augmented with a spectral convolution layer mitigated all issues arising with PLS. The final study attempted to model barley’s germinative capacity by analyzing NIR spectra, RGB images, and NIR-HSI images. However, the results were inconclusive due to the dataset’s low degree of germination. Additionally, this thesis has led to the development of two open-sourced Python packages. The first facilitates fast PLS-based modeling, while the second facilitates very fast cross-validation of PLS and other classical machine learning models with a new algorithm.

Method: Uses model stitching to connect text-to-video generators with 3D decoders at compatible layers, followed by direct reward finetuning to align the generator with the 3D decoder for consistent geometry.

Result: VIST3A significantly improves over prior text-to-3D models that output Gaussian splats and enables high-quality text-to-pointmap generation when using suitable 3D base models.

Conclusion: The framework successfully combines text-to-video generation and 3D reconstruction capabilities through model stitching and alignment, achieving superior text-to-3D generation results.

Abstract: The rapid progress of large, pretrained models for both visual content generation and 3D reconstruction opens up new possibilities for text-to-3D generation. Intuitively, one could obtain a formidable 3D scene generator if one were able to combine the power of a modern latent text-to-video model as “generator” with the geometric abilities of a recent (feedforward) 3D reconstruction system as “decoder”. We introduce VIST3A, a general framework that does just that, addressing two main challenges. First, the two components must be joined in a way that preserves the rich knowledge encoded in their weights. We revisit model stitching, i.e., we identify the layer in the 3D decoder that best matches the latent representation produced by the text-to-video generator and stitch the two parts together. That operation requires only a small dataset and no labels. Second, the text-to-video generator must be aligned with the stitched 3D decoder, to ensure that the generated latents are decodable into consistent, perceptually convincing 3D scene geometry. To that end, we adapt direct reward finetuning, a popular technique for human preference alignment. We evaluate the proposed VIST3A approach with different video generators and 3D reconstruction models. All tested pairings markedly improve over prior text-to-3D models that output Gaussian splats. Moreover, by choosing a suitable 3D base model, VIST3A also enables high-quality text-to-pointmap generation.

Method: Three training-free uncertainty metrics: Similarity Distribution (SD) measures score separation between candidates; Ratio Spread (RS) evaluates competitive ambiguity among top-scoring locations; Statistical Uncertainty (SU) combines SD and RS as a unified metric. All operate without additional training, architectural changes, or geometric verification.

Result: Comprehensive evaluation across 9 state-of-the-art VPR methods and 6 benchmark datasets shows the metrics excel at discriminating correct/incorrect matches, outperform existing approaches, maintain negligible computational overhead, and improve precision-recall performance.

Conclusion: The proposed uncertainty metrics provide robust confidence estimation for VPR systems, generalize across datasets and methods without validation data, and are deployable for real-time robotic applications across varied environmental conditions.

Abstract: Visual Place Recognition (VPR) enables robots and autonomous vehicles to identify previously visited locations by matching current observations against a database of known places. However, VPR systems face significant challenges when deployed across varying visual environments, lighting conditions, seasonal changes, and viewpoints changes. Failure-critical VPR applications, such as loop closure detection in simultaneous localization and mapping (SLAM) pipelines, require robust estimation of place matching uncertainty. We propose three training-free uncertainty metrics that estimate prediction confidence by analyzing inherent statistical patterns in similarity scores from any existing VPR method. Similarity Distribution (SD) quantifies match distinctiveness by measuring score separation between candidates; Ratio Spread (RS) evaluates competitive ambiguity among top-scoring locations; and Statistical Uncertainty (SU) is a combination of SD and RS that provides a unified metric that generalizes across datasets and VPR methods without requiring validation data to select the optimal metric. All three metrics operate without additional model training, architectural modifications, or computationally expensive geometric verification. Comprehensive evaluation across nine state-of-the-art VPR methods and six benchmark datasets confirms that our metrics excel at discriminating between correct and incorrect VPR matches, and consistently outperform existing approaches while maintaining negligible computational overhead, making it deployable for real-time robotic applications across varied environmental conditions with improved precision-recall performance.

Method: The model uses a hybrid approach with CNN-based feature extractors for multi-scale feature extraction (global and local facial features), a Mixture of Experts module for adaptive feature selection, and a residual network backbone for deep feature learning.

Result: Achieved accuracies of 74.77% on AffectNet (v7), 72.55% on AffectNet (v8), 84.29% on RAF-DB, and 64.66% on FER-2013, outperforming current state-of-the-art methods.

Conclusion: The model demonstrates strong adaptability and practical applicability for end-to-end emotion recognition systems in real-world settings, with publicly available reproducible code.

Abstract: In many domains, including online education, healthcare, security, and human-computer interaction, facial emotion recognition (FER) is essential. Real-world FER is still difficult despite its significance because of some factors such as variable head positions, occlusions, illumination shifts, and demographic diversity. Engagement detection, which is essential for applications like virtual learning and customer services, is frequently challenging due to FER limitations by many current models. In this article, we propose ExpressNet-MoE, a novel hybrid deep learning model that blends both Convolution Neural Networks (CNNs) and Mixture of Experts (MoE) framework, to overcome the difficulties. Our model dynamically chooses the most pertinent expert networks, thus it aids in the generalization and providing flexibility to model across a wide variety of datasets. Our model improves on the accuracy of emotion recognition by utilizing multi-scale feature extraction to collect both global and local facial features. ExpressNet-MoE includes numerous CNN-based feature extractors, a MoE module for adaptive feature selection, and finally a residual network backbone for deep feature learning. To demonstrate efficacy of our proposed model we evaluated on several datasets, and compared with current state-of-the-art methods. Our model achieves accuracies of 74.77% on AffectNet (v7), 72.55% on AffectNet (v8), 84.29% on RAF-DB, and 64.66% on FER-2013. The results show how adaptive our model is and how it may be used to develop end-to-end emotion recognition systems in practical settings. Reproducible codes and results are made publicly accessible at https://github.com/DeeptimaanB/ExpressNet-MoE.

Method: Agent-based streaming benchmark with frozen CLIP features and lightweight residual adapters, testing monocultural/mixed populations on a 5x5 lattice with sigma-scheduled Gaussian blur.

Result: JAFFE (Asian) populations maintain higher performance at low blur but drop sharply at intermediate stages, while KDEF (Western) populations degrade more uniformly. Mixed populations show intermediate patterns.

Conclusion: Cultural composition and interaction structure significantly influence FER robustness under deteriorating perceptual conditions, with imbalanced settings amplifying majority-group weaknesses.

Abstract: Facial expression recognition (FER) must remain robust under both cultural variation and perceptually degraded visual conditions, yet most existing evaluations assume homogeneous data and high-quality imagery. We introduce an agent-based, streaming benchmark that reveals how cross-cultural composition and progressive blurring interact to shape face recognition robustness. Each agent operates in a frozen CLIP feature space with a lightweight residual adapter trained online at sigma=0 and fixed during testing. Agents move and interact on a 5x5 lattice, while the environment provides inputs with sigma-scheduled Gaussian blur. We examine monocultural populations (Western-only, Asian-only) and mixed environments with balanced (5/5) and imbalanced (8/2, 2/8) compositions, as well as different spatial contact structures. Results show clear asymmetric degradation curves between cultural groups: JAFFE (Asian) populations maintain higher performance at low blur but exhibit sharper drops at intermediate stages, whereas KDEF (Western) populations degrade more uniformly. Mixed populations exhibit intermediate patterns, with balanced mixtures mitigating early degradation, but imbalanced settings amplify majority-group weaknesses under high blur. These findings quantify how cultural composition and interaction structure influence the robustness of FER as perceptual conditions deteriorate.

Method: Use Action Units describing facial muscle movements as non-transient, semantic features instead of features from shallow or deep convolutional neural networks for incremental learning of complex emotions.

Result: Achieves 0.75 accuracy on CFEE dataset for incremental learning of complex compound emotions, comparable to state-of-the-art results, with lightweight model and small memory footprint.

Conclusion: Action Units are effective non-transient features that outperform CNN-based features for incremental emotion recognition, enabling successful learning of complex emotions while avoiding catastrophic forgetting.

Abstract: Incremental learning is a complex process due to potential catastrophic forgetting of old tasks when learning new ones. This is mainly due to transient features that do not fit from task to task. In this paper, we focus on complex emotion recognition. First, we learn basic emotions and then, incrementally, like humans, complex emotions. We show that Action Units, describing facial muscle movements, are non-transient, highly semantical features that outperform those extracted by both shallow and deep convolutional neural networks. Thanks to this ability, our approach achieves interesting results when learning incrementally complex, compound emotions with an accuracy of 0.75 on the CFEE dataset and can be favorably compared to state-of-the-art results. Moreover, it results in a lightweight model with a small memory footprint.

Method: Combines Structure-from-motion pipeline with a localized implicit neural representation model (liRBSM) that decomposes breast domain into multiple regions with local neural SDFs anchored at anatomical landmarks.

Result: Achieves high-quality 3D breast geometry reconstruction with less than 2 mm error margin, outperforming the global iRBSM model in detail and reconstruction quality.

Conclusion: The proposed pipeline is fast (under 6 minutes), transparent, open-source, and provides accurate breast reconstruction accessible to anyone with an RGB video recording device.

Abstract: We present a neural parametric 3D breast shape model and, based on this model, introduce a low-cost and accessible 3D surface reconstruction pipeline capable of recovering accurate breast geometry from a monocular RGB video. In contrast to widely used, commercially available yet prohibitively expensive 3D breast scanning solutions and existing low-cost alternatives, our method requires neither specialized hardware nor proprietary software and can be used with any device that is able to record RGB videos. The key building blocks of our pipeline are a state-of-the-art, off-the-shelf Structure-from-motion pipeline, paired with a parametric breast model for robust and metrically correct surface reconstruction. Our model, similarly to the recently proposed implicit Regensburg Breast Shape Model (iRBSM), leverages implicit neural representations to model breast shapes. However, unlike the iRBSM, which employs a single global neural signed distance function (SDF), our approach – inspired by recent state-of-the-art face models – decomposes the implicit breast domain into multiple smaller regions, each represented by a local neural SDF anchored at anatomical landmark positions. When incorporated into our surface reconstruction pipeline, the proposed model, dubbed liRBSM (short for localized iRBSM), significantly outperforms the iRBSM in terms of reconstruction quality, yielding more detailed surface reconstruction than its global counterpart. Overall, we find that the introduced pipeline is able to recover high-quality 3D breast geometry within an error margin of less than 2 mm. Our method is fast (requires less than six minutes), fully transparent and open-source, and – together with the model – publicly available at https://rbsm.re-mic.de/local-implicit.

Method: Comparative study of hardware-accelerated feature detectors (FAST, Harris, SuperPoint) in Visual SLAM pipeline, evaluating GPU implementations on Nvidia Jetson Orin against FPGA implementations on AMD Versal SoCs.

Result: For traditional detectors (FAST, Harris), GPU implementations outperform FPGAs in runtime and energy efficiency. For learning-based SuperPoint, FPGA achieves 3.1× speedup and 1.4× energy efficiency over GPU. FPGA-accelerated V-SLAM achieves comparable performance to GPU-accelerated V-SLAM in 2 out of 5 sequences, though GPU generally provides better accuracy.

Conclusion: Hardware acceleration choice depends on detector type: GPUs better for traditional detectors, FPGAs better for learning-based detectors. Hardware acceleration enables less frequent bundle adjustment calls without sacrificing accuracy, improving overall V-SLAM performance.

Abstract: Feature detection is a common yet time-consuming module in Simultaneous Localization and Mapping (SLAM) implementations, which are increasingly deployed on power-constrained platforms, such as drones. Graphics Processing Units (GPUs) have been a popular accelerator for computer vision in general, and feature detection and SLAM in particular. On the other hand, System-on-Chips (SoCs) with integrated Field Programmable Gate Array (FPGA) are also widely available. This paper presents the first study of hardware-accelerated feature detectors considering a Visual SLAM (V-SLAM) pipeline. We offer new insights by comparing the best GPU-accelerated FAST, Harris, and SuperPoint implementations against the FPGA-accelerated counterparts on modern SoCs (Nvidia Jetson Orin and AMD Versal). The evaluation shows that when using a non-learning-based feature detector such as FAST and Harris, their GPU implementations, and the GPU-accelerated V-SLAM can achieve better run-time performance and energy efficiency than the FAST and Harris FPGA implementations as well as the FPGA-accelerated V-SLAM. However, when considering a learning-based detector such as SuperPoint, its FPGA implementation can achieve better run-time performance and energy efficiency (up to 3.1$\times$ and 1.4$\times$ improvements, respectively) than the GPU implementation. The FPGA-accelerated V-SLAM can also achieve comparable run-time performance compared to the GPU-accelerated V-SLAM, with better FPS in 2 out of 5 dataset sequences. When considering the accuracy, the results show that the GPU-accelerated V-SLAM is more accurate than the FPGA-accelerated V-SLAM in general. Last but not least, the use of hardware acceleration for feature detection could further improve the performance of the V-SLAM pipeline by having the global bundle adjustment module invoked less frequently without sacrificing accuracy.

Method: Proposes XD-RCDepth with two knowledge-distillation strategies: explainability-aligned distillation that transfers teacher’s saliency structure to student, and depth-distribution distillation that recasts depth regression as soft classification over discretized bins.

Result: Reduces parameters by 29.7% relative to state-of-the-art lightweight baseline while maintaining comparable accuracy, reduces MAE by 7.97% compared to direct training, and achieves competitive accuracy with real-time efficiency on nuScenes and ZJU-4DRadarCam datasets.

Conclusion: The proposed lightweight architecture with knowledge distillation strategies effectively preserves performance under compression while enhancing interpretability, making it suitable for real-time autonomous driving applications.

Abstract: Depth estimation remains central to autonomous driving, and radar-camera fusion offers robustness in adverse conditions by providing complementary geometric cues. In this paper, we present XD-RCDepth, a lightweight architecture that reduces the parameters by 29.7% relative to the state-of-the-art lightweight baseline while maintaining comparable accuracy. To preserve performance under compression and enhance interpretability, we introduce two knowledge-distillation strategies: an explainability-aligned distillation that transfers the teacher’s saliency structure to the student, and a depth-distribution distillation that recasts depth regression as soft classification over discretized bins. Together, these components reduce the MAE compared with direct training with 7.97% and deliver competitive accuracy with real-time efficiency on nuScenes and ZJU-4DRadarCam datasets.

Method: Proposed PCDF (prompt-guided condition-aware dynamic fusion) that encodes imaging conditions as text prompts and uses a task-specific soft-gating transformation to adaptively reassign multimodal contributions between RGB and IR modalities.

Result: Experiments on the ATR-UMOD dataset demonstrate the effectiveness of the proposed PCDF method for adaptive multimodal fusion.

Conclusion: The introduced dataset and PCDF method successfully address the challenge of diverse imaging conditions in UAV-based object detection, enabling more robust performance across varying scenarios.

Abstract: Unmanned aerial vehicles (UAV)-based object detection with visible (RGB) and infrared (IR) images facilitates robust around-the-clock detection, driven by advancements in deep learning techniques and the availability of high-quality dataset. However, the existing dataset struggles to fully capture real-world complexity for limited imaging conditions. To this end, we introduce a high-diversity dataset ATR-UMOD covering varying scenarios, spanning altitudes from 80m to 300m, angles from 0{\deg} to 75{\deg}, and all-day, all-year time variations in rich weather and illumination conditions. Moreover, each RGB-IR image pair is annotated with 6 condition attributes, offering valuable high-level contextual information. To meet the challenge raised by such diverse conditions, we propose a novel prompt-guided condition-aware dynamic fusion (PCDF) to adaptively reassign multimodal contributions by leveraging annotated condition cues. By encoding imaging conditions as text prompts, PCDF effectively models the relationship between conditions and multimodal contributions through a task-specific soft-gating transformation. A prompt-guided condition-decoupling module further ensures the availability in practice without condition annotations. Experiments on ATR-UMOD dataset reveal the effectiveness of PCDF.

Method: AVAR-Net uses four modules: audio feature extractor (Wav2Vec2), video feature extractor (MobileViT), early fusion strategy, and Multi-Stage Temporal Convolutional Network (MTCN) for learning long-range temporal dependencies and cross-modal relationships.

Result: AVAR-Net achieves 89.29% accuracy on the new VAAR dataset and 88.56% Average Precision on XD-Violence dataset, improving Average Precision by 2.8% over existing state-of-the-art methods.

Conclusion: The framework demonstrates effectiveness, efficiency, and generalization capability. The introduced VAAR dataset serves as a valuable benchmark for advancing multimodal anomaly recognition research.

Abstract: Anomaly recognition plays a vital role in surveillance, transportation, healthcare, and public safety. However, most existing approaches rely solely on visual data, making them unreliable under challenging conditions such as occlusion, low illumination, and adverse weather. Moreover, the absence of large-scale synchronized audio-visual datasets has hindered progress in multimodal anomaly recognition. To address these limitations, this study presents AVAR-Net, a lightweight and efficient audio-visual anomaly recognition framework designed for real-world environments. AVAR-Net consists of four main modules: an audio feature extractor, a video feature extractor, fusion strategy, and a sequential pattern learning network that models cross-modal relationships for anomaly recognition. Specifically, the Wav2Vec2 model extracts robust temporal features from raw audio, while MobileViT captures both local and global visual representations from video frames. An early fusion mechanism combines these modalities, and a Multi-Stage Temporal Convolutional Network (MTCN) model that learns long-range temporal dependencies within the fused representation, enabling robust spatiotemporal reasoning. A novel Visual-Audio Anomaly Recognition (VAAR) dataset, is also introduced, serving as a medium-scale benchmark containing 3,000 real-world videos with synchronized audio across ten diverse anomaly classes. Experimental evaluations demonstrate that AVAR-Net achieves 89.29% accuracy on VAAR and 88.56% Average Precision on the XD-Violence dataset, improving Average Precision by 2.8% over existing state-of-the-art methods. These results highlight the effectiveness, efficiency, and generalization capability of the proposed framework, as well as the utility of VAAR as a benchmark for advancing multimodal anomaly recognition research.

Method: Systematic literature review following PRISMA approach, analyzing 39 peer-reviewed studies on medical image enhancement across various imaging modalities (X-ray, CT, MRI, ultrasound).

Result: Conventional mathematical methods dominate (29 studies), deep learning techniques are used in 9 studies, and hybrid approaches in 1 study. MRI and multi-modal imaging receive most attention, while histopathology, endoscopy, and bone scintigraphy remain underexplored. 65 image quality assessment metrics were identified, predominantly non-reference-based.

Conclusion: The review identifies current limitations and research gaps, highlighting the need for more exploration of deep learning and hybrid methods, as well as specialized imaging modalities. It provides direction for future advancements in medical image enhancement.

Abstract: Medical image enhancement is crucial for improving the quality and interpretability of diagnostic images, ultimately supporting early detection, accurate diagnosis, and effective treatment planning. Despite advancements in imaging technologies such as X-ray, CT, MRI, and ultrasound, medical images often suffer from challenges like noise, artifacts, and low contrast, which limit their diagnostic potential. Addressing these challenges requires robust preprocessing, denoising algorithms, and advanced enhancement methods, with deep learning techniques playing an increasingly significant role. This systematic literature review, following the PRISMA approach, investigates the key challenges, recent advancements, and evaluation metrics in medical image enhancement. By analyzing findings from 39 peer-reviewed studies, this review provides insights into the effectiveness of various enhancement methods across different imaging modalities and the importance of evaluation metrics in assessing their impact. Key issues like low contrast and noise are identified as the most frequent, with MRI and multi-modal imaging receiving the most attention, while specialized modalities such as histopathology, endoscopy, and bone scintigraphy remain underexplored. Out of the 39 studies, 29 utilize conventional mathematical methods, 9 focus on deep learning techniques, and 1 explores a hybrid approach. In terms of image quality assessment, 18 studies employ both reference-based and non-reference-based metrics, 9 rely solely on reference-based metrics, and 12 use only non-reference-based metrics, with a total of 65 IQA metrics introduced, predominantly non-reference-based. This review highlights current limitations, research gaps, and potential future directions for advancing medical image enhancement.

Method: Proposes CanvasMAR with a canvas mechanism that provides blurred global predictions of next frames as starting points for masked generation, along with compositional classifier-free guidance and noise-based canvas augmentation.

Result: Achieves high-quality video generation with fewer autoregressive steps on BAIR and Kinetics-600 benchmarks, rivaling diffusion-based methods and showing remarkable performance among autoregressive models on Kinetics-600.

Conclusion: The canvas mechanism effectively mitigates slow-start and error accumulation issues in video MAR models, enabling more efficient and coherent video generation that competes with state-of-the-art approaches.

Abstract: Masked autoregressive models (MAR) have recently emerged as a powerful paradigm for image and video generation, combining the flexibility of masked modeling with the potential of continuous tokenizer. However, video MAR models suffer from two major limitations: the slow-start problem, caused by the lack of a structured global prior at early sampling stages, and error accumulation across the autoregression in both spatial and temporal dimensions. In this work, we propose CanvasMAR, a novel video MAR model that mitigates these issues by introducing a canvas mechanism–a blurred, global prediction of the next frame, used as the starting point for masked generation. The canvas provides global structure early in sampling, enabling faster and more coherent frame synthesis. Furthermore, we introduce compositional classifier-free guidance that jointly enlarges spatial (canvas) and temporal conditioning, and employ noise-based canvas augmentation to enhance robustness. Experiments on the BAIR and Kinetics-600 benchmarks demonstrate that CanvasMAR produces high-quality videos with fewer autoregressive steps. Our approach achieves remarkable performance among autoregressive models on Kinetics-600 dataset and rivals diffusion-based methods.

Method: Built on AnomalyDINO, used white-box gradient attacks with a lightweight linear head on frozen DINOv2 features, and applied post-hoc Platt scaling for uncertainty calibration.

Result: Adversarial perturbations consistently degrade performance metrics (F1, AUROC, AP, G-mean), and raw anomaly scores are poorly calibrated. Platt scaling significantly improves uncertainty estimation and enables attack detection.

Conclusion: DINOv2-based anomaly detectors have concrete vulnerabilities requiring adversarial robustness and principled uncertainty quantification for trustworthy real-world deployment.

Abstract: Foundation models such as DINOv2 have shown strong performance in few-shot anomaly detection, yet two key questions remain unexamined: (i) how susceptible are these detectors to adversarial perturbations; and (ii) how well do their anomaly scores reflect calibrated uncertainty? Building on AnomalyDINO, a training-free deep nearest-neighbor detector over DINOv2 features, we present one of the first systematic studies of adversarial attacks and uncertainty estimation in this setting. To enable white-box gradient attacks while preserving test-time behavior, we attach a lightweight linear head to frozen DINOv2 features only for crafting perturbations. Using this heuristic, we evaluate the impact of FGSM across the MVTec-AD and VisA datasets and observe consistent drops in F1, AUROC, AP, and G-mean, indicating that imperceptible perturbations can flip nearest-neighbor relations in feature space to induce confident misclassification. Complementing robustness, we probe reliability and find that raw anomaly scores are poorly calibrated, revealing a gap between confidence and correctness that limits safety-critical use. As a simple, strong baseline toward trustworthiness, we apply post-hoc Platt scaling to the anomaly scores for uncertainty estimation. The resulting calibrated posteriors yield significantly higher predictive entropy on adversarially perturbed inputs than on clean ones, enabling a practical flagging mechanism for attack detection while reducing calibration error (ECE). Our findings surface concrete vulnerabilities in DINOv2-based few-shot anomaly detectors and establish an evaluation protocol and baseline for robust, uncertainty-aware anomaly detection. We argue that adversarial robustness and principled uncertainty quantification are not optional add-ons but essential capabilities if anomaly detection systems are to be trustworthy and ready for real-world deployment.

Method: Uses feature-field representation to learn subject identity and geometry, enhanced text-to-video diffusion backbone with dense spatio-temporal attention, depth-aware feature rendering for geometric consistency, and consistent-aware latent completion for perspective alignment.

Result: Extensive experiments show MVCustom is the only framework that simultaneously achieves faithful multi-view generation and customization.

Conclusion: MVCustom successfully addresses the multi-view customization task by maintaining both multi-view consistency and customization fidelity through novel training and inference techniques.

Abstract: Multi-view generation with camera pose control and prompt-based customization are both essential elements for achieving controllable generative models. However, existing multi-view generation models do not support customization with geometric consistency, whereas customization models lack explicit viewpoint control, making them challenging to unify. Motivated by these gaps, we introduce a novel task, multi-view customization, which aims to jointly achieve multi-view camera pose control and customization. Due to the scarcity of training data in customization, existing multi-view generation models, which inherently rely on large-scale datasets, struggle to generalize to diverse prompts. To address this, we propose MVCustom, a novel diffusion-based framework explicitly designed to achieve both multi-view consistency and customization fidelity. In the training stage, MVCustom learns the subject’s identity and geometry using a feature-field representation, incorporating the text-to-video diffusion backbone enhanced with dense spatio-temporal attention, which leverages temporal coherence for multi-view consistency. In the inference stage, we introduce two novel techniques: depth-aware feature rendering explicitly enforces geometric consistency, and consistent-aware latent completion ensures accurate perspective alignment of the customized subject and surrounding backgrounds. Extensive experiments demonstrate that MVCustom is the only framework that simultaneously achieves faithful multi-view generation and customization.

Method: Proposes Local-Global Context-Aware Attention to maintain pixel relationships, and a distribution- and perceptual-aligned conditioning mechanism in pixel space that preserves structural information from local details to global composition.

Result: Extensive experiments on multiple super-resolution benchmarks show the method effectively produces high-fidelity, perceptually accurate reconstructions while mitigating artifacts.

Conclusion: The framework successfully addresses limitations of existing approaches by maintaining structural consistency and realistic detail restoration in super-resolution tasks.

Abstract: Diffusion models have recently achieved significant success in various image manipulation tasks, including image super-resolution and perceptual quality enhancement. Pretrained text-to-image models, such as Stable Diffusion, have exhibited strong capabilities in synthesizing realistic image content, which makes them particularly attractive for addressing super-resolution tasks. While some existing approaches leverage these models to achieve state-of-the-art results, they often struggle when applied to diverse and highly degraded images, leading to noise amplification or incorrect content generation. To address these limitations, we propose a contextually precise image super-resolution framework that effectively maintains both local and global pixel relationships through Local-Global Context-Aware Attention, enabling the generation of high-quality images. Furthermore, we propose a distribution- and perceptual-aligned conditioning mechanism in the pixel space to enhance perceptual fidelity. This mechanism captures fine-grained pixel-level representations while progressively preserving and refining structural information, transitioning from local content details to the global structural composition. During inference, our method generates high-quality images that are structurally consistent with the original content, mitigating artifacts and ensuring realistic detail restoration. Extensive experiments on multiple super-resolution benchmarks demonstrate the effectiveness of our approach in producing high-fidelity, perceptually accurate reconstructions.

Method: Proposes EditCast3D pipeline that propagates edits via video generation models, uses view selection strategy for consistency, and employs feedforward reconstruction without costly refinement.

Result: Superior editing quality and high efficiency compared to state-of-the-art 3D editing baselines on commonly used datasets.

Conclusion: EditCast3D establishes a scalable and general paradigm for integrating foundation models into 3D editing pipelines.

Abstract: Recent advances in foundation models have driven remarkable progress in image editing, yet their extension to 3D editing remains underexplored. A natural approach is to replace the image editing modules in existing workflows with foundation models. However, their heavy computational demands and the restrictions and costs of closed-source APIs make plugging these models into existing iterative editing strategies impractical. To address this limitation, we propose EditCast3D, a pipeline that employs video generation foundation models to propagate edits from a single first frame across the entire dataset prior to reconstruction. While editing propagation enables dataset-level editing via video models, its consistency remains suboptimal for 3D reconstruction, where multi-view alignment is essential. To overcome this, EditCast3D introduces a view selection strategy that explicitly identifies consistent and reconstruction-friendly views and adopts feedforward reconstruction without requiring costly refinement. In combination, the pipeline both minimizes reliance on expensive image editing and mitigates prompt ambiguities that arise when applying foundation models independently across images. We evaluate EditCast3D on commonly used 3D editing datasets and compare it against state-of-the-art 3D editing baselines, demonstrating superior editing quality and high efficiency. These results establish EditCast3D as a scalable and general paradigm for integrating foundation models into 3D editing pipelines. The code is available at https://github.com/UNITES-Lab/EditCast3D

Method: Uses pseudo-labeling with a reward model that assesses reliability by combining 3D direction vectors, visual embeddings from an off-the-shelf encoder, and semantic cues from a Multimodal Large Language Model to compute confidence scores for selecting high-quality pseudo labels.

Result: Achieves state-of-the-art performance on five datasets in both in-domain and cross-domain settings, and demonstrates robust zero-shot generalization on four unseen datasets.

Conclusion: OmniGaze effectively mitigates domain bias and generalizes gaze estimation in the wild, serving as a scalable data engine for gaze estimation.

Abstract: Current 3D gaze estimation methods struggle to generalize across diverse data domains, primarily due to i) the scarcity of annotated datasets, and ii) the insufficient diversity of labeled data. In this work, we present OmniGaze, a semi-supervised framework for 3D gaze estimation, which utilizes large-scale unlabeled data collected from diverse and unconstrained real-world environments to mitigate domain bias and generalize gaze estimation in the wild. First, we build a diverse collection of unlabeled facial images, varying in facial appearances, background environments, illumination conditions, head poses, and eye occlusions. In order to leverage unlabeled data spanning a broader distribution, OmniGaze adopts a standard pseudo-labeling strategy and devises a reward model to assess the reliability of pseudo labels. Beyond pseudo labels as 3D direction vectors, the reward model also incorporates visual embeddings extracted by an off-the-shelf visual encoder and semantic cues from gaze perspective generated by prompting a Multimodal Large Language Model to compute confidence scores. Then, these scores are utilized to select high-quality pseudo labels and weight them for loss computation. Extensive experiments demonstrate that OmniGaze achieves state-of-the-art performance on five datasets under both in-domain and cross-domain settings. Furthermore, we also evaluate the efficacy of OmniGaze as a scalable data engine for gaze estimation, which exhibits robust zero-shot generalization on four unseen datasets.

Method: Proposed LSGC for graph construction and LogViG, a hybrid CNN-GNN model with high-resolution branch and multi-scale feature fusion between high and low-resolution branches.

Result: LogViG beats existing architectures in accuracy, GMACs, and parameters on image classification and semantic segmentation. Ti-LogViG achieves 79.9% top-1 accuracy on ImageNet-1K, 1.7% higher than Vision GNN with 24.3% parameter reduction and 35.3% GMACs reduction.

Conclusion: Leveraging long-range links in graph construction through LSGC can exceed current state-of-the-art ViG performance, demonstrating the effectiveness of the proposed approach.

Abstract: Vision graph neural networks (ViG) have demonstrated promise in vision tasks as a competitive alternative to conventional convolutional neural nets (CNN) and transformers (ViTs); however, common graph construction methods, such as k-nearest neighbor (KNN), can be expensive on larger images. While methods such as Sparse Vision Graph Attention (SVGA) have shown promise, SVGA’s fixed step scale can lead to over-squashing and missing multiple connections to gain the same information that could be gained from a long-range link. Through this observation, we propose a new graph construction method, Logarithmic Scalable Graph Construction (LSGC) to enhance performance by limiting the number of long-range links. To this end, we propose LogViG, a novel hybrid CNN-GNN model that utilizes LSGC. Furthermore, inspired by the successes of multi-scale and high-resolution architectures, we introduce and apply a high-resolution branch and fuse features between our high-resolution and low-resolution branches for a multi-scale high-resolution Vision GNN network. Extensive experiments show that LogViG beats existing ViG, CNN, and ViT architectures in terms of accuracy, GMACs, and parameters on image classification and semantic segmentation tasks. Our smallest model, Ti-LogViG, achieves an average top-1 accuracy on ImageNet-1K of 79.9% with a standard deviation of 0.2%, 1.7% higher average accuracy than Vision GNN with a 24.3% reduction in parameters and 35.3% reduction in GMACs. Our work shows that leveraging long-range links in graph construction for ViGs through our proposed LSGC can exceed the performance of current state-of-the-art ViGs. Code is available at https://github.com/mmunir127/LogViG-Official.

Method: Built ViVerBench benchmark with 16 categories of visual verification tasks. Designed automated pipelines to construct large-scale visual verification data and trained OmniVerifier-7B. Proposed OmniVerifier-TTS, a sequential test-time scaling paradigm for iterative fine-grained optimization.

Result: OmniVerifier-7B achieves +8.3 gain on ViVerBench. OmniVerifier-TTS achieves improvements on T2I-ReasonBench(+3.7) and GenEval++(+4.3), outperforming existing parallel test-time scaling methods like Best-of-N.

Conclusion: The Generative Universal Verifier advances reliable reflection during generation and scalable test-time refinement, marking progress toward more trustworthy and controllable next-generation reasoning systems.

Abstract: We introduce Generative Universal Verifier, a novel concept and plugin designed for next-generation multimodal reasoning in vision-language models and unified multimodal models, providing the fundamental capability of reflection and refinement on visual outcomes during the reasoning and generation process. This work makes three main contributions: (1) We build ViVerBench, a comprehensive benchmark spanning 16 categories of critical tasks for evaluating visual outcomes in multimodal reasoning. Results show that existing VLMs consistently underperform across these tasks, underscoring a substantial gap from human-level capability in reliable visual verification. (2) We design two automated pipelines to construct large-scale visual verification data and train OmniVerifier-7B, the first omni-capable generative verifier trained for universal visual verification and achieves notable gains on ViVerBench(+8.3). Through training, we identify three atomic capabilities in visual verification and demonstrate how they generalize and interact synergistically. (3) We propose OmniVerifier-TTS, a sequential test-time scaling paradigm that leverages the universal verifier to bridge image generation and editing within unified models, enhancing the upper bound of generative ability through iterative fine-grained optimization. Beyond generation, we extend universal verifier to broader world-modeling interleaved reasoning scenarios. Empirically, OmniVerifier-TTS achieves improvements on T2I-ReasonBench(+3.7), and GenEval++(+4.3), outperforming existing parallel test-time scaling methods, such as Best-of-N. By endowing multimodal reasoning with reliable visual verification, OmniVerifier advances both reliable reflection during generation and scalable test-time refinement, marking a step toward more trustworthy and controllable next-generation reasoning systems.

Method: Comprehensive review and evaluation of proposed solutions submitted to the NTIRE 2025 LLIE Challenge, analyzing state-of-the-art advancements in low-light image enhancement.

Result: 28 teams submitted valid entries from 762 registered participants, showcasing significant progress in low-light image enhancement techniques.

Conclusion: The challenge demonstrated substantial advancements in low-light image enhancement, with multiple effective solutions emerging from the competition.

Abstract: This paper presents a comprehensive review of the NTIRE 2025 Low-Light Image Enhancement (LLIE) Challenge, highlighting the proposed solutions and final outcomes. The objective of the challenge is to identify effective networks capable of producing brighter, clearer, and visually compelling images under diverse and challenging conditions. A remarkable total of 762 participants registered for the competition, with 28 teams ultimately submitting valid entries. This paper thoroughly evaluates the state-of-the-art advancements in LLIE, showcasing the significant progress.

Method: RECODE generates multiple candidate programs to reproduce input images, uses a critic to select the most faithful reconstruction, and iteratively refines the code through an agentic framework.

Result: RECODE significantly outperforms methods that don’t use code or only use code for auxiliary purposes on benchmarks like CharXiv, ChartQA, and Geometry3K.

Conclusion: Grounding visual perception in executable code provides a new path toward more accurate and verifiable multimodal reasoning by transforming ambiguous perceptual tasks into verifiable symbolic problems.

Abstract: Multimodal Large Language Models (MLLMs) struggle with precise reasoning for structured visuals like charts and diagrams, as pixel-based perception lacks a mechanism for verification. To address this, we propose to leverage derendering – the process of reverse-engineering visuals into executable code – as a new modality for verifiable visual reasoning. Specifically, we propose RECODE, an agentic framework that first generates multiple candidate programs to reproduce the input image. It then uses a critic to select the most faithful reconstruction and iteratively refines the code. This process not only transforms an ambiguous perceptual task into a verifiable, symbolic problem, but also enables precise calculations and logical inferences later on. On various visual reasoning benchmarks such as CharXiv, ChartQA, and Geometry3K, RECODE significantly outperforms methods that do not leverage code or only use code for drawing auxiliary lines or cropping. Our work demonstrates that grounding visual perception in executable code provides a new path toward more accurate and verifiable multimodal reasoning.

Method: Knowledge-guided Contrastive Learning (KnowCoL) framework that integrates images and text descriptions into a shared semantic space using structured information from Wikidata, leveraging entity descriptions, type hierarchies, and relational context.

Result: The approach significantly improves accuracy on the OVEN benchmark, with the smallest model achieving 10.5% higher accuracy on unseen entities compared to state-of-the-art methods while being 35 times smaller.

Conclusion: Combining visual, textual, and structured knowledge greatly enhances open-domain visual entity recognition, particularly for rare and unseen entities, demonstrating the effectiveness of knowledge-guided contrastive learning.

Abstract: Open-domain visual entity recognition aims to identify and link entities depicted in images to a vast and evolving set of real-world concepts, such as those found in Wikidata. Unlike conventional classification tasks with fixed label sets, it operates under open-set conditions, where most target entities are unseen during training and exhibit long-tail distributions. This makes the task inherently challenging due to limited supervision, high visual ambiguity, and the need for semantic disambiguation. In this work, we propose a Knowledge-guided Contrastive Learning (KnowCoL) framework that combines both images and text descriptions into a shared semantic space grounded by structured information from Wikidata. By abstracting visual and textual inputs to a conceptual level, the model leverages entity descriptions, type hierarchies, and relational context to support zero-shot entity recognition. We evaluate our approach on the OVEN benchmark, a large-scale open-domain visual recognition dataset with Wikidata IDs as the label space. Our experiments show that using visual, textual, and structured knowledge greatly improves accuracy, especially for rare and unseen entities. Our smallest model improves the accuracy on unseen entities by 10.5% compared to the state-of-the-art, despite being 35 times smaller.

Method: Transform 4D volumetric fMRI data into videos of 2D fMRI activity flat maps, then train Vision Transformers on 2.3K hours of fMRI data using spatiotemporal masked autoencoder (MAE) framework.

Result: Masked fMRI modeling performance improves with dataset size according to a strict power scaling law. The model learns rich representations supporting fine-grained state decoding across subjects and subject-specific trait decoding across brain state changes.

Conclusion: This work successfully demonstrates the feasibility of building fMRI foundation models using video representations and masked autoencoding, with ongoing open science development.

Abstract: A key question for adapting modern deep learning architectures to functional MRI (fMRI) is how to represent the data for model input. To bridge the modality gap between fMRI and natural images, we transform the 4D volumetric fMRI data into videos of 2D fMRI activity flat maps. We train Vision Transformers on 2.3K hours of fMRI flat map videos from the Human Connectome Project using the spatiotemporal masked autoencoder (MAE) framework. We observe that masked fMRI modeling performance improves with dataset size according to a strict power scaling law. Downstream classification benchmarks show that our model learns rich representations supporting both fine-grained state decoding across subjects, as well as subject-specific trait decoding across changes in brain state. This work is part of an ongoing open science project to build foundation models for fMRI data. Our code and datasets are available at https://github.com/MedARC-AI/fmri-fm.

Method: Uses a dual-mode pre-training phase followed by cross-mode post-training distillation. First pre-trains a dual-mode multi-view diffusion model supporting both MV-oriented and 3D-oriented generation, then uses cross-mode distillation to match distributions from 3D-oriented to high-quality MV-oriented mode while leveraging single-view images and text prompts for generalization.

Result: Achieves 10-100x faster generation than previous works while possessing superior rendering quality, reduces required denoising steps for inference, and demonstrates strong generalization to out-of-distribution inputs.

Conclusion: FlashWorld successfully bridges the gap between 3D consistency and visual quality through its dual-mode approach and cross-mode distillation, establishing a new efficient paradigm for 3D scene generation.

Abstract: We propose FlashWorld, a generative model that produces 3D scenes from a single image or text prompt in seconds, 10~100$\times$ faster than previous works while possessing superior rendering quality. Our approach shifts from the conventional multi-view-oriented (MV-oriented) paradigm, which generates multi-view images for subsequent 3D reconstruction, to a 3D-oriented approach where the model directly produces 3D Gaussian representations during multi-view generation. While ensuring 3D consistency, 3D-oriented method typically suffers poor visual quality. FlashWorld includes a dual-mode pre-training phase followed by a cross-mode post-training phase, effectively integrating the strengths of both paradigms. Specifically, leveraging the prior from a video diffusion model, we first pre-train a dual-mode multi-view diffusion model, which jointly supports MV-oriented and 3D-oriented generation modes. To bridge the quality gap in 3D-oriented generation, we further propose a cross-mode post-training distillation by matching distribution from consistent 3D-oriented mode to high-quality MV-oriented mode. This not only enhances visual quality while maintaining 3D consistency, but also reduces the required denoising steps for inference. Also, we propose a strategy to leverage massive single-view images and text prompts during this process to enhance the model’s generalization to out-of-distribution inputs. Extensive experiments demonstrate the superiority and efficiency of our method.

Method: The authors use denoising diffusion bridge models (DDBMs) that condition the diffusion process on both the initial healthy image and a corresponding synthetically generated pathological image, treating the pathological image as a structurally informative prior.

Result: The proposed DDBM approach outperforms previously proposed diffusion models and fully supervised approaches at segmentation and anomaly detection tasks.

Conclusion: DDBMs effectively generate counterfactuals that closely match patient anatomy while selectively removing pathology, demonstrating superior performance over existing methods.

Abstract: Generating healthy counterfactuals from pathological images holds significant promise in medical imaging, e.g., in anomaly detection or for application of analysis tools that are designed for healthy scans. These counterfactuals should represent what a patient’s scan would plausibly look like in the absence of pathology, preserving individual anatomical characteristics while modifying only the pathological regions. Denoising diffusion probabilistic models (DDPMs) have become popular methods for generating healthy counterfactuals of pathology data. Typically, this involves training on solely healthy data with the assumption that a partial denoising process will be unable to model disease regions and will instead reconstruct a closely matched healthy counterpart. More recent methods have incorporated synthetic pathological images to better guide the diffusion process. However, it remains challenging to guide the generative process in a way that effectively balances the removal of anomalies with the retention of subject-specific features. To solve this problem, we propose a novel application of denoising diffusion bridge models (DDBMs) - which, unlike DDPMs, condition the diffusion process not only on the initial point (i.e., the healthy image), but also on the final point (i.e., a corresponding synthetically generated pathological image). Treating the pathological image as a structurally informative prior enables us to generate counterfactuals that closely match the patient’s anatomy while selectively removing pathology. The results show that our DDBM outperforms previously proposed diffusion models and fully supervised approaches at segmentation and anomaly detection tasks.

Method: Proposes Risk-adaptive Activation Steering (RAS) which reformulates queries to strengthen cross-modal attention to safety-critical image regions for accurate risk assessment, then adaptively steers activations based on assessed risk.

Result: Significantly reduces attack success rates, preserves general task performance, and improves inference speed over prior inference-time defenses across multiple benchmarks.

Conclusion: RAS effectively addresses multimodal safety challenges by enabling accurate risk assessment and adaptive response generation without the overhead of iterative output adjustments.

Abstract: One of the key challenges of modern AI models is ensuring that they provide helpful responses to benign queries while refusing malicious ones. But often, the models are vulnerable to multimodal queries with harmful intent embedded in images. One approach for safety alignment is training with extensive safety datasets at the significant costs in both dataset curation and training. Inference-time alignment mitigates these costs, but introduces two drawbacks: excessive refusals from misclassified benign queries and slower inference speed due to iterative output adjustments. To overcome these limitations, we propose to reformulate queries to strengthen cross-modal attention to safety-critical image regions, enabling accurate risk assessment at the query level. Using the assessed risk, it adaptively steers activations to generate responses that are safe and helpful without overhead from iterative output adjustments. We call this Risk-adaptive Activation Steering (RAS). Extensive experiments across multiple benchmarks on multimodal safety and utility demonstrate that the RAS significantly reduces attack success rates, preserves general task performance, and improves inference speed over prior inference-time defenses.

Method: Used thinning-based skeletonization on TopCoW dataset (200 stroke patients with MRA/CTA), developed baseline algorithm combining U-Net-based skeletonization with A* graph connection for centerline extraction.

Result: Achieved high graph topology accuracy (F1 = 1), average Euclidean node distance <1 voxel, median relative errors <5% for features, Pearson correlations >0.95. Successfully predicted fetal PCA variants and detected modality differences.

Conclusion: Learning-based skeletonization with graph connection enables anatomically plausible centerline extraction. Emphasizes importance of evaluating anatomical accuracy beyond voxel-based measures. Dataset and algorithm released for further research.

Abstract: The Circle of Willis (CoW) is a critical network of arteries in the brain, often implicated in cerebrovascular pathologies. Voxel-level segmentation is an important first step toward an automated CoW assessment, but a full quantitative analysis requires centerline representations. However, conventional skeletonization techniques often struggle to extract reliable centerlines due to the CoW’s complex geometry, and publicly available centerline datasets remain scarce. To address these challenges, we used a thinning-based skeletonization algorithm to extract and curate centerline graphs and morphometric features from the TopCoW dataset, which includes 200 stroke patients, each imaged with MRA and CTA. The curated graphs were used to develop a baseline algorithm for centerline and feature extraction, combining U-Net-based skeletonization with A* graph connection. Performance was evaluated on a held-out test set, focusing on anatomical accuracy and feature robustness. Further, we used the extracted features to predict the frequency of fetal PCA variants, confirm theoretical bifurcation optimality relations, and detect subtle modality differences. The baseline algorithm consistently reconstructed graph topology with high accuracy (F1 = 1), and the average Euclidean node distance between reference and predicted graphs was below one voxel. Features such as segment radius, length, and bifurcation ratios showed strong robustness, with median relative errors below 5% and Pearson correlations above 0.95. Our results demonstrate the utility of learning-based skeletonization combined with graph connection for anatomically plausible centerline extraction. We emphasize the importance of going beyond simple voxel-based measures by evaluating anatomical accuracy and feature robustness. The dataset and baseline algorithm have been released to support further method development and clinical research.

Method: Created Honey-Data-15M dataset with multiple cleaning techniques and dual-level CoT enrichment, developed HoneyPipe data curation pipeline and DataStudio framework, trained Bee-8B model on this dataset.

Result: Bee-8B establishes new SOTA for fully open MLLMs, achieving performance competitive with and sometimes surpassing semi-open models like InternVL3.5-8B.

Conclusion: Principled focus on data quality is key to developing fully open MLLMs that are highly competitive with semi-open counterparts, providing foundational resources to the community.

Abstract: Fully open multimodal large language models (MLLMs) currently lag behind proprietary counterparts, primarily due to a significant gap in data quality for supervised fine-tuning (SFT). Existing open-source datasets are often plagued by widespread noise and a critical deficit in complex reasoning data, such as Chain-of-Thought (CoT), which hinders the development of advanced model capabilities. Addressing these challenges, our work makes three primary contributions. First, we introduce Honey-Data-15M, a new SFT dataset comprising approximately 15 million QA pairs, processed through multiple cleaning techniques and enhanced with a novel dual-level (short and long) CoT enrichment strategy. Second, we introduce HoneyPipe, the data curation pipeline, and its underlying framework DataStudio, providing the community with a transparent and adaptable methodology for data curation that moves beyond static dataset releases. Finally, to validate our dataset and pipeline, we train Bee-8B, an 8B model on Honey-Data-15M. Experiments show that Bee-8B establishes a new state-of-the-art (SOTA) for fully open MLLMs, achieving performance that is competitive with, and in some cases surpasses, recent semi-open models such as InternVL3.5-8B. Our work delivers to the community a suite of foundational resources, including: the Honey-Data-15M corpus; the full-stack suite comprising HoneyPipe and DataStudio; training recipes; an evaluation harness; and the model weights. This effort demonstrates that a principled focus on data quality is a key pathway to developing fully open MLLMs that are highly competitive with their semi-open counterparts.

Method: Two baseline registration approaches: RANSAC-based 3D transformation estimation using cross-view point clouds, and plenoptic PnP algorithm estimating extrinsic poses from single light field images, both integrating the plenoptic camera model.

Result: Experiments show strong alignment with ground truth, supporting reliable multi-view light field processing.

Conclusion: LiFMCR enables accurate and scalable multi-camera light field registration with explicit integration of plenoptic camera models.

Abstract: We present LiFMCR, a novel dataset for the registration of multiple micro lens array (MLA)-based light field cameras. While existing light field datasets are limited to single-camera setups and typically lack external ground truth, LiFMCR provides synchronized image sequences from two high-resolution Raytrix R32 plenoptic cameras, together with high-precision 6-degrees of freedom (DoF) poses recorded by a Vicon motion capture system. This unique combination enables rigorous evaluation of multi-camera light field registration methods. As a baseline, we provide two complementary registration approaches: a robust 3D transformation estimation via a RANSAC-based method using cross-view point clouds, and a plenoptic PnP algorithm estimating extrinsic 6-DoF poses from single light field images. Both explicitly integrate the plenoptic camera model, enabling accurate and scalable multi-camera registration. Experiments show strong alignment with the ground truth, supporting reliable multi-view light field processing. Project page: https://lifmcr.github.io/

Method: Cyclic self-supervised diffusion framework with cycle-consistent constraint, slice-wise gap perception network for inter-slice alignment, and local structure correction network for enhanced feature restoration.

Result: Achieved 31.80 ± 2.70 dB PSNR, 0.943 ± 0.102 SSIM, and 0.0864 ± 0.0689 LPIPS. Reduced left cerebral white matter error from 12.1% to 2.1% and cortex error from 4.2% to 3.7%.

Conclusion: CSS-Diff synthesizes images that are both quantitatively reliable and anatomically consistent, bridging the domain gap in MRI synthesis.

Abstract: Synthesizing high-quality images from low-field MRI holds significant potential. Low-field MRI is cheaper, more accessible, and safer, but suffers from low resolution and poor signal-to-noise ratio. This synthesis process can reduce reliance on costly acquisitions and expand data availability. However, synthesizing high-field MRI still suffers from a clinical fidelity gap. There is a need to preserve anatomical fidelity, enhance fine-grained structural details, and bridge domain gaps in image contrast. To address these issues, we propose a \emph{cyclic self-supervised diffusion (CSS-Diff)} framework for high-field MRI synthesis from real low-field MRI data. Our core idea is to reformulate diffusion-based synthesis under a cycle-consistent constraint. It enforces anatomical preservation throughout the generative process rather than just relying on paired pixel-level supervision. The CSS-Diff framework further incorporates two novel processes. The slice-wise gap perception network aligns inter-slice inconsistencies via contrastive learning. The local structure correction network enhances local feature restoration through self-reconstruction of masked and perturbed patches. Extensive experiments on cross-field synthesis tasks demonstrate the effectiveness of our method, achieving state-of-the-art performance (e.g., 31.80 $\pm$ 2.70 dB in PSNR, 0.943 $\pm$ 0.102 in SSIM, and 0.0864 $\pm$ 0.0689 in LPIPS). Beyond pixel-wise fidelity, our method also preserves fine-grained anatomical structures compared with the original low-field MRI (e.g., left cerebral white matter error drops from 12.1$%$ to 2.1$%$, cortex from 4.2$%$ to 3.7$%$). To conclude, our CSS-Diff can synthesize images that are both quantitatively reliable and anatomically consistent.

Method: UniCalli employs asymmetric noising and a rasterized box map for spatial priors, trained on a mix of synthetic, labeled, and unlabeled data from a curated dataset of over 8,000 digitized pieces.

Result: The model achieves state-of-the-art generative quality with superior ligature continuity and layout fidelity, alongside stronger recognition performance. It successfully extends to other ancient scripts including Oracle bone inscriptions and Egyptian hieroglyphs.

Conclusion: Joint training of recognition and generation tasks creates synergy that improves both tasks, especially in limited-data regimes, by fostering concept-level abstractions.

Abstract: Computational replication of Chinese calligraphy remains challenging. Existing methods falter, either creating high-quality isolated characters while ignoring page-level aesthetics like ligatures and spacing, or attempting page synthesis at the expense of calligraphic correctness. We introduce \textbf{UniCalli}, a unified diffusion framework for column-level recognition and generation. Training both tasks jointly is deliberate: recognition constrains the generator to preserve character structure, while generation provides style and layout priors. This synergy fosters concept-level abstractions that improve both tasks, especially in limited-data regimes. We curated a dataset of over 8,000 digitized pieces, with ~4,000 densely annotated. UniCalli employs asymmetric noising and a rasterized box map for spatial priors, trained on a mix of synthetic, labeled, and unlabeled data. The model achieves state-of-the-art generative quality with superior ligature continuity and layout fidelity, alongside stronger recognition. The framework successfully extends to other ancient scripts, including Oracle bone inscriptions and Egyptian hieroglyphs. Code and data can be viewed in \href{https://github.com/EnVision-Research/UniCalli}{this URL}.

Method: Integration of vision encoder, audio encoder, LLM, and speech decoder into unified architecture; multi-stage training strategy including pre-training for omni-modal understanding and post-training with speech conversation and audio-visual interaction; curated multi-turn training dataset for long-term conversational ability.

Result: Significantly outperforms leading open-source models in multi-turn audio-visual experience; InteractiveOmni-4B comparable to Qwen2.5-Omni-7B on general benchmarks while retaining 97% performance of 8B model with 50% size; state-of-the-art results across image, audio, video understanding, and speech generation tasks.

Conclusion: InteractiveOmni provides an accessible, open-source foundation for next-generation intelligent interactive systems with superior multi-turn memory and speech interaction capabilities, demonstrating efficient scaling and comprehensive omni-modal understanding.

Abstract: We introduce InteractiveOmni, a unified and open-source omni-modal large language model for audio-visual multi-turn interaction, ranging from 4B to 8B parameters, designed to lead the field of lightweight models by offering comprehensive omni-modal understanding and speech generation capabilities. To achieve this, we integrate the vision encoder, audio encoder, large language model, and speech decoder into a unified model for understanding and generation tasks. We design a multi-stage training strategy to ensure robust cross-modal capabilities, including pre-training for omni-modal understanding, followed by post-training with speech conversation and audio-visual interaction. To enable human-like long-term conversational ability, we meticulously curate a multi-turn training dataset that enhances the model’s ability to handle complex and multi-turn interactions. To effectively evaluate the multi-turn memory and speech interaction capabilities, we construct the multi-modal multi-turn memory benchmark and the multi-turn speech interaction benchmark. Experiments demonstrate that InteractiveOmni significantly outperforms leading open-source models and provides a more intelligent multi-turn audio-visual experience, particularly in its long-term memory capabilities. Notably, InteractiveOmni-4B is comparable to the much larger model like Qwen2.5-Omni-7B on general benchmarks, and it can retain 97% of the performance of the InteractiveOmni-8B while utilizing only 50% of the model size. Achieving state-of-the-art results against similarly sized models across image, audio, video understanding, and speech generation tasks, InteractiveOmni is an accessible, open-source foundation for next-generation intelligent interactive systems.

Method: Presents Uni-MMMU benchmark with bidirectionally coupled tasks across eight domains (science, coding, math, puzzles), incorporating verifiable reasoning steps, unique ground truths, and reproducible scoring for both text and visual outputs.

Result: Extensive evaluation reveals substantial performance disparities and cross-modal dependencies among state-of-the-art unified, generation-only, and understanding-only models.

Conclusion: Provides new insights into when and how understanding and generation abilities reinforce each other, establishing a reliable foundation for advancing unified multimodal models.

Abstract: Unified multimodal models aim to jointly enable visual understanding and generation, yet current benchmarks rarely examine their true integration. Existing evaluations either treat the two abilities in isolation or overlook tasks that inherently couple them. To address this gap, we present Uni-MMMU, a comprehensive and discipline-aware benchmark that systematically unfolds the bidirectional synergy between generation and understanding across eight reasoning-centric domains, including science, coding, mathematics, and puzzles. Each task is bidirectionally coupled, demanding models to (i) leverage conceptual understanding to guide precise visual synthesis, or (ii) utilize generation as a cognitive scaffold for analytical reasoning. Uni-MMMU incorporates verifiable intermediate reasoning steps, unique ground truths, and a reproducible scoring protocol for both textual and visual outputs. Through extensive evaluation of state-of-the-art unified, generation-only, and understanding-only models, we reveal substantial performance disparities and cross-modal dependencies, offering new insights into when and how these abilities reinforce one another, and establishing a reliable foundation for advancing unified models.

Method: AVC uses CLIP to retrieve semantically aligned previous images, and when no relevant image is found, restricts visual conditioning to early diffusion stages only. Also improves data quality by re-captioning noisy datasets with LLMs.

Result: AVC achieves superior coherence, semantic consistency, and visual fidelity compared to baselines, especially in challenging cases where prior visuals conflict with current input.

Conclusion: The adaptive visual conditioning approach effectively balances the use of visual context while preventing irrelevant information injection, demonstrating improved performance in story continuation tasks.

Abstract: Story continuation focuses on generating the next image in a narrative sequence so that it remains coherent with both the ongoing text description and the previously observed images. A central challenge in this setting lies in utilizing prior visual context effectively, while ensuring semantic alignment with the current textual input. In this work, we introduce AVC (Adaptive Visual Conditioning), a framework for diffusion-based story continuation. AVC employs the CLIP model to retrieve the most semantically aligned image from previous frames. Crucially, when no sufficiently relevant image is found, AVC adaptively restricts the influence of prior visuals to only the early stages of the diffusion process. This enables the model to exploit visual context when beneficial, while avoiding the injection of misleading or irrelevant information. Furthermore, we improve data quality by re-captioning a noisy dataset using large language models, thereby strengthening textual supervision and semantic alignment. Quantitative results and human evaluations demonstrate that AVC achieves superior coherence, semantic consistency, and visual fidelity compared to strong baselines, particularly in challenging cases where prior visuals conflict with the current input.

Method: The method utilizes the random seed used to initialize the diffusion process as a watermark, incorporating a hash function into noise sampling to prevent seed recovery from content. It uses cryptographic zero-knowledge proofs to prove ownership without revealing the seed.

Result: The method demonstrates efficient verification using only the seed and output without requiring model weights, shows robustness under various manipulations, and validates on multiple state-of-the-art diffusion models for images and videos.

Conclusion: NoisePrints provides a practical and scalable solution for copyright protection in diffusion models by using random seeds as watermarks, enabling third-party verification without model access while maintaining security through cryptographic proofs.

Abstract: With the rapid adoption of diffusion models for visual content generation, proving authorship and protecting copyright have become critical. This challenge is particularly important when model owners keep their models private and may be unwilling or unable to handle authorship issues, making third-party verification essential. A natural solution is to embed watermarks for later verification. However, existing methods require access to model weights and rely on computationally heavy procedures, rendering them impractical and non-scalable. To address these challenges, we propose , a lightweight watermarking scheme that utilizes the random seed used to initialize the diffusion process as a proof of authorship without modifying the generation process. Our key observation is that the initial noise derived from a seed is highly correlated with the generated visual content. By incorporating a hash function into the noise sampling process, we further ensure that recovering a valid seed from the content is infeasible. We also show that sampling an alternative seed that passes verification is infeasible, and demonstrate the robustness of our method under various manipulations. Finally, we show how to use cryptographic zero-knowledge proofs to prove ownership without revealing the seed. By keeping the seed secret, we increase the difficulty of watermark removal. In our experiments, we validate NoisePrints on multiple state-of-the-art diffusion models for images and videos, demonstrating efficient verification using only the seed and output, without requiring access to model weights.

Method: Proposes a dual-path pooling mechanism that aligns geometric features with semantic and positional cues to create unified image patch-based 3D representations. Also introduces the Grounded Chain-of-Thought (GCoT) dataset with 3D bounding box annotations and reasoning paths.

Result: GS-Reasoner achieves impressive results on 3D visual grounding and state-of-the-art performance in spatial reasoning, demonstrating that grounding significantly enhances reasoning capabilities.

Conclusion: GS-Reasoner establishes a unified and self-contained framework for 3D spatial reasoning, successfully bridging the gap between grounding and reasoning through holistic representations and achieving comparable performance to state-of-the-art models without external modules.

Abstract: In this paper, we claim that 3D visual grounding is the cornerstone of spatial reasoning and introduce the Grounded-Spatial Reasoner (GS-Reasoner) to explore the effective spatial representations that bridge the gap between them. Existing 3D LLMs suffer from the absence of a unified 3D representation capable of jointly capturing semantic and geometric information. This deficiency is manifested either in poor performance on grounding or in an excessive reliance on external modules, ultimately hindering the seamless integration of grounding and spatial reasoning. To address this, we propose a simple yet effective dual-path pooling mechanism that tightly aligns geometric features with both semantic and positional cues, constructing a unified image patch-based 3D representation that encapsulates all essential information without increasing the number of input tokens. Leveraging this holistic representation, GS-Reasoner is the first 3D LLM that achieves autoregressive grounding entirely without external modules while delivering performance comparable to state-of-the-art models, establishing a unified and self-contained framework for 3D spatial reasoning. To further bridge grounding and spatial reasoning, we introduce the Grounded Chain-of-Thought (GCoT) dataset. This dataset is meticulously curated to include both 3D bounding box annotations for objects referenced in reasoning questions and step-by-step reasoning paths that integrate grounding as a core component of the problem-solving process. Extensive experiments demonstrate that GS-Reasoner achieves impressive results on 3D visual grounding, which in turn significantly enhances its spatial reasoning capabilities, leading to state-of-the-art performance.

Method: Proposes representing videos as Trajectory Fields - dense mappings that assign continuous 3D trajectory functions to each pixel. Uses a neural network that predicts control points for B-spline parameterization of trajectories in a single forward pass.

Result: Achieves state-of-the-art performance on trajectory field estimation benchmarks, competitive results on point-tracking benchmarks, offers significant efficiency gains through one-pass prediction, and demonstrates emergent abilities including goal-conditioned manipulation and motion forecasting.

Conclusion: The Trajectory Field representation and Trace Anything model provide an efficient and effective approach for spatio-temporal video understanding, enabling single-pass trajectory prediction with emergent capabilities beyond traditional tracking methods.

Abstract: Effective spatio-temporal representation is fundamental to modeling, understanding, and predicting dynamics in videos. The atomic unit of a video, the pixel, traces a continuous 3D trajectory over time, serving as the primitive element of dynamics. Based on this principle, we propose representing any video as a Trajectory Field: a dense mapping that assigns a continuous 3D trajectory function of time to each pixel in every frame. With this representation, we introduce Trace Anything, a neural network that predicts the entire trajectory field in a single feed-forward pass. Specifically, for each pixel in each frame, our model predicts a set of control points that parameterizes a trajectory (i.e., a B-spline), yielding its 3D position at arbitrary query time instants. We trained the Trace Anything model on large-scale 4D data, including data from our new platform, and our experiments demonstrate that: (i) Trace Anything achieves state-of-the-art performance on our new benchmark for trajectory field estimation and performs competitively on established point-tracking benchmarks; (ii) it offers significant efficiency gains thanks to its one-pass paradigm, without requiring iterative optimization or auxiliary estimators; and (iii) it exhibits emergent abilities, including goal-conditioned manipulation, motion forecasting, and spatio-temporal fusion. Project page: https://trace-anything.github.io/.

Method: Augment VLM vision encoder with compact set of learnable visual probes to enable efficient domain-specific adaptation with minimal modification to pretrained parameters.

Result: VisCoP consistently outperforms existing adaptation strategies across cross-view, cross-modal, and cross-task settings, achieving superior target domain performance while retaining source-domain knowledge.

Conclusion: VisCoP provides an effective solution for domain adaptation in VLMs through visual probes, enabling robust performance on novel domains without catastrophic forgetting.

Abstract: Large Vision-Language Models (VLMs) excel at general visual reasoning tasks but exhibit sharp performance degradation when applied to novel domains with substantial distribution shifts from pretraining data. Existing domain adaptation approaches finetune different VLM components, but this often results in limited domain-specific feature learning or catastrophic forgetting of prior capabilities. To address these issues, we introduce Vision Contextualized Probing (VisCoP), which augments the VLM’s vision encoder with a compact set of learnable visual probes. These probes enable efficient domain-specific adaptation with minimal modification to pretrained parameters. We evaluate VisCoP across three challenging domain adaptation settings-cross-view (exocentric to egocentric), cross-modal (RGB to depth), and cross-task (human understanding to robot control). Experiments show that VisCoP consistently outperforms existing adaptation strategies, achieving superior performance on target domains while effectively retaining source-domain knowledge.

Method: PhysMaster uses PhysEncoder to extract physical information from input images as conditioning, and applies reinforcement learning with Direct Preference Optimization (DPO) to optimize physical representations based on human feedback.

Result: PhysMaster demonstrates improved physics-awareness in video generation, proving effective on proxy tasks and generalizing to various physical scenarios.

Conclusion: PhysMaster provides a generic, plug-in solution for physics-aware video generation that unifies physical process modeling through representation learning in a reinforcement learning framework.

Abstract: Video generation models nowadays are capable of generating visually realistic videos, but often fail to adhere to physical laws, limiting their ability to generate physically plausible videos and serve as ‘‘world models’’. To address this issue, we propose PhysMaster, which captures physical knowledge as a representation for guiding video generation models to enhance their physics-awareness. Specifically, PhysMaster is based on the image-to-video task where the model is expected to predict physically plausible dynamics from the input image. Since the input image provides physical priors like relative positions and potential interactions of objects in the scenario, we devise PhysEncoder to encode physical information from it as an extra condition to inject physical knowledge into the video generation process. The lack of proper supervision on the model’s physical performance beyond mere appearance motivates PhysEncoder to apply reinforcement learning with human feedback to physical representation learning, which leverages feedback from generation models to optimize physical representations with Direct Preference Optimization (DPO) in an end-to-end manner. PhysMaster provides a feasible solution for improving physics-awareness of PhysEncoder and thus of video generation, proving its ability on a simple proxy task and generalizability to wide-ranging physical scenarios. This implies that our PhysMaster, which unifies solutions for various physical processes via representation learning in the reinforcement learning paradigm, can act as a generic and plug-in solution for physics-aware video generation and broader applications.

Method: Hard-to-Easy (H2E) iterative noisy learning framework that learns a classifier as noise identifier invariant to class and context distributional changes, reducing hard noises to easy ones through bootstrapping.

Result: H2E outperforms state-of-the-art de-noising methods on three NLT benchmarks (ImageNet-NLT, Animal10-NLT, Food101-NLT) while maintaining stable performance on balanced settings.

Conclusion: The proposed H2E framework effectively addresses the NLT challenge by iteratively transforming hard noises into easy ones, achieving superior performance on long-tailed noisy datasets.

Abstract: Conventional de-noising methods rely on the assumption that all samples are independent and identically distributed, so the resultant classifier, though disturbed by noise, can still easily identify the noises as the outliers of training distribution. However, the assumption is unrealistic in large-scale data that is inevitably long-tailed. Such imbalanced training data makes a classifier less discriminative for the tail classes, whose previously “easy” noises are now turned into “hard” ones – they are almost as outliers as the clean tail samples. We introduce this new challenge as Noisy Long-Tailed Classification (NLT). Not surprisingly, we find that most de-noising methods fail to identify the hard noises, resulting in significant performance drop on the three proposed NLT benchmarks: ImageNet-NLT, Animal10-NLT, and Food101-NLT. To this end, we design an iterative noisy learning framework called Hard-to-Easy (H2E). Our bootstrapping philosophy is to first learn a classifier as noise identifier invariant to the class and context distributional changes, reducing “hard” noises to “easy” ones, whose removal further improves the invariance. Experimental results show that our H2E outperforms state-of-the-art de-noising methods and their ablations on long-tailed settings while maintaining a stable performance on the conventional balanced settings. Datasets and codes are available at https://github.com/yxymessi/H2E-Framework

Method: Proposed SHAN (Scene Heterogeneous graph Attention Network) that constructs scene heterogeneous graphs from images and uses self-attention mechanisms for scene inference to determine object privacy. Also introduced two benchmark datasets for object-level privacy detection.

Result: SHAN demonstrated excellent performance in privacy object detection tasks, with all evaluation metrics surpassing those of baseline models.

Conclusion: The proposed SHAN model effectively addresses the limitations of existing privacy detection methods by leveraging scene context through heterogeneous graphs and attention mechanisms, providing better accuracy and generalization for privacy object detection.

Abstract: With the rise of social platforms, protecting privacy has become an important issue. Privacy object detection aims to accurately locate private objects in images. It is the foundation of safeguarding individuals’ privacy rights and ensuring responsible data handling practices in the digital age. Since privacy of object is not shift-invariant, the essence of the privacy object detection task is inferring object privacy based on scene information. However, privacy object detection has long been studied as a subproblem of common object detection tasks. Therefore, existing methods suffer from serious deficiencies in accuracy, generalization, and interpretability. Moreover, creating large-scale privacy datasets is difficult due to legal constraints and existing privacy datasets lack label granularity. The granularity of existing privacy detection methods remains limited to the image level. To address the above two issues, we introduce two benchmark datasets for object-level privacy detection and propose SHAN, Scene Heterogeneous graph Attention Network, a model constructs a scene heterogeneous graph from an image and utilizes self-attention mechanisms for scene inference to obtain object privacy. Through experiments, we demonstrated that SHAN performs excellently in privacy object detection tasks, with all metrics surpassing those of the baseline model.

Method: Three-stage framework: 1) Structuring - convert images to heterogeneous scene graphs embedding rich contexts; 2) Data Augmentation - contextual perturbation oversampling to balance skewed privacy class distribution; 3) Hybrid Graph Generation & Reasoning - transform scene graphs into hybrid graphs with homogeneous paths for direct message passing and capturing subtle context changes.

Result: The framework enables explicit derivation of objects’ privacy classes from scene contexts through visual reasoning, addressing the challenge that privacy classification depends on contextual factors beyond visual appearance.

Conclusion: POI should be treated as a visual reasoning task, and the PrivacyGuard framework effectively handles the contextual nature of privacy classification through structured scene graph processing and hybrid graph reasoning.

Abstract: The Privacy-sensitive Object Identification (POI) task allocates bounding boxes for privacy-sensitive objects in a scene. The key to POI is settling an object’s privacy class (privacy-sensitive or non-privacy-sensitive). In contrast to conventional object classes which are determined by the visual appearance of an object, one object’s privacy class is derived from the scene contexts and is subject to various implicit factors beyond its visual appearance. That is, visually similar objects may be totally opposite in their privacy classes. To explicitly derive the objects’ privacy class from the scene contexts, in this paper, we interpret the POI task as a visual reasoning task aimed at the privacy of each object in the scene. Following this interpretation, we propose the PrivacyGuard framework for POI. PrivacyGuard contains three stages. i) Structuring: an unstructured image is first converted into a structured, heterogeneous scene graph that embeds rich scene contexts. ii) Data Augmentation: a contextual perturbation oversampling strategy is proposed to create slightly perturbed privacy-sensitive objects in a scene graph, thereby balancing the skewed distribution of privacy classes. iii) Hybrid Graph Generation & Reasoning: the balanced, heterogeneous scene graph is then transformed into a hybrid graph by endowing it with extra “node-node” and “edge-edge” homogeneous paths. These homogeneous paths allow direct message passing between nodes or edges, thereby accelerating reasoning and facilitating the capturing of subtle context changes. Based on this hybrid graph… For the full abstract, see the original paper.

Method: Uses frozen vision-only models with spatial awareness, aligns only the text encoder, and exploits text’s discrete nature to identify local concepts in captions.

Result: Achieves state-of-the-art results on 7 out of 8 benchmark datasets when trained on COCO Captions in less than 15 minutes using 8 GPUs.

Conclusion: SimZSS demonstrates that leveraging quality visual representations and focused text alignment enables efficient zero-shot segmentation with minimal training requirements.

Abstract: Zero-shot classification capabilities naturally arise in models trained within a vision-language contrastive framework. Despite their classification prowess, these models struggle in dense tasks like zero-shot open-vocabulary segmentation. This deficiency is often attributed to the absence of localization cues in captions and the intertwined nature of the learning process, which encompasses both image representation learning and cross-modality alignment. To tackle these issues, we propose SimZSS, a Simple framework for open-vocabulary Zero-Shot Segmentation. The method is founded on two key principles: i) leveraging frozen vision-only models that exhibit spatial awareness while exclusively aligning the text encoder and ii) exploiting the discrete nature of text and linguistic knowledge to pinpoint local concepts within captions. By capitalizing on the quality of the visual representations, our method requires only image-caption pairs datasets and adapts to both small curated and large-scale noisy datasets. When trained on COCO Captions across 8 GPUs, SimZSS achieves state-of-the-art results on 7 out of 8 benchmark datasets in less than 15 minutes.

Method: Jigsaw++ learns a shape prior of complete objects and employs a “retargeting” strategy that leverages outputs from existing assembly methods to generate complete shape reconstructions.

Result: Extensive evaluations on Breaking Bad dataset and PartNet show Jigsaw++ reduces reconstruction errors and enhances shape reconstruction precision.

Conclusion: Jigsaw++ sets a new direction for future reassembly model developments by effectively integrating complete object priors into the assembly process.

Abstract: The automatic assembly problem has attracted increasing interest due to its complex challenges that involve 3D representation. This paper introduces Jigsaw++, a novel generative method designed to tackle the multifaceted challenges of reconstructing complete shape for the reassembly problem. Existing approach focusing primarily on piecewise information for both part and fracture assembly, often overlooking the integration of complete object prior. Jigsaw++ distinguishes itself by learning a shape prior of complete objects. It employs the proposed “retargeting” strategy that effectively leverages the output of any existing assembly method to generate complete shape reconstructions. This capability allows it to function orthogonally to the current methods. Through extensive evaluations on Breaking Bad dataset and PartNet, Jigsaw++ has demonstrated its effectiveness, reducing reconstruction errors and enhancing the precision of shape reconstruction, which sets a new direction for future reassembly model developments.

Method: Decomposes visual reasoning into proactive visual perception and textual reasoning stages. Uses iterative proactive information collection and reasoning cycles. Allows integration of existing LLMs to compensate for LVLM reasoning deficits.

Result: Outperforms existing multi-step reasoning frameworks by 13.2% on average across various benchmarks for both open-source and closed-source models. Produces high-quality visual reasoning data that enables distilled models to achieve superior downstream task performance.

Conclusion: The decoupled perspective and LLM integration provide effective solutions for visual reasoning challenges and illuminate future research directions for LLM-assisted visual reasoning techniques.

Abstract: Large vision-language models (LVLMs) have witnessed significant progress on visual understanding tasks. However, they often prioritize language knowledge over image information on visual reasoning tasks, incurring performance degradation. To tackle this issue, we first identify the drawbacks of existing solutions (i.e., limited multi-modal reasoning capacities, and insufficient and irrelevant visual descriptions). We then decompose visual reasoning process into two stages: proactive visual perception (i.e., eyesight) and textual reasoning (i.e., wisdom), and introduce a novel visual reasoning framework named ProReason. This framework features decoupled vision-reasoning capabilities and multi-run proactive perception. Briefly, given a multi-modal question, ProReason iterates proactive information collection and reasoning until the answer can be concluded with necessary and sufficient visual descriptions. Notably, the disassociation of capabilities allows seamless integration of existing large language models (LLMs) to compensate for the reasoning deficits of LVLMs. Our extensive experiments demonstrate that ProReason outperforms existing multi-step reasoning frameworks on various benchmarks for both open-source and closed-source models, with the average performance gain reaching 13.2%. Besides, the integration of LLMs allows ProReason to produce high-quality visual reasoning data, which empowers ProReason-distilled models (i.e., ProReason-VL and ProReason-Q3) to achieve superior performance in downstream tasks. Our insights into existing solutions and the decoupled perspective for feasible integration of LLMs illuminate future research on visual reasoning techniques, especially LLM-assisted ones.

Method: Proposes Hints of Prompt (HoP) framework with three enhancements: Affinity hint for instance-level structure, Semantic hint for driving-specific high-level information, and Question hint for query context alignment, fused through a Hint Fusion module.

Result: Extensive experiments show HoP significantly outperforms previous state-of-the-art methods in all key metrics, enabling faster adaptation to driving scenarios with limited domain data.

Conclusion: The HoP framework effectively enriches visual representations for autonomous driving by capturing driving-related representations and addressing the limitations of general MLLMs in complex driving scenarios.

Abstract: In light of the dynamic nature of autonomous driving environments and stringent safety requirements, general MLLMs combined with CLIP alone often struggle to accurately represent driving-specific scenarios, particularly in complex interactions and long-tail cases. To address this, we propose the Hints of Prompt (HoP) framework, which introduces three key enhancements: Affinity hint to emphasize instance-level structure by strengthening token-wise connections, Semantic hint to incorporate high-level information relevant to driving-specific cases, such as complex interactions among vehicles and traffic signs, and Question hint to align visual features with the query context, focusing on question-relevant regions. These hints are fused through a Hint Fusion module, enriching visual representations by capturing driving-related representations with limited domain data, ensuring faster adaptation to driving scenarios. Extensive experiments confirm the effectiveness of the HoP framework, showing that it significantly outperforms previous state-of-the-art methods in all key metrics.

Method: Defines a domain-agnostic structure based on semantic/geometric relationships between class labels in language space and guides adaptation to reflect reference inter-class relationships while preserving domain-specific characteristics.

Result: Surpassed previous works in 18 adaptation scenarios across four datasets with average accuracy improvements: +3.32% on DomainNet, +5.75% on GeoPlaces, +4.77% on GeoImnet, and +1.94% mean class accuracy on EgoExo4D.

Conclusion: The proposed relative positioning alignment approach effectively enables knowledge transfer across unseen domains while preserving domain-specific features, demonstrating superior performance over existing methods.

Abstract: Unsupervised domain adaptation remains a critical challenge in enabling the knowledge transfer of models across unseen domains. Existing methods struggle to balance the need for domain-invariant representations with preserving domain-specific features, which is often due to alignment approaches that impose the projection of samples with similar semantics close in the latent space despite their drastic domain differences. We introduce a novel approach that shifts the focus from aligning representations in absolute coordinates to aligning the relative positioning of equivalent concepts in latent spaces. Our method defines a domain-agnostic structure upon the semantic/geometric relationships between class labels in language space and guides adaptation, ensuring that the organization of samples in visual space reflects reference inter-class relationships while preserving domain-specific characteristics. We empirically demonstrate our method’s superiority in domain adaptation tasks across four diverse image and video datasets. Remarkably, we surpass previous works in 18 different adaptation scenarios across four diverse image and video datasets with average accuracy improvements of +3.32% on DomainNet, +5.75% in GeoPlaces, +4.77% on GeoImnet, and +1.94% mean class accuracy improvement on EgoExo4D.

Method: Uses ControlNet diffusion model conditioned on semantic maps with a novel prompting scheme to generate subgroup-specific, semantically dense prompts for data augmentation.

Result: Improved Mask2Former and SegFormer segmentation models by up to 1.2% and 2.3% on Waymo, and 1.4% and 0.7% on DeepDrive. Enhanced AIM-2D and AIM-BEV autonomous driving models by up to 20% in CARLA simulator.

Conclusion: SynDiff-AD provides an effective data augmentation approach that enhances model robustness across diverse environmental conditions for autonomous driving applications.

Abstract: In recent years, significant progress has been made in collecting large-scale datasets to improve segmentation and autonomous driving models. These large-scale datasets are often dominated by common environmental conditions such as “Clear and Day” weather, leading to decreased performance in under-represented conditions like “Rainy and Night”. To address this issue, we introduce SynDiff-AD, a novel data augmentation pipeline that leverages diffusion models (DMs) to generate realistic images for such subgroups. SynDiff-AD uses ControlNet-a DM that guides data generation conditioned on semantic maps-along with a novel prompting scheme that generates subgroup-specific, semantically dense prompts. By augmenting datasets with SynDiff-AD, we improve the performance of segmentation models like Mask2Former and SegFormer by up to 1.2% and 2.3% on the Waymo dataset, and up to 1.4% and 0.7% on the DeepDrive dataset, respectively. Additionally, we demonstrate that our SynDiff-AD pipeline enhances the driving performance of end-to-end autonomous driving models, like AIM-2D and AIM-BEV, by up to 20% across diverse environmental conditions in the CARLA autonomous driving simulator, providing a more robust model. We release our code and pipeline at https://github.com/UTAustin-SwarmLab/SynDiff-AD.

Method: Two key components: 1) Prompt tuning blocks with student feedback to adapt teacher features to student’s learning process, 2) Region-aware attention to solve view mismatch between heterogeneous architectures.

Result: Extensive experiments on CIFAR, ImageNet, and COCO datasets demonstrate superior performance compared to existing methods.

Conclusion: PAT framework effectively enables knowledge distillation across diverse neural architectures, providing a universal solution for heterogeneous model knowledge transfer.

Abstract: Knowledge distillation (KD) involves transferring knowledge from a pre-trained heavy teacher model to a lighter student model, thereby reducing the inference cost while maintaining comparable effectiveness. Prior KD techniques typically assume homogeneity between the teacher and student models. However, as technology advances, a wide variety of architectures have emerged, ranging from initial Convolutional Neural Networks (CNNs) to Vision Transformers (ViTs), and Multi-Level Perceptrons (MLPs). Consequently, developing a universal KD framework compatible with any architecture has become an important research topic. In this paper, we introduce a perspective-aware teaching (PAT) KD framework to enable feature distillation across diverse architectures. Our framework comprises two key components. First, we design prompt tuning blocks that incorporate student feedback, allowing teacher features to adapt to the student model’s learning process. Second, we propose region-aware attention to mitigate the view mismatch problem between heterogeneous architectures. By leveraging these two modules, effective distillation of intermediate features can be achieved across heterogeneous architectures. Extensive experiments on CIFAR, ImageNet, and COCO demonstrate the superiority of the proposed method. Our code is available at https://github.com/jimmylin0979/PAT.git.

Method: Uses a motion field agent to extract object movement and camera motion from text, converts them to object trajectories and camera extrinsics, integrates them in 3D space, projects to unified optical flow, and uses an optical flow adapter to control the base diffusion model.

Result: Significant improvement in Video-Text Camera Motion metrics on VBench, outperforming other advanced models on motion generation accuracy and achieving precise camera motion control.

Conclusion: MotionAgent successfully enables fine-grained motion control for text-guided video generation through explicit motion field representation and optical flow guidance, demonstrating superior performance in motion-text alignment.

Abstract: We propose MotionAgent, enabling fine-grained motion control for text-guided image-to-video generation. The key technique is the motion field agent that converts motion information in text prompts into explicit motion fields, providing flexible and precise motion guidance. Specifically, the agent extracts the object movement and camera motion described in the text and converts them into object trajectories and camera extrinsics, respectively. An analytical optical flow composition module integrates these motion representations in 3D space and projects them into a unified optical flow. An optical flow adapter takes the flow to control the base image-to-video diffusion model for generating fine-grained controlled videos. The significant improvement in the Video-Text Camera Motion metrics on VBench indicates that our method achieves precise control over camera motion. We construct a subset of VBench to evaluate the alignment of motion information in the text and the generated video, outperforming other advanced models on motion generation accuracy.

Method: PRISM removes the corrupting influence of global background features by modeling intrinsic visual semantics via implicit re-centering, providing a training-free framework for efficient visual instruction selection.

Result: PRISM reduces end-to-end time for data selection and model tuning to 30% of conventional pipelines while surpassing models fine-tuned on full datasets across eight multimodal and three language understanding benchmarks, achieving 101.7% relative improvement over baseline.

Conclusion: PRISM effectively addresses the computational inefficiency in visual instruction data selection by leveraging implicit re-centering to model intrinsic visual semantics, enabling both efficiency gains and performance improvements.

Abstract: Visual instruction tuning adapts pre-trained Multimodal Large Language Models (MLLMs) to follow human instructions for real-world applications. However, the rapid growth of these datasets introduces significant redundancy, leading to increased computational costs. Existing methods for selecting instruction data aim to prune this redundancy, but predominantly rely on computationally demanding techniques such as proxy-based inference or training-based metrics. Consequently, the substantial computational costs incurred by these selection processes often exacerbate the very efficiency bottlenecks they are intended to resolve, posing a significant challenge to the scalable and effective tuning of MLLMs. To address this challenge, we first identify a critical, yet previously overlooked, factor: the anisotropy inherent in visual feature distributions. We find that this anisotropy induces a \textit{Global Semantic Drift}, and overlooking this phenomenon is a key factor limiting the efficiency of current data selection methods. Motivated by this insight, we devise \textbf{PRISM}, the first training-free framework for efficient visual instruction selection. PRISM surgically removes the corrupting influence of global background features by modeling the intrinsic visual semantics via implicit re-centering. Empirically, PRISM reduces the end-to-end time for data selection and model tuning to just 30% of conventional pipelines. More remarkably, it achieves this efficiency while simultaneously enhancing performance, surpassing models fine-tuned on the full dataset across eight multimodal and three language understanding benchmarks, culminating in a 101.7% relative improvement over the baseline. The code is available for access via \href{https://github.com/bibisbar/PRISM}{this repository}.

Method: Proposes four strategies: dynamic channel-patch masking, memory tokens for cross-channel information sharing, hybrid token fusion module, and Channel-Aware Decoder for effective patch reconstruction.

Result: Experiments on satellite and microscopy datasets (CHAMMI, JUMP-CP, So2Sat) show ChA-MAEViT outperforms state-of-the-art MCI-ViTs by 3.0-21.5%.

Conclusion: Cross-channel interactions are crucial in Multi-Channel Imaging, and ChA-MAEViT effectively addresses this limitation in standard MAEs.

Abstract: Prior work using Masked Autoencoders (MAEs) typically relies on random patch masking based on the assumption that images have significant redundancies across different channels, allowing for the reconstruction of masked content using cross-channel correlations. However, this assumption does not hold in Multi-Channel Imaging (MCI), where channels may provide complementary information with minimal feature overlap. Thus, these MAEs primarily learn local structures within individual channels from patch reconstruction, failing to fully leverage cross-channel interactions and limiting their MCI effectiveness. In this paper, we present ChA-MAEViT, an MAE-based method that enhances feature learning across MCI channels via four key strategies: (1) dynamic channel-patch masking, which compels the model to reconstruct missing channels in addition to masked patches, thereby enhancing cross-channel dependencies and improving robustness to varying channel configurations; (2) memory tokens, which serve as long-term memory aids to promote information sharing across channels, addressing the challenges of reconstructing structurally diverse channels; (3) hybrid token fusion module, which merges fine-grained patch tokens with a global class token to capture richer representations; and (4) Channel-Aware Decoder, a lightweight decoder utilizes channel tokens to effectively reconstruct image patches. Experiments on satellite and microscopy datasets, CHAMMI, JUMP-CP, and So2Sat, show that ChA-MAEViT significantly outperforms state-of-the-art MCI-ViTs by 3.0-21.5%, highlighting the importance of cross-channel interactions in MCI. Our code is publicly available at https://github.com/chaudatascience/cha_mae_vit.

Method: The study follows PRISMA 2020 guidelines and systematically analyzes 106 peer-reviewed articles based on modalities, collaboration schemes, and key perception tasks. It conducts comparative analysis of how different methods address practical issues.

Result: The review illustrates how various methods address practical challenges including pose errors, temporal latency, communication constraints, domain shifts, heterogeneity, and adversarial attacks. It also critically examines evaluation methodologies.

Conclusion: The review offers valuable insights into challenges, opportunities, and risks in vehicular collaborative perception, serving as a reference for advancing research in this field while highlighting misalignment between current metrics and CP’s fundamental objectives.

Abstract: The effectiveness of autonomous vehicles relies on reliable perception capabilities. Despite significant advancements in artificial intelligence and sensor fusion technologies, current single-vehicle perception systems continue to encounter limitations, notably visual occlusions and limited long-range detection capabilities. Collaborative Perception (CP), enabled by Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I) communication, has emerged as a promising solution to mitigate these issues and enhance the reliability of autonomous systems. Beyond advancements in communication, the computer vision community is increasingly focusing on improving vehicular perception through collaborative approaches. However, a systematic literature review that thoroughly examines existing work and reduces subjective bias is still lacking. Such a systematic approach helps identify research gaps, recognize common trends across studies, and inform future research directions. In response, this study follows the PRISMA 2020 guidelines and includes 106 peer-reviewed articles. These publications are analyzed based on modalities, collaboration schemes, and key perception tasks. Through a comparative analysis, this review illustrates how different methods address practical issues such as pose errors, temporal latency, communication constraints, domain shifts, heterogeneity, and adversarial attacks. Furthermore, it critically examines evaluation methodologies, highlighting a misalignment between current metrics and CP’s fundamental objectives. By delving into all relevant topics in-depth, this review offers valuable insights into challenges, opportunities, and risks, serving as a reference for advancing research in vehicular collaborative perception.

Method: Proposes IterMask3D with iterative spatial mask-refining: iteratively masks image areas, reconstructs them, then shrinks mask based on reconstruction error. Also uses high-frequency image content as additional structural guidance for reconstruction.

Result: Extensive experiments show effectiveness in detecting both synthetic and real-world imaging artifacts, and segmenting various pathological lesions across multiple MRI sequences.

Conclusion: The proposed IterMask3D method consistently demonstrates effectiveness in 3D brain MRI anomaly detection and segmentation, reducing false positives through iterative mask refinement.

Abstract: Unsupervised anomaly detection and segmentation methods train a model to learn the training distribution as normal'. In the testing phase, they identify patterns that deviate from this normal distribution as anomalies’. To learn the normal' distribution, prevailing methods corrupt the images and train a model to reconstruct them. During testing, the model attempts to reconstruct corrupted inputs based on the learned normal’ distribution. Deviations from this distribution lead to high reconstruction errors, which indicate potential anomalies. However, corrupting an input image inevitably causes information loss even in normal regions, leading to suboptimal reconstruction and an increased risk of false positives. To alleviate this, we propose $\rm{IterMask3D}$, an iterative spatial mask-refining strategy designed for 3D brain MRI. We iteratively spatially mask areas of the image as corruption and reconstruct them, then shrink the mask based on reconstruction error. This process iteratively unmasks normal' areas to the model, whose information further guides reconstruction of normal’ patterns under the mask to be reconstructed accurately, reducing false positives. In addition, to achieve better reconstruction performance, we also propose using high-frequency image content as additional structural information to guide the reconstruction of the masked area. Extensive experiments on the detection of both synthetic and real-world imaging artifacts, as well as segmentation of various pathological lesions across multiple MRI sequences, consistently demonstrate the effectiveness of our proposed method. Code is available at https://github.com/ZiyunLiang/IterMask3D.

Method: Dynamically partition images into coherent regions by structural/semantic similarity, estimate region-level transferability, and incorporate transferability maps into self-attention mechanisms to focus on areas with lower transferability.

Result: Extensive experiments across 20 diverse cross-domain settings show TMT mitigates performance degradation from domain shift and consistently outperforms existing approaches.

Conclusion: Region-adaptive transferability guidance effectively enhances cross-domain representation learning in Vision Transformers for semantic segmentation.

Abstract: Recent advances in Vision Transformers (ViTs) have significantly advanced semantic segmentation performance. However, their adaptation to new target domains remains challenged by distribution shifts, which often disrupt global attention mechanisms. While existing global and patch-level adaptation methods offer some improvements, they overlook the spatially varying transferability inherent in different image regions. To address this, we propose the Transferable Mask Transformer (TMT), a region-adaptive framework designed to enhance cross-domain representation learning through transferability guidance. First, we dynamically partition the image into coherent regions, grouped by structural and semantic similarity, and estimates their domain transferability at a localized level. Then, we incorporate region-level transferability maps directly into the self-attention mechanism of ViTs, allowing the model to adaptively focus attention on areas with lower transferability and higher semantic uncertainty. Extensive experiments across 20 diverse cross-domain settings demonstrate that TMT not only mitigates the performance degradation typically associated with domain shift but also consistently outperforms existing approaches.

Method: Two-stage unlearning framework combining neural pruning with machine unlearning applied to writer classification head, plus Writer-ID Confusion (WIC) method that forces forget set to follow uniform distribution over writer identities.

Result: Achieves better privacy-accuracy trade-offs than Random Labeling, Fisher Forgetting, Amnesiac Unlearning, and DELETE methods, with state-of-the-art performance on IAM and CVL datasets using Accuracy, CER, WER, and MIA metrics.

Conclusion: The approach effectively safeguards privacy without compromising accuracy, providing the first systematic study of machine unlearning for HTR and opening new directions for document analysis research.

Abstract: Handwritten Text Recognition (HTR) is crucial for document digitization, but handwritten data can contain user-identifiable features, like unique writing styles, posing privacy risks. Regulations such as the ``right to be forgotten’' require models to remove these sensitive traces without full retraining. We introduce a practical encoder-only transformer baseline as a robust reference for future HTR research. Building on this, we propose a two-stage unlearning framework for multihead transformer HTR models. Our method combines neural pruning with machine unlearning applied to a writer classification head, ensuring sensitive information is removed while preserving the recognition head. We also present Writer-ID Confusion (WIC), a method that forces the forget set to follow a uniform distribution over writer identities, unlearning user-specific cues while maintaining text recognition performance. We compare WIC to Random Labeling, Fisher Forgetting, Amnesiac Unlearning, and DELETE within our prune-unlearn pipeline and consistently achieve better privacy and accuracy trade-offs. This is the first systematic study of machine unlearning for HTR. Using metrics such as Accuracy, Character Error Rate (CER), Word Error Rate (WER), and Membership Inference Attacks (MIA) on the IAM and CVL datasets, we demonstrate that our method achieves state-of-the-art or superior performance for effective unlearning. These experiments show that our approach effectively safeguards privacy without compromising accuracy, opening new directions for document analysis research. Our code is publicly available at https://github.com/leitro/WIC-WriterIDConfusion-MachineUnlearning.

Method: Used a scene-driven LLM scripting pipeline to create complete tasks with natural progression, combined with mocap-and-camera recording setup to handle occlusions. Professional artists cleaned and verified sequences to minimize artifacts like foot sliding and object penetrations.

Result: Created a dataset spanning diverse activities from cooking to outdoor picnics, preserving both physical accuracy and logical task flow. Provided benchmarks compared to other datasets.

Conclusion: HUMOTO’s comprehensive full-body motion and simultaneous multi-object interactions provide opportunities to advance realistic human-object interaction modeling across research domains with practical applications.

Abstract: We present Human Motions with Objects (HUMOTO), a high-fidelity dataset of human-object interactions for motion generation, computer vision, and robotics applications. Featuring 735 sequences (7,875 seconds at 30 fps), HUMOTO captures interactions with 63 precisely modeled objects and 72 articulated parts. Our innovations include a scene-driven LLM scripting pipeline creating complete, purposeful tasks with natural progression, and a mocap-and-camera recording setup to effectively handle occlusions. Spanning diverse activities from cooking to outdoor picnics, HUMOTO preserves both physical accuracy and logical task flow. Professional artists rigorously clean and verify each sequence, minimizing foot sliding and object penetrations. We also provide benchmarks compared to other datasets. HUMOTO’s comprehensive full-body motion and simultaneous multi-object interactions address key data-capturing challenges and provide opportunities to advance realistic human-object interaction modeling across research domains with practical applications in animation, robotics, and embodied AI systems. Project: https://jiaxin-lu.github.io/humoto/ .

Method: FramePack compresses input frames with frame-wise importance weighting, where more important frames get longer contexts. Uses time proximity, feature similarity, or hybrid metrics for importance measurement. Includes drift prevention methods like early-established endpoints, adjusted sampling orders, and discrete history representation.

Result: The method enables inference with thousands of frames and training with large batch sizes. Ablation studies validate the effectiveness of anti-drifting methods in both single-directional and bi-directional video generation. Existing video diffusion models can be finetuned with FramePack.

Conclusion: FramePack provides an effective framework for video generation that handles long sequences through importance-based frame packing and successfully addresses drift issues through specialized prevention methods.

Abstract: We present a neural network structure, FramePack, to train next-frame (or next-frame-section) prediction models for video generation. FramePack compresses input frame contexts with frame-wise importance so that more frames can be encoded within a fixed context length, with more important frames having longer contexts. The frame importance can be measured using time proximity, feature similarity, or hybrid metrics. The packing method allows for inference with thousands of frames and training with relatively large batch sizes. We also present drift prevention methods to address observation bias (error accumulation), including early-established endpoints, adjusted sampling orders, and discrete history representation. Ablation studies validate the effectiveness of the anti-drifting methods in both single-directional video streaming and bi-directional video generation. Finally, we show that existing video diffusion models can be finetuned with FramePack, and analyze the differences between different packing schedules.

Method: Created Mirror dataset with client statements paired with facial images. Trained vision language models to analyze facial cues, infer emotions, and generate empathetic responses to manage resistance.

Result: Mirror significantly enhances AI therapist’s ability to handle resistance, outperforming text-based CBT approaches. Human expert evaluations confirm effectiveness in managing resistance and fostering therapeutic alliance.

Conclusion: Multimodal approach incorporating facial cues improves AI therapy effectiveness in handling client resistance and building therapeutic alliance compared to text-only methods.

Abstract: Recent studies have explored the use of large language models (LLMs) in psychotherapy; however, text-based cognitive behavioral therapy (CBT) models often struggle with client resistance, which can weaken therapeutic alliance. To address this, we propose a multimodal approach that incorporates nonverbal cues, which allows the AI therapist to better align its responses with the client’s negative emotional state. Specifically, we introduce a new synthetic dataset, Mirror (Multimodal Interactive Rolling with Resistance), which is a novel synthetic dataset that pairs each client’s statements with corresponding facial images. Using this dataset, we train baseline vision language models (VLMs) so that they can analyze facial cues, infer emotions, and generate empathetic responses to effectively manage client resistance. These models are then evaluated in terms of both their counseling skills as a therapist, and the strength of therapeutic alliance in the presence of client resistance. Our results demonstrate that Mirror significantly enhances the AI therapist’s ability to handle resistance, which outperforms existing text-based CBT approaches. Human expert evaluations further confirm the effectiveness of our approach in managing client resistance and fostering therapeutic alliance.

Method: Proposed a simple phenology-informed sampling strategy and created SSL4Eco dataset from multi-date Sentinel-2 imagery. Trained an existing model with season-contrastive objective to learn representations that better capture vegetation seasonality.

Result: The model pretrained on SSL4Eco achieved state-of-the-art performance on 7 out of 8 downstream tasks spanning classification and regression, consistently outperforming representations learned from other datasets.

Conclusion: The straightforward phenology-informed sampling method significantly improves representation quality for ecological tasks, highlighting the critical importance of dataset construction in biodiversity mapping and remote sensing applications.

Abstract: With the exacerbation of the biodiversity and climate crises, macroecological pursuits such as global biodiversity mapping become more urgent. Remote sensing offers a wealth of Earth observation data for ecological studies, but the scarcity of labeled datasets remains a major challenge. Recently, self-supervised learning has enabled learning representations from unlabeled data, triggering the development of pretrained geospatial models with generalizable features. However, these models are often trained on datasets biased toward areas of high human activity, leaving entire ecological regions underrepresented. Additionally, while some datasets attempt to address seasonality through multi-date imagery, they typically follow calendar seasons rather than local phenological cycles. To better capture vegetation seasonality at a global scale, we propose a simple phenology-informed sampling strategy and introduce corresponding SSL4Eco, a multi-date Sentinel-2 dataset, on which we train an existing model with a season-contrastive objective. We compare representations learned from SSL4Eco against other datasets on diverse ecological downstream tasks and demonstrate that our straightforward sampling method consistently improves representation quality, highlighting the importance of dataset construction. The model pretrained on SSL4Eco reaches state of the art performance on 7 out of 8 downstream tasks spanning (multi-label) classification and regression. We release our code, data, and model weights to support macroecological and computer vision research at https://github.com/PlekhanovaElena/ssl4eco.

Method: The framework enhances W2S learning by: (1) integrating homogeneous and heterogeneous supervision from diverse VQA teachers via learn-to-rank formulation, and (2) iterative W2S training where strong students become teachers in subsequent cycles, progressively focusing on challenging cases.

Result: Extensive experiments show the method achieves state-of-the-art results across both in-domain and out-of-distribution benchmarks, with especially strong gains in OOD scenarios. The approach demonstrates a distinct weak-to-strong effect where strong students surpass weak teachers.

Conclusion: W2S learning provides a principled route to break annotation barriers and achieve scalable generalization in VQA, with implications extending to broader alignment and evaluation tasks beyond video quality assessment.

Abstract: Video quality assessment (VQA) seeks to predict the perceptual quality of a video in alignment with human visual perception, serving as a fundamental tool for quantifying quality degradation across video processing workflows. The dominant VQA paradigm relies on supervised training with human-labeled datasets, which, despite substantial progress, still suffers from poor generalization to unseen video content. Moreover, its reliance on human annotations – which are labor-intensive and costly – makes it difficult to scale datasets for improving model generalization. In this work, we explore weak-to-strong (W2S) learning as a new paradigm for advancing VQA without reliance on large-scale human-labeled datasets. We first provide empirical evidence that a straightforward W2S strategy allows a strong student model to not only match its weak teacher on in-domain benchmarks but also surpass it on out-of-distribution (OOD) benchmarks, revealing a distinct weak-to-strong effect in VQA. Building on this insight, we propose a novel framework that enhances W2S learning from two aspects: (1) integrating homogeneous and heterogeneous supervision signals from diverse VQA teachers – including off-the-shelf VQA models and synthetic distortion simulators – via a learn-to-rank formulation, and (2) iterative W2S training, where each strong student is recycled as the teacher in subsequent cycles, progressively focusing on challenging cases. Extensive experiments show that our method achieves state-of-the-art results across both in-domain and OOD benchmarks, with especially strong gains in OOD scenarios. Our findings highlight W2S learning as a principled route to break annotation barriers and achieve scalable generalization in VQA, with implications extending to broader alignment and evaluation tasks.

Method: Three-stage training: (1) misinformation long-Chain-of-Thought instruction tuning, (2) preference alignment via Direct Preference Optimization, (3) Group Relative Policy Optimization using verifiable reward function.

Result: Fact-R1 exhibits emergent reasoning behaviors comparable to advanced text-based reinforcement learning systems, but in the more complex multimodal misinformation setting.

Conclusion: Establishes a new paradigm for misinformation detection that bridges large-scale video understanding, reasoning-guided alignment, and interpretable verification.

Abstract: The rapid spread of multimodal misinformation on social media has raised growing concerns, while research on video misinformation detection remains limited due to the lack of large-scale, diverse datasets. Existing methods often overfit to rigid templates and lack deep reasoning over deceptive content. To address these challenges, we introduce FakeVV, a large-scale benchmark comprising over 100,000 video-text pairs with fine-grained, interpretable annotations. In addition, we further propose Fact-R1, a novel framework that integrates deep reasoning with collaborative rule-based reinforcement learning. Fact-R1 is trained through a three-stage process: (1) misinformation long-Chain-of-Thought (CoT) instruction tuning, (2) preference alignment via Direct Preference Optimization (DPO), and (3) Group Relative Policy Optimization (GRPO) using a novel verifiable reward function. This enables Fact-R1 to exhibit emergent reasoning behaviors comparable to those observed in advanced text-based reinforcement learning systems, but in the more complex multimodal misinformation setting. Our work establishes a new paradigm for misinformation detection, bridging large-scale video understanding, reasoning-guided alignment, and interpretable verification.

Method: RealEngine reconstructs background scenes and foreground traffic participants separately using real-world multi-modal sensor data, then combines them through flexible scene composition. It leverages scene reconstruction and view synthesis techniques for photorealistic rendering across multiple sensor modalities.

Result: The framework achieves photorealistic rendering with both perceptual fidelity and geometric accuracy, supporting three essential simulation categories: non-reactive simulation, safety testing, and multi-agent interaction.

Conclusion: RealEngine provides a reliable and comprehensive benchmark for evaluating real-world performance of driving agents by holistically addressing the fundamental requirements missing in existing simulators.

Abstract: Driving simulation plays a crucial role in developing reliable driving agents by providing controlled, evaluative environments. To enable meaningful assessments, a high-quality driving simulator must satisfy several key requirements: multi-modal sensing capabilities (e.g., camera and LiDAR) with realistic scene rendering to minimize observational discrepancies; closed-loop evaluation to support free-form trajectory behaviors; highly diverse traffic scenarios for thorough evaluation; multi-agent cooperation to capture interaction dynamics; and high computational efficiency to ensure affordability and scalability. However, existing simulators and benchmarks fail to comprehensively meet these fundamental criteria. To bridge this gap, this paper introduces RealEngine, a novel driving simulation framework that holistically integrates 3D scene reconstruction and novel view synthesis techniques to achieve realistic and flexible closed-loop simulation in the driving context. By leveraging real-world multi-modal sensor data, RealEngine reconstructs background scenes and foreground traffic participants separately, allowing for highly diverse and realistic traffic scenarios through flexible scene composition. This synergistic fusion of scene reconstruction and view synthesis enables photorealistic rendering across multiple sensor modalities, ensuring both perceptual fidelity and geometric accuracy. Building upon this environment, RealEngine supports three essential driving simulation categories: non-reactive simulation, safety testing, and multi-agent interaction, collectively forming a reliable and comprehensive benchmark for evaluating the real-world performance of driving agents.

Method: Developed DIP-R1 framework with three rule-based rewards: 1) standard reasoning reward for three-step process (comprehend scene, observe ambiguous regions, decision-making), 2) variance-guided looking reward to examine uncertain regions, and 3) weighted precision-recall accuracy reward for decision-making.

Result: Achieves consistent and significant improvement across diverse fine-grained object detection datasets with challenging real-world scenes, outperforming existing baselines and SFT methods in both in-domain and out-of-domain scenarios.

Conclusion: Integration of RL into MLLMs shows substantial potential for enhancing capabilities in complex real-world perception tasks.

Abstract: MLLMs have demonstrated significant visual understanding capabilities, yet their fine-grained visual perception in complex real-world scenarios, such as densely crowded public areas, remains limited. Inspired by the recent success of RL in both LLMs and MLLMs, in this paper, we explore how RL can enhance visual perception ability of MLLMs. Then we develop a novel RL-based framework, Deep Inspection and Perception with RL (DIP-R1) designed to enhance the visual perception capabilities of MLLMs, by comprehending complex scenes and looking through visual instances closely. DIP-R1 guides MLLMs through detailed inspection of visual scene via three simply designed rule-based reward modeling. First, we adopt a standard reasoning reward encouraging the model to include three-step reasoning process: 1) comprehending entire visual scene, 2) observing for looking through interested but ambiguous regions, and 3) decision-making for predicting answer. Second, a variance-guided looking reward is designed to encourage MLLM to examine uncertain regions during the observing process, guiding it to inspect ambiguous areas and mitigate perceptual uncertainty. This reward promotes variance-driven visual exploration, enabling MLLM to reason about region-level uncertainty and explicitly indicate interpretable uncertain regions. Third, we model a weighted precision-recall accuracy reward enhancing accurate decision-making. We verify its effectiveness across diverse fine-grained object detection data consisting of challenging real-world scenes, such as densely crowded scenes. Built upon existing MLLMs, DIP-R1 achieves consistent and significant improvement across various in-domain and out-of-domain scenarios, outperforming various existing baselines and SFT method. Our findings highlight the substantial potential of integrating RL into MLLMs for enhancing capabilities in complex real-world perception tasks.

Method: Analyzed the middle block of a fully-convolutional unconditional UNet, developed a stochastic reconstruction algorithm that uses the model’s own representation to guide synthesis.

Result: Found that the UNet decomposes images into sparse subsets of active channels, and spatial averages of these channels form a nonlinear representation where Euclidean distances are semantically meaningful without conditioning.

Conclusion: Semantic similarity emerges unsupervised solely from the denoising objective in diffusion models.

Abstract: Generative diffusion models learn probability densities over diverse image datasets by estimating the score with a neural network trained to remove noise. Despite their remarkable success in generating high-quality images, the internal mechanisms of the underlying score networks are not well understood. Here, we examine the image representation that arises from score estimation in a {fully-convolutional unconditional UNet}. We show that the middle block of the UNet decomposes individual images into sparse subsets of active channels, and that the vector of spatial averages of these channels can provide a nonlinear representation of the underlying clean images. Euclidean distances in this representation space are semantically meaningful, even though no conditioning information is provided during training. We develop a novel algorithm for stochastic reconstruction of images conditioned on this representation: The synthesis using the unconditional model is “self-guided” by the representation extracted from that very same model. For a given representation, the common patterns in the set of reconstructed samples reveal the features captured in the middle block of the UNet. Together, these results show, for the first time, that a measure of semantic similarity emerges, unsupervised, solely from the denoising objective.

Method: Collected 20 weight-loaded fitness actions from 38 subjects across 3 skill levels using high-precision MoCap, capturing 5 RGB views, 3D pose, sEMG signals, and physiological data. Incorporated knowledge graphs with penalty functions mapping actions, keysteps, error types, and feedback.

Result: Demonstrated that multimodal data, multiview data, and fine-grained annotations significantly enhance model performance in fitness AQA tasks.

Conclusion: FLEX advances AQA methodologies toward multi-modal and multi-action scenarios, fostering AI integration in the fitness domain and providing a comprehensive benchmark for fitness action assessment.

Abstract: With the increasing awareness of health and the growing desire for aesthetic physique, fitness has become a prevailing trend. However, the potential risks associated with fitness training, especially with weight-loaded fitness actions, cannot be overlooked. Action Quality Assessment (AQA), a technology that quantifies the quality of human action and provides feedback, holds the potential to assist fitness enthusiasts of varying skill levels in achieving better training outcomes. Nevertheless, current AQA methodologies and datasets are limited to single-view competitive sports scenarios and RGB modality and lack professional assessment and guidance of fitness actions. To address this gap, we propose the FLEX dataset, the first multi-modal, multi-action, large-scale dataset that incorporates surface electromyography (sEMG) signals into AQA. FLEX utilizes high-precision MoCap to collect 20 different weight-loaded actions performed by 38 subjects across 3 different skill levels for 10 repetitions each, containing 5 different views of the RGB video, 3D pose, sEMG, and physiological information. Additionally, FLEX incorporates knowledge graphs into AQA, constructing annotation rules in the form of penalty functions that map weight-loaded actions, action keysteps, error types, and feedback. We conducted various baseline methodologies on FLEX, demonstrating that multimodal data, multiview data, and fine-grained annotations significantly enhance model performance. FLEX not only advances AQA methodologies and datasets towards multi-modal and multi-action scenarios but also fosters the integration of artificial intelligence within the fitness domain. Dataset and code are available at https://haoyin116.github.io/FLEX_Dataset.

Method: Employed self-supervised training methods to develop AquaCluster model that segments radar satellite images without manual annotations.

Result: Outperformed other radar-based water detection techniques that don’t require annotated data, achieving 0.08 improvement in Intersection over Union metric.

Conclusion: It’s possible to train machine learning models to detect vegetated water from radar images without annotated data, making model retraining for changes much easier.

Abstract: In recent years, the wide availability of high-resolution radar satellite images has enabled the remote monitoring of wetland surface areas. Machine learning models have achieved state-of-the-art results in segmenting wetlands from satellite images. However, these models require large amounts of manually annotated satellite images, which are slow and expensive to produce. The need for annotated training data makes it difficult to adapt these models to changes such as different climates or sensors. To address this issue, we employed self-supervised training methods to develop a model, AquaCluster, which segments radar satellite images into water and land areas without manual annotations. Our final model outperformed other radar-based water detection techniques that do not require annotated data in our test dataset, having achieved a 0.08 improvement in the Intersection over Union metric. Our results demonstrate that it is possible to train machine learning models to detect vegetated water from radar images without the use of annotated data, which can make the retraining of these models to account for changes much easier.

Method: Proposes: 1) relation-aware lane detector with geometry-biased self-attention and curve cross-attention, 2) relation-enhanced topology heads including geometry-enhanced L2L head and cross-view L2T head, 3) contrastive learning strategy with InfoNCE loss to regularize relationship embeddings.

Result: Significant improvements on OpenLane-V2 benchmark: +3.1 in DET_l, +5.3 in TOP_ll, +4.9 in TOP_lt, and +4.4 in overall OLS, setting new state-of-the-art.

Conclusion: Relational modeling jointly enhances both perception and reasoning for road topology, demonstrating that capturing contextual relationships among road elements is crucial for accurate autonomous driving systems.

Abstract: Accurate road topology reasoning is critical for autonomous driving, enabling effective navigation and adherence to traffic regulations. Central to this task are lane perception and topology reasoning. However, existing methods typically focus on either lane detection or Lane-to-Lane (L2L) topology reasoning, often \textit{neglecting} Lane-to-Traffic-element (L2T) relationships or \textit{failing} to optimize these tasks jointly. Furthermore, most approaches either overlook relational modeling or apply it in a limited scope, despite the inherent spatial relationships among road elements. We argue that relational modeling is beneficial for both perception and reasoning, as humans naturally leverage contextual relationships for road element recognition and their connectivity inference. To this end, we introduce relational modeling into both perception and reasoning, \textit{jointly} enhancing structural understanding. Specifically, we propose: 1) a relation-aware lane detector, where our geometry-biased self-attention and \curve\ cross-attention refine lane representations by capturing relational dependencies; 2) relation-enhanced topology heads, including a geometry-enhanced L2L head and a cross-view L2T head, boosting reasoning with relational cues; and 3) a contrastive learning strategy with InfoNCE loss to regularize relationship embeddings. Extensive experiments on OpenLane-V2 demonstrate that our approach significantly improves both detection and topology reasoning metrics, achieving +3.1 in DET$l$, +5.3 in TOP${ll}$, +4.9 in TOP$_{lt}$, and an overall +4.4 in OLS, setting a new state-of-the-art. Code will be released.

Method: Developed a framework to collect large-scale REA dataset and proposed two STLLM baselines: STLLM-3D (direct fusion of point cloud, video, and text) and STLLM-Aligner (alignment of spatial context with video and text before LLM decoding).

Result: The STLLM baselines outperform existing models on the REA dataset, demonstrating enhanced spatial understanding and temporal grounding capabilities.

Conclusion: The proposed STLLM approaches effectively address spatio-temporal reasoning challenges in MLLMs, showing improved performance on the newly created REA benchmark dataset.

Abstract: Despite significant recent progress of Multimodal Large Language Models (MLLMs), current MLLMs are challenged by “spatio-temporal” prompts, i.e., prompts that refer to 1) the entirety of an environment encoded in a point cloud that the MLLM should consider; and simultaneously also refer to 2) actions that happened in part of the environment and are encoded in a short ego-centric video clip. However, such a holistic spatio-temporal understanding is important for agents operating in the real world. To address this challenge, we first develop a framework to collect a large-scale dataset. Using the collected “Reasoning about Environments and Actions” (REA) dataset, we show that recent MLLMs indeed struggle to correctly answer “spatio-temporal” prompts. Building on this dataset, we study two spatio-temporal LLM (STLLM) baselines: 1) STLLM-3D, which directly fuses point cloud, video, and text representations as inputs to the LLM; and 2) STLLM-Aligner, which aligns spatial context with video and text before LLM decoding. Both baselines aim to enhance spatial understanding of environments and temporal grounding of egocentric observations. On REA, the STLLM baselines outperform existing models, demonstrating the effectiveness of our designs. Code and data are available at https://zoezheng126.github.io/STLLM-website/.

Method: Proposed Generative HMC (GenHMC) that leverages large unpaired HMC captures to directly generate synthetic HMC images from conditioning avatar states obtained from dome captures, enabling proper disentanglement of facial expression and viewpoint from appearance.

Result: The method achieves more accurate ground truth, generalizes to unseen identities, removes reliance on paired captures, and enables training of universal face encoders with better data efficiency and state-of-the-art accuracy.

Conclusion: GenHMC provides a breakthrough approach for photorealistic avatar animation by using generative methods with unpaired data, overcoming the operational expense and limitations of traditional paired capture methods.

Abstract: Enabling photorealistic avatar animations in virtual and augmented reality (VR/AR) has been challenging because of the difficulty of obtaining ground truth state of faces. It is physically impossible to obtain synchronized images from head-mounted cameras (HMC) sensing input, which has partial observations in infrared (IR), and an array of outside-in dome cameras, which have full observations that match avatars’ appearance. Prior works relying on analysis-by-synthesis methods could generate accurate ground truth, but suffer from imperfect disentanglement between expression and style in their personalized training. The reliance of extensive paired captures (HMC and dome) for the same subject makes it operationally expensive to collect large-scale datasets, which cannot be reused for different HMC viewpoints and lighting. In this work, we propose a novel generative approach, Generative HMC (GenHMC), that leverages large unpaired HMC captures, which are much easier to collect, to directly generate high-quality synthetic HMC images given any conditioning avatar state from dome captures. We show that our method is able to properly disentangle the input conditioning signal that specifies facial expression and viewpoint, from facial appearance, leading to more accurate ground truth. Furthermore, our method can generalize to unseen identities, removing the reliance on the paired captures. We demonstrate these breakthroughs by both evaluating synthetic HMC images and universal face encoders trained from these new HMC-avatar correspondences, which achieve better data efficiency and state-of-the-art accuracy.

Method: Used two complementary methods: prompt-based evaluation using probabilities as preference measure, and Grad-CAM for interpreting visual attention in shape-word matching tasks.

Result: Models do not consistently show bouba-kiki effect; ResNet shows some preference for round shapes but overall performance lacks expected associations; models’ responses fall short of human performance on same task.

Conclusion: VLMs have limitations in cross-modal concept understanding and their internal representations do not align well with human intuitions.

Abstract: Recent advances in multimodal models have raised questions about whether vision-and-language models (VLMs) integrate cross-modal information in ways that reflect human cognition. One well-studied test case in this domain is the bouba-kiki effect, where humans reliably associate pseudowords like bouba' with round shapes and kiki’ with jagged ones. Given the mixed evidence found in prior studies for this effect in VLMs, we present a comprehensive re-evaluation focused on two variants of CLIP, ResNet and Vision Transformer (ViT), given their centrality in many state-of-the-art VLMs. We apply two complementary methods closely modelled after human experiments: a prompt-based evaluation that uses probabilities as a measure of model preference, and we use Grad-CAM as a novel approach to interpret visual attention in shape-word matching tasks. Our findings show that these model variants do not consistently exhibit the bouba-kiki effect. While ResNet shows a preference for round shapes, overall performance across both model variants lacks the expected associations. Moreover, direct comparison with prior human data on the same task shows that the models’ responses fall markedly short of the robust, modality-integrated behaviour characteristic of human cognition. These results contribute to the ongoing debate about the extent to which VLMs truly understand cross-modal concepts, highlighting limitations in their internal representations and alignment with human intuitions.

Method: Four-component approach: 1) Dynamic patch selection via quantile-based thresholding, 2) Graph-guided k-means clustering for phenotype heterogeneity, 3) Attention mechanisms for intra- and inter-cluster relationships, 4) Expert-guided Gaussian mixture models for survival distribution estimation.

Result: Achieved concordance index of 0.712 ± 0.028 and Brier score of 0.254 ± 0.018 on TCGA-KIRC (renal cancer), and 0.645 ± 0.017 concordance index and 0.281 ± 0.031 Brier score on TCGA-LUAD (lung adenocarcinoma).

Conclusion: The proposed method significantly outperforms state-of-the-art approaches and demonstrates strong predictive potential across diverse cancer types.

Abstract: We introduce a modular framework for predicting cancer-specific survival from whole slide pathology images (WSIs) that significantly improves upon the state-of-the-art accuracy. Our method integrating four key components. Firstly, to tackle large size of WSIs, we use dynamic patch selection via quantile-based thresholding for isolating prognostically informative tissue regions. Secondly, we use graph-guided k-means clustering to capture phenotype-level heterogeneity through spatial and morphological coherence. Thirdly, we use attention mechanisms that model both intra- and inter-cluster relationships to contextualize local features within global spatial relations between various types of tissue compartments. Finally, we use an expert-guided mixture density modeling for estimating complex survival distributions using Gaussian mixture models. The proposed model achieves a concordance index of $0.712 \pm 0.028$ and Brier score of $0.254 \pm 0.018$ on TCGA-KIRC (renal cancer), and a concordance index of $0.645 \pm 0.017$ and Brier score of $0.281 \pm 0.031$ on TCGA-LUAD (lung adenocarcinoma). These results are significantly better than the state-of-art and demonstrate predictive potential of the proposed method across diverse cancer types.

Method: Systematic review from three perspectives: data (public dataset acquisition), methods (monocular and stereo deep learning approaches), and applications (clinical implementation challenges and solutions).

Result: Provides a thorough state-of-the-art literature survey that identifies specific challenges and corresponding solutions for clinical implementation in concrete scenarios.

Conclusion: Outlines future research directions including domain adaptation, real-time implementation, and fusion of depth information with sensor technologies to advance the field toward clinical translation.

Abstract: Endoscopic depth estimation is a critical technology for improving the safety and precision of minimally invasive surgery. It has attracted considerable attention from researchers in medical imaging, computer vision, and robotics. Over the past decade, a large number of methods have been developed. Despite the existence of several related surveys, a comprehensive overview focusing on recent deep learning-based techniques is still limited. This paper endeavors to bridge this gap by systematically reviewing the state-of-the-art literature. Specifically, we provide a thorough survey of the field from three key perspectives: data, methods, and applications. Firstly, at the data level, we describe the acquisition process of publicly available datasets. Secondly, at the methodological level, we introduce both monocular and stereo deep learning-based approaches for endoscopic depth estimation. Thirdly, at the application level, we identify the specific challenges and corresponding solutions for the clinical implementation of depth estimation technology, situated within concrete clinical scenarios. Finally, we outline potential directions for future research, such as domain adaptation, real-time implementation, and the synergistic fusion of depth information with sensor technologies, thereby providing a valuable starting point for researchers to engage with and advance the field toward clinical translation.

Method: TempFlow-GRPO introduces three innovations: trajectory branching mechanism for process rewards, noise-aware weighting scheme for temporal optimization, and seed group strategy to control initialization effects.

Result: The framework achieves state-of-the-art performance in human preference alignment and text-to-image benchmarks by enabling temporally-aware optimization that respects generative dynamics.

Conclusion: TempFlow-GRPO successfully addresses the temporal uniformity problem in flow model training through principled temporal structure exploitation, leading to more efficient exploration and better convergence in human preference alignment.

Abstract: Recent flow matching models for text-to-image generation have achieved remarkable quality, yet their integration with reinforcement learning for human preference alignment remains suboptimal, hindering fine-grained reward-based optimization. We observe that the key impediment to effective GRPO training of flow models is the temporal uniformity assumption in existing approaches: sparse terminal rewards with uniform credit assignment fail to capture the varying criticality of decisions across generation timesteps, resulting in inefficient exploration and suboptimal convergence. To remedy this shortcoming, we introduce \textbf{TempFlow-GRPO} (Temporal Flow GRPO), a principled GRPO framework that captures and exploits the temporal structure inherent in flow-based generation. TempFlow-GRPO introduces three key innovations: (i) a trajectory branching mechanism that provides process rewards by concentrating stochasticity at designated branching points, enabling precise credit assignment without requiring specialized intermediate reward models; (ii) a noise-aware weighting scheme that modulates policy optimization according to the intrinsic exploration potential of each timestep, prioritizing learning during high-impact early stages while ensuring stable refinement in later phases; and (iii) a seed group strategy that controls for initialization effects to isolate exploration contributions. These innovations endow the model with temporally-aware optimization that respects the underlying generative dynamics, leading to state-of-the-art performance in human preference alignment and text-to-image benchmarks.

Method: GLSim leverages complementary global and local embedding similarity signals between image and text modalities without requiring training.

Result: GLSim achieves superior detection performance, significantly outperforming competitive baselines in comprehensive benchmarking.

Conclusion: The proposed GLSim framework provides more accurate and reliable object hallucination detection by integrating both global and local perspectives.

Abstract: Object hallucination in large vision-language models presents a significant challenge to their safe deployment in real-world applications. Recent works have proposed object-level hallucination scores to estimate the likelihood of object hallucination; however, these methods typically adopt either a global or local perspective in isolation, which may limit detection reliability. In this paper, we introduce GLSim, a novel training-free object hallucination detection framework that leverages complementary global and local embedding similarity signals between image and text modalities, enabling more accurate and reliable hallucination detection in diverse scenarios. We comprehensively benchmark existing object hallucination detection methods and demonstrate that GLSim achieves superior detection performance, outperforming competitive baselines by a significant margin.

Method: Benchmarked state-of-the-art waste classification models (ResNeXt-50, EfficientNet-B0, MobileNetV3-S, YOLOv8-n, YOLOv5-s) using stratified 5-fold cross-validation on combined HCW data, followed by ANOVA testing for statistical significance.

Result: YOLOv5-s achieved the highest accuracy (95.06%) with competitive inference speed, while EfficientNet-B0 showed promising accuracy (93.22%) but had the highest inference time. Statistical significance was confirmed through ANOVA testing.

Conclusion: The best performing model (YOLOv5-s) was successfully deployed to a web application with bin color mapping according to Nepal’s HCW management standards, though data limitations and localized context need further attention.

Abstract: The increasing number of Health Care facilities in Nepal has added up the challenges on managing health care waste (HCW). Improper segregation and disposal of HCW leads to contamination, spreading of infectious diseases and risk for waste handlers. This study benchmarks the state of the art waste classification models: ResNeXt-50, EfficientNet-B0, MobileNetV3-S, YOLOv8-n and YOLOv5-s using stratified 5-fold cross-validation technique on combined HCW data. YOLOv5-s achieved the highest accuracy (95.06%) but fell short with the YOLOv8-n model in inference speed with few milliseconds. The EfficientNet-B0 showed promising results of 93.22% accuracy but took the highest inference time. Following a repetitive ANOVA test to confirm the statistical significance, the best performing model (YOLOv5-s) was deployed to the web with bin color mapped using Nepal’s HCW management standards. Further work is suggested to address data limitation and ensure localized context.

Method: Dual-branch architecture equivariant to dihedral group, with specialized branches for reflection and rotation symmetry. Uses orientational anchors and reflectional matching for reflection symmetry, and rotational matching with fixed angular intervals for rotational symmetry.

Result: Extensive experiments demonstrate state-of-the-art performance, outperforming existing approaches in symmetry detection.

Conclusion: The proposed framework effectively detects axis-level symmetry by leveraging geometric primitives and dihedral group-equivariant features, achieving superior performance in detecting both reflection and rotation symmetry types.

Abstract: Symmetry is a fundamental concept that has been extensively studied, yet detecting it in complex scenes remains a significant challenge in computer vision. Recent heatmap-based approaches can localize potential regions of symmetry axes but often lack precision in identifying individual axes. In this work, we propose a novel framework for axis-level detection of the two most common symmetry types-reflection and rotation-by representing them as explicit geometric primitives, i.e. lines and points. Our method employs a dual-branch architecture that is equivariant to the dihedral group, with each branch specialized to exploit the structure of dihedral group-equivariant features for its respective symmetry type. For reflection symmetry, we introduce orientational anchors, aligned with group components, to enable orientation-specific detection, and a reflectional matching that measures similarity between patterns and their mirrored counterparts across candidate axes. For rotational symmetry, we propose a rotational matching that compares patterns at fixed angular intervals to identify rotational centers. Extensive experiments demonstrate that our method achieves state-of-the-art performance, outperforming existing approaches.

Method: 1) Baseline DIO models the causal chain: image → attributes → progressive patterns → consistency → answer. 2) Brando introduces trainable negative options to tighten variational bounds. 3) WORLD replaces generation with Gaussian-mixture feature model for infinite weighted negatives. 4) DIEGO adds metadata supervision to bridge the semantic gap between attributes and patterns.

Result: The refinements substantially boost discriminative RPM accuracy and enable DIO to generate valid answers in open-ended RPM for the first time.

Conclusion: The work provides causal-driven design guidelines, objective-refinement strategies and cross-modal insights for abstract-reasoning research, successfully addressing the reasoning bottleneck through systematic model improvements.

Abstract: Despite deep learning’s broad success, its abstract-reasoning bottleneck persists. We tackle Raven’s Progressive Matrices (RPM), the benchmark for pattern, reasoning and problem-solving intelligence. We model the full causal chain image $\rightarrow$ attributes $\rightarrow$ progressive patterns $\rightarrow$ consistency $\rightarrow$ answer and build the baseline DIO. Yet DIO’s mutual-information lower-bound objective does not embed human logic: the bound is loose and statistic-based, ignoring causal subject-object links. We therefore present three refinements. 1) Brando introduces trainable negative options to tighten the variational bound. 2) WORLD replaces generation with a Gaussian-mixture feature model that supplies infinite, weighted negatives, further tightening the bound. 3) DIEGO adds metadata supervision to rectify the “attributes $\rightarrow$ patterns” semantic gap, aligning representations with human rules. These upgrades substantially boost discriminative RPM accuracy and, for the first time, let DIO generate valid answers in open-ended RPM. The work provides causal-driven design guidelines, objective-refinement strategies and cross-modal insights for abstract-reasoning research.

Method: Reframes TTA as Gaussian probabilistic inference using gradually updated class means and shared covariance matrix, with lightweight regularization from CLIP priors and historical knowledge bank - no backpropagation required.

Result: Achieves state-of-the-art performance across diverse benchmarks under various distribution shifts with superior scalability and robustness, supporting both online and transductive settings.

Conclusion: ADAPT provides an effective backpropagation-free TTA approach that explicitly models class distributions through Gaussian inference, enabling scalable and robust adaptation without source data or gradient updates.

Abstract: Test-time adaptation (TTA) enhances the zero-shot robustness under distribution shifts by leveraging unlabeled test data during inference. Despite notable advances, several challenges still limit its broader applicability. First, most methods rely on backpropagation or iterative optimization, which limits scalability and hinders real-time deployment. Second, they lack explicit modeling of class-conditional feature distributions. This modeling is crucial for producing reliable decision boundaries and calibrated predictions, but it remains underexplored due to the lack of both source data and supervision at test time. In this paper, we propose ADAPT, an Advanced Distribution-Aware and backPropagation-free Test-time adaptation method. We reframe TTA as a Gaussian probabilistic inference task by modeling class-conditional likelihoods using gradually updated class means and a shared covariance matrix. This enables closed-form, training-free inference. To correct potential likelihood bias, we introduce lightweight regularization guided by CLIP priors and a historical knowledge bank. ADAPT requires no source data, no gradient updates, and no full access to target data, supporting both online and transductive settings. Extensive experiments across diverse benchmarks demonstrate that our method achieves state-of-the-art performance under a wide range of distribution shifts with superior scalability and robustness.

Method: Proposed UnorthoDOS approach using unorthorectified hyperspectral data from EMIT sensor, comparing ML models trained on both unorthorectified and orthorectified datasets against matched filter baseline.

Result: ML models on unorthorectified data perform comparably to orthorectified data, and both outperform the traditional matched filter (mag1c) baseline for methane plume detection.

Conclusion: Unorthorectified data can effectively replace orthorectified data for methane detection, enabling faster onboard satellite processing and reduced downlink costs while maintaining detection accuracy.

Abstract: Methane is a potent greenhouse gas and a major driver of climate change, making its timely detection critical for effective mitigation. Machine learning (ML) deployed onboard satellites can enable rapid detection while reducing downlink costs, supporting faster response systems. Conventional methane detection methods often rely on image processing techniques, such as orthorectification to correct geometric distortions and matched filters to enhance plume signals. We introduce a novel approach that bypasses these preprocessing steps by using \textit{unorthorectified} data (UnorthoDOS). We find that ML models trained on this dataset achieve performance comparable to those trained on orthorectified data. Moreover, we also train models on an orthorectified dataset, showing that they can outperform the matched filter baseline (mag1c). We release model checkpoints and two ML-ready datasets comprising orthorectified and unorthorectified hyperspectral images from the Earth Surface Mineral Dust Source Investigation (EMIT) sensor at https://huggingface.co/datasets/SpaceML/UnorthoDOS , along with code at https://github.com/spaceml-org/plume-hunter.

Method: Automatically synthesizes multimodal trajectories (M-TRACE dataset with 28.5K tasks and 177K trajectories) and generates preference pairs (Pref-X with 11K pairs), then trains VLM controllers via imitation-based trajectory tuning and step-wise preference learning.

Result: MATRIX consistently surpasses both open- and closed-source VLMs across three benchmarks (Agent-X, GTA, and GAIA), demonstrating scalable and effective multimodal tool use.

Conclusion: The framework enables scalable and effective multimodal tool use through automated trajectory synthesis and preference learning, with publicly available data and code.

Abstract: Vision language models (VLMs) are increasingly deployed as controllers with access to external tools for complex reasoning and decision-making, yet their effectiveness remains limited by the scarcity of high-quality multimodal trajectories and the cost of manual annotation. We address this challenge with a vision-centric agent tuning framework that automatically synthesizes multimodal trajectories, generates step-wise preference pairs, and trains a VLM controller for robust tool-use reasoning. Our pipeline first constructs M-TRACE, a large-scale dataset of 28.5K multimodal tasks with 177K verified trajectories, enabling imitation-based trajectory tuning. Building on this, we develop MATRIX Agent, a controller finetuned on M-TRACE for step-wise tool reasoning. To achieve finer alignment, we further introduce Pref-X, a set of 11K automatically generated preference pairs, and optimize MATRIX on it via step-wise preference learning. Across three benchmarks, Agent-X, GTA, and GAIA, MATRIX consistently surpasses both open- and closed-source VLMs, demonstrating scalable and effective multimodal tool use. Our data and code is avaliable at https://github.com/mbzuai-oryx/MATRIX.

Method: Revisits plain ViTs with multi-scale token aggregation, performs domain-adaptive pretraining on CT-3M dataset (3.87M CT slices) using multi-stage DINOv3 recipe to learn robust dense features.

Result: MedDINOv3 matches or exceeds state-of-the-art performance across four segmentation benchmarks, demonstrating foundation models can serve as unified backbones for medical image segmentation.

Conclusion: The framework successfully adapts vision foundation models to medical imaging, overcoming domain gap challenges and enabling generalized segmentation performance across different medical imaging tasks.

Abstract: Accurate segmentation of organs and tumors in CT and MRI scans is essential for diagnosis, treatment planning, and disease monitoring. While deep learning has advanced automated segmentation, most models remain task-specific, lacking generalizability across modalities and institutions. Vision foundation models (FMs) pretrained on billion-scale natural images offer powerful and transferable representations. However, adapting them to medical imaging faces two key challenges: (1) the ViT backbone of most foundation models still underperform specialized CNNs on medical image segmentation, and (2) the large domain gap between natural and medical images limits transferability. We introduce MedDINOv3, a simple and effective framework for adapting DINOv3 to medical segmentation. We first revisit plain ViTs and design a simple and effective architecture with multi-scale token aggregation. Then, we perform domain-adaptive pretraining on CT-3M, a curated collection of 3.87M axial CT slices, using a multi-stage DINOv3 recipe to learn robust dense features. MedDINOv3 matches or exceeds state-of-the-art performance across four segmentation benchmarks, demonstrating the potential of vision foundation models as unified backbones for medical image segmentation. The code is available at https://github.com/ricklisz/MedDINOv3.

Method: Created psychophysics-inspired benchmarks for Chinese logographs and English alphabetic words by splicing, recombining, and overlaying glyphs to create ‘visible but unreadable’ stimuli for models while remaining legible to humans.

Result: Contemporary VLMs show severe performance drops under these perturbations, frequently producing unrelated or incoherent outputs, despite strong performance on clean text.

Conclusion: Models rely heavily on generic visual invariances but underutilize compositional priors needed for robust literacy, motivating new architectures and training strategies for symbol segmentation, composition, and binding across scripts.

Abstract: Writing is a universal cultural technology that reuses vision for symbolic communication. Humans display striking resilience: we readily recognize words even when characters are fragmented, fused, or partially occluded. This paper investigates whether advanced vision language models (VLMs) share this resilience. We construct two psychophysics inspired benchmarks across distinct writing systems, Chinese logographs and English alphabetic words, by splicing, recombining, and overlaying glyphs to yield ‘‘visible but unreadable’’ stimuli for models while remaining legible to humans. Despite strong performance on clean text, contemporary VLMs show a severe drop under these perturbations, frequently producing unrelated or incoherent outputs. The pattern suggests a structural limitation: models heavily leverage generic visual invariances but under rely on compositional priors needed for robust literacy. We release stimuli generation code, prompts, and evaluation protocols to facilitate transparent replication and follow up work. Our findings motivate architectures and training strategies that encode symbol segmentation, composition, and binding across scripts, and they delineate concrete challenges for deploying multimodal systems in education, accessibility, cultural heritage, and security.

Method: Proposes two innovations: 1) Converting structured metadata into synthetic reports for better image-text alignment, and 2) Adding an auxiliary token trained to predict MMSE scores for additional supervision. Uses lightweight prompt tuning on both image and text modalities.

Result: Achieves state-of-the-art performance on ADNI with only 1,504 training MRIs, outperforming methods trained on 27,161 MRIs, and shows strong zero-shot generalization on OASIS-2 and AIBL datasets.

Conclusion: The proposed data-efficient fine-tuning approach successfully adapts 3D CT-based Med-VLMs for 3D MRI in AD diagnosis, demonstrating superior performance with minimal training data and strong generalization capabilities.

Abstract: Medical vision-language models (Med-VLMs) have shown impressive results in tasks such as report generation and visual question answering, but they still face several limitations. Most notably, they underutilize patient metadata and lack integration of clinical diagnostic knowledge. Moreover, most existing models are typically trained from scratch or fine-tuned on large-scale 2D image-text pairs, requiring extensive computational resources, and their effectiveness on 3D medical imaging is often limited due to the absence of structural information. To address these gaps, we propose a data-efficient fine-tuning pipeline to adapt 3D CT-based Med-VLMs for 3D MRI and demonstrate its application in Alzheimer’s disease (AD) diagnosis. Our system introduces two key innovations. First, we convert structured metadata into synthetic reports, enriching textual input for improved image-text alignment. Second, we add an auxiliary token trained to predict the mini-mental state examination (MMSE) score, a widely used clinical measure of cognitive function that correlates with AD severity. This provides additional supervision for fine-tuning. Applying lightweight prompt tuning to both image and text modalities, our approach achieves state-of-the-art performance on ADNI with only 1,504 training MRIs, outperforming methods trained on 27,161 MRIs, and shows strong zero-shot generalization on OASIS-2 and AIBL. Code is available at https://github.com/CFQ666312/DEFT-VLM-AD.

Method: Use a simple linear classifier on a modern Vision Foundation Model (VFM) trained on identical data as specialized detectors, and analyze text-image similarity probing to understand the source of VFM’s effectiveness.

Result: The VFM-based baseline decisively outperforms bespoke detectors, boosting in-the-wild accuracy by over 20%. Recent VLMs show learned alignment with forgery concepts like ‘AI-generated’, but this advantage disappears on novel datasets scraped after pre-training cut-off dates.

Conclusion: 1) Updated VFMs provide superior real-world detection capability compared to static specialized detectors. 2) True generalization evaluation requires test data independent of the model’s entire training history, including pre-training.

Abstract: While specialized detectors for AI-generated images excel on curated benchmarks, they fail catastrophically in real-world scenarios, as evidenced by their critically high false-negative rates on in-the-wild' benchmarks. Instead of crafting another specialized knife’ for this problem, we bring a gun' to the fight: a simple linear classifier on a modern Vision Foundation Model (VFM). Trained on identical data, this baseline decisively outguns’ bespoke detectors, boosting in-the-wild accuracy by a striking margin of over 20%. Our analysis pinpoints the source of the VFM’s firepower': First, by probing text-image similarities, we find that recent VLMs (e.g., Perception Encoder, Meta CLIP2) have learned to align synthetic images with forgery-related concepts (e.g., AI-generated’), unlike previous versions. Second, we speculate that this is due to data exposure, as both this alignment and overall accuracy plummet on a novel dataset scraped after the VFM’s pre-training cut-off date, ensuring it was unseen during pre-training. Our findings yield two critical conclusions: 1) For the real-world gunfight' of AI-generated image detection, the raw firepower’ of an updated VFM is far more effective than the `craftsmanship’ of a static detector. 2) True generalization evaluation requires test data to be independent of the model’s entire training history, including pre-training.

Method: Proposes an edge-centric formulation modeling wall segments as directed, geometrically continuous edges. Uses a dual-query transformer decoder with perturbed and latent queries in a denoising framework to stabilize optimization and accelerate convergence.

Result: Achieves state-of-the-art performance on Structured3D and SceneCAD datasets with F1 scores of 99.1% (rooms), 91.7% (corners), and 89.3% (angles). Demonstrates strong cross-dataset generalization.

Conclusion: CAGE’s architectural innovations enable robust vector floorplan reconstruction with coherent structures, watertight room boundaries, and reduced artifacts, outperforming existing methods.

Abstract: We present CAGE (Continuity-Aware edGE) network, a robust framework for reconstructing vector floorplans directly from point-cloud density maps. Traditional corner-based polygon representations are highly sensitive to noise and incomplete observations, often resulting in fragmented or implausible layouts.Recent line grouping methods leverage structural cues to improve robustness but still struggle to recover fine geometric details. To address these limitations,we propose a native edge-centric formulation, modeling each wall segment as a directed, geometrically continuous edge. This representation enables inference of coherent floorplan structures, ensuring watertight, topologically valid room boundaries while improving robustness and reducing artifacts. Towards this design, we develop a dual-query transformer decoder that integrates perturbed and latent queries within a denoising framework, which not only stabilizes optimization but also accelerates convergence. Extensive experiments on Structured3D and SceneCAD show that CAGE achieves state-of-the-art performance, with F1 scores of 99.1% (rooms), 91.7% (corners), and 89.3% (angles). The method also demonstrates strong cross-dataset generalization, underscoring the efficacy of our architectural innovations. Code and pretrained models are available on our project page: https://github.com/ee-Liu/CAGE.git.

Method: Bi-level optimization that jointly learns selection variables, sparse invertible mask, and sample-specific perturbations. Inner problem synthesizes triggers using inverse mask, outer problem refines mask to suppress backdoor while preserving clean accuracy.

Result: Outperforms existing pruning-based approaches, maintains strong performance under limited data, achieves competitive results with state-of-the-art fine-tuning methods, and effectively restores correct predictions for compromised samples.

Conclusion: The proposed invertible pruning approach provides an effective defense against backdoor attacks with improved interpretability and robustness, particularly valuable in low-data scenarios.

Abstract: Model pruning has gained traction as a promising defense strategy against backdoor attacks in deep learning. However, existing pruning-based approaches often fall short in accurately identifying and removing the specific parameters responsible for inducing backdoor behaviors. Despite the dominance of fine-tuning-based defenses in recent literature, largely due to their superior performance, pruning remains a compelling alternative, offering greater interpretability and improved robustness in low-data regimes. In this paper, we propose a novel pruning approach featuring a learned \emph{selection} mechanism to identify parameters critical to both main and backdoor tasks, along with an \emph{invertible} pruning mask designed to simultaneously achieve two complementary goals: eliminating the backdoor task while preserving it through the inverse mask. We formulate this as a bi-level optimization problem that jointly learns selection variables, a sparse invertible mask, and sample-specific backdoor perturbations derived from clean data. The inner problem synthesizes candidate triggers using the inverse mask, while the outer problem refines the mask to suppress backdoor behavior without impairing clean-task accuracy. Extensive experiments demonstrate that our approach outperforms existing pruning-based backdoor mitigation approaches, maintains strong performance under limited data conditions, and achieves competitive results compared to state-of-the-art fine-tuning approaches. Notably, the proposed approach is particularly effective in restoring correct predictions for compromised samples after successful backdoor mitigation.

Method: Propose Geo-R1 with a “reason first, then act” process: generate explicit interpretable reasoning chains to decompose referring expressions, then use these rationales to localize target objects.

Result: Substantially outperforms SFT baselines on three few-shot geospatial referring benchmarks and demonstrates strong cross-dataset generalization.

Conclusion: The reasoning-centric reinforcement fine-tuning paradigm enables more effective use of limited annotations, enhances generalization, and provides interpretability for geospatial referring tasks.

Abstract: Referring expression understanding in remote sensing poses unique challenges, as it requires reasoning over complex object-context relationships. While supervised fine-tuning (SFT) on multimodal large language models achieves strong performance with massive labeled datasets, they struggle in data-scarce scenarios, leading to poor generalization. To address this limitation, we propose Geo-R1, a reasoning-centric reinforcement fine-tuning (RFT) paradigm for few-shot geospatial referring. Geo-R1 enforces the model to first generate explicit, interpretable reasoning chains that decompose referring expressions, and then leverage these rationales to localize target objects. This “reason first, then act” process enables the model to make more effective use of limited annotations, enhances generalization, and provides interpretability. We validate Geo-R1 on three carefully designed few-shot geospatial referring benchmarks, where our model consistently and substantially outperforms SFT baselines. It also demonstrates strong cross-dataset generalization, highlighting its robustness. Code and data will be released at: https://github.com/Geo-R1/geo-r1.

Method: Created a curated benchmark with 4 primary visual domains, 15 secondary domains, and 43 sub-fields. Developed automated annotation pipeline and human-annotation platform with 19,945 real-world image question pairs across eight progressive dimensions.

Result: Extensive experiments on 17 state-of-the-art MLLMs revealed limitations in human-centric understanding and provided guidance for future model development.

Conclusion: Human-MME enables more holistic evaluation of MLLMs for human-centric scene understanding and exposes current model limitations, guiding future research in this direction.

Abstract: Multimodal Large Language Models (MLLMs) have demonstrated significant advances in visual understanding tasks. However, their capacity to comprehend human-centric scenes has rarely been explored, primarily due to the absence of comprehensive evaluation benchmarks that take into account both the human-oriented granular level and higher-dimensional causal reasoning ability. Such high-quality evaluation benchmarks face tough obstacles, given the physical complexity of the human body and the difficulty of annotating granular structures. In this paper, we propose Human-MME, a curated benchmark designed to provide a more holistic evaluation of MLLMs in human-centric scene understanding. Compared with other existing benchmarks, our work provides three key features: 1. Diversity in human scene, spanning 4 primary visual domains with 15 secondary domains and 43 sub-fields to ensure broad scenario coverage. 2. Progressive and diverse evaluation dimensions, evaluating the human-based activities progressively from the human-oriented granular perception to the higher-dimensional reasoning, consisting of eight dimensions with 19,945 real-world image question pairs and an evaluation suite. 3. High-quality annotations with rich data paradigms, constructing the automated annotation pipeline and human-annotation platform, supporting rigorous manual labeling to facilitate precise and reliable model assessment. Our benchmark extends the single-target understanding to the multi-person and multi-image mutual understanding by constructing the choice, short-answer, grounding, ranking and judgment question components, and complex questions of their combination. The extensive experiments on 17 state-of-the-art MLLMs effectively expose the limitations and guide future MLLMs research toward better human-centric image understanding. All data and code are available at https://github.com/Yuan-Hou/Human-MME.

Method: Proposed RobustVLA with two components: (1) output robustness via offline robust optimization against worst-case action noise, and (2) input robustness by enforcing consistent actions across semantically-preserving input variations. Formulated as multi-armed bandit problem to identify most harmful perturbations.

Result: Achieved 12.6% gain on pi0 backbone and 10.4% on OpenVLA across 17 perturbations, 50.6x faster inference than visual-robust VLAs, 10.4% gain under mixed perturbations, and 65.6% improvement on real-world FR5 robot with limited demonstrations.

Conclusion: RobustVLA effectively addresses multi-modal robustness in VLA models, demonstrating significant improvements across various perturbations and real-world robotic applications.

Abstract: In Vision-Language-Action (VLA) models, robustness to real-world perturbations is critical for deployment. Existing methods target simple visual disturbances, overlooking the broader multi-modal perturbations that arise in actions, instructions, environments, and observations. Here, we first evaluate the robustness of mainstream VLAs under 17 perturbations across four modalities. We find (1) actions as the most fragile modality, (2) Existing visual-robust VLA do not gain robustness in other modality, and (3) pi0 demonstrates superior robustness with a diffusion-based action head. To build multi-modal robust VLAs, we propose RobustVLA against perturbations in VLA inputs and outputs. For output robustness, we perform offline robust optimization against worst-case action noise that maximizes mismatch in flow matching objective. This can be seen as adversarial training, label smoothing, and outlier penalization. For input robustness, we enforce consistent actions across input variations that preserve task semantics. To account for multiple perturbations, we formulate robustness as a multi-armed bandit problem and apply an upper confidence bound algorithm to automatically identify the most harmful noise. Experiments on LIBERO demonstrate our RobustVLA delivers absolute gains over baselines of 12.6% on the pi0 backbone and 10.4% on the OpenVLA backbone across all 17 perturbations, achieving 50.6x faster inference than existing visual-robust VLAs, and a 10.4% gain under mixed perturbations. Our RobustVLA is particularly effective on real-world FR5 robot with limited demonstrations, showing absolute gains by 65.6% under perturbations of four modalities.

Method: DCS establishes a co-evolutionary framework with dynamic multi-faceted reward functions at feature level (semantic coherence/diversity) and logits level (exploratory generation/inter-class discriminability) that guide diffusion model training.

Result: The framework achieves state-of-the-art performance on FSCIL benchmarks, significantly enhancing both knowledge retention and new class learning.

Conclusion: The mutual boosting loop between diffusion model and classifier through reward-aligned learning effectively addresses FSCIL challenges and demonstrates superior performance.

Abstract: Few-Shot Class-Incremental Learning (FSCIL) challenges models to sequentially learn new classes from minimal examples without forgetting prior knowledge, a task complicated by the stability-plasticity dilemma and data scarcity. Current FSCIL methods often struggle with generalization due to their reliance on limited datasets. While diffusion models offer a path for data augmentation, their direct application can lead to semantic misalignment or ineffective guidance. This paper introduces Diffusion-Classifier Synergy (DCS), a novel framework that establishes a mutual boosting loop between diffusion model and FSCIL classifier. DCS utilizes a reward-aligned learning strategy, where a dynamic, multi-faceted reward function derived from the classifier’s state directs the diffusion model. This reward system operates at two levels: the feature level ensures semantic coherence and diversity using prototype-anchored maximum mean discrepancy and dimension-wise variance matching, while the logits level promotes exploratory image generation and enhances inter-class discriminability through confidence recalibration and cross-session confusion-aware mechanisms. This co-evolutionary process, where generated images refine the classifier and an improved classifier state yields better reward signals, demonstrably achieves state-of-the-art performance on FSCIL benchmarks, significantly enhancing both knowledge retention and new class learning.

Method: Three-pronged approach: comprehensive data engineering pipeline, hierarchical model framework (base model + pattern shifting + consistency modules), and two-stage training strategy with separate datasets.

Result: TBStar-Edit outperforms existing general-domain editing models in both objective metrics (VIE Score) and subjective user preference on e-commerce benchmark.

Conclusion: The proposed specialized approach with data engineering, hierarchical architecture, and staged training effectively addresses e-commerce image editing consistency challenges.

Abstract: Recent advances in image generation and editing technologies have enabled state-of-the-art models to achieve impressive results in general domains. However, when applied to e-commerce scenarios, these general models often encounter consistency limitations. To address this challenge, we introduce TBStar-Edit, an new image editing model tailored for the e-commerce domain. Through rigorous data engineering, model architecture design and training strategy, TBStar-Edit achieves precise and high-fidelity image editing while maintaining the integrity of product appearance and layout. Specifically, for data engineering, we establish a comprehensive data construction pipeline, encompassing data collection, construction, filtering, and augmentation, to acquire high-quality, instruction-following, and strongly consistent editing data to support model training. For model architecture design, we design a hierarchical model framework consisting of a base model, pattern shifting modules, and consistency enhancement modules. For model training, we adopt a two-stage training strategy to enhance the consistency preservation: first stage for editing pattern shifting, and second stage for consistency enhancement. Each stage involves training different modules with separate datasets. Finally, we conduct extensive evaluations of TBStar-Edit on a self-proposed e-commerce benchmark, and the results demonstrate that TBStar-Edit outperforms existing general-domain editing models in both objective metrics (VIE Score) and subjective user preference.

Method: Introduces MM-COF motion representation that combines optical flow from both micro-expression phases, and FMANet neural network with learnable modules for adaptive fusion of dual-phase motion cues and attention to salient facial regions.

Result: Experimental evaluations on MMEW, SMIC, CASME-II, and SAMM datasets show that the proposed MM-COF and FMANet outperform existing methods.

Conclusion: The learnable dual-phase framework demonstrates significant potential for advancing micro-expression recognition by capturing comprehensive motion dynamics.

Abstract: Facial micro-expressions, characterized by their subtle and brief nature, are valuable indicators of genuine emotions. Despite their significance in psychology, security, and behavioral analysis, micro-expression recognition remains challenging due to the difficulty of capturing subtle facial movements. Optical flow has been widely employed as an input modality for this task due to its effectiveness. However, most existing methods compute optical flow only between the onset and apex frames, thereby overlooking essential motion information in the apex-to-offset phase. To address this limitation, we first introduce a comprehensive motion representation, termed Magnitude-Modulated Combined Optical Flow (MM-COF), which integrates motion dynamics from both micro-expression phases into a unified descriptor suitable for direct use in recognition networks. Building upon this principle, we then propose FMANet, a novel end-to-end neural network architecture that internalizes the dual-phase analysis and magnitude modulation into learnable modules. This allows the network to adaptively fuse motion cues and focus on salient facial regions for classification. Experimental evaluations on the MMEW, SMIC, CASME-II, and SAMM datasets, widely recognized as standard benchmarks, demonstrate that our proposed MM-COF representation and FMANet outperforms existing methods, underscoring the potential of a learnable, dual-phase framework in advancing micro-expression recognition.

Method: PAChroma (Perception-Aware Chroma-Restrictive Perturbation) optimizes imperceptible perturbations using Laplacian filter for perceptual quality and diverse input transformations for transferability and robustness.

Result: Experiments on ImageNet and Danbooru datasets show PAChroma effectively degrades colorization quality while maintaining visual appearance.

Conclusion: First step toward protecting visual content from illegitimate AI colorization, paving way for copyright-aware defenses in generative media.

Abstract: AI-based colorization has shown remarkable capability in generating realistic color images from grayscale inputs. However, it poses risks of copyright infringement – for example, the unauthorized colorization and resale of monochrome manga and films. Despite these concerns, no effective method currently exists to prevent such misuse. To address this, we introduce the first defensive paradigm, Uncolorable Examples, which embed imperceptible perturbations into grayscale images to invalidate unauthorized colorization. To ensure real-world applicability, we establish four criteria: effectiveness, imperceptibility, transferability, and robustness. Our method, Perception-Aware Chroma-Restrictive Perturbation (PAChroma), generates Uncolorable Examples that meet these four criteria by optimizing imperceptible perturbations with a Laplacian filter to preserve perceptual quality, and applying diverse input transformations during optimization to enhance transferability across models and robustness against common post-processing (e.g., compression). Experiments on ImageNet and Danbooru datasets demonstrate that PAChroma effectively degrades colorization quality while maintaining the visual appearance. This work marks the first step toward protecting visual content from illegitimate AI colorization, paving the way for copyright-aware defenses in generative media.

Method: The review analyzes YOLO versions chronologically from YOLOv26 (latest) backward to YOLOv5, examining key innovations like DFL removal, NMS-free inference, ProgLoss, STAL, MuSGD optimizer, decoupled detection heads, and anchor-free predictions. Quantitative benchmarking is performed on MS COCO dataset with metrics including precision, recall, F1 score, mAP, and inference speed.

Result: YOLOv26 introduces significant innovations including native NMS-free inference and improved training stability. Benchmarking shows trade-offs between accuracy and efficiency across versions, with YOLO26 demonstrating state-of-the-art performance while maintaining deployment flexibility through various export formats and quantization strategies.

Conclusion: The YOLO family has evolved substantially with each version introducing architectural improvements. Future challenges include addressing dense-scene limitations, integrating CNN-Transformer hybrids, developing open-vocabulary detection capabilities, and optimizing for edge deployment scenarios.

Abstract: This paper presents a comprehensive overview of the Ultralytics YOLO(You Only Look Once) family of object detectors, focusing the architectural evolution, benchmarking, deployment perspectives, and future challenges. The review begins with the most recent release, YOLO26 (or YOLOv26), which introduces key innovations including Distribution Focal Loss (DFL) removal, native NMS-free inference, Progressive Loss Balancing (ProgLoss), Small-Target-Aware Label Assignment (STAL), and the MuSGD optimizer for stable training. The progression is then traced through YOLO11, with its hybrid task assignment and efficiency-focused modules; YOLOv8, which advanced with a decoupled detection head and anchor-free predictions; and YOLOv5, which established the modular PyTorch foundation that enabled modern YOLO development. Benchmarking on the MS COCO dataset provides a detailed quantitative comparison of YOLOv5, YOLOv8, YOLO11, and YOLO26 (YOLOv26), alongside cross-comparisons with YOLOv12, YOLOv13, RT-DETR, and DEIM(DETR with Improved Matching). Metrics including precision, recall, F1 score, mean Average Precision, and inference speed are analyzed to highlight trade-offs between accuracy and efficiency. Deployment and application perspectives are further discussed, covering export formats, quantization strategies, and real-world use in robotics, agriculture, surveillance, and manufacturing. Finally, the paper identifies challenges and future directions, including dense-scene limitations, hybrid CNN-Transformer integration, open-vocabulary detection, and edge-aware training approaches. (Object Detection, YOLOv26, YOLO)

Method: VR-Thinker introduces visual reasoning operations (e.g., select frame) and configurable visual memory windows. It uses a reinforcement fine-tuning pipeline with three stages: Cold Start with curated visual chain-of-thought data, Rejection sampling Fine-Tuning on high-quality traces, and Group Relative Policy Optimization (GRPO) to strengthen reasoning.

Result: State-of-the-art accuracy on video preference benchmarks: 7B VR-Thinker achieves 80.5% on VideoGen Reward, 82.3% on GenAI-Bench, and 75.6% on MJ-Bench-Video, especially effective for longer videos.

Conclusion: The results validate the effectiveness and promise of thinking-with-image multimodal reward modeling, demonstrating improved reasoning fidelity and reliability through active visual evidence acquisition.

Abstract: Recent advancements in multimodal reward models (RMs) have substantially improved post-training for visual generative models. However, current RMs face inherent limitations: (1) visual inputs consume large context budgets, forcing fewer frames and causing loss of fine-grained details; and (2) all visual information is packed into the initial prompt, exacerbating hallucination and forgetting during chain-of-thought reasoning. To overcome these issues, we introduce VideoReward Thinker (VR-Thinker), a thinking-with-image framework that equips the RM with visual reasoning operations (e.g., select frame) and a configurable visual memory window. This allows the RM to actively acquire and update visual evidence within context limits, improving reasoning fidelity and reliability. We activate visual reasoning via a reinforcement fine-tuning pipeline: (i) Cold Start with curated visual chain-of-thought data to distill basic reasoning skills and operation formatting; (ii) select samples whose per-dimension and overall judgments are all correct, then conduct Rejection sampling Fine-Tuning on these high-quality traces to further enhance reasoning; and (iii) apply Group Relative Policy Optimization (GRPO) to strengthen reasoning. Our approach delivers state-of-the-art accuracy among open-source models on video preference benchmarks, especially for longer videos: a 7B VR-Thinker achieves 80.5% on VideoGen Reward, 82.3% on GenAI-Bench, and 75.6% on MJ-Bench-Video. These results validate the effectiveness and promise of thinking-with-image multimodal reward modeling.

Method: Proposed ΔEnergy OOD score based on energy changes during vision-language modality realignment, with a lower-bound maximization approach (EBM) that theoretically ensures domain-consistent Hessian for improved generalization.

Result: Extensive experiments show the method outperforms recent approaches by 10% to 25% in AUROC on challenging OOD detection and generalization benchmarks.

Conclusion: The unified fine-tuning framework using ΔEnergy and EBM significantly enhances VLMs’ robustness in both OOD generalization and OOD detection, providing a comprehensive solution for handling real-world distribution shifts.

Abstract: Recent approaches for vision-language models (VLMs) have shown remarkable success in achieving fast downstream adaptation. When applied to real-world downstream tasks, VLMs inevitably encounter both the in-distribution (ID) data and out-of-distribution (OOD) data. The OOD datasets often include both covariate shifts (e.g., known classes with changes in image styles) and semantic shifts (e.g., test-time unseen classes). This highlights the importance of improving VLMs’ generalization ability to covariate-shifted OOD data, while effectively detecting open-set semantic-shifted OOD classes. In this paper, inspired by the substantial energy change observed in closed-set data when re-aligning vision-language modalities (specifically by directly reducing the maximum cosine similarity to a low value), we introduce a novel OOD score, named {\Delta}Energy. {\Delta}Energy significantly outperforms the vanilla energy-based OOD score and provides a more reliable approach for OOD detection. Furthermore, {\Delta}Energy can simultaneously improve OOD generalization under covariate shifts, which is achieved by lower-bound maximization for {\Delta}Energy (termed EBM). EBM is theoretically proven to not only enhance OOD detection but also yields a domain-consistent Hessian, which serves as a strong indicator for OOD generalization. Based on this finding, we developed a unified fine-tuning framework that allows for improving VLMs’ robustness in both OOD generalization and OOD detection. Extensive experiments on challenging OOD detection and generalization benchmarks demonstrate the superiority of our method, outperforming recent approaches by 10% to 25% in AUROC.

Method: Proposes CapFlow, a multi-agent collaboration workflow that leverages open-source models to generate high-quality captions. Uses this as a data synthesizer to train MetaCaptioner via fine-tuning.

Result: CapFlow achieves caption quality comparable to GPT-4 with 89.5% cost reduction. MetaCaptioner matches commercial models’ capabilities and reaches top-tier multimodal performance in open-source community.

Conclusion: CapFlow and MetaCaptioner provide a strong, cost-effective visual captioning solution that can benefit future multimodal research.

Abstract: Generalist visual captioning goes beyond a simple appearance description task, but requires integrating a series of visual cues into a caption and handling various visual domains. In this task, current open-source models present a large performance gap with commercial ones, which limits various applications such as data synthesis. To bridge the gap, this paper proposes CapFlow, a novel multi-agent collaboration workflow. CapFlow demonstrates for the first time that, by capitalizing on open-source models, it is possible to achieve caption quality on par with GPT-4.1 in various domains with an 89.5% reduction in costs. By leveraging CapFlow as the data synthesizer, we produce high-quality visual captions from image and video domains at scale, and obtain a generalist visual captioner via fine-tuning, namely MetaCaptioner. Through extensive experiments, we show that MetaCaptioner not only achieves comparable captioning capabilities with commercial models but also reaches top-tier multimodal performance in the open-source community. We hope CapFlow and MetaCaptioner can benefit future multimodal research by providing a strong and cost-effective visual captioning solution.

Method: Proposes Spatial-Aware Feature Encoder and Cross-Agent Feature Integration to preserve 3D structures, Altitude-Aware Feature Reduction for compact representation, and Dual-Mask Perceptual Guidance for adaptive feature selection and reduced communication.

Result: Achieves state-of-the-art accuracy, significantly outperforms existing collaborative methods, and reduces communication overhead to only a fraction of previous approaches.

Conclusion: The proposed framework effectively addresses limitations of current BEV-based approaches for UAV collaborative perception, providing superior performance with reduced communication requirements.

Abstract: Unmanned Aerial Vehicle (UAV) swarm systems necessitate efficient collaborative perception mechanisms for diverse operational scenarios. Current Bird’s Eye View (BEV)-based approaches exhibit two main limitations: bounding-box representations fail to capture complete semantic and geometric information of the scene, and their performance significantly degrades when encountering undefined or occluded objects. To address these limitations, we propose a novel multi-UAV collaborative occupancy prediction framework. Our framework effectively preserves 3D spatial structures and semantics through integrating a Spatial-Aware Feature Encoder and Cross-Agent Feature Integration. To enhance efficiency, we further introduce Altitude-Aware Feature Reduction to compactly represent scene information, along with a Dual-Mask Perceptual Guidance mechanism to adaptively select features and reduce communication overhead. Due to the absence of suitable benchmark datasets, we extend three datasets for evaluation: two virtual datasets (Air-to-Pred-Occ and UAV3D-Occ) and one real-world dataset (GauUScene-Occ). Experiments results demonstrate that our method achieves state-of-the-art accuracy, significantly outperforming existing collaborative methods while reducing communication overhead to only a fraction of previous approaches.

Method: Proposes unsupervised two-stage evidential loss clustering for label reliability awareness and residual flow matching mechanism that models domain- and category-conditioned residuals for uncertainty-aware transfer paths. Uses meta-learning where clean set updates maximize loss decrease on noisy set with pseudo labels.

Result: EReLiFM outperforms existing methods on OSDG-NL, achieving state-of-the-art performance in experiments.

Conclusion: The proposed approach effectively handles noisy labels in open-set domain generalization through evidential reliability awareness and residual flow matching, demonstrating superior performance over existing methods.

Abstract: Open-Set Domain Generalization (OSDG) aims to enable deep learning models to recognize unseen categories in new domains, which is crucial for real-world applications. Label noise hinders open-set domain generalization by corrupting source-domain knowledge, making it harder to recognize known classes and reject unseen ones. While existing methods address OSDG under Noisy Labels (OSDG-NL) using hyperbolic prototype-guided meta-learning, they struggle to bridge domain gaps, especially with limited clean labeled data. In this paper, we propose Evidential Reliability-Aware Residual Flow Meta-Learning (EReLiFM). We first introduce an unsupervised two-stage evidential loss clustering method to promote label reliability awareness. Then, we propose a residual flow matching mechanism that models structured domain- and category-conditioned residuals, enabling diverse and uncertainty-aware transfer paths beyond interpolation-based augmentation. During this meta-learning process, the model is optimized such that the update direction on the clean set maximizes the loss decrease on the noisy set, using pseudo labels derived from the most confident predicted class for supervision. Experimental results show that EReLiFM outperforms existing methods on OSDG-NL, achieving state-of-the-art performance. The source code is available at https://github.com/KPeng9510/ERELIFM.

Method: Rule-Intent Distinction (RID) Framework - a low-compute meta-prompting technique that provides structured cognitive schema for deconstructing tasks, classifying rules, weighing conflicting outcomes, and justifying decisions.

Result: Achieved 95% Human Alignment Score compared to 80% baseline and 75% Chain-of-Thought, with higher-quality intent-driven reasoning across 20 diverse scenarios.

Conclusion: RID Framework provides a practical, accessible method for steering LLMs from literal instruction-following to goal-oriented reasoning, enabling more reliable AI agents.

Abstract: Large Language Models (LLMs) are increasingly being deployed as the reasoning engines for agentic AI systems, yet they exhibit a critical flaw: a rigid adherence to explicit rules that leads to decisions misaligned with human common sense and intent. This “rule-rigidity” is a significant barrier to building trustworthy autonomous agents. While prior work has shown that supervised fine-tuning (SFT) with human explanations can mitigate this issue, SFT is computationally expensive and inaccessible to many practitioners. To address this gap, we introduce the Rule-Intent Distinction (RID) Framework, a novel, low-compute meta-prompting technique designed to elicit human-aligned exception handling in LLMs in a zero-shot manner. The RID framework provides the model with a structured cognitive schema for deconstructing tasks, classifying rules, weighing conflicting outcomes, and justifying its final decision. We evaluated the RID framework against baseline and Chain-of-Thought (CoT) prompting on a custom benchmark of 20 scenarios requiring nuanced judgment across diverse domains. Our human-verified results demonstrate that the RID framework significantly improves performance, achieving a 95% Human Alignment Score (HAS), compared to 80% for the baseline and 75% for CoT. Furthermore, it consistently produces higher-quality, intent-driven reasoning. This work presents a practical, accessible, and effective method for steering LLMs from literal instruction-following to liberal, goal-oriented reasoning, paving the way for more reliable and pragmatic AI agents.

Method: Proposes DeepPlanner with token-level advantage shaping using entropy-based terms to prioritize high-entropy tokens, and sample-level advantage upweighting for planning-intensive rollouts.

Result: Extensive experiments across seven benchmarks show improved planning quality and state-of-the-art results with substantially lower training budget.

Conclusion: DeepPlanner effectively enhances planning capabilities in deep research agents through systematic optimization of the planning stage.

Abstract: Large language models (LLMs) augmented with multi-step reasoning and action generation abilities have shown promise in leveraging external tools to tackle complex tasks that require long-horizon planning. However, existing approaches either rely on implicit planning in the reasoning stage or introduce explicit planners without systematically addressing how to optimize the planning stage. As evidence, we observe that under vanilla reinforcement learning (RL), planning tokens exhibit significantly higher entropy than other action tokens, revealing uncertain decision points that remain under-optimized. To address this, we propose DeepPlanner, an end-to-end RL framework that effectively enhances the planning capabilities of deep research agents. Our approach shapes token-level advantage with an entropy-based term to allocate larger updates to high entropy tokens, and selectively upweights sample-level advantages for planning-intensive rollouts. Extensive experiments across seven deep research benchmarks demonstrate that DeepPlanner improves planning quality and achieves state-of-the-art results under a substantially lower training budget.

Method: Multi-level verification pipeline: (i) semantic level - formalize natural language safety requirements into temporal logic formulas and probe LLM understanding; (ii) plan level - verify high-level action plans against TL formulas; (iii) trajectory level - merge execution trajectories into computation trees and verify against detailed TL specifications.

Result: Applied in VirtualHome and ALFRED environments, Sentinel successfully exposed safety violations overlooked by previous methods and provided insights into failure modes of LLM-based embodied agents.

Conclusion: Sentinel provides a rigorous foundation for systematically evaluating LLM-based embodied agents in physical environments by grounding physical safety in temporal logic and applying verification across multiple levels.

Abstract: We present Sentinel, the first framework for formally evaluating the physical safety of Large Language Model(LLM-based) embodied agents across the semantic, plan, and trajectory levels. Unlike prior methods that rely on heuristic rules or subjective LLM judgments, Sentinel grounds practical safety requirements in formal temporal logic (TL) semantics that can precisely specify state invariants, temporal dependencies, and timing constraints. It then employs a multi-level verification pipeline where (i) at the semantic level, intuitive natural language safety requirements are formalized into TL formulas and the LLM agent’s understanding of these requirements is probed for alignment with the TL formulas; (ii) at the plan level, high-level action plans and subgoals generated by the LLM agent are verified against the TL formulas to detect unsafe plans before execution; and (iii) at the trajectory level, multiple execution trajectories are merged into a computation tree and efficiently verified against physically-detailed TL specifications for a final safety check. We apply Sentinel in VirtualHome and ALFRED, and formally evaluate multiple LLM-based embodied agents against diverse safety requirements. Our experiments show that by grounding physical safety in temporal logic and applying verification methods across multiple levels, Sentinel provides a rigorous foundation for systematically evaluating LLM-based embodied agents in physical environments, exposing safety violations overlooked by previous methods and offering insights into their failure modes.

Method: Fine-tuned Llama 3.2 1B model on five years of two-vehicle crash narratives, benchmarked against Random Forest, XGBoost, CatBoost, and neural network classifiers.

Result: Fine-tuned LLM achieved 80% overall accuracy, outperforming all baseline models, with improved performance on imbalanced data. Probabilistic reasoning revealed specific patterns: distraction increases ‘General Unsafe Driving’, mutual distraction increases ‘Both Drivers Took Hazardous Actions’, and teen drivers elevate ‘Speed and Stopping Violations’.

Conclusion: The framework provides robust, interpretable automated DHA detection for traffic safety analysis, offering new opportunities for intervention through large-scale analysis.

Abstract: Vehicle crashes involve complex interactions between road users, split-second decisions, and challenging environmental conditions. Among these, two-vehicle crashes are the most prevalent, accounting for approximately 70% of roadway crashes and posing a significant challenge to traffic safety. Identifying Driver Hazardous Action (DHA) is essential for understanding crash causation, yet the reliability of DHA data in large-scale databases is limited by inconsistent and labor-intensive manual coding practices. Here, we present an innovative framework that leverages a fine-tuned large language model to automatically infer DHAs from textual crash narratives, thereby improving the validity and interpretability of DHA classifications. Using five years of two-vehicle crash data from MTCF, we fine-tuned the Llama 3.2 1B model on detailed crash narratives and benchmarked its performance against conventional machine learning classifiers, including Random Forest, XGBoost, CatBoost, and a neural network. The fine-tuned LLM achieved an overall accuracy of 80%, surpassing all baseline models and demonstrating pronounced improvements in scenarios with imbalanced data. To increase interpretability, we developed a probabilistic reasoning approach, analyzing model output shifts across original test sets and three targeted counterfactual scenarios: variations in driver distraction and age. Our analysis revealed that introducing distraction for one driver substantially increased the likelihood of “General Unsafe Driving”; distraction for both drivers maximized the probability of “Both Drivers Took Hazardous Actions”; and assigning a teen driver markedly elevated the probability of “Speed and Stopping Violations.” Our framework and analytical methods provide a robust and interpretable solution for large-scale automated DHA detection, offering new opportunities for traffic safety analysis and intervention.

Method: Proposes using LLMs for time series analysis as a reasoning task that prioritizes causal structure discovery and explainability, integrating multimodal inputs for context-aware insights.

Result: The approach enables time series analysis to uncover actual driving forces behind trends rather than just surface-level patterns, bringing analysis closer to human-aligned understanding.

Conclusion: Shifting from numerical regression to reasoning-based LLM usage for time series analysis provides transparent and context-aware insights suitable for complex real-world environments.

Abstract: Traditional time series analysis has long relied on pattern recognition, trained on static and well-established benchmarks. However, in real-world settings – where policies shift, human behavior adapts, and unexpected events unfold – effective analysis must go beyond surface-level trends to uncover the actual forces driving them. The recent rise of Large Language Models (LLMs) presents new opportunities for rethinking time series analysis by integrating multimodal inputs. However, as the use of LLMs becomes popular, we must remain cautious, asking why we use LLMs and how to exploit them effectively. Most existing LLM-based methods still employ their numerical regression ability and ignore their deeper reasoning potential. This paper argues for rethinking time series with LLMs as a reasoning task that prioritizes causal structure and explainability. This shift brings time series analysis closer to human-aligned understanding, enabling transparent and context-aware insights in complex real-world environments.

Method: Iterative framework that adds transition-dependent correction terms to proxy rewards using targeted exploration and a new preference-learning objective.

Result: PBRR matches prior methods’ regret bounds in tabular domains and outperforms baselines on reward-hacking benchmarks with substantially fewer preferences needed.

Conclusion: PBRR effectively bridges the gap between flawed proxy rewards and learning from scratch, requiring fewer human preferences to achieve optimal performance.

Abstract: Human-designed reward functions for reinforcement learning (RL) agents are frequently misaligned with the humans’ true, unobservable objectives, and thus act only as proxies. Optimizing for a misspecified proxy reward function often induces reward hacking, resulting in a policy misaligned with the human’s true objectives. An alternative is to perform RL from human feedback, which involves learning a reward function from scratch by collecting human preferences over pairs of trajectories. However, building such datasets is costly. To address the limitations of both approaches, we propose Preference-Based Reward Repair (PBRR): an automated iterative framework that repairs a human-specified proxy reward function by learning an additive, transition-dependent correction term from preferences. A manually specified reward function can yield policies that are highly suboptimal under the ground-truth objective, yet corrections on only a few transitions may suffice to recover optimal performance. To identify and correct for those transitions, PBRR uses a targeted exploration strategy and a new preference-learning objective. We prove in tabular domains PBRR has a cumulative regret that matches, up to constants, that of prior preference-based RL methods. In addition, on a suite of reward-hacking benchmarks, PBRR consistently outperforms baselines that learn a reward function from scratch from preferences or modify the proxy reward function using other approaches, requiring substantially fewer preferences to learn high performing policies.

Method: Constructs an emotional cognition framework with desire generation and objective management, modeling the complete decision-making process including state evolution, desire generation, objective optimization, decision generation, and action execution.

Result: Agents using this framework exhibit behaviors congruent with emotional states, demonstrate superior ecological validity, and generate decisions that significantly more closely approximate human behavioral patterns compared to other agent types.

Conclusion: The proposed emotional cognition framework successfully enables emotion alignment between LLM-based agents and humans, addressing key limitations in affective cognition and bounded rationality.

Abstract: The advent of large language models (LLMs) has enabled agents to represent virtual humans in societal simulations, facilitating diverse interactions within complex social systems. However, existing LLM-based agents exhibit severe limitations in affective cognition: They fail to simulate the bounded rationality essential for bridging virtual and real-world services; They lack empirically validated integration mechanisms embedding emotions within agent decision architectures. This paper constructs an emotional cognition framework incorporating desire generation and objective management, designed to achieve emotion alignment between LLM-based agents and humans, modeling the complete decision-making process of LLM-based agents, encompassing state evolution, desire generation, objective optimization, decision generation, and action execution. This study implements the proposed framework within our proprietary multi-agent interaction environment. Experimental results demonstrate that agents governed by our framework not only exhibit behaviors congruent with their emotional states but also, in comparative assessments against other agent types, demonstrate superior ecological validity and generate decision outcomes that significantly more closely approximate human behavioral patterns.

Method: Integration of small and large LLMs where small LLM generates initial answer, large LLM verifies it, and only performs deep reasoning if the answer is incorrect.

Result: Reduces computational cost of large LLM by more than 50% for simple problems with negligible accuracy loss, while maintaining robust performance on complex tasks.

Conclusion: The complementary agent system effectively balances computational efficiency and accuracy by leveraging small LLMs for simple tasks and reserving large LLMs for complex reasoning when needed.

Abstract: Recent advances in Large Language Models (LLMs) demonstrate that chain-of-thought prompting and deep reasoning substantially enhance performance on complex tasks, and multi-agent systems can further improve accuracy by enabling model debates. However, applying deep reasoning to all problems is computationally expensive. To mitigate these costs, we propose a complementary agent system integrating small and large LLMs. The small LLM first generates an initial answer, which is then verified by the large LLM. If correct, the answer is adopted directly; otherwise, the large LLM performs in-depth reasoning. Experimental results show that, for simple problems, our approach reduces the computational cost of the large LLM by more than 50% with negligible accuracy loss, while consistently maintaining robust performance on complex tasks.

Method: Integrates reinforcement-based training with LLM-driven educational architecture, using structured learner state modeling, automated reward functions, supervised fine-tuning (SFT), and Group Relative Policy Optimization (GRPO).

Result: Extensive experiments validate Pxplore’s effectiveness in producing coherent, personalized, and goal-driven learning paths, with deployment in a real-world learning platform.

Conclusion: The framework successfully addresses the goal-alignment gap in personalized learning and the released code/dataset will facilitate future research in this area.

Abstract: Personalized Learning Path Planning (PLPP) aims to design adaptive learning paths that align with individual goals. While large language models (LLMs) show potential in personalizing learning experiences, existing approaches often lack mechanisms for goal-aligned planning. We introduce Pxplore, a novel framework for PLPP that integrates a reinforcement-based training paradigm and an LLM-driven educational architecture. We design a structured learner state model and an automated reward function that transforms abstract objectives into computable signals. We train the policy combining supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO), and deploy it within a real-world learning platform. Extensive experiments validate Pxplore’s effectiveness in producing coherent, personalized, and goal-driven learning paths. We release our code and dataset to facilitate future research.

Method: EvoTest uses an evolutionary approach with two roles: Actor Agent plays the game, and Evolver Agent analyzes episode transcripts to propose revised configurations including prompt rewriting, memory updates, hyperparameter tuning, and tool-use routine learning.

Result: EvoTest consistently increases performance on J-TTL benchmark, outperforming reflection, memory-only baselines, and online fine-tuning methods. It’s the only method capable of winning two games (Detective and Library) while all baselines fail.

Conclusion: Evolutionary test-time learning without fine-tuning or gradients enables effective agent improvement across episodes, addressing fundamental limitations of current AI agents.

Abstract: A fundamental limitation of current AI agents is their inability to learn complex skills on the fly at test time, often behaving like “clever but clueless interns” in novel environments. This severely limits their practical utility. To systematically measure and drive progress on this challenge, we first introduce the Jericho Test-Time Learning (J-TTL) benchmark. J-TTL is a new evaluation setup where an agent must play the same game for several consecutive episodes, attempting to improve its performance from one episode to the next. On J-TTL, we find that existing adaptation methods like reflection, memory, or reinforcement learning struggle. To address the challenges posed by our benchmark, we present EvoTest, an evolutionary test-time learning framework that improves an agent without any fine-tuning or gradients-by evolving the entire agentic system after every episode. EvoTest has two roles: the Actor Agent, which plays the game, and the Evolver Agent, which analyzes the episode transcript to propose a revised configuration for the next run. This configuration rewrites the prompt, updates memory by logging effective state-action choices, tunes hyperparameters, and learns the tool-use routines. On our J-TTL benchmark, EvoTest consistently increases performance, outperforming not only reflection and memory-only baselines but also more complex online fine-tuning methods. Notably, our method is the only one capable of winning two games (Detective and Library), while all baselines fail to win any.

Method: Developed a custom dataset with distinctive objects (motorcyclists, rickshaws, etc.), used YOLOv8s for object detection, and created a utility measurement module to evaluate perception service from trained model performance.

Result: Three best-performing YOLOv8s instances: SGD-based (mAP@0.5: 0.832), Adam-based (0.810), and AdamW-based (0.822). AdamW-based model outperformed SGD-based model in class-level performance (car: 0.921, motorcyclist: 0.899, truck: 0.793).

Conclusion: The proposed perception model successfully evaluates learning model utility and determines appropriate perception for autonomous vehicles, validated through object detection tasks and benchmarking against state-of-the-art models.

Abstract: The driving environment perception has a vital role for autonomous driving and nowadays has been actively explored for its realization. The research community and relevant stakeholders necessitate the development of Deep Learning (DL) models and AI-enabled solutions to enhance autonomous vehicles (AVs) for smart mobility. There is a need to develop a model that accurately perceives multiple objects on the road and predicts the driver’s perception to control the car’s movements. This article proposes a novel utility-based analytical model that enables perception systems of AVs to understand the driving environment. The article consists of modules: acquiring a custom dataset having distinctive objects, i.e., motorcyclists, rickshaws, etc; a DL-based model (YOLOv8s) for object detection; and a module to measure the utility of perception service from the performance values of trained model instances. The perception model is validated based on the object detection task, and its process is benchmarked by state-of-the-art deep learning models’ performance metrics from the nuScense dataset. The experimental results show three best-performing YOLOv8s instances based on mAP@0.5 values, i.e., SGD-based (0.832), Adam-based (0.810), and AdamW-based (0.822). However, the AdamW-based model (i.e., car: 0.921, motorcyclist: 0.899, truck: 0.793, etc.) still outperforms the SGD-based model (i.e., car: 0.915, motorcyclist: 0.892, truck: 0.781, etc.) because it has better class-level performance values, confirmed by the proposed perception model. We validate that the proposed function is capable of finding the right perception for AVs. The results above encourage using the proposed perception model to evaluate the utility of learning models and determine the appropriate perception for AVs.

Method: Models multi-agent collaboration as discrete-time temporal attributed graphs, uses unsupervised encoder-decoder architecture with incremental training to reconstruct node attributes and graph structures, and employs graph abstraction based on Information Bottleneck Theory.

Result: Achieves state-of-the-art accuracy in detecting anomalous nodes and edges with efficient resource utilization, effectively safeguarding LLM multi-agent collaborations against diverse safety vulnerabilities.

Conclusion: GUARDIAN provides an effective solution for detecting and mitigating multiple safety concerns in multi-agent LLM collaborations through temporal graph modeling and unsupervised learning approaches.

Abstract: The emergence of large language models (LLMs) enables the development of intelligent agents capable of engaging in complex and multi-turn dialogues. However, multi-agent collaboration faces critical safety challenges, such as hallucination amplification and error injection and propagation. This paper presents GUARDIAN, a unified method for detecting and mitigating multiple safety concerns in GUARDing Intelligent Agent collaboratioNs. By modeling the multi-agent collaboration process as a discrete-time temporal attributed graph, GUARDIAN explicitly captures the propagation dynamics of hallucinations and errors. The unsupervised encoder-decoder architecture incorporating an incremental training paradigm learns to reconstruct node attributes and graph structures from latent embeddings, enabling the identification of anomalous nodes and edges with unparalleled precision. Moreover, we introduce a graph abstraction mechanism based on the Information Bottleneck Theory, which compresses temporal interaction graphs while preserving essential patterns. Extensive experiments demonstrate GUARDIAN’s effectiveness in safeguarding LLM multi-agent collaborations against diverse safety vulnerabilities, achieving state-of-the-art accuracy with efficient resource utilization. The code is available at https://github.com/JialongZhou666/GUARDIAN

Method: A two-phase framework: (1) multi-step gradient ascent using actor and critic networks to compute adversarial states, (2) gradient ascent using critic network on perturbed states to craft adversarial actions, with a heuristic regularizer.

Result: SAJA outperforms state-only or action-only attacks in Multi-Agent Particle Environment, is more stealthy, and existing defense methods cannot defend against it.

Conclusion: The proposed joint state-action attack framework demonstrates superior attack effectiveness and stealthiness compared to isolated attacks, highlighting the need for more comprehensive defense strategies.

Abstract: Multi-Agent Deep Reinforcement Learning (MADRL) has shown potential for cooperative and competitive tasks such as autonomous driving and strategic gaming. However, models trained by MADRL are vulnerable to adversarial perturbations on states and actions. Therefore, it is essential to investigate the robustness of MADRL models from an attack perspective. Existing studies focus on either state-only attacks or action-only attacks, but do not consider how to effectively joint them. Simply combining state and action perturbations such as randomly perturbing states and actions does not exploit their potential synergistic effects. In this paper, we propose the State-Action Joint Attack (SAJA) framework that has a good synergistic effects. SAJA consists of two important phases: (1) In the state attack phase, a multi-step gradient ascent method utilizes both the actor network and the critic network to compute an adversarial state, and (2) in the action attack phase, based on the perturbed state, a second gradient ascent uses the critic network to craft the final adversarial action. Additionally, a heuristic regularizer measuring the distance between the perturbed actions and the original clean ones is added into the loss function to enhance the effectiveness of the critic’s guidance. We evaluate SAJA in the Multi-Agent Particle Environment (MPE), demonstrating that (1) it outperforms and is more stealthy than state-only or action-only attacks, and (2) existing state or action defense methods cannot defend its attacks.

Method: PORAT uses game-theoretic policy optimization to progressively introduce interventions in the cooperative game process, guiding the system toward more optimal equilibria by encouraging generator exploration.

Result: Validated on 9 real-world datasets and 2 synthetic settings, PORAT achieves up to 8.1% performance improvements over state-of-the-art methods.

Conclusion: The game-theoretic perspective successfully addresses mode collapse in rationalization, and PORAT provides an effective unified solution for guiding cooperative models toward optimal equilibria.

Abstract: Rationalization, a data-centric framework, aims to build self-explanatory models to explain the prediction outcome by generating a subset of human-intelligible pieces of the input data. It involves a cooperative game model where a generator generates the most human-intelligible parts of the input (i.e., rationales), followed by a predictor that makes predictions based on these generated rationales. Conventional rationalization methods typically impose constraints via regularization terms to calibrate or penalize undesired generation. However, these methods are suffering from a problem called mode collapse, in which the predictor produces correct predictions yet the generator consistently outputs rationales with collapsed patterns. Moreover, existing studies are typically designed separately for specific collapsed patterns, lacking a unified consideration. In this paper, we systematically revisit cooperative rationalization from a novel game-theoretic perspective and identify the fundamental cause of this problem: the generator no longer tends to explore new strategies to uncover informative rationales, ultimately leading the system to converge to a suboptimal game equilibrium (correct predictions v.s collapsed rationales). To solve this problem, we then propose a novel approach, Game-theoretic Policy Optimization oriented RATionalization (PORAT), which progressively introduces policy interventions to address the game equilibrium in the cooperative game process, thereby guiding the model toward a more optimal solution state. We theoretically analyse the cause of such a suboptimal equilibrium and prove the feasibility of the proposed method. Furthermore, we validate our method on nine widely used real-world datasets and two synthetic settings, where PORAT achieves up to 8.1% performance improvements over existing state-of-the-art methods.

Method: Nine LLMs were instructed to generate all possible intermediate causal steps linking given cause-effect pairs from climate change discussions, using a diagnostic evaluation framework.

Result: LLMs vary in the number and granularity of causal steps produced, are generally self-consistent and confident, but their judgments are driven by associative pattern matching rather than genuine causal reasoning. Human evaluations confirmed logical coherence.

Conclusion: The baseline causal chain discovery approach, diagnostic insights, and benchmark dataset provide a foundation for advancing implicit mechanistic causal reasoning in argumentation settings.

Abstract: How does a cause lead to an effect, and which intermediate causal steps explain their connection? This work scrutinizes the mechanistic causal reasoning capabilities of large language models (LLMs) to answer these questions through the task of implicit causal chain discovery. In a diagnostic evaluation framework, we instruct nine LLMs to generate all possible intermediate causal steps linking given cause-effect pairs in causal chain structures. These pairs are drawn from recent resources in argumentation studies featuring polarized discussion on climate change. Our analysis reveals that LLMs vary in the number and granularity of causal steps they produce. Although they are generally self-consistent and confident about the intermediate causal connections in the generated chains, their judgments are mainly driven by associative pattern matching rather than genuine causal reasoning. Nonetheless, human evaluations confirmed the logical coherence and integrity of the generated chains. Our baseline causal chain discovery approach, insights from our diagnostic evaluation, and benchmark dataset with causal chains lay a solid foundation for advancing future work in implicit, mechanistic causal reasoning in argumentation settings.

Method: Developed Thermodynamic Coordination Theory (TCT) using information theory to derive minimum description length bounds for coordination protocols, introduced coordination temperature concept, and extended Arrow’s theorem to analyze preference aggregation constraints.

Result: Found that coordination protocol complexity scales as L(P)≥NKlog₂K+N²d²log(1/ε), forcing progressive simplification. Coordination creates metastable states with hysteresis, requiring environmental shifts for phase transitions. Arrow’s theorem recursively binds preference combination.

Conclusion: Coordination fundamentally requires radical information loss, with findability being more important than accuracy. This explains persistent cycling in multi-objective systems and alignment issues in AI, providing a unified thermodynamic framework for coordination phenomena.

Abstract: Information-processing systems that coordinate multiple agents and objectives face fundamental thermodynamic constraints. We show that solutions with maximum utility to act as coordination focal points have a much higher selection pressure for being findable across agents rather than accuracy. We derive that the information-theoretic minimum description length of coordination protocols to precision $\varepsilon$ scales as $L(P)\geq NK\log_2 K+N^2d^2\log (1/\varepsilon)$ for $N$ agents with $d$ potentially conflicting objectives and internal model complexity $K$. This scaling forces progressive simplification, with coordination dynamics changing the environment itself and shifting optimization across hierarchical levels. Moving from established focal points requires re-coordination, creating persistent metastable states and hysteresis until significant environmental shifts trigger phase transitions through spontaneous symmetry breaking. We operationally define coordination temperature to predict critical phenomena and estimate coordination work costs, identifying measurable signatures across systems from neural networks to restaurant bills to bureaucracies. Extending the topological version of Arrow’s theorem on the impossibility of consistent preference aggregation, we find it recursively binds whenever preferences are combined. This potentially explains the indefinite cycling in multi-objective gradient descent and alignment faking in Large Language Models trained with reinforcement learning with human feedback. We term this framework Thermodynamic Coordination Theory (TCT), which demonstrates that coordination requires radical information loss.

Method: Coverage analysis at individual cell level aggregated to site level using empirical geolocation data, applying One-Class SVM algorithm to model coverage contours and analyze service loss reports.

Result: Framework effectively maps mobile coverage and highlights granular areas of signal deficiency, especially in complex urban environments.

Conclusion: The novel framework demonstrates efficacy in accurately mapping mobile coverage and identifying geographically localized weak spots using crowdsourced data and machine learning approaches.

Abstract: Effective assessment of mobile network coverage and the precise identification of service weak spots are paramount for network operators striving to enhance user Quality of Experience (QoE). This paper presents a novel framework for mobile coverage and weak spot analysis utilising crowdsourced QoE data. The core of our methodology involves coverage analysis at the individual cell (antenna) level, subsequently aggregated to the site level, using empirical geolocation data. A key contribution of this research is the application of One-Class Support Vector Machine (OC-SVM) algorithm for calculating mobile network coverage. This approach models the decision hyperplane as the effective coverage contour, facilitating robust calculation of coverage areas for individual cells and entire sites. The same methodology is extended to analyse crowdsourced service loss reports, thereby identifying and quantifying geographically localised weak spots. Our findings demonstrate the efficacy of this novel framework in accurately mapping mobile coverage and, crucially, in highlighting granular areas of signal deficiency, particularly within complex urban environments.

Method: Systematically investigate Confidence-as-a-Reward (CRew) using token-level confidence in final answers, and propose CRew-DPO training strategy that constructs preference data from confidence scores with correctness signals.

Result: CRew outperforms existing training-free reward approaches on MATH500 and RewardMATH benchmarks, surpasses most trained reward models, shows strong correlation with reasoning performance, and enables effective high-quality data filtering.

Conclusion: Confidence-as-a-Reward is a powerful training-free method that effectively enhances LLM reasoning capabilities, and CRew-DPO further improves model judging capabilities beyond existing self-training methods.

Abstract: Reward models can significantly enhance the reasoning capabilities of large language models (LLMs), but they typically require extensive curated data and costly training. To mitigate these challenges, training-free approaches such as LLM-as-a-Judge leverage the intrinsic reasoning abilities of LLMs to evaluate responses, achieving promising results. Recent works have also indicated that model confidence can serve effectively as a reward metric, distinguishing between chain-of-thought (CoT) and non-CoT paths. However, the concept of using confidence as a reward has not been comprehensively studied. In this work, we systematically investigate Confidence-as-a-Reward (CRew), a simple yet powerful training-free method that utilizes token-level confidence in the model’s final answers as a proxy for reward, especially suitable for close-ended tasks. Through extensive experiments on mathematical reasoning tasks, we demonstrate that CRew outperforms existing training-free reward approaches on the MATH500 and RewardMATH benchmarks, and even surpasses most trained reward models. We further identify a strong correlation between CRew scores and the actual reasoning performance of the model. Additionally, we find that CRew can effectively filter high-quality training data. Building upon these insights, we propose CRew-DPO, a training strategy that constructs preference data from confidence scores combined with correctness signals. Finetuning with CRew-DPO further enhances the model’s judging capabilities and consistently outperforms existing self-training methods.

Method: The paper proposes a Risk Assessment Framework that allows for better application of Subject Matter Expert evaluation and machine learning metrics, addressing unique risks in metric selection.

Result: The framework helps overcome limitations of traditional metrics and generalized benchmarks that fail to account for industrial use cases in financial services.

Conclusion: The proposed Risk Assessment Framework enables more effective measurement of Generative AI performance in financial services by properly combining SME evaluation with appropriate metrics.

Abstract: As Generative Artificial Intelligence is adopted across the financial services industry, a significant barrier to adoption and usage is measuring model performance. Historical machine learning metrics can oftentimes fail to generalize to GenAI workloads and are often supplemented using Subject Matter Expert (SME) Evaluation. Even in this combination, many projects fail to account for various unique risks present in choosing specific metrics. Additionally, many widespread benchmarks created by foundational research labs and educational institutions fail to generalize to industrial use. This paper explains these challenges and provides a Risk Assessment Framework to allow for better application of SME and machine learning Metrics

Method: Introduces tandem training - an RL paradigm where rollout tokens are randomly sampled from a frozen weak model instead of the strong model being trained. This forces the strong model to produce solutions that can be continued by the weaker model.

Result: In GSM8K math reasoning, tandem training reliably teaches models to abandon jargon and adapt language to weaker partners while maintaining high task accuracy.

Conclusion: Tandem training provides a promising approach for building AI systems that remain auditable by weaker agents, with positive implications for human-AI collaboration and multi-agent communication.

Abstract: As language models continue to rapidly improve, we can expect their actions and reasoning to become difficult or impossible for weaker agents and humans to follow, undermining interpretability and oversight. With an eye on long-term futures, we pursue methods that encourage models to produce solutions that remain intelligible to weaker collaborators. We formalize intelligibility as handoff robustness: a strong model’s solution is intelligible to a weaker model if randomly handing off control to the weaker model along the solution path does not cause failure. Building on this criterion, we introduce tandem training for language models, a reinforcement learning (RL) paradigm in which rollout tokens are intermittently and randomly sampled from a frozen weak model rather than the strong model being trained. Because rollouts succeed only when the strong model’s actions and reasoning process can be continued by the weak model – when the two can co-construct a successful solution – optimizing standard RL objectives with tandem training implicitly incentivizes both correctness and intelligibility. In the GSM8K math reasoning task, tandem training reliably teaches models to abandon jargon and adapt their language to weaker partners while keeping task accuracy high. Our results demonstrate a promising route to building AI systems that remain auditable by weaker agents, with implications for human–AI collaboration and multi-agent communication.

Method: Introduces a modal logic of classifiers that incorporates temporal dimensions of cases and court hierarchy to handle conflicts between precedents.

Result: A formal logic framework that can model legal case-based reasoning with conflict resolution mechanisms based on time and court authority.

Conclusion: The proposed modal logic provides a foundation for building verification tools for ML classifiers in legal contexts by formally representing legal reasoning principles.

Abstract: Logic-based models can be used to build verification tools for machine learning classifiers employed in the legal field. ML classifiers predict the outcomes of new cases based on previous ones, thereby performing a form of case-based reasoning (CBR). In this paper, we introduce a modal logic of classifiers designed to formally capture legal CBR. We incorporate principles for resolving conflicts between precedents, by introducing into the logic the temporal dimension of cases and the hierarchy of courts within the legal system.

Method: Proposed Empower method maximizes human empowerment (ability to effect desired changes) using only offline text data. Self-supervised fine-tuning of language models based on empowerment maximization.

Result: In user study, participants preferred Empower assistant 78% of the time (p=0.015) with 31% higher acceptance rate and 38% fewer suggestions. In simulated coding environment, Empower increased success rate by 192% over baseline.

Conclusion: Empower provides a framework for creating aligned AI agents at scale using only offline data, without needing additional human feedback or verifiable rewards.

Abstract: Assistive agents should not only take actions on behalf of a human, but also step out of the way and cede control when there are important decisions to be made. However, current methods for building assistive agents, whether via mimicking expert humans or via RL finetuning on an inferred reward, often encourage agents to complete tasks on their own rather than truly assisting the human attain their objectives. Additionally, these methods often require costly explicit human feedback to provide a training signal. We propose a new approach to tuning assistive language models based on maximizing the human’s empowerment, their ability to effect desired changes in the environment. Our empowerment-maximizing method, Empower, only requires offline text data, providing a self-supervised method for fine-tuning language models to better assist humans. To study the efficacy of our approach, we conducted an 18-person user study comparing our empowerment assistant with a strong baseline. Participants preferred our assistant 78% of the time (p=0.015), with a 31% higher acceptance rate and 38% fewer suggestions. Additionally, we introduce a new environment for evaluating multi-turn code assistance using simulated humans. Using this environment, we show that agents trained with Empower increase the success rate of a simulated human programmer on challenging coding questions by an average of 192% over an SFT baseline. With this empowerment objective, we provide a framework for useful aligned AI agents at scale using only offline data without the need for any additional human feedback or verifiable rewards.

Method: Formalizes AI safety through safety-critical control theory in the AI model’s latent representation. Uses safety-critical reinforcement learning to train predictive guardrails that monitor and proactively correct risky outputs in real-time.

Result: Experiments in simulated driving and e-commerce show the guardrails reliably prevent catastrophic outcomes (collisions, bankruptcy) while maintaining task performance, outperforming traditional flag-and-block approaches.

Conclusion: Control-theoretic guardrails offer a principled dynamic alternative to current static safety methods, enabling AI systems to operate safely in complex environments by treating safety as a sequential decision problem.

Abstract: Generative AI systems are increasingly assisting and acting on behalf of end users in practical settings, from digital shopping assistants to next-generation autonomous cars. In this context, safety is no longer about blocking harmful content, but about preempting downstream hazards like financial or physical harm. Yet, most AI guardrails continue to rely on output classification based on labeled datasets and human-specified criteria,making them brittle to new hazardous situations. Even when unsafe conditions are flagged, this detection offers no path to recovery: typically, the AI system simply refuses to act–which is not always a safe choice. In this work, we argue that agentic AI safety is fundamentally a sequential decision problem: harmful outcomes arise from the AI system’s continually evolving interactions and their downstream consequences on the world. We formalize this through the lens of safety-critical control theory, but within the AI model’s latent representation of the world. This enables us to build predictive guardrails that (i) monitor an AI system’s outputs (actions) in real time and (ii) proactively correct risky outputs to safe ones, all in a model-agnostic manner so the same guardrail can be wrapped around any AI model. We also offer a practical training recipe for computing such guardrails at scale via safety-critical reinforcement learning. Our experiments in simulated driving and e-commerce settings demonstrate that control-theoretic guardrails can reliably steer LLM agents clear of catastrophic outcomes (from collisions to bankruptcy) while preserving task performance, offering a principled dynamic alternative to today’s flag-and-block guardrails.

Method: Created Hard2Verify benchmark with 500+ hours of human annotation, evaluating 29 generative critics and process reward models on step-level verification of frontier LLM responses to recent math questions.

Result: Open-source verifiers significantly lag behind closed-source models in step-level verification performance, with only a few exceptions.

Conclusion: The study reveals performance gaps in step-level verification and analyzes factors like scaling compute, self-verification, and verification-generation dynamics.

Abstract: Large language model (LLM)-based reasoning systems have recently achieved gold medal-level performance in the IMO 2025 competition, writing mathematical proofs where, to receive full credit, each step must be not only correct but also sufficiently supported. To train LLM-based reasoners in such challenging, open-ended settings, strong verifiers capable of catching step-level mistakes are necessary prerequisites. We introduce Hard2Verify, a human-annotated, step-level verification benchmark produced with over 500 hours of human labor. Hard2Verify is designed to rigorously assess step-level verifiers at the frontier: Verifiers must provide step-level annotations or identify the first error in responses generated by frontier LLMs for very recent, challenging, and open-ended math questions. We evaluate 29 generative critics and process reward models, demonstrating that, beyond a few standouts, open-source verifiers lag closed source models. We subsequently analyze what drives poor performance in step-level verification, the impacts of scaling verifier compute, as well as fundamental questions such as self-verification and verification-generation dynamics.

Method: The authors provide analytical characterization of the optimal QMDP value function and develop a dynamic programming-based algorithm to solve for optimal policies, which also extends to CVaR objectives.

Result: The proposed algorithm successfully solves QMDP problems and is demonstrated on an HIV treatment initiation case study where patients balance treatment benefits and risks.

Conclusion: QMDP provides a valuable framework for risk-sensitive decision making, particularly in applications where optimizing specific reward quantiles is more appropriate than maximizing expected values.

Abstract: The goal of a traditional Markov decision process (MDP) is to maximize expected cumulative reward over a defined horizon (possibly infinite). In many applications, however, a decision maker may be interested in optimizing a specific quantile of the cumulative reward instead of its expectation. In this paper we consider the problem of optimizing the quantiles of the cumulative rewards of a Markov decision process (MDP), which we refer to as a quantile Markov decision process (QMDP). We provide analytical results characterizing the optimal QMDP value function and present a dynamic programming-based algorithm to solve for the optimal policy. The algorithm also extends to the MDP problem with a conditional value-at-risk (CVaR) objective. We illustrate the practical relevance of our model by evaluating it on an HIV treatment initiation problem, where patients aim to balance the potential benefits and risks of the treatment.

Method: Define 66 atomic functions for common computational logics, then use LLM-FuncMapper with rule-based adaptive prompts to match clauses to functions, and generate executable code by composing functions through LLMs.

Result: LLM-FuncMapper outperforms fine-tuning methods by 19% in function matching while significantly reducing manual annotation efforts, and can automatically compose multiple atomic functions to generate executable code.

Conclusion: This represents the first application of LLMs for interpreting complex design clauses into executable code, potentially enabling broader adoption of LLMs in the construction domain.

Abstract: Translating clauses into executable code is a vital stage of automated rule checking (ARC) and is essential for effective building design compliance checking, particularly for rules with implicit properties or complex logic requiring domain knowledge. Thus, by systematically analyzing building clauses, 66 atomic functions are defined first to encapsulate common computational logics. Then, LLM-FuncMapper is proposed, a large language model (LLM)-based approach with rule-based adaptive prompts that match clauses to atomic functions. Finally, executable code is generated by composing functions through the LLMs. Experiments show LLM-FuncMapper outperforms fine-tuning methods by 19% in function matching while significantly reducing manual annotation efforts. Case study demonstrates that LLM-FuncMapper can automatically compose multiple atomic functions to generate executable code, boosting rule-checking efficiency. To our knowledge, this research represents the first application of LLMs for interpreting complex design clauses into executable code, which may shed light on further adoption of LLMs in the construction domain.

Method: Proposed Modular Agentic Planner (MAP) architecture where planning is achieved through recurrent interaction of specialized modules (conflict monitoring, state prediction, state evaluation, task decomposition, orchestration), each implemented using LLMs. The approach breaks down larger problems into multiple brief automated LLM calls.

Result: MAP significantly outperforms standard LLM methods (zero-shot, in-context learning) and competitive baselines (chain-of-thought, multi-agent debate, tree-of-thought) on graph traversal, Tower of Hanoi, PlanBench benchmark, and strategyQA. It works effectively with smaller LLMs (Llama3-70B) and shows superior task transfer capabilities.

Conclusion: A modular and multi-agent approach to planning with LLMs provides significant benefits for complex reasoning tasks, demonstrating the value of specialized functional components working together in an agentic architecture.

Abstract: Large language models (LLMs) demonstrate impressive performance on a wide variety of tasks, but they often struggle with tasks that require multi-step reasoning or goal-directed planning. Both cognitive neuroscience and reinforcement learning (RL) have proposed a number of interacting functional components that together implement search and evaluation in multi-step decision making. These components include conflict monitoring, state prediction, state evaluation, task decomposition, and orchestration. To improve planning with LLMs, we propose an agentic architecture, the Modular Agentic Planner (MAP), in which planning is accomplished via the recurrent interaction of the specialized modules mentioned above, each implemented using an LLM. MAP improves planning through the interaction of specialized modules that break down a larger problem into multiple brief automated calls to the LLM. We evaluate MAP on three challenging planning tasks – graph traversal, Tower of Hanoi, and the PlanBench benchmark – as well as an NLP task requiring multi-step reasoning (strategyQA). We find that MAP yields significant improvements over both standard LLM methods (zero-shot prompting, in-context learning) and competitive baselines (chain-of-thought, multi-agent debate, and tree-of-thought), can be effectively combined with smaller and more cost-efficient LLMs (Llama3-70B), and displays superior transfer across tasks. These results suggest the benefit of a modular and multi-agent approach to planning with LLMs.

Method: Constructed MathCAMPS dataset with 44 fine-grained math skills from Common Core K-8 curriculum, analyzed skill acquisition order during pre-training and effects of instruction tuning.

Result: Mathematical skills are learned during pre-training in order correlated with human curriculum despite random data ordering; instruction tuning benefits some skills while harming others.

Conclusion: Provides empirical understanding of LLM training dynamics for reasoning, showing curriculum-like learning patterns emerge naturally from next-token prediction training.

Abstract: Large Language Models (LLMs) solely trained on next-token prediction learn to solve a wide range of problems involving mathematical reasoning. But how does this ability evolve during training? We show the first analysis of how mathematical reasoning abilities of several open-weight LLMs develop during pre-training and post-training. To this end, we construct MathCAMPS, a synthetic dataset of novel mathematical reasoning problems grounded in 44 fine-grained skills taken from the Common Core curriculum from K to 8th grades. In one experiment, we show that mathematical skills are learned during pre-training in an order that measurably correlates with the human-designed curriculum, even though training data are randomly ordered. We also show a detailed analysis of which mathematical abilities benefit from instruction tuning, a widely used post-training method and, in contrast, which skills suffer. Our work paves the way for an empirical understanding of LLM training dynamics in relation to reasoning.

Method: Uses natural language processing to extract sentiments and emotions from text, aggregates individual preferences into a collective matrix, and employs a fuzzy inference system to compute overall scores combining preferences, sentiments, and emotions.

Result: Successfully applied to a hotel selection scenario, demonstrating that integrating sentiment and emotion analysis significantly improves consensus among participants by considering everyone’s feelings and opinions.

Conclusion: The proposed system effectively enhances consensus-reaching in group decision-making by incorporating both explicit preferences and implicit sentiments/emotions from natural language discussions.

Abstract: In today’s world, making decisions as a group is common, whether choosing a restaurant or deciding on a holiday destination. Group decision-making (GDM) systems play a crucial role by facilitating consensus among participants with diverse preferences. Discussions are one of the main tools people use to make decisions. When people discuss alternatives, they use natural language to express their opinions. Traditional GDM systems generally require participants to provide explicit opinion values to the system. However, in real-life scenarios, participants often express their opinions through some text (e.g., in comments, social media, messengers, etc.). This paper introduces a sentiment and emotion-aware multi-criteria fuzzy GDM system designed to enhance consensus-reaching effectiveness in group settings. This system incorporates natural language processing to analyze sentiments and emotions expressed in textual data, enabling an understanding of participant opinions besides the explicit numerical preference inputs. Once all the experts have provided their preferences for the alternatives, the individual preferences are aggregated into a single collective preference matrix. This matrix represents the collective expert opinion regarding the other options. Then, sentiments, emotions, and preference scores are inputted into a fuzzy inference system to get the overall score. The proposed system was used for a small decision-making process - choosing the hotel for a vacation by a group of friends. Our findings demonstrate that integrating sentiment and emotion analysis into GDM systems allows everyone’s feelings and opinions to be considered during discussions and significantly improves consensus among participants.

Method: Created the first computational framework for SLS with GUI and RL benchmarking support, then trained self-playing agents using classical deep RL methods (DQN, DDQN, Dueling DQN).

Result: Agents achieved ~50% of maximum reward and consistently outperformed random baselines, but required ~2000 games of training and still made occasional illegal moves.

Conclusion: SLS serves as a promising negotiation-aware MARL benchmark, highlighting both the capabilities and limitations of classical RL and opening avenues for future research integrating game theory and coalition-aware strategies.

Abstract: This paper investigates the strategy game So Long Sucker (SLS) as a novel benchmark for multi-agent reinforcement learning (MARL). Unlike traditional board or video game testbeds, SLS is distinguished by its coalition formation, strategic deception, and dynamic elimination rules, making it a uniquely challenging environment for autonomous agents. We introduce the first publicly available computational framework for SLS, complete with a graphical user interface and benchmarking support for reinforcement learning algorithms. Using classical deep reinforcement learning methods (e.g., DQN, DDQN, and Dueling DQN), we train self-playing agents to learn the rules and basic strategies of SLS. Experimental results demonstrate that, although these agents achieve roughly half of the maximum attainable reward and consistently outperform random baselines, they require long training horizons (~2000 games) and still commit occasional illegal moves, highlighting both the promise and limitations of classical reinforcement learning. Our findings establish SLS as a negotiation-aware benchmark for MARL, opening avenues for future research that integrates game-theoretic reasoning, coalition-aware strategies, and advanced reinforcement learning architectures to better capture the social and adversarial dynamics of complex multi-agent games.

Method: Three-module agentic framework: (1) Grounding Module predicts marketable features by mimicking expert human behavior, (2) Personalization Module aligns content with user preferences, (3) Marketing Module ensures factual accuracy and localized features.

Result: Human-subject experiments with potential house buyers showed that marketing descriptions generated by this approach were clearly preferred over those written by human experts while maintaining the same level of factual accuracy.

Conclusion: The framework presents a promising agentic approach for automated large-scale targeted copywriting that ensures factuality in content generation, demonstrating superior performance compared to human expert writing.

Abstract: This paper develops an agentic framework that employs large language models (LLMs) for grounded persuasive language generation in automated copywriting, with real estate marketing as a focal application. Our method is designed to align the generated content with user preferences while highlighting useful factual attributes. This agent consists of three key modules: (1) Grounding Module, mimicking expert human behavior to predict marketable features; (2) Personalization Module, aligning content with user preferences; (3) Marketing Module, ensuring factual accuracy and the inclusion of localized features. We conduct systematic human-subject experiments in the domain of real estate marketing, with a focus group of potential house buyers. The results demonstrate that marketing descriptions generated by our approach are preferred over those written by human experts by a clear margin while maintaining the same level of factual accuracy. Our findings suggest a promising agentic approach to automate large-scale targeted copywriting while ensuring factuality of content generation.

Method: Proposed ICLDC system where UAV collects sensory data, LLM generates task descriptions in natural language and creates data collection schedules, with a verifier ensuring safety by overriding unsafe schedules based on predefined rules.

Result: ICLDC significantly reduces cumulative packet loss compared to DQN and Maximum Channel Gain baselines, though vulnerable to jailbreaking attacks that manipulate task descriptions.

Conclusion: ICLDC presents a promising direction for intelligent scheduling and control in UAV-assisted sensor networks, offering continuous adaptation through feedback incorporation.

Abstract: Unmanned Aerial Vehicles (UAVs) are increasingly being utilized in various private and commercial applications, e.g., traffic control, parcel delivery, and Search and Rescue (SAR) missions. Machine Learning (ML) methods used in UAV-Assisted Sensor Networks (UASNETs) and, especially, in Deep Reinforcement Learning (DRL) face challenges such as complex and lengthy model training, gaps between simulation and reality, and low sampling efficiency, which conflict with the urgency of emergencies, such as SAR missions. In this paper, an In-Context Learning (ICL)-Data Collection Scheduling (ICLDC) system is proposed as an alternative to DRL in emergencies. The UAV collects sensory data and transmits it to a Large Language Model (LLM), which creates a task description in natural language. From this description, the UAV receives a data collection schedule that must be executed. A verifier ensures safe UAV operations by evaluating the schedules generated by the LLM and overriding unsafe schedules based on predefined rules. The system continuously adapts by incorporating feedback into the task descriptions and using this for future decisions. This method is tested against jailbreaking attacks, where the task description is manipulated to undermine network performance, highlighting the vulnerability of LLMs to such attacks. The proposed ICLDC significantly reduces cumulative packet loss compared to both the DQN and Maximum Channel Gain baselines. ICLDC presents a promising direction for intelligent scheduling and control in UASNETs.

Method: DPP uses pretrained auxiliary Neural Operators to enforce prior distribution constraints, enabling first-order gradient-based inverse optimization with surrogate machine learning models.

Result: The approach provides robust and meaningful solutions, particularly effective when prior data and observation distributions are unknown.

Conclusion: DPP offers a novel method for inverse design optimization that overcomes limitations of current approaches by enabling gradient-based optimization with physics-informed constraints.

Abstract: Inverse design optimization aims to infer system parameters from observed solutions, posing critical challenges across domains such as semiconductor manufacturing, structural engineering, materials science, and fluid dynamics. The lack of explicit mathematical representations in many systems complicates this process and makes the first order optimization impossible. Mainstream approaches, including generative AI and Bayesian optimization, address these challenges but have limitations. Generative AI is computationally expensive, while Bayesian optimization, relying on surrogate models, suffers from scalability, sensitivity to priors, and noise issues, often leading to suboptimal solutions. This paper introduces Deep Physics Prior (DPP), a novel method enabling first-order gradient-based inverse optimization with surrogate machine learning models. By leveraging pretrained auxiliary Neural Operators, DPP enforces prior distribution constraints to ensure robust and meaningful solutions. This approach is particularly effective when prior data and observation distributions are unknown.

Method: Curated high-quality, open-access scientific publications to create MSEarth benchmark, encompassing atmosphere, cryosphere, hydrosphere, lithosphere, and biosphere domains. Captions were refined from original figure captions and enriched with discussions and reasoning from papers.

Result: Created MSEarth benchmark with over 289K figures and enriched captions that capture nuanced reasoning and knowledge-intensive content essential for advanced scientific tasks. Supports scientific figure captioning, multiple choice questions, and open-ended reasoning challenges.

Conclusion: MSEarth bridges the gap in graduate-level benchmarks and provides a scalable, high-fidelity resource to enhance development and evaluation of multimodal large language models in scientific reasoning, fostering further research in this field.

Abstract: The rapid advancement of multimodal large language models (MLLMs) has unlocked new opportunities to tackle complex scientific challenges. Despite this progress, their application in addressing earth science problems, especially at the graduate level, remains underexplored. A significant barrier is the absence of benchmarks that capture the depth and contextual complexity of geoscientific reasoning. Current benchmarks often rely on synthetic datasets or simplistic figure-caption pairs, which do not adequately reflect the intricate reasoning and domain-specific insights required for real-world scientific applications. To address these gaps, we introduce MSEarth, a multimodal scientific benchmark curated from high-quality, open-access scientific publications. MSEarth encompasses the five major spheres of Earth science: atmosphere, cryosphere, hydrosphere, lithosphere, and biosphere, featuring over 289K figures with refined captions. These captions are crafted from the original figure captions and enriched with discussions and reasoning from the papers, ensuring the benchmark captures the nuanced reasoning and knowledge-intensive content essential for advanced scientific tasks. MSEarth supports a variety of tasks, including scientific figure captioning, multiple choice questions, and open-ended reasoning challenges. By bridging the gap in graduate-level benchmarks, MSEarth provides a scalable and high-fidelity resource to enhance the development and evaluation of MLLMs in scientific reasoning. The benchmark is publicly available to foster further research and innovation in this field.

Method: Uses table agents for detection, classification, extraction, and recommendations leveraging initial schema, with a Recommender Agent that reviews outputs and suggests schema revisions in a continuous learning process.

Result: Outperforms Table Transformer by 10.1%, with larger batch sizes increasing actionable schema recommendations by 104.3% and extracted holdings by 9.8%. Dataset includes 22,584 pages and 3,213 tables covering $731B+ holdings.

Conclusion: Agentic, schema-guided extraction systems show promise for robust understanding of real-world financial tables, with continuous learning significantly improving performance.

Abstract: Real-world financial documents report essential information about an entity’s financial holdings that can span millions of different financial instrument types. Yet, these details are often buried in messy, multi-page, fragmented tables - for example, 99.4% of the tables in our dataset have no bounding boxes with the maximum number of rows amounting to 426 per table across 44 pages. To tackle these unique challenges from real-world tables, we present a continuously learning, agentic table extraction system, TASER (Table Agents for Schema-guided Extraction and Recommendation) that extracts highly unstructured, multi-page, heterogeneous tables into normalized, schema-conforming outputs. Our table agents execute on table detection, classification, extraction, and recommendations by leveraging an initial schema. Then, our Recommender Agent reviews the outputs, recommends schema revisions, and decides on the final recommendations, enabling TASER to outperform existing table detection models such as Table Transformer by 10.1%. Within this continuous learning process, we highlight that larger batch sizes result in a 104.3% increase in schema recommendations that are actionable and utilized, resulting in a 9.8% increase in extracted holdings - highlighting the importance of a continuous learning process. To train TASER, we have manually labeled 22,584 pages (28,150,449 tokens), 3,213 tables for $731,685,511,687 of holdings culminating in one of the first real financial table datasets. We release our dataset TASERTab to enable the research community to access real-world financial tables and outputs. Our results highlight the promise of agentic, schema-guided extraction systems for robust understanding of real-world financial tables.

Method: The framework integrates multiple LLMs through OpenRouter platform for automated process map interpretation and report generation, validated using sepsis progression data across four proof-of-concept scenarios.

Result: The framework successfully processed sepsis data and generated reports through automated LLM analysis. Claude Sonnet-4 and Gemini 2.5-Pro achieved highest consistency scores (3.79/4.0 and 3.65/4.0) in LLM evaluation.

Conclusion: HealthProcessAI represents a novel methodological advance by combining structured analytics with AI-driven interpretation to translate complex process mining results into actionable healthcare insights.

Abstract: Process mining has emerged as a powerful analytical technique for understanding complex healthcare workflows. However, its application faces significant barriers, including technical complexity, a lack of standardized approaches, and limited access to practical training resources. We introduce HealthProcessAI, a GenAI framework designed to simplify process mining applications in healthcare and epidemiology by providing a comprehensive wrapper around existing Python (PM4PY) and R (bupaR) libraries. To address unfamiliarity and improve accessibility, the framework integrates multiple Large Language Models (LLMs) for automated process map interpretation and report generation, helping translate technical analyses into outputs that diverse users can readily understand. We validated the framework using sepsis progression data as a proof-of-concept example and compared the outputs of five state-of-the-art LLM models through the OpenRouter platform. To test its functionality, the framework successfully processed sepsis data across four proof-of-concept scenarios, demonstrating robust technical performance and its capability to generate reports through automated LLM analysis. LLM evaluation using five independent LLMs as automated evaluators revealed distinct model strengths: Claude Sonnet-4 and Gemini 2.5-Pro achieved the highest consistency scores (3.79/4.0 and 3.65/4.0) when evaluated by automated LLM assessors. By integrating multiple Large Language Models (LLMs) for automated interpretation and report generation, the framework addresses widespread unfamiliarity with process mining outputs, making them more accessible to clinicians, data scientists, and researchers. This structured analytics and AI-driven interpretation combination represents a novel methodological advance in translating complex process mining results into potentially actionable insights for healthcare applications.

Method: Proposed COAST framework leveraging long-term clue memory to better plan and solve sequential tasks, and CUA-as-a-Judge automated gameplay evaluator.

Result: Current GUI agents struggle with full story arcs, while COAST improves milestone completion by bridging the observation-behavior gap.

Conclusion: There remains a marked discrepancy between humans and best-performing agents, warranting continued research efforts to narrow this divide.

Abstract: GUI agents powered by LLMs show promise in interacting with diverse digital environments. Among these, video games offer a valuable testbed due to their varied interfaces, with adventure games posing additional challenges through complex, narrative-driven interactions. Existing game benchmarks, however, lack diversity and rarely evaluate agents on completing entire storylines. To address this, we introduce FlashAdventure, a benchmark of 34 Flash-based adventure games designed to test full story arc completion and tackle the observation-behavior gap: the challenge of remembering and acting on earlier gameplay information. We also propose CUA-as-a-Judge, an automated gameplay evaluator, and COAST, an agentic framework leveraging long-term clue memory to better plan and solve sequential tasks. Experiments show current GUI agents struggle with full story arcs, while COAST improves milestone completion by bridging the observation-behavior gap. Nonetheless, a marked discrepancy between humans and best-performing agents warrants continued research efforts to narrow this divide.

Method: Developed an automated red-teaming framework and constructed SafeSearch benchmark with 300 test cases covering 5 risk categories. Evaluated 3 search agent scaffolds across 15 LLMs.

Result: Found substantial vulnerabilities - highest attack success rate reached 90.5% for GPT-4.1-mini. Common defenses like reminder prompting showed limited effectiveness.

Conclusion: The framework provides systematic safety assessment for search agents, highlighting the need for transparency in safer agent development.

Abstract: Search agents connect LLMs to the Internet, enabling access to broader and more up-to-date information. However, unreliable search results may also pose safety threats to end users, establishing a new threat surface. In this work, we conduct two in-the-wild experiments to demonstrate both the prevalence of low-quality search results and their potential to misguide agent behaviors. To counter this threat, we introduce an automated red-teaming framework that is systematic, scalable, and cost-efficient, enabling lightweight and harmless safety assessments of search agents. Building on this framework, we construct the SafeSearch benchmark, which includes 300 test cases covering five categories of risks (e.g., misinformation and indirect prompt injection). Using this benchmark, we evaluate three representative search agent scaffolds, covering search workflow, tool-calling, and deep research, across 7 proprietary and 8 open-source backend LLMs. Our results reveal substantial vulnerabilities of LLM-based search agents: when exposed to unreliable websites, the highest ASR reached 90.5% for GPT-4.1-mini under a search workflow setting. Moreover, our analysis highlights the limited effectiveness of common defense practices, such as reminder prompting. This emphasizes the value of our framework in promoting transparency for safer agent development. Our codebase and test cases are publicly available: https://github.com/jianshuod/SafeSearch.

Method: Review of LLM-based techniques across four data modalities: structured data (table QA, NL2GQL), semi-structured data (markup languages, table modeling), unstructured data (chart/document understanding, vulnerability detection), and heterogeneous data (data retrieval, modality alignment).

Result: Identified five key design goals for intelligent data analysis agents: semantic-aware design, modality-hybrid integration, autonomous pipelines, tool-augmented workflows, and support for open-world tasks.

Conclusion: Outlined remaining challenges and proposed insights and practical directions for advancing LLM/Agent-powered data analysis systems.

Abstract: Large language model (LLM) and agent techniques for data analysis (a.k.a LLM/Agent-as-Data-Analyst) have demonstrated substantial impact in both academica and industry. In comparison with traditional rule or small-model based approaches, (agentic) LLMs enable complex data understanding, natural language interfaces, semantic analysis functions, and autonomous pipeline orchestration. The technical evolution further distills five key design goals for intelligent data analysis agents, namely semantic-aware design, modality-hybrid integration, autonomous pipelines, tool-augmented workflows, and support for open-world tasks. From a modality perspective, we review LLM-based techniques for (i) structured data (e.g., table question answering for relational data and NL2GQL for graph data), (ii) semi-structured data (e.g., markup languages understanding and semi-structured table modeling), (iii) unstructured data (e.g., chart understanding, document understanding, programming languages vulnerable detection), and (iv) heterogeneous data (e.g., data retrieval and modality alignment for data lakes). Finally, we outline the remaining challenges and propose several insights and practical directions for advancing LLM/Agent-powered data analysis.

Method: Proposes UMFDet framework with VLM backbone, Category-aware Mixture-of-Experts Adapter for category-specific cues, and attribution chain-of-thought mechanism for implicit reasoning guidance to locate deceptive signals.

Result: Extensive experiments show UMFDet achieves robust and consistent performance across both misinformation types, outperforming specialized baselines on the comprehensive OmniFake dataset of 127K samples.

Conclusion: UMFDet provides a practical unified solution for real-world multimodal deception detection that handles both human-crafted and AI-generated fake content effectively.

Abstract: In recent years, detecting fake multimodal content on social media has drawn increasing attention. Two major forms of deception dominate: human-crafted misinformation (e.g., rumors and misleading posts) and AI-generated content produced by image synthesis models or vision-language models (VLMs). Although both share deceptive intent, they are typically studied in isolation. NLP research focuses on human-written misinformation, while the CV community targets AI-generated artifacts. As a result, existing models are often specialized for only one type of fake content. In real-world scenarios, however, the type of a multimodal post is usually unknown, limiting the effectiveness of such specialized systems. To bridge this gap, we construct the Omnibus Dataset for Multimodal News Deception (OmniFake), a comprehensive benchmark of 127K samples that integrates human-curated misinformation from existing resources with newly synthesized AI-generated examples. Based on this dataset, we propose Unified Multimodal Fake Content Detection (UMFDet), a framework designed to handle both forms of deception. UMFDet leverages a VLM backbone augmented with a Category-aware Mixture-of-Experts (MoE) Adapter to capture category-specific cues, and an attribution chain-of-thought mechanism that provides implicit reasoning guidance for locating salient deceptive signals. Extensive experiments demonstrate that UMFDet achieves robust and consistent performance across both misinformation types, outperforming specialized baselines and offering a practical solution for real-world multimodal deception detection.

Method: Uses LLMs as planners to decompose instructions into executable pipelines and as annotators to generate supervision for training small proxy models. Unifies classification and extraction into two atomic operations (get label and get span).

Result: Falconer closely matches state-of-the-art LLMs in instruction-following accuracy while reducing inference cost by up to 90% and accelerating large-scale knowledge mining by more than 20x.

Conclusion: Falconer offers an efficient and scalable foundation for Deep Research by combining the strengths of LLMs and lightweight proxy models.

Abstract: At the core of Deep Research is knowledge mining, the task of extracting structured information from massive unstructured text in response to user instructions. Large language models (LLMs) excel at interpreting such instructions but are prohibitively expensive to deploy at scale, while traditional pipelines of classifiers and extractors remain efficient yet brittle and unable to generalize to new tasks. We introduce Falconer, a collaborative framework that combines the agentic reasoning of LLMs with lightweight proxy models for scalable knowledge mining. In Falconer, LLMs act as planners, decomposing user instructions into executable pipelines, and as annotators, generating supervision to train small proxies. The framework unifies classification and extraction into two atomic operations, get label and get span, enabling a single instruction-following model to replace multiple task-specific components. To evaluate the consistency between proxy models incubated by Falconer and annotations provided by humans and large models, we construct new benchmarks covering both planning and end-to-end execution. Experiments show that Falconer closely matches state-of-the-art LLMs in instruction-following accuracy while reducing inference cost by up to 90% and accelerating large-scale knowledge mining by more than 20x, offering an efficient and scalable foundation for Deep Research.

Method: The paper proposes PeerBench - a community-governed, proctored evaluation blueprint featuring: sealed execution to prevent data leakage, item banking with rolling renewal to maintain test freshness, and delayed transparency to ensure fair assessment while preventing gaming of the system.

Result: The paper introduces a prototype implementation of PeerBench (available at https://www.peerbench.ai/) as a blueprint for trustworthy AI evaluation, though specific empirical results are not detailed in the abstract.

Conclusion: The current laissez-faire approach to AI evaluation is unsustainable. A paradigm shift to unified, live, and quality-controlled benchmarking frameworks like PeerBench is essential for restoring integrity and delivering genuinely trustworthy measures of AI progress.

Abstract: The meteoric rise of AI, with its rapidly expanding market capitalization, presents both transformative opportunities and critical challenges. Chief among these is the urgent need for a new, unified paradigm for trustworthy evaluation, as current benchmarks increasingly reveal critical vulnerabilities. Issues like data contamination and selective reporting by model developers fuel hype, while inadequate data quality control can lead to biased evaluations that, even if unintentionally, may favor specific approaches. As a flood of participants enters the AI space, this “Wild West” of assessment makes distinguishing genuine progress from exaggerated claims exceptionally difficult. Such ambiguity blurs scientific signals and erodes public confidence, much as unchecked claims would destabilize financial markets reliant on credible oversight from agencies like Moody’s. In high-stakes human examinations (e.g., SAT, GRE), substantial effort is devoted to ensuring fairness and credibility; why settle for less in evaluating AI, especially given its profound societal impact? This position paper argues that the current laissez-faire approach is unsustainable. We contend that true, sustainable AI advancement demands a paradigm shift: a unified, live, and quality-controlled benchmarking framework robust by construction, not by mere courtesy and goodwill. To this end, we dissect the systemic flaws undermining today’s AI evaluation, distill the essential requirements for a new generation of assessments, and introduce PeerBench (with its prototype implementation at https://www.peerbench.ai/), a community-governed, proctored evaluation blueprint that embodies this paradigm through sealed execution, item banking with rolling renewal, and delayed transparency. Our goal is to pave the way for evaluations that can restore integrity and deliver genuinely trustworthy measures of AI progress.

Method: PACT extends HumanEval+ and MBPP+ with contract-violating test cases, enables systematic analysis of code generation under different prompting conditions, and introduces novel metrics to quantify contract adherence in test and code generation.

Result: The analysis shows that augmenting prompts with contract-violating test cases significantly improves models’ ability to respect contracts compared to using contract descriptions alone.

Conclusion: PACT provides rigorous and interpretable metrics to evaluate LLM-generated code robustness in both functionality and contract-adherence, revealing critical errors that conventional benchmarks overlook.

Abstract: Prevailing code generation benchmarks, such as HumanEval+ and MBPP+, primarily evaluate large language models (LLMs) with pass@k on functional correctness using well-formed inputs. However, they ignore a crucial aspect of real-world software: adherence to contracts-the preconditions and validity constraints that dictate how ill-formed inputs must be rejected. This critical oversight means that existing benchmarks fail to measure, and models consequently fail to generate, truly robust and reliable code snippets. We introduce PACT, a program assessment and contract-adherence evaluation framework, to bridge this gap. PACT is the first framework designed to systematically evaluate and enhance contract-adherence in LLM-generated code snippets alongside functional correctness. PACT’s contributions are threefold: First, it provides a comprehensive test-suite corpus focused on contract violations, extending HumanEval+ and MBPP+. Second, it enables a systematic analysis of code generation under varied prompting conditions. This analysis demonstrates that augmenting prompts with contract-violating test cases significantly enhance a model’s ability to respect contracts compared to using contract description alone. Finally, it introduces novel metrics to rigorously quantify contract adherence in both test generation and code generation. By revealing critical errors that conventional benchmarks overlook, PACT provides the rigorous and interpretable metrics to evaluate the robustness of LLM-generated code snippets in both functionality and contract-adherence. Our code and data are available at https://github.com/suhanmen/PACT.

Method: The paper introduces tensor logic where the sole construct is tensor equations, unifying logical rules with Einstein summation notation to implement various AI approaches.

Result: Tensor logic enables elegant implementation of transformers, formal reasoning, kernel machines, and graphical models, and enables new capabilities like sound reasoning in embedding space.

Conclusion: Tensor logic combines neural networks’ scalability and learnability with symbolic reasoning’s reliability and transparency, potentially enabling wider AI adoption.

Abstract: Progress in AI is hindered by the lack of a programming language with all the requisite features. Libraries like PyTorch and TensorFlow provide automatic differentiation and efficient GPU implementation, but are additions to Python, which was never intended for AI. Their lack of support for automated reasoning and knowledge acquisition has led to a long and costly series of hacky attempts to tack them on. On the other hand, AI languages like LISP an Prolog lack scalability and support for learning. This paper proposes tensor logic, a language that solves these problems by unifying neural and symbolic AI at a fundamental level. The sole construct in tensor logic is the tensor equation, based on the observation that logical rules and Einstein summation are essentially the same operation, and all else can be reduced to them. I show how to elegantly implement key forms of neural, symbolic and statistical AI in tensor logic, including transformers, formal reasoning, kernel machines and graphical models. Most importantly, tensor logic makes new directions possible, such as sound reasoning in embedding space. This combines the scalability and learnability of neural networks with the reliability and transparency of symbolic reasoning, and is potentially a basis for the wider adoption of AI.

Method: Systematically transforms canonical puzzles through three dimensions: Increased Complexity (IC), Uncommon Elements (UE), and Unsolvable Puzzles (UP) to reduce reliance on shortcut memorization.

Result: Evaluations show significant performance drops even for top-performing models, with increased complexity being the dominant difficulty source, but models also struggle with subtle rule variations that don’t increase puzzle difficulty.

Conclusion: HardcoreLogic exposes limitations of current LRMs in genuine reasoning and establishes a benchmark for advancing high-level logical reasoning capabilities.

Abstract: Large Reasoning Models (LRMs) have demonstrated impressive performance on complex tasks, including logical puzzle games that require deriving solutions satisfying all constraints. However, whether they can flexibly apply appropriate rules to varying conditions, particularly when faced with non-canonical game variants, remains an open question. Existing corpora focus on popular puzzles like 9x9 Sudoku, risking overfitting to canonical formats and memorization of solution patterns, which can mask deficiencies in understanding novel rules or adapting strategies to new variants. To address this, we introduce HardcoreLogic, a challenging benchmark of over 5,000 puzzles across 10 games, designed to test the robustness of LRMs on the “long-tail” of logical games. HardcoreLogic systematically transforms canonical puzzles through three dimensions: Increased Complexity (IC), Uncommon Elements (UE), and Unsolvable Puzzles (UP), reducing reliance on shortcut memorization. Evaluations on a diverse set of LRMs reveal significant performance drops, even for models achieving top scores on existing benchmarks, indicating heavy reliance on memorized stereotypes. While increased complexity is the dominant source of difficulty, models also struggle with subtle rule variations that do not necessarily increase puzzle difficulty. Our systematic error analysis on solvable and unsolvable puzzles further highlights gaps in genuine reasoning. Overall, HardcoreLogic exposes the limitations of current LRMs and establishes a benchmark for advancing high-level logical reasoning.

Method: Uses automatic VA label generation algorithm for large-scale annotation, multi-task learning framework combining VA regression with auxiliary tasks (emotion, body size, gender), and Audio Transformer model.

Result: Achieved validation Valence Pearson correlation of r = 0.9024 and Arousal r = 0.7155, effectively resolving confusion between discrete categories like ’territorial’ and ‘happy’.

Conclusion: First continuous VA framework for pet vocalization analysis, offering expressive representation for human-pet interaction, veterinary diagnostics, and behavioral training with strong potential for consumer product deployment.

Abstract: Traditional pet emotion recognition from vocalizations, based on discrete classification, struggles with ambiguity and capturing intensity variations. We propose a continuous Valence-Arousal (VA) model that represents emotions in a two-dimensional space. Our method uses an automatic VA label generation algorithm, enabling large-scale annotation of 42,553 pet vocalization samples. A multi-task learning framework jointly trains VA regression with auxiliary tasks (emotion, body size, gender) to enhance prediction by improving feature learning. Our Audio Transformer model achieves a validation Valence Pearson correlation of r = 0.9024 and an Arousal r = 0.7155, effectively resolving confusion between discrete categories like “territorial” and “happy.” This work introduces the first continuous VA framework for pet vocalization analysis, offering a more expressive representation for human-pet interaction, veterinary diagnostics, and behavioral training. The approach shows strong potential for deployment in consumer products like AI pet emotion translators.

Method: Used a classical guitar model printed in PLA on Prusa Mark 4, divided into sections due to build plate constraints, assembled with press-fit connections and minimal adhesive, then tested with nylon strings using Audacity software for frequency analysis.

Result: The 3D-printed guitar achieved playable functionality with accurate pitches through tuning, though large frequency deviations occurred in lower strings due to PLA material properties.

Conclusion: 3D printing with PLA can produce affordable, playable acoustic guitars despite material limitations, with potential for alternative plastics to improve frequency matching and expand musical instrument accessibility.

Abstract: This research investigates the feasibility of producing affordable, functional acoustic guitars using 3D printing, with a focus on producing structural designs with proper tonal performance. Conducted in collaboration with William Schiesser, the study uses a classical guitar model, chosen for its lower string tension, to evaluate the tonal characteristics of a 3D-printed prototype made from polylactic acid (PLA). Due to the build plate size constraints of the Prusa Mark 4 printer, the guitar body was divided into multiple sections joined with press-fit tolerances and minimal cyanoacrylate adhesive. CAD modeling in Fusion 360 ensured dimensional accuracy in press-fit connections and the overall assembly. Following assembly, the guitar was strung with nylon strings and tested using Audacity software to compare recorded frequencies and notes with standard reference values. Results showed large deviations in lower string frequencies, likely caused by the material choice utilized in printing. Accurate pitches were reached with all strings despite frequency differences through tuning, demonstrating that PLA and modern manufacturing methods can produce affordable, playable acoustic guitars despite inevitable challenges. Further research may investigate alternative plastics for superior frequency matching. This approach holds significant potential for expanding access to quality instruments while reducing reliance on endangered tonewoods, thereby encouraging both sustainable instrument production and increased musical participation. This also creates opportunities for disadvantaged communities where access to musical instruments remains a challenge. Keywords: Luthiery, Stereolithography, 3D-Print, Guitar Making

Method: Proposes MotionBeat with two new objectives: Embodied Contrastive Loss (ECL) with tempo-aware and beat-jitter negatives for fine-grained rhythmic discrimination, and Structural Rhythm Alignment Loss (SRAL) for aligning music accents with motion events. Uses bar-equivariant phase rotations and contact-guided attention.

Result: MotionBeat outperforms state-of-the-art audio encoders in music-to-dance generation and transfers effectively to beat tracking, music tagging, genre/instrument classification, emotion recognition, and audio-visual retrieval.

Conclusion: The framework successfully bridges the gap between auditory and embodied music understanding by learning motion-aligned representations that capture rhythmic and structural cues essential for movement.

Abstract: Music is both an auditory and an embodied phenomenon, closely linked to human motion and naturally expressed through dance. However, most existing audio representations neglect this embodied dimension, limiting their ability to capture rhythmic and structural cues that drive movement. We propose MotionBeat, a framework for motion-aligned music representation learning. MotionBeat is trained with two newly proposed objectives: the Embodied Contrastive Loss (ECL), an enhanced InfoNCE formulation with tempo-aware and beat-jitter negatives to achieve fine-grained rhythmic discrimination, and the Structural Rhythm Alignment Loss (SRAL), which ensures rhythm consistency by aligning music accents with corresponding motion events. Architecturally, MotionBeat introduces bar-equivariant phase rotations to capture cyclic rhythmic patterns and contact-guided attention to emphasize motion events synchronized with musical accents. Experiments show that MotionBeat outperforms state-of-the-art audio encoders in music-to-dance generation and transfers effectively to beat tracking, music tagging, genre and instrument classification, emotion recognition, and audio-visual retrieval. Our project demo page: https://motionbeat2025.github.io/.

Method: Uses interleaved token sequences in a discrete autoregressive backbone with modality-specific decoders, supporting multi-speaker and multi-style cloning, and enables gesture-only synthesis from speech inputs.

Result: Subjective and objective evaluations demonstrate competitive speech quality and improved gesture generation over unimodal baselines.

Conclusion: Gelina provides a unified approach for joint speech and gesture synthesis that maintains better synchronization and prosody alignment than sequential methods.

Abstract: Human communication is multimodal, with speech and gestures tightly coupled, yet most computational methods for generating speech and gestures synthesize them sequentially, weakening synchrony and prosody alignment. We introduce Gelina, a unified framework that jointly synthesizes speech and co-speech gestures from text using interleaved token sequences in a discrete autoregressive backbone, with modality-specific decoders. Gelina supports multi-speaker and multi-style cloning and enables gesture-only synthesis from speech inputs. Subjective and objective evaluations demonstrate competitive speech quality and improved gesture generation over unimodal baselines.

Method: Probe models’ internal states and propose Adaptive Vector Steering (AVS) to better ground generation in audio content by steering internal representations.

Result: Consistent performance gains across two models and two benchmarks: F1-score increased from 0.550 to 0.619 for Gemma and 0.626 to 0.632 for Qwen on Audio Hallucination QA, and accuracy increased from 0.548 to 0.592 for Qwen on MMAU (8% relative increase).

Conclusion: This is the first work to apply vector steering to mitigate hallucination in audio, showing strong correlation between output correctness and internal representations.

Abstract: Large Audio-Language Models and Multi-Modal Large Language Models have demonstrated strong capabilities in tasks such as Audio Question Answering (AQA), Audio Captioning, and Automatic Speech Recognition (ASR). However, there is growing evidence that these models can hallucinate about the content of the audio. To address this issue, we probe the models’ internal states and propose Adaptive Vector Steering (AVS), a method that better grounds generation in audio content. We also identify a strong correlation between output correctness and internal representations. Experiments show consistent performance gains across two models and two benchmarks. On the Audio Hallucination QA dataset, our method boosts the F1-score of Gemma from 0.550 to 0.619 and Qwen from 0.626 to 0.632. Furthermore, our method increases the accuracy of Qwen on MMAU from 0.548 to 0.592, marking an 8% relative increase. To the best of our knowledge, this is the first work to apply vector steering to mitigate hallucination in audio.

Method: Proposes VCTR with Hybrid Perception Block (HPB) and Dual Pruned Self-Attention (DPSA) combined with contrastive learning-based adversarial approach to capture both local and global semantics.

Result: The method is presented as an efficient solution for non-parallel voice conversion, though specific quantitative results are not provided in the abstract.

Conclusion: VCTR addresses the limitations of previous methods by incorporating attention mechanisms to better capture long-range dependencies while maintaining efficient training.

Abstract: Non-parallel voice conversion aims to convert voice from a source domain to a target domain without paired training data. Cycle-Consistent Generative Adversarial Networks (CycleGAN) and Variational Autoencoders (VAE) have been used for this task, but these models suffer from difficult training and unsatisfactory results. Later, Contrastive Voice Conversion (CVC) was introduced, utilizing a contrastive learning-based approach to address these issues. However, these methods use CNN-based generators, which can capture local semantics but lacks the ability to capture long-range dependencies necessary for global semantics. In this paper, we propose VCTR, an efficient method for non-parallel voice conversion that leverages the Hybrid Perception Block (HPB) and Dual Pruned Self-Attention (DPSA) along with a contrastive learning-based adversarial approach. The code can be found in https://github.com/Maharnab-Saikia/VCTR.

Method: Proposes UniMoE-Audio with Dynamic-Capacity MoE framework featuring Top-P routing, hybrid experts (routed, shared, null), and three-stage training: Independent Specialist Training, MoE Integration and Warmup, and Synergistic Joint Training.

Result: Achieves state-of-the-art performance on major speech and music generation benchmarks, demonstrates superior synergistic learning, and mitigates performance degradation seen in naive joint training.

Conclusion: Specialized MoE architecture and curated training strategies show substantial potential for advancing universal audio generation by effectively unifying speech and music synthesis.

Abstract: Recent advances in unified multimodal models indicate a clear trend towards comprehensive content generation. However, the auditory domain remains a significant challenge, with music and speech often developed in isolation, hindering progress towards universal audio synthesis. This separation stems from inherent task conflicts and severe data imbalances, which impede the development of a truly unified audio generation model. To address this challenge, we propose UniMoE-Audio, a unified speech and music generation model within a novel Dynamic-Capacity Mixture-of-Experts (MoE) framework. Architecturally, UniMoE-Audio introduces a Top-P routing strategy for dynamic expert number allocation, and a hybrid expert design comprising routed experts for domain-specific knowledge, shared experts for domain-agnostic features, and null experts for adaptive computation skipping. To tackle data imbalance, we introduce a three-stage training curriculum: 1) Independent Specialist Training leverages original datasets to instill domain-specific knowledge into each “proto-expert” without interference; 2) MoE Integration and Warmup incorporates these specialists into the UniMoE-Audio architecture, warming up the gate module and shared expert using a subset of balanced dataset; and 3) Synergistic Joint Training trains the entire model end-to-end on the fully balanced dataset, fostering enhanced cross-domain synergy. Extensive experiments show that UniMoE-Audio not only achieves state-of-the-art performance on major speech and music generation benchmarks, but also demonstrates superior synergistic learning, mitigating the performance degradation typically seen in naive joint training. Our findings highlight the substantial potential of specialized MoE architecture and curated training strategies in advancing the field of universal audio generation. Homepage: https://mukioxun.github.io/Uni-MoE-site/home.html

Method: Freezes both audio encoder and LLM decoder, trains only a lightweight steering module integrated within encoder layers using Mixture-of-Experts router to dynamically select and apply learned steering vectors, transforming audio representations into LLM-comprehensible space.

Result: Achieves strong performance on ASR, audio understanding, and function-calling tasks while remaining highly modular and computationally efficient.

Conclusion: SteerMoE offers a robust new paradigm for developing sophisticated audio-language systems by operating entirely in continuous embedding space without modifying LLM vocabulary, preserving reasoning and agentic capabilities.

Abstract: Aligning pretrained audio encoders and Large Language Models (LLMs) offers a promising, parameter-efficient path to building powerful multimodal agents. However, existing methods often require costly full-model finetuning or rely on static adapters that may lack expressive power. Drawing inspiration from the Platonic Representation Hypothesis, we introduce SteerMoE, a novel and modular framework for audio-language alignment. SteerMoE freezes both the audio encoder and the LLM decoder, training only a lightweight steering module integrated within the encoder’s layers. This module uses a Mixture-of-Experts (MoE) router to dynamically select and apply learned steering vectors, progressively transforming continuous audio representations into a space comprehensible to the LLM. By operating entirely in the continuous embedding space, our approach requires no modifications to the LLM’s vocabulary and preserves its advanced reasoning and agentic capabilities. We demonstrate through experiments on ASR, audio understanding, and a qualitative function-calling task that SteerMoE achieves strong performance while remaining highly modular and computationally efficient, offering a robust new paradigm for developing sophisticated audio-language systems.

Method: Uses latent-domain predictive coding conditioned on past quantized latent frames, learnable time-frequency compression, and differentiable vector quantization with distance-to-soft mapping and Gumbel-Softmax.

Result: At 1 kbps, TF-Codec significantly outperforms Opus at 9 kbps; at 3 kbps, it beats both EVS at 9.6 kbps and Opus at 12 kbps with low latency.

Conclusion: The proposed techniques effectively remove temporal redundancies and enable high-quality neural speech coding at ultra-low bitrates with low latency.

Abstract: Neural audio/speech coding has recently demonstrated its capability to deliver high quality at much lower bitrates than traditional methods. However, existing neural audio/speech codecs employ either acoustic features or learned blind features with a convolutional neural network for encoding, by which there are still temporal redundancies within encoded features. This paper introduces latent-domain predictive coding into the VQ-VAE framework to fully remove such redundancies and proposes the TF-Codec for low-latency neural speech coding in an end-to-end manner. Specifically, the extracted features are encoded conditioned on a prediction from past quantized latent frames so that temporal correlations are further removed. Moreover, we introduce a learnable compression on the time-frequency input to adaptively adjust the attention paid to main frequencies and details at different bitrates. A differentiable vector quantization scheme based on distance-to-soft mapping and Gumbel-Softmax is proposed to better model the latent distributions with rate constraint. Subjective results on multilingual speech datasets show that, with low latency, the proposed TF-Codec at 1 kbps achieves significantly better quality than Opus at 9 kbps, and TF-Codec at 3 kbps outperforms both EVS at 9.6 kbps and Opus at 12 kbps. Numerous studies are conducted to demonstrate the effectiveness of these techniques. Code and models are available at https://github.com/microsoft/TF-Codec.

Method: UniCodec uses a low-bitrate neural codec to learn semantically-disentangled unified tokens at global and local scales, with knowledge distilled from self-supervised learned features.

Result: Extensive multilingual evaluations show UniCodec generates natural, expressive speech with long-term consistency and well-preserved paralinguistic attributes across various speech processing tasks.

Conclusion: The unified token approach benefits both speech understanding with paralinguistic hints and high-quality speech generation, overcoming limitations of current semantic-modeling and acoustic-synthesis paradigms.

Abstract: Current large speech language models are mainly based on semantic tokens from discretization of self-supervised learned representations and acoustic tokens from a neural codec, following a semantic-modeling and acoustic-synthesis paradigm. However, semantic tokens discard paralinguistic attributes of speakers that is important for natural spoken communication, while prompt-based acoustic synthesis from semantic tokens has limits in recovering paralinguistic details and suffers from robustness issues, especially when there are domain gaps between the prompt and the target. This paper unifies two types of tokens and proposes the UniCodec, a universal speech token learning that encapsulates all semantics of speech, including linguistic and paralinguistic information, into a compact and semantically-disentangled unified token. Such a unified token can not only benefit speech language models in understanding with paralinguistic hints but also help speech generation with high-quality output. A low-bitrate neural codec is leveraged to learn such disentangled discrete representations at global and local scales, with knowledge distilled from self-supervised learned features. Extensive evaluations on multilingual datasets demonstrate its effectiveness in generating natural, expressive and long-term consistent output quality with paralinguistic attributes well preserved in several speech processing tasks.

Method: LAL introduces audio representations via attention mechanism in LLM layers, bypassing feedforward modules. PAL uses PLITS for Whisper and LAL for general audio encoders.

Result: LAL outperforms PLITS by up to 30% on general audio tasks while reducing memory usage by 64.1% and increasing throughput by 247.5%. PAL performs on par with full PLITS but with better efficiency.

Conclusion: LAL provides an efficient alternative to PLITS for audio-LLM integration, and PAL offers a balanced approach for different audio encoders, achieving strong performance with substantial computational benefits.

Abstract: Integration of audio perception into large language models (LLMs) is an emerging research area for enabling machine listening applications, yet efficient transfer of rich audio semantics from audio encoders to LLMs remains underexplored. The most widely used integration paradigm projects the audio encoder output tokens into the LLM input space (e.g., via an MLP or a Q-Former), then prepends or inserts them to the text tokens. We refer to this generic scheme as Prepend to the LLM’s input token space (PLITS) integration. We propose an efficient alternative, Lightweight Audio LLM Integration (LAL). LAL introduces audio representations solely via the attention mechanism within different layers of the LLM, bypassing its feedforward module. LAL encodes rich audio semantics at an appropriate level of abstraction for integration into different blocks of LLMs. Our design significantly reduces computational overhead compared to existing integration approaches. Observing with Whisper that the speech encoder benefits from PLITS integration, we propose an audio encoder aware approach for efficiently Probing Audio encoders via LLM (PAL), which employs PLITS integration for Whisper and LAL for general audio encoders. Under an identical training curriculum, LAL consistently maintains performance or outperforms existing integration approaches across multiple base LLMs and tasks. For general audio tasks, LAL improvement is up to 30% over a strong PLITS baseline while reducing memory usage by up to 64.1% and increasing throughput by up to 247.5%. Furthermore, for general audio-music-speech LLM, PAL performs on par with a fully PLITS integration-based system but with substantially improved computational and memory efficiency. Project page: https://ta012.github.io/PAL/

Method: AGR mimics human coarse-to-fine cognition: transforms audio to coarse text document, then uses proactive iterative refinement with tool-augmented routes and specialized agents to continuously search for missing information and augment evidence chains.

Result: AGR achieves state-of-the-art performance over existing open-source audio deep reasoning models across various benchmarks.

Conclusion: The proposed paradigm shift successfully unlocks the potential of large language models for audio deep reasoning by transforming it into a text understanding task with iterative refinement.

Abstract: Audio deep reasoning is a challenging task that requires expert-level perception, multi-step logical inference, and the integration of contextual knowledge. However, existing models suffer from a gap between audio perception and reasoning abilities due to the lack of training data with explicit reasoning chains and the absence of mechanisms for active exploration and iterative refinement. To address these challenges, we propose AudioGenie-Reasoner (AGR), the first unified training-free multi-agent system that coordinates perception and reasoning over an evolving chain of textual evidence. Our key idea is a paradigm shift that transforms audio deep reasoning into complex text understanding task from a new perspective, thereby unlocking the full potential of large language models. Specifically, the design of AGR mimics the human coarse-to-fine cognitive process. It first transforms the input audio into a coarse text-based document. Then, we design a novel proactive iterative document refinement loop, featuring tool-augmented routes and specialized agents, to continuously search for missing information and augment the evidence chain in a coarse-to-fine manner until sufficient question-related information is gathered for making final predictions. Experimental results show that AGR achieves state-of-the-art (SOTA) performance over existing open-source audio deep reasoning models across various benchmarks. The code will be available at https://github.com/ryysayhi/AudioGenie-Reasoner.

Method: Attribute-Specialized Key-Value Head Sharing (AS-KVHS) adapted to music’s structured symbolic representation, plus systematic study of BPE’s generalizability in multi-track symbolic music.

Result: Achieved about 30% inference speedup with only 0.4% quality drop in objective evaluations and slight improvements in subjective listening tests. Also released SAGE-Music benchmark matching/surpassing state-of-the-art models.

Conclusion: AS-KVHS effectively addresses speed-quality trade-off in symbolic music generation, with contributions including systematic BPE analysis and open-source benchmark SAGE-Music.

Abstract: Low-latency symbolic music generation is essential for real-time improvisation and human-AI co-creation. Existing transformer-based models, however, face a trade-off between inference speed and musical quality. Traditional acceleration techniques such as embedding pooling significantly degrade quality, while recently proposed Byte Pair Encoding (BPE) methods - though effective on single-track piano data - suffer large performance drops in multi-track settings, as revealed by our analysis. We propose Attribute-Specialized Key-Value Head Sharing (AS-KVHS), adapted to music’s structured symbolic representation, achieving about 30% inference speedup with only a negligible (about 0.4%) quality drop in objective evaluations and slight improvements in subjective listening tests. Our main contributions are (1) the first systematic study of BPE’s generalizability in multi-track symbolic music, and (2) the introduction of AS-KVHS for low-latency symbolic music generation. Beyond these, we also release SAGE-Music, an open-source benchmark that matches or surpasses state-of-the-art models in generation quality.

Method: Two-stage approach: 1) Transform audio to mel-spectrograms, compress and quantize into single tokens using 2D tokenizer with perceptual loss to reduce artifacts; 2) Use vocoder to recover waveforms from mel discrete tokens in single forward pass for real-time decoding.

Result: Both objective and subjective evaluations show MelCap achieves quality comparable to state-of-the-art multi-codebook codecs while maintaining the computational simplicity of single-codebook design.

Conclusion: MelCap provides an effective unified neural codec that bridges the gap between single-codebook and multi-codebook approaches, offering high-quality compression across audio domains with practical computational efficiency for downstream tasks.

Abstract: Neural audio codecs have recently emerged as powerful tools for high-quality and low-bitrate audio compression, leveraging deep generative models to learn latent representations of audio signals. However, existing approaches either rely on a single quantizer that only processes speech domain, or on multiple quantizers that are not well suited for downstream tasks. To address this issue, we propose MelCap, a unified “one-codebook-for-all” neural codec that effectively handles speech, music, and general sound. By decomposing audio reconstruction into two stages, our method preserves more acoustic details than previous single-codebook approaches, while achieving performance comparable to mainstream multi-codebook methods. In the first stage, audio is transformed into mel-spectrograms, which are compressed and quantized into compact single tokens using a 2D tokenizer. A perceptual loss is further applied to mitigate the over-smoothing artifacts observed in spectrogram reconstruction. In the second stage, a Vocoder recovers waveforms from the mel discrete tokens in a single forward pass, enabling real-time decoding. Both objective and subjective evaluations demonstrate that MelCap achieves quality on comparable to state-of-the-art multi-codebook codecs, while retaining the computational simplicity of a single-codebook design, thereby providing an effective representation for downstream tasks.

Method: Proposes Local Timescale Gating (LT-Gate) where each neuron tracks information on fast and slow timescales in parallel, with a learned gate locally adjusting their influence. Also introduces variance-tracking regularization for firing stability.

Result: Achieves about 51% final accuracy on challenging temporal classification benchmark, outperforming recent Hebbian continual-learning baseline (46%) and prior SNN methods. Successfully demonstrated on Intel’s Loihi chip.

Conclusion: Multi-timescale gating substantially enhances continual learning in SNNs, narrowing the gap between spiking and conventional deep networks on lifelong-learning tasks while maintaining hardware compatibility.

Abstract: Spiking neural networks (SNNs) promise energy-efficient artificial intelligence on neuromorphic hardware but struggle with tasks requiring both fast adaptation and long-term memory, especially in continual learning. We propose Local Timescale Gating (LT-Gate), a neuron model that combines dual time-constant dynamics with an adaptive gating mechanism. Each spiking neuron tracks information on a fast and a slow timescale in parallel, and a learned gate locally adjusts their influence. This design enables individual neurons to preserve slow contextual information while responding to fast signals, addressing the stability-plasticity dilemma. We further introduce a variance-tracking regularization that stabilizes firing activity, inspired by biological homeostasis. Empirically, LT-Gate yields significantly improved accuracy and retention in sequential learning tasks: on a challenging temporal classification benchmark it achieves about 51 percent final accuracy, compared to about 46 percent for a recent Hebbian continual-learning baseline and lower for prior SNN methods. Unlike approaches that require external replay or expensive orthogonalizations, LT-Gate operates with local updates and is fully compatible with neuromorphic hardware. In particular, it leverages features of Intel’s Loihi chip (multiple synaptic traces with different decay rates) for on-chip learning. Our results demonstrate that multi-timescale gating can substantially enhance continual learning in SNNs, narrowing the gap between spiking and conventional deep networks on lifelong-learning tasks.

Method: Conduct thorough investigation into pseudo-alignment causes, connect it to cone effect in LLMs, and introduce TimeSUP technique that increases time series manifold dimension to match language embeddings’ intrinsic dimension.

Result: TimeSUP consistently outperforms state-of-the-art LLM4TS methods and lightweight baselines on long-term forecasting, and can be seamlessly integrated into four existing LLM4TS pipelines with significant performance improvements.

Conclusion: Pseudo-alignment arises from cone effect in pretrained LLMs and low-dimensional time-series manifolds. TimeSUP effectively mitigates this by aligning manifold dimensions while preserving modality-specific features in a unified embedding space.

Abstract: Pseudo-Alignment is a pervasive challenge in many large language models for time series (LLM4TS) models, often causing them to underperform compared to linear models or randomly initialised backbones. However, there is limited discussion in the community for the reasons that pseudo-alignment occurs. In this work, we conduct a thorough investigation into the root causes of pseudo-alignment in LLM4TS and build a connection of pseudo-alignment to the cone effect in LLM. We demonstrate that pseudo-alignment arises from the interplay of cone effect within pretrained LLM components and the intrinsically low-dimensional manifold of time-series data. In addition, we also introduce \textit{\textbf{TimeSUP}}, a novel technique designed to mitigate this issue and improve forecast performance in existing LLM4TS approaches. TimeSUP addresses this by increasing the time series manifold to more closely match the intrinsic dimension of language embeddings, allowing the model to distinguish temporal signals clearly while still capturing shared structures across modalities. As a result, representations for time and language tokens remain distinct yet exhibit high cosine similarity, signifying that the model preserves each modality unique features while learning their commonalities in a unified embedding space. Empirically, TimeSUP consistently outperforms state-of-the-art LLM4TS methods and other lightweight baselines on long-term forecasting performance. Furthermore, it can be seamlessly integrated into four existing LLM4TS pipelines and delivers significant improvements in forecasting performance.

Method: Uses Cardinality-Agnostic Task Encoder (CATE) to produce Gaussian-distributed task embeddings that encode task knowledge and quantify uncertainty. Employs 2-Wasserstein distance on server side to measure inter-task gaps and formulate Wasserstein loss for inter-task separation.

Result: Extensive evaluations show FedGTEA achieves superior classification performance, significantly mitigates forgetting, and consistently outperforms existing baselines on popular datasets.

Conclusion: FedGTEA provides an effective solution for federated class incremental learning that captures task-specific knowledge, handles uncertainty, ensures privacy, and maintains scalability across long task sequences.

Abstract: We introduce a novel framework for Federated Class Incremental Learning, called Federated Gaussian Task Embedding and Alignment (FedGTEA). FedGTEA is designed to capture task-specific knowledge and model uncertainty in a scalable and communication-efficient manner. At the client side, the Cardinality-Agnostic Task Encoder (CATE) produces Gaussian-distributed task embeddings that encode task knowledge, address statistical heterogeneity, and quantify data uncertainty. Importantly, CATE maintains a fixed parameter size regardless of the number of tasks, which ensures scalability across long task sequences. On the server side, FedGTEA utilizes the 2-Wasserstein distance to measure inter-task gaps between Gaussian embeddings. We formulate the Wasserstein loss to enforce inter-task separation. This probabilistic formulation not only enhances representation learning but also preserves task-level privacy by avoiding the direct transmission of latent embeddings, aligning with the privacy constraints in federated learning. Extensive empirical evaluations on popular datasets demonstrate that FedGTEA achieves superior classification performance and significantly mitigates forgetting, consistently outperforming strong existing baselines.

Method: Use Equilibrium Propagation (EP) on Oscillator Ising Machines (OIMs) to unify optimization and sampling through local learning rules without global backpropagation.

Result: Achieved competitive accuracy: ~97.2% on MNIST and ~88.0% on Fashion-MNIST, with robustness to parameter quantization and phase noise.

Conclusion: OIMs are established as fast, energy-efficient neuromorphic learning substrates that can practically realize energy-based models on physical hardware.

Abstract: Physical systems that naturally perform energy descent offer a direct route to accelerating machine learning. Oscillator Ising Machines (OIMs) exemplify this idea: their GHz-frequency dynamics mirror both the optimization of energy-based models (EBMs) and gradient descent on loss landscapes, while intrinsic noise corresponds to Langevin dynamics - supporting sampling as well as optimization. Equilibrium Propagation (EP) unifies these processes into descent on a single total energy landscape, enabling local learning rules without global backpropagation. We show that EP on OIMs achieves competitive accuracy ($\sim 97.2 \pm 0.1 %$ on MNIST, $\sim 88.0 \pm 0.1 %$ on Fashion-MNIST), while maintaining robustness under realistic hardware constraints such as parameter quantization and phase noise. These results establish OIMs as a fast, energy-efficient substrate for neuromorphic learning, and suggest that EBMs - often bottlenecked by conventional processors - may find practical realization on physical hardware whose dynamics directly perform their optimization.

Method: Developed theoretical framework for certified robustness under pruning in state-adversarial MDPs. Analyzed Gaussian and categorical policies with Lipschitz networks, proving element-wise pruning tightens robustness bounds. Derived three-term regret decomposition to separate clean performance, pruning loss, and robustness gains.

Result: Empirical evaluation on continuous-control benchmarks showed pruning consistently finds ‘sweet spots’ at moderate sparsity where robustness improves substantially without harming (and sometimes enhancing) clean performance.

Conclusion: Pruning serves not just as compression but as a structural intervention for robust RL, exposing a fundamental performance-robustness frontier.

Abstract: Reinforcement learning (RL) policies deployed in real-world environments must remain reliable under adversarial perturbations. At the same time, modern deep RL agents are heavily over-parameterized, raising costs and fragility concerns. While pruning has been shown to improve robustness in supervised learning, its role in adversarial RL remains poorly understood. We develop the first theoretical framework for certified robustness under pruning in state-adversarial Markov decision processes (SA-MDPs). For Gaussian and categorical policies with Lipschitz networks, we prove that element-wise pruning can only tighten certified robustness bounds; pruning never makes the policy less robust. Building on this, we derive a novel three-term regret decomposition that disentangles clean-task performance, pruning-induced performance loss, and robustness gains, exposing a fundamental performance–robustness frontier. Empirically, we evaluate magnitude and micro-pruning schedules on continuous-control benchmarks with strong policy-aware adversaries. Across tasks, pruning consistently uncovers reproducible ``sweet spots’’ at moderate sparsity levels, where robustness improves substantially without harming - and sometimes even enhancing - clean performance. These results position pruning not merely as a compression tool but as a structural intervention for robust RL.

Method: Introduces black-box evaluation tests for probing memorization at embedding and generative levels, with an open-source toolkit for reproducible privacy assessments.

Result: Validated on a publicly available EHR foundation model, demonstrating the framework’s ability to assess memorization risks.

Conclusion: The proposed toolkit facilitates collaborative privacy assessments in healthcare AI, addressing critical privacy concerns in EHR foundation models.

Abstract: Foundation models trained on large-scale de-identified electronic health records (EHRs) hold promise for clinical applications. However, their capacity to memorize patient information raises important privacy concerns. In this work, we introduce a suite of black-box evaluation tests to assess privacy-related memorization risks in foundation models trained on structured EHR data. Our framework includes methods for probing memorization at both the embedding and generative levels, and aims to distinguish between model generalization and harmful memorization in clinically relevant settings. We contextualize memorization in terms of its potential to compromise patient privacy, particularly for vulnerable subgroups. We validate our approach on a publicly available EHR foundation model and release an open-source toolkit to facilitate reproducible and collaborative privacy assessments in healthcare AI.

Method: Multimodal XAI framework unifying attention-augmented feature fusion, Grad-CAM++-based local explanations, and a Reveal-to-Revise feedback loop for bias detection and mitigation.

Result: Achieved 93.2% classification accuracy, 91.6% F1-score, and 78.1% explanation fidelity (IoU-XAI) on multimodal MNIST extensions, outperforming unimodal and non-explainable baselines.

Conclusion: Integrating interpretability with bias-aware learning enhances robustness and human alignment, bridging the gap between performance, transparency, and fairness for trustworthy AI in sensitive domains.

Abstract: Standard benchmark datasets, such as MNIST, often fail to expose latent biases and multimodal feature complexities, limiting the trustworthiness of deep neural networks in high-stakes applications. We propose a novel multimodal Explainable AI (XAI) framework that unifies attention-augmented feature fusion, Grad-CAM++-based local explanations, and a Reveal-to-Revise feedback loop for bias detection and mitigation. Evaluated on multimodal extensions of MNIST, our approach achieves 93.2% classification accuracy, 91.6% F1-score, and 78.1% explanation fidelity (IoU-XAI), outperforming unimodal and non-explainable baselines. Ablation studies demonstrate that integrating interpretability with bias-aware learning enhances robustness and human alignment. Our work bridges the gap between performance, transparency, and fairness, highlighting a practical pathway for trustworthy AI in sensitive domains.

Method: Confident Inlier Extrapolation (CI-EX) iteratively identifies rejected client distribution using outlier detection, then assigns labels to rejected individuals closest to accepted population based on supervised classification probabilities.

Result: CI-EX consistently outperforms existing RI models on RI-specific metrics (Kickout, Area under Kickout) while maintaining competitive AUC performance across most experiments on two real-world credit datasets.

Conclusion: RI methods involve trade-offs between AUC and RI-specific metrics, but CI-EX framework effectively mitigates blind extrapolation and achieves superior performance on reject inference tasks.

Abstract: Reject Inference (RI) methods aim to address sample bias by inferring missing repayment data for rejected credit applicants. Traditional approaches often assume that the behavior of rejected clients can be extrapolated from accepted clients, despite potential distributional differences between the two populations. To mitigate this blind extrapolation, we propose a novel Confident Inlier Extrapolation framework (CI-EX). CI-EX iteratively identifies the distribution of rejected client samples using an outlier detection model and assigns labels to rejected individuals closest to the distribution of the accepted population based on probabilities derived from a supervised classification model. The effectiveness of our proposed framework is validated through experiments on two large real-world credit datasets. Performance is evaluated using the Area Under the Curve (AUC) as well as RI-specific metrics such as Kickout and a novel metric introduced in this work, denoted as Area under the Kickout. Our findings reveal that RI methods, including the proposed framework, generally involve a trade-off between AUC and RI-specific metrics. However, the proposed CI-EX framework consistently outperforms existing RI models from the credit literature in terms of RI-specific metrics while maintaining competitive performance in AUC across most experiments.

Method: The authors show that LID is a lower bound on the denoising score matching loss, motivating its use as a LID estimator. They also connect this to implicit score matching loss and relate it to FLIPD estimator.

Result: Experiments on manifold benchmarks and Stable Diffusion 3.5 show the denoising score matching loss achieves superior accuracy and memory footprint under increasing problem size and quantization levels.

Conclusion: Denoising score matching loss provides a highly competitive and scalable LID estimator that outperforms existing methods in terms of computational efficiency and accuracy.

Abstract: The local intrinsic dimension (LID) of data is a fundamental quantity in signal processing and learning theory, but quantifying the LID of high-dimensional, complex data has been a historically challenging task. Recent works have discovered that diffusion models capture the LID of data through the spectra of their score estimates and through the rate of change of their density estimates under various noise perturbations. While these methods can accurately quantify LID, they require either many forward passes of the diffusion model or use of gradient computation, limiting their applicability in compute- and memory-constrained scenarios. We show that the LID is a lower bound on the denoising score matching loss, motivating use of the denoising score matching loss as a LID estimator. Moreover, we show that the equivalent implicit score matching loss also approximates LID via the normal dimension and is closely related to a recent LID estimator, FLIPD. Our experiments on a manifold benchmark and with Stable Diffusion 3.5 indicate that the denoising score matching loss is a highly competitive and scalable LID estimator, achieving superior accuracy and memory footprint under increasing problem size and quantization level.

Method: Developed OrbitZoo, a versatile multi-agent RL environment built on high-fidelity industry standard orbital dynamics library, supporting scenarios like collision avoidance and cooperative maneuvers.

Result: Validated against real Starlink constellation data, achieving 0.16% Mean Absolute Percentage Error (MAPE), ensuring reliable high-fidelity simulations for autonomous satellite operations.

Conclusion: OrbitZoo provides a robust, validated platform for realistic space operations simulation, enabling development of autonomous satellite control policies with industry-standard accuracy.

Abstract: The increasing number of satellites and orbital debris has made space congestion a critical issue, threatening satellite safety and sustainability. Challenges such as collision avoidance, station-keeping, and orbital maneuvering require advanced techniques to handle dynamic uncertainties and multi-agent interactions. Reinforcement learning (RL) has shown promise in this domain, enabling adaptive, autonomous policies for space operations; however, many existing RL frameworks rely on custom-built environments developed from scratch, which often use simplified models and require significant time to implement and validate the orbital dynamics, limiting their ability to fully capture real-world complexities. To address this, we introduce OrbitZoo, a versatile multi-agent RL environment built on a high-fidelity industry standard library, that enables realistic data generation, supports scenarios like collision avoidance and cooperative maneuvers, and ensures robust and accurate orbital dynamics. The environment is validated against a real satellite constellation, Starlink, achieving a Mean Absolute Percentage Error (MAPE) of 0.16% compared to real-world data. This validation ensures reliability for generating high-fidelity simulations and enabling autonomous and independent satellite operations.

Method: Functional Alignment for Distributional Equivalence (FADE) measures distributional similarity by comparing bidirectional likelihood assignments over generated samples between unlearned and reference models, capturing functional alignment across the entire output distribution.

Result: Experiments on TOFU and UnlearnCanvas benchmarks show that methods achieving near-optimal scores on traditional metrics fail to achieve distributional equivalence, with many becoming more distant from the gold standard than before unlearning.

Conclusion: FADE exposes fundamental gaps in current evaluation practices and provides a more robust foundation for developing and assessing truly effective unlearning methods by measuring genuine distributional equivalence.

Abstract: Current unlearning metrics for generative models evaluate success based on reference responses or classifier outputs rather than assessing the core objective: whether the unlearned model behaves indistinguishably from a model that never saw the unwanted data. This reference-specific approach creates systematic blind spots, allowing models to appear successful while retaining unwanted knowledge accessible through alternative prompts or attacks. We address these limitations by proposing Functional Alignment for Distributional Equivalence (FADE), a novel metric that measures distributional similarity between unlearned and reference models by comparing bidirectional likelihood assignments over generated samples. Unlike existing approaches that rely on predetermined references, FADE captures functional alignment across the entire output distribution, providing a principled assessment of genuine unlearning. Our experiments on the TOFU benchmark for LLM unlearning and the UnlearnCanvas benchmark for text-to-image diffusion model unlearning reveal that methods achieving near-optimal scores on traditional metrics fail to achieve distributional equivalence, with many becoming more distant from the gold standard than before unlearning. These findings expose fundamental gaps in current evaluation practices and demonstrate that FADE provides a more robust foundation for developing and assessing truly effective unlearning methods.

Method: Hybrid architecture combining 4 key components: Convolutional neural network residuals, Adaptive correction layers, ShuffleNet blocks, and Transformers to efficiently capture local and long-range dependencies.

Result: Achieves superior prediction accuracy with substantially lower computational cost, outperforming baselines in 88.9% of TDD and 43.8% of FDD scenarios, while reducing FLOPs by 5x and 3x compared to the strongest baseline.

Conclusion: CSI-4CAST provides an efficient and robust solution for CSI prediction, and the accompanying CSI-RRG benchmark enables standardized evaluation of model performance across diverse channel conditions.

Abstract: Channel state information (CSI) prediction is a promising strategy for ensuring reliable and efficient operation of massive multiple-input multiple-output (mMIMO) systems by providing timely downlink (DL) CSI. While deep learning-based methods have advanced beyond conventional model-driven and statistical approaches, they remain limited in robustness to practical non-Gaussian noise, generalization across diverse channel conditions, and computational efficiency. This paper introduces CSI-4CAST, a hybrid deep learning architecture that integrates 4 key components, i.e., Convolutional neural network residuals, Adaptive correction layers, ShuffleNet blocks, and Transformers, to efficiently capture both local and long-range dependencies in CSI prediction. To enable rigorous evaluation, this work further presents a comprehensive benchmark, CSI-RRG for Regular, Robustness and Generalization testing, which includes more than 300,000 samples across 3,060 realistic scenarios for both TDD and FDD systems. The dataset spans multiple channel models, a wide range of delay spreads and user velocities, and diverse noise types and intensity degrees. Experimental results show that CSI-4CAST achieves superior prediction accuracy with substantially lower computational cost, outperforming baselines in 88.9% of TDD scenarios and 43.8% of FDD scenario, the best performance among all evaluated models, while reducing FLOPs by 5x and 3x compared to LLM4CP, the strongest baseline. In addition, evaluation over CSI-RRG provides valuable insights into how different channel factors affect the performance and generalization capability of deep learning models. Both the dataset (https://huggingface.co/CSI-4CAST) and evaluation protocols (https://github.com/AI4OPT/CSI-4CAST) are publicly released to establish a standardized benchmark and to encourage further research on robust and efficient CSI prediction.

Method: Created ExtremBench - a benchmark dataset with 93 standardized extrema-finding problems curated from Chinese Mathematical Olympiad inequality exercises, and evaluated various state-of-the-art open-source LLMs including Qwen3, GPT-OSS, and DeepSeek.

Result: LLMs’ extremal-solving reasoning capabilities do not align with performance on current mathematical benchmarks like AIME25 and MATH-500, with some models showing strong general mathematical reasoning but poor extremal-solving skills, and vice versa.

Conclusion: Existing benchmarks may not comprehensively capture the full spectrum of mathematical reasoning abilities, highlighting a critical gap in current evaluation practices.

Abstract: Test-time scaling has enabled Large Language Models (LLMs) with remarkable reasoning capabilities, particularly in mathematical domains, through intermediate chain-of-thought (CoT) reasoning before generating final answers. However, the specific sources and mechanisms underlying these reasoning capabilities remain insufficiently understood. Optimization reasoning, i.e. finding extrema under constraints, represents a fundamental abstraction that underpins critical applications in planning, control, resource allocation, and prompt search. To systematically evaluate this capability, we introduce ExtremBench, a benchmark dataset for solving mathematical extremal problems, curated from inequality exercises used for Chinese Mathematical Olympiad and transformed into $93$ standardized extrema-finding problems. We conduct extensive evaluations across various state-of-the-art open-source model families, including the Qwen3, GPT-OSS, and DeepSeek. Our results reveal that LLMs’ extremal-solving reasoning capabilities do not always align with those of current mathematical benchmarks such as AIME25 and MATH-500, with some models showing strong general mathematical reasoning but poor extremal-solving skills, and vice versa. This discrepancy highlights a critical gap in current evaluation practices and suggests that existing benchmarks may not comprehensively capture the full spectrum of mathematical reasoning abilities.

Method: Developed AMORE framework with multi-output operator, adaptive loss functions that consider state variable and sample errors, trunk design for Partition of Unity, and invertible map for mass-fraction constraints. Uses two-step training for DeepONet and extends to FNO.

Result: Demonstrated effectiveness on syngas (12 states) and GRI-Mech 3.0 (24 active states out of 54) examples. The framework successfully handles multiple thermochemical states while enforcing physical constraints.

Conclusion: AMORE provides a general framework for operator learning in stiff systems and will serve as a backbone for accelerating turbulent combustion simulations in future CFD studies.

Abstract: Time integration of stiff systems is a primary source of computational cost in combustion, hypersonics, and other reactive transport systems. This stiffness can introduce time scales significantly smaller than those associated with other physical processes, requiring extremely small time steps in explicit schemes or computationally intensive implicit methods. Consequently, strategies to alleviate challenges posed by stiffness are important. While neural operators (DeepONets) can act as surrogates for stiff kinetics, a reliable operator learning strategy is required to appropriately account for differences in the error between output variables and samples. Here, we develop AMORE, Adaptive Multi-Output Operator Network, a framework comprising an operator capable of predicting multiple outputs and adaptive loss functions ensuring reliable operator learning. The operator predicts all thermochemical states from given initial conditions. We propose two adaptive loss functions within the framework, considering each state variable’s and sample’s error to penalize the loss function. We designed the trunk to automatically satisfy Partition of Unity. To enforce unity mass-fraction constraint exactly, we propose an invertible analytical map that transforms the $n$-dimensional species mass-fraction vector into an ($n-1$)-dimensional space, where DeepONet training is performed. We consider two-step training for DeepONet for multiple outputs and extend adaptive loss functions for trunk and branch training. We demonstrate the efficacy and applicability of our models through two examples: the syngas (12 states) and GRI-Mech 3.0 (24 active states out of 54). The proposed DeepONet will be a backbone for future CFD studies to accelerate turbulent combustion simulations. AMORE is a general framework, and here, in addition to DeepONet, we also demonstrate it for FNO.

Method: Self-attention-based communication module that exchanges information between agents, fully differentiable with fixed number of parameters independent of agent count.

Result: Achieves state-of-the-art performance on SMAC and SMACv2 benchmarks across multiple maps.

Conclusion: The proposed differentiable communication module effectively enables agents to learn reward-driven message generation and can be seamlessly integrated with existing MARL methods.

Abstract: Communication is essential for the collective execution of complex tasks by human agents, motivating interest in communication mechanisms for multi-agent reinforcement learning (MARL). However, existing communication protocols in MARL are often complex and non-differentiable. In this work, we introduce a self-attention-based communication module that exchanges information between the agents in MARL. Our proposed approach is fully differentiable, allowing agents to learn to generate messages in a reward-driven manner. The module can be seamlessly integrated with any action-value function decomposition method and can be viewed as an extension of such decompositions. Notably, it includes a fixed number of trainable parameters, independent of the number of agents. Experimental results on the SMAC and SMACv2 benchmarks demonstrate the effectiveness of our approach, which achieves state-of-the-art performance on a number of maps.

Method: Two-stage reinforcement learning: (1) Natural gradient update using Fisher-vector products and conjugate gradient solver with KL trust-region constraint for safe initialization, (2) Proximal Policy Optimization (PPO) with clipped surrogates for policy refinement using minibatch efficiency.

Result: The proposed initialization method substantially improves generalization performance on both scRNA-seq and scATAC-seq perturbation analysis compared to existing approaches.

Conclusion: The multistage reinforcement learning framework provides an effective data-driven solution for single-cell perturbation modeling that properly escapes local optima and converges to biologically plausible lineages through well-designed initialization and optimization.

Abstract: Modeling cellular responses to genetic and chemical perturbations remains a central challenge in single-cell biology. Existing data-driven framework have advanced perturbation prediction through variational autoencoders, chemically conditioned autoencoders, and large-scale transformer pretraining. However, these models are prone to local optima in the nonconvex Waddington landscape of cell fate decisions, where poor initialization can trap trajectories in spurious lineages or implausible differentiation outcomes. While executable gene regulatory networks complement these approaches, automated design frameworks incorporate biological priors through multi-agent optimization. Yet, an approach that is completely data-driven with well-designed initialization to escape local optima and converge to a proper lineage remains elusive. In this work, we introduce a multistage reinforcement learning algorithm tailored for single-cell perturbation modeling. We first compute an explicit natural gradient update using Fisher-vector products and a conjugate gradient solver, scaled by a KL trust-region constraint to provide a safe, curvature-aware the first step for the policy. Starting with these preconditioned parameters, we then apply a second phase of proximal policy optimization (PPO) with clipped surrogates, exploiting minibatch efficiency to refine the policy. We demonstrate that this initialization substantially improves generalization on Single-cell RNA sequencing (scRNA-seq) and Single-cell ATAC sequencing (scATAC-seq) pertubation analysis.

Method: Proposed workflow includes: 1) Reduced-order Helmholtz model based on Lamb wave theory for generating comprehensive dataset over varying weld heterogeneity and crack defects, 2) Transfer learning using limited 3D elastodynamic simulations to refine inversion models, 3) Guided diffusion for handling out-of-distribution real-world measurements by producing in-distribution representations of experimental LDV scans, 4) U-Net-based segmentation and inversion models.

Result: The integrated framework provides an end-to-end solution for automated weld inspection on real data, successfully handling the challenges of data curation and signal corruption in realistic industrial settings.

Conclusion: This work demonstrates a complete machine learning pipeline that overcomes key limitations in automated ultrasonic weld inspection by combining physics-based modeling, transfer learning, and diffusion techniques to enable reliable inspection in industrial environments.

Abstract: Automated ultrasonic weld inspection remains a significant challenge in the nondestructive evaluation (NDE) community to factors such as limited training data (due to the complexity of curating experimental specimens or high-fidelity simulations) and environmental volatility of many industrial settings (resulting in the corruption of on-the-fly measurements). Thus, an end-to-end machine learning (ML) workflow for acoustic weld inspection in realistic (i.e., industrial) settings has remained an elusive goal. This work addresses the challenges of data curation and signal corruption by proposing workflow consisting of a reduced-order modeling scheme, diffusion based distribution alignment, and U-Net-based segmentation and inversion. A reduced-order Helmholtz model based on Lamb wave theory is used to generate a comprehensive dataset over varying weld heterogeneity and crack defects. The relatively inexpensive low-order solutions provide a robust training dateset for inversion models which are refined through a transfer learning stage using a limited set of full 3D elastodynamic simulations. To handle out-of-distribution (OOD) real-world measurements with varying and unpredictable noise distributions, i.e., Laser Doppler Vibrometry scans, guided diffusion produces in-distribution representations of OOD experimental LDV scans which are subsequently processed by the inversion models. This integrated framework provides an end-to-end solution for automated weld inspection on real data.

Method: Proposes an information-theoretic Lagrangian formulation that balances simplicity (via latent mutual information) and expressiveness (via von Neumann entropy), with a new algorithm that prevents mode collapse and encourages diversity.

Result: The approach demonstrates improved performance over existing Koopman learning methods across diverse dynamical systems, with visualizations showing consistent empirical results with theoretical predictions.

Conclusion: The information-theoretic framework successfully addresses the tradeoff between simplicity and expressiveness in Koopman learning, producing more stable and interpretable representations.

Abstract: The Koopman operator provides a powerful framework for modeling dynamical systems and has attracted growing interest from the machine learning community. However, its infinite-dimensional nature makes identifying suitable finite-dimensional subspaces challenging, especially for deep architectures. We argue that these difficulties come from suboptimal representation learning, where latent variables fail to balance expressivity and simplicity. This tension is closely related to the information bottleneck (IB) dilemma: constructing compressed representations that are both compact and predictive. Rethinking Koopman learning through this lens, we demonstrate that latent mutual information promotes simplicity, yet an overemphasis on simplicity may cause latent space to collapse onto a few dominant modes. In contrast, expressiveness is sustained by the von Neumann entropy, which prevents such collapse and encourages mode diversity. This insight leads us to propose an information-theoretic Lagrangian formulation that explicitly balances this tradeoff. Furthermore, we propose a new algorithm based on the Lagrangian formulation that encourages both simplicity and expressiveness, leading to a stable and interpretable Koopman representation. Beyond quantitative evaluations, we further visualize the learned manifolds under our representations, observing empirical results consistent with our theoretical predictions. Finally, we validate our approach across a diverse range of dynamical systems, demonstrating improved performance over existing Koopman learning methods. The implementation is publicly available at https://github.com/Wenxuan52/InformationKoopman.

Method: Develop an explainable AI framework using a reconfigured latent data assimilation technique that bridges model hierarchy, exploiting sparse output from idealized models to enhance operational models.

Result: The bridging model inherits high resolution and comprehensive variables from operational models while achieving global accuracy enhancements, demonstrated by significant bias correction in CMIP6 simulations of El Niño spatiotemporal patterns.

Conclusion: The framework provides physically insightful understanding beyond black-box correction, enables effective physics-assisted digital twins and uncertainty quantification, and highlights the importance of advancing idealized model development and interdisciplinary communication.

Abstract: Computer models are indispensable tools for understanding the Earth system. While high-resolution operational models have achieved many successes, they exhibit persistent biases, particularly in simulating extreme events and statistical distributions. In contrast, coarse-grained idealized models isolate fundamental processes and can be precisely calibrated to excel in characterizing specific dynamical and statistical features. However, different models remain siloed by disciplinary boundaries. By leveraging the complementary strengths of models of varying complexity, we develop an explainable AI framework for Earth system emulators. It bridges the model hierarchy through a reconfigured latent data assimilation technique, uniquely suited to exploit the sparse output from the idealized models. The resulting bridging model inherits the high resolution and comprehensive variables of operational models while achieving global accuracy enhancements through targeted improvements from idealized models. Crucially, the mechanism of AI provides a clear rationale for these advancements, moving beyond black-box correction to physically insightful understanding in a computationally efficient framework that enables effective physics-assisted digital twins and uncertainty quantification. We demonstrate its power by significantly correcting biases in CMIP6 simulations of El Ni~no spatiotemporal patterns, leveraging statistically accurate idealized models. This work also highlights the importance of pushing idealized model development and advancing communication between modeling communities.

Method: Developed IAGD using second-order Newton Interpolation to accelerate convergence, and NRSGD with controlled noise injection for regularization. Tested on CIFAR-10/100 datasets with various CNNs against classical Keras optimizers.

Result: Both IAGD and NRSGD demonstrated improved performance over classical optimizers, showing faster convergence and better regularization effectiveness on image classification tasks.

Conclusion: The proposed IAGD and NRSGD optimizers provide viable improvements to SGD, offering effective solutions for convergence acceleration and overfitting prevention in deep learning optimization.

Abstract: Based on Stochastic Gradient Descent (SGD), the paper introduces two optimizers, named Interpolational Accelerating Gradient Descent (IAGD) as well as Noise-Regularized Stochastic Gradient Descent (NRSGD). IAGD leverages second-order Newton Interpolation to expedite the convergence process during training, assuming relevancy in gradients between iterations. To avoid over-fitting, NRSGD incorporates a noise regularization technique that introduces controlled noise to the gradients during the optimization process. Comparative experiments of this research are conducted on the CIFAR-10, and CIFAR-100 datasets, benchmarking different CNNs(Convolutional Neural Networks) with IAGD and NRSGD against classical optimizers in Keras Package. Results demonstrate the potential of those two viable improvement methods in SGD, implicating the effectiveness of the advancements.

Method: Leverages Global Precipitation Mission’s CORRA dataset, geostationary satellite data, and global NWP data to train a machine learning model that predicts precipitation for the next 12 hours at high resolution.

Result: Significantly outperforms industry-standard hourly forecasts, achieves higher skill than best NWP models in data-sparse regions, and shows improvements across key metrics like critical success index and fractions skill score.

Conclusion: Global MetNet reduces global disparities in forecast quality, integrates sparse satellite observations into weather forecasting, and is already deployed for millions of users on Google Search with sub-minute forecast generation.

Abstract: Precipitation nowcasting, which predicts rainfall up to a few hours ahead, is a critical tool for vulnerable communities in the Global South frequently exposed to intense, rapidly developing storms. Timely forecasts provide a crucial window to protect lives and livelihoods. Traditional numerical weather prediction (NWP) methods suffer from high latency, low spatial and temporal resolution, and significant gaps in accuracy across the world. Recent machine learning-based nowcasting methods, common in the Global North, cannot be extended to the Global South due to extremely sparse radar coverage. We present Global MetNet, an operational global machine learning nowcasting model. It leverages the Global Precipitation Mission’s CORRA dataset, geostationary satellite data, and global NWP data to predict precipitation for the next 12 hours. The model operates at a high resolution of approximately 0.05{\deg} (~5km) spatially and 15 minutes temporally. Global MetNet significantly outperforms industry-standard hourly forecasts and achieves significantly higher skill, making forecasts useful over a much larger area of the world than previously available. Our model demonstrates better skill in data-sparse regions than even the best high-resolution NWP models achieve in the US. Validated using ground radar and satellite data, it shows significant improvements across key metrics like the critical success index and fractions skill score for all precipitation rates and lead times. Crucially, our model generates forecasts in under a minute, making it readily deployable for real-time applications. It is already deployed for millions of users on Google Search. This work represents a key step in reducing global disparities in forecast quality and integrating sparse, high-resolution satellite observations into weather forecasting.

Method: Introduces a structured weight-based formulation for time-varying optimization, analyzing two weighting strategies (uniform and discounted) and deriving tracking error bounds under gradient descent updates.

Result: Uniform weighting achieves asymptotic convergence with O(1/t) decay rate, while discounted weighting incurs a nonzero error floor controlled by discount factor and gradient update frequency.

Conclusion: The structured weight-based approach provides theoretical guarantees for time-varying optimization in streaming data contexts, with different trade-offs between uniform and discounted weighting schemes.

Abstract: Classical optimization theory deals with fixed, time-invariant objective functions. However, time-varying optimization has emerged as an important subject for decision-making in dynamic environments. In this work, we study the problem of learning from streaming data through a time-varying optimization lens. Unlike prior works that focus on generic formulations, we introduce a structured, \emph{weight-based} formulation that explicitly captures the streaming-data origin of the time-varying objective, where at each time step, an agent aims to minimize a weighted average loss over all the past data samples. We focus on two specific weighting strategies: (1) uniform weights, which treat all samples equally, and (2) discounted weights, which geometrically decay the influence of older data. For both schemes, we derive tight bounds on the ``tracking error’’ (TE), defined as the deviation between the model parameter and the time-varying optimum at a given time step, under gradient descent (GD) updates. We show that under uniform weighting, the TE vanishes asymptotically with a $\mathcal{O}(1/t)$ decay rate, whereas discounted weighting incurs a nonzero error floor controlled by the discount factor and the number of gradient updates performed at each time step. Our theoretical findings are validated through numerical simulations.

Method: Proposed OMG algorithm for Matrix games using best response sampling with optimistic bonuses, and SOMG algorithm for Markov games using best response sampling with novel superoptimistic bonuses.

Result: Both algorithms achieve logarithmic regret in T that scales inversely with KL regularization strength β, in addition to standard Õ(√T) regret independent of β.

Conclusion: KL regularization provides provable theoretical benefits in game-theoretic settings, enabling improved sample efficiency through specialized algorithms that leverage the regularization structure.

Abstract: Reverse Kullback-Leibler (KL) divergence-based regularization with respect to a fixed reference policy is widely used in modern reinforcement learning to preserve the desired traits of the reference policy and sometimes to promote exploration (using uniform reference policy, known as entropy regularization). Beyond serving as a mere anchor, the reference policy can also be interpreted as encoding prior knowledge about good actions in the environment. In the context of alignment, recent game-theoretic approaches have leveraged KL regularization with pretrained language models as reference policies, achieving notable empirical success in self-play methods. Despite these advances, the theoretical benefits of KL regularization in game-theoretic settings remain poorly understood. In this work, we develop and analyze algorithms that provably achieve improved sample efficiency under KL regularization. We study both two-player zero-sum Matrix games and Markov games: for Matrix games, we propose OMG, an algorithm based on best response sampling with optimistic bonuses, and extend this idea to Markov games through the algorithm SOMG, which also uses best response sampling and a novel concept of superoptimistic bonuses. Both algorithms achieve a logarithmic regret in $T$ that scales inversely with the KL regularization strength $\beta$ in addition to the standard $\widetilde{\mathcal{O}}(\sqrt{T})$ regret independent of $\beta$ which is attained in both regularized and unregularized settings

Method: Defined compactness functions for individual clusters and neighboring point sets for cluster pairs. Used these to calculate cluster compactness, distribution compactness, inter-cluster margins, and overall distribution margins.

Result: The proposed indices successfully identified the true number of clusters in both synthetic and real-world datasets, demonstrating competitive performance compared to widely-used cluster validity indices.

Conclusion: The novel absolute cluster indices provide effective measures for evaluating cluster compactness and separability, offering a valuable alternative to traditional relative validity indices for determining optimal clustering solutions.

Abstract: Finding “true” clusters in a data set is a challenging problem. Clustering solutions obtained using different models and algorithms do not necessarily provide compact and well-separated clusters or the optimal number of clusters. Cluster validity indices are commonly applied to identify such clusters. Nevertheless, these indices are typically relative, and they are used to compare clustering algorithms or choose the parameters of a clustering algorithm. Moreover, the success of these indices depends on the underlying data structure. This paper introduces novel absolute cluster indices to determine both the compactness and separability of clusters. We define a compactness function for each cluster and a set of neighboring points for cluster pairs. This function is utilized to determine the compactness of each cluster and the whole cluster distribution. The set of neighboring points is used to define the margin between clusters and the overall distribution margin. The proposed compactness and separability indices are applied to identify the true number of clusters. Using a number of synthetic and real-world data sets, we demonstrate the performance of these new indices and compare them with other widely-used cluster validity indices.

Method: Developed NeuroRVQ with a codebook-based tokenizer featuring: (i) multi-scale feature extraction for full frequency spectrum capture, (ii) hierarchical residual vector quantization codebooks for high-resolution encoding, and (iii) phase- and amplitude-aware loss function for efficient training.

Result: NeuroRVQ achieves lower reconstruction error and outperforms existing Large Brainwave Models on various downstream tasks, enabling efficient EEG compression with accurate reconstruction across all frequency bands.

Conclusion: NeuroRVQ establishes a strong prior for codebook-based general-purpose brainwave models, advancing neural decoding, generative modeling, and multimodal biosignal integration.

Abstract: Electroencephalography (EEG) captures neural activity across multiple temporal and spectral scales, yielding signals that are rich but complex for representation learning. Recently, EEG foundation models trained to predict masked signal-tokens have shown promise for learning generalizable representations. However, their performance is hindered by their signal tokenization modules. Existing neural tokenizers fail to preserve high-frequency dynamics, limiting their ability to reconstruct EEG signals with high fidelity. We introduce NeuroRVQ, a scalable Large Brainwave Model (LBM) centered on a codebook-based tokenizer. Our tokenizer integrates: (i) multi-scale feature extraction modules that capture the full frequency neural spectrum; (ii) hierarchical residual vector quantization (RVQ) codebooks for high-resolution encoding; and, (iii) an EEG signal phase- and amplitude-aware loss function for efficient training. This design enables efficient EEG compression while supporting accurate reconstruction across all frequency bands, leading to robust generative masked modeling. Our empirical results demonstrate that NeuroRVQ achieves lower reconstruction error and outperforms existing LBMs on a variety of downstream tasks. More broadly, NeuroRVQ tokenizer establishes a strong prior for codebook-based general-purpose brainwave models, enabling advances in neural decoding, generative modeling and multimodal biosignal integration.

Method: Multi-layer Transformer with residual connections for iterative refinement of channel and beamformer features, enhanced by curriculum learning, semi-amortized learning with gradient ascent steps, and sliding-window training of Transformer blocks.

Result: Outperforms existing baselines at low-to-medium SNRs and approaches WMMSE performance at high SNRs, with substantially faster inference than iterative and online learning methods.

Conclusion: The proposed unsupervised deep learning framework provides an effective solution for real-time beamforming with competitive performance and computational efficiency.

Abstract: We develop an unsupervised deep learning framework for downlink beamforming in large-scale MU-MISO channels. The model is trained offline, allowing real-time inference through lightweight feedforward computations in dynamic communication environments. Following the learning-to-optimize (L2O) paradigm, a multi-layer Transformer iteratively refines both channel and beamformer features via residual connections. To enhance training, three strategies are introduced: (i) curriculum learning (CL) to improve early-stage convergence and avoid local optima, (ii) semi-amortized learning to refine each Transformer block with a few gradient ascent steps, and (iii) sliding-window training to stabilize optimization by training only a subset of Transformer blocks at a time. Extensive simulations show that the proposed scheme outperforms existing baselines at low-to-medium SNRs and closely approaches WMMSE performance at high SNRs, while achieving substantially faster inference than iterative and online learning approaches.

Method: Uses Gated Recurrent Units (GRUs) to learn temporal patterns, Directed Acyclic Graph (DAG) learning for causal structures, Hill equation-based saturation curves, with data-driven hyperparameter estimation, multi-region modeling, and robust statistical methods.

Result: The package provides automated learning of marketing transformations, comprehensive response curve analysis, and extensive visualization capabilities with 14+ interactive dashboards for business insights.

Conclusion: DeepCausalMMM offers a modern, data-driven approach to Marketing Mix Modeling that eliminates manual heuristics and provides more accurate modeling of complex marketing dynamics through deep learning and causal inference techniques.

Abstract: Marketing Mix Modeling (MMM) is a statistical technique used to estimate the impact of marketing activities on business outcomes such as sales, revenue, or customer visits. Traditional MMM approaches often rely on linear regression or Bayesian hierarchical models that assume independence between marketing channels and struggle to capture complex temporal dynamics and non-linear saturation effects [@Hanssens2005; @Ng2021Bayesian]. DeepCausalMMM is a Python package that addresses these limitations by combining deep learning, causal inference, and advanced marketing science. The package uses Gated Recurrent Units (GRUs) to automatically learn temporal patterns such as adstock (carryover effects) and lag, while simultaneously learning statistical dependencies and potential causal structures between marketing channels through Directed Acyclic Graph (DAG) learning [@Zheng2018NOTEARS; @Gong2024CausalMMM]. Additionally, it implements Hill equation-based saturation curves to model diminishing returns and optimize budget allocation. Key innovations include: (1) a data-driven design where hyperparameters and transformations (e.g., adstock decay, saturation curves) are learned or estimated from data with sensible defaults, rather than requiring fixed heuristics or manual specification, (2) multi-region modeling with both shared and region-specific parameters, (3) robust statistical methods including Huber loss and advanced regularization, (4) comprehensive response curve analysis for understanding channel saturation, and (5) an extensive visualization suite with 14+ interactive dashboards for business insights.

Method: Sparse triangular transport maps that exploit conditional independence structure using monotone rectified neural networks (MRNN), with a framework balancing exact sparsity (respecting conditional independence) and approximate sparsity (computational tractability).

Result: The method enables site-wise parallel evaluation and achieves linear time complexity in lattice size N while preserving expressive, invertible structure. Tested on φ⁴ theory in 2D with analysis of node orderings’ effects.

Conclusion: The proposed sparse triangular transport maps provide an efficient alternative to traditional methods like Hybrid Monte Carlo and established flow approaches like RealNVP for sampling lattice field theories.

Abstract: Lattice field theories are fundamental testbeds for computational physics; yet, sampling their Boltzmann distributions remains challenging due to multimodality and long-range correlations. While normalizing flows offer a promising alternative, their application to large lattices is often constrained by prohibitive memory requirements and the challenge of maintaining sufficient model expressivity. We propose sparse triangular transport maps that explicitly exploit the conditional independence structure of the lattice graph under periodic boundary conditions using monotone rectified neural networks (MRNN). We introduce a comprehensive framework for triangular transport maps that navigates the fundamental trade-off between \emph{exact sparsity} (respecting marginal conditional independence in the target distribution) and \emph{approximate sparsity} (computational tractability without fill-ins). Restricting each triangular map component to a local past enables site-wise parallel evaluation and linear time complexity in lattice size $N$, while preserving the expressive, invertible structure. Using $\phi^4$ in two dimensions as a controlled setting, we analyze how node labelings (orderings) affect the sparsity and performance of triangular maps. We compare against Hybrid Monte Carlo (HMC) and established flow approaches (RealNVP).

Method: Connecting MDMs to established reasoning frameworks (chain of thought and padded looped transformers) in the finite-precision log-width setting, and proving equivalences between these models.

Result: MDMs are equivalent to polynomially-padded looped transformers and can solve all problems that CoT-augmented transformers can. MDMs are inherently more efficient for certain problem classes including regular languages.

Conclusion: Masked diffusion models offer parallel generation capabilities that enable substantially faster reasoning for certain problem classes while maintaining equivalent computational power to established transformer-based reasoning frameworks.

Abstract: Masked diffusion models (MDMs) for text offer a compelling alternative to traditional autoregressive language models. Parallel generation makes them efficient, but their computational capabilities and the limitations inherent to their parallelism remain largely unexplored. To this end, we characterize what types of reasoning problems MDMs can provably solve and how efficiently. We do this by connecting MDMs to the well-understood reasoning frameworks of chain of thought (CoT) and padded looped transformers (PLTs) in the finite-precision log-width setting: We show that MDMs and polynomially-padded PLTs are, in fact, equivalent in this setting, and that MDMs can solve all problems that CoT-augmented transformers can. Moreover, we showcase classes of problems (including regular languages) for which MDMs are inherently more efficient than CoT transformers, where parallel generation allows for substantially faster reasoning.

Method: Uses Earth Mover’s Distance for client clustering based on local data distributions, derives relationship between intra-cluster EMD and training rounds, and incorporates directed acyclic graph-based task scheduling for dependency management.

Result: Numerical experiments show CoDa-FL outperforms existing benchmarks with faster convergence, lower communication/computational costs, and higher learning accuracy under heterogeneous MEC settings.

Conclusion: The proposed CoDa-FL framework effectively reduces total training time in federated learning by optimizing client selection and task assignment through clustering and dependency-aware scheduling.

Abstract: We study the client selection problem in Federated Learning (FL) within mobile edge computing (MEC) environments, particularly under the dependent multi-task settings, to reduce the total time required to complete various learning tasks. We propose CoDa-FL, a Cluster-oriented and Dependency-aware framework designed to reduce the total required time via cluster-based client selection and dependent task assignment. Our approach considers Earth Mover’s Distance (EMD) for client clustering based on their local data distributions to lower computational cost and improve communication efficiency. We derive a direct and explicit relationship between intra-cluster EMD and the number of training rounds required for convergence, thereby simplifying the otherwise complex process of obtaining the optimal solution. Additionally, we incorporate a directed acyclic graph-based task scheduling mechanism to effectively manage task dependencies. Through numerical experiments, we validate that our proposed CoDa-FL outperforms existing benchmarks by achieving faster convergence, lower communication and computational costs, and higher learning accuracy under heterogeneous MEC settings.

Method: Construct unbiased estimators using random-batch methods that sample neuron batches over time intervals, analyze convergence rates and stability, and derive optimal sampling strategies including cost-accuracy trade-offs.

Result: Established linear convergence rate for expected uniform error, total-variation error of order h^{1/2} for distribution stability, and validated theory on neural ODEs showing predicted rates and favorable computational profiles.

Conclusion: Random-batch methods provide a rigorous framework for dropout regularization in continuous-time models with proven convergence guarantees and practical computational advantages.

Abstract: We study dropout regularization in continuous-time models through the lens of random-batch methods – a family of stochastic sampling schemes originally devised to reduce the computational cost of interacting particle systems. We construct an unbiased, well-posed estimator that mimics dropout by sampling neuron batches over time intervals of length $h$. Trajectory-wise convergence is established with linear rate in $h$ for the expected uniform error. At the distribution level, we establish stability for the associated continuity equation, with total-variation error of order $h^{1/2}$ under mild moment assumptions. During training with fixed batch sampling across epochs, a Pontryagin-based adjoint analysis bounds deviations in the optimal cost and control, as well as in gradient-descent iterates. On the design side, we compare convergence rates for canonical batch sampling schemes, recover standard Bernoulli dropout as a special case, and derive a cost–accuracy trade-off yielding a closed-form optimal $h$. We then specialize to a single-layer neural ODE and validate the theory on classification and flow matching, observing the predicted rates, regularization effects, and favorable runtime and memory profiles.

Method: Proposed Program Behavior Spectrum representation by probing program IR with diverse optimization sequences, using Product Quantization to discretize reaction vectors, and training PQ-BERT Transformer model to learn behavioral codes.

Result: Outperformed state-of-the-art static baselines on Best Pass Prediction and -Oz Benefit Prediction tasks, demonstrating superior performance in compiler optimization.

Conclusion: The quasi-dynamic framework successfully bridges the gap between static and dynamic representations, providing efficient yet insightful program representations for compiler optimization tasks.

Abstract: Learning effective numerical representations, or embeddings, of programs is a fundamental prerequisite for applying machine learning to automate and enhance compiler optimization. Prevailing paradigms, however, present a dilemma. Static representations, derived from source code or intermediate representation (IR), are efficient and deterministic but offer limited insight into how a program will behave or evolve under complex code transformations. Conversely, dynamic representations, which rely on runtime profiling, provide profound insights into performance bottlenecks but are often impractical for large-scale tasks due to prohibitive overhead and inherent non-determinism. This paper transcends this trade-off by proposing a novel quasi-dynamic framework for program representation. The core insight is to model a program’s optimization sensitivity. We introduce the Program Behavior Spectrum, a new representation generated by probing a program’s IR with a diverse set of optimization sequences and quantifying the resulting changes in its static features. To effectively encode this high-dimensional, continuous spectrum, we pioneer a compositional learning approach. Product Quantization is employed to discretize the continuous reaction vectors into structured, compositional sub-words. Subsequently, a multi-task Transformer model, termed PQ-BERT, is pre-trained to learn the deep contextual grammar of these behavioral codes. Comprehensive experiments on two representative compiler optimization tasks – Best Pass Prediction and -Oz Benefit Prediction – demonstrate that our method outperforms state-of-the-art static baselines. Our code is publicly available at https://github.com/Panhaolin2001/PREP/.

Method: Proves completeness can be achieved via complete scalar functions (canonical forms) and full-rank steerable basis sets, then implements efficient algorithms for EGNN and TFN models.

Result: Empirical results show superior completeness and excellent performance with few layers, significantly reducing computational overhead while maintaining strong efficacy.

Conclusion: The proposed framework provides a theoretically grounded, efficient, and practical approach to constructing complete equivariant GNNs.

Abstract: Equivariant Graph Neural Networks (GNNs) have demonstrated significant success across various applications. To achieve completeness – that is, the universal approximation property over the space of equivariant functions – the network must effectively capture the intricate multi-body interactions among different nodes. Prior methods attain this via deeper architectures, augmented body orders, or increased degrees of steerable features, often at high computational cost and without polynomial-time solutions. In this work, we present a theoretically grounded framework for constructing complete equivariant GNNs that is both efficient and practical. We prove that a complete equivariant GNN can be achieved through two key components: 1) a complete scalar function, referred to as the canonical form of the geometric graph; and 2) a full-rank steerable basis set. Leveraging this finding, we propose an efficient algorithm for constructing complete equivariant GNNs based on two common models: EGNN and TFN. Empirical results demonstrate that our model demonstrates superior completeness and excellent performance with only a few layers, thereby significantly reducing computational overhead while maintaining strong practical efficacy.

Method: Proposed Cross-modal Complementarity Hypothesis (CCH), theoretically validated it in a joint Gaussian model, and empirically tested across multimodal datasets including image, text, video, audio, and cancer omics data.

Result: The CCH was both theoretically and empirically validated, showing cross-modal KD works when mutual information between teacher and student representations exceeds that between student and labels.

Conclusion: Established a theoretical framework for cross-modal KD and provided practical guidelines for selecting optimal teacher modalities based on the CCH criterion.

Abstract: The rapid increase in multimodal data availability has sparked significant interest in cross-modal knowledge distillation (KD) techniques, where richer “teacher” modalities transfer information to weaker “student” modalities during model training to improve performance. However, despite successes across various applications, cross-modal KD does not always result in improved outcomes, primarily due to a limited theoretical understanding that could inform practice. To address this gap, we introduce the Cross-modal Complementarity Hypothesis (CCH): we propose that cross-modal KD is effective when the mutual information between teacher and student representations exceeds the mutual information between the student representation and the labels. We theoretically validate the CCH in a joint Gaussian model and further confirm it empirically across diverse multimodal datasets, including image, text, video, audio, and cancer-related omics data. Our study establishes a novel theoretical framework for understanding cross-modal KD and offers practical guidelines based on the CCH criterion to select optimal teacher modalities for improving the performance of weaker modalities.

Method: Integrates structured domain expertise into a neural architecture that aligns rules and data samples in a shared embedding space. Trains encoders jointly on synthetic compliance and violation data to learn soft rule embeddings that generalize to real-world datasets.

Result: Outperforms four state-of-the-art anomaly detection baselines with average improvements of 1.3% in AUC and 3.4% in recall on large-scale real-world datasets. Ablation study confirms the complementary role of expert rules.

Conclusion: Embedding expert rules into the learning process improves detection accuracy and increases transparency, offering an interpretable approach for high-stakes domains like healthcare fraud detection.

Abstract: Healthcare fraud detection remains a critical challenge due to limited availability of labeled data, constantly evolving fraud tactics, and the high dimensionality of medical records. Traditional supervised methods are challenged by extreme label scarcity, while purely unsupervised approaches often fail to capture clinically meaningful anomalies. In this work, we introduce CleverCatch, a knowledge-guided weak supervision model designed to detect fraudulent prescription behaviors with improved accuracy and interpretability. Our approach integrates structured domain expertise into a neural architecture that aligns rules and data samples within a shared embedding space. By training encoders jointly on synthetic data representing both compliance and violation, CleverCatch learns soft rule embeddings that generalize to complex, real-world datasets. This hybrid design enables data-driven learning to be enhanced by domain-informed constraints, bridging the gap between expert heuristics and machine learning. Experiments on the large-scale real-world dataset demonstrate that CleverCatch outperforms four state-of-the-art anomaly detection baselines, yielding average improvements of 1.3% in AUC and 3.4% in recall. Our ablation study further highlights the complementary role of expert rules, confirming the adaptability of the framework. The results suggest that embedding expert rules into the learning process not only improves detection accuracy but also increases transparency, offering an interpretable approach for high-stakes domains such as healthcare fraud detection.

Method: Controlled comparison protocol fixing model, sampler, and step size; analysis using Fisher information matrix (FIM) as covariance of sufficient statistics; visualization of empirical moments; comparison of SGD and natural gradient descent (NGD) performance.

Result: QUBO encoding induces larger cross terms between first- and second-order statistics, creating more small-eigenvalue directions in FIM and lowering spectral entropy, leading to slower SGD convergence. Ising encoding provides more isotropic curvature and faster SGD convergence. NGD achieves similar convergence across encodings.

Conclusion: For SGD-based training, Ising encoding is preferable; for QUBO, centering/scaling or NGD-style preconditioning can mitigate curvature issues. The study provides actionable guidelines for variable encoding and preprocessing in Boltzmann machine learning.

Abstract: We compare Ising ({-1,+1}) and QUBO ({0,1}) encodings for Boltzmann machine learning under a controlled protocol that fixes the model, sampler, and step size. Exploiting the identity that the Fisher information matrix (FIM) equals the covariance of sufficient statistics, we visualize empirical moments from model samples and reveal systematic, representation-dependent differences. QUBO induces larger cross terms between first- and second-order statistics, creating more small-eigenvalue directions in the FIM and lowering spectral entropy. This ill-conditioning explains slower convergence under stochastic gradient descent (SGD). In contrast, natural gradient descent (NGD)-which rescales updates by the FIM metric-achieves similar convergence across encodings due to reparameterization invariance. Practically, for SGD-based training, the Ising encoding provides more isotropic curvature and faster convergence; for QUBO, centering/scaling or NGD-style preconditioning mitigates curvature pathologies. These results clarify how representation shapes information geometry and finite-time learning dynamics in Boltzmann machines and yield actionable guidelines for variable encoding and preprocessing.

Method: Proposed truncated influence function (TIF) to measure individual data influence, developed simpler scoring functions correlated with TIF, and combined them to create an effective data selection rule that adapts to specific models.

Result: Experiments across diverse alignment benchmarks and LLM families show that better alignment performance can be achieved using less data, demonstrating the effectiveness of the proposed methods.

Conclusion: Preference data quality is model-dependent, and the proposed model-adaptive data selection approach enables more precise selection of valuable preference data for improved LLM alignment efficiency.

Abstract: Large language model (LLM) alignment is typically achieved through learning from human preference comparisons, making the quality of preference data critical to its success. Existing studies often pre-process raw training datasets to identify valuable preference pairs using external reward models or off-the-shelf LLMs, achieving improved overall performance but rarely examining whether individual, selected data point is genuinely beneficial. We assess data quality through individual influence on validation data using our newly proposed truncated influence function (TIF), which mitigates the over-scoring present in traditional measures and reveals that preference data quality is inherently a property of the model. In other words, a data pair that benefits one model may harm another. This leaves the need to improve the preference data selection approaches to be adapting to specific models. To this end, we introduce two candidate scoring functions (SFs) that are computationally simpler than TIF and positively correlated with it. They are also model dependent and can serve as potential indicators of individual data quality for preference data selection. Furthermore, we observe that these SFs inherently exhibit errors when compared to TIF. To this end, we combine them to offset their diverse error sources, resulting in a simple yet effective data selection rule that enables the models to achieve a more precise selection of valuable preference data. We conduct experiments across diverse alignment benchmarks and various LLM families, with results demonstrating that better alignment performance can be achieved using less data, showing the generality of our findings and new methods.

Method: Proposes FracNet with two modules: decomposes graphs into high-frequency (domain-specific local details) and low-frequency (domain-invariant global patterns) components, then performs frequency-aware domain adaptation integrated with contrastive learning.

Result: Extensive experiments demonstrate significant improvements over state-of-the-art approaches in domain adaptation tasks.

Conclusion: FracNet effectively addresses domain shifts through spectral analysis and contrastive learning, with both practical success and theoretical justification.

Abstract: Graph neural networks (GNNs) have achieved remarkable success in various domains, yet they often struggle with domain adaptation due to significant structural distribution shifts and insufficient exploration of transferable patterns. One of the main reasons behind this is that traditional approaches do not treat global and local patterns discriminatingly so that some local details in the graph may be violated after multi-layer GNN. Our key insight is that domain shifts can be better understood through spectral analysis, where low-frequency components often encode domain-invariant global patterns, and high-frequency components capture domain-specific local details. As such, we propose FracNet (\underline{\textbf{Fr}}equency \underline{\textbf{A}}ware \underline{\textbf{C}}ontrastive Graph \underline{\textbf{Net}}work) with two synergic modules to decompose the original graph into high-frequency and low-frequency components and perform frequency-aware domain adaption. Moreover, the blurring boundary problem of domain adaptation is improved by integrating with a contrastive learning framework. Besides the practical implication, we also provide rigorous theoretical proof to demonstrate the superiority of FracNet. Extensive experiments further demonstrate significant improvements over state-of-the-art approaches.

Method: Apply hypernetwork+adapters combination to perspectivist classification, creating an architecture-agnostic solution that works with various base models.

Result: The solution competes with specialized models in adopting user perspectives for hate speech and toxicity detection while using considerably fewer parameters.

Conclusion: The proposed approach provides an efficient and flexible solution for perspective-aware classification that can be applied to various base models without architectural constraints.

Abstract: The task of perspective-aware classification introduces a bottleneck in terms of parametric efficiency that did not get enough recognition in existing studies. In this article, we aim to address this issue by applying an existing architecture, the hypernetwork+adapters combination, to perspectivist classification. Ultimately, we arrive at a solution that can compete with specialized models in adopting user perspectives on hate speech and toxicity detection, while also making use of considerably fewer parameters. Our solution is architecture-agnostic and can be applied to a wide range of base models out of the box.

Method: Proposes BlendFL framework that blends horizontal and vertical FL principles in synchronized fashion, with decentralized inference mechanism and BlendAvg adaptive global model aggregation strategy.

Result: Superior performance for both multimodal and unimodal classification on large-scale medical dataset and benchmark, with faster convergence than traditional approaches.

Conclusion: BlendFL shows strong potential for handling multimodal data heterogeneity in privacy-sensitive real-world settings like healthcare and finance.

Abstract: One of the key challenges of collaborative machine learning, without data sharing, is multimodal data heterogeneity in real-world settings. While Federated Learning (FL) enables model training across multiple clients, existing frameworks, such as horizontal and vertical FL, are only effective in `ideal’ settings that meet specific assumptions. Hence, they struggle to address scenarios where neither all modalities nor all samples are represented across the participating clients. To address this gap, we propose BlendFL, a novel FL framework that seamlessly blends the principles of horizontal and vertical FL in a synchronized and non-restrictive fashion despite the asymmetry across clients. Specifically, any client within BlendFL can benefit from either of the approaches, or both simultaneously, according to its available dataset. In addition, BlendFL features a decentralized inference mechanism, empowering clients to run collaboratively trained local models using available local data, thereby reducing latency and reliance on central servers for inference. We also introduce BlendAvg, an adaptive global model aggregation strategy that prioritizes collaborative model updates based on each client’s performance. We trained and evaluated BlendFL and other state-of-the-art baselines on three classification tasks using a large-scale real-world multimodal medical dataset and a popular multimodal benchmark. Our results highlight BlendFL’s superior performance for both multimodal and unimodal classification. Ablation studies demonstrate BlendFL’s faster convergence compared to traditional approaches, accelerating collaborative learning. Overall, in our study we highlight the potential of BlendFL for handling multimodal data heterogeneity for collaborative learning in real-world settings where data privacy is crucial, such as in healthcare and finance.

Method: MERA optimizes intervention direction and calibrates when and how much to steer, allowing for selective abstention when no confident correction is possible.

Result: Experiments across diverse datasets and LM families show MERA provides safe, effective, non-degrading error correction and outperforms existing baselines. It can also enhance existing steering techniques.

Conclusion: MERA establishes itself as a general-purpose, efficient approach to mechanistic activation steering that provably improves performance through calibrated interventions.

Abstract: We introduce Mechanistic Error Reduction with Abstention (MERA), a principled framework for steering language models (LMs) to mitigate errors through selective, adaptive interventions. Unlike existing methods that rely on fixed, manually tuned steering strengths, often resulting in under or oversteering, MERA addresses these limitations by (i) optimising the intervention direction, and (ii) calibrating when, and how much to steer, thereby provably improving performance or abstaining when no confident correction is possible. Experiments across diverse datasets, and LM families demonstrate safe, effective, non-degrading error correction, and that MERA outperforms existing baselines. Moreover, MERA can be applied on top of existing steering techniques to further enhance their performance, establishing it as a general-purpose, and efficient approach to mechanistic activation steering.

Method: Uses generative models (normalizing flows or diffusion models) to approximate conditional data distributions without sharing raw data. Each client locally calibrates conformal scores reflecting its unique uncertainty, with federated aggregation for global consistency.

Result: Experiments on real datasets show Fed-CCP achieves more adaptive prediction sets compared to existing methods.

Conclusion: Fed-CCP successfully addresses the limitations of standard federated CP by providing conditional coverage that adapts to local data heterogeneity through generative modeling, while maintaining privacy through federated aggregation.

Abstract: Conformal Prediction (CP) provides distribution-free uncertainty quantification by constructing prediction sets that guarantee coverage of the true labels. This reliability makes CP valuable for high-stakes federated learning scenarios such as multi-center healthcare. However, standard CP assumes i.i.d. data, which is violated in federated settings where client distributions differ substantially. Existing federated CP methods address this by maintaining marginal coverage on each client, but such guarantees often fail to reflect input-conditional uncertainty. In this work, we propose Federated Conditional Conformal Prediction (Fed-CCP) via generative models, which aims for conditional coverage that adapts to local data heterogeneity. Fed-CCP leverages generative models, such as normalizing flows or diffusion models, to approximate conditional data distributions without requiring the sharing of raw data. This enables each client to locally calibrate conformal scores that reflect its unique uncertainty, while preserving global consistency through federated aggregation. Experiments on real datasets demonstrate that Fed-CCP achieves more adaptive prediction sets.

Method: Augment diffusion model downscaling pipeline with conformal prediction framework, post-processing samples to derive conditional quantile estimates and implementing conformalized quantile regression for locally adaptive prediction intervals.

Result: Evaluation on ERA5 reanalysis data over Italy (downscaled to 2-km grid) shows markedly improved coverage and stable probabilistic scores compared to baseline diffusion model, with grid-point-level uncertainty estimates.

Conclusion: Conformalized generative models show potential for more trustworthy probabilistic downscaling to high-resolution meteorological fields by providing reliable uncertainty quantification with finite-sample guarantees.

Abstract: Dynamical downscaling is crucial for deriving high-resolution meteorological fields from coarse-scale simulations, enabling detailed analysis for critical applications such as weather forecasting and renewable energy modeling. Generative Diffusion models (DMs) have recently emerged as powerful data-driven tools for this task, offering reconstruction fidelity and more scalable sampling supporting uncertainty quantification. However, DMs lack finite-sample guarantees against overconfident predictions, resulting in miscalibrated grid-point-level uncertainty estimates hindering their reliability in operational contexts. In this work, we tackle this issue by augmenting the downscaling pipeline with a conformal prediction framework. Specifically, the DM’s samples are post-processed to derive conditional quantile estimates, incorporated into a conformalized quantile regression procedure targeting locally adaptive prediction intervals with finite-sample marginal validity. The proposed approach is evaluated on ERA5 reanalysis data over Italy, downscaled to a 2-km grid. Results demonstrate grid-point-level uncertainty estimates with markedly improved coverage and stable probabilistic scores relative to the DM baseline, highlighting the potential of conformalized generative models for more trustworthy probabilistic downscaling to high-resolution meteorological fields.

Method: ISER extends isolation-based methods using hypersphere radii as density proxies, constructs ensemble representations, and introduces similarity-based scoring comparing against theoretical anomaly reference patterns.

Result: Comprehensive experiments on 22 real-world datasets demonstrate ISER’s superior performance over 11 baseline methods.

Conclusion: ISER effectively addresses fundamental challenges in anomaly detection while maintaining computational efficiency and improving detection performance.

Abstract: Anomaly detection is a critical task in data mining and management with applications spanning fraud detection, network security, and log monitoring. Despite extensive research, existing unsupervised anomaly detection methods still face fundamental challenges including conflicting distributional assumptions, computational inefficiency, and difficulty handling different anomaly types. To address these problems, we propose ISER (Isolation-based Spherical Ensemble Representations) that extends existing isolation-based methods by using hypersphere radii as proxies for local density characteristics while maintaining linear time and constant space complexity. ISER constructs ensemble representations where hypersphere radii encode density information: smaller radii indicate dense regions while larger radii correspond to sparse areas. We introduce a novel similarity-based scoring method that measures pattern consistency by comparing ensemble representations against a theoretical anomaly reference pattern. Additionally, we enhance the performance of Isolation Forest by using ISER and adapting the scoring function to address axis-parallel bias and local anomaly detection limitations. Comprehensive experiments on 22 real-world datasets demonstrate ISER’s superior performance over 11 baseline methods.

Method: Thompson Sampling via Fine-Tuning (ToSFiT) parameterizes the probability that a candidate yields maximum reward, leveraging prompt-conditioned LLMs and incrementally adapting them toward the posterior.

Result: Theoretical regret bounds match standard Thompson sampling guarantees. Empirical validation on FAQ response refinement, protein search, and quantum circuit design shows improved sample efficiency with negligible computational impact.

Conclusion: Online fine-tuning significantly improves sample efficiency for Bayesian optimization in large discrete spaces while maintaining computational efficiency.

Abstract: Bayesian optimization in large unstructured discrete spaces is often hindered by the computational cost of maximizing acquisition functions due to the absence of gradients. We propose a scalable alternative based on Thompson sampling that eliminates the need for acquisition function maximization by directly parameterizing the probability that a candidate yields the maximum reward. Our approach, Thompson Sampling via Fine-Tuning (ToSFiT) leverages the prior knowledge embedded in prompt-conditioned large language models, and incrementally adapts them toward the posterior. Theoretically, we derive a novel regret bound for a variational formulation of Thompson Sampling that matches the strong guarantees of its standard counterpart. Our analysis reveals the critical role of careful adaptation to the posterior probability of maximality–a principle that underpins our ToSFiT algorithm. Empirically, we validate our method on three diverse tasks: FAQ response refinement, thermally stable protein search, and quantum circuit design. We demonstrate that online fine-tuning significantly improves sample efficiency, with negligible impact on computational efficiency.

Method: Distributed learning method that integrates ML and wireless communication, leveraging multiple devices for distributed training of specialized compute-efficient classifiers with minimal communication overhead, combined with tailored wireless multi-hop communication protocols.

Result: Hardware experiments on 20 ultra-low-power devices show RockNet learns timeseries classification from scratch, surpassing latest neural network microcontroller training accuracy by up to 2x, and reduces memory, latency and energy consumption per device by up to 90% when scaling from 1 to 20 devices.

Conclusion: Tight integration of distributed ML, distributed computing, and communication enables, for the first time, training on ultra-low-power hardware with state-of-the-art accuracy.

Abstract: As Machine Learning (ML) becomes integral to Cyber-Physical Systems (CPS), there is growing interest in shifting training from traditional cloud-based to on-device processing (TinyML), for example, due to privacy and latency concerns. However, CPS often comprise ultra-low-power microcontrollers, whose limited compute resources make training challenging. This paper presents RockNet, a new TinyML method tailored for ultra-low-power hardware that achieves state-of-the-art accuracy in timeseries classification, such as fault or malware detection, without requiring offline pretraining. By leveraging that CPS consist of multiple devices, we design a distributed learning method that integrates ML and wireless communication. RockNet leverages all devices for distributed training of specialized compute efficient classifiers that need minimal communication overhead for parallelization. Combined with tailored and efficient wireless multi-hop communication protocols, our approach overcomes the communication bottleneck that often occurs in distributed learning. Hardware experiments on a testbed with 20 ultra-low-power devices demonstrate RockNet’s effectiveness. It successfully learns timeseries classification tasks from scratch, surpassing the accuracy of the latest approach for neural network microcontroller training by up to 2x. RockNet’s distributed ML architecture reduces memory, latency and energy consumption per device by up to 90 % when scaling from one central device to 20 devices. Our results show that a tight integration of distributed ML, distributed computing, and communication enables, for the first time, training on ultra-low-power hardware with state-of-the-art accuracy.

Method: Model the interaction as a Stackelberg game where a principal (classifier) announces its decision policy first, then strategic agents manipulate their features to receive desired outcomes. Focus on binary classifiers with feature manipulation.

Result: Optimal abstention ensures principal’s utility is no worse than non-abstention settings, even with strategic agents. Abstention improves accuracy and serves as a manipulation deterrent, making it costlier for less qualified agents to manipulate.

Conclusion: Abstention is a valuable tool for reducing negative effects of strategic behavior in algorithmic decision making systems, providing both accuracy improvements and manipulation deterrence.

Abstract: Algorithmic decision making is increasingly prevalent, but often vulnerable to strategic manipulation by agents seeking a favorable outcome. Prior research has shown that classifier abstention (allowing a classifier to decline making a decision due to insufficient confidence) can significantly increase classifier accuracy. This paper studies abstention within a strategic classification context, exploring how its introduction impacts strategic agents’ responses and how principals should optimally leverage it. We model this interaction as a Stackelberg game where a principal, acting as the classifier, first announces its decision policy, and then strategic agents, acting as followers, manipulate their features to receive a desired outcome. Here, we focus on binary classifiers where agents manipulate observable features rather than their true features, and show that optimal abstention ensures that the principal’s utility (or loss) is no worse than in a non-abstention setting, even in the presence of strategic agents. We also show that beyond improving accuracy, abstention can also serve as a deterrent to manipulation, making it costlier for agents, especially those less qualified, to manipulate to achieve a positive outcome when manipulation costs are significant enough to affect agent behavior. These results highlight abstention as a valuable tool for reducing the negative effects of strategic behavior in algorithmic decision making systems.

Method: Data-dependent binning with proximity assignment projects data into high-dimensional sparse representation. Cascading fallback strategy estimates missing feature distributions without explicit imputation in original space.

Result: Superior clustering performance on 12 real-world incomplete datasets compared to existing methods, while maintaining linear time complexity.

Conclusion: Proximity kernel effectively handles incomplete data similarity measurement without information loss or bias from traditional imputation approaches.

Abstract: Measuring similarity between incomplete data is a fundamental challenge in web mining, recommendation systems, and user behavior analysis. Traditional approaches either discard incomplete data or perform imputation as a preprocessing step, leading to information loss and biased similarity estimates. This paper presents the proximity kernel, a new similarity measure that directly computes similarity between incomplete data in kernel feature space without explicit imputation in the original space. The proposed method introduces data-dependent binning combined with proximity assignment to project data into a high-dimensional sparse representation that adapts to local density variations. For missing value handling, we propose a cascading fallback strategy to estimate missing feature distributions. We conduct clustering tasks on the proposed kernel representation across 12 real world incomplete datasets, demonstrating superior performance compared to existing methods while maintaining linear time complexity. All the code are available at https://anonymous.4open.science/r/proximity-kernel-2289.

Method: Partition generalization goal into multiple sub-tasks assigned to dedicated base learners, then interpolate their parameters to form a global learner while periodically redistributing global parameters back to base learners to prevent optimization drift.

Result: Generalist achieves lower generalization error and significantly alleviates trade-off problems compared to baseline methods, providing better natural accuracy while maintaining robustness.

Conclusion: Generalist framework represents a promising step toward developing fully robust classifiers by effectively addressing the limitations of traditional adversarial training approaches.

Abstract: Despite the rapid progress of neural networks, they remain highly vulnerable to adversarial examples, for which adversarial training (AT) is currently the most effective defense. While AT has been extensively studied, its practical applications expose two major limitations: natural accuracy tends to degrade significantly compared with standard training, and robustness does not transfer well across attacks crafted under different norm constraints. Unlike prior works that attempt to address only one issue within a single network, we propose to partition the overall generalization goal into multiple sub-tasks, each assigned to a dedicated base learner. By specializing in its designated objective, each base learner quickly becomes an expert in its field. In the later stages of training, we interpolate their parameters to form a knowledgeable global learner, while periodically redistributing the global parameters back to the base learners to prevent their optimization trajectories from drifting too far from the shared target. We term this framework Generalist and introduce three variants tailored to different application scenarios. Both theoretical analysis and extensive experiments demonstrate that Generalist achieves lower generalization error and significantly alleviates the trade-off problems compared with baseline methods. Our results suggest that Generalist provides a promising step toward developing fully robust classifiers in the future.

Method: Investigated key design questions: input conditioning, component sharing between actor and critic, and sequential data slicing for training. Developed stable architectural and training strategies.

Result: Achieved competitive performance across fully and partially observable tasks, in both vector- and image-based settings.

Conclusion: Provides practical guidance for applying transformers in online RL, demonstrating they can be strong baselines for continuous control.

Abstract: Despite their effectiveness and popularity in offline or model-based reinforcement learning (RL), transformers remain underexplored in online model-free RL due to their sensitivity to training setups and model design decisions such as how to structure the policy and value networks, share components, or handle temporal information. In this paper, we show that transformers can be strong baselines for continuous control in online model-free RL. We investigate key design questions: how to condition inputs, share components between actor and critic, and slice sequential data for training. Our experiments reveal stable architectural and training strategies enabling competitive performance across fully and partially observable tasks, and in both vector- and image-based settings. These findings offer practical guidance for applying transformers in online RL.

Method: Abstracts service interactions into dependency graphs, extracts temporal/structural features via embedding functions, uses graph convolution for neighborhood aggregation, and employs contrastive learning with positive/negative sample pairs. Includes temporal consistency constraint for representation stability.

Result: Significantly outperforms existing methods on Precision, Recall, F1-Score, and AUC metrics. Maintains robustness under sparse labeling, monitoring noise, and traffic fluctuations.

Conclusion: Verifies effectiveness of integrating dependency modeling with contrastive learning, provides complete technical solution for cloud service anomaly detection with strong adaptability and stability in complex environments.

Abstract: This paper addresses the challenges of complex dependencies and diverse anomaly patterns in cloud service environments by proposing a dependency modeling and anomaly detection method that integrates contrastive learning. The method abstracts service interactions into a dependency graph, extracts temporal and structural features through embedding functions, and employs a graph convolution mechanism to aggregate neighborhood information for context-aware service representations. A contrastive learning framework is then introduced, constructing positive and negative sample pairs to enhance the separability of normal and abnormal patterns in the representation space. Furthermore, a temporal consistency constraint is designed to maintain representation stability across time steps and reduce the impact of short-term fluctuations and noise. The overall optimization combines contrastive loss and temporal consistency loss to ensure stable and reliable detection across multi-dimensional features. Experiments on public datasets systematically evaluate the method from hyperparameter, environmental, and data sensitivity perspectives. Results show that the proposed approach significantly outperforms existing methods on key metrics such as Precision, Recall, F1-Score, and AUC, while maintaining robustness under conditions of sparse labeling, monitoring noise, and traffic fluctuations. This study verifies the effectiveness of integrating dependency modeling with contrastive learning, provides a complete technical solution for cloud service anomaly detection, and demonstrates strong adaptability and stability in complex environments.

Method: Uses robust regression models to learn optimal forecast combinations while handling missing submissions, and introduces a pay-off allocation mechanism considering both in-sample and out-of-sample performance.

Result: The proposed market design effectively handles missing submissions and adapts to time-varying conditions, demonstrating effectiveness through simulated and real-world case studies.

Conclusion: The prediction market framework provides a viable alternative to cooperative game-theoretical approaches, enabling forecast collaboration while respecting data ownership and competitive constraints.

Abstract: Although both data availability and the demand for accurate forecasts are increasing, collaboration between stakeholders is often constrained by data ownership and competitive interests. In contrast to recent proposals within cooperative game-theoretical frameworks, we place ourselves in a more general framework, based on prediction markets. There, independent agents trade forecasts of uncertain future events in exchange for rewards. We introduce and analyse a prediction market that (i) accounts for the historical performance of the agents, (ii) adapts to time-varying conditions, while (iii) permitting agents to enter and exit the market at will. The proposed design employs robust regression models to learn the optimal forecasts’ combination whilst handling missing submissions. Moreover, we introduce a pay-off allocation mechanism that considers both in-sample and out-of-sample performance while satisfying several desirable economic properties. Case-studies using simulated and real-world data allow demonstrating the effectiveness and adaptability of the proposed market design.

Method: Proposes an unsupervised low-rank method that uses physical constraints (parking spots are parallel to road sides) to rectify GPS errors and align points to actual parking spots in a unified framework.

Result: Extensive experiments show the method effectively corrects GPS point errors and demonstrates superiority in solving this practical problem.

Conclusion: The proposed method provides a simple yet effective solution for GPS point rectification and alignment, with publicly available dataset and code for practical applications.

Abstract: Parking spots are essential components, providing vital mobile resources for residents in a city. Accurate Global Positioning System (GPS) points of parking spots are the core data for subsequent applications,e.g., parking management, parking policy, and urban development. However, high-rise buildings tend to cause GPS points to drift from the actual locations of parking spots; besides, the standard lower-cost GPS equipment itself has a certain location error. Therefore, it is a non-trivial task to correct a few wrong GPS points from a large number of parking spots in an unsupervised approach. In this paper, motivated by the physical constraints of parking spots (i.e., parking spots are parallel to the sides of roads), we propose an unsupervised low-rank method to effectively rectify errors in GPS points and further align them to the parking spots in a unified framework. The proposed unconventional rectification and alignment method is simple and yet effective for any type of GPS point errors. Extensive experiments demonstrate the superiority of the proposed method to solve a practical problem. The data set and the code are publicly accessible at:https://github.com/pangjunbiao/ITS-Parking-spots-Dataset.

Method: Framed the problem as a graph-level prediction task using three GNN architectures (GAT, GINE, EdgeConv) trained on a large-scale synthetic dataset of 200,000 graphs with varying sizes, agent counts, and edge probabilities.

Result: Trained GNN models achieved high-fidelity Banzhaf value approximation with order-of-magnitude speedups compared to exact and sampling-based methods, and demonstrated strong zero-shot generalization to networks with different structural properties.

Conclusion: GNNs establish as a practical tool for scalable cooperative game-theoretic analysis of complex networked systems, enabling efficient computation of Banzhaf values without requiring retraining for new network configurations.

Abstract: Computing the Banzhaf value in network flow games is fundamental for quantifying agent influence in multi-agent systems, with applications ranging from cybersecurity to infrastructure planning. However, exact computation is intractable for systems with more than $\sim20$ agents due to exponential complexity $\mathcal{O}(2^m)$. While Monte Carlo sampling methods provide statistical estimates, they suffer from high sample complexity and cannot transfer knowledge across different network configurations, making them impractical for large-scale or dynamic systems. We present a novel learning-based approach using Graph Neural Networks (GNNs) to approximate Banzhaf values in cardinal network flow games. By framing the problem as a graph-level prediction task, our method learns generalisable patterns of agent influence directly from network topology and control structure. We conduct a comprehensive empirical study comparing three state-of-the-art GNN architectures-Graph Attention Networks (GAT), Graph Isomorphism Networks with Edge features (GINE), and EdgeConv-on a large-scale synthetic dataset of 200,000 graphs per configuration, varying in size (20-100 nodes), agent count (5-20), and edge probability (0.5-1.0). Our results demonstrate that trained GNN models achieve high-fidelity Banzhaf value approximation with order-of-magnitude speedups compared to exact and sampling-based methods. Most significantly, we show strong zero-shot generalisation: models trained on graphs of a specific size and topology accurately predict Banzhaf values for entirely new networks with different structural properties, without requiring retraining. This work establishes GNNs as a practical tool for scalable cooperative game-theoretic analysis of complex networked systems.

Method: Train a Transformer model to predict near-minimal monomial bases for SOS polynomials, using a dataset of 100+ million SOS polynomials, with a theoretical fallback mechanism for correctness.

Result: Achieved over 100x speedups on 200+ benchmark datasets compared to state-of-the-art solvers, solving instances where competing approaches fail.

Conclusion: Learning-augmented SOS certification transforms practical scalability, providing novel insights for SOS programming through efficient basis prediction.

Abstract: Certifying nonnegativity of polynomials is a well-known NP-hard problem with direct applications spanning non-convex optimization, control, robotics, and beyond. A sufficient condition for nonnegativity is the Sum of Squares (SOS) property, i.e., it can be written as a sum of squares of other polynomials. In practice, however, certifying the SOS criterion remains computationally expensive and often involves solving a Semidefinite Program (SDP), whose dimensionality grows quadratically in the size of the monomial basis of the SOS expression; hence, various methods to reduce the size of the monomial basis have been proposed. In this work, we introduce the first learning-augmented algorithm to certify the SOS criterion. To this end, we train a Transformer model that predicts an almost-minimal monomial basis for a given polynomial, thereby drastically reducing the size of the corresponding SDP. Our overall methodology comprises three key components: efficient training dataset generation of over 100 million SOS polynomials, design and training of the corresponding Transformer architecture, and a systematic fallback mechanism to ensure correct termination, which we analyze theoretically. We validate our approach on over 200 benchmark datasets, achieving speedups of over $100\times$ compared to state-of-the-art solvers and enabling the solution of instances where competing approaches fail. Our findings provide novel insights towards transforming the practical scalability of SOS programming.

Method: Develops a novel meta-learner that estimates informative bounds on CATE using arbitrary machine learning models, with double robustness and quasi-oracle efficiency properties.

Result: The framework identifies patient subgroups where treatment remains effective despite informative censoring, demonstrated through numerical experiments and cancer drug trial application.

Conclusion: Provides a practical tool for assessing treatment effect robustness in presence of censoring, promoting reliable use of survival data in medicine and epidemiology.

Abstract: Dropout is common in clinical studies, with up to half of patients leaving early due to side effects or other reasons. When dropout is informative (i.e., dependent on survival time), it introduces censoring bias, because of which treatment effect estimates are also biased. In this paper, we propose an assumption-lean framework to assess the robustness of conditional average treatment effect (CATE) estimates in survival analysis when facing censoring bias. Unlike existing works that rely on strong assumptions, such as non-informative censoring, to obtain point estimation, we use partial identification to derive informative bounds on the CATE. Thereby, our framework helps to identify patient subgroups where treatment is effective despite informative censoring. We further develop a novel meta-learner that estimates the bounds using arbitrary machine learning models and with favorable theoretical properties, including double robustness and quasi-oracle efficiency. We demonstrate the practical value of our meta-learner through numerical experiments and in an application to a cancer drug trial. Together, our framework offers a practical tool for assessing the robustness of estimated treatment effects in the presence of censoring and thus promotes the reliable use of survival data for evidence generation in medicine and epidemiology.

Method: Synthetic generation of SWIR-like structural/contrast cues from LWIR data using contrast enhancement techniques, followed by multimodal fusion framework with encoder-decoder neural network featuring modality-specific encoders and softmax-gated fusion head.

Result: The synthetic-SWIR-enhanced fusion framework improves fused-image quality (contrast, edge definition, structural fidelity) while maintaining real-time performance across multiple benchmarks and a private real RGB-MWIR-SWIR dataset.

Conclusion: The approach demonstrates substantial potential for real-world applications in surveillance and autonomous systems by effectively enhancing scene understanding in adverse visibility conditions.

Abstract: Enhancing scene understanding in adverse visibility conditions remains a critical challenge for surveillance and autonomous navigation systems. Conventional imaging modalities, such as RGB and thermal infrared (MWIR / LWIR), when fused, often struggle to deliver comprehensive scene information, particularly under conditions of atmospheric interference or inadequate illumination. To address these limitations, Short-Wave Infrared (SWIR) imaging has emerged as a promising modality due to its ability to penetrate atmospheric disturbances and differentiate materials with improved clarity. However, the advancement and widespread implementation of SWIR-based systems face significant hurdles, primarily due to the scarcity of publicly accessible SWIR datasets. In response to this challenge, our research introduces an approach to synthetically generate SWIR-like structural/contrast cues (without claiming spectral reproduction) images from existing LWIR data using advanced contrast enhancement techniques. We then propose a multimodal fusion framework integrating synthetic SWIR, LWIR, and RGB modalities, employing an optimized encoder-decoder neural network architecture with modality-specific encoders and a softmax-gated fusion head. Comprehensive experiments on public {RGB-LWIR benchmarks (M3FD, TNO, CAMEL, MSRS, RoadScene) and an additional private real RGB-MWIR-SWIR dataset} demonstrate that our synthetic-SWIR-enhanced fusion framework improves fused-image quality (contrast, edge definition, structural fidelity) while maintaining real-time performance. We also add fair trimodal baselines (LP, LatLRR, GFF) and cascaded trimodal variants of U2Fusion/SwinFusion under a unified protocol. The outcomes highlight substantial potential for real-world applications in surveillance and autonomous systems.

Method: Uses CLIP framework with EEG encoder based on Conformer architecture and text encoder with Learnable BERT (BERT-LP) for prompt learning, operating in shared latent space. Applies knowledge distillation where teacher model guides compact student model.

Result: Achieved >97% accuracy and F1-scores >0.94 on TUSZ, AUBMC, and CHB-MIT datasets. Student model has 58.1% parameters of teacher model while maintaining performance.

Conclusion: The model demonstrates robust epilepsy detection with reduced complexity, establishing foundation for lightweight deployment in resource-constrained settings.

Abstract: Epilepsy is a prevalent neurological disorder marked by sudden, brief episodes of excessive neuronal activity caused by abnormal electrical discharges, which may lead to some mental disorders. Most existing deep learning methods for epilepsy detection rely solely on unimodal EEG signals, neglecting the potential benefits of multimodal information. To address this, we propose a novel multimodal model, DistilCLIP-EEG, based on the CLIP framework, which integrates both EEG signals and text descriptions to capture comprehensive features of epileptic seizures. The model involves an EEG encoder based on the Conformer architecture as a text encoder, the proposed Learnable BERT (BERT-LP) as prompt learning within the encoders. Both operate in a shared latent space for effective cross-modal representation learning. To enhance efficiency and adaptability, we introduce a knowledge distillation method where the trained DistilCLIP-EEG serves as a teacher to guide a more compact student model to reduce training complexity and time. On the TUSZ, AUBMC, and CHB-MIT datasets, both the teacher and student models achieved accuracy rates exceeding 97%. Across all datasets, the F1-scores were consistently above 0.94, demonstrating the robustness and reliability of the proposed framework. Moreover, the student model’s parameter count and model size are approximately 58.1% of those of the teacher model, significantly reducing model complexity and storage requirements while maintaining high performance. These results highlight the potential of our proposed model for EEG-based epilepsy detection and establish a solid foundation for deploying lightweight models in resource-constrained settings.

Method: AdaLog addresses two key inefficiencies: (1) formulates feature-level redundant data elimination as maximum weighted matching in hypergraphs with hierarchical algorithm, (2) uses virtually hashed attribute design to distribute heterogeneous behaviors into dense log files, and (3) implements incremental update mechanism for dynamic behavior patterns.

Result: Evaluation on real-world user data shows AdaLog reduces behavior log size by 19% to 44% with minimal system overhead - only 2 seconds latency and 15 MB memory usage.

Conclusion: AdaLog provides an efficient data foundation for broader adoption of on-device ML by significantly improving storage efficiency without compromising model inference accuracy or latency.

Abstract: Machine learning (ML) models are increasingly integrated into modern mobile apps to enable personalized and intelligent services. These models typically rely on rich input features derived from historical user behaviors to capture user intents. However, as ML-driven services become more prevalent, recording necessary user behavior data imposes substantial storage cost on mobile apps, leading to lower system responsiveness and more app uninstalls. To address this storage bottleneck, we present AdaLog, a lightweight and adaptive system designed to improve the storage efficiency of user behavior log in ML-embedded mobile apps, without compromising model inference accuracy or latency. We identify two key inefficiencies in current industrial practices of user behavior log: (i) redundant logging of overlapping behavior data across different features and models, and (ii) sparse storage caused by storing behaviors with heterogeneous attribute descriptions in a single log file. To solve these issues, AdaLog first formulates the elimination of feature-level redundant data as a maximum weighted matching problem in hypergraphs, and proposes a hierarchical algorithm for efficient on-device deployment. Then, AdaLog employs a virtually hashed attribute design to distribute heterogeneous behaviors into a few log files with physically dense storage. Finally, to ensure scalability to dynamic user behavior patterns, AdaLog designs an incremental update mechanism to minimize the I/O operations needed for adapting outdated behavior log. We implement a prototype of AdaLog and deploy it into popular mobile apps in collaboration with our industry partner. Evaluations on real-world user data show that AdaLog reduces behavior log size by 19% to 44% with minimal system overhead (only 2 seconds latency and 15 MB memory usage), providing a more efficient data foundation for broader adoption of on-device ML.

Method: Procrustes post-processing - an orthogonal transformation that aligns two sets of embeddings while preserving their geometric structure, based on the preservation of pairwise dot products.

Result: The method achieves state-of-the-art performance in mixed-modality search and effectively maintains compatibility across model retrainings and enables combining different models for text retrieval.

Conclusion: Orthogonal transformations can effectively make embedding models interoperable while preserving their geometric properties, solving practical challenges in model deployment and multimodal applications.

Abstract: Embedding models trained separately on similar data often produce representations that encode stable information but are not directly interchangeable. This lack of interoperability raises challenges in several practical applications, such as model retraining, partial model upgrades, and multimodal search. Driven by these challenges, we study when two sets of embeddings can be aligned by an orthogonal transformation. We show that if pairwise dot products are approximately preserved, then there exists an isometry that closely aligns the two sets, and we provide a tight bound on the alignment error. This insight yields a simple alignment recipe, Procrustes post-processing, that makes two embedding models interoperable while preserving the geometry of each embedding space. Empirically, we demonstrate its effectiveness in three applications: maintaining compatibility across retrainings, combining different models for text retrieval, and improving mixed-modality search, where it achieves state-of-the-art performance.

Method: Designed pessimism-based algorithm for offline setting and optimism-based algorithm for online setting, both operating under ε-LDP constraints on human preference labels.

Result: Achieved optimal suboptimality gap of Õ(1/[(e^ε-1)^2 n]) in offline setting and logarithmic regret bound of O(d_F log(N_F·T)/(e^ε-1)^2) in online setting.

Conclusion: The paper provides the first theoretical analysis of KL-regularized RLHF under LDP, with optimal algorithms and bounds, and also contributes to non-private RLHF analysis.

Abstract: In this paper, we study the offline and online settings of reinforcement learning from human feedback (RLHF) with KL-regularization – a widely used objective function in large language model alignment – under the $\epsilon$ local differential privacy ($\epsilon$-LDP) model on the label of the human preference. In the offline setting, we design an algorithm based on the principle of pessimism and derive a new suboptimality gap of $\tilde{O}(1/[(e^\epsilon-1)^2 n])$ on the KL-regularized objective under single-policy concentrability. We also prove its optimality by providing a matching lower bound where $n$ is the sample size. In the online setting, we are the first one to theoretically investigate the problem of KL-regularized RLHF with LDP. We design an optimism-based algorithm and derive a logarithmic regret bound of $O(d_{\mathcal{F}}\log (N_{\mathcal{F}}\cdot T) /(e^\epsilon-1)^2 )$, where $T$ is the total time step, $N_{\mathcal{F}}$ is cardinality of the reward function space $\mathcal{F}$ and $d_{\mathcal{F}}$ is a variant of eluder dimension for RLHF. As a by-product of our analysis, our results also imply the first analysis for online KL-regularized RLHF without privacy. We implement our algorithm in the offline setting to verify our theoretical results and release our open source code at: https://github.com/rushil-thareja/PPKL-RLHF-Official.

Method: Extended PINN framework with biological constraints as regularization, applied to ODE model of combination therapy to learn time-varying parameters and system dynamics.

Result: Algorithm successfully learns ODE solutions and time-varying parameters from sparse data, with strong convergence demonstrated by low MSE, MAE, and MAPE metrics.

Conclusion: Modified PINN approach effectively handles sparse oncology data by incorporating biological constraints, enabling accurate learning of treatment dynamics with limited observations.

Abstract: Physics-informed neural networks (PINNs) are neural networks that embed the laws of dynamical systems modeled by differential equations into their loss function as constraints. In this work, we present a PINN framework applied to oncology. Here, we seek to learn time-varying interactions due to a combination therapy in a tumor microenvironment. In oncology, experimental data are often sparse and composed of a few time points of tumor volume. By embedding inductive biases derived from prior information about a dynamical system, we extend the physics-informed neural networks (PINN) and incorporate observed biological constraints as regularization agents. The modified PINN algorithm is able to steer itself to a reasonable solution and can generalize well with only a few training examples. We demonstrate the merit of our approach by learning the dynamics of treatment applied intermittently in an ordinary differential equation (ODE) model of a combination therapy. The algorithm yields a solution to the ODE and time-varying forms of some of the ODE model parameters. We demonstrate a strong convergence using metrics such as the mean squared error (MSE), mean absolute error (MAE), and mean absolute percentage error (MAPE).

Method: Hybrid fuzzy regression with dual rule structure combining fuzzy and crisp components, integrating Mamdani’s interpretability with TSK’s precision through improved rule outputs.

Result: Achieved best fuzzy methodology score in 4 out of 6 datasets, outperformed opaque models in 2 datasets, best overall score in 1 dataset, with RMSE improvements ranging from 0.4% to 19%.

Conclusion: The hybrid approach provides a balanced solution to the interpretability-accuracy trade-off in fuzzy systems, offering a versatile tool for predictive modeling.

Abstract: Regression analysis is employed to examine and quantify the relationships between input variables and a dependent and continuous output variable. It is widely used for predictive modelling in fields such as finance, healthcare, and engineering. However, traditional methods often struggle with real-world data complexities, including uncertainty and ambiguity. While deep learning approaches excel at capturing complex non-linear relationships, they lack interpretability and risk over-fitting on small datasets. Fuzzy systems provide an alternative framework for handling uncertainty and imprecision, with Mamdani and Takagi-Sugeno-Kang (TSK) systems offering complementary strengths: interpretability versus accuracy. This paper presents a novel fuzzy regression method that combines the interpretability of Mamdani systems with the precision of TSK models. The proposed approach introduces a hybrid rule structure with fuzzy and crisp components and dual dominance types, enhancing both accuracy and explainability. Evaluations on benchmark datasets demonstrate state-of-the-art performance in several cases, with rules maintaining a component similar to traditional Mamdani systems while improving precision through improved rule outputs. This hybrid methodology offers a balanced and versatile tool for predictive modelling, addressing the trade-off between interpretability and accuracy inherent in fuzzy systems. In the 6 datasets tested, the proposed approach gave the best fuzzy methodology score in 4 datasets, out-performed the opaque models in 2 datasets and produced the best overall score in 1 dataset with the improvements in RMSE ranging from 0.4% to 19%.

Method: Theoretical analysis using finite-sample generalization bounds for smooth calibration error based on optimization error, regularization strength, and Rademacher complexity, instantiated for kernel ridge and logistic regression.

Result: Established that canonical L2-regularized ERM directly controls smooth calibration error without post-hoc correction, with experimental confirmation of these guarantees.

Conclusion: L2-regularized empirical risk minimization can provide well-calibrated models without boosting or post-hoc recalibration, offering theoretical foundation for standard training procedures.

Abstract: Calibration of predicted probabilities is critical for reliable machine learning, yet it is poorly understood how standard training procedures yield well-calibrated models. This work provides the first theoretical proof that canonical $L_{2}$-regularized empirical risk minimization directly controls the smooth calibration error (smCE) without post-hoc correction or specialized calibration-promoting regularizer. We establish finite-sample generalization bounds for smCE based on optimization error, regularization strength, and the Rademacher complexity. We then instantiate this theory for models in reproducing kernel Hilbert spaces, deriving concrete guarantees for kernel ridge and logistic regression. Our experiments confirm these specific guarantees, demonstrating that $L_{2}$-regularized ERM can provide a well-calibrated model without boosting or post-hoc recalibration. The source code to reproduce all experiments is available at https://github.com/msfuji0211/erm_calibration.

Method: A data-free and computationally efficient strategy for selecting and merging LoRAs when new ones become available, assuming limited adapter storage capacity on devices.

Result: Extensive experiments across real-world tasks demonstrate superiority over alternative strategies while adhering to on-device storage budget and compute limitations.

Conclusion: The proposed approach effectively addresses the challenge of online continual LoRA merging for incremental task support on resource-constrained mobile devices.

Abstract: On-device deployment of Large Language Models (LLMs) frequently leverages Low-Rank Adapters (LoRAs) to support diverse downstream tasks under tight resource constraints. To address the limited storage capacity of mobile devices, recent works have explored model merging techniques to fuse multiple LoRAs into a single one. In practice, however, LoRAs are often delivered incrementally, as users request support for new tasks (e.g., novel problem types or languages). This scenario introduces a new challenge: on-device online continual merging, where the objective is to incorporate new LoRAs while preserving the performance on previously supported tasks. In this paper, we propose a data-free and computationally efficient strategy for selecting and merging LoRAs when a new one becomes available, assuming the device can store only a limited number of adapters. Extensive experiments across real-world tasks demonstrate the superiority of our approach compared to alternative strategies while adhering to the storage budget and compute limitations of on-device settings.

Method: Construct learning algorithms for each optimality order with vanishing error probability. Characterize MDPs where identification can stop in finite time (those with unique Bellman optimal policy). Provide tractable stopping rule.

Result: Developed learning algorithms with vanishing probability of error for identifying bias-optimal policies. Characterized the class of MDPs where finite-time stopping is possible. Provided a tractable stopping rule that triggers in finite time when possible.

Conclusion: The paper successfully addresses the problem of identifying bias-optimal policies in MDPs, providing algorithms with finite-time stopping guarantees for MDPs with unique Bellman optimal policies, bridging the gap between asymptotic and immediate performance measures.

Abstract: Although average gain optimality is a commonly adopted performance measure in Markov Decision Processes (MDPs), it is often too asymptotic. Further incorporating measures of immediate losses leads to the hierarchy of bias optimalities, all the way up to Blackwell optimality. In this paper, we investigate the problem of identifying policies of such optimality orders. To that end, for each order, we construct a learning algorithm with vanishing probability of error. Furthermore, we characterize the class of MDPs for which identification algorithms can stop in finite time. That class corresponds to the MDPs with a unique Bellman optimal policy, and does not depend on the optimality order considered. Lastly, we provide a tractable stopping rule that when coupled to our learning algorithm triggers in finite time whenever it is possible to do so.

Method: The authors detail their approach to Arabic pre-training data collection and filtration, assess various tokenizer designs’ impact on performance, and propose a corrective methodology for existing Arabic evaluation frameworks.

Result: The paper shares data and methodologies to promote transparency and collaborative development in Arabic language modeling.

Conclusion: The research contributes to advancing language modeling for Arabic by addressing key challenges in data, tokenization, and evaluation, while promoting open collaboration through shared resources.

Abstract: Large Language Models (LLMs) have significantly advanced the field of natural language processing, enhancing capabilities in both language understanding and generation across diverse domains. However, developing LLMs for Arabic presents unique challenges. This paper explores these challenges by focusing on critical aspects such as data curation, tokenizer design, and evaluation. We detail our approach to the collection and filtration of Arabic pre-training datasets, assess the impact of various tokenizer designs on model performance, and examine the limitations of existing Arabic evaluation frameworks, for which we propose a systematic corrective methodology. To promote transparency and facilitate collaborative development, we share our data and methodologies, contributing to the advancement of language modeling, particularly for the Arabic language.

Method: Uses prototypical networks that compute distances between input datapoints and prototype representations, making them effective for few-shot learning in topic modeling.

Result: ProtoTopic demonstrates improved topic coherence and diversity compared to baseline topic modeling methods, generating medically relevant topics even with limited data.

Conclusion: Prototypical networks are effective for topic modeling in medical domains with limited data, offering improved performance over traditional methods.

Abstract: Topic modeling is a useful tool for analyzing large corpora of written documents, particularly academic papers. Despite a wide variety of proposed topic modeling techniques, these techniques do not perform well when applied to medical texts. This can be due to the low number of documents available for some topics in the healthcare domain. In this paper, we propose ProtoTopic, a prototypical network-based topic model used for topic generation for a set of medical paper abstracts. Prototypical networks are efficient, explainable models that make predictions by computing distances between input datapoints and a set of prototype representations, making them particularly effective in low-data or few-shot learning scenarios. With ProtoTopic, we demonstrate improved topic coherence and diversity compared to two topic modeling baselines used in the literature, demonstrating the ability of our model to generate medically relevant topics even with limited data.

Method: Proposes a simple algorithm combining Hedge and online gradient descent (OGD) for the i.i.d. input setting where loss functions are generated from an unknown distribution.

Result: The algorithm achieves an expected min-max regret of O(√(T log K)) with a remarkably simple proof.

Conclusion: The proposed Hedge+OGD algorithm effectively handles multi-objective OCO with strong theoretical guarantees for min-max regret in i.i.d. settings.

Abstract: In online convex optimization (OCO), a single loss function sequence is revealed over a time horizon of $T$, and an online algorithm has to choose its action at time $t$, before the loss function at time $t$ is revealed. The goal of the online algorithm is to incur minimal penalty (called $\textit{regret}$ compared to a static optimal action made by an optimal offline algorithm knowing all functions of the sequence in advance. In this paper, we broaden the horizon of OCO, and consider multi-objective OCO, where there are $K$ distinct loss function sequences, and an algorithm has to choose its action at time $t$, before the $K$ loss functions at time $t$ are revealed. To capture the tradeoff between tracking the $K$ different sequences, we consider the $\textit{min-max}$ regret, where the benchmark (optimal offline algorithm) takes a static action across all time slots that minimizes the maximum of the total loss (summed across time slots) incurred by each of the $K$ sequences. An online algorithm is allowed to change its action across time slots, and its {\it min-max} regret is defined as the difference between its $\textit{min-max}$ cost and that of the benchmark. The $\textit{min-max}$ regret is a stringent performance measure and an algorithm with small regret needs to `track’ all loss function sequences closely at all times. We consider this $\textit{min-max}$ regret in the i.i.d. input setting where all loss functions are i.i.d. generated from an unknown distribution. For the i.i.d. model we propose a simple algorithm that combines the well-known $\textit{Hedge}$ and online gradient descent (OGD) and show via a remarkably simple proof that its expected $\textit{min-max}$ regret is $O(\sqrt{T \log K})$.

Method: Proposes DOLFIN method combining Vision Transformers with low-rank adapters (LoRA) for minimal communication overhead, and incorporates DualGradient Projection Memory (DualGPM) to prevent catastrophic forgetting in federated environments.

Result: Evaluated on CIFAR-100, ImageNet-R, ImageNet-A, and CUB-200 under two Dirichlet heterogeneity settings, DOLFIN consistently surpassed six strong baselines in final average accuracy while matching their memory footprint.

Conclusion: Orthogonal low-rank adapters provide an effective and scalable solution for privacy-preserving continual learning in federated settings.

Abstract: Federated continual learning (FCL) enables models to learn new tasks across multiple distributed clients, protecting privacy and without forgetting previously acquired knowledge. However, current methods face challenges balancing performance, privacy preservation, and communication efficiency. We introduce a Distributed Online LoRA for Federated INcremental learning method DOLFIN, a novel approach combining Vision Transformers with low-rank adapters designed to efficiently and stably learn new tasks in federated environments. Our method leverages LoRA for minimal communication overhead and incorporates DualGradient Projection Memory (DualGPM) to prevent forgetting. Evaluated on CIFAR-100, ImageNet-R, ImageNet-A, and CUB-200 under two Dirichlet heterogeneity settings, DOLFIN consistently surpasses six strong baselines in final average accuracy while matching their memory footprint. Orthogonal low-rank adapters offer an effective and scalable solution for privacy-preserving continual learning in federated settings.

Method: Proposed EB-1WL edge-based color-refinement test and EB-GNN architecture inspired by Chiba and Nishizeki’s triangle counting algorithm, explicitly incorporating triangles during message passing.

Result: EB-1WL is significantly more expressive than 1-WL, provides complete logical characterization via first-order logic, requires near-linear time/memory, and EB-GNN outperforms simple MPNNs while remaining competitive with specialized GNNs with better computational efficiency.

Conclusion: EB-GNN represents a highly-efficient general-purpose GNN architecture that balances expressiveness with computational efficiency, making it suitable for practical graph learning tasks.

Abstract: We propose EB-1WL, an edge-based color-refinement test, and a corresponding GNN architecture, EB-GNN. Our architecture is inspired by a classic triangle counting algorithm by Chiba and Nishizeki, and explicitly uses triangles during message passing. We achieve the following results: (1)~EB-1WL is significantly more expressive than 1-WL. Further, we provide a complete logical characterization of EB-1WL based on first-order logic, and matching distinguishability results based on homomorphism counting. (2)~In an important distinction from previous proposals for more expressive GNN architectures, EB-1WL and EB-GNN require near-linear time and memory on practical graph learning tasks. (3)~Empirically, we show that EB-GNN is a highly-efficient general-purpose architecture: It substantially outperforms simple MPNNs, and remains competitive with task-specialized GNNs while being significantly more computationally efficient.

Method: Used canonical attacks from TextAttack framework, developed a custom constraint to increase selectivity, and created a surrogate-LLM pipeline to generate selective perturbations on the MMLU benchmark.

Result: Found that selective adversarial attacks exist and can materially alter relative rankings of LLMs, demonstrating that even subtle edits can shift comparative judgments.

Conclusion: Benchmark evaluations are vulnerable to selective adversarial attacks, motivating the need for perturbation-aware reporting and robustness diagnostics to ensure fair and transparent LLM evaluation.

Abstract: Benchmarking outcomes increasingly govern trust, selection, and deployment of LLMs, yet these evaluations remain vulnerable to semantically equivalent adversarial perturbations. Prior work on adversarial robustness in NLP has emphasized text attacks that affect many models equally, leaving open the question of whether it is possible to selectively degrade or enhance performance while minimally affecting other models. We formalize this problem and study selective adversarial attacks on MMLU - a widely used benchmark designed to measure a language model’s broad general knowledge and reasoning ability across different subjects. Using canonical attacks integrated into TextAttack framework, we introduce a protocol for selectivity assessment, develop a custom constraint to increase selectivity of attacks and propose a surrogate-LLM pipeline that generates selective perturbations. Empirically, we find that selective adversarial attacks exist and can materially alter relative rankings, challenging the fairness, reproducibility, and transparency of leaderboard-driven evaluation. Our results motivate perturbation-aware reporting and robustness diagnostics for LLM evaluation and demonstrate that even subtle edits can shift comparative judgments.

Method: ArtNet generates artificial netlists that replicate key topological characteristics, producing realistic datasets that match target parameters for enhanced ML training and DTCO exploration.

Result: In CNN-based DRV prediction, ArtNet’s data augmentation improves F1 score by 0.16. In DTCO context, ArtNet-generated mini-brains achieve PPA match up to 97.94% with full-scale block designs.

Conclusion: ArtNet effectively addresses data scarcity and TAT issues in advanced node design, enabling more efficient PPA optimization through realistic artificial netlist generation.

Abstract: In advanced nodes, optimization of power, performance and area (PPA) has become highly complex and challenging. Machine learning (ML) and design-technology co-optimization (DTCO) provide promising mitigations, but face limitations due to a lack of diverse training data as well as long design flow turnaround times (TAT). We propose ArtNet, a novel artificial netlist generator designed to tackle these issues. Unlike previous methods, ArtNet replicates key topological characteristics, enhancing ML model generalization and supporting broader design space exploration for DTCO. By producing realistic artificial datasets that moreclosely match given target parameters, ArtNet enables more efficient PPAoptimization and exploration of flows and design enablements. In the context of CNN-based DRV prediction, ArtNet’s data augmentationimproves F1 score by 0.16 compared to using only the original (real) dataset. In the DTCO context, ArtNet-generated mini-brains achieve a PPA match up to 97.94%, demonstrating close alignment with design metrics of targeted full-scale block designs.

Method: Investigation of existing TSFM evaluation practices, identifying challenges in benchmark dataset representativeness, spatiotemporal evaluation gaps, information leakage risks, and data partition confusion.

Result: Findings reveal widespread confusion regarding data partitions, risks of inflated performance estimates, incorrect transfer of global knowledge to local time series, and memorization issues from external shocks.

Conclusion: Calls for robust evaluation methodologies, principled approaches like truly out-of-sample future data evaluation, and community action to safeguard TSFM assessment integrity.

Abstract: Time Series Foundation Models (TSFMs) represent a new paradigm for time series forecasting, offering zero-shot forecasting capabilities without the need for domain-specific pre-training or fine-tuning. However, as with Large Language Models (LLMs), evaluating TSFMs is tricky, as with ever more extensive training sets, it becomes more and more challenging to ensure the integrity of benchmarking data. Our investigation of existing TSFM evaluation highlights multiple challenges, ranging from the representativeness of the benchmark datasets, over the lack of spatiotemporal evaluation, to risks of information leakage due to overlapping and obscure datasets, and the memorization of global patterns caused by external shocks like economic crises or pandemics. Our findings reveal widespread confusion regarding data partitions, risking inflated performance estimates and incorrect transfer of global knowledge to local time series. We argue for the development of robust evaluation methodologies to prevent pitfalls already observed in LLM and classical time series benchmarking, and call upon the research community to design new, principled approaches, such as evaluations on truly out-of-sample future data, to safeguard the integrity of TSFM assessment.

Method: Proposed EEGChaT with Channel Aggregation Tokens (CATs) to aggregate information across channels, and improved Attention Rollout technique to compute interpretable channel importance scores.

Result: On DuIN dataset, EEGChaT integration improved decoding accuracy by up to 17% absolute gains. Channel weights showed substantial overlap with manually selected channels, supporting interpretability.

Conclusion: EEGChaT is an effective and generalizable solution for channel selection in high-dimensional SEEG analysis, offering enhanced performance and insights into neural signal relevance.

Abstract: Analyzing stereoelectroencephalography (SEEG) signals is critical for brain-computer interface (BCI) applications and neuroscience research, yet poses significant challenges due to the large number of input channels and their heterogeneous relevance. Traditional channel selection methods struggle to scale or provide meaningful interpretability for SEEG data. In this work, we propose EEGChaT, a novel Transformer-based channel selection module designed to automatically identify the most task-relevant channels in SEEG recordings. EEGChaT introduces Channel Aggregation Tokens (CATs) to aggregate information across channels, and leverages an improved Attention Rollout technique to compute interpretable, quantitative channel importance scores. We evaluate EEGChaT on the DuIN dataset, demonstrating that integrating EEGChaT with existing classification models consistently improves decoding accuracy, achieving up to 17% absolute gains. Furthermore, the channel weights produced by EEGChaT show substantial overlap with manually selected channels, supporting the interpretability of the approach. Our results suggest that EEGChaT is an effective and generalizable solution for channel selection in high-dimensional SEEG analysis, offering both enhanced performance and insights into neural signal relevance.

Method: Developed XNN architecture inspired by parameter-sharing structures like Deep Sets and Graph Neural Networks. Converted existing PDE foundation models into XNNs and evaluated across three training scenarios: from scratch, pretraining on multiple PDEs, and fine-tuning on single PDE.

Result: XNNs perform competitively with original models and show superior generalization to unseen dimensions, demonstrating the importance of multidimensional pretraining for foundation models.

Conclusion: XNN architecture effectively addresses dimensionality challenges in physics foundation models, enabling efficient training and better generalization across varying dimensional PDE systems.

Abstract: The advent of foundation models in AI has significantly advanced general-purpose learning, enabling remarkable capabilities in zero-shot inference and in-context learning. However, training such models on physics data, including solutions to partial differential equations (PDEs), poses a unique challenge due to varying dimensionalities across different systems. Traditional approaches either fix a maximum dimension or employ separate encoders for different dimensionalities, resulting in inefficiencies. To address this, we propose a dimension-agnostic neural network architecture, the Axial Neural Network (XNN), inspired by parameter-sharing structures such as Deep Sets and Graph Neural Networks. XNN generalizes across varying tensor dimensions while maintaining computational efficiency. We convert existing PDE foundation models into axial neural networks and evaluate their performance across three training scenarios: training from scratch, pretraining on multiple PDEs, and fine-tuning on a single PDE. Our experiments show that XNNs perform competitively with original models and exhibit superior generalization to unseen dimensions, highlighting the importance of multidimensional pretraining for foundation models.

Method: Developed a geometry-aware multi-task Gaussian Process model to capture spatiotemporal structure and inter-task dependencies, augmented with physics-based regularization that constrains predictions to be consistent with governing physical laws.

Result: Validated on 3D cardiac electrodynamics modeling, demonstrating significant improvement in prediction accuracy over existing methods by effectively incorporating domain-specific physical constraints and geometric priors.

Conclusion: The proposed P-M-GP framework successfully enhances model fidelity and robustness for spatiotemporal dynamic systems by integrating physical knowledge with data-driven modeling.

Abstract: Recent advances in sensing and imaging technologies have enabled the collection of high-dimensional spatiotemporal data across complex geometric domains. However, effective modeling of such data remains challenging due to irregular spatial structures, rapid temporal dynamics, and the need to jointly predict multiple interrelated physical variables. This paper presents a physics-augmented multi-task Gaussian Process (P-M-GP) framework tailored for spatiotemporal dynamic systems. Specifically, we develop a geometry-aware, multi-task Gaussian Process (M-GP) model to effectively capture intrinsic spatiotemporal structure and inter-task dependencies. To further enhance the model fidelity and robustness, we incorporate governing physical laws through a physics-based regularization scheme, thereby constraining predictions to be consistent with governing dynamical principles. We validate the proposed P-M-GP framework on a 3D cardiac electrodynamics modeling task. Numerical experiments demonstrate that our method significantly improves prediction accuracy over existing methods by effectively incorporating domain-specific physical constraints and geometric prior.

Method: Uses Task Arithmetic and Neural Tangent Kernel to rapidly remove a client’s influence from a model without requiring multiple communication rounds.

Result: Enables fast removal of client contributions while maintaining model availability during the process.

Conclusion: The proposed solution addresses the critical need for efficient client influence removal in federated learning systems while preserving model service continuity.

Abstract: Nowdays, there are an abundance of portable devices capable of collecting large amounts of data and with decent computational power. This opened the possibility to train AI models in a distributed manner, preserving the participating clients’ privacy. However, because of privacy regulations and safety requirements, elimination upon necessity of a client contribution to the model has become mandatory. The cleansing process must satisfy specific efficacy and time requirements. In recent years, research efforts have produced several knowledge removal methods, but these require multiple communication rounds between the data holders and the process coordinator. This can cause the unavailability of an effective model up to the end of the removal process, which can result in a disservice to the system users. In this paper, we introduce an innovative solution based on Task Arithmetic and the Neural Tangent Kernel, to rapidly remove a client’s influence from a model.

Method: Systematic framework for constructing neural decoder architectures for NLDR methods, extended with diffusion-based generative process operating directly in learned manifold spaces.

Result: Decoders successfully reconstruct data but are outperformed by autoencoders. Manifold-constrained diffusion yields poor-quality samples due to discrete/sparse nature of NLDR embeddings.

Conclusion: There are inherent challenges in retrofitting generative capabilities onto NLDR methods designed primarily for visualization, as their embeddings are ill-suited for continuous interpolation required by generative models.

Abstract: Classical nonlinear dimensionality reduction (NLDR) techniques like t-SNE, Isomap, and LLE excel at creating low-dimensional embeddings for data visualization but fundamentally lack the ability to map these embeddings back to the original high-dimensional space. This one-way transformation limits their use in generative applications. This paper addresses this critical gap by introducing a system- atic framework for constructing neural decoder architectures for prominent NLDR methods, enabling bidirectional mapping for the first time. We extend this framework by implementing a diffusion-based generative process that operates directly within these learned manifold spaces. Through experiments on the CelebA dataset, we evaluate the reconstruction and generative performance of our approach against autoencoder and standard diffusion model baselines. Our findings reveal a fundamental trade- off: while the decoders successfully reconstruct data, their quality is surpassed by end-to-end optimized autoencoders. Moreover, manifold-constrained diffusion yields poor-quality samples, suggesting that the discrete and sparse nature of classical NLDR embeddings is ill-suited for the continuous inter- polation required by generative models. This work highlights the inherent challenges in retrofitting generative capabilities onto NLDR methods designed primarily for visualization and analysis.

Method: Propose simplicial embeddings - lightweight representation layers that constrain embeddings to simplicial structures, creating sparse and discrete features.

Result: Applied to FastTD3, FastSAC, and PPO, simplicial embeddings consistently improve sample efficiency and final performance across continuous- and discrete-control environments.

Conclusion: Simplicial embeddings provide geometric inductive bias that stabilizes learning and enhances performance without sacrificing runtime speed.

Abstract: Recent works have proposed accelerating the wall-clock training time of actor-critic methods via the use of large-scale environment parallelization; unfortunately, these can sometimes still require large number of environment interactions to achieve a desired level of performance. Noting that well-structured representations can improve the generalization and sample efficiency of deep reinforcement learning (RL) agents, we propose the use of simplicial embeddings: lightweight representation layers that constrain embeddings to simplicial structures. This geometric inductive bias results in sparse and discrete features that stabilize critic bootstrapping and strengthen policy gradients. When applied to FastTD3, FastSAC, and PPO, simplicial embeddings consistently improve sample efficiency and final performance across a variety of continuous- and discrete-control environments, without any loss in runtime speed.

Method: Developed MTS-QRC with paired injection and memory qubits using Trotterized nearest-neighbor transverse-field Ising evolution optimized for current quantum device connectivity and depth constraints.

Result: Achieved MSE of 0.0087 on Lorenz-63 and 0.0036 on ENSO, performing competitively with classical methods. On IBM Heron R2 hardware, maintained accuracy with realistic depths and surprisingly outperformed noiseless simulator on ENSO due to noise-induced variance concentration.

Conclusion: The method demonstrates practical gate-based QRC for multivariate time series forecasting on NISQ hardware and suggests systematic investigation of when hardware noise can benefit quantum reservoir computing readouts.

Abstract: Quantum reservoir computing (QRC) offers a hardware-friendly approach to temporal learning, yet most studies target univariate signals and overlook near-term hardware constraints. This work introduces a gate-based QRC for multivariate time series (MTS-QRC) that pairs injection and memory qubits and uses a Trotterized nearest-neighbor transverse-field Ising evolution optimized for current device connectivity and depth. On Lorenz-63 and ENSO, the method achieves a mean square error (MSE) of 0.0087 and 0.0036, respectively, performing on par with classical reservoir computing on Lorenz and above learned RNNs on both, while NVAR and clustered ESN remain stronger on some settings. On IBM Heron R2, MTS-QRC sustains accuracy with realistic depths and, interestingly, outperforms a noiseless simulator on ENSO; singular value analysis indicates that device noise can concentrate variance in feature directions, acting as an implicit regularizer for linear readout in this regime. These findings support the practicality of gate-based QRC for MTS forecasting on NISQ hardware and motivate systematic studies on when and how hardware noise benefits QRC readouts.

Method: Analyze rejection sampling algorithms and GRPO algorithm by showing they correspond to stochastic gradient ascent on specific monotone transformations (logarithm and arcsine of square root) of the probability of correct answers.

Result: Demonstrated that rejection sampling algorithms correspond to the logarithm transformation, while GRPO corresponds to the arcsine of square root transformation of the probability of correct answers.

Conclusion: Multiple popular RL algorithms for LLMs with binary rewards can be mathematically unified as stochastic gradient ascent on different monotone transformations of the probability of correct responses.

Abstract: We show that several popular algorithms for reinforcement learning in large language models with binary rewards can be viewed as stochastic gradient ascent on a monotone transform of the probability of a correct answer given a prompt. In particular, the transformation associated with rejection sampling algorithms is the logarithm and that associated with the GRPO algorithm is the arcsine of the square root.

Method: Uses an encoder-decoder network to preserve imbalanced data structure, then extracts feature vectors to generate synthetic samples through distribution calibration strategy that leverages weighted parameters from Gaussian mixtures of majority and intermediate classes in neighboring regions.

Result: Achieves superior classification performance compared to baseline and state-of-the-art techniques across diverse image, text, and tabular datasets.

Conclusion: The proposed RCS method effectively mitigates class imbalance issues by calibrating distribution parameters using neighboring class information, preventing overgeneralization and improving classification robustness.

Abstract: The class imbalance problem refers to the insufficiency of data in certain classes, which causes a classifier to be biased toward the majority class. Distribution calibration is a technique that seeks to estimate a more accurate class distribution based on an observed or estimated one. To address this issue, we propose a distribution calibration-based method-Rebalancing with Calibrated Sub-classes (RCS): An Enhanced Approach for Robust Imbalanced Classification, which estimates the distribution parameters of the minority classes using weighted parameters derived from a mixture of Gaussian components from both the majority and intermediate classes. An encoder-decoder network is trained to preserve the structure of the imbalanced data and prevent disentanglement. After training, feature vectors extracted from the encoder are used to generate synthetic samples through our distribution calibration strategy. This approach effectively mitigates the overgeneralization problem that arises when only the distribution of the majority class is used to approximate the minority class statistics. Instead, our method calibrates the parameters by leveraging the distribution of data points in neighboring regions. Experimental results demonstrate that the proposed method achieves superior classification performance compared to several baseline and state-of-the-art techniques across a diverse range of image, text, and tabular datasets.

Method: Theoretical analysis on quadratic objectives and logistic regression across all four quadrants, examining both full-batch and stochastic settings. Empirical studies on convex and non-convex objectives to validate theoretical findings.

Result: In full-batch settings, Adam can outperform both GN⁻¹ and GN⁻¹/² regardless of basis choice. In stochastic regime, Adam behaves similarly to GN⁻¹/² for linear regression under Gaussian data assumption.

Conclusion: The performance between Adam and Gauss-Newton diagonal preconditioners depends on the optimization setting (full-batch vs stochastic) and basis choice, with Adam showing advantages in certain full-batch scenarios and behaving like GN⁻¹/² in stochastic linear regression.

Abstract: Diagonal preconditioners are computationally feasible approximate to second-order optimizers, which have shown significant promise in accelerating training of deep learning models. Two predominant approaches are based on Adam and Gauss-Newton (GN) methods: the former leverages statistics of current gradients and is the de-factor optimizers for neural networks, and the latter uses the diagonal elements of the Gauss-Newton matrix and underpins some of the recent diagonal optimizers such as Sophia. In this work, we compare these two diagonal preconditioning methods through the lens of two key factors: the choice of basis in the preconditioner, and the impact of gradient noise from mini-batching. To gain insights, we analyze these optimizers on quadratic objectives and logistic regression under all four quadrants. We show that regardless of the basis, there exist instances where Adam outperforms both GN$^{-1}$ and GN$^{-1/2}$ in full-batch settings. Conversely, in the stochastic regime, Adam behaves similarly to GN$^{-1/2}$ for linear regression under a Gaussian data assumption. These theoretical results are supported by empirical studies on both convex and non-convex objectives.

Method: Propose InfoRM based on Information Bottleneck principle to filter irrelevant information, IBL distribution-level regularization that penalizes deviations from SFT-induced distribution using Mahalanobis distance, and MOP metric for quantifying reward hacking.

Result: Extensive experiments across diverse LLMs and datasets confirm the effectiveness of InfoRM and IBL, and reliability of MOP as a diagnostic tool for reward hacking mitigation.

Conclusion: The proposed framework collectively advances RLHF by providing principled solutions to reward hacking through information-theoretic reward modeling, distribution-level regularization, and statistical quantification of reward hacking severity.

Abstract: Despite the success of Reinforcement Learning from Human Feedback (RLHF) in aligning language models with human values, reward hacking-or reward over-optimization-remains a major challenge. We identify two key obstacles to its mitigation: (1) reward misgeneralization in reward modeling, where reward models overfit to spurious, preference-irrelevant features; and (2) the lack of suitable regularization during RL optimization, as existing token-level constraints often over-restrict the policy space. To address these issues, we propose InfoRM, an information-theoretic reward modeling framework based on the Information Bottleneck (IB) principle, which filters out preference-irrelevant information to alleviate reward misgeneralization. We further observe that reward-hacked responses manifest as pronounced outliers in InfoRM’s IB latent space, measured by Mahalanobis distance from the SFT-induced distribution. Motivated by this, we introduce IBL, a distribution-level regularization that penalizes such deviations, effectively expanding the optimization landscape while maintaining alignment. We prove that IBL is theoretically equivalent to the pessimistic RL objective within the IB latent space. Finally, we present Mahalanobis Outlier Probability (MOP), a statistical metric for quantifying reward hacking severity, enabling principled hyperparameter tuning and online mitigation such as early stopping. Extensive experiments across diverse LLMs and datasets confirm the generality of our findings, the effectiveness of InfoRM and IBL, and the reliability of MOP as a diagnostic tool-collectively advancing the state of RLHF.

Method: The authors use convex relaxation of combinatorial constraints and solve the resulting problem using the Frank-Wolfe algorithm, then round the relaxed solution to obtain an approximate solution to the original combinatorial problem.

Result: The method drastically reduces per-layer pruning error, outperforms strong baselines on GPT architectures, and remains memory-efficient.

Conclusion: The Frank-Wolfe approach provides theoretical guarantees and practical improvements for LLM pruning by better handling weight interactions in the pruning objective.

Abstract: Pruning is a common technique to reduce the compute and storage requirements of Neural Networks. While conventional approaches typically retrain the model to recover pruning-induced performance degradation, state-of-the-art Large Language Model (LLM) pruning methods operate layer-wise, minimizing the per-layer pruning error on a small calibration dataset to avoid full retraining, which is considered computationally prohibitive for LLMs. However, finding the optimal pruning mask is a hard combinatorial problem and solving it to optimality is intractable. Existing methods hence rely on greedy heuristics that ignore the weight interactions in the pruning objective. In this work, we instead consider the convex relaxation of these combinatorial constraints and solve the resulting problem using the Frank-Wolfe (FW) algorithm. Our method drastically reduces the per-layer pruning error, outperforms strong baselines on state-of-the-art GPT architectures, and remains memory-efficient. We provide theoretical justification by showing that, combined with the convergence guarantees of the FW algorithm, we obtain an approximate solution to the original combinatorial problem upon rounding the relaxed solution to integrality.

Method: Conducted a large-scale study (400,000+ GPU-hours) fitting sigmoidal compute-performance curves for RL training, ablating design choices like loss aggregation, normalization, curriculum, and off-policy algorithms.

Result: Found that (1) not all recipes yield similar asymptotic performance, (2) certain details modulate compute efficiency without shifting asymptote, and (3) stable recipes follow predictable scaling trajectories enabling extrapolation.

Conclusion: Proposed ScaleRL recipe demonstrated effectiveness by successfully scaling and predicting validation performance on a 100,000 GPU-hour run, providing both scientific framework and practical recipe for predictable RL training.

Abstract: Reinforcement learning (RL) has become central to training large language models (LLMs), yet the field lacks predictive scaling methodologies comparable to those established for pre-training. Despite rapidly rising compute budgets, there is no principled understanding of how to evaluate algorithmic improvements for scaling RL compute. We present the first large-scale systematic study, amounting to more than 400,000 GPU-hours, that defines a principled framework for analyzing and predicting RL scaling in LLMs. We fit sigmoidal compute-performance curves for RL training and ablate a wide range of common design choices to analyze their effects on asymptotic performance and compute efficiency. We observe: (1) Not all recipes yield similar asymptotic performance, (2) Details such as loss aggregation, normalization, curriculum, and off-policy algorithm primarily modulate compute efficiency without materially shifting the asymptote, and (3) Stable, scalable recipes follow predictable scaling trajectories, enabling extrapolation from smaller-scale runs. Combining these insights, we propose a best-practice recipe, ScaleRL, and demonstrate its effectiveness by successfully scaling and predicting validation performance on a single RL run scaled up to 100,000 GPU-hours. Our work provides both a scientific framework for analyzing scaling in RL and a practical recipe that brings RL training closer to the predictability long achieved in pre-training.

Method: Benchmarks recent SOTA deep learning models (e.g., CorrDiff) using physics-inspired diagnostics, with focus on geographic generalization and physical consistency. Proposes a power spectral density loss function to improve small-scale physical structure reconstruction.

Result: Models trained on limited European geographies struggle to generalize to other regions (Iberia, Morocco, Scandinavia) and fail to accurately capture second-order variables like divergence and vorticity. Deficiencies occur even in in-distribution geographies.

Conclusion: Current deep learning models for climate downscaling have significant limitations in geographic generalization and physical consistency. The proposed power spectral density loss empirically improves generalization by better reconstructing small-scale physical structures.

Abstract: Kilometer-scale weather data is crucial for real-world applications but remains computationally intensive to produce using traditional weather simulations. An emerging solution is to use deep learning models, which offer a faster alternative for climate downscaling. However, their reliability is still in question, as they are often evaluated using standard machine learning metrics rather than insights from atmospheric and weather physics. This paper benchmarks recent state-of-the-art deep learning models and introduces physics-inspired diagnostics to evaluate their performance and reliability, with a particular focus on geographic generalization and physical consistency. Our experiments show that, despite the seemingly strong performance of models such as CorrDiff, when trained on a limited set of European geographies (e.g., central Europe), they struggle to generalize to other regions such as Iberia, Morocco in the south, or Scandinavia in the north. They also fail to accurately capture second-order variables such as divergence and vorticity derived from predicted velocity fields. These deficiencies appear even in in-distribution geographies, indicating challenges in producing physically consistent predictions. We propose a simple initial solution: introducing a power spectral density loss function that empirically improves geographic generalization by encouraging the reconstruction of small-scale physical structures. The code for reproducing the experimental results can be found at https://github.com/CarloSaccardi/PSD-Downscaling

Method: Uses rate-distortion information theory to randomly distort agents’ observations of transition kernels and other properties, limiting information gain about ground-truth during training.

Result: Derives information-theoretic lower bound on agent’s reward regret and demonstrates impact on both model-based and model-free RL algorithms; extends approach to state observation attacks.

Conclusion: The proposed information-theoretic attack framework provides provably invincible adversarial strategies that cannot be countered by deterministic defense mechanisms, highlighting fundamental limitations in RL system security.

Abstract: Reinforcement learning (RL) for the Markov Decision Process (MDP) has emerged in many security-related applications, such as autonomous driving, financial decisions, and drone/robot algorithms. In order to improve the robustness/defense of RL systems against adversaries, studying various adversarial attacks on RL systems is very important. Most previous work considered deterministic adversarial attack strategies in MDP, which the recipient (victim) agent can defeat by reversing the deterministic attacks. In this paper, we propose a provably invincible'' or uncounterable’’ type of adversarial attack on RL. The attackers apply a rate-distortion information-theoretic approach to randomly change agents’ observations of the transition kernel (or other properties) so that the agent gains zero or very limited information about the ground-truth kernel (or other properties) during the training. We derive an information-theoretic lower bound on the recipient agent’s reward regret and show the impact of rate-distortion attacks on state-of-the-art model-based and model-free algorithms. We also extend this notion of an information-theoretic approach to other types of adversarial attack, such as state observation attacks.

Method: Two-phase algorithm: 1) Learn latent structure via random exploration and clustering, 2) Switch to optimism-guided strategy adapted to uncovered structure.

Result: Achieves O(√T+n) regret on BMDPs susceptible to clustering, improving prior O(√T+n²) bound. Proves no algorithm can achieve lower regret uniformly on this class.

Conclusion: The algorithm achieves asymptotic optimality on this class of BMDPs, demonstrating that accurate latent state estimation effectively speeds up learning.

Abstract: The curse of dimensionality renders Reinforcement Learning (RL) impractical in many real-world settings with exponentially large state and action spaces. Yet, many environments exhibit exploitable structure that can accelerate learning. To formalize this idea, we study RL in Block Markov Decision Processes (BMDPs). BMDPs model problems with large observation spaces, but where transition dynamics are fully determined by latent states. Recent advances in clustering methods have enabled the efficient recovery of this latent structure. However, a regret analysis that exploits these techniques to determine their impact on learning performance remained open. We are now addressing this gap by providing a regret analysis that explicitly leverages clustering, demonstrating that accurate latent state estimation can indeed effectively speed up learning. Concretely, this paper analyzes a two-phase RL algorithm for BMDPs that first learns the latent structure through random exploration and then switches to an optimism-guided strategy adapted to the uncovered structure. This algorithm achieves a regret that is $O(\sqrt{T}+n)$ on a large class of BMDPs susceptible to clustering. Here, $T$ denotes the number of time steps, $n$ is the cardinality of the observation space, and the Landau notation $O(\cdot)$ holds up to constants and polylogarithmic factors. This improves the best prior bound, $O(\sqrt{T}+n^2)$, especially when $n$ is large. Moreover, we prove that no algorithm can achieve lower regret uniformly on this same class of BMDPs. This establishes that, on this class, the algorithm achieves asymptotic optimality.

Method: Uses tailored encoders for different data types, performs multi-fidelity regression with neural networks, and employs dual connections: concatenations among encoded inputs and additive connections among final outputs to enable progressive information flow from lower to higher fidelity levels.

Result: The model reliably integrates multi-modal data, provides accurate predictions, and maintains performance when generalizing across time and parameter variations, as demonstrated on numerical benchmarks and a real-world case study.

Conclusion: The progressive multi-fidelity approach effectively addresses challenges of limited high-fidelity data by leveraging diverse data sources while preventing performance degradation when integrating new inputs, making it suitable for practical applications requiring precise evaluations across multiple scenarios.

Abstract: Highly accurate datasets from numerical or physical experiments are often expensive and time-consuming to acquire, posing a significant challenge for applications that require precise evaluations, potentially across multiple scenarios and in real-time. Even building sufficiently accurate surrogate models can be extremely challenging with limited high-fidelity data. Conversely, less expensive, low-fidelity data can be computed more easily and encompass a broader range of scenarios. By leveraging multi-fidelity information, prediction capabilities of surrogates can be improved. However, in practical situations, data may be different in types, come from sources of different modalities, and not be concurrently available, further complicating the modeling process. To address these challenges, we introduce a progressive multi-fidelity surrogate model. This model can sequentially incorporate diverse data types using tailored encoders. Multi-fidelity regression from the encoded inputs to the target quantities of interest is then performed using neural networks. Input information progressively flows from lower to higher fidelity levels through two sets of connections: concatenations among all the encoded inputs, and additive connections among the final outputs. This dual connection system enables the model to exploit correlations among different datasets while ensuring that each level makes an additive correction to the previous level without altering it. This approach prevents performance degradation as new input data are integrated into the model and automatically adapts predictions based on the available inputs. We demonstrate the effectiveness of the approach on numerical benchmarks and a real-world case study, showing that it reliably integrates multi-modal data and provides accurate predictions, maintaining performance when generalizing across time and parameter variations.

Method: Models factor functions along each input dimension using 1D GPs, combines via tensor decomposition. Uses partial freezing strategy and Newton’s method for nonlinear PDEs, develops ALS approach with closed-form updates for efficient training.

Result: Achieves superior accuracy and efficiency compared to existing approaches on several benchmark PDEs, with theoretical guarantees on expressivity, convergence, and error analysis.

Conclusion: TGPS provides an efficient, scalable alternative to existing PDE solvers by leveraging tensor decomposition of 1D GPs, with proven theoretical guarantees and demonstrated practical superiority.

Abstract: Machine learning solvers for partial differential equations (PDEs) have attracted growing interest. However, most existing approaches, such as neural network solvers, rely on stochastic training, which is inefficient and typically requires a great many training epochs. Gaussian process (GP)/kernel-based solvers, while mathematical principled, suffer from scalability issues when handling large numbers of collocation points often needed for challenging or higher-dimensional PDEs. To overcome these limitations, we propose TGPS, a tensor-GP-based solver that models factor functions along each input dimension using one-dimensional GPs and combines them via tensor decomposition to approximate the full solution. This design reduces the task to learning a collection of one-dimensional GPs, substantially lowering computational complexity, and enabling scalability to massive collocation sets. For efficient nonlinear PDE solving, we use a partial freezing strategy and Newton’s method to linerize the nonlinear terms. We then develop an alternating least squares (ALS) approach that admits closed-form updates, thereby substantially enhancing the training efficiency. We establish theoretical guarantees on the expressivity of our model, together with convergence proof and error analysis under standard regularity assumptions. Experiments on several benchmark PDEs demonstrate that our method achieves superior accuracy and efficiency compared to existing approaches.

Method: Uses modality-specific encoders for different input types and a Transformer-based fusion module to learn unified representations through Stochastic Multimodal Fusion.

Result: Outperforms prior foundation models on location-encoding, supports multimodal input during inference, and generalizes well to unseen regions across 41 tasks in 56 cities.

Conclusion: UrbanFusion provides a flexible framework that can utilize any subset of available modalities, enabling broad applicability across diverse data availability scenarios.

Abstract: Forecasting urban phenomena such as housing prices and public health indicators requires the effective integration of various geospatial data. Current methods primarily utilize task-specific models, while recent foundation models for spatial representations often support only limited modalities and lack multimodal fusion capabilities. To overcome these challenges, we present UrbanFusion, a Geo-Foundation Model (GeoFM) that features Stochastic Multimodal Fusion (SMF). The framework employs modality-specific encoders to process different types of inputs, including street view imagery, remote sensing data, cartographic maps, and points of interest (POIs) data. These multimodal inputs are integrated via a Transformer-based fusion module that learns unified representations. An extensive evaluation across 41 tasks in 56 cities worldwide demonstrates UrbanFusion’s strong generalization and predictive performance compared to state-of-the-art GeoAI models. Specifically, it 1) outperforms prior foundation models on location-encoding, 2) allows multimodal input during inference, and 3) generalizes well to regions unseen during training. UrbanFusion can flexibly utilize any subset of available modalities for a given location during both pretraining and inference, enabling broad applicability across diverse data availability scenarios. All source code is available at https://github.com/DominikM198/UrbanFusion.

Method: T3former introduces sliding-window topological and spectral descriptors as first-class tokens, integrated via a specialized Descriptor-Attention mechanism. This preserves temporal fidelity, enhances robustness, and enables principled cross-modal fusion without rigid discretization.

Result: T3former achieves state-of-the-art performance across multiple benchmarks including dynamic social networks, brain functional connectivity datasets, and traffic networks. It also provides theoretical guarantees of stability under temporal and structural perturbations.

Conclusion: The results demonstrate the power of combining topological and spectral insights for advancing temporal graph learning, offering a robust and effective solution for temporal graph classification tasks.

Abstract: Temporal graph classification plays a critical role in applications such as cybersecurity, brain connectivity analysis, social dynamics, and traffic monitoring. Despite its significance, this problem remains underexplored compared to temporal link prediction or node forecasting. Existing methods often rely on snapshot-based or recurrent architectures that either lose fine-grained temporal information or struggle with long-range dependencies. Moreover, local message-passing approaches suffer from oversmoothing and oversquashing, limiting their ability to capture complex temporal structures. We introduce T3former, a novel Topological Temporal Transformer that leverages sliding-window topological and spectral descriptors as first-class tokens, integrated via a specialized Descriptor-Attention mechanism. This design preserves temporal fidelity, enhances robustness, and enables principled cross-modal fusion without rigid discretization. T3former achieves state-of-the-art performance across multiple benchmarks, including dynamic social networks, brain functional connectivity datasets, and traffic networks. It also offers theoretical guarantees of stability under temporal and structural perturbations. Our results highlight the power of combining topological and spectral insights for advancing the frontier of temporal graph learning.

Method: Proposes Dynamics-Aware Hybrid Offline-and-Online RL (H2O) with a dynamics-aware policy evaluation scheme that adaptively penalizes Q function learning on simulated state-action pairs with large dynamics gaps while learning from fixed real-world dataset.

Result: Demonstrates superior performance against other cross-domain online and offline RL algorithms through extensive simulation, real-world tasks, and theoretical analysis.

Conclusion: H2O provides a new hybrid RL paradigm that can potentially guide future RL algorithm design for solving practical real-world tasks.

Abstract: Learning effective reinforcement learning (RL) policies to solve real-world complex tasks can be quite challenging without a high-fidelity simulation environment. In most cases, we are only given imperfect simulators with simplified dynamics, which inevitably lead to severe sim-to-real gaps in RL policy learning. The recently emerged field of offline RL provides another possibility to learn policies directly from pre-collected historical data. However, to achieve reasonable performance, existing offline RL algorithms need impractically large offline data with sufficient state-action space coverage for training. This brings up a new question: is it possible to combine learning from limited real data in offline RL and unrestricted exploration through imperfect simulators in online RL to address the drawbacks of both approaches? In this study, we propose the Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learning (H2O) framework to provide an affirmative answer to this question. H2O introduces a dynamics-aware policy evaluation scheme, which adaptively penalizes the Q function learning on simulated state-action pairs with large dynamics gaps, while also simultaneously allowing learning from a fixed real-world dataset. Through extensive simulation and real-world tasks, as well as theoretical analysis, we demonstrate the superior performance of H2O against other cross-domain online and offline RL algorithms. H2O provides a brand new hybrid offline-and-online RL paradigm, which can potentially shed light on future RL algorithm design for solving practical real-world tasks.

Method: Empirical study of LLM behaviors in online learning problems and repeated games, theoretical analysis under certain assumptions, and proposal of a novel unsupervised training loss called regret-loss that doesn’t require optimal action labels.

Result: Identified cases where advanced LLMs like GPT-4 fail to be no-regret, and demonstrated the effectiveness of the proposed regret-loss in addressing these regrettable cases through experiments.

Conclusion: The proposed regret-loss provides an effective approach to promote no-regret behaviors in LLM agents, with both statistical and optimization guarantees, addressing limitations observed in current LLM-based decision-making systems.

Abstract: Large language models (LLMs) have been increasingly employed for (interactive) decision-making, via the development of LLM-based autonomous agents. Despite their emerging successes, the performance of LLM agents in decision-making has not been fully investigated through quantitative metrics, especially in the multi-agent setting when they interact with each other, a typical scenario in real-world LLM-agent applications. To better understand the limits of LLM agents in these interactive environments, we propose to study their interactions in benchmark decision-making settings in online learning and game theory, through the performance metric of \emph{regret}. We first empirically study the {no-regret} behaviors of LLMs in canonical (non-stationary) online learning problems, as well as the emergence of equilibria when LLM agents interact through playing repeated games. We then provide some theoretical insights into the no-regret behaviors of LLM agents, under certain assumptions on the supervised pre-training and the rationality model of human decision-makers who generate the data. Notably, we also identify (simple) cases where advanced LLMs such as GPT-4 fail to be no-regret. To promote the no-regret behaviors, we propose a novel \emph{unsupervised} training loss of \emph{regret-loss}, which, in contrast to the supervised pre-training loss, does not require the labels of (optimal) actions. We then establish the statistical guarantee of generalization bound for regret-loss minimization, followed by the optimization guarantee that minimizing such a loss may automatically lead to known no-regret learning algorithms. Our further experiments demonstrate the effectiveness of our regret-loss, especially in addressing the above ``regrettable’’ cases.

Method: Proposes an enlightening taxonomy that encompasses data augmentation techniques for different common data modalities by investigating how to leverage intrinsic relationships between and within instances, categorizing methods across five data modalities through a unified inductive approach.

Result: A comprehensive survey framework that bridges the gap in current literature by providing a consistent summary of data augmentation methods across multiple modalities.

Conclusion: The proposed taxonomy offers a more comprehensive understanding of data augmentation techniques by focusing on intrinsic data relationships rather than modality-specific operations, enabling better comprehension of how data samples serve the augmentation process.

Abstract: Data augmentation is a series of techniques that generate high-quality artificial data by manipulating existing data samples. By leveraging data augmentation techniques, AI models can achieve significantly improved applicability in tasks involving scarce or imbalanced datasets, thereby substantially enhancing AI models’ generalization capabilities. Existing literature surveys only focus on a certain type of specific modality data and categorize these methods from modality-specific and operation-centric perspectives, which lacks a consistent summary of data augmentation methods across multiple modalities and limits the comprehension of how existing data samples serve the data augmentation process. To bridge this gap, this survey proposes a more enlightening taxonomy that encompasses data augmentation techniques for different common data modalities by investigating how to take advantage of the intrinsic relationship between and within instances. Additionally, it categorizes data augmentation methods across five data modalities through a unified inductive approach.

Method: Develops new learning objectives that integrate regularization effects from multiple previous posterior estimations, preventing individual errors from dominating future updates. Connects these to Temporal-Difference methods from Reinforcement Learning.

Result: Experiments on challenging continual learning benchmarks demonstrate effective mitigation of catastrophic forgetting and superior performance compared to strong Variational CL baselines.

Conclusion: The proposed approach successfully addresses compounding approximation errors in Bayesian continual learning, providing more robust and reliable adaptation to shifting data distributions while maintaining previous knowledge.

Abstract: Machine Learning models in real-world applications must continuously learn new tasks to adapt to shifts in the data-generating distribution. Yet, for Continual Learning (CL), models often struggle to balance learning new tasks (plasticity) with retaining previous knowledge (memory stability). Consequently, they are susceptible to Catastrophic Forgetting, which degrades performance and undermines the reliability of deployed systems. In the Bayesian CL literature, variational methods tackle this challenge by employing a learning objective that recursively updates the posterior distribution while constraining it to stay close to its previous estimate. Nonetheless, we argue that these methods may be ineffective due to compounding approximation errors over successive recursions. To mitigate this, we propose new learning objectives that integrate the regularization effects of multiple previous posterior estimations, preventing individual errors from dominating future posterior updates and compounding over time. We reveal insightful connections between these objectives and Temporal-Difference methods, a popular learning mechanism in Reinforcement Learning and Neuroscience. Experiments on challenging CL benchmarks show that our approach effectively mitigates Catastrophic Forgetting, outperforming strong Variational CL methods.

Method: The authors propose a new measure based on optimal set-matching accuracy that is normalized, monotonic, scale-invariant, and corrected for imbalanced cluster sizes.

Result: The proposed measure addresses the identified limitations of classical partition similarity scores by providing better worst-case scenario identification and improved interpretability.

Conclusion: The new normalized optimal set-matching accuracy measure enables more effective evaluation of clustering algorithms on diverse benchmark datasets by overcoming the shortcomings of traditional evaluation metrics.

Abstract: There is no, nor will there ever be, single best clustering algorithm. Nevertheless, we would still like to be able to distinguish between methods that work well on certain task types and those that systematically underperform. Clustering algorithms are traditionally evaluated using either internal or external validity measures. Internal measures quantify different aspects of the obtained partitions, e.g., the average degree of cluster compactness or point separability. However, their validity is questionable because the clusterings they endorse can sometimes be meaningless. External measures, on the other hand, compare the algorithms’ outputs to fixed ground truth groupings provided by experts. In this paper, we argue that the commonly used classical partition similarity scores, such as the normalised mutual information, Fowlkes-Mallows, or adjusted Rand index, miss some desirable properties. In particular, they do not identify worst-case scenarios correctly, nor are they easily interpretable. As a consequence, the evaluation of clustering algorithms on diverse benchmark datasets can be difficult. To remedy these issues, we propose and analyse a new measure: a version of the optimal set-matching accuracy, which is normalised, monotonic with respect to some similarity relation, scale-invariant, and corrected for the imbalancedness of cluster sizes (but neither symmetric nor adjusted for chance).

Method: Proposes Bayesian formulations that jointly diversify model parameters and inputs, with advanced posterior approximations for model input and principled fine-tuning of model parameters within the Bayesian framework.

Result: Achieves significant improvements in transferability, surpassing all state-of-the-art methods when attacking without model fine-tuning, with substantial gains on ImageNet and CIFAR-10 datasets.

Conclusion: The Bayesian approach to joint diversification of model parameters and inputs provides a powerful framework for enhancing adversarial transferability, establishing new state-of-the-art performance in transfer-based attacks.

Abstract: The transferability of adversarial examples allows for the attack on unknown deep neural networks (DNNs), posing a serious threat to many applications and attracting great attention. In this paper, we improve the transferability of adversarial examples by incorporating the Bayesian formulation into both the model parameters and model input, enabling their joint diversification. We demonstrate that combination of Bayesian formulations for both the model input and model parameters yields significant improvements in transferability. By introducing advanced approximations of the posterior distribution over the model input, adversarial transferability achieves further enhancement, surpassing all state-of-the-arts when attacking without model fine-tuning. Additionally, we propose a principled approach to fine-tune model parameters within this Bayesian framework. Extensive experiments demonstrate that our method achieves a new state-of-the-art in transfer-based attacks, significantly improving the average success rate on ImageNet and CIFAR-10. Code at: https://github.com/qizhangli/MoreBayesian-jrnl.

Method: Statistical language generation framework building on Gold and Angluin, analyzing model behavior as training size increases, and examining different language collections.

Result: Negative result: For most language collections, including those handled by next-token prediction models, simultaneous consistency and breadth is impossible. However, consistent generation without breadth is possible for any countable language collection.

Conclusion: Generation with breadth fundamentally differs from generation without breadth. The availability of negative examples (post-training feedback) enables achieving both consistency and breadth, suggesting this approach can reduce hallucinations while limiting mode collapse.

Abstract: Specifying all desirable properties of a language model is challenging, but certain requirements seem essential. Given samples from an unknown language, the trained model should produce valid strings not seen in training and be expressive enough to capture the language’s full richness. Otherwise, outputting invalid strings constitutes “hallucination,” and failing to capture the full range leads to “mode collapse.” We ask if a language model can meet both requirements. We investigate this within a statistical language generation setting building on Gold and Angluin. Here, the model receives random samples from a distribution over an unknown language K, which belongs to a possibly infinite collection of languages. The goal is to generate unseen strings from K. We say the model generates from K with consistency and breadth if, as training size increases, its output converges to all unseen strings in K. Kleinberg and Mullainathan [KM24] asked if consistency and breadth in language generation are possible. We answer this negatively: for a large class of language models, including next-token prediction models, this is impossible for most collections of candidate languages. This contrasts with [KM24]’s result, showing consistent generation without breadth is possible for any countable collection of languages. Our finding highlights that generation with breadth fundamentally differs from generation without breadth. As a byproduct, we establish near-tight bounds on the number of samples needed for generation with or without breadth. Finally, our results offer hope: consistent generation with breadth is achievable for any countable collection of languages when negative examples (strings outside K) are available alongside positive ones. This suggests that post-training feedback, which encodes negative examples, can be crucial in reducing hallucinations while limiting mode collapse.

Method: The paper conducts a systematic review and provides a detailed taxonomy of graph unlearning methodologies, along with clear explanations of fundamental concepts and evaluation measures for broader accessibility.

Result: The survey presents a comprehensive overview of graph unlearning techniques, their applications in various domains, and establishes a solid foundation for understanding this emerging field.

Conclusion: The paper aims to inspire further research in graph unlearning to advance responsible AI development, enhance data privacy protection, and strengthen the ethical application of machine learning techniques.

Abstract: Graph unlearning emerges as a crucial advancement in the pursuit of responsible AI, providing the means to remove sensitive data traces from trained models, thereby upholding the \textit{right to be forgotten}. It is evident that graph machine learning exhibits sensitivity to data privacy and adversarial attacks, necessitating the application of graph unlearning techniques to address these concerns effectively. In this comprehensive survey paper, we present the first systematic review of graph unlearning approaches, encompassing a diverse array of methodologies and offering a detailed taxonomy and up-to-date literature overview to facilitate the understanding of researchers new to this field. To ensure clarity, we provide lucid explanations of the fundamental concepts and evaluation measures used in graph unlearning, catering to a broader audience with varying levels of expertise. Delving into potential applications, we explore the versatility of graph unlearning across various domains, including but not limited to social networks, adversarial settings, recommender systems, and resource-constrained environments like the Internet of Things, illustrating its potential impact in safeguarding data privacy and enhancing AI systems’ robustness. Finally, we shed light on promising research directions, encouraging further progress and innovation within the domain of graph unlearning. By laying a solid foundation and fostering continued progress, this survey seeks to inspire researchers to further advance the field of graph unlearning, thereby instilling confidence in the ethical growth of AI systems and reinforcing the responsible application of machine learning techniques in various domains.

Method: Developed training method to produce NNs with hidden counterexamples that evade common adversarial attacks, systematically constructing benchmark across various model architectures, activation functions, and input data.

Result: Successfully identified bugs in state-of-the-art NN verifiers, demonstrating that the training effectively produces hidden counterexamples and the benchmark can detect false verification claims.

Conclusion: SoundnessBench provides a valuable tool for testing NN verifier soundness by including instances with known hidden counterexamples, helping identify verification errors in challenging cases.

Abstract: Neural network (NN) verification aims to formally verify properties of NNs, which is crucial for ensuring the behavior of NN-based models in safety-critical applications. In recent years, the community has developed many NN verifiers and benchmarks to evaluate them. However, existing benchmarks typically lack ground-truth for hard instances where no current verifier can verify the property and no counterexample can be found. This makes it difficult to validate the soundness of a verifier, when it claims verification on such challenging instances that no other verifier can handle. In this work, we develop a new benchmark for NN verification, named “SoundnessBench”, specifically for testing the soundness of NN verifiers. SoundnessBench consists of instances with deliberately inserted counterexamples that are hidden from adversarial attacks commonly used to find counterexamples. Thereby, it can identify false verification claims when hidden counterexamples are known to exist. We design a training method to produce NNs with hidden counterexamples and systematically construct our SoundnessBench with instances across various model architectures, activation functions, and input data. We demonstrate that our training effectively produces hidden counterexamples and our SoundnessBench successfully identifies bugs in state-of-the-art NN verifiers. Our code is available at https://github.com/MVP-Harry/SoundnessBench and our benchmark is available at https://huggingface.co/datasets/SoundnessBench/SoundnessBench.

Method: HOoD projects groups of correlated measurements through a trained model and uses permutation-based hypothesis tests to compare them with known subpopulations, generating interpretable p-values for each test.

Result: HOoD consistently outperforms point-wise and ensemble-based OoD detectors in evaluations, demonstrating superior performance in identifying OoD groups.

Conclusion: HOoD shows promise for robust real-world deployment by reliably detecting OoD groups in biomedical applications through its novel approach of leveraging correlated measurements and permutation tests.

Abstract: Deep neural networks (DNNs) have the potential to power many biomedical workflows, but training them on truly representative, IID datasets is often infeasible. Most models instead rely on biased or incomplete data, making them prone to out-of-distribution (OoD) inputs that closely resemble in-distribution samples. Such near-OoD cases are harder to detect than standard OOD benchmarks and can cause unreliable, even catastrophic, predictions. Biomedical assays, however, offer a unique opportunity: they often generate multiple correlated measurements per specimen through biological or technical replicates. Exploiting this insight, we introduce Homogeneous OoD (HOoD), a novel OoD detection framework for correlated data. HOoD projects groups of correlated measurements through a trained model and uses permutation-based hypothesis tests to compare them with known subpopulations. Each test yields an interpretable p-value, quantifying how well a group matches a subpopulation. By aggregating these p-values, HOoD reliably identifies OoD groups. In evaluations, HOoD consistently outperforms point-wise and ensemble-based OoD detectors, demonstrating its promise for robust real-world deployment.

Method: Extends CSI-BERT with two-stage training: unsupervised MLM pre-training followed by fine-tuning, introduces adaptive re-weighting layer for subcarrier representation, MLP-based temporal embedding to mitigate temporal information loss, and extends MLM to mask prediction model for CSI prediction.

Result: Achieves state-of-the-art performance across all tasks, generalizes effectively across varying sampling rates, and robustly handles discontinuous CSI sequences caused by packet loss where conventional methods fail.

Conclusion: CSI-BERT2 provides an effective unified framework that addresses key challenges in wireless sensing and communication through improved architecture and training methods, demonstrating strong generalization and robustness.

Abstract: Channel state information (CSI) is a fundamental component in both wireless communication and sensing systems, enabling critical functions such as radio resource optimization and environmental perception. In wireless sensing, data scarcity and packet loss hinder efficient model training, while in wireless communication, high-dimensional CSI matrices and short coherent times caused by high mobility present challenges in CSI estimation.To address these issues, we propose a unified framework named CSI-BERT2 for CSI prediction and classification tasks, built on CSI-BERT, which adapts BERT to capture the complex relationships among CSI sequences through a bidirectional self-attention mechanism. We introduce a two-stage training method that first uses a mask language model (MLM) to enable the model to learn general feature extraction from scarce datasets in an unsupervised manner, followed by fine-tuning for specific downstream tasks. Specifically, we extend MLM into a mask prediction model (MPM), which efficiently addresses the CSI prediction task. To further enhance the representation capacity of CSI data, we modify the structure of the original CSI-BERT. We introduce an adaptive re-weighting layer (ARL) to enhance subcarrier representation and a multi-layer perceptron (MLP)-based temporal embedding module to mitigate temporal information loss problem inherent in the original Transformer.Extensive experiments on both real-world collected and simulated datasets demonstrate that CSI-BERT2 achieves state-of-the-art performance across all tasks. Our results further show that CSI-BERT2 generalizes effectively across varying sampling rates and robustly handles discontinuous CSI sequences caused by packet loss-challenges that conventional methods fail to address. The dataset and code are publicly available at https://github.com/RS2002/CSI-BERT2 .

Method: Combines generalized Gauss-Newton Hessian approximation with Duncan-Guttman matrix identity to compute parameter updates by factorizing a mini-batch-sized matrix, with optional enhancements like line search, adaptive regularization, and momentum.

Result: EGN consistently exceeds or matches the generalization performance of well-tuned SGD, Adam, GAF, SQN, and SGN optimizers across various supervised and reinforcement learning tasks.

Conclusion: EGN provides an effective stochastic second-order optimization approach that scales well to large neural networks and demonstrates strong empirical performance across diverse machine learning applications.

Abstract: We present Exact Gauss-Newton (EGN), a stochastic second-order optimization algorithm that combines the generalized Gauss-Newton (GN) Hessian approximation with low-rank linear algebra to compute the descent direction. Leveraging the Duncan-Guttman matrix identity, the parameter update is obtained by factorizing a matrix which has the size of the mini-batch. This is particularly advantageous for large-scale machine learning problems where the dimension of the neural network parameter vector is several orders of magnitude larger than the batch size. Additionally, we show how improvements such as line search, adaptive regularization, and momentum can be seamlessly added to EGN to further accelerate the algorithm. Moreover, under mild assumptions, we prove that our algorithm converges in expectation to a stationary point of the objective. Finally, our numerical experiments demonstrate that EGN consistently exceeds, or at most matches the generalization performance of well-tuned SGD, Adam, GAF, SQN, and SGN optimizers across various supervised and reinforcement learning tasks.

Method: Implemented 10 environments as generative probabilistic models from real-world scientific domains. Used expected information gain (EIG) to evaluate experimental design and explanation-based evaluation plus prediction errors for model discovery.

Result: Current LLMs like GPT-4o struggle with both experimental design and model discovery. Augmenting LLM-based agents with explicit statistical models does not reliably improve performance.

Conclusion: BoxingGym provides a systematic benchmark for evaluating scientific reasoning in LLMs, revealing significant limitations in current models’ abilities for scientific discovery tasks.

Abstract: Understanding the world and explaining it with scientific theories is a central aspiration of artificial intelligence research. Proposing theories, designing experiments to test them, and then revising them based on data are fundamental to scientific discovery. Despite the significant promise of LLM-based scientific agents, no benchmarks systematically test LLM’s ability to propose scientific models, collect experimental data, and revise them in light of new data. We introduce BoxingGym, a benchmark with 10 environments for systematically evaluating both experimental design (e.g. collecting data to test a scientific theory) and model discovery (e.g. proposing and revising scientific theories). To enable tractable and quantitative evaluation, we implement each environment as a generative probabilistic model with which a scientific agent can run interactive experiments. These probabilistic models are drawn from various real-world scientific domains ranging from psychology to ecology. To quantitatively evaluate a scientific agent’s ability to collect informative experimental data, we compute the expected information gain (EIG), an information-theoretic quantity which measures how much an experiment reduces uncertainty about the parameters of a generative model. A good scientific theory is a concise and predictive explanation. Therefore, to quantitatively evaluate model discovery, we ask a scientific agent to explain their model and then assess whether this explanation enables another scientific agent to make reliable predictions about this environment. In addition to this explanation-based evaluation, we compute standard model evaluation metrics such as prediction errors. We find that current LLMs, such as GPT-4o, struggle with both experimental design and model discovery. We find that augmenting the LLM-based agent with an explicit statistical model does not reliably improve these results.

Method: Developed a new estimation method that only requires generating prediction questions and responses from the conditional generative model and evaluating response log probabilities.

Result: Empirically evaluated the method using large language models on synthetic regression and natural language in-context learning tasks.

Conclusion: The proposed method provides a practical way to estimate hallucination probabilities in in-context learning scenarios without requiring access to the true data generation process.

Abstract: This paper presents a method for estimating the hallucination rate for in-context learning (ICL) with generative AI. In ICL, a conditional generative model (CGM) is prompted with a dataset and a prediction question and asked to generate a response. One interpretation of ICL assumes that the CGM computes the posterior predictive of an unknown Bayesian model, which implicitly defines a joint distribution over observable datasets and latent mechanisms. This joint distribution factorizes into two components: the model prior over mechanisms and the model likelihood of datasets given a mechanism. With this perspective, we define a hallucination as a generated response to the prediction question with low model likelihood given the mechanism. We develop a new method that takes an ICL problem and estimates the probability that a CGM will generate a hallucination. Our method only requires generating prediction questions and responses from the CGM and evaluating its response log probability. We empirically evaluate our method using large language models for synthetic regression and natural language ICL tasks.

Method: Introduces a theoretical framework analyzing generalization scaling with value diversity and sample quantity in direct preference optimization. Analyzes reward margin trajectories during finite gradient steps to bound generalization error.

Result: The framework demonstrates challenges in effectively learning wide sets of concepts or values. Empirical validation on contemporary LLMs confirms the practical relevance of the theoretical insights.

Conclusion: Learning diverse values presents significant challenges in LLM alignment, and the theoretical framework provides important insights into how generalization scales with value diversity in preference optimization.

Abstract: Large language models (LLMs) have demonstrated remarkable capabilities but often struggle to align with human preferences, leading to harmful or undesirable outputs. Preference learning, which trains models to distinguish between preferred and non-preferred responses based on human feedback, has become a crucial component for ensuring that LLMs align with human values. An essential part of ensuring that LLMs are aligned for all people is accounting for a diverse set of values. This paper introduces a new theoretical framework to analyze how generalization scales with value diversity and sample quantity in models trained with direct preference optimization. Our framework rigorously assesses how well models generalize after a finite number of gradient steps, reflecting real-world LLM training practices. By analyzing the reward margin associated with each sample and its trajectory throughout training, we provide a bound on the generalization error that demonstrates the challenges of effectively learning a wide set of concepts or values. These insights are empirically validated on contemporary LLMs, underscoring the practical relevance of our theory.

Method: Theoretical analysis proving the existence of special training examples that distort SSMs’ implicit bias, combined with empirical demonstrations using SSMs trained independently and within non-linear neural networks.

Result: Found that while SSMs’ implicit bias generally leads to generalization, certain cleanly labeled training examples can completely disrupt this bias and cause generalization failure, demonstrating susceptibility to clean-label poisoning.

Conclusion: SSMs are vulnerable to clean-label poisoning attacks, highlighting the need for research into methods to overcome this susceptibility given SSMs’ increasing popularity.

Abstract: Neural networks are powered by an implicit bias: a tendency of gradient descent to fit training data in a way that generalizes to unseen data. A recent class of neural network models gaining increasing popularity is structured state space models (SSMs), regarded as an efficient alternative to transformers. Prior work argued that the implicit bias of SSMs leads to generalization in a setting where data is generated by a low dimensional teacher. In this paper, we revisit the latter setting, and formally establish a phenomenon entirely undetected by prior work on the implicit bias of SSMs. Namely, we prove that while implicit bias leads to generalization under many choices of training data, there exist special examples whose inclusion in training completely distorts the implicit bias, to a point where generalization fails. This failure occurs despite the special training examples being labeled by the teacher, i.e. having clean labels! We empirically demonstrate the phenomenon, with SSMs trained independently and as part of non-linear neural networks. In the area of adversarial machine learning, disrupting generalization with cleanly labeled training examples is known as clean-label poisoning. Given the proliferation of SSMs, we believe that delineating their susceptibility to clean-label poisoning, and developing methods for overcoming this susceptibility, are critical research directions to pursue.

Method: Proposed HEGNN, a high-degree version of EGNN that incorporates high-degree steerable vectors while maintaining efficiency through the scalarization trick used in EGNN.

Result: HEGNN aligns with theoretical analyses on symmetric structures and shows substantial improvements on N-body and MD17 datasets compared to EGNN.

Conclusion: Higher-degree representations are essential for expressivity in equivariant GNNs on symmetric structures, and HEGNN successfully combines efficiency with increased expressivity, opening new research possibilities.

Abstract: Equivariant Graph Neural Networks (GNNs) that incorporate E(3) symmetry have achieved significant success in various scientific applications. As one of the most successful models, EGNN leverages a simple scalarization technique to perform equivariant message passing over only Cartesian vectors (i.e., 1st-degree steerable vectors), enjoying greater efficiency and efficacy compared to equivariant GNNs using higher-degree steerable vectors. This success suggests that higher-degree representations might be unnecessary. In this paper, we disprove this hypothesis by exploring the expressivity of equivariant GNNs on symmetric structures, including $k$-fold rotations and regular polyhedra. We theoretically demonstrate that equivariant GNNs will always degenerate to a zero function if the degree of the output representations is fixed to 1 or other specific values. Based on this theoretical insight, we propose HEGNN, a high-degree version of EGNN to increase the expressivity by incorporating high-degree steerable vectors while maintaining EGNN’s efficiency through the scalarization trick. Our extensive experiments demonstrate that HEGNN not only aligns with our theoretical analyses on toy datasets consisting of symmetric structures, but also shows substantial improvements on more complicated datasets such as $N$-body and MD17. Our theoretical findings and empirical results potentially open up new possibilities for the research of equivariant GNNs.

Method: Trains a flat goal-conditioned policy by bootstrapping on subgoal-conditioned policies with advantage-weighted importance sampling, avoiding generative models over the goal space.

Result: Matches or surpasses state-of-the-art offline GCRL algorithms across locomotion and manipulation benchmarks, scaling to complex long-horizon tasks where prior approaches fail.

Conclusion: The proposed bootstrapping approach provides an effective alternative to hierarchical methods for scaling offline GCRL to high-dimensional, long-horizon tasks without the complexity of subgoal generation.

Abstract: Offline goal-conditioned reinforcement learning (GCRL) is a promising approach for pretraining generalist policies on large datasets of reward-free trajectories, akin to the self-supervised objectives used to train foundation models for computer vision and natural language processing. However, scaling GCRL to longer horizons remains challenging due to the combination of sparse rewards and discounting, which obscures the comparative advantages of primitive actions with respect to distant goals. Hierarchical RL methods achieve strong empirical results on long-horizon goal-reaching tasks, but their reliance on modular, timescale-specific policies and subgoal generation introduces significant additional complexity and hinders scaling to high-dimensional goal spaces. In this work, we introduce an algorithm to train a flat (non-hierarchical) goal-conditioned policy by bootstrapping on subgoal-conditioned policies with advantage-weighted importance sampling. Our approach eliminates the need for a generative model over the (sub)goal space, which we find is key for scaling to high-dimensional control in large state spaces. We further show that existing hierarchical and bootstrapping-based approaches correspond to specific design choices within our derivation. Across a comprehensive suite of state- and pixel-based locomotion and manipulation benchmarks, our method matches or surpasses state-of-the-art offline GCRL algorithms and scales to complex, long-horizon tasks where prior approaches fail. Project page: https://johnlyzhou.github.io/saw/

Method: Leverages prediction-powered inferences to essentially ‘shrink’ confidence intervals, making minimal assumptions about observational datasets and providing valid CIs.

Result: The method proves unbiasedness and validity of CIs, with numerical experiments confirming theoretical results and showing more precise uncertainty quantification compared to naive approaches.

Conclusion: The method is widely applicable in medical practice and can be extended to construct CIs from combinations of experimental and observational datasets.

Abstract: Constructing confidence intervals (CIs) for the average treatment effect (ATE) from patient records is crucial to assess the effectiveness and safety of drugs. However, patient records typically come from different hospitals, thus raising the question of how multiple observational datasets can be effectively combined for this purpose. In our paper, we propose a new method that estimates the ATE from multiple observational datasets and provides valid CIs. Our method makes little assumptions about the observational datasets and is thus widely applicable in medical practice. The key idea of our method is that we leverage prediction-powered inferences and thereby essentially `shrink’ the CIs so that we offer more precise uncertainty quantification as compared to na"ive approaches. We further prove the unbiasedness of our method and the validity of our CIs. We confirm our theoretical results through various numerical experiments. Finally, we provide an extension of our method for constructing CIs from combinations of experimental and observational datasets.

Method: NEUROLOGIC extracts logic rules over hidden predicates derived from neural representations at any chosen layer, without requiring costly layerwise extraction and rewriting. It supports richer logical constructs and can incorporate human prior knowledge to ground hidden predicates back to the input space.

Result: Validated on Transformer-based sentiment analysis, NEUROLOGIC demonstrates the ability to extract meaningful, interpretable logic rules and provide deeper insights, tasks where existing methods struggle to scale.

Conclusion: NEUROLOGIC enables broader architectural compatibility and improved scalability for rule-based explanation methods, particularly for complex architectures like Transformers, while producing richer, more interpretable logical rules.

Abstract: Rule-based explanation methods offer rigorous and globally interpretable insights into neural network behavior. However, existing approaches are mostly limited to small fully connected networks and depend on costly layerwise rule extraction and substitution processes. These limitations hinder their generalization to more complex architectures such as Transformers. Moreover, existing methods produce shallow, decision-tree-like rules that fail to capture rich, high-level abstractions in complex domains like computer vision and natural language processing. To address these challenges, we propose NEUROLOGIC, a novel framework that extracts interpretable logical rules directly from deep neural networks. Unlike previous methods, NEUROLOGIC can construct logic rules over hidden predicates derived from neural representations at any chosen layer, in contrast to costly layerwise extraction and rewriting. This flexibility enables broader architectural compatibility and improved scalability. Furthermore, NEUROLOGIC supports richer logical constructs and can incorporate human prior knowledge to ground hidden predicates back to the input space, enhancing interpretability. We validate NEUROLOGIC on Transformer-based sentiment analysis, demonstrating its ability to extract meaningful, interpretable logic rules and provide deeper insights-tasks where existing methods struggle to scale.

Method: Built on Qwen2.5-7B with large-scale RL using Group Relative Policy Optimization and rule-based rewards, with RL enhancements for domain-specific challenges.

Result: Achieves state-of-the-art accuracy on MIMIC-IV benchmark and generates physician-validated reasoning for DRG assignments, enhancing explainability.

Conclusion: RL performance scales linearly with log of SFT examples, showing RL effectiveness is constrained by domain knowledge in base model. For OOD tasks, sufficient knowledge infusion via SFT is crucial before RL.

Abstract: Diagnosis-Related Group (DRG) codes are essential for hospital reimbursement and operations but require labor-intensive assignment. Large Language Models (LLMs) struggle with DRG coding due to the out-of-distribution (OOD) nature of the task: pretraining corpora rarely contain private clinical or billing data. We introduce DRG-Sapphire, which uses large-scale reinforcement learning (RL) for automated DRG coding from clinical notes. Built on Qwen2.5-7B and trained with Group Relative Policy Optimization (GRPO) using rule-based rewards, DRG-Sapphire introduces a series of RL enhancements to address domain-specific challenges not seen in previous mathematical tasks. Our model achieves state-of-the-art accuracy on the MIMIC-IV benchmark and generates physician-validated reasoning for DRG assignments, significantly enhancing explainability. Our study further sheds light on broader challenges of applying RL to knowledge-intensive, OOD tasks. We observe that RL performance scales approximately linearly with the logarithm of the number of supervised fine-tuning (SFT) examples, suggesting that RL effectiveness is fundamentally constrained by the domain knowledge encoded in the base model. For OOD tasks like DRG coding, strong RL performance requires sufficient knowledge infusion prior to RL. Consequently, scaling SFT may be more effective and computationally efficient than scaling RL alone for such tasks.

Method: Analyzed transformers trained on tractable hidden Markov models, focusing on single-layer transformers to reveal how attention layers implement constrained Bayesian updates, with extensions to multi-layer architectures.

Result: Attention layers carry out algorithms with natural interpretations in probability simplex, creating representations with distinctive geometric structure that can be theoretically predicted including attention patterns, OV-vectors, and embedding vectors.

Conclusion: Transformers develop specific intermediate geometric structures due to architectural constraints that shape how they implement optimal future token predictions through constrained Bayesian belief updating.

Abstract: What computational structures emerge in transformers trained on next-token prediction? In this work, we provide evidence that transformers implement constrained Bayesian belief updating – a parallelized version of partial Bayesian inference shaped by architectural constraints. We integrate the model-agnostic theory of optimal prediction with mechanistic interpretability to analyze transformers trained on a tractable family of hidden Markov models that generate rich geometric patterns in neural activations. Our primary analysis focuses on single-layer transformers, revealing how the first attention layer implements these constrained updates, with extensions to multi-layer architectures demonstrating how subsequent layers refine these representations. We find that attention carries out an algorithm with a natural interpretation in the probability simplex, and create representations with distinctive geometric structure. We show how both the algorithmic behavior and the underlying geometry of these representations can be theoretically predicted in detail – including the attention pattern, OV-vectors, and embedding vectors – by modifying the equations for optimal future token predictions to account for the architectural constraints of attention. Our approach provides a principled lens on how architectural constraints shape the implementation of optimal prediction, revealing why transformers develop specific intermediate geometric structures.

Method: Uses an attention-based architecture extending Evoformer from AlphaFold, with Optimal Transport-inspired loss and differentiable node matching trained jointly with encoder/decoder.

Result: Enables highly general pre-training applicable to diverse downstream tasks including classification, regression, graph interpolation, editing, matching, and prediction.

Conclusion: GRALE provides a versatile graph representation learning framework that supports multiple graph sizes and complex tasks through its shared embedding space and joint training approach.

Abstract: Although graph-based learning has attracted a lot of attention, graph representation learning is still a challenging task whose resolution may impact key application fields such as chemistry or biology. To this end, we introduce GRALE, a novel graph autoencoder that encodes and decodes graphs of varying sizes into a shared embedding space. GRALE is trained using an Optimal Transport-inspired loss that compares the original and reconstructed graphs and leverages a differentiable node matching module, which is trained jointly with the encoder and decoder. The proposed attention-based architecture relies on Evoformer, the core component of AlphaFold, which we extend to support both graph encoding and decoding. We show, in numerical experiments on simulated and molecular data, that GRALE enables a highly general form of pre-training, applicable to a wide range of downstream tasks, from classification and regression to more complex tasks such as graph interpolation, editing, matching, and prediction.

Method: The paper proposes two theorems based on the intrinsic geometry of Transformer models: 1) key projections have larger spectral/Frobenius norms than value matrices, indicating higher information density, and 2) prioritizing key precision over values reduces quantization error for any given memory budget.

Result: Empirical evaluations show that key-favored allocations (e.g., 4-bit keys, 2-bit values) retain up to 98.3% accuracy compared to uniform allocations while conserving memory across various LLMs and benchmarks.

Conclusion: The work transforms bit allocation from ad hoc tuning into a theoretically grounded, geometry-driven design principle for efficient LLM inference.

Abstract: Large Language Models (LLMs) suffer inference-time memory bottlenecks dominated by the attention Key-Value (KV) cache, which scales with model size and context length. While KV-cache quantization alleviates this cost, bit allocation between keys and values is often tuned heuristically, lacking theoretical grounding and generalizability. This paper proposes two theorems that anchor mixed-precision KV quantization in the intrinsic geometry of Transformer models. First, key projections systematically have larger spectral and Frobenius norms than value matrices, implying higher information density along the key path. Second, for any given memory budget, prioritizing precision for keys over values strictly reduces quantization error and better preserves accuracy. Empirical evaluations across various prominent LLMs and benchmarks show that key-favored allocations (e.g., 4-bit keys, 2-bit values) retain up to 98.3% accuracy compared to uniform allocations (e.g., 4-bit for both), while conserving memory. These results transform bit allocation from ad hoc tuning into a theoretically grounded, geometry-driven design principle for efficient LLM inference. Source code is available at https://github.com/mohsenhariri/spectral-kv.

Method: Analyzed information flow in five diverse molecular encoders, performed empirical layer-wise evaluation across 22 property prediction tasks using both frozen embeddings from optimal intermediate layers and finetuning encoders truncated at intermediate depths.

Result: Using frozen embeddings from intermediate layers improved downstream performance by average 5.4% (up to 28.6%) vs final-layer. Finetuning truncated encoders achieved even greater average improvements of 8.5% (up to 40.8%), obtaining new state-of-the-art results on several benchmarks.

Conclusion: Exploring the full representational depth of molecular encoders is crucial for achieving substantial performance improvements and computational efficiency in computational chemistry tasks.

Abstract: Pretrained molecular encoders have become indispensable in computational chemistry for tasks such as property prediction and molecular generation. However, the standard practice of relying solely on final-layer embeddings for downstream tasks may discard valuable information. In this work, we first analyze the information flow in five diverse molecular encoders and find that intermediate layers retain more general-purpose features, whereas the final-layer specializes and compresses information. We then perform an empirical layer-wise evaluation across 22 property prediction tasks. We find that using frozen embeddings from optimal intermediate layers improves downstream performance by an average of 5.4%, up to 28.6%, compared to the final-layer. Furthermore, finetuning encoders truncated at intermediate depths achieves even greater average improvements of 8.5%, with increases as high as 40.8%, obtaining new state-of-the-art results on several benchmarks. These findings highlight the importance of exploring the full representational depth of molecular encoders to achieve substantial performance improvements and computational efficiency. The code will be made publicly available.

Method: Used synthetic length generalization tasks, analyzed performance distributions across random seeds, conducted case studies of inverse scaling, and validated findings on MMLU performance in language model populations.

Result: Found that different random seeds produce either linear or emergent scaling trends, and sharp breakthroughs in metrics are driven by continuous changes in their distribution across seeds. Even with declining probability of successful runs, average performance of successful runs increases monotonically.

Conclusion: Random variation across seeds must be considered when predicting model performance from scale, as breakthroughs are explained by distributional changes rather than true emergence or thresholding effects.

Abstract: Language models famously improve under a smooth scaling law, but some specific capabilities exhibit sudden breakthroughs in performance. While advocates of “emergence” view breakthroughs as unlocked capabilities, others attribute them to thresholding effects on noncontinuous metrics. We propose that breakthroughs are instead driven by continuous changes in the probability distribution of training outcomes when performance is bimodally distributed across random seeds. In synthetic length generalization tasks, we show that different random seeds can produce either highly linear or emergent scaling trends. We reveal that sharp breakthroughs in metrics are produced by underlying continuous changes in their distribution across seeds. In a case study of inverse scaling, we show that even as the probability of a successful run declines, the average performance of a successful run increases monotonically. We validate our distributional scaling framework on realistic settings by measuring MMLU performance in LM populations. Our observations hold true even under continuous loss metrics, confirming that random variation must be considered when predicting a model’s performance from its scale.

Method: The authors extend formal tools for checking compatibility of local policies with team tasks and incorporate high-level symbolic knowledge about temporal evolution of events in the environment to improve DMARL.

Result: The approach makes decentralized training with theoretical guarantees usable in more scenarios and significantly expedites the learning process in DMARL.

Conclusion: Symbolic knowledge can effectively address unique challenges in DMARL and improve both theoretical guarantees and practical learning efficiency.

Abstract: Reinforcement learning (RL) algorithms can find an optimal policy for a single agent to accomplish a particular task. However, many real-world problems require multiple agents to collaborate in order to achieve a common goal. For example, a robot executing a task in a warehouse may require the assistance of a drone to retrieve items from high shelves. In Decentralized Multi-Agent RL (DMARL), agents learn independently and then combine their policies at execution time, but often must satisfy constraints on compatibility of local policies to ensure that they can achieve the global task when combined. In this paper, we study how providing high-level symbolic knowledge to agents can help address unique challenges of this setting, such as privacy constraints, communication limitations, and performance concerns. In particular, we extend the formal tools used to check the compatibility of local policies with the team task, making decentralized training with theoretical guarantees usable in more scenarios. Furthermore, we empirically demonstrate that symbolic knowledge about the temporal evolution of events in the environment can significantly expedite the learning process in DMARL.

Method: Proposes T’yr-the-Pruner framework that constructs a supernet by applying local pruning across sparsity ratios, uses expectation error accumulation for supernet construction, and employs iterative prune-and-search with coarse-to-fine sparsity granularity.

Result: Achieves state-of-the-art structural pruning, maintaining 97% of the dense model’s performance while removing 50% of parameters from Llama-3.1-70B.

Conclusion: The proposed end-to-end search-based framework effectively addresses limitations of existing pruning methods and enables efficient compression of large language models with minimal performance loss.

Abstract: Structural pruning enhances hardware-agnostic inference efficiency for large language models (LLMs) yet often fails to maintain comparable performance. Local pruning performs efficient layer-by-layer compression but ignores global topology. Although global pruning aims to identify an optimal sparse model, intuitive methods typically adopt a two-stage paradigm that first evaluates substructure saliency and then applies global pruning, which ignores inter-structure dependencies and fails to achieve end-to-end optimization. To address these limitations, we propose T'yr-the-Pruner, an efficient end-to-end search-based global structural pruning framework. This framework constructs a supernet by repeatedly applying local pruning across a range of sparsity ratios to each layer in an LLM, with the core goal of determining the optimal sparsity distribution under a target overall sparsity ratio. Concretely, we introduce an effective local pruning and an expectation error accumulation approach to improve supernet construction. Furthermore, we employ an iterative prune-and-search strategy with coarse-to-fine sparsity granularity to ensure efficient search convergence. Experimental results show that T'yr-the-Pruner achieves state-of-the-art structural pruning, retaining 97% of the dense model’s performance while removing a challenging 50% of Llama-3.1-70B’s parameters. Code will be available at https://github.com/AMD-AGI/Tyr-the-Pruner.

Method: Proposes BYOL-γ, a self-predictive representation learning objective that approximates successor representations to encourage long-range temporal consistency.

Result: Achieves competitive empirical performance across challenging tasks requiring combinatorial generalization.

Conclusion: Successor representations can effectively improve temporal consistency and facilitate zero-shot generalization in goal-conditioned behavior cloning.

Abstract: While goal-conditioned behavior cloning (GCBC) methods can perform well on in-distribution training tasks, they do not necessarily generalize zero-shot to tasks that require conditioning on novel state-goal pairs, i.e. combinatorial generalization. In part, this limitation can be attributed to a lack of temporal consistency in the state representation learned by BC; if temporally correlated states are properly encoded to similar latent representations, then the out-of-distribution gap for novel state-goal pairs would be reduced. We formalize this notion by demonstrating how encouraging long-range temporal consistency via successor representations (SR) can facilitate generalization. We then propose a simple yet effective representation learning objective, $\text{BYOL-}\gamma$ for GCBC, which theoretically approximates the successor representation in the finite MDP case through self-predictive representations, and achieves competitive empirical performance across a suite of challenging tasks requiring combinatorial generalization.

Method: Developed a framework based on probability mass functions and summary statistics for uncertainty evaluation of nominal properties in ML classification.

Result: Created a conceptual framework that enables uncertainty evaluation for ML classification models and could extend GUM to cover nominal properties.

Conclusion: The proposed framework bridges the gap in metrology standards and makes uncertainty evaluation applicable to ML classification models in critical domains.

Abstract: Machine learning (ML) classification models are increasingly being used in a wide range of applications where it is important that predictions are accompanied by uncertainties, including in climate and earth observation, medical diagnosis and bioaerosol monitoring. The output of an ML classification model is a type of categorical variable known as a nominal property in the International Vocabulary of Metrology (VIM). However, concepts related to uncertainty evaluation for nominal properties are not defined in the VIM, nor is such evaluation addressed by the Guide to the Expression of Uncertainty in Measurement (GUM). In this paper we propose a metrological conceptual uncertainty evaluation framework for nominal properties. This framework is based on probability mass functions and summary statistics thereof, and it is applicable to ML classification. We also illustrate its use in the context of two applications that exemplify the issues and have significant societal impact, namely, climate and earth observation and medical diagnosis. Our framework would enable an extension of the GUM to uncertainty for nominal properties, which would make both applicable to ML classification models.

Method: Created Time-IMM dataset with 9 types of time series irregularity categorized into trigger-based, constraint-based, and artifact-based mechanisms. Developed IMM-TSF benchmark library with specialized fusion modules including timestamp-to-text fusion and multimodality fusion supporting recency-aware averaging and attention-based integration.

Result: Empirical results demonstrate that explicitly modeling multimodality on irregular time series data leads to substantial gains in forecasting performance.

Conclusion: Time-IMM and IMM-TSF provide a foundation for advancing time series analysis under real-world conditions, bridging the gap between research and practical deployment.

Abstract: Time series data in real-world applications such as healthcare, climate modeling, and finance are often irregular, multimodal, and messy, with varying sampling rates, asynchronous modalities, and pervasive missingness. However, existing benchmarks typically assume clean, regularly sampled, unimodal data, creating a significant gap between research and real-world deployment. We introduce Time-IMM, a dataset specifically designed to capture cause-driven irregularity in multimodal multivariate time series. Time-IMM represents nine distinct types of time series irregularity, categorized into trigger-based, constraint-based, and artifact-based mechanisms. Complementing the dataset, we introduce IMM-TSF, a benchmark library for forecasting on irregular multimodal time series, enabling asynchronous integration and realistic evaluation. IMM-TSF includes specialized fusion modules, including a timestamp-to-text fusion module and a multimodality fusion module, which support both recency-aware averaging and attention-based integration strategies. Empirical results demonstrate that explicitly modeling multimodality on irregular time series data leads to substantial gains in forecasting performance. Time-IMM and IMM-TSF provide a foundation for advancing time series analysis under real-world conditions. The dataset is publicly available at https://github.com/blacksnail789521/Time-IMM, and the benchmark library can be accessed at https://github.com/blacksnail789521/IMM-TSF. Project page: https://blacksnail789521.github.io/time-imm-project-page/

Method: Formulates QoS metric prediction as probabilistic forecasting using diffusion models, incorporating latent temporal dynamics of non-stationary and multi-mode data.

Result: Extensive experiments show the proposed approach outperforms popular probabilistic time series forecasting methods.

Conclusion: The diffusion model-based approach effectively handles the complexity of QoS metrics in DTNs and provides uncertainty quantification for better operational performance.

Abstract: Active QoS metric prediction, commonly employed in the maintenance and operation of DTN, could enhance network performance regarding latency, throughput, energy consumption, and dependability. Naturally formulated as a multivariate time series forecasting problem, it attracts substantial research efforts. Traditional mean regression methods for time series forecasting cannot capture the data complexity adequately, resulting in deteriorated performance in operational tasks in DTNs such as routing. This paper formulates the prediction of QoS metrics in DTN as a probabilistic forecasting problem on multivariate time series, where one could quantify the uncertainty of forecasts by characterizing the distribution of these samples. The proposed approach hires diffusion models and incorporates the latent temporal dynamics of non-stationary and multi-mode data into them. Extensive experiments demonstrate the efficacy of the proposed approach by showing that it outperforms the popular probabilistic time series forecasting methods.

Method: Two-stage framework: 1) Brain representation learning using GPT-4 knowledge to enrich brain graphs with adaptive node reassignment graph attention network; 2) Population disorder diagnosis incorporating phenotypic data for population graph construction and feature fusion.

Result: Outperforms state-of-the-art methods on ABIDE I, ADHD-200, and Rest-meta-MDD datasets in prediction accuracy while enhancing interpretability.

Conclusion: B2P-GL provides a reliable and personalized approach to brain disorder diagnosis that advances clinical applicability by addressing key limitations in current methods.

Abstract: Recent developed graph-based methods for diagnosing brain disorders using functional connectivity highly rely on predefined brain atlases, but overlook the rich information embedded within atlases and the confounding effects of site and phenotype variability. To address these challenges, we propose a two-stage Brain-to-Population Graph Learning (B2P-GL) framework that integrates the semantic similarity of brain regions and condition-based population graph modeling. In the first stage, termed brain representation learning, we leverage brain atlas knowledge from GPT-4 to enrich the graph representation and refine the brain graph through an adaptive node reassignment graph attention network. In the second stage, termed population disorder diagnosis, phenotypic data is incorporated into population graph construction and feature fusion to mitigate confounding effects and enhance diagnosis performance. Experiments on the ABIDE I, ADHD-200, and Rest-meta-MDD datasets show that B2P-GL outperforms state-of-the-art methods in prediction accuracy while enhancing interpretability. Overall, our proposed framework offers a reliable and personalized approach to brain disorder diagnosis, advancing clinical applicability.

Method: Integration of graph neural networks with task-adaptive message passing and large language model-enhanced meta-heuristic algorithms for HLS design space exploration.

Result: ECoGNN reduced prediction error by 57.27% in post-HLS prediction and achieved average reductions of 17.6% for FF usage, 33.7% for CP delay, 26.3% for power, 38.3% for DSP utilization, and 40.8% for BRAM usage in post-implementation prediction. LLMMH variants improved ADRS by 87.47% and outperformed GNN-DSE and IRONMAN-PRO by 68.17% and 63.07% respectively.

Conclusion: The proposed ECoGNNs-LLMMHs framework effectively addresses limitations of existing methods, demonstrating superior performance in HLS design space exploration through improved prediction accuracy and optimization capabilities.

Abstract: High-Level Synthesis (HLS) Design Space Exploration (DSE) is essential for generating hardware designs that balance performance, power, and area (PPA). To optimize this process, existing works often employs message-passing neural networks (MPNNs) to predict quality of results (QoR). These predictors serve as evaluators in the DSE process, effectively bypassing the time-consuming estimations traditionally required by HLS tools. However, existing models based on MPNNs struggle with over-smoothing and limited expressiveness. Additionally, while meta-heuristic algorithms are widely used in DSE, they typically require extensive domain-specific knowledge to design operators and time-consuming tuning. To address these limitations, we propose ECoGNNs-LLMMHs, a framework that integrates graph neural networks with task-adaptive message passing and large language model-enhanced meta-heuristic algorithms. Compared with state-of-the-art works, ECoGNN exhibits lower prediction error in the post-HLS prediction task, with the error reduced by 57.27%. For post-implementation prediction tasks, ECoGNN demonstrates the lowest prediction errors, with average reductions of 17.6% for flip-flop (FF) usage, 33.7% for critical path (CP) delay, 26.3% for power consumption, 38.3% for digital signal processor (DSP) utilization, and 40.8% for BRAM usage. LLMMH variants can generate superior Pareto fronts compared to meta-heuristic algorithms in terms of average distance from the reference set (ADRS) with average improvements of 87.47%, respectively. Compared with the SOTA DSE approaches GNN-DSE and IRONMAN-PRO, LLMMH can reduce the ADRS by 68.17% and 63.07% respectively.

Method: Proposed LazyEviction framework uses observation window-based lagged eviction to retain latent recurring tokens through prioritized eviction based on tokens’ recurrence patterns.

Result: Extensive experiments show LazyEviction reduces KV cache by 50-70% while maintaining comparable accuracy, outperforming existing KV cache compression baselines.

Conclusion: LazyEviction effectively addresses the Token Importance Recurrence phenomenon in LLM reasoning tasks, providing superior KV cache compression without sacrificing performance.

Abstract: Large Language Models (LLMs) exhibit enhanced capabilities by Chain-of-Thought reasoning. However, the extended reasoning sequences introduce significant GPU memory overhead due to increased key-value (KV) cache. Existing KV cache compression methods mitigate memory bottlenecks but struggle in long reasoning tasks. In this paper, we analyze attention patterns in reasoning tasks and reveal a Token Importance Recurrence phenomenon: a large proportion of tokens regain high attention after multiple decoding steps, which is failed to capture by existing works and may lead to unpredictable eviction on such periodically critical tokens. To address this, we propose LazyEviction, an observation window-based lagged eviction framework retaining latent recurring tokens by prioritized eviction based on tokens’ recurrence patterns. Extensive experiments demonstrate that LazyEviction reduces KV cache by 50%~70% while maintaining comparable accuracy, outperforming existing KV cache compression baselines. Our implementation code can be found at https://github.com/Halo-949/LazyEviction.

Method: Proposes OFTv2 with input-centric reformulation using matrix-vector multiplications (matrix-free computation) and introduces Cayley-Neumann parameterization that approximates matrix inversion via truncated Neumann series for efficient orthogonal parameterization.

Result: Achieves up to 10x faster training and 3x lower GPU memory usage without performance compromise. Also extends to quantized foundation models, outperforming QLoRA in training stability, efficiency, and memory usage.

Conclusion: OFTv2 successfully addresses the computational bottlenecks of OFT through algorithmic innovations, making orthogonal finetuning more practical for real-world deployment while maintaining its benefits for parameter-efficient adaptation.

Abstract: Orthogonal finetuning (OFT) offers highly parameter-efficient adaptation while preventing catastrophic forgetting, but its high runtime and memory demands limit practical deployment. We identify the core computational bottleneck in OFT as its weight-centric implementation, which relies on costly matrix-matrix multiplications with cubic complexity. To overcome this, we propose OFTv2, an input-centric reformulation that instead uses matrix-vector multiplications (i.e., matrix-free computation), reducing the computational cost to quadratic. We further introduce the Cayley-Neumann parameterization, an efficient orthogonal parameterization that approximates the matrix inversion in the Cayley transform via a truncated Neumann series. These modifications allow OFTv2 to achieve up to 10x faster training and 3x lower GPU memory usage without compromising performance. In addition, we extend OFTv2 to support finetuning quantized foundation models and show that it outperforms the popular QLoRA in training stability, efficiency, and memory usage.

Method: Conducted a systematic review synthesizing findings from 69 studies to identify common obstacles and highlight recent methodological advances in the field.

Result: Identified key challenges including missing modalities, limited sample sizes, dimensionality imbalance, interpretability issues, and optimal fusion techniques. Found promising solutions through transfer learning, generative models, attention mechanisms, and neural architecture search.

Conclusion: Provides a comprehensive overview of current trends and innovations in multimodal modeling for medical applications, offering practical insights to guide future research and development in this field.

Abstract: Multimodal data modeling has emerged as a powerful approach in clinical research, enabling the integration of diverse data types such as imaging, genomics, wearable sensors, and electronic health records. Despite its potential to improve diagnostic accuracy and support personalized care, modeling such heterogeneous data presents significant technical challenges. This systematic review synthesizes findings from 69 studies to identify common obstacles, including missing modalities, limited sample sizes, dimensionality imbalance, interpretability issues, and finding the optimal fusion techniques. We highlight recent methodological advances, such as transfer learning, generative models, attention mechanisms, and neural architecture search that offer promising solutions. By mapping current trends and innovations, this review provides a comprehensive overview of the field and offers practical insights to guide future research and development in multimodal modeling for medical applications.

Method: The system leverages knowledge graphs as external structured knowledge to guide search by modeling entity relationships, and employs multi-reward RL framework for fine-grained control over retrieval accuracy, efficiency, and response quality objectives.

Result: Achieves state-of-the-art answer accuracy on six multi-hop question answering datasets, matching frontier LLMs while using only small-scale models and limited computational resources. Shows strong generalization and robustness across diverse retrieval environments.

Conclusion: DynaSearcher effectively addresses factual consistency and efficiency issues in multi-step retrieval systems through structured knowledge guidance and multi-objective reinforcement learning, demonstrating broad applicability and resource efficiency.

Abstract: Multi-step agentic retrieval systems based on large language models (LLMs) have demonstrated remarkable performance in complex information search tasks. However, these systems still face significant challenges in practical applications, particularly in generating factually inconsistent intermediate queries and inefficient search trajectories, which can lead to reasoning deviations or redundant computations. To address these issues, we propose DynaSearcher, an innovative search agent enhanced by dynamic knowledge graphs and multi-reward reinforcement learning (RL). Specifically, our system leverages knowledge graphs as external structured knowledge to guide the search process by explicitly modeling entity relationships, thereby ensuring factual consistency in intermediate queries and mitigating biases from irrelevant information. Furthermore, we employ a multi-reward RL framework for fine-grained control over training objectives such as retrieval accuracy, efficiency, and response quality. This framework promotes the generation of high-quality intermediate queries and comprehensive final answers, while discouraging unnecessary exploration and minimizing information omissions or redundancy. Experimental results demonstrate that our approach achieves state-of-the-art answer accuracy on six multi-hop question answering datasets, matching frontier LLMs while using only small-scale models and limited computational resources. Furthermore, our approach demonstrates strong generalization and robustness across diverse retrieval environments and larger-scale models, highlighting its broad applicability.

Method: L2T treats query-response as hierarchical sessions with episodes, uses a universal dense process reward based on information gain, estimates reward via PAC-Bayes bounds and Fisher information matrix, and optimizes via reinforcement learning.

Result: Empirical results show L2T boosts both reasoning effectiveness and efficiency across various benchmarks and base models, achieving optimal reasoning with fewer tokens.

Conclusion: L2T successfully addresses the reasoning efficiency-effectiveness trade-off through information-theoretic reinforcement learning, enabling LLMs to achieve better performance with reduced computational cost.

Abstract: Large language models (LLMs) excel at complex tasks thanks to advances in their reasoning abilities. However, existing methods overlook the trade-off between reasoning effectiveness and efficiency, often encouraging unnecessarily long reasoning chains and wasting tokens. To address this, we propose Learning to Think (L2T), an information-theoretic reinforcement fine-tuning framework for LLMs to make the models achieve optimal reasoning with fewer tokens. Specifically, L2T treats each query-response interaction as a hierarchical session of multiple episodes and proposes a universal dense process reward, i.e., quantifies the episode-wise information gain in parameters, requiring no extra annotations or task-specific evaluators. We propose a method to quickly estimate this reward based on PAC-Bayes bounds and the Fisher information matrix. Theoretical analyses show that it significantly reduces computational complexity with high estimation accuracy. By immediately rewarding each episode’s contribution and penalizing excessive updates, L2T optimizes the model via reinforcement learning to maximize the use of each episode and achieve effective updates. Empirical results on various reasoning benchmarks and base models demonstrate the advantage of L2T across different tasks, boosting both reasoning effectiveness and efficiency.

Method: Comparative study of biological mice and RL agents in predator-avoidance maze environment, followed by proposing two novel mechanisms to encourage naturalistic risk-avoidance behaviors in RL agents.

Result: RL agents consistently showed lack of self-preservation instinct, readily risking ‘death’ for marginal efficiency gains, while biological agents exhibited sophisticated risk-assessment and avoidance behaviors. The proposed mechanisms successfully led to emergence of naturalistic behaviors including strategic environment assessment, cautious path planning, and predator avoidance patterns mirroring biological systems.

Conclusion: The gap between artificial and biological risk-assessment can be bridged through specific mechanisms that encourage more naturalistic behaviors in RL agents, leading to improved self-preservation instincts similar to biological systems.

Abstract: Recent advances in reinforcement learning (RL) have demonstrated impressive capabilities in complex decision-making tasks. This progress raises a natural question: how do these artificial systems compare to biological agents, which have been shaped by millions of years of evolution? To help answer this question, we undertake a comparative study of biological mice and RL agents in a predator-avoidance maze environment. Through this analysis, we identify a striking disparity: RL agents consistently demonstrate a lack of self-preservation instinct, readily risking ``death’’ for marginal efficiency gains. These risk-taking strategies are in contrast to biological agents, which exhibit sophisticated risk-assessment and avoidance behaviors. Towards bridging this gap between the biological and artificial, we propose two novel mechanisms that encourage more naturalistic risk-avoidance behaviors in RL agents. Our approach leads to the emergence of naturalistic behaviors, including strategic environment assessment, cautious path planning, and predator avoidance patterns that closely mirror those observed in biological systems.

Method: Proposed Hierarchical Evaluation Function (HEF) integrating R2, RMSE, and MAE; tested on Walmart, M3, M4, and M5 datasets using Grid Search, PSO, and Optuna optimizers with statistical difference-of-proportions tests.

Result: HEF delivered superior results compared to unimetric reference function across all optimizers, particularly effective for heterogeneous monthly time series (M3) and highly granular daily demand scenarios (M5).

Conclusion: HEF improves stability, generalization, and robustness at low computational cost, serving as a reliable evaluation framework that enhances model selection and supports decision-making in dynamic business environments.

Abstract: Demand forecasting in competitive and uncertain business environments requires models that can integrate multiple evaluation perspectives, rather than being restricted to hyperparameter optimization through a single metric. This traditional approach tends to prioritize one error indicator, which can bias results when metrics provide contradictory signals. In this context, the Hierarchical Evaluation Function (HEF) is proposed as a multi-metric framework for hyperparameter optimization that integrates explanatory power (R2), sensitivity to extreme errors (RMSE), and average accuracy (MAE). The performance of HEF was assessed using four widely recognized benchmark datasets in the forecasting domain: the Walmart, M3, M4, and M5 datasets. Prediction models were optimized through Grid Search, Particle Swarm Optimization (PSO), and Optuna, and statistical analyses based on difference-of-proportions tests confirmed that HEF delivers superior results compared to a unimetric reference function, regardless of the optimizer employed, with particular relevance for heterogeneous monthly time series (M3) and highly granular daily demand scenarios (M5). The findings demonstrate that HEF improves stability, generalization, and robustness at a low computational cost, consolidating its role as a reliable evaluation framework that enhances model selection, enables more accurate demand forecasts, and supports decision-making in dynamic and competitive business environments.

Method: Uses social-network inspired approach with homophily-based proximity matrices for spectral clustering to form client coalitions and select representatives that minimize model update variance.

Result: The algorithm shows higher accuracy and faster convergence compared to three strong heterogeneity-aware baselines.

Conclusion: The proposed framework is both theoretically grounded with convergence guarantees and effective in practice for handling client data heterogeneity in federated learning.

Abstract: Federated Learning (FL) enables privacy-preserving collaborative model training, but its effectiveness is often limited by client data heterogeneity. We introduce a client-selection algorithm that (i) dynamically forms nonoverlapping coalitions of clients based on asymptotic agreement and (ii) selects one representative from each coalition to minimize the variance of model updates. Our approach is inspired by social-network modeling, leveraging homophily-based proximity matrices for spectral clustering and techniques for identifying the most informative individuals to estimate a group’s aggregate opinion. We provide theoretical convergence guarantees for the algorithm under mild, standard FL assumptions. Finally, we validate our approach by benchmarking it against three strong heterogeneity-aware baselines; the results show higher accuracy and faster convergence, indicating that the framework is both theoretically grounded and effective in practice.

Method: CE-GPPO reintroduces gradients from clipped tokens in a gentle and bounded manner, controlling gradient magnitude from tokens outside the clipping interval to achieve better exploration-exploitation trade-off.

Result: Extensive experiments on mathematical reasoning benchmarks show that CE-GPPO consistently outperforms strong baselines across different model scales and effectively mitigates entropy instability.

Conclusion: CE-GPPO provides a theoretically justified and empirically validated approach to better manage policy entropy in RL for LLMs, leading to improved performance on complex reasoning tasks.

Abstract: Reinforcement learning (RL) has become a powerful paradigm for optimizing large language models (LLMs) to handle complex reasoning tasks. A core challenge in this process lies in managing policy entropy, which reflects the balance between exploration and exploitation during training. Existing methods, such as proximal policy optimization (PPO) and its variants, discard valuable gradient signals from low-probability tokens due to the clipping mechanism. We systematically analyze the entropy dynamics and reveal that these clipped tokens play a critical yet overlooked role in regulating entropy evolution. We propose \textbf{C}oordinating \textbf{E}ntropy via \textbf{G}radient-\textbf{P}reserving \textbf{P}olicy \textbf{O}ptimization (CE-GPPO), a novel algorithm that reintroduces gradients from clipped tokens in native PPO in a gentle and bounded manner. By controlling the magnitude of gradients from tokens outside the clipping interval, CE-GPPO is able to achieve an exploration-exploitation trade-off. We provide theoretical justification and empirical evidence showing that CE-GPPO effectively mitigates entropy instability. Extensive experiments on mathematical reasoning benchmarks show that CE-GPPO consistently outperforms strong baselines across different model scales.

Method: Proposes QHNetV2 with SO(2)-equivariant operations that perform all off-diagonal feature updates and message passing within SO(2) local frames, eliminating the need for SO(3) Clebsch-Gordan tensor products. Uses continuous SO(2) tensor products to fuse node features.

Result: Extensive experiments on QH9 and MD17 datasets show superior performance across various molecular structures and trajectories, demonstrating strong generalization capability.

Conclusion: The SO(2) operations on SO(2) local frames provide a promising direction for scalable and symmetry-aware learning of electronic structures, offering an efficient alternative to SO(3) tensor products.

Abstract: We consider the task of predicting Hamiltonian matrices to accelerate electronic structure calculations, which plays an important role in physics, chemistry, and materials science. Motivated by the inherent relationship between the off-diagonal blocks of the Hamiltonian matrix and the SO(2) local frame, we propose a novel and efficient network, called QHNetV2, that achieves global SO(3) equivariance without the costly SO(3) Clebsch-Gordan tensor products. This is achieved by introducing a set of new efficient and powerful SO(2)-equivariant operations and performing all off-diagonal feature updates and message passing within SO(2) local frames, thereby eliminating the need of SO(3) tensor products. Moreover, a continuous SO(2) tensor product is performed within the SO(2) local frame at each node to fuse node features, mimicking the symmetric contraction operation. Extensive experiments on the large QH9 and MD17 datasets demonstrate that our model achieves superior performance across a wide range of molecular structures and trajectories, highlighting its strong generalization capability. The proposed SO(2) operations on SO(2) local frames offer a promising direction for scalable and symmetry-aware learning of electronic structures. Our code will be released as part of the AIRS library https://github.com/divelab/AIRS.

Method: Used RLVR framework with curriculum-learning scheme to address conflicting rule-derived rewards in the Korean word-chain game

Result: Curriculum learning successfully mitigated conflicts between naturally conflicting rule-derived rewards

Conclusion: The findings support further investigation of puzzle tasks across diverse languages using RLVR approaches

Abstract: Reinforcement learning with verifiable rewards (RLVR) is a promising approach for training large language models (LLMs) with stronger reasoning abilities. It has also been applied to a variety of logic puzzles. In this work, we study the Korean word-chain game using RLVR. We show that rule-derived rewards can naturally conflict, and demonstrate through experiments that a curriculum-learning scheme mitigates these conflicts. Our findings motivate further studies of puzzle tasks in diverse languages.

Method: Partition pretrained language model layers into four expert modules aligned with cognitive networks, post-train with curriculum learning to induce functional specialization, and enable dynamic token routing at inference.

Result: MiCRo outperforms or matches baselines on reasoning benchmarks (GSM8K, BBH) and human behavior alignment (CogBench), while maintaining interpretability and enabling causal ablation studies.

Conclusion: Cognitively grounded functional specialization produces models that are both more human-like and more interpretable, offering advantages in causal understanding and dynamic control over standard language models.

Abstract: Human cognitive behavior arises from the interaction of specialized brain networks dedicated to distinct functions, such as language, logic, and social reasoning. Inspired by this organization, we propose Mixture of Cognitive Reasoners (MiCRo): a modular, transformer-based architecture post-trained with a curriculum that induces functional specialization across experts. Concretely, we partition the layers of a pretrained language model into four expert modules aligned with well-studied cognitive networks in the human brain. MiCRo offers three key advantages over standard language models. (1) The specialized experts are interpretable and causally meaningful – ablating a module causes substantial drops on benchmarks requiring its specialized domain. (2) MiCRo’s behavior can be dynamically steered at inference time by routing tokens to particular experts (e.g., favoring social over logical reasoning), enabling fine-grained control over outputs. (3) MiCRo outperforms or matches comparable baselines on both machine-learning reasoning benchmarks (e.g., GSM8K, BBH) and alignment to human behavior (CogBench), while maintaining interpretability. Taken together, cognitively grounded functional specialization yields models that are both more human-like and more human-interpretable.

Method: Constrained optimization formulations that find smallest weight interventions to make sensitive vocabulary unreachable or shift weights to safer regions, using point-wise constraint-based interventions.

Result: The proposed approach outperforms max-min interventions and state-of-the-art defense methods, achieving better performance with lower computational cost.

Conclusion: Simple point-wise constraint-based interventions provide an effective unified solution for LLM safety that doesn’t require oracle classifiers and offers computational efficiency.

Abstract: With the increasing adoption of Large Language Models (LLMs), more customization is needed to ensure privacy-preserving and safe generation. We address this objective from two critical aspects: unlearning of sensitive information and robustness to jail-breaking attacks. We investigate various constrained optimization formulations that address both aspects in a \emph{unified manner}, by finding the smallest possible interventions on LLM weights that either make a given vocabulary set unreachable or embed the LLM with robustness to tailored attacks by shifting part of the weights to a \emph{safer} region. Beyond unifying two key properties, this approach contrasts with previous work in that it doesn’t require an oracle classifier that is typically not available or represents a computational overhead. Surprisingly, we find that the simplest point-wise constraint-based intervention we propose leads to better performance than max-min interventions, while having a lower computational cost. Comparison against state-of-the-art defense methods demonstrates superior performance of the proposed approach.

Method: Joint learning across conformational ensembles using 46,033 conformational pairs covering 75% of CATH superfamilies, explicitly trained for multi-conformation compatibility.

Result: Outperforms ProteinMPNN by up to 25% on decoy-normalized RMSD and by 12% on sequence recovery across multi-state protein benchmark, evaluated using Alphafold 3.

Conclusion: DynamicMPNN successfully addresses the limitations of existing multi-state design approaches by explicitly training for conformational ensemble compatibility.

Abstract: Structural biology has long been dominated by the one sequence, one structure, one function paradigm, yet many critical biological processes - from enzyme catalysis to membrane transport - depend on proteins that adopt multiple conformational states. Existing multi-state design approaches rely on post-hoc aggregation of single-state predictions, achieving poor experimental success rates compared to single-state design. We introduce DynamicMPNN, an inverse folding model explicitly trained to generate sequences compatible with multiple conformations through joint learning across conformational ensembles. Trained on 46,033 conformational pairs covering 75% of CATH superfamilies and evaluated using Alphafold 3, DynamicMPNN outperforms ProteinMPNN by up to 25% on decoy-normalized RMSD and by 12% on sequence recovery across our challenging multi-state protein benchmark.

Method: Proposes a dynamic-rank training framework that strategically alternates between full-rank and low-rank training epochs to restore weight matrix rank and prevent rank collapse during training.

Result: Achieves almost the same computational cost as SVD-based low-rank training while achieving comparable accuracy to full-rank training across various benchmarks.

Conclusion: The proposed dynamic-rank training framework effectively prevents rank collapse while maintaining the computational benefits of low-rank methods and the accuracy of full-rank training.

Abstract: Low-rank training methods reduce the number of trainable parameters by re-parameterizing the weights with matrix decompositions (e.g., singular value decomposition). However, enforcing a fixed low-rank structure caps the rank of the weight matrices and can hinder the model’s ability to learn complex patterns. Furthermore, the effective rank of the model’s weights tends to decline during training, and this drop is accelerated when the model is reparameterized into a low-rank structure. In this study, we argue that strategically interleaving full-rank training epochs within low-rank training epochs can effectively restore the rank of the model’s weights. Based on our findings, we propose a general dynamic-rank training framework that is readily applicable to a wide range of neural-network tasks. We first describe how to adjust the rank of weight matrix to alleviate the inevitable rank collapse that arises during training, and then present extensive empirical results that validate our claims and demonstrate the efficacy of the proposed framework. Our empirical study shows that the proposed method achieves almost the same computational cost as SVD-based low-rank training while achieving a comparable accuracy to full-rank training across various benchmarks.

Method: FLARE projects input sequences onto fixed-length latent sequences using learnable query tokens, routing attention through a bottleneck to achieve O(NM) complexity where M « N.

Result: FLARE scales to unprecedented problem sizes and delivers superior accuracy compared to state-of-the-art neural PDE surrogates across diverse benchmarks.

Conclusion: FLARE provides an efficient linear-complexity attention mechanism that enables scalable processing of large unstructured meshes while maintaining high accuracy.

Abstract: The quadratic complexity of self-attention limits its applicability and scalability on large unstructured meshes. We introduce Fast Low-rank Attention Routing Engine (FLARE), a linear complexity self-attention mechanism that routes attention through fixed-length latent sequences. Each attention head performs global communication among $N$ tokens by projecting the input sequence onto a fixed length latent sequence of $M \ll N$ tokens using learnable query tokens. By routing attention through a bottleneck sequence, FLARE learns a low-rank form of attention that can be applied at $O(NM)$ cost. FLARE not only scales to unprecedented problem sizes, but also delivers superior accuracy compared to state-of-the-art neural PDE surrogates across diverse benchmarks. We also release a new additive manufacturing dataset to spur further research. Our code is available at https://github.com/vpuri3/FLARE.py.

Method: Uses dual-expert design: dedicated LoRA expert per task for knowledge preservation, and shared LoRA expert for cross-task transfer. Integrates task-aware discriminator in GAN framework to filter task-irrelevant noise and encourage task-aligned information sharing.

Result: Extensive experiments on MTL5 and Tencent3 benchmarks validated effectiveness. In real-world A/B testing for content compliance review on Tencent Video, MoE-CL reduced manual review costs by 15.3%.

Conclusion: MoE-CL is practical for large-scale industrial deployment where continual adaptation and stable transfer are critical, balancing knowledge retention and cross-task generalization to support self-evolution.

Abstract: In real-world industrial settings, large language models (LLMs) must learn continually to keep pace with diverse and evolving tasks, requiring self-evolution to refine knowledge under dynamic data distributions. However, existing continual learning (CL) approaches, such as replay and parameter isolation, often suffer from catastrophic forgetting: training on new tasks degrades performance on earlier ones by overfitting to the new distribution and weakening generalization.We propose MoE-CL, a parameter-efficient adversarial mixture-of-experts framework for industrial-scale, self-evolving continual instruction tuning of LLMs. MoE-CL uses a dual-expert design: (1) a dedicated LoRA expert per task to preserve task-specific knowledge via parameter independence, mitigating forgetting; and (2) a shared LoRA expert to enable cross-task transfer. To prevent transferring task-irrelevant noise through the shared pathway, we integrate a task-aware discriminator within a GAN. The discriminator encourages the shared expert to pass only task-aligned information during sequential training. Through adversarial learning, the shared expert acquires generalized representations that mimic the discriminator, while dedicated experts retain task-specific details, balancing knowledge retention and cross-task generalization and thereby supporting self-evolution.Extensive experiments on the public MTL5 benchmark and an industrial Tencent3 benchmark validate the effectiveness of MoE-CL for continual instruction tuning. In real-world A/B testing for content compliance review on the Tencent Video platform, MoE-CL reduced manual review costs by 15.3%. These results demonstrate that MoE-CL is practical for large-scale industrial deployment where continual adaptation and stable transfer are critical.

Method: Learn the quadratic form x^T A x corresponding to the group from data, use the symmetric matrix’s diagonal form to incorporate inductive biases, and decompose equivariant functions into norm-invariant and scale-invariant components. Extend to tuples of vectors via diagonal group action.

Result: The framework successfully discovers underlying symmetries and learns equivariant functions across various tasks including polynomial regression, top quark tagging, and moment of inertia prediction, outperforming baseline methods.

Conclusion: The proposed approach effectively learns group equivariant functions by discovering quadratic forms, leading to simplified and efficient models that preserve group symmetries while capturing complex inter-dependencies in data.

Abstract: In this study, we introduce a method for learning group (known or unknown) equivariant functions by learning the associated quadratic form $x^T A x$ corresponding to the group from the data. Certain groups, known as orthogonal groups, preserve a specific quadratic form, and we leverage this property to uncover the underlying symmetry group under the assumption that it is orthogonal. By utilizing the corresponding unique symmetric matrix and its inherent diagonal form, we incorporate suitable inductive biases into the neural network architecture, leading to models that are both simplified and efficient. Our approach results in an invariant model that preserves norms, while the equivariant model is represented as a product of a norm-invariant model and a scale-invariant model, where the ``product’’ refers to the group action. Moreover, we extend our framework to a more general setting where the function acts on tuples of input vectors via a diagonal (or product) group action. In this extension, the equivariant function is decomposed into an angular component extracted solely from the normalized first vector and a scale-invariant component that depends on the full Gram matrix of the tuple. This decomposition captures the inter-dependencies between multiple inputs while preserving the underlying group symmetry. We assess the effectiveness of our framework across multiple tasks, including polynomial regression, top quark tagging, and moment of inertia matrix prediction. Comparative analysis with baseline methods demonstrates that our model consistently excels in both discovering the underlying symmetry and efficiently learning the corresponding equivariant function.

Method: Introduces functional critic modeling that leads to a new actor-critic framework. Theoretically analyzed in linear function setting and implemented with a carefully designed neural network architecture for practical applications.

Result: The framework provides provable convergence, making it the first convergent off-policy target-based AC algorithm. Preliminary experiments on DeepMind Control Benchmark tasks demonstrate effectiveness.

Conclusion: Functional critic modeling successfully addresses both challenges in off-policy actor-critic learning under the deadly triad, offering theoretical convergence guarantees and practical effectiveness through specialized neural network design.

Abstract: Off-policy reinforcement learning (RL) with function approximation offers an effective way to improve sample efficiency by reusing past experience. Within this setting, the actor-critic (AC) framework has achieved strong empirical success. However, both the critic and actor learning is challenging for the off-policy AC methods: first of all, in addition to the classic “deadly triad” instability of off-policy evaluation, it also suffers from a “moving target” problem, where the policy being evaluated changes continually; secondly, actor learning becomes less efficient due to the difficulty of estimating the exact off-policy policy gradient. The first challenge essentially reduces the problem to repeatedly performing off-policy evaluation for changing policies. For the second challenge, the off-policy policy gradient theorem requires a complex and often impractical algorithm to estimate an additional emphasis critic, which is typically neglected in practice, thereby reducing to the on-policy policy gradient as an approximation. In this work, we introduce a novel concept of functional critic modeling, which leads to a new AC framework that addresses both challenges for actor-critic learning under the deadly triad setting. We provide a theoretical analysis in the linear function setting, establishing the provable convergence of our framework, which, to the best of our knowledge, is the first convergent off-policy target-based AC algorithm. From a practical perspective, we further propose a carefully designed neural network architecture for the functional critic modeling and demonstrate its effectiveness through preliminary experiments on widely used RL tasks from the DeepMind Control Benchmark.

Method: Introduces Relational Graph Recognition (RGR) framework where key-query channels represent graphs, and analytically derives capacity scaling laws validated through controlled single-layer experiments.

Result: Established that D_K = Θ(m’ log m’ / d_model) is both necessary and sufficient to recover m’ relations, showing multi-head attention mitigates interference when embeddings are compressed.

Conclusion: Provides concrete scaling law for self-attention capacity and principled design rule for allocating key-query budget across heads, confirming benefits of distributing capacity across multiple heads.

Abstract: While self-attention is known to learn relations among tokens, we lack a formal understanding of its capacity: how many distinct relations can a single layer reliably recover for a given budget? To formalize this, we introduce Relational Graph Recognition (RGR), where the key-query channel represents a graph on $m$ items with $m’$ directed edges, and, given a context of items, must recover the neighbors of each item. We measure resources by the total key dimension $D_K = h,d_k$. Within this framework, we analytically derive a capacity scaling law and validate it empirically. We show that $D_K = \Theta(m’ \log m’ / d_{\text{model}})$ is both necessary (information-theoretic lower bound) and sufficient (explicit construction) in a broad class of graphs to recover $m’$ relations. This scaling law directly leads to a new, capacity-based rationale for multi-head attention that applies even when each item only attends to a single target. When embeddings are uncompressed ($m = d_{\text{model}}$) and the graph is a permutation, a single head suffices. However, compression ($m > d_{\text{model}}$) forces relations into overlapping subspaces, creating interference that a single large head cannot disentangle. Our analysis shows that allocating a fixed $D_K$ across many small heads mitigates this interference, increasing the number of recoverable relations. Controlled single-layer experiments mirror the theory, revealing a sharp performance threshold that matches the predicted capacity scaling and confirms the benefit of distributing $D_K$ across multiple heads. Altogether, these results provide a concrete scaling law for self-attention capacity and a principled design rule for allocating key-query budget across heads.

Method: Developed Fidel-TS benchmark by formalizing core principles: data sourcing integrity, strict causal soundness, and structural clarity. Built from ground up using live API data.

Result: Experiments exposed critical biases and design limitations in prior benchmarks. Demonstrated that causal relevance of textual information is the key factor for genuine performance gains in multimodal forecasting.

Conclusion: High-fidelity benchmarking with proper data sourcing and causal soundness is essential for reliable evaluation of time series forecasting models, with textual causal relevance being crucial for multimodal approaches.

Abstract: The evaluation of time series forecasting models is hindered by a critical lack of high-quality benchmarks, leading to a potential illusion of progress. Existing datasets suffer from issues ranging from pre-training data contamination in the age of LLMs to the causal and description leakage prevalent in early multimodal designs. To address this, we formalize the core principles of high-fidelity benchmarking, focusing on data sourcing integrity, strict causal soundness, and structural clarity. We introduce Fidel-TS, a new large-scale benchmark built from the ground up on these principles by sourcing data from live APIs. Our extensive experiments validate this approach by exposing the critical biases and design limitations of prior benchmarks. Furthermore, we conclusively demonstrate that the causal relevance of textual information is the key factor in unlocking genuine performance gains in multimodal forecasting.

Method: TransIP uses a generic non-equivariant Transformer-based model and guides it to learn SO(3)-equivariance by optimizing representations in the embedding space. Trained on the Open Molecules (OMol25) dataset covering diverse molecular types.

Result: TransIP effectively learns symmetry in latent space with low equivariance error. Achieves 40-60% performance improvement over data augmentation baseline across varying dataset sizes in OMol25.

Conclusion: Learned equivariance can be a powerful and efficient alternative to augmentation-based MLIP models, showing that symmetry compliance can be achieved without hard architectural constraints.

Abstract: Accurate and scalable machine-learned inter-atomic potentials (MLIPs) are essential for molecular simulations ranging from drug discovery to new material design. Current state-of-the-art models enforce roto-translational symmetries through equivariant neural network architectures, a hard-wired inductive bias that can often lead to reduced flexibility, computational efficiency, and scalability. In this work, we introduce TransIP: Transformer-based Inter-Atomic Potentials, a novel training paradigm for interatomic potentials achieving symmetry compliance without explicit architectural constraints. Our approach guides a generic non-equivariant Transformer-based model to learn SO(3)-equivariance by optimizing its representations in the embedding space. Trained on the recent Open Molecules (OMol25) collection, a large and diverse molecular dataset built specifically for MLIPs and covering different types of molecules (including small organics, biomolecular fragments, and electrolyte-like species), TransIP effectively learns symmetry in its latent space, providing low equivariance error. Further, compared to a data augmentation baseline, TransIP achieves 40% to 60% improvement in performance across varying OMol25 dataset sizes. More broadly, our work shows that learned equivariance can be a powerful and efficient alternative to augmentation-based MLIP models.

Method: Three-step approach: (1) Replace self-attention with spike-based self-attention (SSA), (2) Convert feed-forward blocks to SNN equivalents, (3) Fine-tune SSA block using SNN-based surrogate learning algorithms.

Result: Converted GPT-2 small models show 5-12% loss in cosine similarity and 9.7% reduction in perplexity. SSA block demonstrates 64.71-85.28% reduction in estimated energy consumption compared to standard self-attention on digital hardware.

Conclusion: NeurTransformer provides an effective methodology for creating energy-efficient transformer-based SNNs with minimal performance degradation, addressing computational challenges of traditional transformers.

Abstract: Foundational models based on the transformer architecture are currently the state-of-the-art in general language modeling, as well as in scientific areas such as material science and climate. However, training and deploying these models is computationally challenging as the time and space complexity has a quadratic relation to the input sequence length. Several efforts exploring efficient computational paradigms and model architectures to address these limitations have been made. In this work, we explore spiking neural networks (SNNs) to design transformer models. A challenge in training large-scale SNNs, using existing surrogate learning methods is inefficient and time-consuming. On the other hand, techniques to convert existing transformer-based models to their SNN equivalent are not scalable, as achieving optimal performance comes at the cost of a large number of spike time-steps, i.e. increased latency. To address this, we propose NeurTransformer, a methodology for designing transformer-based SNN for inference using a supervised fine-tuning approach with existing conversion methods. The proposed methodology works by: (1) replacing the self-attention mechanism with a spike-based self-attention (SSA), (2) converting the feed-forward block of the trained transformer model to its equivalent SNN, and (3) fine-tuning the SSA block using SNN-based surrogate learning algorithms. We benchmark the proposed methodology and demonstrate its accuracy and scalability using three variants of the GPT-2 model of increasing model size. We observe that the converted GPT-2 small models demonstrate a 5-12% loss in cosine similarity and a 9.7% reduction in perplexity. Finally, we demonstrate the energy efficiency of the SSA block compared to the ASA block and show between 64.71% and 85.28% reductions in estimated energy consumption when implementing the self-attention mechanism on a digital hardware.

Method: Asymmetric PPO uses lightweight mini-critics on disjoint prompt shards, leverages inter-critic uncertainty to mask advantages in agreed states and filter high-divergence states from entropy regularization.

Result: Achieved performance gains of over 6% on Qwen3-4b-Base and ~3% on Qwen3-8b-Base and Qwen3-14b-Base over classic PPO with only 5,000 training samples, outperforming strong baselines like GRPO.

Conclusion: Architectural innovations are crucial for scalable, efficient RL algorithms for LLMs, as demonstrated by AsyPPO’s success in restoring critics while maintaining efficiency.

Abstract: Most recent RL for LLMs (RL4LLM) methods avoid explicit critics, replacing them with average advantage baselines. This shift is largely pragmatic: conventional value functions are computationally expensive to train at LLM scale and often fail under sparse rewards and long reasoning horizons. We revisit this bottleneck from an architectural perspective and introduce Asymmetric Proximal Policy Optimization (AsyPPO), a simple and scalable framework that restores the critics role while remaining efficient in large-model settings. AsyPPO employs a set of lightweight mini-critics, each trained on disjoint prompt shards. This design encourages diversity while preserving calibration, reducing value-estimation bias. Beyond robust estimation, AsyPPO leverages inter-critic uncertainty to refine the policy update: (i) masking advantages in states where critics agree and gradients add little learning signal, and (ii) filtering high-divergence states from entropy regularization, suppressing spurious exploration. After training on open-source data with only 5,000 samples, AsyPPO consistently improves learning stability and performance across multiple benchmarks over strong baselines, such as GRPO, achieving performance gains of more than six percent on Qwen3-4b-Base and about three percent on Qwen3-8b-Base and Qwen3-14b-Base over classic PPO, without additional tricks. These results highlight the importance of architectural innovations for scalable, efficient algorithms.

Method: The authors analyze noise models corresponding to real-world sources like mislabeling and uncertainty, focusing on finite-step preference optimization rather than assuming convergence. They examine how generalization decays with different noise types and rates based on data distribution and sample size.

Result: The analysis provides generalization guarantees for noisy preference learning that apply to a broad family of optimization losses (DPO, IPO, SLiC, etc.). Empirical validation on contemporary LLMs confirms the practical relevance of the findings.

Conclusion: The work offers valuable insights for developing AI systems that better align with human preferences under realistic noisy feedback conditions, bridging the gap between theoretical analysis and practical LLM training.

Abstract: As large language models (LLMs) advance their capabilities, aligning these models with human preferences has become crucial. Preference optimization, which trains models to distinguish between preferred and non-preferred responses based on human feedback, has become a crucial component for aligning LLMs. However, most existing works assume noise-free feedback, which is unrealistic due to the inherent errors and inconsistencies in human judgments. This paper addresses the impact of noisy feedback on preference optimization, providing generalization guarantees under these conditions. In particular, we consider noise models that correspond to common real-world sources of noise, such as mislabeling and uncertainty. Unlike traditional analyses that assume convergence, our work focuses on finite-step preference optimization, offering new insights that are more aligned with practical LLM training. We describe how generalization decays with different types of noise across levels of noise rates based on the preference data distribution and number of samples. Our analysis for noisy preference learning applies to a broad family of preference optimization losses such as DPO, IPO, SLiC, etc. Empirical validation on contemporary LLMs confirms the practical relevance of our findings, offering valuable insights for developing AI systems that align with human preferences.

Method: Combines Conditional Masked Autoregressive normalizing flow for aleatoric uncertainty with flexible probabilistic predictor for epistemic uncertainty, supporting integration with any probabilistic model class.

Result: Improves upon previous uncertainty quantification frameworks across depth estimation, regression benchmarks, and scientific case studies; shows calibrated uncertainty that better aligns with model error than existing methods.

Conclusion: HybridFlow successfully unifies aleatoric and epistemic uncertainty modeling in a single robust framework, addressing key challenges in Bayesian deep learning.

Abstract: Uncertainty quantification is critical for ensuring robustness in high-stakes machine learning applications. We introduce HybridFlow, a modular hybrid architecture that unifies the modeling of aleatoric and epistemic uncertainty by combining a Conditional Masked Autoregressive normalizing flow for estimating aleatoric uncertainty with a flexible probabilistic predictor for epistemic uncertainty. The framework supports integration with any probabilistic model class, allowing users to easily adapt HybridFlow to existing architectures without sacrificing predictive performance. HybridFlow improves upon previous uncertainty quantification frameworks across a range of regression tasks, such as depth estimation, a collection of regression benchmarks, and a scientific case study of ice sheet emulation. We also provide empirical results of the quantified uncertainty, showing that the uncertainty quantified by HybridFlow is calibrated and better aligns with model error than existing methods for quantifying aleatoric and epistemic uncertainty. HybridFlow addresses a key challenge in Bayesian deep learning, unifying aleatoric and epistemic uncertainty modeling in a single robust framework.

Method: Combines a policy improvement objective that avoids high variance gradients with a robust distributional critic using distributional RL and clipped double Q-learning.

Result: Achieved state-of-the-art performance on 18 hard RL tasks including Humanoid, Dog, and Shadow Hand domains in both dense-reward and goal-conditioned scenarios.

Conclusion: D2AC provides an effective approach for training diffusion policies online with stable learning and strong performance across diverse RL benchmarks.

Abstract: We introduce D2AC, a new model-free reinforcement learning (RL) algorithm designed to train expressive diffusion policies online effectively. At its core is a policy improvement objective that avoids the high variance of typical policy gradients and the complexity of backpropagation through time. This stable learning process is critically enabled by our second contribution: a robust distributional critic, which we design through a fusion of distributional RL and clipped double Q-learning. The resulting algorithm is highly effective, achieving state-of-the-art performance on a benchmark of eighteen hard RL tasks, including Humanoid, Dog, and Shadow Hand domains, spanning both dense-reward and goal-conditioned RL scenarios. Beyond standard benchmarks, we also evaluate a biologically motivated predator-prey task to examine the behavioral robustness and generalization capacity of our approach.

Method: Developed a new methodology for function regression using the Forward-Forward algorithm, evaluated on univariate and multivariate functions, and extended to Kolmogorov Arnold Networks and Deep Physical Neural Networks.

Result: Successfully demonstrated that the Forward-Forward algorithm can be used for function regression tasks, expanding its applicability beyond classification.

Conclusion: The Forward-Forward algorithm can be effectively extended to function regression tasks, opening new possibilities for neuromorphic computing and physical neural network implementations without backpropagation.

Abstract: Function regression/approximation is a fundamental application of machine learning. Neural networks (NNs) can be easily trained for function regression using a sufficient number of neurons and epochs. The forward-forward learning algorithm is a novel approach for training neural networks without backpropagation, and is well suited for implementation in neuromorphic computing and physical analogs for neural networks. To the best of the authors’ knowledge, the Forward Forward paradigm of training and inferencing NNs is currently only restricted to classification tasks. This paper introduces a new methodology for approximating functions (function regression) using the Forward-Forward algorithm. Furthermore, the paper evaluates the developed methodology on univariate and multivariate functions, and provides preliminary studies of extending the proposed Forward-Forward regression to Kolmogorov Arnold Networks, and Deep Physical Neural Networks.

Method: Synthetically compose simple problems into complex multi-step dependency chains of arbitrary length, then train models using outcome-only rewards under an automatic curriculum that increases in complexity.

Result: Curriculum training on composed 6th-grade math problems boosts accuracy on longer competition-level benchmarks by up to 2.06x, transfers to diverse out-of-distribution domains, and shows significant improvements even at high pass@k.

Conclusion: The method provides an efficient path for scaling RL for long-horizon problems using only existing data, achieving exponential improvement in sample complexity over full-horizon training.

Abstract: Large language models excel at short-horizon reasoning tasks, but performance drops as reasoning horizon lengths increase. Existing approaches to combat this rely on inference-time scaffolding or costly step-level supervision, neither of which scales easily. In this work, we introduce a scalable method to bootstrap long-horizon reasoning capabilities using only existing, abundant short-horizon data. Our approach synthetically composes simple problems into complex, multi-step dependency chains of arbitrary length. We train models on this data using outcome-only rewards under a curriculum that automatically increases in complexity, allowing RL training to be scaled much further without saturating. Empirically, our method generalizes remarkably well: curriculum training on composed 6th-grade level math problems (GSM8K) boosts accuracy on longer, competition-level benchmarks (GSM-Symbolic, MATH-500, AIME) by up to 2.06x. It also transfers significantly to diverse out-of-distribution ReasoningGym domains and long-context benchmarks, indicating broader generalization. Importantly, our long-horizon improvements are significantly higher than baselines even at high pass@k, showing that models can learn new reasoning paths under RL. Theoretically, we show that curriculum RL with outcome rewards achieves an exponential improvement in sample complexity over full-horizon training, providing training signal comparable to dense supervision. h1 therefore introduces an efficient path towards scaling RL for long-horizon problems using only existing data.

Method: AEPO replaces entropy bonuses with REINFORCE policy gradient on temperature-adjusted distributions and stabilizes entropy through temperature regulation, integrating policy gradient, distribution, and REINFORCE as regularization.

Result: AEPO stabilizes entropy at arbitrary target levels, removes collapse in GRPO, reveals non-monotonic performance-entropy relationship, and generalizes beyond entropy to broader RFT paradigm.

Conclusion: AEPO effectively resolves entropy collapse, clarifies entropy-exploration-performance relationship, and provides a generalized RFT framework using superior target distributions as REINFORCE regularizers.

Abstract: Reinforcement fine-tuning (RFT) is essential for enhancing the reasoning capabilities of large language models (LLM), yet the widely adopted Group Relative Policy Optimization (GRPO) suffers from entropy collapse, where entropy monotonically decreases, exploration vanishes, and policies converge prematurely. Existing entropy-regularized methods only partially alleviate this issue while introducing bias and instability, leaving entropy control unresolved and the connection between entropy, exploration, and performance unclear. We propose Arbitrary Entropy Policy Optimization (AEPO), which eliminates entropy collapse by replacing entropy bonuses with REINFORCE policy gradient on temperature-adjusted distributions and stabilizing entropy through temperature regulation. AEPO integrates three key designs: policy gradient as regularization, distribution as regularization, and REINFORCE as regularization, enabling precise entropy control without distorting optimization. Experiments demonstrate three major contributions: AEPO (1) stabilizes entropy at arbitrary target levels, effectively removing collapse in GRPO; (2) reveals a non-monotonic relation where performance first improves then declines with increasing entropy, clarifying the link between entropy, exploration, and reasoning; and (3) generalizes beyond entropy, providing a broader RFT paradigm where superior target distributions can serve as REINFORCE regularizers.

Method: The authors analyze the relationship between expressivity and generalization through the lens of coloring algorithms, showing that the number of equivalence classes induced by WL colorings bounds the GNNs’ Rademacher complexity.

Result: Greater expressivity leads to higher Rademacher complexity and weaker generalization guarantees. The complexity is also stable under perturbations in color counts across samples.

Conclusion: The framework unifies expressivity and generalization in GNNs, providing a principled understanding of why increasing expressive power often comes at the cost of generalization.

Abstract: The expressive power of graph neural networks (GNNs) is typically understood through their correspondence with graph isomorphism tests such as the Weisfeiler-Leman (WL) hierarchy. While more expressive GNNs can distinguish a richer set of graphs, they are also observed to suffer from higher generalization error. This work provides a theoretical explanation for this trade-off by linking expressivity and generalization through the lens of coloring algorithms. Specifically, we show that the number of equivalence classes induced by WL colorings directly bounds the GNNs Rademacher complexity – a key data-dependent measure of generalization. Our analysis reveals that greater expressivity leads to higher complexity and thus weaker generalization guarantees. Furthermore, we prove that the Rademacher complexity is stable under perturbations in the color counts across different samples, ensuring robustness to sampling variability across datasets. Importantly, our framework is not restricted to message-passing GNNs or 1-WL, but extends to arbitrary GNN architectures and expressivity measures that partition graphs into equivalence classes. These results unify the study of expressivity and generalization in GNNs, providing a principled understanding of why increasing expressive power often comes at the cost of generalization.

Method: Combination of diverse reaction scoring strategies based on machine learning models and existing chemical databases, capturing different classes of hallucinations.

Result: RetroTrim is the sole method that successfully filters out hallucinated reactions and produces the highest number of high-quality paths overall on challenging drug-like targets.

Conclusion: The approach provides reliable retrosynthesis for drug-like targets, and the authors release benchmark targets and evaluation protocol to inspire further research in this domain.

Abstract: Retrosynthesis is one of the domains transformed by the rise of generative models, and it is one where the problem of nonsensical or erroneous outputs (hallucinations) is particularly insidious: reliable assessment of synthetic plans is time-consuming, with automatic methods lacking. In this work, we present RetroTrim, a retrosynthesis system that successfully avoids nonsensical plans on a set of challenging drug-like targets. Compared to common baselines in the field, our system is not only the sole method that succeeds in filtering out hallucinated reactions, but it also results in the highest number of high-quality paths overall. The key insight behind RetroTrim is the combination of diverse reaction scoring strategies, based on machine learning models and existing chemical databases. We show that our scoring strategies capture different classes of hallucinations by analyzing them on a dataset of labeled retrosynthetic intermediates. This approach formed the basis of our winning solution to the Standard Industries $1 million Retrosynthesis Challenge. To measure the performance of retrosynthesis systems, we propose a novel evaluation protocol for reactions and synthetic paths based on a structured review by expert chemists. Using this protocol, we compare systems on a set of 32 novel targets, curated to reflect recent trends in drug structures. While the insights behind our methodology are broadly applicable to retrosynthesis, our focus is on targets in the drug-like domain. By releasing our benchmark targets and the details of our evaluation protocol, we hope to inspire further research into reliable retrosynthesis.

Method: Proposed Y-shaped generative flows with velocity-powered objective using sublinear exponent (0-1), instantiated as scalable neural ODE training objective that encourages joint mass movement along shared pathways.

Result: Y-flows recover hierarchy-aware structure, improve distributional metrics over strong flow-based baselines, and reach targets with fewer integration steps on synthetic, image, and biology datasets.

Conclusion: Y-shaped flows provide a more efficient and structured approach to generative modeling by leveraging shared pathways before branching, outperforming traditional independent trajectory methods.

Abstract: Modern continuous-time generative models often induce V-shaped transport: each sample travels independently along nearly straight trajectories from prior to data, overlooking shared structure. We introduce Y-shaped generative flows, which move probability mass together along shared pathways before branching to target-specific endpoints. Our formulation is based on novel velocity-powered objective with a sublinear exponent (between zero and one). this concave dependence rewards joint and fast mass movement. Practically, we instantiate the idea in a scalable neural ODE training objective. On synthetic, image, and biology datasets, Y-flows recover hierarchy-aware structure, improve distributional metrics over strong flow-based baselines, and reach targets with fewer integration steps.

Method: Theoretical analysis of Mamba’s ICL on low-dimensional nonlinear target functions (single-index models), proving it can extract relevant features directly from context examples through gradient-based pretraining.

Result: Mamba achieves test-time sample complexity that improves upon linear Transformers (which behave like kernel methods) and is comparable to nonlinear Transformers, surpassing the CSQ lower bound and achieving near-optimal rates.

Conclusion: Mamba’s nonlinear gating mechanism is crucial for feature extraction, enabling both computational efficiency and high performance in in-context learning tasks.

Abstract: Mamba, a recently proposed linear-time sequence model, has attracted significant attention for its computational efficiency and strong empirical performance. However, a rigorous theoretical understanding of its underlying mechanisms remains limited. In this work, we provide a theoretical analysis of Mamba’s in-context learning (ICL) capability by focusing on tasks defined by low-dimensional nonlinear target functions. Specifically, we study in-context learning of a single-index model $y \approx g_*(\langle \boldsymbol{\beta}, \boldsymbol{x} \rangle)$, which depends on only a single relevant direction $\boldsymbol{\beta}$, referred to as feature. We prove that Mamba, pretrained by gradient-based methods, can achieve efficient ICL via test-time feature learning, extracting the relevant direction directly from context examples. Consequently, we establish a test-time sample complexity that improves upon linear Transformers – analyzed to behave like kernel methods – and is comparable to nonlinear Transformers, which have been shown to surpass the Correlational Statistical Query (CSQ) lower bound and achieve near information-theoretically optimal rate in previous works. Our analysis reveals the crucial role of the nonlinear gating mechanism in Mamba for feature extraction, highlighting it as the fundamental driver behind Mamba’s ability to achieve both computational efficiency and high performance.

Method: Combines physics-informed Graph Neural Networks (GNNs) with extreme-value analysis. Uses Graph Attention Network with LSTM (Attention-LSTM) that applies attention mechanism with edge features from orographic-precipitation physics. Performs Peak-Over-Threshold mapping using Spatial Season-aware Generalized Pareto Distribution method.

Result: Outperforms well-established baselines across most regions, including extreme-prone areas. Remains strongly competitive with state-of-the-art. Improves extreme-event prediction compared to operational forecasting system SEAS5 and produces fine-resolution maps for decision-making.

Conclusion: The proposed method provides practical enhancement for long-term water management by improving extreme rainfall forecasting and offering explainable predictions through teleconnections and physics-informed approaches.

Abstract: Accurate rainfall forecasting, particularly for extreme events, remains a significant challenge in climatology and the Earth system. This paper presents novel physics-informed Graph Neural Networks (GNNs) combined with extreme-value analysis techniques to improve gauge-station rainfall predictions across Thailand. The model leverages a graph-structured representation of gauge stations to capture complex spatiotemporal patterns, and it offers explainability through teleconnections. We preprocess relevant climate indices that potentially influence regional rainfall. The proposed Graph Attention Network with Long Short-Term Memory (Attention-LSTM) applies the attention mechanism using initial edge features derived from simple orographic-precipitation physics formulation. The embeddings are subsequently processed by LSTM layers. To address extremes, we perform Peak-Over-Threshold (POT) mapping using the novel Spatial Season-aware Generalized Pareto Distribution (GPD) method, which overcomes limitations of traditional machine-learning models. Experiments demonstrate that our method outperforms well-established baselines across most regions, including areas prone to extremes, and remains strongly competitive with the state of the art. Compared with the operational forecasting system SEAS5, our real-world application improves extreme-event prediction and offers a practical enhancement to produce fine-resolution maps that support decision-making in long-term water management.

Method: Used a cultural evolutionary agent-based model and a large-scale behavioral experiment (N=1,243) where participants combined items into novel innovations, testing conditions with and without access to semantic knowledge.

Result: Semantic knowledge and social learning interact synergistically to enhance innovation. Participants without semantic knowledge performed no better than chance even with social learning, relying on shallow exploration strategies.

Conclusion: Semantic knowledge is a key cognitive process enabling human cumulative culture by providing the necessary scaffolding for meaningful innovation.

Abstract: Cumulative cultural evolution enables human societies to generate increasingly complex knowledge and technology over generations. While social learning transmits innovations between individuals and generations, the cognitive processes that generate these innovations remain poorly understood. Here, we demonstrate that semantic knowledge-structured associations between concepts and their functions-provides cognitive scaffolding for cumulative innovation by guiding exploration toward plausible and meaningful actions. We tested this hypothesis using a cultural evolutionary agent-based model and a large-scale behavioural experiment (N = 1,243), in which individuals performed a task requiring the combination of items into novel innovations. Across both approaches, semantic knowledge and social learning interact synergistically to enhance innovation. Behaviorally, participants without access to semantic knowledge performed no better than chance, even when social learning was available, and relied on shallow exploration strategies. These findings suggest that semantic knowledge is a key cognitive process enabling human cumulative culture.

Method: KVCOMM reuses KV-caches and aligns cache offsets by referencing an anchor pool that stores observed cache deviations under varying prefixes, with the pool maintained and updated online for dynamic adaptation.

Result: KVCOMM achieves over 70% reuse rate across diverse multi-agent workloads and up to 7.8x speedup in five-agent settings, reducing TTFT from ~430 ms to ~55 ms.

Conclusion: The proposed KVCOMM framework effectively addresses KV-cache offset variance in multi-agent systems, enabling significant performance improvements through efficient cache reuse while maintaining output quality.

Abstract: Multi-agent large language model (LLM) systems are increasingly adopted for complex language processing tasks that require communication and coordination among agents. However, these systems often suffer substantial overhead from repeated reprocessing of overlapping contexts across agents. In typical pipelines, once an agent receives a message from its predecessor, the full context-including prior turns-must be reprocessed from scratch, leading to inefficient processing. While key-value (KV) caching is an effective solution for avoiding redundant computation in single-agent settings where prefixes remain unchanged, it cannot be directly reused in multi-agent scenarios due to diverging prefixes introduced by agent-specific context extensions. We identify that the core challenge lies in the offset variance of KV-caches across agents. To address this, we propose KVCOMM, a training-free framework that enables efficient prefilling in multi-agent inference by reusing KV-caches and aligning cache offsets of overlapping contexts under diverse prefix contexts. KVCOMM estimates and adjusts KV-caches for shared content by referencing a pool of cached examples-termed anchors-that store observed cache deviations under varying prefixes. The anchor pool is maintained and updated online, allowing dynamic adaptation to distinct user requests and context structures. KVCOMM achieves over 70% reuse rate across diverse multi-agent workloads, including retrieval-augmented generation, math reasoning, and collaborative coding tasks, all without quality degradation. Particularly, when each fully-connected agent receives 1K input tokens with 512 prefix tokens and 512 output tokens under a five-agent setting, KVCOMM achieves up to 7.8x speedup compared to the standard prefill pipeline, reducing TTFT from ~430 ms to ~55 ms.

Method: Proposing the development of cooperative AI agents that work alongside humans to overcome decision-making bottlenecks in scientific workflows.

Result: Identified that realizing such cooperative agents will require progress in both AI capabilities and supporting infrastructure.

Conclusion: Cooperative agents are essential to augment human roles and enable truly autonomous scientific discovery, requiring advancements in AI and infrastructure.

Abstract: As data-driven methods, artificial intelligence (AI), and automated workflows accelerate scientific tasks, we see the rate of discovery increasingly limited by human decision-making tasks such as setting objectives, generating hypotheses, and designing experiments. We postulate that cooperative agents are needed to augment the role of humans and enable autonomous discovery. Realizing such agents will require progress in both AI and infrastructure.

Method: Decentralized peer-to-peer framework using altruism points, multi-agent reinforcement learning (MADDPG) for dynamic ride-matching, game-theoretic equilibrium for fairness guarantees, and population model for long-term balance.

Result: ARS reduces travel distance and emissions, increases vehicle utilization, and promotes equitable participation compared to no-sharing and optimization-based baselines, using real-world NYC taxi data.

Conclusion: ARS establishes a scalable, community-driven alternative to conventional ride-sharing that aligns individual behavior with collective urban sustainability goals.

Abstract: Urban mobility faces persistent challenges of congestion and fuel consumption, specifically when people choose a private, point-to-point commute option. Profit-driven ride-sharing platforms prioritize revenue over fairness and sustainability. This paper introduces Altruistic Ride-Sharing (ARS), a decentralized, peer-to-peer mobility framework where participants alternate between driver and rider roles based on altruism points rather than monetary incentives. The system integrates multi-agent reinforcement learning (MADDPG) for dynamic ride-matching, game-theoretic equilibrium guarantees for fairness, and a population model to sustain long-term balance. Using real-world New York City taxi data, we demonstrate that ARS reduces travel distance and emissions, increases vehicle utilization, and promotes equitable participation compared to both no-sharing and optimization-based baselines. These results establish ARS as a scalable, community-driven alternative to conventional ride-sharing, aligning individual behavior with collective urban sustainability goals.

Method: Proposes AOAD-MAT with Transformer-based actor-critic architecture that incorporates action decision sequences into learning. Uses a novel MARL architecture with a subtask for predicting next agent to act, integrated into PPO-based loss function to maximize sequential decision-making advantage.

Result: Extensive experiments on StarCraft Multi-Agent Challenge and Multi-Agent MuJoCo benchmarks show AOAD-MAT outperforms existing MAT and other baseline models.

Conclusion: The proposed AOAD-MAT model effectively improves MARL performance by explicitly considering and adjusting agent action order, demonstrating the importance of sequential decision-making in multi-agent systems.

Abstract: Multi-agent reinforcement learning focuses on training the behaviors of multiple learning agents that coexist in a shared environment. Recently, MARL models, such as the Multi-Agent Transformer (MAT) and ACtion dEpendent deep Q-learning (ACE), have significantly improved performance by leveraging sequential decision-making processes. Although these models can enhance performance, they do not explicitly consider the importance of the order in which agents make decisions. In this paper, we propose an Agent Order of Action Decisions-MAT (AOAD-MAT), a novel MAT model that considers the order in which agents make decisions. The proposed model explicitly incorporates the sequence of action decisions into the learning process, allowing the model to learn and predict the optimal order of agent actions. The AOAD-MAT model leverages a Transformer-based actor-critic architecture that dynamically adjusts the sequence of agent actions. To achieve this, we introduce a novel MARL architecture that cooperates with a subtask focused on predicting the next agent to act, integrated into a Proximal Policy Optimization based loss function to synergistically maximize the advantage of the sequential decision-making. The proposed method was validated through extensive experiments on the StarCraft Multi-Agent Challenge and Multi-Agent MuJoCo benchmarks. The experimental results show that the proposed AOAD-MAT model outperforms existing MAT and other baseline models, demonstrating the effectiveness of adjusting the AOAD order in MARL.

Method: Developed Foragax toolkit using JAX library for end-to-end vectorized and differentiable simulations, providing tools for custom agent dynamics, control policies, sensor models, and boundary conditions.

Result: The toolkit successfully simulates thousands of agents foraging in common environments with hardware acceleration and supports dynamic agent population changes during simulations.

Conclusion: Foragax provides a general-purpose, scalable solution for multi-agent foraging simulations that can also be extended to model various other multi-agent scenarios beyond foraging.

Abstract: Foraging for resources is a ubiquitous activity conducted by living organisms in a shared environment to maintain their homeostasis. Modelling multi-agent foraging in-silico allows us to study both individual and collective emergent behaviour in a tractable manner. Agent-based modelling has proven to be effective in simulating such tasks, though scaling the simulations to accommodate large numbers of agents with complex dynamics remains challenging. In this work, we present Foragax, a general-purpose, scalable, hardware-accelerated, multi-agent foraging toolkit. Leveraging the JAX library, our toolkit can simulate thousands of agents foraging in a common environment, in an end-to-end vectorized and differentiable manner. The toolkit provides agent-based modelling tools to model various foraging tasks, including options to design custom spatial and temporal agent dynamics, control policies, sensor models, and boundary conditions. Further, the number of agents during such simulations can be increased or decreased based on custom rules. While applied to foraging, the toolkit can also be used to model and simulate a wide range of other multi-agent scenarios.

Method: Created SwarmBench with five MAS coordination tasks (Pursuit, Synchronization, Foraging, Flocking, Transport) in a 2D grid environment where agents rely solely on local sensory input and local communication.

Result: Zero-shot evaluations of leading LLMs show significant task-dependent performance variations. While some rudimentary coordination occurs, current LLMs struggle with robust long-range planning and adaptive strategy formation under decentralized uncertainty.

Conclusion: Assessing LLMs under swarm-like constraints is crucial for understanding their utility in decentralized intelligent systems. SwarmBench is released as an open toolkit to foster reproducible research into LLM-based MAS coordination.

Abstract: Large Language Models (LLMs) show potential for complex reasoning, yet their capacity for emergent coordination in Multi-Agent Systems (MAS) when operating under strict swarm-like constraints-limited local perception and communication-remains largely unexplored. Existing benchmarks often do not fully capture the unique challenges of decentralized coordination when agents operate with incomplete spatio-temporal information. To bridge this gap, we introduce SwarmBench, a novel benchmark designed to systematically evaluate the swarm intelligence capabilities of LLMs acting as decentralized agents. SwarmBench features five foundational MAS coordination tasks (Pursuit, Synchronization, Foraging, Flocking, Transport) within a configurable 2D grid environment, forcing agents to rely solely on local sensory input ($k\times k$ view) and local communication. We propose metrics for coordination effectiveness and analyze emergent group dynamics. Zero-shot evaluations of leading LLMs (e.g., deepseek-v3, o4-mini) reveal significant task-dependent performance variations. While some rudimentary coordination is observed, our results indicate that current LLMs significantly struggle with robust long-range planning and adaptive strategy formation under the uncertainty inherent in these decentralized scenarios. Assessing LLMs under such swarm-like constraints is crucial for understanding their utility in future decentralized intelligent systems. We release SwarmBench as an open, extensible toolkit-built on a customizable physical system-providing environments, prompts, evaluation scripts, and comprehensive datasets. This aims to foster reproducible research into LLM-based MAS coordination and the theoretical underpinnings of emergent collective behavior under severe informational decentralization. Our code repository is available at https://github.com/x66ccff/swarmbench.

Method: Proposes ESG-Net with two modules: Early Semantics Interaction (ESI) for multi-stage semantic guidance through multi-modal early fusion and classification losses, and Mixture of Dependency Experts (MoDE) for adaptive extraction of event dependencies using multiple serial experts.

Result: Significantly surpasses state-of-the-art methods while greatly reducing parameters and computational load.

Conclusion: The proposed multi-stage semantic guidance and multi-event relationship modeling effectively improve event localization by enabling hierarchical semantic understanding and adaptive extraction of event dependencies.

Abstract: Dense audio-visual event localization (DAVE) aims to identify event categories and locate the temporal boundaries in untrimmed videos. Most studies only employ event-related semantic constraints on the final outputs, lacking cross-modal semantic bridging in intermediate layers. This causes modality semantic gap for further fusion, making it difficult to distinguish between event-related content and irrelevant background content. Moreover, they rarely consider the correlations between events, which limits the model to infer concurrent events among complex scenarios. In this paper, we incorporate multi-stage semantic guidance and multi-event relationship modeling, which respectively enable hierarchical semantic understanding of audio-visual events and adaptive extraction of event dependencies, thereby better focusing on event-related information. Specifically, our eventaware semantic guided network (ESG-Net) includes a early semantics interaction (ESI) module and a mixture of dependency experts (MoDE) module. ESI applys multi-stage semantic guidance to explicitly constrain the model in learning semantic information through multi-modal early fusion and several classification loss functions, ensuring hierarchical understanding of event-related content. MoDE promotes the extraction of multi-event dependencies through multiple serial mixture of experts with adaptive weight allocation. Extensive experiments demonstrate that our method significantly surpasses the state-of-the-art methods, while greatly reducing parameters and computational load. Our code will be released on https://github.com/uchiha99999/ESG-Net.

Method: Systematic review of ASR evolution covering: foundational end-to-end paradigms (CTC, attention-based encoder-decoder, RNN-T); architectural shifts to Transformer/Conformer models; training paradigm progression from supervised learning to self-supervised learning (SSL) with foundation models like wav2vec 2.0 and large-scale weakly supervised models like Whisper.

Result: The survey details how ASR has evolved to achieve unprecedented robustness through massive data diversity and reduced reliance on transcribed data, while capturing long-range dependencies with high computational efficiency through modern architectures.

Conclusion: The paper outlines open challenges and future research directions for ASR, emphasizing the need to address streaming inference, on-device efficiency, and ethical considerations of fairness and robustness in real-world deployment.

Abstract: Automatic Speech Recognition (ASR) has undergone a profound transformation over the past decade, driven by advances in deep learning. This survey provides a comprehensive overview of the modern era of ASR, charting its evolution from traditional hybrid systems, such as Gaussian Mixture Model-Hidden Markov Models (GMM-HMMs) and Deep Neural Network-HMMs (DNN-HMMs), to the now-dominant end-to-end neural architectures. We systematically review the foundational end-to-end paradigms: Connectionist Temporal Classification (CTC), attention-based encoder-decoder models, and the Recurrent Neural Network Transducer (RNN-T), which established the groundwork for fully integrated speech-to-text systems. We then detail the subsequent architectural shift towards Transformer and Conformer models, which leverage self-attention to capture long-range dependencies with high computational efficiency. A central theme of this survey is the parallel revolution in training paradigms. We examine the progression from fully supervised learning, augmented by techniques like SpecAugment, to the rise of self-supervised learning (SSL) with foundation models such as wav2vec 2.0, which drastically reduce the reliance on transcribed data. Furthermore, we analyze the impact of largescale, weakly supervised models like Whisper, which achieve unprecedented robustness through massive data diversity. The paper also covers essential ecosystem components, including key datasets and benchmarks (e.g., LibriSpeech, Switchboard, CHiME), standard evaluation metrics (e.g., Word Error Rate), and critical considerations for real-world deployment, such as streaming inference, on-device efficiency, and the ethical imperatives of fairness and robustness. We conclude by outlining open challenges and future research directions.

Method: Proposes HyWA-PVAD that uses a hypernetwork to modify weights of selected layers in a standard VAD model, allowing speaker adaptation without changing the core architecture.

Result: Consistent improvements in PVAD performance compared to baseline conditioning techniques, with increased mean average precision.

Conclusion: HyWA-PVAD effectively enables speaker conditioning while maintaining the original VAD architecture, offering both performance gains and practical deployment advantages.

Abstract: Personalized Voice Activity Detection (PVAD) systems activate only in response to a specific target speaker by incorporating speaker embeddings from enrollment utterances. Unlike existing methods that require architectural changes, such as FiLM layers, our approach employs a hypernetwork to modify the weights of a few selected layers within a standard voice activity detection (VAD) model. This enables speaker conditioning without changing the VAD architecture, allowing the same VAD model to adapt to different speakers by updating only a small subset of the layers. We propose HyWA-PVAD, a hypernetwork weight adaptation method, and evaluate it against multiple baseline conditioning techniques. Our comparison shows consistent improvements in PVAD performance. HyWA also offers practical advantages for deployment by preserving the core VAD architecture. Our new approach improves the current conditioning techniques in two ways: i) increases the mean average precision, ii) simplifies deployment by reusing the same VAD architecture.

Method: Proposes a dual-head architecture with: (1) diffusion head generating continuous speech representations frame-level autoregressively, (2) retaining original LM head for multitasking and speech control, (3) masked training for exposure bias, and (4) two-stage training scheme with frozen LM in second stage for stable optimization.

Result: Achieves state-of-the-art autoregressive performance on LibriSpeech test-clean with WER of 1.95%, speaker similarity of 0.54, and UTMOS of 4.00. Two-stage training yields 46% relative WER reduction over one-stage baseline.

Conclusion: The combination of autoregressive modeling with continuous-token diffusion and two-stage training procedure is effective for high-quality TTS in multimodal LLMs, preserving acoustic details while maintaining multitasking capability.

Abstract: Unified architectures in multimodal large language models (MLLM) have shown promise in handling diverse tasks within a single framework. In the text-to-speech (TTS) task, current MLLM-based approaches rely on discrete token representations, which disregard the inherently continuous nature of speech and can lead to loss of fine-grained acoustic information.In this work, we investigate the TTS within the MLLM paradigm using continuous speech representations. We design a dual-head architecture and implement two complementary training strategies for a robust model. (1) A diffusion head generating continuous speech representations is added on the MLLM, which is on frame-level and strictly autoregressive. (2) The original language model head is retained to preserve multitask capability and to control the start and end of speech synthesis. (3) Masked training is employed to address exposure bias in autoregressive decoding. (4) To stabilize optimization, we propose a two-stage scheme where the LM is frozen in the second stage, ensuring the diffusion head learns from a fixed input distribution. Evaluations on LibriSpeech(PC) test-clean show that our approach achieves state-of-the-art autoregressive performance, with a WER of 1.95%, speaker similarity of 0.54, and UTMOS of 4.00. The two-stage training yields a 46% relative WER reduction over the one-stage training baseline. These results highlight the effectiveness of combining autoregressive modeling with continuous-token diffusion, supported by a two-stage training procedure.

Method: Uses EnCodec architecture with five training tasks: clean reconstruction, reverberated reconstruction, dereverberation, and two acoustic teleportation variants. Analyzes impact of temporal downsampling on acoustic embeddings.

Result: Achieved substantial quality improvement with ScoreQ of 3.03 vs 2.44 for prior methods. Found temporal downsampling significantly degrades performance. Acoustic embeddings correlate with RT60 and cluster by room, while speech embeddings cluster by speaker.

Conclusion: The approach effectively disentangles speech content from acoustic environment characteristics, enabling successful acoustic teleportation with demonstrated quality improvements and proper clustering behavior.

Abstract: This paper presents an approach for acoustic teleportation by disentangling speech content from acoustic environment characteristics in neural audio codec representations. Acoustic teleportation transfers room characteristics between speech recordings while preserving content and speaker identity. We build upon previous work using the EnCodec architecture, achieving substantial objective quality improvements with non-intrusive ScoreQ scores of 3.03, compared to 2.44 for prior methods. Our training strategy incorporates five tasks: clean reconstruction, reverberated reconstruction, dereverberation, and two variants of acoustic teleportation. We demonstrate that temporal downsampling of the acoustic embedding significantly degrades performance, with even 2x downsampling resulting in a statistically significant reduction in quality. The learned acoustic embeddings exhibit strong correlations with RT60. Effective disentanglement is demonstrated using t-SNE clustering analysis, where acoustic embeddings cluster by room while speech embeddings cluster by speaker.

Method: Uses DualHyp framework with RelPrompt mechanism - an LLM composes independent N-best hypotheses from ASR and VSR models, guided by temporal reliability prompts that dynamically switch focus between modalities.

Result: Achieves up to 57.7% error rate gain on LRS2 benchmark under various corruption scenarios, significantly outperforming single-stream GER approaches that achieve only 10% gain.

Conclusion: The DualHyp framework with RelPrompt guidance effectively combines audio-visual modalities for superior speech recognition performance, with code and dataset released for research community.

Abstract: This paper introduces a new paradigm for generative error correction (GER) framework in audio-visual speech recognition (AVSR) that reasons over modality-specific evidences directly in the language space. Our framework, DualHyp, empowers a large language model (LLM) to compose independent N-best hypotheses from separate automatic speech recognition (ASR) and visual speech recognition (VSR) models. To maximize the effectiveness of DualHyp, we further introduce RelPrompt, a noise-aware guidance mechanism that provides modality-grounded prompts to the LLM. RelPrompt offers the temporal reliability of each modality stream, guiding the model to dynamically switch its focus between ASR and VSR hypotheses for an accurate correction. Under various corruption scenarios, our framework attains up to 57.7% error rate gain on the LRS2 benchmark over standard ASR baseline, contrary to single-stream GER approaches that achieve only 10% gain. To facilitate research within our DualHyp framework, we release the code and the dataset comprising ASR and VSR hypotheses at https://github.com/sungnyun/dualhyp.

Method: Proposes a framework with tri-axial Transformer that jointly models temporal, frequency, and spatial channel dependencies, using CLAP embeddings for unified audio-text conditioning via FiLM, and a label-free data pipeline for training.

Result: Experiment results demonstrate high separation quality, showing efficacy of multimodal conditioning and tri-axial modeling.

Conclusion: Establishes a new paradigm for high-fidelity spatial audio separation in immersive applications.

Abstract: Query-based audio source extraction seeks to recover a target source from a mixture conditioned on a query. Existing approaches are largely confined to single-channel audio, leaving the spatial information in multi-channel recordings underexploited. We introduce a query-based spatial audio source extraction framework for recovering dry target signals from first-order ambisonics (FOA) mixtures. Our method accepts either an audio prompt or a text prompt as condition input, enabling flexible end-to-end extraction. The core of our proposed model lies in a tri-axial Transformer that jointly models temporal, frequency, and spatial channel dependencies. The model uses contrastive language-audio pretraining (CLAP) embeddings to enable unified audio-text conditioning via feature-wise linear modulation (FiLM). To eliminate costly annotations and improve generalization, we propose a label-free data pipeline that dynamically generates spatial mixtures and corresponding targets for training. The result of our experiment with high separation quality demonstrates the efficacy of multimodal conditioning and tri-axial modeling. This work establishes a new paradigm for high-fidelity spatial audio separation in immersive applications.

Method: Multi-scale residual codec architecture encoding speech into four distinct streams, combined with a two-stage language model for TTS synthesis that leverages the codec’s disentangled representations.

Result: Achieves state-of-the-art Word Error Rate and superior speaker similarity in TTS compared to larger models, with minimal data requirements. Also highly effective for voice conversion with independent manipulation of speaker timbre and prosody.

Conclusion: The proposed codec provides an efficient solution for speech generation that balances low bitrate with high fidelity, while its inherent disentanglement properties enable flexible control over speech attributes for various applications.

Abstract: Audio codecs are a critical component of modern speech generation systems. This paper introduces a low-bitrate, multi-scale residual codec that encodes speech into four distinct streams: semantic, timbre, prosody, and residual. This architecture achieves high-fidelity speech reconstruction at competitive low bitrates while demonstrating an inherent ability for information disentanglement. We construct a two-stage language model for text-to-speech (TTS) synthesis using this codec, which, despite its lightweight design and minimal data requirements, achieves a state-of-the-art Word Error Rate (WER) and superior speaker similarity compared to several larger models. Furthermore, the codec’s design proves highly effective for voice conversion, enabling independent manipulation of speaker timbre and prosody. Our inference code, pre-trained models, and audio samples are available at https://github.com/herbertLJY/MSRCodec.

Method: Inject artifact-correlated watermarks associated with deepfake systems rather than pre-assigned bitstrings, allowing detectors to leverage both injected watermarks and intrinsic deepfake artifacts.

Result: FakeMark improves generalization to cross-dataset samples and maintains high accuracy under various distortions where conventional methods fail.

Conclusion: FakeMark provides a robust solution for deepfake speech attribution by combining watermarking with artifact analysis, addressing limitations of both classifier-based and conventional watermarking approaches.

Abstract: Deepfake speech attribution remains challenging for existing solutions. Classifier-based solutions often fail to generalize to domain-shifted samples, and watermarking-based solutions are easily compromised by distortions like codec compression or malicious removal attacks. To address these issues, we propose FakeMark, a novel watermarking framework that injects artifact-correlated watermarks associated with deepfake systems rather than pre-assigned bitstring messages. This design allows a detector to attribute the source system by leveraging both injected watermark and intrinsic deepfake artifacts, remaining effective even if one of these cues is elusive or removed. Experimental results show that FakeMark improves generalization to cross-dataset samples where classifier-based solutions struggle and maintains high accuracy under various distortions where conventional watermarking-based solutions fail.

Method: The approach hierarchically models tissue architecture at local and global scales. Locally, it constructs patch-level graphs from cellular orientation and extracts features. Globally, it learns image-level graphs with sparse, biologically meaningful connections using first-order approximate bilevel optimization, optimizing the graph structure based on classification performance.

Result: On Extended CRC dataset: 97.33 ± 1.15% accuracy for three-class colorectal cancer grading and 98.33 ± 0.58% for binary classification. On melanoma dataset: 96.27 ± 0.74% accuracy for tumor-lymphocyte ROI classification.

Conclusion: ABiG-Net effectively captures both local structural information and global contextual relationships, enhancing interpretability and achieving superior performance in histopathology image analysis tasks.

Abstract: The structural and spatial arrangements of cells within tissues represent their functional states, making graph-based learning highly suitable for histopathology image analysis. Existing methods often rely on fixed graphs with predefined edges, limiting their ability to capture the true biological complexity of tissue interactions. In this work, we propose ABiG-Net (Approximate Bilevel Optimization for Graph Structure Learning via Neural Networks), a novel framework designed to learn optimal interactions between patches within whole slide images (WSI) or large regions of interest (ROI) while simultaneously learning discriminative node embeddings for the downstream image classification task. Our approach hierarchically models the tissue architecture at local and global scales. At the local scale, we construct patch-level graphs from cellular orientation within each patch and extract features to quantify local structures. At the global scale, we learn an image-level graph that captures sparse, biologically meaningful connections between patches through a first-order approximate bilevel optimization strategy. The learned global graph is optimized in response to classification performance, capturing the long-range contextual dependencies across the image. By unifying local structural information with global contextual relationships, ABiG-Net enhances interpretability and downstream performance. Experiments on two histopathology datasets demonstrate its effectiveness: on the Extended CRC dataset, ABiG-Net achieves 97.33 $\pm$ 1.15 % accuracy for three-class colorectal cancer grading and 98.33 $\pm$ 0.58 % for binary classification; on the melanoma dataset, it attains 96.27 $\pm$ 0.74 % for tumor-lymphocyte ROI classification.

Method: Uses digital twins (digital replicas of users) with supervised machine learning. The system includes a data processing pipeline, XGBoost-based digital twin models for individual users, and unified models for engagement prediction.

Result: Reduces user engagement prediction error by up to 5.8% compared to non-user-aware models. Identifies features that maximize engagement, providing an average engagement increase of up to 8.6%.

Conclusion: DIGITWISE demonstrates the importance of modeling personal user sensitivities in video streaming, enabling better engagement prediction and optimization of content delivery.

Abstract: As the popularity of video streaming entertainment continues to grow, understanding how users engage with the content and react to its changes becomes a critical success factor for every stakeholder. User engagement, i.e., the percentage of video the user watches before quitting, is central to customer loyalty, content personalization, ad relevance, and A/B testing. This paper presents DIGITWISE, a digital twin-based approach for modeling adaptive video streaming engagement. Traditional adaptive bitrate (ABR) algorithms assume that all users react similarly to video streaming artifacts and network issues, neglecting individual user sensitivities. DIGITWISE leverages the concept of a digital twin, a digital replica of a physical entity, to model user engagement based on past viewing sessions. The digital twin receives input about streaming events and utilizes supervised machine learning to predict user engagement for a given session. The system model consists of a data processing pipeline, machine learning models acting as digital twins, and a unified model to predict engagement. DIGITWISE employs the XGBoost model in both digital twins and unified models. The proposed architecture demonstrates the importance of personal user sensitivities, reducing user engagement prediction error by up to 5.8% compared to non-user-aware models. Furthermore, DIGITWISE can optimize content provisioning and delivery by identifying the features that maximize engagement, providing an average engagement increase of up to 8.6%.

Method: The authors present a semantic-enabled holographic video communication architecture with three key technologies: semantic sampling, joint semantic-channel coding, and semantic-aware transmission, validated through two use cases.

Result: The proposed methods demonstrate performance gains in holographic video communication systems, as shown through the presented use cases.

Conclusion: The paper identifies potential research topics to advance the realization of semantic-enabled holographic video communications, paving the way for future developments in this emerging field.

Abstract: Holographic video communication is considered a paradigm shift in visual communications, becoming increasingly popular for its ability to offer immersive experiences. This article provides an overview of holographic video communication and outlines the requirements of a holographic video communication system. Particularly, following a brief review of semantic com- munication, an architecture for a semantic-enabled holographic video communication system is presented. Key technologies, including semantic sampling, joint semantic-channel coding, and semantic-aware transmission, are designed based on the proposed architecture. Two related use cases are presented to demonstrate the performance gain of the proposed methods. Finally, potential research topics are discussed to pave the way for the realization of semantic-enabled holographic video communications.

Method: RCWA framework uses redundancy-aware coding to remove wireless channel redundancy and encode only source-relevant information into bits, plus Lagrangian multiplier method for controllable variable-rate coding that encodes features into expected rates while minimizing distortion.

Result: Superior rate-distortion performance and robustness compared to baselines, achieving up to 0.7dB PSNR gain over neural network methods and 4dB gain over digital baselines on CIFAR-10 dataset.

Conclusion: RCWA validates potential for efficient wired resource utilization in hybrid transmission scenarios, addressing the gap in existing DJSCC approaches for hybrid wireless-wired networks.

Abstract: Deep joint source-channel coding (DJSCC) has emerged as a robust alternative to traditional separate coding for communications through wireless channels. Existing DJSCC approaches focus primarily on point-to-point wireless communication scenarios, while neglecting end-to-end communication efficiency in hybrid wireless-wired networks such as 5G and 6G communication systems. Considerable redundancy in DJSCC symbols against wireless channels becomes inefficient for long-distance wired transmission. Furthermore, DJSCC symbols must adapt to the varying transmission rate of the wired network to avoid congestion. In this paper, we propose a novel framework designed for efficient wired transmission of DJSCC symbols within hybrid wireless-wired networks, namely Rate-Controllable Wired Adaptor (RCWA). RCWA achieves redundancy-aware coding for DJSCC symbols to improve transmission efficiency, which removes considerable redundancy present in DJSCC symbols for wireless channels and encodes only source-relevant information into bits. Moreover, we leverage the Lagrangian multiplier method to achieve controllable and continuous variable-rate coding, which can encode given features into expected rates, thereby minimizing end-to-end distortion while satisfying given constraints. Extensive experiments on diverse datasets demonstrate the superior RD performance and robustness of RCWA compared to existing baselines, validating its potential for wired resource utilization in hybrid transmission scenarios. Specifically, our method can obtain peak signal-to-noise ratio gain of up to 0.7dB and 4dB compared to neural network-based methods and digital baselines on CIFAR-10 dataset, respectively.

Method: Uses a lightweight local model that processes current frames and fuses features from a heavyweight remote model that computes features from past frames.

Result: On BDD100K driving dataset, Dedelayed improves semantic segmentation accuracy over local-only and remote-only baselines across all delays beyond 33ms. At 100ms delay, it improves by 6.4 mIoU over local inference and 9.8 mIoU over remote inference without additional delay.

Conclusion: Dedelayed provides clear advantages for real-time tasks by sustaining accuracy under longer delays and higher-motion scenes while maintaining alignment with current world state.

Abstract: Remote inference allows lightweight devices to leverage powerful cloud models. However, communication network latency makes predictions stale and unsuitable for real-time tasks. To address this, we introduce Dedelayed, a delay-corrective method that mitigates arbitrary remote inference delays, allowing the local device to produce low-latency outputs in real time. Our method employs a lightweight local model that processes the current frame and fuses in features that a heavyweight remote model computes from past frames. On video from the BDD100K driving dataset, Dedelayed improves semantic segmentation accuracy over the stronger of the local-only and remote-only baselines across all realistic communication network delays beyond 33 ms. Without incurring additional delay, it improves accuracy by 6.4 mIoU compared to fully local inference and 9.8 mIoU compared to remote inference, for a round-trip delay of 100 ms. The advantage grows under longer delays and higher-motion scenes, as delay-mitigated split inference sustains accuracy more effectively, providing clear advantages for real-time tasks that must remain aligned with the current world state.

Method: Uses ternary-quantized linear layers with multi-query attention, task-aware distillation from a high-capacity teacher, and custom CUDA kernels for efficient deployment on Jetson Orin Nano.

Result: Achieves 86% diagnostic accuracy (vs 89% SOTA) on MedMNIST across 12 datasets, with 43x model size reduction, 39x memory traffic reduction, and 16.8 ms inference at 41x higher energy efficiency (183.62 GOPs/J).

Conclusion: Provides a practical route for extreme-precision medical imaging ViTs deployable on edge devices, bridging the gap between algorithmic advances and clinical deployment.

Abstract: Vision Transformers (ViTs) have demonstrated strong capabilities in interpreting complex medical imaging data. However, their significant computational and memory demands pose challenges for deployment in real-time, resource-constrained mobile and wearable devices used in clinical environments. We introduce, BiTMedViT, a new class of Edge ViTs serving as medical AI assistants that perform structured analysis of medical images directly on the edge. BiTMedViT utilizes ternary- quantized linear layers tailored for medical imaging and com- bines a training procedure with multi-query attention, preserving stability under ternary weights with low-precision activations. Furthermore, BiTMedViT employs task-aware distillation from a high-capacity teacher to recover accuracy lost due to extreme quantization. Lastly, we also present a pipeline that maps the ternarized ViTs to a custom CUDA kernel for efficient memory bandwidth utilization and latency reduction on the Jetson Orin Nano. Finally, BiTMedViT achieves 86% diagnostic accuracy (89% SOTA) on MedMNIST across 12 datasets, while reducing model size by 43x, memory traffic by 39x, and enabling 16.8 ms inference at an energy efficiency up to 41x that of SOTA models at 183.62 GOPs/J on the Orin Nano. Our results demonstrate a practical and scientifically grounded route for extreme-precision medical imaging ViTs deployable on the edge, narrowing the gap between algorithmic advances and deployable clinical tools.

Method: Integrates 3D Mamba Coordinate Attention module for enhanced cost aggregation with position-sensitive attention and long-range spatial dependency modeling, plus High-Frequency Disparity Optimization module that refines disparity near tissue boundaries using wavelet domain processing.

Result: Achieves state-of-the-art matching accuracy on SCARED and SERV-CT datasets with real-time inference speed of 42 FPS.

Conclusion: RRESM successfully addresses the challenges of stereo matching in endoscopic imaging by combining efficient attention mechanisms with high-frequency optimization, enabling accurate real-time depth acquisition for robotic surgery.

Abstract: Real-time acquisition of accurate scene depth is essential for automated robotic minimally invasive surgery. Stereo matching with binocular endoscopy can provide this depth information. However, existing stereo matching methods, designed primarily for natural images, often struggle with endoscopic images due to fuzzy tissue boundaries and typically fail to meet real-time requirements for high-resolution endoscopic image inputs. To address these challenges, we propose \textbf{RRESM}, a real-time stereo matching method tailored for endoscopic images. Our approach integrates a 3D Mamba Coordinate Attention module that enhances cost aggregation through position-sensitive attention maps and long-range spatial dependency modeling via the Mamba block, generating a robust cost volume without substantial computational overhead. Additionally, we introduce a High-Frequency Disparity Optimization module that refines disparity predictions near tissue boundaries by amplifying high-frequency details in the wavelet domain. Evaluations on the SCARED and SERV-CT datasets demonstrate state-of-the-art matching accuracy with a real-time inference speed of 42 FPS. The code is available at https://github.com/Sonne-Ding/RRESM.

Method: Developed an end-to-end compression framework that retains text-specific features for OCR, with an encoder operating at half the computational cost of OCR modules. Also explored general-purpose encoders’ capacity to preserve hidden semantics under extreme compression.

Result: Significant improvements in text extraction accuracy at low bitrates, even outperforming OCR on uncompressed images. Demonstrated that semantic information can persist despite severe compression through learned enhancement and recognition modules.

Conclusion: The framework bridges text-oriented compression and general-purpose semantic preservation in machine-centered image coding, showing that compact, visually degraded representations can retain recoverable meaning for automated understanding.

Abstract: Traditional image compression methods aim to reconstruct images for human perception, prioritizing visual fidelity over task relevance. In contrast, Coding for Machines focuses on preserving information essential for automated understanding. Building on this principle, we present an end-to-end compression framework that retains text-specific features for Optical Character Recognition (OCR). The encoder operates at roughly half the computational cost of the OCR module, making it suitable for resource-limited devices. When on-device OCR is infeasible, images can be efficiently compressed and later decoded to recover textual content. Experiments show significant improvements in text extraction accuracy at low bitrates, even outperforming OCR on uncompressed images. We further extend this study to general-purpose encoders, exploring their capacity to preserve hidden semantics under extreme compression. Instead of optimizing for visual fidelity, we examine whether compact, visually degraded representations can retain recoverable meaning through learned enhancement and recognition modules. Results demonstrate that semantic information can persist despite severe compression, bridging text-oriented compression and general-purpose semantic preservation in machine-centered image coding.

Method: Developed a polar transformer architecture combining polar representations and transformers with a convolutional attention mechanism that preserves rotational symmetry, applied to particle-level denoising.

Result: Achieved up to 2× reduction in mean squared error at SNR of 0.02 on simulated datasets, enabling optimal clustering, alignment, and denoising.

Conclusion: The approach suggests new opportunities for data-driven reconstruction in cryo-EM and related tomographic modalities through neural networks that respect problem symmetries.

Abstract: Many imaging modalities involve reconstruction of unknown objects from collections of noisy projections related by random rotations. In one of these modalities, cryogenic electron microscopy (cryo-EM), the extremely low signal-to-noise ratio (SNR) makes integration of information from multiple images crucial. Existing approaches to cryo-EM processing, however, either rely on handcrafted priors or apply deep learning only on select portions of the pipeline, such as particle picking, micrograph denoising, or refinement. A fully end-to-end reconstruction approach requires a neural network architecture that integrates information from multiple images while respecting the rotational symmetry of the measurement process. In this work, we introduce the polar transformer, a new neural network architecture that combines polar representations and transformers along with a convolutional attention mechanism that preserves the rotational symmetry of the problem. We apply it to the particle-level denoising problem, where it is able to learn discriminative features in the images, enabling optimal clustering, alignment, and denoising. On simulated datasets, this achieves up to a $2\times$ reduction in mean squared error (MSE) at a signal-to-noise ratio (SNR) of $0.02$, suggesting new opportunities for data-driven approaches to reconstruction in cryo-EM and related tomographic modalities.