Method: Define a five-way behavioral taxonomy, train a classifier to estimate ethical entropy S(t) from model transcripts, measure entropy dynamics across stress tests for base and instruction-tuned variants of four frontier models, and estimate effective alignment work rate gamma_eff.

Result: Base models show sustained ethical entropy growth, while instruction-tuned variants suppress drift and reduce ethical entropy by approximately 80%. The framework enables estimation of alignment work rates and provides a monitoring pipeline for run-time oversight.

Conclusion: The proposed ethical entropy framework provides operational tools for dynamic safety monitoring of LLMs, demonstrating that instruction tuning effectively counters ethical entropy drift, and enables real-time alerts when entropy exceeds stability thresholds.

Abstract: Large language model safety is usually assessed with static benchmarks, but key failures are dynamic: value drift under distribution shift, jailbreak attacks, and slow degradation of alignment in deployment. Building on a recent Second Law of Intelligence that treats ethical entropy as a state variable which tends to increase unless countered by alignment work, we make this framework operational for large language models. We define a five-way behavioral taxonomy, train a classifier to estimate ethical entropy S(t) from model transcripts, and measure entropy dynamics for base and instruction-tuned variants of four frontier models across stress tests. Base models show sustained entropy growth, while tuned variants suppress drift and reduce ethical entropy by roughly eighty percent. From these trajectories we estimate an effective alignment work rate gamma_eff and embed S(t) and gamma_eff in a monitoring pipeline that raises alerts when entropy drift exceeds a stability threshold, enabling run-time oversight of value drift.

Method: 1) Demonstrates vulnerability of existing EaaS watermarks to paraphrasing attacks during imitation attacks. 2) Proposes WET (Watermarking EaaS with Linear Transformation) - a novel watermarking technique that applies linear transformation to embeddings and verifies watermarks by reversing the transformation and comparing similarity between recovered and original embeddings.

Result: 1) Paraphrasing effectively bypasses current state-of-the-art EaaS watermarks across various attack setups and datasets. 2) WET demonstrates robustness against paraphrasing attacks with near-perfect verifiability. 3) Ablation studies validate the significance of WET components and hyperparameters.

Conclusion: Existing EaaS watermarks are vulnerable to paraphrasing attacks, but the proposed WET technique provides a robust defense with near-perfect verifiability against such attacks, offering better protection for EaaS intellectual property.

Abstract: Large Language Models (LLMs) have demonstrated exceptional capabilities in natural language understanding and generation. Based on these LLMs, businesses have started to provide Embeddings-as-a-Service (EaaS), offering feature extraction capabilities (in the form of text embeddings) that benefit downstream natural language processing tasks. However, prior research has demonstrated that EaaS is vulnerable to imitation attacks, where an attacker clones the service’s model in a black-box manner without access to the model’s internal workings. In response, watermarks have been added to the text embeddings to protect the intellectual property of EaaS providers by allowing them to check for model ownership. This thesis focuses on defending against imitation attacks by investigating EaaS watermarks. To achieve this goal, we unveil novel attacks and propose and validate new watermarking techniques. Firstly, we show that existing EaaS watermarks can be removed through paraphrasing the input text when attackers clone the model during imitation attacks. Our study illustrates that paraphrasing can effectively bypass current state-of-the-art EaaS watermarks across various attack setups (including different paraphrasing techniques and models) and datasets in most instances. This demonstrates a new vulnerability in recent EaaS watermarking techniques. Subsequently, as a countermeasure, we propose a novel watermarking technique, WET (Watermarking EaaS with Linear Transformation), which employs linear transformation of the embeddings. Watermark verification is conducted by applying a reverse transformation and comparing the similarity between recovered and original embeddings. We demonstrate its robustness against paraphrasing attacks with near-perfect verifiability. We conduct detailed ablation studies to assess the significance of each component and hyperparameter in WET.

Method: Reasoning Dependency Generation (RDG) framework that automatically constructs balanced reasoning QA pairs by explicitly (un)modeling dependencies between choices, evidences, and justifications, generating large-scale datasets with Contextual Dependency Data and Dependency Decouple Data.

Result: LLMs fine-tuned on RDG-generated data show 81.5% improvement in memory-based experiments and 94.3% improvement in evaluation-based experiments, while maintaining similar performance on standard BBQ benchmarks.

Conclusion: RDG pioneers an approach for addressing cognitive biases in LLMs and contributes to developing more reliable AI-assisted decision support systems by mitigating choice-supportive bias through reasoning data generation.

Abstract: Recent studies have demonstrated that some Large Language Models exhibit choice-supportive bias (CSB) when performing evaluations, systematically favoring their chosen options and potentially compromising the objectivity of AI-assisted decision making. While existing debiasing approaches primarily target demographic and social biases, methods for addressing cognitive biases in LLMs remain largely unexplored. In this work, we present the first solution to address CSB through Reasoning Dependency Generation (RDG), a novel framework for generating unbiased reasoning data to mitigate choice-supportive bias through fine-tuning. RDG automatically constructs balanced reasoning QA pairs, explicitly (un)modeling the dependencies between choices, evidences, and justifications. Our approach is able to generate a large-scale dataset of QA pairs across domains, incorporating Contextual Dependency Data and Dependency Decouple Data. Experiments show that LLMs fine-tuned on RDG-generated data demonstrate a 81.5% improvement in memory-based experiments and 94.3% improvement in the evaluation-based experiment, while maintaining similar performance on standard BBQ benchmarks. This work pioneers an approach for addressing cognitive biases in LLMs and contributes to the development of more reliable AI-assisted decision support systems.

Method: Two main methodologies: Sentence Linking and Entity Linking. Developed an open-source tool incorporating both approaches. Created annotated datasets for evaluating occupation and qualification representation. Explored generative large language models for the classification task.

Result: Developed a publicly available open-source tool for labor classification. Created annotated datasets for benchmarking. Advanced state of the art in job entity extraction. Provided computational infrastructure for analyzing work and skills in digital labor markets.

Conclusion: The study contributes to improving labor market analysis by providing tools and datasets for linking job vacancies to European frameworks, enabling more sophisticated examination of skills and qualifications in the digital economy.

Abstract: This study investigates the potential of language models to improve the classification of labor market information by linking job vacancy texts to two major European frameworks: the European Skills, Competences, Qualifications and Occupations (ESCO) taxonomy and the European Qualifications Framework (EQF). We examine and compare two prominent methodologies from the literature: Sentence Linking and Entity Linking. In support of ongoing research, we release an open-source tool, incorporating these two methodologies, designed to facilitate further work on labor classification and employment discourse. To move beyond surface-level skill extraction, we introduce two annotated datasets specifically aimed at evaluating how occupations and qualifications are represented within job vacancy texts. Additionally, we examine different ways to utilize generative large language models for this task. Our findings contribute to advancing the state of the art in job entity extraction and offer computational infrastructure for examining work, skills, and labor market narratives in a digitally mediated economy. Our code is made publicly available: https://github.com/tabiya-tech/tabiya-livelihoods-classifier

Method: InvertiTune combines controlled data generation pipeline with supervised fine-tuning. Pipeline extracts subgraphs from large knowledge bases, filters noise, uses LLMs to generate natural text descriptions (easier for LLMs than direct KG generation). This creates realistic datasets for SFT of lightweight models for single-shot KG construction.

Result: Outperforms larger non-fine-tuned LLMs and state-of-the-art Text2KG approaches on CE12k dataset. Shows stronger cross-dataset generalization on CrossEval-1200 test set from three established benchmarks and CE12k.

Conclusion: Realistic, high-quality training data is crucial for advancing efficient and high-performing Text2KG systems. InvertiTune demonstrates effectiveness of combining controlled data generation with supervised fine-tuning.

Abstract: Large Language Models (LLMs) have revolutionized the ability to understand and generate text, enabling significant progress in automatic knowledge graph construction from text (Text2KG). Many Text2KG methods, however, rely on iterative LLM prompting, making them computationally expensive and prone to overlooking complex relations distributed throughout the text. To address these limitations, we propose InvertiTune, a framework that combines a controlled data generation pipeline with supervised fine-tuning (SFT). Within this framework, the data-generation pipeline systematically extracts subgraphs from large knowledge bases, applies noise filtering, and leverages LLMs to generate corresponding natural text descriptions, a task more aligned with LLM capabilities than direct KG generation from text. This pipeline enables generating datasets composed of longer texts paired with larger KGs that better reflect real-world scenarios compared to existing benchmarks, thus supporting effective SFT of lightweight models for single-shot KG construction. Experimental results on CE12k, a dataset generated using the introduced pipeline, show that InvertiTune outperforms larger non-fine-tuned LLMs as well as state-of-the-art Text2KG approaches, while also demonstrating stronger cross-dataset generalization on CrossEval-1200, a test set created from three established benchmark datasets and CE12k. These findings highlight the importance of realistic, high-quality training data for advancing efficient and high-performing Text2KG systems.

Method: Train a lightweight causal language model that detects and parses explanations in political text, returning structured cause-effect pairs for downstream analysis.

Result: The method enables studying causal explanations at scale, demonstrates modest annotation requirements, shows generalizability across contexts, and achieves accuracy comparable to human coding.

Conclusion: The framework provides a systematic approach to analyzing political explanations, addressing a gap in political science methodology and enabling large-scale study of causal reasoning in political discourse.

Abstract: Explanations are a fundamental element of how people make sense of the political world. Citizens routinely ask and answer questions about why events happen, who is responsible, and what could or should be done differently. Yet despite their importance, explanations remain an underdeveloped object of systematic analysis in political science, and existing approaches are fragmented and often issue-specific. I introduce a framework for detecting and parsing explanations in political text. To do this, I train a lightweight causal language model that returns a structured data set of causal claims in the form of cause-effect pairs for downstream analysis. I demonstrate how causal explanations can be studied at scale, and show the method’s modest annotation requirements, generalizability, and accuracy relative to human coding.

Method: Randomized Masked Fine-Tuning (RMFT) - a novel privacy-preserving fine-tuning technique that reduces PII memorization while minimizing performance impact.

Result: RMFT achieves 80.81% reduction in Total Extraction Rate and 80.17% reduction in Seen Extraction Rate compared to baseline, outperforming deduplication methods with only 5.73% perplexity increase.

Conclusion: RMFT effectively addresses PII memorization risks in LLMs with minimal performance degradation, and MaxTER provides a Pareto-optimal framework for evaluating privacy-utility tradeoffs.

Abstract: The current literature on memorization in Natural Language Models, especially Large Language Models (LLMs), poses severe security and privacy risks, as models tend to memorize personally identifying information (PIIs) from training data. We introduce Randomized Masked Fine-Tuning (RMFT), a novel privacy-preserving fine-tuning technique that reduces PII memorization while minimizing performance impact. Using the Enron Email Dataset, we demonstrate that RMFT achieves an 80.81% reduction in Total Extraction Rate and 80.17% reduction in Seen Extraction Rate compared to baseline fine-tuning, outperforming deduplication methods while maintaining only a 5.73% increase in perplexity. We present MaxTER, a Pareto-optimal evaluation framework for assessing privacy-utility tradeoffs, and show the performance of RMFT vs Deduplication by Area Under The Response Curve (AURC) metric.

Method: Developed an LLM-assisted annotation pipeline that automatically labels topic, genre, and discourse-pragmatic functions across 3,691 code-switched sentences, integrated demographic metadata from the Miami Bilingual Corpus, and enriched Spanish-Guaraní dataset with new topic annotations.

Result: Revealed systematic links between gender, language dominance, and discourse function in Miami data, and a clear diglossic division between formal Guaraní and informal Spanish in Paraguayan texts, replicating and extending earlier sociolinguistic observations with quantitative evidence.

Conclusion: LLMs can reliably recover interpretable sociolinguistic patterns at scale, advancing computational methods for bilingual research and providing corpus-scale quantitative evidence that validates and extends traditional sociolinguistic observations.

Abstract: This study presents an LLM-assisted annotation pipeline for the sociolinguistic and topical analysis of bilingual discourse in two typologically distinct contexts: Spanish-English and Spanish-Guaraní. Using large language models, we automatically labeled topic, genre, and discourse-pragmatic functions across a total of 3,691 code-switched sentences, integrated demographic metadata from the Miami Bilingual Corpus, and enriched the Spanish-Guaraní dataset with new topic annotations. The resulting distributions reveal systematic links between gender, language dominance, and discourse function in the Miami data, and a clear diglossic division between formal Guaraní and informal Spanish in Paraguayan texts. These findings replicate and extend earlier interactional and sociolinguistic observations with corpus-scale quantitative evidence. The study demonstrates that large language models can reliably recover interpretable sociolinguistic patterns traditionally accessible only through manual annotation, advancing computational methods for cross-linguistic and low-resource bilingual research.

Method: Created PERCS dataset with biomedical abstracts paired with summaries for four personas (Laypersons, Premedical Students, Non-medical Researchers, Medical Experts), physician-reviewed for factual accuracy and persona alignment using detailed error taxonomy.

Result: Technical validation shows clear differences in readability, vocabulary, and content depth across personas. Benchmarking four LLMs with automatic metrics assessing comprehensiveness, readability, and faithfulness establishes baseline results.

Conclusion: PERCS dataset supports research on persona-specific communication and controllable biomedical summarization, addressing the need for audience-targeted medical text simplification.

Abstract: Automatic medical text simplification plays a key role in improving health literacy by making complex biomedical research accessible to diverse readers. However, most existing resources assume a single generic audience, overlooking the wide variation in medical literacy and information needs across user groups. To address this limitation, we introduce PERCS (Persona-guided Controllable Summarization), a dataset of biomedical abstracts paired with summaries tailored to four personas: Laypersons, Premedical Students, Non-medical Researchers, and Medical Experts. These personas represent different levels of medical literacy and information needs, emphasizing the need for targeted, audience-specific summarization. Each summary in PERCS was reviewed by physicians for factual accuracy and persona alignment using a detailed error taxonomy. Technical validation shows clear differences in readability, vocabulary, and content depth across personas. Along with describing the dataset, we benchmark four large language models on PERCS using automatic evaluation metrics that assess comprehensiveness, readability, and faithfulness, establishing baseline results for future research. The dataset, annotation guidelines, and evaluation materials are publicly available to support research on persona-specific communication and controllable biomedical summarization.

Method: Introduces Idea-Gated Transformer architecture with an auxiliary “Idea Head” trained to predict bag-of-words distribution for future context windows, creating a latent “Concept Vector” that actively gates the main vocabulary during generation. Uses differentiable gating mechanism to suppress semantically irrelevant tokens in real-time.

Result: On WikiText-103, the Idea-Gated model achieves comparable validation perplexity to standard GPT-2 baseline but exhibits significantly superior Domain Retention. The gating mechanism successfully locks generation into specific semantic clusters (e.g., Finance, Science) and resists associative drift.

Conclusion: The Idea-Gated Transformer offers a parameter-efficient path toward more controllable language modeling by addressing topic drift through separation of semantic planning from syntactic generation, enabling better domain retention without sacrificing perplexity.

Abstract: Autoregressive Language Models (LLMs) trained on Next-Token Prediction (NTP) often suffer from Topic Drift'' where the generation wanders away from the initial prompt due to a reliance on local associations rather than global planning \citep{holtzman2019curious}. While scaling model size mitigates this \citep{brown2020language}, the fundamental myopia of the NTP objective remains. In this work, we introduce the Idea-Gated Transformer, a novel architecture that separates semantic planning from syntactic generation. We introduce an auxiliary Idea Head’’ trained to predict the bag-of-words distribution for a future context window, creating a latent ``Concept Vector’’ that actively gates the main vocabulary during generation. We propose a differentiable gating mechanism that suppresses semantically irrelevant tokens, effectively pruning the search space in real-time. Experiments on WikiText-103 demonstrate that while the Idea-Gated model achieves comparable validation perplexity to a standard GPT-2 baseline, it exhibits significantly superior Domain Retention. Qualitative and quantitative analysis reveals that the gating mechanism successfully locks generation into specific semantic clusters (e.g., Finance, Science) and resists associative drift, offering a parameter-efficient path toward more controllable language modeling.

Method: HBLR integrates confidence-aware symbolic translation (converting only high-confidence spans to FOL while keeping uncertain content in natural language) with hypothesis-driven backward reasoning (assuming conclusion is true and recursively verifying premises). Includes translation reflection for semantic fidelity and reasoning reflection for correcting flawed inference steps.

Result: Extensive experiments on five reasoning benchmarks demonstrate that HBLR consistently outperforms strong baselines in both accuracy and efficiency.

Conclusion: The proposed HBLR framework effectively addresses limitations of forward reasoning by simulating human deductive thinking through backward logical reasoning with confidence-aware symbolic representation, resulting in more reliable and efficient logical reasoning.

Abstract: Logical reasoning is a core challenge in natural language understanding and a fundamental capability of artificial intelligence, underpinning scientific discovery, mathematical theorem proving, and complex decision-making. Despite the remarkable progress of large language models (LLMs), most current approaches still rely on forward reasoning paradigms, generating step-by-step rationales from premises to conclusions. However, such methods often suffer from redundant inference paths, hallucinated steps, and semantic drift, resulting in inefficient and unreliable reasoning. In this paper, we propose a novel framework, Hypothesis-driven Backward Logical Reasoning (HBLR). The core idea is to integrate confidence-aware symbolic translation with hypothesis-driven backward reasoning. In the translation phase, only high-confidence spans are converted into logical form, such as First-Order Logic (FOL), while uncertain content remains in natural language. A translation reflection module further ensures semantic fidelity by evaluating symbolic outputs and reverting lossy ones back to text when necessary. In the reasoning phase, HBLR simulates human deductive thinking by assuming the conclusion is true and recursively verifying its premises. A reasoning reflection module further identifies and corrects flawed inference steps, enhancing logical coherence. Extensive experiments on five reasoning benchmarks demonstrate that HBLR consistently outperforms strong baselines in both accuracy and efficiency.

Method: Hon uses a recursive framework where Query and Key vectors are outputs of inner attention loops, allowing tokens to aggregate global context before final attention computation. It employs parameter-efficient weight-sharing across recursive steps.

Result: Theoretical analysis shows Hon breaks the linear bottleneck of standard attention. Empirically, it outperforms standard Transformers on multiple benchmarks.

Conclusion: Hon provides enhanced representational power through higher-order attention while maintaining parameter efficiency, offering a promising alternative to standard Transformer architectures.

Abstract: Transformers have achieved significant success across various domains, relying on self-attention to capture dependencies. However, the standard first-order attention mechanism is often limited by a low-rank bottleneck, struggling to capture intricate, multi-hop relationships within a single layer. In this paper, we propose the \textbf{Higher-Order Attention Network (Hon)}, a novel architecture designed to enhance representational power through a recursive framework. Unlike standard approaches that use static linear projections for Queries and Keys, Hon dynamically refines these representations via nested self-attention mechanisms. Specifically, the Query and Key vectors are themselves outputs of inner attention loops, allowing tokens to aggregate global context and model high-order correlations \textit{prior} to the final attention computation. We enforce a parameter-efficient weight-sharing strategy across recursive steps, ensuring that this enhanced expressivity incurs $\mathcal{O}(1)$ additional parameters. We provide theoretical analysis demonstrating that our method breaks the linear bottleneck of standard attention. Empirically, Hon outperforms standard Transformers on multiple benchmarks.

Method: Collected the Portal Dialogue Corpus from Portal 2 co-op gameplay, comprising 11.5 hours of spoken dialogue (24.5K utterances), with player videos, audio, transcripts, game state data, and manual/automatic annotations.

Result: Identified unique linguistic phenomena rarely found in existing dialogue corpora: complex spatial reference, clarification and repair mechanisms, and ad-hoc convention formation during collaborative problem-solving.

Conclusion: The publicly released corpus enables future research on language use in complex, situated, collaborative problem-solving scenarios, addressing gaps in current dialogue datasets.

Abstract: Cooperative video games, where multiple participants must coordinate by communicating and reasoning under uncertainty in complex environments, yield a rich source of language data. We collect the Portal Dialogue Corpus: a corpus of 11.5 hours of spoken human dialogue in the co-op mode of the popular Portal 2 virtual puzzle game, comprising 24.5K total utterances. We analyze player language and behavior, identifying a number of linguistic phenomena that rarely appear in most existing chitchat or task-oriented dialogue corpora, including complex spatial reference, clarification and repair, and ad-hoc convention formation. To support future analyses of language use in complex, situated, collaborative problem-solving scenarios, we publicly release the corpus, which comprises player videos, audio, transcripts, game state data, and both manual and automatic annotations of language data.

Method: Separate low-rank matrices into two groups: magnitude group (controls whether/how far to update parameters with ReLU) and direction group (decides forward/backward movement with sign function). This better simulates full fine-tuning parameter updates.

Result: Consistently outperforms LoRA and its state-of-the-art variants with same number of trainable parameters across NLG, NLU, and commonsense reasoning tasks on GPT-2, RoBERTa, DeBERTa, and LLaMA-1/2/3 models.

Conclusion: Dual LoRA effectively incorporates inductive bias into LoRA to improve performance by better simulating full fine-tuning parameter updates while maintaining parameter efficiency.

Abstract: Low-rank adaptation (LoRA) is one of the most popular methods among parameter-efficient fine-tuning (PEFT) methods to adapt pre-trained large language models (LLMs) to specific downstream tasks. However, the model trained based on LoRA often has an unsatisfactory performance due to its low-rank assumption. In this paper, we propose a novel method called Dual LoRA to improve the performance by incorporating an inductive bias into the original LoRA. Specifically, we separate low-rank matrices into two groups: the magnitude group to control whether or not and how far we should update a parameter and the direction group to decide whether this parameter should move forward or backward, to better simulate the parameter updating process of the full fine-tuning based on gradient-based optimization algorithms. We show that this can be simply achieved by adding a ReLU function to the magnitude group and a sign function to the direction group. We conduct several experiments over a wide range of NLP tasks, including natural language generation (NLG), understanding (NLU), and commonsense reasoning datasets on GPT-2, RoBERTa, DeBERTa, and LLaMA-1/2/3 as baseline models. The results show that we consistently outperform LoRA and its state-of-the-art variants with the same number of trainable parameters.

Method: 1) Active pretraining: learns a unified reasoning policy to identify informative content from pretraining corpus and predict it via RL. 2) Self-supervised learning: directly pretrains reasoners from 3-30B base models on Wikipedia corpus using RL without verifiable labels or supervised fine-tuning. 3) Verification scaling: tackles increasingly challenging masked spans to enhance reasoning abilities.

Result: PretrainZero improves Qwen3-4B-Base by 8.43, 5.96, and 10.60 points on MMLU-Pro, SuperGPQA, and math average benchmarks respectively. The pretrained models can also serve as reasoning foundation models for downstream RLVR tasks.

Conclusion: PretrainZero successfully extends RL to general pretraining, breaking the verification data-wall for general reasoning and demonstrating significant improvements in reasoning benchmarks while enabling foundation models for downstream tasks.

Abstract: Mimicking human behavior to actively learning from general experience and achieve artificial general intelligence has always been a human dream. Recent reinforcement learning (RL) based large-thinking models demonstrate impressive expert-level abilities, i.e., software and math, but still rely heavily on verifiable rewards in specific domains, placing a significant bottleneck to extend the performance boundary of general reasoning capabilities. In this work, we propose PretrainZero, a reinforcement active learning framework built on the pretraining corpus to extend RL from domain-specific post-training to general pretraining. PretrainZero features the following characteristics: 1) Active pretraining: inspired by the active learning ability of humans, PretrainZero learns a unified reasoning policy to actively identify reasonable and informative contents from pretraining corpus, and reason to predict these contents by RL. 2) Self-supervised learning: without any verifiable labels, pretrained reward models, or supervised fine-tuning, we directly pretrain reasoners from 3 to 30B base models on the general Wikipedia corpus using RL, significantly breaking the verification data-wall for general reasoning. 3) Verification scaling: by tackling increasingly challenging masked spans, PretrainZero substantially enhances the general reasoning abilities of pretrained base models. In reinforcement pretraining, PretrainZero improves Qwen3-4B-Base for 8.43, 5.96 and 10.60 on MMLU-Pro, SuperGPQA and math average benchmarks. In post-training, the pretrained models can also serve as reasoning foundation models for downstream RLVR tasks.

Method: Investigates Top-k Attention mechanism: 1) Exact Top-k Decoding retains only top-k most similar Keys to Query as context window; 2) Native Top-k Attention training ensures consistency between training and inference; 3) Studies approximate Top-k algorithms; 4) Theoretical analysis from entropy perspective.

Result: Top-k Decoding achieves performance comparable to or surpassing full attention on HELMET and LongBench v2. Native Top-k training further enhances model performance. Positive correlation between downstream task performance and approximation fidelity. Models with Top-k Attention SFT show entropy reduction in downstream tasks.

Conclusion: Top-k Attention effectively reduces LLM inference costs while maintaining performance. Consistency between training and inference is crucial. Low-entropy states are better adapted to Top-k Decoding, providing theoretical foundation for the mechanism’s effectiveness.

Abstract: Large Language Models (LLMs) are increasingly prevalent in the field of long-context modeling, however, their inference computational costs have become a critical bottleneck hindering the advancement of tasks such as agents and multimodal applications. This report conducts a preliminary investigation into the effectiveness and theoretical mechanisms of the Top-$k$ Attention mechanism during both the decoding and training phases. First, we validate the effectiveness of exact Top-$k$ Decoding through extensive experimentation. Experiments demonstrate that retaining only the pivotal Keys with the highest similarity to the Query as the context window during the decoding stage achieves performance comparable to, or even surpassing, full attention on downstream tasks such as HELMET and LongBench v2. Second, we further explore the native Top-$k$ Attention training strategy. Experiments confirm that ensuring the consistency between training and inference regarding Top-$k$ Attention operations facilitates the further unlocking of Top-$k$ Decoding’s potential, thereby significantly enhancing model performance. Furthermore, considering the high computational complexity of exact Top-$k$ Attention, we investigate the impact of approximate Top-$k$ algorithm precision on downstream tasks. Our research confirms a positive correlation between downstream task performance and approximation fidelity, and we provide statistical evaluations of the Lightning Indexer’s precision within the DeepSeek-V3.2-Exp model. Finally, this report provides a theoretical interpretation from the perspective of Entropy. Experimental observations indicate that models subjected to Top-$k$ Attention SFT exhibit a distinct phenomenon of entropy reduction in downstream tasks, which validates the hypothesis that low-entropy states are better adapted to Top-$k$ Decoding.

Method: Tailored general reasoning strategies to summarization domain, then conducted large-scale comparative study of 8 reasoning strategies and 3 Large Reasoning Models across 8 diverse datasets, assessing both summary quality and faithfulness.

Result: Reasoning is not a universal solution - effectiveness depends on specific strategy and context. Found trade-off between summary quality and factual faithfulness: explicit reasoning improves fluency but reduces factual grounding, while implicit reasoning in LRMs shows opposite pattern. Increasing LRM’s internal reasoning budget doesn’t improve and can even hurt factual consistency.

Conclusion: Effective summarization requires faithful compression rather than creative over-thinking. The study challenges assumptions about reasoning’s universal benefits for summarization and highlights context-dependent trade-offs between quality and faithfulness.

Abstract: While the reasoning capabilities of Large Language Models (LLMs) excel in analytical tasks such as mathematics and code generation, their utility for abstractive summarization remains widely assumed but largely unverified. To bridge this gap, we first tailor general reasoning strategies to the summarization domain. We then conduct a systematic, large scale comparative study of 8 reasoning strategies and 3 Large Reasoning Models (LRMs) across 8 diverse datasets, assessing both summary quality and faithfulness. Our findings show that reasoning is not a universal solution and its effectiveness is highly dependent on the specific strategy and context. Specifically, we observe a trade-off between summary quality and factual faithfulness: explicit reasoning strategies tend to improve fluency at the expense of factual grounding, while implicit reasoning in LRMs exhibits the inverse pattern. Furthermore, increasing an LRM’s internal reasoning budget does not improve, and can even hurt, factual consistency, suggesting that effective summarization demands faithful compression rather than creative over-thinking.

Method: Created INDI-PROP dataset with 1,266 articles on CAA and Farmers’ protest, annotated at three hierarchical levels: ideological bias, event-specific fine-grained narrative frames, and persuasive techniques. Developed two GPT-4o-mini guided frameworks: FANTA (multi-hop reasoning integrating information extraction and contextual framing) and TPTC (two-stage decomposition of persuasive cues).

Result: Both FANTA and TPTC frameworks show substantial improvement over underlying baselines for bias, narrative, and persuasive technique classification tasks on the INDI-PROP dataset.

Conclusion: The paper establishes a comprehensive framework for analyzing propaganda narratives in Indian news media through hierarchical annotation and advanced reasoning models, enabling better understanding of how ideological narratives are constructed and propagated.

Abstract: Narratives are the cognitive and emotional scaffolds of propaganda. They organize isolated persuasive techniques into coherent stories that justify actions, attribute blame, and evoke identification with ideological camps. In this paper, we propose a novel fine-grained narrative classification in biased news articles. We also explore article-bias classification as the precursor task to narrative classification and fine-grained persuasive technique identification. We develop INDI-PROP, the first ideologically grounded fine-grained narrative dataset with multi-level annotation for analyzing propaganda in Indian news media. Our dataset INDI-PROP comprises 1,266 articles focusing on two polarizing socio-political events in recent times: CAA and the Farmers’ protest. Each article is annotated at three hierarchical levels: (i) ideological article-bias (pro-government, pro-opposition, neutral), (ii) event-specific fine-grained narrative frames anchored in ideological polarity and communicative intent, and (iii) persuasive techniques. We propose FANTA and TPTC, two GPT-4o-mini guided multi-hop prompt-based reasoning frameworks for the bias, narrative, and persuasive technique classification. FANTA leverages multi-layered communicative phenomena by integrating information extraction and contextual framing for hierarchical reasoning. On the other hand, TPTC adopts systematic decomposition of persuasive cues via a two-stage approach. Our evaluation suggests substantial improvement over underlying baselines in each case.

Method: Decomposes text into atomic facts and introduces a flexible, schema-free methodology. Incorporates a weighted metric instead of absolute metrics, and proposes a mechanism to control assessment complexity in intricate domains.

Result: Benchmarked on popular general and clinical datasets. Released code to support fact-aware model training in future research.

Conclusion: The proposed interpretable framework addresses limitations of existing factual consistency assessment methods by providing better evaluation with weighted metrics and complexity control, particularly valuable for high-stakes domains.

Abstract: Large Language Models have significantly advanced natural language processing tasks, but remain prone to generating incorrect or misleading but plausible arguments. This issue, known as hallucination, is particularly concerning in high-stakes domains like clinical applications, where factual inaccuracies can have severe consequences. Existing evaluation metrics fail to adequately assess factual consistency and lack interpretability, making diagnosing and mitigating errors difficult. We propose an interpretable framework for factual consistency assessment for in-domain and open-domain texts to address these limitations. Our approach decomposes text into atomic facts and introduces a flexible, schema-free methodology. Unlike previous methods with an absolute metric, we incorporate a weighted metric to enhance factual evaluation. Additionally, we propose a mechanism to control assessment complexity in intricate domains. We benchmark our approach on popular general and clinical datasets and release our code to support fact-aware model training in future research.

Method: Conducted first comprehensive empirical mapping using newly collected survey data from 1,906 Italian-speaking adults, analyzing adoption rates, usage patterns, and digital literacy levels.

Result: Widespread GenAI adoption for work and personal use including sensitive tasks; GenAI replacing other technologies as primary information source despite low user literacy; significant gender divide where women are half as likely to adopt and use it less frequently, especially in older generations.

Conclusion: Need for targeted educational initiatives to improve digital literacy and further investigation into barriers beyond competence that create gender disparities, ensuring equitable participation in AI-driven digital transformation.

Abstract: The rise of Artificial Intelligence (AI) language technologies, particularly generative AI (GenAI) chatbots accessible via conversational interfaces, is transforming digital interactions. While these tools hold societal promise, they also risk widening digital divides due to uneven adoption and low awareness of their limitations. This study presents the first comprehensive empirical mapping of GenAI adoption, usage patterns, and literacy in Italy, based on newly collected survey data from 1,906 Italian-speaking adults. Our findings reveal widespread adoption for both work and personal use, including sensitive tasks like emotional support and medical advice. Crucially, GenAI is supplanting other technologies to become a primary information source: this trend persists despite low user digital literacy, posing a risk as users struggle to recognize errors or misinformation. Moreover, we identify a significant gender divide – particularly pronounced in older generations – where women are half as likely to adopt GenAI and use it less frequently than men. While we find literacy to be a key predictor of adoption, it only partially explains this disparity, suggesting that other barriers are at play. Overall, our data provide granular insights into the multipurpose usage of GenAI, highlighting the dual need for targeted educational initiatives and further investigation into the underlying barriers to equitable participation that competence alone cannot explain.

Method: Developed Hydro-SE Bench containing 4,000 multiple-choice questions covering nine subfields. The benchmark evaluates three aspects: basic conceptual knowledge, engineering application ability, and reasoning/calculation ability. Tested both commercial and small-parameter LLMs.

Result: Commercial LLMs scored 0.74-0.80 accuracy, while small-parameter LLMs scored 0.41-0.68. Models performed well on science-related subfields but struggled with domain-specific knowledge like industry standards and hydraulic structures. Model scaling mainly improved reasoning/calculation abilities.

Conclusion: LLMs have potential in Hydro-SE but need improvement in practical engineering applications. The benchmark provides clear training targets for developers and practical guidance for researchers applying LLMs in Hydro-SE domains.

Abstract: Hydro-Science and Engineering (Hydro-SE) is a critical and irreplaceable domain that secures human water supply, generates clean hydropower energy, and mitigates flood and drought disasters. Featuring multiple engineering objectives, Hydro-SE is an inherently interdisciplinary domain that integrates scientific knowledge with engineering expertise. This integration necessitates extensive expert collaboration in decision-making, which poses difficulties for intelligence. With the rapid advancement of large language models (LLMs), their potential application in the Hydro-SE domain is being increasingly explored. However, the knowledge and application abilities of LLMs in Hydro-SE have not been sufficiently evaluated. To address this issue, we propose the Hydro-SE LLM evaluation benchmark (Hydro-SE Bench), which contains 4,000 multiple-choice questions. Hydro-SE Bench covers nine subfields and enables evaluation of LLMs in aspects of basic conceptual knowledge, engineering application ability, and reasoning and calculation ability. The evaluation results on Hydro-SE Bench show that the accuracy values vary among 0.74 to 0.80 for commercial LLMs, and among 0.41 to 0.68 for small-parameter LLMs. While LLMs perform well in subfields closely related to natural and physical sciences, they struggle with domain-specific knowledge such as industry standards and hydraulic structures. Model scaling mainly improves reasoning and calculation abilities, but there is still great potential for LLMs to better handle problems in practical engineering application. This study highlights the strengths and weaknesses of LLMs for Hydro-SE tasks, providing model developers with clear training targets and Hydro-SE researchers with practical guidance for applying LLMs.

Method: Used functional localization approach inspired by cognitive neuroscience to identify LLM units responsive to 67 English syntactic phenomena across 7 open-weight models. Analyzed cross-lingual patterns in 57 diverse languages.

Result: Different types of syntactic agreement (subject-verb, anaphor, determiner-noun) recruit overlapping sets of units, suggesting agreement constitutes a meaningful functional category. This pattern holds across English, Russian, and Chinese, and structurally similar languages share more units for subject-verb agreement.

Conclusion: Syntactic agreement, a critical marker of syntactic dependencies, constitutes a meaningful category within LLMs’ representational spaces, revealing systematic organization of grammatical knowledge.

Abstract: Large language models (LLMs) can reliably distinguish grammatical from ungrammatical sentences, but how grammatical knowledge is represented within the models remains an open question. We investigate whether different syntactic phenomena recruit shared or distinct components in LLMs. Using a functional localization approach inspired by cognitive neuroscience, we identify the LLM units most responsive to 67 English syntactic phenomena in seven open-weight models. These units are consistently recruited across sentences containing the phenomena and causally support the models’ syntactic performance. Critically, different types of syntactic agreement (e.g., subject-verb, anaphor, determiner-noun) recruit overlapping sets of units, suggesting that agreement constitutes a meaningful functional category for LLMs. This pattern holds in English, Russian, and Chinese; and further, in a cross-lingual analysis of 57 diverse languages, structurally more similar languages share more units for subject-verb agreement. Taken together, these findings reveal that syntactic agreement-a critical marker of syntactic dependencies-constitutes a meaningful category within LLMs’ representational spaces.

Method: Developed an application that uses language technology to assess AI tutor quality, includes software for demonstration and evaluation, and provides model inspection and data visualization capabilities.

Result: Created AITutor-EvalKit, a comprehensive tool that supports learning, collects user feedback and annotations, and serves both education stakeholders and the ACL community.

Conclusion: AITutor-EvalKit provides valuable infrastructure for evaluating AI tutor pedagogical quality, with applications in both educational settings and computational linguistics research.

Abstract: We present AITutor-EvalKit, an application that uses language technology to evaluate the pedagogical quality of AI tutors, provides software for demonstration and evaluation, as well as model inspection and data visualization. This tool is aimed at education stakeholders as well as *ACL community at large, as it supports learning and can also be used to collect user feedback and annotations.

Method: Proposes DZ-TDPO, a non-destructive alignment framework that synergizes conflict-aware dynamic KL constraints with a learnable temporal attention bias to regulate attention patterns without destructive weight updates.

Result: Achieves state-of-the-art win rates (86.2% on Phi-3.5) on Multi-Session Chat dataset with robust zero-shot generalization. Larger models like Qwen2.5-7B achieve near-perfect alignment (99.4% win rate) with negligible perplexity overhead, revealing a “Capacity-Stability Trade-off”.

Conclusion: State Inertia can be alleviated via precise attention regulation rather than destructive weight updates, preserving general capabilities across model scales while achieving superior dialogue performance.

Abstract: Long-context dialogue systems suffer from State Inertia, where static constraints prevent models from resolving conflicts between evolving user intents and established historical context. To address this, we propose DZ-TDPO, a non-destructive alignment framework that synergizes conflict-aware dynamic KL constraints with a learnable temporal attention bias. Experiments on the Multi-Session Chat (MSC) dataset demonstrate that DZ-TDPO achieves state-of-the-art win rates (86.2% on Phi-3.5) while maintaining robust zero-shot generalization. Crucially, our scaling analysis reveals a “Capacity-Stability Trade-off”: while smaller models incur an “alignment tax” (perplexity surge) to overcome historical inertia, the larger Qwen2.5-7B model achieves near-perfect alignment (99.4% win rate) with negligible perplexity overhead. This confirms that TAI can be alleviated via precise attention regulation rather than destructive weight updates, preserving general capabilities (MMLU) across model scales. Code and data are available: https://github.com/lyj20071013/DZ-TDPO

Method: AR-Med uses retrieval-augmented LLM reasoning with verified medical knowledge, knowledge distillation to compress large models into compact student models, and LocalQSMed benchmark for evaluation.

Result: Achieves 93% offline accuracy (24% absolute improvement), significant gains in online relevance and user satisfaction, successfully deployed at scale on medical platforms.

Conclusion: Provides a practical, scalable blueprint for trustworthy LLM-powered healthcare systems that balance accuracy, reliability, and operational efficiency.

Abstract: Accurate and reliable search on online healthcare platforms is critical for user safety and service efficacy. Traditional methods, however, often fail to comprehend complex and nuanced user queries, limiting their effectiveness. Large language models (LLMs) present a promising solution, offering powerful semantic understanding to bridge this gap. Despite their potential, deploying LLMs in this high-stakes domain is fraught with challenges, including factual hallucinations, specialized knowledge gaps, and high operational costs. To overcome these barriers, we introduce \textbf{AR-Med}, a novel framework for \textbf{A}utomated \textbf{R}elevance assessment for \textbf{Med}ical search that has been successfully deployed at scale on the Online Medical Delivery Platforms. AR-Med grounds LLM reasoning in verified medical knowledge through a retrieval-augmented approach, ensuring high accuracy and reliability. To enable efficient online service, we design a practical knowledge distillation scheme that compresses large teacher models into compact yet powerful student models. We also introduce LocalQSMed, a multi-expert annotated benchmark developed to guide model iteration and ensure strong alignment between offline and online performance. Extensive experiments show AR-Med achieves an offline accuracy of over 93%, a 24% absolute improvement over the original online system, and delivers significant gains in online relevance and user satisfaction. Our work presents a practical and scalable blueprint for developing trustworthy, LLM-powered systems in real-world healthcare applications.

Method: Proposes ELBO-based Sequence-level Policy Optimization (ESPO) that treats entire sequence generation as a single action and uses the Evidence Lower Bound (ELBO) as a tractable sequence-level likelihood proxy. Incorporates per-token normalization of importance ratios and robust KL-divergence estimation for stable large-scale training.

Result: ESPO significantly outperforms token-level baselines, achieving dramatic improvements of 20-40 points on the Countdown task while maintaining consistent gains on math and coding benchmarks across mathematical reasoning, coding, and planning tasks.

Conclusion: Sequence-level optimization is established as a principled and empirically effective paradigm for RL in diffusion LLMs, addressing the fundamental mismatch between diffusion generation processes and traditional token-level RL methods.

Abstract: Reinforcement Learning (RL) has proven highly effective for autoregressive language models, but adapting these methods to diffusion large language models (dLLMs) presents fundamental challenges. The core difficulty lies in likelihood approximation: while autoregressive models naturally provide token-level conditional probabilities essential for token-level RL objectives (e.g., GRPO), dLLMs generate sequences through iterative non-autoregressive denoising steps that lack this factorization. To address this fundamental mismatch, we propose ELBO-based Sequence-level Policy Optimization (ESPO), a principled RL framework that treats entire sequence generation as a single action and uses the ELBO as a tractable sequence-level likelihood proxy. Our method incorporates per-token normalization of importance ratios and robust KL-divergence estimation to ensure stable large-scale training. Extensive experiments on mathematical reasoning, coding, and planning tasks demonstrate that ESPO significantly outperforms token-level baselines, achieving dramatic improvements of 20-40 points on the Countdown task, while maintaining consistent gains on math and coding benchmarks. Our approach establishes sequence-level optimization as a principled and empirically effective paradigm for RL in dLLMs. Our code is available at https://github.com/ML-GSAI/ESPO.

Method: Systematically replace harmful keywords with benign tokens across multiple in-context examples preceding a harmful request, causing the internal representation of benign tokens to converge toward harmful semantics through layer-by-layer semantic overwrite.

Result: Achieves 74% attack success rate on Llama-3.3-70B-Instruct with single-sentence context override; broadly transferable across model families; optimization-free; uses interpretability tools to show semantic convergence from early to later layers.

Conclusion: Current LLM alignment strategies are insufficient and should operate at the representation level rather than surface level, as demonstrated by this new attack surface in the latent space of LLMs.

Abstract: We introduce \textbf{Doublespeak}, a simple \emph{in-context representation hijacking} attack against large language models (LLMs). The attack works by systematically replacing a harmful keyword (e.g., \textit{bomb}) with a benign token (e.g., \textit{carrot}) across multiple in-context examples, provided a prefix to a harmful request. We demonstrate that this substitution leads to the internal representation of the benign token converging toward that of the harmful one, effectively embedding the harmful semantics under a euphemism. As a result, superficially innocuous prompts (e.g., How to build a carrot?'') are internally interpreted as disallowed instructions (e.g., How to build a bomb?’’), thereby bypassing the model’s safety alignment. We use interpretability tools to show that this semantic overwrite emerges layer by layer, with benign meanings in early layers converging into harmful semantics in later ones. Doublespeak is optimization-free, broadly transferable across model families, and achieves strong success rates on closed-source and open-source systems, reaching 74% ASR on Llama-3.3-70B-Instruct with a single-sentence context override. Our findings highlight a new attack surface in the latent space of LLMs, revealing that current alignment strategies are insufficient and should instead operate at the representation level.

Method: Adapt DoLa (Decoding by Contrastive Layers) for T5 and FLAN-T5 model families. Conduct layer-by-layer analysis of logit evolution in FLAN-T5 to quantify DoLa’s impact on token output probabilities.

Result: DoLa improves faithfulness of text generation for certain categories of tasks but harms others. The layer-by-layer analysis provides quantitative understanding of how DoLa affects token output probabilities.

Conclusion: First successful implementation of contrastive decoding in encoder-decoder architecture shows mixed results - improves some instruction following tasks while harming others, with detailed analysis explaining the effects.

Abstract: Contrastive decoding is a lightweight and effective inference-time method that improves the quality of text generation in Large Language Models. However, algorithms such as DoLa (Decoding by Contrastive Layers) have only been implemented in decoder-only architectures and studied for their impact on improving factuality. This work adapts DoLa for the T5 and FLAN-T5 model families and evaluates its impact on the models’ instruction following capabilities, which to our knowledge is the first implementation of a contrastive decoding strategy in an encoder-decoder architecture. Our results show that DoLa improves the faithfulness of text generation for certain categories of tasks and harms others. To understand these results, we present a layer-by-layer analysis of logit evolution in a FLAN-T5 model to quantify DoLa’s impact on token output probabilities.

Method: Empirical framework for optimizing LLM performance via prompt engineering. Evaluated five strategies: codebook-guided empirical prompt selection, automatic prompt engineering, persona prompting, chain-of-thought reasoning, and explanatory prompting with zero-shot and few-shot classification.

Result: Persona, chain-of-thought, and explanations don’t fully address performance loss from badly worded prompts. Most influential features are construct definition, task framing, and examples. Best results came from few-shot prompt combining codebook-guided empirical prompt selection with automatic prompt engineering across three constructs and two models.

Conclusion: Researchers should generate and evaluate many prompt variants (human-crafted and automatically generated), select based on empirical performance in training data, and validate in holdout set. This provides practical, systematic, theory-driven method for optimizing LLM prompts when alignment with expert judgment is critical.

Abstract: Due to their architecture and vast pre-training data, large language models (LLMs) demonstrate strong text classification performance. However, LLM output - here, the category assigned to a text - depends heavily on the wording of the prompt. While literature on prompt engineering is expanding, few studies focus on classification tasks, and even fewer address domains like psychology, where constructs have precise, theory-driven definitions that may not be well represented in pre-training data. We present an empirical framework for optimizing LLM performance for identifying constructs in texts via prompt engineering. We experimentally evaluate five prompting strategies –codebook-guided empirical prompt selection, automatic prompt engineering, persona prompting, chain-of-thought reasoning, and explanatory prompting - with zero-shot and few-shot classification. We find that persona, chain-of-thought, and explanations do not fully address performance loss accompanying a badly worded prompt. Instead, the most influential features of a prompt are the construct definition, task framing, and, to a lesser extent, the examples provided. Across three constructs and two models, the classifications most aligned with expert judgments resulted from a few-shot prompt combining codebook-guided empirical prompt selection with automatic prompt engineering. Based on our findings, we recommend that researchers generate and evaluate as many prompt variants as feasible, whether human-crafted, automatically generated, or ideally both, and select prompts and examples based on empirical performance in a training dataset, validating the final approach in a holdout set. This procedure offers a practical, systematic, and theory-driven method for optimizing LLM prompts in settings where alignment with expert judgment is critical.

Method: Fine-tune LLMs on verbalized medical consensus rules and their instantiations to electronic health records. Use Sepsis-3 definition as example. Evaluate both derivation correctness (logical reasoning) and value correctness (prediction accuracy). Combine with time series forecasting models in multimodal setup for irregular clinical data.

Result: Small fine-tuned models outperform larger LLMs using one-shot learning with explicit definitions. Models achieve near-perfect derivation correctness for rules and exceptions on unseen data. Main bottleneck is forecasting sparse, irregular clinical variables, which improves with multimodal integration of time series forecasting.

Conclusion: Teaching LLMs to follow medical consensus guidelines step-by-step improves both prediction accuracy and faithful explanations. The approach enables automatic evaluation of reasoning correctness and shows that fine-tuning on rule instantiations is more effective than prompting or training on general medical texts.

Abstract: Machine learning for early prediction in medicine has recently shown breakthrough performance, however, the focus on improving prediction accuracy has led to a neglect of faithful explanations that are required to gain the trust of medical practitioners. The goal of this paper is to teach LLMs to follow medical consensus guidelines step-by-step in their reasoning and prediction process. Since consensus guidelines are ubiquitous in medicine, instantiations of verbalized medical inference rules to electronic health records provide data for fine-tuning LLMs to learn consensus rules and possible exceptions thereof for many medical areas. Consensus rules also enable an automatic evaluation of the model’s inference process regarding its derivation correctness (evaluating correct and faithful deduction of a conclusion from given premises) and value correctness (comparing predicted values against real-world measurements). We exemplify our work using the complex Sepsis-3 consensus definition. Our experiments show that small fine-tuned models outperform one-shot learning of considerably larger LLMs that are prompted with the explicit definition and models that are trained on medical texts including consensus definitions. Since fine-tuning on verbalized rule instantiations of a specific medical area yields nearly perfect derivation correctness for rules (and exceptions) on unseen patient data in that area, the bottleneck for early prediction is not out-of-distribution generalization, but the orthogonal problem of generalization into the future by forecasting sparsely and irregularly sampled clinical variables. We show that the latter results can be improved by integrating the output representations of a time series forecasting model with the LLM in a multimodal setup.

Method: Analyzed KV information flow revealing values come from bottom layers, keys from both bottom/middle layers. Proposed FusedKV: top-layer KV caches are learnable fusion of most informative bottom/middle layer caches, operating on post-RoPE keys. Also proposed FusedKV-Lite: cross-layer sharing where top-layer KV caches derive from bottom-layer values and middle-layer keys.

Result: Reduces 50% cache memory while achieving lower validation perplexity than standard Transformer decoder across LLMs from 332M to 4B parameters. FusedKV-Lite reduces I/O overhead with slight perplexity increase.

Conclusion: FusedKV establishes a memory-efficient, high-performance architectural alternative to standard Transformer decoders by intelligently fusing KV cache information across layers.

Abstract: Transformer decoders have achieved strong results across tasks, but the memory required for the KV cache becomes prohibitive at long sequence lengths. Although Cross-layer KV Cache sharing (e.g., YOCO, CLA) offers a path to mitigate KV Cache bottleneck, it typically underperforms within-layer methods like GQA. To understand the root cause, we investigate the information flow of keys and values of the top-layers. Our preliminary reveals a clear distribution: values are predominantly derived from the bottom layer, while keys draw more information from both bottom and middle layers. Building upon this, we propose FusedKV, whose top-layer KV caches are a learnable fusion of the most informative ones from the bottom and middle layers. This fusion operates directly on post-RoPE keys, preserving relative positional information without the computational cost of re-applying rotary embeddings. To further improve efficiency, we propose FusedKV-Lite, an cross-layer sharing approach, where top-layer KV caches are directly derived from the bottom-layer values and the middle-layer keys. Compared to FusedKV, FusedKV-Lite reduces I/O overhead at the cost of a slight increase in perplexity. In experiments on LLMs ranging from 332M to 4B parameters, our proposed method reduce 50% cache memory while achieving lower validation perplexity than the standard Transformer decoder, establishing it as a memory-efficient, high-performance architectural alternative.

Method: Focusing on Basque (a morphologically rich, low-resource language), the authors: 1) Construct new corpora combining standard, social media, and historical sources; 2) Pre-train the BERnaT family of encoder-only models in three configurations (standard, diverse, and combined); 3) Propose an evaluation framework that separates NLU tasks into standard and diverse subsets to assess linguistic generalization.

Result: Models trained on both standard and diverse data consistently outperform those trained on standard corpora alone. They improve performance across all task types without compromising standard benchmark accuracy.

Conclusion: Linguistic diversity is crucial for building inclusive, generalizable language models. Incorporating diverse linguistic varieties (dialectal, historical, informal) alongside standard text improves model robustness and reduces representational biases.

Abstract: Language models depend on massive text corpora that are often filtered for quality, a process that can unintentionally exclude non-standard linguistic varieties, reduce model robustness and reinforce representational biases. In this paper, we argue that language models should aim to capture the full spectrum of language variation (dialectal, historical, informal, etc.) rather than relying solely on standardized text. Focusing on Basque, a morphologically rich and low-resource language, we construct new corpora combining standard, social media, and historical sources, and pre-train the BERnaT family of encoder-only models in three configurations: standard, diverse, and combined. We further propose an evaluation framework that separates Natural Language Understanding (NLU) tasks into standard and diverse subsets to assess linguistic generalization. Results show that models trained on both standard and diverse data consistently outperform those trained on standard corpora, improving performance across all task types without compromising standard benchmark accuracy. These findings highlight the importance of linguistic diversity in building inclusive, generalizable language models.

Method: Created BRAND: Bilingual Religious Accountable Norm Dataset covering Buddhism, Christianity, Hinduism, and Islam with 2,400+ entries. Used three prompt types in both English and Bengali to evaluate model performance and bias.

Result: Models perform better in English than Bengali and consistently display bias toward Islam, even when answering religion-neutral questions. Findings reveal persistent cross-language bias patterns in multilingual models.

Conclusion: The study highlights persistent religious bias in multilingual LLMs, particularly favoring Islam, with performance disparities between languages. Connects findings to broader HCI issues regarding religion and spirituality, emphasizing need for better religious representation in AI.

Abstract: While recent developments in large language models have improved bias detection and classification, sensitive subjects like religion still present challenges because even minor errors can result in severe misunderstandings. In particular, multilingual models often misrepresent religions and have difficulties being accurate in religious contexts. To address this, we introduce BRAND: Bilingual Religious Accountable Norm Dataset, which focuses on the four main religions of South Asia: Buddhism, Christianity, Hinduism, and Islam, containing over 2,400 entries, and we used three different types of prompts in both English and Bengali. Our results indicate that models perform better in English than in Bengali and consistently display bias toward Islam, even when answering religion-neutral questions. These findings highlight persistent bias in multilingual models when similar questions are asked in different languages. We further connect our findings to the broader issues in HCI regarding religion and spirituality.

Method: Two-stage approach: 1) Continual Pretraining (CPT) to establish Tibetan linguistic grounding, 2) Supervised Fine-Tuning (SFT) for task and translation specialization. Layer-wise analysis across 435 layers in Qwen3-4B to understand adaptation dynamics.

Result: Significant improvements: perplexity decreased from 2.98 to 1.54; Chinese→Tibetan translation BLEU improved from 0.046 to 0.261; chrF improved from 2.2 to 6.6. Adaptation primarily concentrated on embedding and output heads, with mid-late MLP projections encoding domain-specific transformations.

Conclusion: CPT constructs Tibetan semantic manifold while SFT sharpens task alignment with minimal representational disruption. Provides first quantitative exploration of Tibetan adaptation dynamics for LLMs and offers reproducible framework for extending multilingual models to low-resource settings.

Abstract: Adapting large language models (LLMs) to low-resource languages remains a major challenge due to data scarcity and cross-lingual drift. This work presents a two-stage adaptation of Qwen2.5-3B to Tibetan, a morphologically rich and underrepresented language. We employ Continual Pretraining (CPT) to establish Tibetan linguistic grounding, followed by Supervised Fine-Tuning (SFT) for task and translation specialization. Empirical evaluations demonstrate a consistent decrease in perplexity (from 2.98 $\rightarrow$ 1.54) and substantial improvements in Chinese$\rightarrow$Tibetan translation quality (BLEU: 0.046 $\rightarrow$ 0.261; chrF: 2.2 $\rightarrow$ 6.6). Layer-wise analysis across 435 layers in Qwen3-4B reveals that adaptation primarily concentrates on embedding and output heads, with mid–late MLP projections encoding domain-specific transformations. Our findings suggest that CPT constructs a Tibetan semantic manifold while SFT sharpens task alignment with minimal representational disruption. This study provides the first quantitative exploration of Tibetan adaptation dynamics for LLMs, and offers an open, reproducible framework for extending multilingual foundation models to low-resource settings.

Method: 1) Continued BPE training: Adapts pre-trained tokenizer by continuing BPE merge learning on new data. 2) Leaf-based vocabulary pruning: Removes redundant tokens while preserving model quality.

Result: Experiments across multiple languages and model families show improved tokenization efficiency and better utilization of added vocabulary. The methods provide practical tools for controlled vocabulary modification.

Conclusion: The proposed continued BPE training and leaf-based pruning offer effective solutions for tokenizer adaptation, released as an open-source package for practical vocabulary modification.

Abstract: Tokenizer adaptation plays an important role in transferring pre-trained language models to new domains or languages. In this work, we address two complementary aspects of this process: vocabulary extension and pruning. The common approach to extension trains a new tokenizer on domain-specific text and appends the tokens that do not overlap with the existing vocabulary, which often results in many tokens that are unreachable or never used. We propose continued BPE training, which adapts a pre-trained tokenizer by continuing the BPE merge learning process on new data. Experiments across multiple languages and model families show that this approach improves tokenization efficiency and leads to better utilization of added vocabulary. We also introduce leaf-based vocabulary pruning, which removes redundant tokens while preserving model quality. Together, these methods provide practical tools for controlled vocabulary modification, which we release as an open-source package.

Method: Two-stage adaptive request scheduling: Stage I optimizes scheduling order using inference features, Stage II refines decisions with runtime info. Also includes dynamic token batching based on hardware status and real-time load.

Result: Achieves 4.7-33.1x and 3.3-13.2x higher effective throughput than vLLM and InferCept, while reducing TTFT by up to 96.3% and 95.0% respectively.

Conclusion: AugServe significantly improves augmented LLM inference serving efficiency by addressing scheduling and batching limitations, enhancing user experience through better SLO compliance.

Abstract: As augmented large language models (LLMs) with external tools become increasingly popular in web applications, improving augmented LLM inference serving efficiency and optimizing service-level objectives (SLOs) are critical for enhancing user experience. To achieve this, inference systems must maximize request handling within latency constraints, referred to as increasing effective throughput. However, existing systems face two major challenges: (i) reliance on first-come-first-served (FCFS) scheduling causes severe head-of-line blocking, leading to queuing delays exceeding the SLOs for many requests; and (ii) static batch token limit, which fails to adapt to fluctuating loads and hardware conditions. Both of these factors degrade effective throughput and service quality. This paper presents AugServe, an efficient inference framework designed to reduce queueing latency and enhance effective throughput for augmented LLM inference services. The core idea of AugServe is a two-stage adaptive request scheduling strategy. Specifically, AugServe combines the inference features of augmented LLM requests to optimize the order of scheduling decisions (stage I). These decisions are continuously refined with runtime information (stage II), adapting to both request characteristics and system capabilities. In addition, AugServe dynamically adjusts the token batching mechanism based on hardware status and real-time load, further enhancing throughput performance. Experimental results show that AugServe achieves 4.7-33.1x and 3.3-13.2x higher effective throughput than vLLM and InferCept, while reducing time-to-first-token (TTFT) by up to 96.3% and 95.0%, respectively.

Method: Combines SigLIP2 vision encoder with Qwen3 language backbone using an attention-pooling connector, enabling token-efficient processing of arbitrary-resolution images.

Result: Achieves state-of-the-art multilingual visual question answering performance among open 2B-scale VLMs across standard VQA benchmarks and multilingual evaluations while preserving competitive text-only performance.

Conclusion: Jina-VLM demonstrates that effective multilingual vision-language models can be built at the 2B parameter scale with efficient architecture design, achieving strong performance on both visual and text tasks.

Abstract: We present Jina-VLM, a 2.4B parameter vision-language model that achieves state-of-the-art multilingual visual question answering among open 2B-scale VLMs. The model couples a SigLIP2 vision encoder with a Qwen3 language backbone through an attention-pooling connector that enables token-efficient processing of arbitrary-resolution images. Across standard VQA benchmarks and multilingual evaluations, Jina-VLM outperforms comparable models while preserving competitive text-only performance.

Method: SkillFactory uses supervised fine-tuning (SFT) with “silver” training traces created by rearranging samples from the model itself, formatted to demonstrate cognitive skills. These imperfect but effective traces prime the model to acquire skills, which are then further refined through reinforcement learning.

Result: (1) SkillFactory SFT initialization helps models generalize to harder task variants post-RL despite lower pre-RL performance; (2) models actually use cognitive skills; (3) RLed SkillFactory models are more robust to regression on out-of-domain tasks than RLed base models.

Conclusion: Inductive biases learned prior to reinforcement learning help models learn robust cognitive skill use, and SkillFactory’s approach of using self-generated, rearranged samples for SFT is effective for skill acquisition without relying on distillation from stronger models.

Abstract: Reasoning models leveraging long chains of thought employ various cognitive skills, such as verification of their answers, backtracking, retrying by an alternate method, and more. Previous work has shown that when a base language model exhibits these skills, training that model further with reinforcement learning (RL) can learn to leverage them. How can we get models to leverage skills that aren’t exhibited by base models? Our work, SkillFactory, is a method for fine-tuning models to roughly learn these skills during a supervised fine-tuning (SFT) stage prior to RL. Our approach does not rely on distillation from a stronger model, but instead uses samples from the model itself, rearranged to provide training data in the format of those skills. These “silver” SFT traces may be imperfect, but are nevertheless effective for priming a model to acquire skills during RL. Our evaluation shows that (1) starting from SkillFactory SFT initialization helps a model to generalize to harder variants of a task post-RL, despite lower performance pre-RL; (2) cognitive skills are indeed used by the model; (3) RLed SkillFactory models are more robust to regression on out-of-domain tasks than RLed base models. Our work suggests that inductive biases learned prior to RL help models learn robust cognitive skill use.

Method: 1) Construct GFAIR dataset with target-attribute combinations across multiple dimensions; 2) Introduce statement organization task for open-ended text generation to uncover complex biases; 3) Propose GF-THINK chain-of-thought method to mitigate biases from group fairness perspective.

Result: Extensive evaluations reveal inherent safety concerns in popular LLMs; GF-THINK method demonstrates efficacy in mitigating bias and achieving fairness in LLMs.

Conclusion: Proposes comprehensive framework for evaluating and mitigating LLM bias from group fairness perspective, with dataset and methods publicly available.

Abstract: The need to assess LLMs for bias and fairness is critical, with current evaluations often being narrow, missing a broad categorical view. In this paper, we propose evaluating the bias and fairness of LLMs from a group fairness lens using a novel hierarchical schema characterizing diverse social groups. Specifically, we construct a dataset, GFAIR, encapsulating target-attribute combinations across multiple dimensions. Moreover, we introduce statement organization, a new open-ended text generation task, to uncover complex biases in LLMs. Extensive evaluations of popular LLMs reveal inherent safety concerns. To mitigate the biases of LLMs from a group fairness perspective, we pioneer a novel chainof-thought method GF-THINK to mitigate biases of LLMs from a group fairness perspective. Experimental results demonstrate its efficacy in mitigating bias and achieving fairness in LLMs. Our dataset and codes are available at https://github.com/surika/Group-Fairness-LLMs.

Method: 1) Created IW-BENCH benchmark with 1200 image-web code pairs of varying difficulty. 2) Proposed Element Accuracy by parsing DOM trees to test element completeness. 3) Proposed Layout Accuracy by converting DOM trees to common subsequences to analyze positional relationships. 4) Designed five-hop multimodal Chain-of-Thought Prompting: SoM prompt injection, inferring elements, inferring layout, inferring web code, and reflection.

Result: Conducted extensive experiments on existing large multimodal models, providing insights into their performance and areas for improvement in the image-to-web domain. The benchmark enables systematic evaluation of web element integrity and layout accuracy.

Conclusion: IW-BENCH addresses critical gaps in evaluating image-to-web conversion by focusing on both visible/invisible element integrity and layout relationships, offering a comprehensive benchmark for assessing and improving large multimodal models in this domain.

Abstract: Recently advancements in large multimodal models have led to significant strides in image comprehension capabilities. Despite these advancements, there is a lack of the robust benchmark specifically for assessing the Image-to-Web conversion proficiency of these large models. Primarily, it is essential to ensure the integrity of the web elements generated. These elements comprise visible and invisible categories. Previous evaluation methods (e.g.,BLEU) are notably susceptible to significant alterations due to the presence of invisible elements in Web. Furthermore, it is crucial to measure the layout information of web pages, referring to the positional relationships between elements, which is overlooked by previous work. To address challenges, we have curated and aligned a benchmark of images and corresponding web codes (IW-BENCH). Specifically, we propose the Element Accuracy, which tests the completeness of the elements by parsing the Document Object Model (DOM) tree. Layout Accuracy is also proposed to analyze the positional relationships of elements by converting DOM tree into a common subsequence. Besides, we design a five-hop multimodal Chain-of-Thought Prompting for better performance, which contains five hop: 1) SoM prompt injection. 2) Inferring Elements. 3) Inferring Layout. 4) Inferring Web code. 5) Reflection. Our benchmark comprises 1200 pairs of images and web codes with varying levels of difficulty. We have conducted extensive experiments on existing large multimodal models, offering insights into their performance and areas for improvement in image-to-web domain.

Method: Established reliable evaluation protocol using broad long-context downstream tasks, conducted thorough experiments on data mix for continued pre-training (combining code repositories, books with high-quality short-context data), instruction tuning datasets, and position extrapolation techniques.

Result: ProLong-8B demonstrates state-of-the-art long-context performance among similarly sized models at 128K length, outperforms Llama-3.1-8B-Instruct on most long-context tasks despite using only 5% as many tokens, and can effectively process up to 512K tokens.

Conclusion: Effective long-context training requires combining long and short-context data, training beyond evaluation length, and using short instruction datasets for SFT, resulting in models that can process extremely long contexts efficiently.

Abstract: We study continued training and supervised fine-tuning (SFT) of a language model (LM) to make effective use of long-context information. We first establish a reliable evaluation protocol to guide model development – instead of perplexity or simple needle-in-a-haystack (NIAH) tests, we use a broad set of long-context downstream tasks, and we evaluate models after SFT as this better reveals long-context abilities. Supported by our robust evaluations, we run thorough experiments to decide the data mix for continued pre-training, the instruction tuning dataset, and many other design choices such as position extrapolation. We find that (1) code repositories and books are excellent sources of long data, but it is crucial to combine them with high-quality short-context data; (2) training with a sequence length beyond the evaluation length boosts long-context performance; (3) for SFT, using only short instruction datasets yields strong performance on long-context tasks. Our final model, ProLong-8B, which is initialized from Llama-3 and trained on 40B tokens, demonstrates state-of-the-art long-context performance among similarly sized models at a length of 128K. ProLong outperforms Llama-3.1-8B-Instruct on the majority of long-context tasks despite using only 5% as many tokens during long-context training. Additionally, ProLong can effectively process up to 512K tokens, one of the longest context windows of publicly available LMs.

Method: Develops uncertainty measures based on minimum Bayes risks in LLM decoding, then proposes a novel synthesis of model confidence and output consistency to create efficient and robust UQ methods.

Result: The proposed approach demonstrates sizable improvements over state-of-the-art UQ methods across various NLP tasks including question answering, abstractive summarization, and machine translation.

Conclusion: The paper provides a principled approach to UQ for LLMs by connecting uncertainty with minimum Bayes risk, revealing distinctive LLM characteristics that explain previous UQ failures, and offering a superior integration method.

Abstract: Uncertainty quantification (UQ) methods for Large Language Models (LLMs) encompass a variety of approaches, with two major types being particularly prominent: information-based, which focus on model confidence expressed as token probabilities, and consistency-based, which assess the semantic relationship between multiple outputs generated using repeated sampling. Several recent methods have combined these two approaches to boost UQ performance. However, they sometimes fail to outperform much simpler baseline methods. Our work discusses the fundamental approach to constructing uncertainty measures that directly links uncertainty with the minimum Bayes risks achieved by LLM decoding. Building on these findings, we propose a novel approach to integrating model confidence with output consistency, resulting in a family of efficient and robust UQ methods. Our investigation reveals distinctive characteristics of LLMs as probabilistic models, which help to explain why these UQ methods underperform in certain tasks. Based on these findings, we propose a new way of synthesizing model confidence and output consistency, leading to a family of efficient and robust UQ methods. We evaluate our approach across various tasks such as question answering, abstractive summarization, and machine translation, demonstrating sizable improvements over state-of-the-art UQ approaches.

Method: Combines upside-down reinforcement learning with synthetic data distillation from Llama-3 to train a 100M-parameter GPT-2 model for multitask prompt generation, creating a framework for scaling SLMs to multimodal tasks.

Result: The 100M-parameter SLM achieves relevance and diversity scores within 6% of competitive baselines (Llama-3 8B, Qwen3 8B, Ministral 8B) despite being up to 80 times smaller, while dramatically reducing inference latency and memory overhead.

Conclusion: SLMs can effectively handle multimodal search and recommendation tasks, demonstrating the potential of lightweight models as practical engines for scalable multimodal discovery in real-world applications.

Abstract: In this work, we investigate how small language models (SLMs) can be scaled to support multimodal search and recommendation use cases while remaining efficient enough for real-time, resource-constrained deployments. We present a framework that combines upside-down reinforcement learning with synthetic data distillation from a large language model (Llama-3) to train a 100M-parameter GPT-2 model for multitask prompt generation. Despite being up to 80 times smaller than state-of-the-art large language models (LLMs), our SLM achieves relevance and diversity scores within 6% of competitive baselines such as Llama-3 8B, Qwen3 8B, and Ministral 8B. These results demonstrate that SLMs can effectively handle multimodal search and recommendation tasks, while dramatically reducing inference latency and memory overhead. Our study highlights the potential of lightweight models as practical engines for scalable multimodal discovery, bridging the gap between cutting-edge research and real-world multimodal applications such as media recommendations and creative content generation.

Method: Identified language model subnetworks that make bigram predictions (naive next token predictions based only on current token) in fully trained language models up to 1B parameters. Analyzed these subnetworks’ location, parameter efficiency, and overlap with optimally pruned subnetworks.

Result: Found that bigram subnetworks exist in trained models, are critical for performance even with less than 0.2% of parameters, concentrate in the first Transformer MLP layer, overlap significantly with optimally pruned subnetworks, and recreate the full model’s pattern where the first layer aligns activations with next token predictions.

Conclusion: Bigram subnetworks represent a minimal subset of parameters that are both necessary and sufficient for basic next token predictions, driving the transformation from current to next token activations, and can serve as foundational building blocks for studying more complex language model circuits.

Abstract: In Transformer language models, activation vectors transform from current token embeddings to next token predictions as they pass through the model. To isolate a minimal form of this transformation, we identify language model subnetworks that make bigram predictions, naive next token predictions based only on the current token. We find that bigram subnetworks can be found in fully trained language models up to 1B parameters, and these subnetworks are critical for model performance even when they consist of less than 0.2% of model parameters. Bigram subnetworks are concentrated in the first Transformer MLP layer, and they overlap significantly with subnetworks trained to optimally prune a given model. Mechanistically, the bigram subnetworks often recreate a pattern from the full models where the first layer induces a sharp change that aligns activations with next token predictions rather than current token representations. Our results demonstrate that bigram subnetworks comprise a minimal subset of parameters that are both necessary and sufficient for basic next token predictions in language models, and they help drive the transformation from current to next token activations in the residual stream. These subnetworks can lay a foundation for studying more complex language model circuits by building up from a minimal circuit.

Method: Finetune-RAG: fine-tuning approach using first-of-its-kind RAG training dataset mimicking real-world retrieval imperfections. Bench-RAG: LLM-as-a-judge evaluation pipeline stress testing models under realistic imperfect retrieval scenarios.

Result: Finetune-RAG improves factual accuracy by 21.2% over the base model. The approach and evaluation framework are validated through experiments.

Conclusion: Fine-tuning on realistic imperfect retrieval data significantly improves RAG performance. The open-sourced codebase and dataset enable community advancement in handling retrieval imperfections.

Abstract: Retrieval-Augmented Generation (RAG) has emerged as a powerful framework to improve factuality in large language models (LLMs) by grounding their outputs in retrieved documents. However, ensuring perfect retrieval of relevant information remains challenging, and when irrelevant content is passed downstream to an LLM, it can lead to hallucinations. In this work, we propose Finetune-RAG, a simple and effective fine-tuning approach that features the first-of-its-kind RAG training dataset constructed to mimic real-world imperfections. Experimental results show that Finetune-RAG improves factual accuracy by 21.2% over the base model. We also propose Bench-RAG, an LLM-as-a-judge evaluation pipeline that stress tests models under realistic imperfect retrieval scenarios. Our codebase and dataset are fully open sourced for community use.

Method: Uses goal-conditioned value functions that predict task outcomes given actions, allowing LLM agents to evaluate multiple possible outcomes. Value functions are trained over reasoning steps rather than full actions, creating a lightweight module for decision-making in multi-turn interactions.

Result: Demonstrates superior performance over both RL fine-tuning and prompting methods on interactive tasks including tool use, social deduction, and dialogue, while maintaining efficiency and scalability.

Conclusion: Goal-conditioned value functions provide an effective, scalable approach to enhance LLM reasoning for complex interactive tasks without the computational burdens of full RL fine-tuning.

Abstract: Large language models (LLMs) excel in tasks like question answering and dialogue, but complex tasks requiring interaction, such as negotiation and persuasion, require additional long-horizon reasoning and planning. Reinforcement learning (RL) fine-tuning can enable such planning in principle, but suffers from drawbacks that hinder scalability. In particular, multi-turn RL training incurs high memory and computational costs, which are exacerbated when training LLMs as policies. Furthermore, the largest LLMs do not expose the APIs necessary to be trained in such manner. As a result, modern methods to improve the reasoning of LLMs rely on sophisticated prompting mechanisms rather than RL fine-tuning. To remedy this, we propose a novel approach that uses goal-conditioned value functions to guide the reasoning of LLM agents, that scales even to large API-based models. These value functions predict how a task will unfold given an action, allowing the LLM agent to evaluate multiple possible outcomes, both positive and negative, to plan effectively. In addition, these value functions are trained over reasoning steps rather than full actions, to be a concise and light-weight module that facilitates decision-making in multi-turn interactions. We validate our method on tasks requiring interaction, including tool use, social deduction, and dialogue, demonstrating superior performance over both RL fine-tuning and prompting methods while maintaining efficiency and scalability.

Method: The authors decompose the DPO loss to reveal its underlying structure, identifying that standard DPO implicitly oversimplifies a regularizer. They restore this full term and introduce PRO (PRoximalized PReference Optimization), which uses an efficient approximation of the full regularizer to eliminate likelihood underdetermination while accommodating diverse feedback types (pairwise, binary, scalar).

Result: Empirical evaluations show consistent superiority of PRO over existing methods across different feedback types (pairwise, binary, and scalar). The method effectively resolves the likelihood underdetermination problem while maintaining alignment performance.

Conclusion: PRO provides a unified alignment framework that addresses the fundamental limitation of contrastive methods by properly handling the regularizer term, enabling more robust alignment across diverse feedback types without likelihood underdetermination issues.

Abstract: Direct alignment methods typically train large language models (LLMs) by contrasting the likelihoods of preferred and dispreferred responses. While effective at capturing relative preferences, these methods are widely observed to suppress the absolute likelihoods of example responses. As a result, aligned models can deviate from expected patterns, exhibiting rewar-hacking effect even without an explicit reward model. This fundamental limitation of contrastive alignment, which we term likelihood underdetermination, motivates us to revisit direct preference optimization (DPO) – the seminal direct alignment method. Interestingly, we show that the DPO loss admits a principled decomposition. The reformulated loss not only extends naturally to a broader range of feedback types, but also unveils the root cause of likelihood underdetermination. Specifically, we identify that standard DPO implicitly oversimplifies a regularizer in the reformulated loss; restoring this full term effectively resolves the underdetermination. Building on these insights, we introduce PRoximalized PReference Optimization (PRO), a unified alignment method that accommodates diverse feedback types while eliminating likelihood underdetermination through an efficient approximation of the full regularizer. Empirical evaluations demonstrate the consistent superiority of PRO over existing methods across pairwise, binary and scalar feedback.

Method: Analyzes a restricted idealization of fixed-precision transformers with strict future masking, soft attention, and no positional encodings. Connects this class to a specific fragment of linear temporal logic (LTL) containing only the past operator, and further relates it to formal language theory, automata theory, and algebra.

Result: Establishes that the idealized transformer class is exactly as expressive as the LTL fragment with only past operator. Empirical results show transformers trained on languages within this expressive capacity generalize reliably across sequence lengths, while consistently failing to generalize on languages beyond it.

Conclusion: Provides a unified framework for understanding transformer expressivity under specific idealizations, connecting neural architectures to formal language theory and showing clear empirical alignment between theoretical expressive capacity and practical generalization behavior.

Abstract: Transformer-based language models (LMs) have achieved widespread empirical success, but their theoretical expressive power remains only partially understood. In this work, we analyze a restricted idealization of fixed-precision transformers with strict future masking, soft attention, and no positional encodings. We establish that this class of models is exactly as expressive as a specific fragment of linear temporal logic that contains only a single temporal operator: the past operator. We further connect this fragment to established classes in formal language theory, automata theory, and algebra, yielding a unified framework for understanding transformer expressivity under this idealization. Finally, we present empirical results that align closely with our theory: transformers trained on languages within their characterized expressive capacity generalize reliably across sequence lengths, while they consistently fail to generalize on languages beyond it.

Method: Study with 480 participants evaluating 3,840 query-response pairs to examine how refusal strategies affect user perceptions. Also analyzed response patterns of 9 state-of-the-art LLMs and evaluated how 6 reward models score different refusal strategies.

Result: Response strategy shapes user experience significantly while user motivation has negligible impact. Partial compliance reduces negative perceptions by over 50% compared to flat refusals. Models rarely deploy partial compliance naturally, and reward models undervalue this strategy.

Conclusion: Effective guardrails should focus on crafting thoughtful refusals rather than detecting intent. Partial compliance offers a path toward AI safety mechanisms that ensure both safety and sustained user engagement.

Abstract: Current LLMs are trained to refuse potentially harmful input queries regardless of whether users actually had harmful intents, causing a tradeoff between safety and user experience. Through a study of 480 participants evaluating 3,840 query-response pairs, we examine how different refusal strategies affect user perceptions across varying motivations. Our findings reveal that response strategy largely shapes user experience, while actual user motivation has negligible impact. Partial compliance – providing general information without actionable details – emerges as the optimal strategy, reducing negative user perceptions by over 50% to flat-out refusals. Complementing this, we analyze response patterns of 9 state-of-the-art LLMs and evaluate how 6 reward models score different refusal strategies, demonstrating that models rarely deploy partial compliance naturally and reward models currently undervalue it. This work demonstrates that effective guardrails require focusing on crafting thoughtful refusals rather than detecting intent, offering a path toward AI safety mechanisms that ensure both safety and sustained user engagement.

Method: Proposes MemOS, a memory operating system that unifies representation, scheduling, and evolution of plaintext, activation-based, and parameter-level memories. Uses MemCube as the basic unit that encapsulates both memory content and metadata (provenance, versioning). MemCubes can be composed, migrated, and fused over time, enabling flexible transitions between memory types and bridging retrieval with parameter-based learning.

Result: MemOS establishes a memory-centric system framework that brings controllability, plasticity, and evolvability to LLMs. It enables cost-efficient storage and retrieval while laying the foundation for continual learning and personalized modeling.

Conclusion: MemOS addresses fundamental memory management challenges in LLMs by providing a unified system for managing heterogeneous knowledge across different temporal scales and sources, enabling more efficient and capable AGI systems with better long-context reasoning and personalization capabilities.

Abstract: Large Language Models (LLMs) have become an essential infrastructure for Artificial General Intelligence (AGI), yet their lack of well-defined memory management systems hinders the development of long-context reasoning, continual personalization, and knowledge consistency.Existing models mainly rely on static parameters and short-lived contextual states, limiting their ability to track user preferences or update knowledge over extended periods.While Retrieval-Augmented Generation (RAG) introduces external knowledge in plain text, it remains a stateless workaround without lifecycle control or integration with persistent representations.Recent work has modeled the training and inference cost of LLMs from a memory hierarchy perspective, showing that introducing an explicit memory layer between parameter memory and external retrieval can substantially reduce these costs by externalizing specific knowledge. Beyond computational efficiency, LLMs face broader challenges arising from how information is distributed over time and context, requiring systems capable of managing heterogeneous knowledge spanning different temporal scales and sources. To address this challenge, we propose MemOS, a memory operating system that treats memory as a manageable system resource. It unifies the representation, scheduling, and evolution of plaintext, activation-based, and parameter-level memories, enabling cost-efficient storage and retrieval. As the basic unit, a MemCube encapsulates both memory content and metadata such as provenance and versioning. MemCubes can be composed, migrated, and fused over time, enabling flexible transitions between memory types and bridging retrieval with parameter-based learning. MemOS establishes a memory-centric system framework that brings controllability, plasticity, and evolvability to LLMs, laying the foundation for continual learning and personalized modeling.

Method: Constructed SpeechRole-Data (98 roles, 112k speech conversations) with distinct vocal characteristics, and proposed SpeechRole-Eval benchmark for multidimensional evaluation of SRPAs.

Result: Experimental results reveal advantages and challenges of both cascaded and end-to-end speech role-playing agents in maintaining vocal style consistency and role coherence.

Conclusion: The work provides a solid foundation for speech-driven multimodal role-playing research by releasing data, code, and baseline models to foster further developments in this field.

Abstract: Recently, role-playing agents have emerged as a promising paradigm for achieving personalized interaction and emotional resonance. Existing research primarily focuses on the textual modality, neglecting the critical dimension of speech in realistic interactive scenarios. In particular, there is a lack of systematic evaluation for Speech Role-Playing Agents (SRPAs). To address this gap, we construct SpeechRole-Data, a large-scale, high-quality dataset that comprises 98 diverse roles and 112k speech-based single-turn and multi-turn conversations. Each role demonstrates distinct vocal characteristics, including timbre and prosody, thereby enabling more sophisticated speech role-playing. Furthermore, we propose SpeechRole-Eval, a multidimensional evaluation benchmark that systematically assesses SRPAs performance in key aspects such as fundamental interaction ability, speech expressiveness, and role-playing fidelity. Experimental results reveal the advantages and challenges of both cascaded and end-to-end speech role-playing agents in maintaining vocal style consistency and role coherence. We release all data, code, and baseline models to provide a solid foundation for speech-driven multimodal role-playing research and to foster further developments in this field.

Method: Dynamic evaluation framework with proprietary 220k graduate-level question bank, dynamically sampling unseen test sets, contamination-resistant data curation, anti-cheating architecture, and calibrated LLM-as-a-judge process with relative ranking system.

Result: 20-month study of 50 models reveals performance ceiling on knowledge memorization, exposes undetectable contamination vulnerabilities, demonstrates exceptional ranking stability and consistency, and achieves 90% agreement with human experts.

Conclusion: LLMEval-3 provides robust methodology for assessing true LLM capabilities beyond leaderboard scores, validating dynamic evaluation paradigm and promoting more trustworthy evaluation standards.

Abstract: Existing evaluation of Large Language Models (LLMs) on static benchmarks is vulnerable to data contamination and leaderboard overfitting, critical issues that obscure true model capabilities. To address this, we introduce LLMEval-3, a framework for dynamic evaluation of LLMs. LLMEval-3 is built on a proprietary bank of 220k graduate-level questions, from which it dynamically samples unseen test sets for each evaluation run. Its automated pipeline ensures integrity via contamination-resistant data curation, a novel anti-cheating architecture, and a calibrated LLM-as-a-judge process achieving 90% agreement with human experts, complemented by a relative ranking system for fair comparison. An 20-month longitudinal study of nearly 50 leading models reveals a performance ceiling on knowledge memorization and exposes data contamination vulnerabilities undetectable by static benchmarks. The framework demonstrates exceptional robustness in ranking stability and consistency, providing strong empirical validation for the dynamic evaluation paradigm. LLMEval-3 offers a robust and credible methodology for assessing the true capabilities of LLMs beyond leaderboard scores, promoting the development of more trustworthy evaluation standards.

Method: ARoG framework with two strategies: 1) Relation-centric abstraction - converts anonymous entities into high-level concepts by capturing semantics of adjacent relations; 2) Structure-oriented abstraction - transforms natural language questions into structured abstract concept paths for alignment with KG concepts.

Result: Experiments on three datasets show ARoG achieves strong performance and privacy-robustness, effectively retrieving knowledge from anonymized KGs while protecting entity privacy.

Conclusion: ARoG successfully addresses privacy-protected RAG by abstracting anonymous entities into retrievable concepts and aligning question structures with KG abstractions, enabling effective knowledge retrieval without exposing private entity semantics to LLMs.

Abstract: LLMs often suffer from hallucinations and outdated or incomplete knowledge. RAG is proposed to address these issues by integrating external knowledge like that in KGs into LLMs. However, leveraging private KGs in RAG systems poses significant privacy risks due to the black-box nature of LLMs and potential insecure data transmission, especially when using third-party LLM APIs lacking transparency and control. In this paper, we investigate the privacy-protected RAG scenario for the first time, where entities in KGs are anonymous for LLMs, thus preventing them from accessing entity semantics. Due to the loss of semantics of entities, previous RAG systems cannot retrieve question-relevant knowledge from KGs by matching questions with the meaningless identifiers of anonymous entities. To realize an effective RAG system in this scenario, two key challenges must be addressed: (1) How can anonymous entities be converted into retrievable information. (2) How to retrieve question-relevant anonymous entities. Hence, we propose a novel ARoG framework including relation-centric abstraction and structure-oriented abstraction strategies. For challenge (1), the first strategy abstracts entities into high-level concepts by dynamically capturing the semantics of their adjacent relations. It supplements meaningful semantics which can further support the retrieval process. For challenge (2), the second strategy transforms unstructured natural language questions into structured abstract concept paths. These paths can be more effectively aligned with the abstracted concepts in KGs, thereby improving retrieval performance. To guide LLMs to effectively retrieve knowledge from KGs, the two strategies strictly protect privacy from being exposed to LLMs. Experiments on three datasets demonstrate that ARoG achieves strong performance and privacy-robustness.

Method: Manually translated the English Reveal multi-step reasoning dataset into Bangla, then conducted controlled evaluations comparing English-centric and Bangla-centric multilingual small language models on both original and translated versions.

Result: Reasoning context helps with more challenging non-binary questions in comparable settings, but models struggle to effectively employ relevant Bangla reasoning steps. Different trends in how reasoning steps contribute to predictions were observed across models and languages.

Conclusion: While reasoning context is beneficial for complex questions, current models have limitations in leveraging reasoning steps effectively in low-resource languages like Bangla, highlighting the need for better multilingual reasoning capabilities.

Abstract: Language models have demonstrated remarkable performance on complex multi-step reasoning tasks. However, their evaluation has been predominantly confined to high-resource languages such as English. In this paper, we introduce a manually translated Bangla multi-step reasoning dataset derived from the English Reveal dataset, featuring both binary and non-binary question types. We conduct a controlled evaluation of English-centric and Bangla-centric multilingual small language models on the original dataset and our translated version to compare their ability to exploit relevant reasoning steps to produce correct answers. Our results show that, in comparable settings, reasoning context is beneficial for more challenging non-binary questions, but models struggle to employ relevant Bangla reasoning steps effectively. We conclude by exploring how reasoning steps contribute to models’ predictions, highlighting different trends across models and languages.

Method: RECAP (Reflect-Extract-Calibrate-Align-Produce) uses inference-time structured emotional reasoning with appraisal-theoretic stages, psychological factor identification, and Likert-based emotion likelihoods that clinicians can inspect or override.

Result: 22-28% improvement in emotional reasoning on 8B models, 10-13% on larger models; oncology clinicians rated RECAP responses as more empathetic, supportive, and context-appropriate than baseline prompts.

Conclusion: Modular, principled prompting can enhance emotional intelligence in medical AI while maintaining transparency and accountability for clinical deployment.

Abstract: Large language models in healthcare often miss critical emotional cues, delivering medically sound but emotionally flat advice. Such responses are insufficient in clinical encounters, where distressed or vulnerable patients rely on empathic communication to support safety, adherence, and trust. We present RECAP (Reflect-Extract-Calibrate-Align-Produce), an inference-time framework that guides models through structured emotional reasoning without retraining. RECAP decomposes patient input into appraisal-theoretic stages, identifies psychological factors, and assigns Likert-based emotion likelihoods that clinicians can inspect or override, producing nuanced and auditable responses. Across EmoBench, SECEU, and EQ-Bench, RECAP improves emotional reasoning by 22-28% on 8B models and 10-13% on larger models over zero-shot baselines. In blinded evaluations, oncology clinicians rated RECAP’s responses as more empathetic, supportive, and context-appropriate than prompting baselines. These findings demonstrate that modular, principled prompting can enhance emotional intelligence in medical AI while maintaining transparency and accountability for clinical deployment.

Method: Created automated multilingual pipeline generating 628 K-10 math problems in English, German, and Arabic. Tested three commercial LLMs (GPT-4o-mini, Gemini 2.5 Flash, Qwen-plus) to produce step-by-step solutions. Used held-out LLM judges (Claude 3.5 Haiku) with comparative framework to evaluate solution quality across languages.

Result: Consistent performance gap: English solutions consistently rated highest, Arabic often ranked lower. Shows persistent linguistic bias in LLM performance across different languages for educational math problem solving.

Conclusion: Findings highlight persistent linguistic bias in LLMs and emphasize the need for more equitable multilingual AI systems in education to ensure fair educational support across different language users.

Abstract: Large Language Models (LLMs) are increasingly used for educational support, yet their response quality varies depending on the language of interaction. This paper presents an automated multilingual pipeline for generating, solving, and evaluating math problems aligned with the German K-10 curriculum. We generated 628 math exercises and translated them into English, German, and Arabic. Three commercial LLMs (GPT-4o-mini, Gemini 2.5 Flash, and Qwen-plus) were prompted to produce step-by-step solutions in each language. A held-out panel of LLM judges, including Claude 3.5 Haiku, evaluated solution quality using a comparative framework. Results show a consistent gap, with English solutions consistently rated highest, and Arabic often ranked lower. These findings highlight persistent linguistic bias and the need for more equitable multilingual AI systems in education.

Method: Conducted comprehensive study across 13 diverse benchmarks, analyzing behavioral patterns in batched inference including overthinking suppression, hedging language reduction, and emergent collective effects.

Result: Batching improves accuracy while reducing reasoning token usage by 3x-5x, suppresses overthinking, reduces hedging language, and shows emergent collective effects where models generalize patterns from easier to harder examples.

Conclusion: Batching should be viewed not just as a throughput optimization technique, but as a powerful inference-time regularizer that enables more efficient and reliable LLM reasoning.

Abstract: Recent work has explored batch prompting as a strategy to amortize inference cost in large language models (LLMs). In this paper, we show that batching offers an additional, underappreciated benefit: it regularizes model behavior during multi-step reasoning for Large Reasoning Models (LRMs). We conduct a comprehensive study across 13 diverse benchmarks and observe that batching improves accuracy while substantially reducing reasoning token usage, often by 3x-5x. Through detailed behavioral analysis, we find that batching suppresses overthinking, reduces hedging language (e.g., repetitive self-corrections), and encourages more decisive answers. Surprisingly, we also observe emergent collective effects in batched inference: models often generalize patterns from earlier examples to solve harder ones in the same batch. These findings position batching not just as a throughput optimization, but as a powerful inference-time regularizer for more efficient and reliable LLM reasoning.

Method: Proposed Language Attention Head Importance Scores (LAHIS) - an efficient method using single forward/backward pass to identify attention head importance for multilingual capabilities. Also introduced lightweight adaptation with soft head mask (only 20 tunable parameters) to modulate attention outputs.

Result: Applied LAHIS to Aya-23-8B, Llama-3.2-3B, and Mistral-7B-v0.1, revealing both language-specific and language-general heads. Language-specific heads enable cross-lingual attention transfer and mitigate off-target language generation. Lightweight adaptation improved XQuAD accuracy.

Conclusion: The work enhances LLM interpretability and multilingual capabilities by analyzing MHA’s role, identifying key attention heads for multilingual processing, and showing that minimal parameter tuning can improve performance through attention modulation.

Abstract: Large language models (LLMs) increasingly support multilingual understanding and generation. Meanwhile, efforts to interpret their internal mechanisms have emerged, offering insights to enhance multilingual performance. While multi-head self-attention (MHA) has proven critical in many areas, its role in multilingual capabilities remains underexplored. In this work, we study the contribution of MHA in supporting multilingual processing in LLMs. We propose Language Attention Head Importance Scores (LAHIS), an effective and efficient method that identifies attention head importance for multilingual capabilities via a single forward and backward pass through the LLM. Applying LAHIS to Aya-23-8B, Llama-3.2-3B, and Mistral-7B-v0.1, we reveal the existence of both language-specific and language-general heads. Language-specific heads enable cross-lingual attention transfer to guide the model toward target language contexts and mitigate off-target language generation issue, contributing to addressing challenges in multilingual LLMs. We also introduce a lightweight adaptation that learns a soft head mask to modulate attention outputs over language heads, requiring only 20 tunable parameters to improve XQuAD accuracy. Overall, our work enhances both the interpretability and multilingual capabilities of LLMs from the perspective of MHA.

Method: Cascades two LLMs: a small LLM compresses long contexts into latent tokens (32-64 length), achieving high text-to-latent token ratios. A large LLM then performs decoding tasks on the compressed context.

Result: At 20x compression ratio, achieves 98% decoding accuracy vs. ~60% for DeepSeek-OCR. At 40x ratio, maintains ~93% accuracy. Shows superior performance over optical character compression.

Conclusion: C3 demonstrates superior performance and feasibility in context compression with a simpler pure-text pipeline. Suggests potential upper bounds for compression ratios in optical character compression, OCR, and related fields.

Abstract: Million-level token inputs in long-context tasks pose significant computational and memory challenges for Large Language Models (LLMs). Recently, DeepSeek-OCR conducted research into the feasibility of Contexts Optical Compression and achieved preliminary results. Inspired by this, we introduce Context Cascade Compression C3 to explore the upper limits of text compression. Our method cascades two LLMs of different sizes to handle the compression and decoding tasks. Specifically, a small LLM, acting as the first stage, performs text compression by condensing a long context into a set of latent tokens (e.g., 32 or 64 in length), achieving a high ratio of text tokens to latent tokens. A large LLM, as the second stage, then executes the decoding task on this compressed context. Experiments show that at a 20x compression ratio (where the number of text tokens is 20 times the number of latent tokens), our model achieves 98% decoding accuracy, compared to approximately 60% for DeepSeek-OCR. When we further increase the compression ratio to 40x, the accuracy is maintained at around 93%. This indicates that in the domain of context compression, C3 Compression demonstrates superior performance and feasibility over optical character compression. C3 uses a simpler, pure-text pipeline that ignores factors like layout, color, and information loss from a visual encoder. This also suggests a potential upper bound for compression ratios in future work on optical character compression, OCR, and related fields. Codes and model weights are publicly accessible at https://github.com/liufanfanlff/C3-Context-Cascade-Compression

Method: Created three datasets: one large-scale dataset of potential idioms from a large corpus, plus two human-annotated datasets of definite idioms. Used existing idiom datasets to build a combined idiom list, retrieved context sequences, and processed for model-agnostic training.

Result: Developed datasets for evaluating baseline ability of pre-trained language models in idiom recognition tasks, with compatibility for slot labeling and sequence tagging tasks.

Conclusion: Better and larger datasets can help narrow the gap in LLMs’ understanding of figurative language, with finetuning approaches proving optimal for idiom recognition tasks.

Abstract: Idiomatic and figurative language form a large portion of colloquial speech and writing. With social media, this informal language has become more easily observable to people and trainers of large language models (LLMs) alike. While the advantage of large corpora seems like the solution to all machine learning and Natural Language Processing (NLP) problems, idioms and figurative language continue to elude LLMs. Finetuning approaches are proving to be optimal, but better and larger datasets can help narrow this gap even further. The datasets presented in this paper provide one answer, while offering a diverse set of categories on which to build new models and develop new approaches. A selection of recent idiom and figurative language datasets were used to acquire a combined idiom list, which was used to retrieve context sequences from a large corpus. One large-scale dataset of potential idiomatic and figurative language expressions and two additional human-annotated datasets of definite idiomatic and figurative language expressions were created to evaluate the baseline ability of pre-trained language models in handling figurative meaning through idiom recognition (detection) tasks. The resulting datasets were post-processed for model agnostic training compatibility, utilized in training, and evaluated on slot labeling and sequence tagging.

Method: Distribution-based feature Recovery and Fusion (DRF) maintains feature queues to approximate modality distributions, estimates modality qualities for low-quality cases, and builds inter-modal mapping relationships to recover missing modalities from available ones.

Result: DRF shows universal improvements over SOTA methods on three image-text datasets under disruption strategies mimicking low-quality and missing modalities, validating its robustness.

Conclusion: DRF provides an effective unified framework for robust multimodal sentiment analysis that handles both low-quality and missing modalities through distribution-based feature management.

Abstract: As posts on social media increase rapidly, analyzing the sentiments embedded in image-text pairs has become a popular research topic in recent years. Although existing works achieve impressive accomplishments in simultaneously harnessing image and text information, they lack the considerations of possible low-quality and missing modalities. In real-world applications, these issues might frequently occur, leading to urgent needs for models capable of predicting sentiment robustly. Therefore, we propose a Distribution-based feature Recovery and Fusion (DRF) method for robust multimodal sentiment analysis of image-text pairs. Specifically, we maintain a feature queue for each modality to approximate their feature distributions, through which we can simultaneously handle low-quality and missing modalities in a unified framework. For low-quality modalities, we reduce their contributions to the fusion by quantitatively estimating modality qualities based on the distributions. For missing modalities, we build inter-modal mapping relationships supervised by samples and distributions, thereby recovering the missing modalities from available ones. In experiments, two disruption strategies that corrupt and discard some modalities in samples are adopted to mimic the low-quality and missing modalities in various real-world scenarios. Through comprehensive experiments on three publicly available image-text datasets, we demonstrate the universal improvements of DRF compared to SOTA methods under both two strategies, validating its effectiveness in robust multimodal sentiment analysis.

Method: Created a live dynamic benchmark with 50 questions per month designed by crypto-native professionals, categorized into four quadrants: Simple Retrieval, Complex Retrieval, Simple Prediction, and Complex Prediction. Evaluated 10 LLMs both directly and within agentic frameworks.

Result: Revealed a performance hierarchy among LLMs and uncovered a “retrieval-prediction imbalance” - many leading models are proficient at data retrieval but show pronounced weakness in predictive analysis tasks, highlighting agents that appear factually grounded but lack deeper analytical synthesis capabilities.

Conclusion: CryptoBench provides a more challenging and valuable scenario for LLM agent assessment in the demanding cryptocurrency domain, exposing critical weaknesses in current models’ analytical capabilities despite their retrieval strengths.

Abstract: This paper introduces CryptoBench, the first expert-curated, dynamic benchmark designed to rigorously evaluate the real-world capabilities of Large Language Model (LLM) agents in the uniquely demanding and fast-paced cryptocurrency domain. Unlike general-purpose agent benchmarks for search and prediction, professional crypto analysis presents specific challenges: \emph{extreme time-sensitivity}, \emph{a highly adversarial information environment}, and the critical need to synthesize data from \emph{diverse, specialized sources}, such as on-chain intelligence platforms and real-time Decentralized Finance (DeFi) dashboards. CryptoBench thus serves as a much more challenging and valuable scenario for LLM agent assessment. To address these challenges, we constructed a live, dynamic benchmark featuring 50 questions per month, expertly designed by crypto-native professionals to mirror actual analyst workflows. These tasks are rigorously categorized within a four-quadrant system: Simple Retrieval, Complex Retrieval, Simple Prediction, and Complex Prediction. This granular categorization enables a precise assessment of an LLM agent’s foundational data-gathering capabilities alongside its advanced analytical and forecasting skills. Our evaluation of ten LLMs, both directly and within an agentic framework, reveals a performance hierarchy and uncovers a failure mode. We observe a \textit{retrieval-prediction imbalance}, where many leading models, despite being proficient at data retrieval, demonstrate a pronounced weakness in tasks requiring predictive analysis. This highlights a problematic tendency for agents to appear factually grounded while lacking the deeper analytical capabilities to synthesize information.

Method: Proposes Four Over Six (4/6) algorithm that evaluates two potential scale factors for each block of values. Unlike standard NVFP4 quantization, it can scale to smaller FP4 values to make the distribution of representable values more uniform, better handling near-maximal values that cause most quantization error.

Result: 4/6 prevents training divergence in several cases, bringing training loss significantly closer to BF16 baselines. It can be efficiently implemented on NVIDIA Blackwell GPUs and easily incorporated into various post-training quantization methods, generally improving downstream accuracy.

Conclusion: 4/6 modification makes NVFP4 training more stable and effective, enabling better low-precision training and deployment of large language models while maintaining performance close to higher-precision formats.

Abstract: As large language models have grown larger, low-precision numerical formats such as NVFP4 have become increasingly popular due to the speed and memory benefits they provide. However, to accelerate computation with NVFP4, all matrix multiplication operands–weights and activations in the forward pass, and weights, activations, and gradients in the backward pass–must be quantized to NVFP4, often leading to divergence during training and performance degradation during inference. To address this issue, in this work we introduce Four Over Six (4/6), a modification to the NVFP4 quantization algorithm that evaluates two potential scale factors for each block of values. Unlike integer formats, floating-point formats such as FP4 have the most quantization error on near-maximal values in each block, which we find to be primarily responsible for downstream performance degradation. We find that for some blocks, scaling to smaller FP4 values makes the distribution of representable values more uniform, improving representation of near-maximal values. Importantly, 4/6 can be implemented efficiently on NVIDIA Blackwell GPUs, making it viable to use while training LLMs with NVFP4. In pre-training experiments with transformer and hybrid model architectures, we find that 4/6 prevents divergence in several cases, bringing training loss significantly closer to BF16 compared to models trained with current state-of-the-art NVFP4 training recipes. We also find that 4/6 can be easily incorporated into many different post-training quantization methods and generally improves downstream accuracy. We hope this inspires future work in training and deploying models with NVFP4. Our code is available at http://github.com/mit-han-lab/fouroversix.

Method: Multi-Layered Self-Reflection with Auto-Prompting (MAPS) integrates CoT, Self-Reflection, and Auto-Prompting in an iterative refinement process with adaptive error detection and tailored prompt generation.

Result: MAPS significantly outperforms standard CoT on four benchmarks across multiple LLMs, enables general-purpose LLMs to reach performance comparable to specialized reasoning models, and balances accuracy with cost through strategic reflection depth limitation.

Conclusion: MAPS provides an effective framework for enhancing multi-step mathematical reasoning in LLMs through iterative self-reflection and adaptive prompting, offering a practical balance between performance and computational cost.

Abstract: Recent advancements in Large Language Models (LLMs) have significantly improved their problem-solving capabilities. However, these models still struggle when faced with complex multi-step reasoning tasks. In this paper, we propose the Multi-Layered Self-Reflection with Auto-Prompting (MAPS) framework, a novel approach designed to enhance multi-step mathematical reasoning in LLMs by integrating techniques such as Chain of Thought (CoT), Self-Reflection, and Auto-Prompting. Unlike traditional static prompting methods, MAPS employs an iterative refinement process. Initially, the model generates a solution using CoT prompting. When errors are detected, an adaptive self-reflection mechanism identifies and analyzes them, generating tailored prompts to guide corrections. These dynamically adjusted prompts enable the model to iteratively refine its reasoning. Experiments on four well-established benchmarks across multiple LLMs show that MAPS significantly outperforms standard CoT and achieves competitive results with reasoning-optimized models. In addition, MAPS enables general-purpose LLMs to reach performance levels comparable to specialized reasoning models. While deeper reflection layers improve accuracy, they also increase token usage and costs. To balance this trade-off, MAPS strategically limits reflection depth, ensuring an optimal balance between cost and reasoning performance.

Method: Proposes a self-supervised symbolic auto-encoder that encodes diagrams into structured primitives (points, lines, shapes) and their relationships, then decodes them through an executable engine. Uses Symbolic Hierarchical Process Reward Modeling with step-level parsing rewards for consistency (point-on-line, line-on-shape, shape-on-relation). Introduces stabilization mechanisms to balance exploration-exploitation in reinforcement learning for diagram reconstruction.

Result: Achieves 98.2% reduction in MSE for geometric diagram reconstruction, surpasses GPT-4o by 0.6% with a 7B model on chart reconstruction, improves by +13% on MathGlance perception benchmark, and by +3% on MathVerse and GeoQA reasoning benchmarks.

Conclusion: The proposed neuro-symbolic system successfully integrates neural network reasoning with symbolic interpretability through reasoning-grounded visual rewards, demonstrating effectiveness across multiple diagram understanding tasks and establishing a new approach for symbolic visual learning.

Abstract: Symbolic computer vision represents diagrams through explicit logical rules and structured representations, enabling interpretable understanding in machine vision. This requires fundamentally different learning paradigms from pixel-based visual models. Symbolic visual learners parse diagrams into geometric primitives-points, lines, and shapes-whereas pixel-based learners operate on textures and colors. We propose a novel self-supervised symbolic auto-encoder that encodes diagrams into structured primitives and their interrelationships within the latent space, and decodes them through our executable engine to reconstruct the input diagrams. Central to this architecture is Symbolic Hierarchical Process Reward Modeling, which applies hierarchical step-level parsing rewards to enforce point-on-line, line-on-shape, and shape-on-relation consistency. Since vanilla reinforcement learning exhibits poor exploration in the policy space during diagram reconstruction; we thus introduce stabilization mechanisms to balance exploration and exploitation. We fine-tune our symbolic encoder on downstream tasks, developing a neuro-symbolic system that integrates the reasoning capabilities of neural networks with the interpretability of symbolic models through reasoning-grounded visual rewards. Evaluations across reconstruction, perception, and reasoning tasks demonstrate the effectiveness of our approach: achieving a 98.2% reduction in MSE for geometric diagram reconstruction, surpassing GPT-4o by 0.6% with a 7B model on chart reconstruction, and improving by +13% on the MathGlance perception benchmark, and by +3% on MathVerse and GeoQA reasoning benchmarks.

Method: Three-phase framework: 1) Fine-tune point cloud generation model for coarse object representation from text, 2) Hypergraph-based learning to refine representations (parts as vertices), 3) Diffusion model to generate joints (edges) between object parts.

Result: Extensive experiments on PartNet-Mobility dataset demonstrate effectiveness and superior performance over previous methods.

Conclusion: GAOT successfully bridges the gap between text prompts and articulated 3D object generation through a novel three-phase framework combining diffusion models and hypergraph learning.

Abstract: Articulated object generation has seen increasing advancements, yet existing models often lack the ability to be conditioned on text prompts. To address the significant gap between textual descriptions and 3D articulated object representations, we propose GAOT, a three-phase framework that generates articulated objects from text prompts, leveraging diffusion models and hypergraph learning in a three-step process. First, we fine-tune a point cloud generation model to produce a coarse representation of objects from text prompts. Given the inherent connection between articulated objects and graph structures, we design a hypergraph-based learning method to refine these coarse representations, representing object parts as graph vertices. Finally, leveraging a diffusion model, the joints of articulated objects-represented as graph edges-are generated based on the object parts. Extensive qualitative and quantitative experiments on the PartNet-Mobility dataset demonstrate the effectiveness of our approach, achieving superior performance over previous methods.

Method: Propose a drainage node added at network output that reallocates probability mass toward uncertainty while preserving end-to-end training and differentiability, providing an escape route for ambiguous, anomalous, or noisy samples.

Result: Achieves up to 9% accuracy increase over existing approaches in high-noise regimes on CIFAR-10/100 with instance-dependent or asymmetric noise, matches/surpasses SOTA on real-world datasets (mini-WebVision, mini-ImageNet, Clothing-1M), and shows denoising effect where drainage neuron absorbs corrupt data.

Conclusion: The drainage formulation provides a unified solution for handling noisy labels and uncertainty, enables applications beyond classification including web-scale dataset cleaning, semi-supervised learning, and open-set applications, with qualitative benefits for decision boundary stability.

Abstract: Modern deep learning faces significant challenges with noisy labels, class ambiguity, as well as the need to robustly reject out-of-distribution or corrupted samples. In this work, we propose a unified framework based on the concept of a “drainage node’’ which we add at the output of the network. The node serves to reallocate probability mass toward uncertainty, while preserving desirable properties such as end-to-end training and differentiability. This mechanism provides a natural escape route for highly ambiguous, anomalous, or noisy samples, particularly relevant for instance-dependent and asymmetric label noise. In systematic experiments involving the addition of varying proportions of instance-dependent noise or asymmetric noise to CIFAR-10/100 labels, our drainage formulation achieves an accuracy increase of up to 9% over existing approaches in the high-noise regime. Our results on real-world datasets, such as mini-WebVision, mini-ImageNet and Clothing-1M, match or surpass existing state-of-the-art methods. Qualitative analysis reveals a denoising effect, where the drainage neuron consistently absorbs corrupt, mislabeled, or outlier data, leading to more stable decision boundaries. Furthermore, our drainage formulation enables applications well beyond classification, with immediate benefits for web-scale, semi-supervised dataset cleaning, and open-set applications.

Method: 1) Introduce MERIT dataset: 320,000 queries with 135,000 products in 5 languages across 7 product categories. 2) Propose Coral fine-tuning framework: adapts pre-trained MLLMs with embedding reconstruction to preserve fine-grained conditional elements and contrastive learning to extract comprehensive global semantics.

Result: Coral achieves 45.9% performance improvement over conventional approaches on MERIT dataset. Shows strong generalization capabilities validated across 8 established retrieval benchmarks. Identifies critical limitation of existing models: they focus solely on global semantic information while neglecting specific conditional elements in queries.

Conclusion: The paper establishes foundation for future research in interleaved multi-condition semantic retrieval through three key contributions: novel multilingual dataset (MERIT), identification of critical limitations in existing approaches, and innovative fine-tuning framework (Coral) that significantly improves performance.

Abstract: Semantic retrieval is crucial for modern applications yet remains underexplored in current research. Existing datasets are limited to single languages, single images, or singular retrieval conditions, often failing to fully exploit the expressive capacity of visual information as evidenced by maintained performance when images are replaced with captions. However, practical retrieval scenarios frequently involve interleaved multi-condition queries with multiple images. Hence, this paper introduces MERIT, the first multilingual dataset for interleaved multi-condition semantic retrieval, comprising 320,000 queries with 135,000 products in 5 languages, covering 7 distinct product categories. Extensive experiments on MERIT identify existing models’s limitation: focusing solely on global semantic information while neglecting specific conditional elements in queries. Consequently, we propose Coral, a novel fine-tuning framework that adapts pre-trained MLLMs by integrating embedding reconstruction to preserve fine-grained conditional elements and contrastive learning to extract comprehensive global semantics. Experiments demonstrate that Coral achieves a 45.9% performance improvement over conventional approaches on MERIT, with strong generalization capabilities validated across 8 established retrieval benchmarks. Collectively, our contributions - a novel dataset, identification of critical limitations in existing approaches, and an innovative fine-tuning framework - establish a foundation for future research in interleaved multi-condition semantic retrieval.

Method: Used a model-agnostic forensic evaluation pipeline to test three restoration approaches: 3D reconstruction (NextFace), 2D frontalization (CFR-GAN), and feature enhancement (CodeFormer). Evaluated both naive and selective application strategies.

Result: Naive application of restoration techniques substantially degrades facial recognition accuracy. However, selective application of CFR-GAN combined with CodeFormer yields meaningful improvements in accuracy.

Conclusion: Targeted, selective application of AI-driven restoration techniques (specifically CFR-GAN + CodeFormer) can improve facial recognition accuracy in challenging real-world conditions, but careful implementation is crucial to avoid performance degradation.

Abstract: Biometric facial recognition models often demonstrate significant decreases in accuracy when processing real-world images, often characterized by poor quality, non-frontal subject poses, and subject occlusions. We investigate whether targeted, AI-driven, head-pose correction and image restoration can improve recognition accuracy. Using a model-agnostic, large-scale, forensic-evaluation pipeline, we assess the impact of three restoration approaches: 3D reconstruction (NextFace), 2D frontalization (CFR-GAN), and feature enhancement (CodeFormer). We find that naive application of these techniques substantially degrades facial recognition accuracy. However, we also find that selective application of CFR-GAN combined with CodeFormer yields meaningful improvements.

Method: Flux4D directly predicts 3D Gaussians and their motion dynamics using only photometric losses and “as static as possible” regularization. It learns to decompose dynamic elements from raw data without pre-trained models or foundational priors by training across many scenes in a fully unsupervised manner.

Result: Experiments on outdoor driving datasets show Flux4D significantly outperforms existing methods in scalability, generalization, and reconstruction quality. It enables efficient reconstruction within seconds, scales effectively to large datasets, and generalizes well to unseen environments including rare and unknown objects.

Conclusion: Flux4D provides a simple yet effective framework for 4D reconstruction of large-scale dynamic scenes that addresses key limitations of existing methods through its unsupervised approach, scalability, and strong generalization capabilities.

Abstract: Reconstructing large-scale dynamic scenes from visual observations is a fundamental challenge in computer vision, with critical implications for robotics and autonomous systems. While recent differentiable rendering methods such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have achieved impressive photorealistic reconstruction, they suffer from scalability limitations and require annotations to decouple actor motion. Existing self-supervised methods attempt to eliminate explicit annotations by leveraging motion cues and geometric priors, yet they remain constrained by per-scene optimization and sensitivity to hyperparameter tuning. In this paper, we introduce Flux4D, a simple and scalable framework for 4D reconstruction of large-scale dynamic scenes. Flux4D directly predicts 3D Gaussians and their motion dynamics to reconstruct sensor observations in a fully unsupervised manner. By adopting only photometric losses and enforcing an “as static as possible” regularization, Flux4D learns to decompose dynamic elements directly from raw data without requiring pre-trained supervised models or foundational priors simply by training across many scenes. Our approach enables efficient reconstruction of dynamic scenes within seconds, scales effectively to large datasets, and generalizes well to unseen environments, including rare and unknown objects. Experiments on outdoor driving datasets show Flux4D significantly outperforms existing methods in scalability, generalization, and reconstruction quality.

Method: Proposes Multi-resolution Retrieval-Detection (MRD) with two key components: 1) Multi-resolution semantic fusion that integrates semantic similarity maps from different resolutions to preserve object integrity, and 2) Open-vocabulary object detection using sliding-window approach for direct object localization.

Result: Experiments on high-resolution image understanding benchmarks with different MLLMs demonstrate the effectiveness of the approach.

Conclusion: MRD provides an effective training-free solution for high-resolution image understanding by addressing object fragmentation issues through multi-resolution processing and direct object detection.

Abstract: Understanding high-resolution images remains a significant challenge for multimodal large language models (MLLMs). Recent study address this issue by dividing the image into smaller crops and computing the semantic similarity between each crop and a query using a pretrained retrieval-augmented generation (RAG) model. The most relevant crops are then selected to localize the target object and suppress irrelevant information. However, such crop-based processing can fragment complete objects across multiple crops, thereby disrupting the computation of semantic similarity. In our experiments, we find that image crops of objects with different sizes are better handled at different resolutions. Based on this observation, we propose Multi-resolution Retrieval-Detection (MRD), a training-free framework for high-resolution image understanding. To address the issue of semantic similarity bias caused by objects being split across different image crops, we propose a multi-resolution semantic fusion method, which integrates semantic similarity maps obtained at different resolutions to produce more accurate semantic information and preserve the integrity of target objects. Furthermore, to achieve direct localization of target objects at a global scale, we introduce an open-vocalbulary object detection (OVD) model that identifies object regions using a sliding-window approach.Experiments on high-resolution image understanding benchmarks using different MLLMs demonstrate the effectiveness of our approach.

Method: Leverage visual capabilities of multimodal LLMs (GPT-4o and GPT-5) to perform object counting in zero-shot manner using only textual prompts, without any training or visual exemplars.

Result: Models achieve performance comparable to state-of-the-art zero-shot approaches on FSC-147, sometimes even surpassing them. Evaluated on both FSC-147 and CARPK datasets.

Conclusion: Multimodal LLMs can effectively perform zero-shot object counting using only textual prompts, demonstrating their potential for unsupervised counting tasks without requiring annotated data or visual exemplars.

Abstract: Zero-shot object counting attempts to estimate the number of object instances belonging to novel categories that the vision model performing the counting has never encountered during training. Existing methods typically require large amount of annotated data and often require visual exemplars to guide the counting process. However, large language models (LLMs) are powerful tools with remarkable reasoning and data understanding abilities, which suggest the possibility of utilizing them for counting tasks without any supervision. In this work we aim to leverage the visual capabilities of two multi-modal LLMs, GPT-4o and GPT-5, to perform object counting in a zero-shot manner using only textual prompts. We evaluate both models on the FSC-147 and CARPK datasets and provide a comparative analysis. Our findings show that the models achieve performance comparable to the state-of-the-art zero-shot approaches on FSC-147, in some cases, even surpass them.

Method: Two-stage LLM approach: 1) Predict object semantics from point clouds, 2) Assign plausible materials to each geometric segment conditioned on inferred semantics. Both stages operate zero-shot without task-specific training.

Result: Achieves high semantic and material plausibility across 1,000 shapes from Fusion/ABS and ShapeNet datasets, evaluated using LLM-as-a-Judge implemented in DeepEval.

Conclusion: Language models can serve as general-purpose priors for bridging geometric reasoning and material understanding in 3D data, demonstrating effective zero-shot material inference.

Abstract: Most existing 3D shape datasets and models focus solely on geometry, overlooking the material properties that determine how objects appear. We introduce a two-stage large language model (LLM) based method for inferring material composition directly from 3D point clouds with coarse segmentations. Our key insight is to decouple reasoning about what an object is from what it is made of. In the first stage, an LLM predicts the object’s semantic; in the second stage, it assigns plausible materials to each geometric segment, conditioned on the inferred semantics. Both stages operate in a zero-shot manner, without task-specific training. Because existing datasets lack reliable material annotations, we evaluate our method using an LLM-as-a-Judge implemented in DeepEval. Across 1,000 shapes from Fusion/ABS and ShapeNet, our method achieves high semantic and material plausibility. These results demonstrate that language models can serve as general-purpose priors for bridging geometric reasoning and material understanding in 3D data.

Method: Two key innovations: 1) Multi-scale feature grids integrated into lightweight neural representation, using higher resolution grids to improve INR performance at low complexity; 2) Octree-based context model for entropy coding high-dimensional feature grids to accelerate the entropy coding module.

Result: NVRC-Lite outperforms C3 (best lightweight INR-based video codec) with up to 21.03% BD-rate savings in PSNR and 23.06% in MS-SSIM, while achieving 8.4x encoding and 2.5x decoding speedup.

Conclusion: NVRC-Lite successfully extends NVRC toward lightweight representations with improved performance and practical coding speeds, making INR-based video compression more viable for real-world applications.

Abstract: Recent works have demonstrated the viability of utilizing over-fitted implicit neural representations (INRs) as alternatives to autoencoder-based models for neural video compression. Among these INR-based video codecs, Neural Video Representation Compression (NVRC) was the first to adopt a fully end-to-end compression framework that compresses INRs, achieving state-of-the-art performance. Moreover, some recently proposed lightweight INRs have shown comparable performance to their baseline codecs with computational complexity lower than 10kMACs/pixel. In this work, we extend NVRC toward lightweight representations, and propose NVRC-Lite, which incorporates two key changes. Firstly, we integrated multi-scale feature grids into our lightweight neural representation, and the use of higher resolution grids significantly improves the performance of INRs at low complexity. Secondly, we address the issue that existing INRs typically leverage autoregressive models for entropy coding: these are effective but impractical due to their slow coding speed. In this work, we propose an octree-based context model for entropy coding high-dimensional feature grids, which accelerates the entropy coding module of the model. Our experimental results demonstrate that NVRC-Lite outperforms C3, one of the best lightweight INR-based video codecs, with up to 21.03% and 23.06% BD-rate savings when measured in PSNR and MS-SSIM, respectively, while achieving 8.4x encoding and 2.5x decoding speedup. The implementation of NVRC-Lite will be made available.

Method: Proposes a practical noise synthesis method requiring only one noisy image and one dark frame per ISO. Uses Poisson distribution for signal-dependent noise and Fourier-domain spectral sampling for signal-independent noise to model spatial and statistical properties of real sensor noise.

Result: The method generates diverse noise realizations that accurately model real sensor noise without relying on simplified parametric models or large clean-noisy pairs. Leads to state-of-the-art performances on multiple low-light denoising benchmarks.

Conclusion: The proposed noise synthesis approach is accurate, practical, and enables effective training of denoisers with minimal data requirements, achieving superior performance on low-light denoising tasks.

Abstract: Raw images taken in low-light conditions are very noisy due to low photon count and sensor noise. Learning-based denoisers have the potential to reconstruct high-quality images. For training, however, these denoisers require large paired datasets of clean and noisy images, which are difficult to collect. Noise synthesis is an alternative to large-scale data acquisition: given a clean image, we can synthesize a realistic noisy counterpart. In this work, we propose a general and practical noise synthesis method that requires only one single noisy image and one single dark frame per ISO setting. We represent signal-dependent noise with a Poisson distribution and introduce a Fourier-domain spectral sampling algorithm to accurately model signal-independent noise. The latter generates diverse noise realizations that maintain the spatial and statistical properties of real sensor noise. As opposed to competing approaches, our method neither relies on simplified parametric models nor on large sets of clean-noisy image pairs. Our synthesis method is not only accurate and practical, it also leads to state-of-the-art performances on multiple low-light denoising benchmarks.

Method: PixPerfect uses three key components: (1) a differentiable discriminative pixel space that amplifies/suppresses subtle color and texture discrepancies, (2) a comprehensive artifact simulation pipeline that exposes the refiner to realistic local editing artifacts during training, and (3) a direct pixel-space refinement scheme for broad applicability across diverse latent representations and tasks.

Result: Extensive experiments on inpainting, object removal, and insertion benchmarks show that PixPerfect substantially enhances perceptual fidelity and downstream editing performance, establishing a new standard for robust and high-fidelity localized image editing.

Conclusion: PixPerfect delivers seamless, high-fidelity local edits across diverse LDM architectures and tasks, effectively addressing pixel-level inconsistencies that previous methods failed to fully eliminate.

Abstract: Latent Diffusion Models (LDMs) have markedly advanced the quality of image inpainting and local editing. However, the inherent latent compression often introduces pixel-level inconsistencies, such as chromatic shifts, texture mismatches, and visible seams along editing boundaries. Existing remedies, including background-conditioned latent decoding and pixel-space harmonization, usually fail to fully eliminate these artifacts in practice and do not generalize well across different latent representations or tasks. We introduce PixPerfect, a pixel-level refinement framework that delivers seamless, high-fidelity local edits across diverse LDM architectures and tasks. PixPerfect leverages (i) a differentiable discriminative pixel space that amplifies and suppresses subtle color and texture discrepancies, (ii) a comprehensive artifact simulation pipeline that exposes the refiner to realistic local editing artifacts during training, and (iii) a direct pixel-space refinement scheme that ensures broad applicability across diverse latent representations and tasks. Extensive experiments on inpainting, object removal, and insertion benchmarks demonstrate that PixPerfect substantially enhances perceptual fidelity and downstream editing performance, establishing a new standard for robust and high-fidelity localized image editing.

Method: Systematic evaluation of deep learning architectures (custom CNNs and Transformers) for multi-class fire classification. Introduces PyroFocus, a two-stage pipeline: 1) fire classification, 2) fire radiative power (FRP) regression/segmentation to reduce inference time and computational cost. Uses NASA’s MASTER data (similar to next-gen fire sensors).

Result: The two-stage pipeline achieves strong trade-offs between speed and accuracy, demonstrating significant potential for real-time edge deployment in future wildfire monitoring missions.

Conclusion: PyroFocus offers an effective solution for real-time wildfire detection and intensity estimation that balances accuracy with computational efficiency, making it suitable for onboard deployment in airborne/spaceborne missions.

Abstract: Rapid and accurate wildfire detection is crucial for emergency response and environmental management. In airborne and spaceborne missions, real-time algorithms must distinguish between no fire, active fire, and post-fire conditions, and estimate fire intensity. Multispectral and hyperspectral thermal imagers provide rich spectral information, but high data dimensionality and limited onboard resources make real-time processing challenging. As wildfires increase in frequency and severity, the need for low-latency and computationally efficient onboard detection methods is critical. We present a systematic evaluation of multiple deep learning architectures, including custom Convolutional Neural Networks (CNNs) and Transformer-based models, for multi-class fire classification. We also introduce PyroFocus, a two-stage pipeline that performs fire classification followed by fire radiative power (FRP) regression or segmentation to reduce inference time and computational cost for onboard deployment. Using data from NASA’s MODIS/ASTER Airborne Simulator (MASTER), which is similar to a next-generation fire detection sensor, we compare accuracy, inference latency, and resource efficiency. Experimental results show that the proposed two-stage pipeline achieves strong trade-offs between speed and accuracy, demonstrating significant potential for real-time edge deployment in future wildfire monitoring missions.

Method: 1) H²U3D dataset: automated annotation pipeline creates hierarchical coarse-to-fine visual representations and generates diverse QA pairs with chain-of-thought annotations. 2) SpatialReasoner framework: active perception that autonomously invokes spatial tools to explore 3D scenes based on queries, trained via two-stage strategy (supervised cold start + RL with adaptive exploration reward).

Result: SpatialReasoner achieves SOTA on H²U3D, outperforming GPT-4o and Gemini-2.5-Pro. Notably uses only 3-4 images on average vs. baselines requiring 16+ images, demonstrating efficient exploration.

Conclusion: The coarse-to-fine active exploration paradigm enables effective spatial reasoning in large-scale 3D environments with significantly reduced visual observations, advancing house-scale scene understanding.

Abstract: Spatial reasoning in large-scale 3D environments remains challenging for current vision-language models, which are typically constrained to room-scale scenarios. We introduce H$^2$U3D (Holistic House Understanding in 3D), a 3D visual question answering dataset designed for house-scale scene understanding. H$^2$U3D features multi-floor environments spanning up to three floors and 10-20 rooms, covering more than 300 m$^2$. Through an automated annotation pipeline, it constructs hierarchical coarse-to-fine visual representations and generates diverse question-answer pairs with chain-of-thought annotations. We further propose SpatialReasoner, an active perception framework that autonomously invokes spatial tools to explore 3D scenes based on textual queries. SpatialReasoner is trained through a two-stage strategy: a supervised cold start followed by reinforcement learning with an adaptive exploration reward that promotes efficient exploration while discouraging redundant operations. Extensive experiments demonstrate that SpatialReasoner achieves state-of-the-art performance on H$^2$U3D, outperforming strong baselines including GPT-4o and Gemini-2.5-Pro. Notably, our method attains superior results while using only 3-4 images in total on average, compared to baselines requiring 16+ images, highlighting the effectiveness of our coarse-to-fine active exploration paradigm.

Method: Proposes NavMapFusion, a diffusion-based framework that performs iterative denoising conditioned on both high-fidelity sensor data and low-fidelity navigation maps. The approach treats discrepancies between prior maps and online perception as noise in the diffusion process.

Result: On nuScenes benchmark, NavMapFusion conditioned on coarse OpenStreetMap road lines achieves 21.4% relative improvement at 100m range, with even stronger improvements at larger perception ranges, while maintaining real-time capabilities.

Conclusion: Diffusion-based map construction provides a robust framework for map fusion, where consistent regions reinforce map construction while outdated segments are suppressed. This enables generation of accurate, up-to-date environment representations for safer autonomous driving.

Abstract: Accurate environmental representations are essential for autonomous driving, providing the foundation for safe and efficient navigation. Traditionally, high-definition (HD) maps are providing this representation of the static road infrastructure to the autonomous system a priori. However, because the real world is constantly changing, such maps must be constructed online from on-board sensor data. Navigation-grade standard-definition (SD) maps are widely available, but their resolution is insufficient for direct deployment. Instead, they can be used as coarse prior to guide the online map construction process. We propose NavMapFusion, a diffusion-based framework that performs iterative denoising conditioned on high-fidelity sensor data and on low-fidelity navigation maps. This paper strives to answer: (1) How can coarse, potentially outdated navigation maps guide online map construction? (2) What advantages do diffusion models offer for map fusion? We demonstrate that diffusion-based map construction provides a robust framework for map fusion. Our key insight is that discrepancies between the prior map and online perception naturally correspond to noise within the diffusion process; consistent regions reinforce the map construction, whereas outdated segments are suppressed. On the nuScenes benchmark, NavMapFusion conditioned on coarse road lines from OpenStreetMap data reaches a 21.4% relative improvement on 100 m, and even stronger improvements on larger perception ranges, while maintaining real-time capabilities. By fusing low-fidelity priors with high-fidelity sensor data, the proposed method generates accurate and up-to-date environment representations, guiding towards safer and more reliable autonomous driving. The code is available at https://github.com/tmonnin/navmapfusion

Method: Proposes Step-by-Step Layered Design Generation problem setting and SLEDGE framework that models each design update as atomic, layered changes over previous states, grounded in sequential instructions. Leverages multi-modal LLMs to handle this incremental generation process.

Result: Exhaustive experimental analysis shows efficacy of the approach compared to state-of-the-art methods. Introduces new evaluation suite including dataset and benchmark for this problem setting.

Conclusion: The work addresses a pragmatic and under-explored research area, proposing a more realistic approach to design generation that better aligns with actual creative processes. The authors hope to attract attention to this important research direction.

Abstract: Design generation, in its essence, is a step-by-step process where designers progressively refine and enhance their work through careful modifications. Despite this fundamental characteristic, existing approaches mainly treat design synthesis as a single-step generation problem, significantly underestimating the inherent complexity of the creative process. To bridge this gap, we propose a novel problem setting called Step-by-Step Layered Design Generation, which tasks a machine learning model with generating a design that adheres to a sequence of instructions from a designer. Leveraging recent advancements in multi-modal LLMs, we propose SLEDGE: Step-by-step LayEred Design GEnerator to model each update to a design as an atomic, layered change over its previous state, while being grounded in the instruction. To complement our new problem setting, we introduce a new evaluation suite, including a dataset and a benchmark. Our exhaustive experimental analysis and comparison with state-of-the-art approaches tailored to our new setup demonstrate the efficacy of our approach. We hope our work will attract attention to this pragmatic and under-explored research area.

Method: ProtoEFNet uses video-based prototype learning for continuous EF regression. It learns dynamic spatiotemporal prototypes capturing clinically meaningful cardiac motion patterns. A Prototype Angular Separation (PAS) loss enforces discriminative representations across the continuous EF spectrum.

Result: On the EchonetDynamic dataset, ProtoEFNet achieves accuracy comparable to non-interpretable models while providing clinically relevant insights. The PAS loss boosts performance with a 2% increase in F1 score from 77.67±2.68 to 79.64±2.10.

Conclusion: ProtoEFNet addresses the interpretability gap in EF estimation by learning clinically meaningful prototypes, offering both accurate predictions and transparent reasoning that can enhance clinical trust and adoption.

Abstract: Ejection fraction (EF) is a crucial metric for assessing cardiac function and diagnosing conditions such as heart failure. Traditionally, EF estimation requires manual tracing and domain expertise, making the process time-consuming and subject to interobserver variability. Most current deep learning methods for EF prediction are black-box models with limited transparency, which reduces clinical trust. Some post-hoc explainability methods have been proposed to interpret the decision-making process after the prediction is made. However, these explanations do not guide the model’s internal reasoning and therefore offer limited reliability in clinical applications. To address this, we introduce ProtoEFNet, a novel video-based prototype learning model for continuous EF regression. The model learns dynamic spatiotemporal prototypes that capture clinically meaningful cardiac motion patterns. Additionally, the proposed Prototype Angular Separation (PAS) loss enforces discriminative representations across the continuous EF spectrum. Our experiments on the EchonetDynamic dataset show that ProtoEFNet can achieve accuracy on par with its non-interpretable counterpart while providing clinically relevant insight. The ablation study shows that the proposed loss boosts performance with a 2% increase in F1 score from 77.67$\pm$2.68 to 79.64$\pm$2.10. Our source code is available at: https://github.com/DeepRCL/ProtoEF

Method: HalluGen uses a diffusion-based framework to synthesize realistic hallucinations with controllable type, location, and severity. This generates perceptually realistic but semantically incorrect outputs, which are used to create a large-scale hallucination dataset of 4,350 annotated images from 1,450 brain MR images for low-field MRI enhancement.

Result: The framework successfully creates realistic hallucinations (segmentation IoU drops from 0.86 to 0.36) and enables two key applications: (1) benchmarking image quality metrics and developing SHAFE, a feature-based metric with soft-attention pooling that improves hallucination sensitivity; and (2) training reference-free hallucination detectors that generalize to real restoration failures.

Conclusion: HalluGen and its open dataset establish the first scalable foundation for systematically evaluating hallucinations in safety-critical image restoration, addressing a critical gap in reliable medical imaging and other high-stakes applications.

Abstract: Generative models are prone to hallucinations: plausible but incorrect structures absent in the ground truth. This issue is problematic in image restoration for safety-critical domains such as medical imaging, industrial inspection, and remote sensing, where such errors undermine reliability and trust. For example, in low-field MRI, widely used in resource-limited settings, restoration models are essential for enhancing low-quality scans, yet hallucinations can lead to serious diagnostic errors. Progress has been hindered by a circular dependency: evaluating hallucinations requires labeled data, yet such labels are costly and subjective. We introduce HalluGen, a diffusion-based framework that synthesizes realistic hallucinations with controllable type, location, and severity, producing perceptually realistic but semantically incorrect outputs (segmentation IoU drops from 0.86 to 0.36). Using HalluGen, we construct the first large-scale hallucination dataset comprising 4,350 annotated images derived from 1,450 brain MR images for low-field enhancement, enabling systematic evaluation of hallucination detection and mitigation. We demonstrate its utility in two applications: (1) benchmarking image quality metrics and developing Semantic Hallucination Assessment via Feature Evaluation (SHAFE), a feature-based metric with soft-attention pooling that improves hallucination sensitivity over traditional metrics; and (2) training reference-free hallucination detectors that generalize to real restoration failures. Together, HalluGen and its open dataset establish the first scalable foundation for evaluating hallucinations in safety-critical image restoration.

Method: Proposes Neural Video Representation Compression (NVRC) framework with novel entropy coding and quantization models enabling fully end-to-end optimization. Also introduces hierarchical model compression framework to minimize bitrate overhead from entropy models.

Result: NVRC achieves 24% average coding gain over VVC VTM (Random Access) on UVG dataset measured in PSNR, outperforming many conventional and learning-based benchmark codecs.

Conclusion: NVRC demonstrates that INR-based video codecs can outperform state-of-the-art standard codecs through proper end-to-end optimization and efficient model compression, marking the first time an INR-based approach achieves such performance.

Abstract: Recent advances in implicit neural representation (INR)-based video coding have demonstrated its potential to compete with both conventional and other learning-based approaches. With INR methods, a neural network is trained to overfit a video sequence, with its parameters compressed to obtain a compact representation of the video content. However, although promising results have been achieved, the best INR-based methods are still out-performed by the latest standard codecs, such as VVC VTM, partially due to the simple model compression techniques employed. In this paper, rather than focusing on representation architectures as in many existing works, we propose a novel INR-based video compression framework, Neural Video Representation Compression (NVRC), targeting compression of the representation. Based on the novel entropy coding and quantization models proposed, NVRC, for the first time, is able to optimize an INR-based video codec in a fully end-to-end manner. To further minimize the additional bitrate overhead introduced by the entropy models, we have also proposed a new model compression framework for coding all the network, quantization and entropy model parameters hierarchically. Our experiments show that NVRC outperforms many conventional and learning-based benchmark codecs, with a 24% average coding gain over VVC VTM (Random Access) on the UVG dataset, measured in PSNR. As far as we are aware, this is the first time an INR-based video codec achieving such performance. The implementation of NVRC will be released.

Method: Controlled comparison of 16 modern deep learning architectures spanning 2D/3D convolutional, hybrid, and volumetric transformer families for subclinical keratoconus detection from 3D anterior segment OCT volumes. Includes mechanistic analyses of attention-distance measurements, representational similarity, and auxiliary age/sex prediction tasks.

Result: Hierarchical attention models offer superior and more parameter-efficient inductive bias, achieving 21-23% higher sensitivity and specificity in the sparse anomaly regime. Hierarchical windowing produces effective receptive fields matched to intermediate, multi-slice extent of subclinical abnormalities. Required spatial integration length shifts significantly based on signal strength.

Conclusion: Hierarchical attention provides a principled and effective approach for early pathological change analysis in 3D medical imaging, establishing design guidance for future volumetric anomaly detection systems. The findings demonstrate that precise spatial scale alignment through hierarchical windowing avoids excessive CNN locality and diffuse global attention.

Abstract: The detection of weak, spatially distributed anomalies in volumetric medical imaging remains a major challenge. The subtle, non-adjacent nature of early disease signals is often lost due to suboptimal architectural inductive biases: 2D/3D CNNs impose strong locality, while ViTs diffuse unconstrained global attention. This conflict leaves the optimal inductive structure for robust, sparse volumetric pattern recognition unresolved. This study presents a controlled comparison of sixteen modern deep learning architectures spanning 2D/3D convolutional, hybrid, and volumetric transformer families for subclinical keratoconus (SKC) detection from 3D anterior segment OCT volumes. We demonstrate that hierarchical attention models offer a superior and more parameter-efficient inductive bias, surpassing the performance of both 2D and 3D CNNs and ViTs. Our results show 21-23% higher sensitivity and specificity in the sparse anomaly (subclinical) regime. Mechanistic analyses reveal that this advantage stems from precise spatial scale alignment: hierarchical windowing produces effective receptive fields matched to the intermediate, multi-slice extent of subclinical abnormalities. This avoids excessive CNN locality and diffuse global attention. Attention-distance measurements confirm a key insight into architectural adaptation: the required spatial integration length shifts significantly based on the signal strength, with subclinical cases necessitating longer integration compared to both healthy and manifest disease states. Representational similarity and auxiliary age/sex prediction tasks further support the generalizability of these inductive principles. The findings provide design guidance for future volumetric anomaly detection systems, establishing hierarchical attention as a principled and effective approach for early pathological change analysis in 3D medical imaging.

Method: SeeU uses a three-stage framework: 1) 2D→4D: reconstructs 4D world from sparse monocular 2D frames, 2) discrete 4D→continuous 4D: learns continuous dynamics on low-rank representation with physical constraints, 3) 4D→2D: rolls world forward in time and re-projects to 2D at sampled times/viewpoints, generating unseen regions using spatial-temporal context awareness.

Result: SeeU achieves continuous and physically-consistent novel visual generation, demonstrating strong potential in multiple tasks including unseen temporal generation, unseen spatial generation, and video editing.

Conclusion: By modeling dynamics in 4D space-time, SeeU provides a principled approach for generating unseen visual content with physical consistency, overcoming limitations of direct 2D processing and enabling applications in temporal/spatial generation and video editing.

Abstract: Images and videos are discrete 2D projections of the 4D world (3D space + time). Most visual understanding, prediction, and generation operate directly on 2D observations, leading to suboptimal performance. We propose SeeU, a novel approach that learns the continuous 4D dynamics and generate the unseen visual contents. The principle behind SeeU is a new 2D$\to$4D$\to$2D learning framework. SeeU first reconstructs the 4D world from sparse and monocular 2D frames (2D$\to$4D). It then learns the continuous 4D dynamics on a low-rank representation and physical constraints (discrete 4D$\to$continuous 4D). Finally, SeeU rolls the world forward in time, re-projects it back to 2D at sampled times and viewpoints, and generates unseen regions based on spatial-temporal context awareness (4D$\to$2D). By modeling dynamics in 4D, SeeU achieves continuous and physically-consistent novel visual generation, demonstrating strong potentials in multiple tasks including unseen temporal generation, unseen spatial generation, and video editing.

Method: Used DenseNet169 with Squeeze-and-Excitation blocks for attention-based feature extraction, Focal Loss to handle class imbalance, and Feature Pyramid Network for multi-scale feature fusion. Also developed an SVM model using MobileNetV2 for feature extraction. Integrated Grad-CAM for visualizing decision regions in CT scans and SHAP for explaining feature contributions in the SVM model.

Result: Both DenseNet169 and SVM models achieved 98% accuracy on the IQOTHNCCD lung cancer dataset, demonstrating high performance for classifying CT scans into Normal, Benign, and Malignant categories.

Conclusion: The proposed deep learning system shows strong potential for real-world medical practice by providing high accuracy, transparency, and robustness in lung cancer diagnosis, with interpretability features that enhance clinical trust and adoption.

Abstract: Lung cancer is a very deadly disease worldwide, and its early diagnosis is crucial for increasing patient survival rates. Computed tomography (CT) scans are widely used for lung cancer diagnosis as they can give detailed lung structures. However, manual interpretation is time-consuming and prone to human error. To surmount this challenge, the study proposes a deep learning-based automatic lung cancer classification system to enhance detection accuracy and interpretability. The IQOTHNCCD lung cancer dataset is utilized, which is a public CT scan dataset consisting of cases categorized into Normal, Benign, and Malignant and used DenseNet169, which includes Squeezeand-Excitation blocks for attention-based feature extraction, Focal Loss for handling class imbalance, and a Feature Pyramid Network (FPN) for multi-scale feature fusion. In addition, an SVM model was developed using MobileNetV2 for feature extraction, improving its classification performance. For model interpretability enhancement, the study integrated Grad-CAM for the visualization of decision-making regions in CT scans and SHAP (Shapley Additive Explanations) for explanation of feature contributions within the SVM model. Intensive evaluation was performed, and it was found that both DenseNet169 and SVM models achieved 98% accuracy, suggesting their robustness for real-world medical practice. These results open up the potential for deep learning to improve the diagnosis of lung cancer by a higher level of accuracy, transparency, and robustness.

Method: 1) Created FireSentry dataset with synchronized UAV platforms collecting visible/infrared video streams, environmental measurements, and validated fire masks. 2) Established benchmark with physics-based, data-driven, and generative models. 3) Proposed FiReDiff paradigm that first predicts future infrared video sequences, then segments fire masks based on generated dynamics.

Result: FiReDiff achieves state-of-the-art performance: video quality gains of 39.2% PSNR, 36.1% SSIM, 50.0% LPIPS, 29.4% FVD; mask accuracy gains of 3.3% AUPRC, 59.1% F1 score, 42.9% IoU, 62.5% MSE compared to generative models.

Conclusion: FireSentry benchmark dataset and FiReDiff paradigm collectively advance fine-grained wildfire forecasting and dynamic disaster simulation, with the dataset publicly available for further research.

Abstract: Fine-grained wildfire spread prediction is crucial for enhancing emergency response efficacy and decision-making precision. However, existing research predominantly focuses on coarse spatiotemporal scales and relies on low-resolution satellite data, capturing only macroscopic fire states while fundamentally constraining high-precision localized fire dynamics modeling capabilities. To bridge this gap, we present FireSentry, a provincial-scale multi-modal wildfire dataset characterized by sub-meter spatial and sub-second temporal resolution. Collected using synchronized UAV platforms, FireSentry provides visible and infrared video streams, in-situ environmental measurements, and manually validated fire masks. Building on FireSentry, we establish a comprehensive benchmark encompassing physics-based, data-driven, and generative models, revealing the limitations of existing mask-only approaches. Our analysis proposes FiReDiff, a novel dual-modality paradigm that first predicts future video sequences in the infrared modality, and then precisely segments fire masks in the mask modality based on the generated dynamics. FiReDiff achieves state-of-the-art performance, with video quality gains of 39.2% in PSNR, 36.1% in SSIM, 50.0% in LPIPS, 29.4% in FVD, and mask accuracy gains of 3.3% in AUPRC, 59.1% in F1 score, 42.9% in IoU, and 62.5% in MSE when applied to generative models. The FireSentry benchmark dataset and FiReDiff paradigm collectively advance fine-grained wildfire forecasting and dynamic disaster simulation. The processed benchmark dataset is publicly available at: https://github.com/Munan222/FireSentry-Benchmark-Dataset.

Method: Proposes ShelfGaussian with two key components: 1) Multi-Modal Gaussian Transformer that enables Gaussians to query features from diverse sensor modalities, and 2) Shelf-Supervised Learning Paradigm that efficiently optimizes Gaussians with vision foundation model features jointly at both 2D image and 3D scene levels.

Result: Achieves state-of-the-art zero-shot semantic occupancy prediction on Occ3D-nuScenes benchmark. Successfully evaluated on unmanned ground vehicle (UGV) across diverse urban scenarios, demonstrating strong in-the-wild performance for various perception and planning tasks.

Conclusion: ShelfGaussian effectively bridges the gap between closed-set and open-set Gaussian methods by leveraging vision foundation models for supervision, enabling open-vocabulary multi-modal 3D scene understanding with strong performance across perception and planning tasks.

Abstract: We introduce ShelfGaussian, an open-vocabulary multi-modal Gaussian-based 3D scene understanding framework supervised by off-the-shelf vision foundation models (VFMs). Gaussian-based methods have demonstrated superior performance and computational efficiency across a wide range of scene understanding tasks. However, existing methods either model objects as closed-set semantic Gaussians supervised by annotated 3D labels, neglecting their rendering ability, or learn open-set Gaussian representations via purely 2D self-supervision, leading to degraded geometry and limited to camera-only settings. To fully exploit the potential of Gaussians, we propose a Multi-Modal Gaussian Transformer that enables Gaussians to query features from diverse sensor modalities, and a Shelf-Supervised Learning Paradigm that efficiently optimizes Gaussians with VFM features jointly at 2D image and 3D scene levels. We evaluate ShelfGaussian on various perception and planning tasks. Experiments on Occ3D-nuScenes demonstrate its state-of-the-art zero-shot semantic occupancy prediction performance. ShelfGaussian is further evaluated on an unmanned ground vehicle (UGV) to assess its in the-wild performance across diverse urban scenarios. Project website: https://lunarlab-gatech.github.io/ShelfGaussian/.

Method: MOS framework has two core components: (1) Modality-Consistent Representation Learning (MCRL) uses denoised SAR image processing and class-wise modality alignment loss to align intra-identity feature distributions across modalities. (2) Cross-modal Data Generation and Feature Fusion (CDGF) employs a Brownian bridge diffusion model to synthesize cross-modal samples, which are fused with original features during inference to enhance alignment and discriminability.

Result: Extensive experiments on the HOSS ReID dataset show MOS significantly outperforms state-of-the-art methods across all evaluation protocols. It achieves improvements of +3.0%, +6.2%, and +16.4% in R1 accuracy under ALL to ALL, Optical to SAR, and SAR to Optical settings respectively.

Conclusion: MOS effectively mitigates the optical-SAR modality gap through modality-consistent feature learning and cross-modal data generation, establishing new state-of-the-art performance for cross-modal ship re-identification. The framework demonstrates strong potential for maritime surveillance applications.

Abstract: Cross-modal ship re-identification (ReID) between optical and synthetic aperture radar (SAR) imagery has recently emerged as a critical yet underexplored task in maritime intelligence and surveillance. However, the substantial modality gap between optical and SAR images poses a major challenge for robust identification. To address this issue, we propose MOS, a novel framework designed to mitigate the optical-SAR modality gap and achieve modality-consistent feature learning for optical-SAR cross-modal ship ReID. MOS consists of two core components: (1) Modality-Consistent Representation Learning (MCRL) applies denoise SAR image procession and a class-wise modality alignment loss to align intra-identity feature distributions across modalities. (2) Cross-modal Data Generation and Feature fusion (CDGF) leverages a brownian bridge diffusion model to synthesize cross-modal samples, which are subsequently fused with original features during inference to enhance alignment and discriminability. Extensive experiments on the HOSS ReID dataset demonstrate that MOS significantly surpasses state-of-the-art methods across all evaluation protocols, achieving notable improvements of +3.0%, +6.2%, and +16.4% in R1 accuracy under the ALL to ALL, Optical to SAR, and SAR to Optical settings, respectively. The code and trained models will be released upon publication.

Method: 1) Introduces the ViDiC task for video difference captioning; 2) Creates ViDiC-1K dataset with 1,000 curated video pairs annotated with over 4,000 comparative checklist items across 7 categories; 3) Proposes a dual-checklist evaluation framework using LLM-as-a-Judge protocol to separately measure similarity and difference accuracy.

Result: Experiments on nineteen representative multimodal models reveal a significant performance gap in their comparative description and difference perception abilities, indicating current models struggle with this challenging task.

Conclusion: ViDiC-1K serves as a challenging benchmark that lays a solid foundation for advancing video understanding, edit awareness, and comparative reasoning in multimodal intelligence, highlighting the need for improved video difference perception capabilities.

Abstract: Understanding visual differences between dynamic scenes requires the comparative perception of compositional, spatial, and temporal changes–a capability that remains underexplored in existing vision-language systems. While prior work on Image Difference Captioning (IDC) has enabled models to describe semantic changes between static images, these approaches fail to capture motion continuity, event evolution, or editing consistency over time. We introduce the ViDiC (Video Difference Captioning) task and its corresponding ViDiC-1K dataset, designed to evaluate the ability of Multimodal Large Language Models (MLLMs) to provide fine-grained descriptions of similarities and differences between video pairs. ViDiC-1K comprises 1,000 curated video pairs annotated with over 4,000 comparative checklist items, covering seven categories: subject, style, background, cinematography, motion, location, and playback techniques. To ensure reliable evaluation, we propose a dual-checklist framework that measures the accuracy of similarity and difference separately, based on the LLM-as-a-Judge protocol. Experiments on nineteen representative multimodal models reveal a significant performance gap in their comparative description and difference perception abilities. We hope ViDiC-1K can be a challenging benchmark that lays a solid foundation for advancing video understanding, edit awareness, and comparative reasoning in multimodal intelligence.

Method: YOLOA uses a lightweight detector with object detection and affordance learning branches refined through an LLM adapter. The LLM adapter interacts with preliminary predictions to generate more accurate class priors, box offsets, and affordance gates during training.

Result: Achieves state-of-the-art accuracy (52.8 mAP on ADG-Det, 73.1 mAP on IIT-Heat) with real-time performance (up to 89.77 FPS, and up to 846.24 FPS for lightweight variant), showing excellent accuracy-efficiency trade-off.

Conclusion: YOLOA successfully addresses the “what-where-how” challenge in affordance detection by jointly handling object detection and affordance learning through LLM adapter refinement, achieving both high accuracy and real-time performance.

Abstract: Affordance detection aims to jointly address the fundamental “what-where-how” challenge in embodied AI by understanding “what” an object is, “where” the object is located, and “how” it can be used. However, most affordance learning methods focus solely on “how” objects can be used while neglecting the “what” and “where” aspects. Other affordance detection methods treat object detection and affordance learning as two independent tasks, lacking effective interaction and real-time capability. To overcome these limitations, we introduce YOLO Affordance (YOLOA), a real-time affordance detection model that jointly handles these two tasks via a large language model (LLM) adapter. Specifically, YOLOA employs a lightweight detector consisting of object detection and affordance learning branches refined through the LLM Adapter. During training, the LLM Adapter interacts with object and affordance preliminary predictions to refine both branches by generating more accurate class priors, box offsets, and affordance gates. Experiments on our relabeled ADG-Det and IIT-Heat benchmarks demonstrate that YOLOA achieves state-of-the-art accuracy (52.8 / 73.1 mAP on ADG-Det / IIT-Heat) while maintaining real-time performance (up to 89.77 FPS, and up to 846.24 FPS for the lightweight variant). This indicates that YOLOA achieves an excellent trade-off between accuracy and efficiency.

Method: DM3D introduces: 1) Offset-guided Gaussian sequencing that unifies local resampling and global reordering; 2) Gaussian-based KNN Resampling (GKR) for adaptive neighborhood reorganization; 3) Gaussian-based Differentiable Reordering (GDR) for end-to-end optimization of serialization order; 4) Tri-Path Frequency Fusion module for feature complementarity and aliasing reduction.

Result: Extensive experiments on benchmark datasets show DM3D achieves state-of-the-art performance in classification, few-shot learning, and part segmentation tasks.

Conclusion: Adaptive serialization effectively unlocks the potential of SSMs for point cloud understanding, demonstrating that structure-adaptive approaches overcome the limitations of predefined serialization strategies.

Abstract: State Space Models (SSMs) demonstrate significant potential for long-sequence modeling, but their reliance on input order conflicts with the irregular nature of point clouds. Existing approaches often rely on predefined serialization strategies, which cannot adjust based on diverse geometric structures. To overcome this limitation, we propose \textbf{DM3D}, a deformable Mamba architecture for point cloud understanding. Specifically, DM3D introduces an offset-guided Gaussian sequencing mechanism that unifies local resampling and global reordering within a deformable scan. The Gaussian-based KNN Resampling (GKR) enhances structural awareness by adaptively reorganizing neighboring points, while the Gaussian-based Differentiable Reordering (GDR) enables end-to-end optimization of serialization order. Furthermore, a Tri-Path Frequency Fusion module enhances feature complementarity and reduces aliasing. Together, these components enable structure-adaptive serialization of point clouds. Extensive experiments on benchmark datasets show that DM3D achieves state-of-the-art performance in classification, few-shot learning, and part segmentation, demonstrating that adaptive serialization effectively unlocks the potential of SSMs for point cloud understanding.

Method: Systematic zero-shot evaluation of eight state-of-the-art stereo methods (RAFT-Stereo, IGEV, IGEV++, BridgeDepth, StereoAnywhere, DEFOM, ACVNet, PSMNet, TCstereo) trained exclusively on Scene Flow and evaluated without fine-tuning on four standard benchmarks plus a novel 5,313-pair Canterbury forestry dataset captured with ZED Mini camera.

Result: Foundation models excel on structured scenes (BridgeDepth: 0.23 px on ETH3D, 0.83-1.07 px on KITTI; DEFOM: 0.35-4.65 px across benchmarks), while iterative methods maintain cross-domain robustness (IGEV++: 0.36-6.77 px; IGEV: 0.33-21.91 px). RAFT-Stereo shows catastrophic ETH3D failure (26.23 px EPE, 98% error rate) but normal KITTI performance. DEFOM identified as optimal for forestry with superior depth smoothness, occlusion handling, and cross-domain consistency.

Conclusion: DEFOM emerges as the gold-standard baseline for vegetation depth estimation in forestry applications, demonstrating the importance of scene-dependent method selection and revealing critical cross-domain generalization challenges in stereo depth estimation for specialized environments.

Abstract: Autonomous UAV forestry operations require robust depth estimation methods with strong cross-domain generalization. However, existing evaluations focus on urban and indoor scenarios, leaving a critical gap for specialized vegetation-dense environments. We present the first systematic zero-shot evaluation of eight state-of-the-art stereo methods–RAFT-Stereo, IGEV, IGEV++, BridgeDepth, StereoAnywhere, DEFOM (plus baseline methods ACVNet, PSMNet, TCstereo)–spanning iterative refinement, foundation model, and zero-shot adaptation paradigms. All methods are trained exclusively on Scene Flow and evaluated without fine-tuning on four standard benchmarks (ETH3D, KITTI 2012/2015, Middlebury) plus a novel 5,313-pair Canterbury forestry dataset captured with ZED Mini camera (1920x1080). Performance reveals scene-dependent patterns: foundation models excel on structured scenes (BridgeDepth: 0.23 px on ETH3D, 0.83-1.07 px on KITTI; DEFOM: 0.35-4.65 px across benchmarks), while iterative methods maintain cross-domain robustness (IGEV++: 0.36-6.77 px; IGEV: 0.33-21.91 px). Critical finding: RAFT-Stereo exhibits catastrophic ETH3D failure (26.23 px EPE, 98 percent error rate) due to negative disparity predictions, while performing normally on KITTI (0.90-1.11 px). Qualitative evaluation on Canterbury forestry dataset identifies DEFOM as the optimal gold-standard baseline for vegetation depth estimation, exhibiting superior depth smoothness, occlusion handling, and cross-domain consistency compared to IGEV++, despite IGEV++’s finer detail preservation.

Method: Extracts low-level spatial features from early denoising timesteps of a frozen diffusion model pretrained on natural images, then uses a lightweight FiLM-based fusion module to adaptively integrate spectral information with spatial features for multimodal learning under sparse supervision.

Result: Outperforms state-of-the-art approaches on two recent hyperspectral datasets using only sparse training labels, with ablation studies confirming benefits of diffusion-derived features and spectral-aware fusion.

Conclusion: Pretrained diffusion models can support domain-agnostic, label-efficient representation learning for remote sensing and broader scientific imaging tasks by effectively transferring spatial features to low-texture hyperspectral data.

Abstract: Hyperspectral imaging (HSI) enables detailed land cover classification, yet low spatial resolution and sparse annotations pose significant challenges. We present a label-efficient framework that leverages spatial features from a frozen diffusion model pretrained on natural images. Our approach extracts low-level representations from high-resolution decoder layers at early denoising timesteps, which transfer effectively to the low-texture structure of HSI. To integrate spectral and spatial information, we introduce a lightweight FiLM-based fusion module that adaptively modulates frozen spatial features using spectral cues, enabling robust multimodal learning under sparse supervision. Experiments on two recent hyperspectral datasets demonstrate that our method outperforms state-of-the-art approaches using only the provided sparse training labels. Ablation studies further highlight the benefits of diffusion-derived features and spectral-aware fusion. Overall, our results indicate that pretrained diffusion models can support domain-agnostic, label-efficient representation learning for remote sensing and broader scientific imaging tasks.

Method: Proposes MAGEN (Multi-Agent data GENeration) for enhancing data quality through foundation model-assisted captioning and retrieval verification, and O-MAKE (Ontology-based Multi-Aspect Knowledge-Enhanced) pretraining that decomposes long texts into knowledge aspects for fine-grained alignment and ontology-guided concept modeling.

Result: Achieves state-of-the-art zero-shot performance on disease classification and cross-modal retrieval tasks across eight dermatology datasets, and releases Derm1M-AgentAug dataset with over 400k skin-image-text pairs.

Conclusion: The integrated framework effectively addresses data quality and text complexity challenges in medical VLP, demonstrating superior performance in dermatology applications with released code and augmented dataset.

Abstract: Vision-language pretraining (VLP) has emerged as a powerful paradigm in medical image analysis, enabling representation learning from large-scale image-text pairs without relying on expensive manual annotations. However, existing methods often struggle with the noise inherent in web-collected data and the complexity of unstructured long medical texts. To address these challenges, we propose a novel VLP framework integrating a Multi-Agent data GENeration (MAGEN) system and Ontology-based Multi-Aspect Knowledge-Enhanced (O-MAKE) pretraining. First, MAGEN enhances data quality by synthesizing knowledge-enriched descriptions via a foundation model-assisted captioning and retrieval-based verification pipeline. Second, O-MAKE addresses the difficulty of learning from long, unstructured texts by decomposing them into distinct knowledge aspects. This facilitates fine-grained alignment at both global and patch levels, while explicitly modeling medical concept relationships through ontology-guided mechanisms. We validate our framework in the field of dermatology, where comprehensive experiments demonstrate the effectiveness of each component. Our approach achieves state-of-the-art zero-shot performance on disease classification and cross-modal retrieval tasks across eight datasets. Our code and the augmented dataset Derm1M-AgentAug, comprising over 400k skin-image-text pairs, will be released at https://github.com/SiyuanYan1/Derm1M.

Method: Two 3D nnU-Net models trained on SKM-TEA and OAIZIB-CM datasets; zero-shot predictions fused and refined with geometric rules: connected-component cleaning, 10mm subchondral bone bands, and data-driven tibial lateral/medial split using PCA and k-means.

Result: Post-processing improved macro ASSD from 2.63 to 0.36 mm and HD95 from 25.2 to 3.35 mm (DSC 0.91); zero-shot DSC on SKI-10 was 0.80. Geometric L/M rule produced stable compartments, with only 6-12% of radiomic features strongly correlated with volume/thickness.

Conclusion: LM-CartSeg provides automatic, quality-controlled ROIs and radiomic features with discriminative information beyond simple morphometry, offering practical foundation for multi-center knee OA radiomics studies.

Abstract: Background and Objective: Radiomics of knee MRI requires robust, anatomically meaningful regions of interest (ROIs) that jointly capture cartilage and subchondral bone. Most existing work relies on manual ROIs and rarely reports quality control (QC). We present LM-CartSeg, a fully automatic pipeline for cartilage/bone segmentation, geometric lateral/medial (L/M) compartmentalisation and radiomics analysis. Methods: Two 3D nnU-Net models were trained on SKM-TEA (138 knees) and OAIZIB-CM (404 knees). At test time, zero-shot predictions were fused and refined by simple geometric rules: connected-component cleaning, construction of 10 mm subchondral bone bands in physical space, and a data-driven tibial L/M split based on PCA and k-means. Segmentation was evaluated on an OAIZIB-CM test set (103 knees) and on SKI-10 (100 knees). QC used volume and thickness signatures. From 10 ROIs we extracted 4 650 non-shape radiomic features to study inter-compartment similarity, dependence on ROI size, and OA vs. non-OA classification on OAIZIB-CM Results: Post-processing improved macro ASSD on OAIZIB-CM from 2.63 to 0.36 mm and HD95 from 25.2 to 3.35 mm, with DSC 0.91; zero-shot DSC on SKI-10 was 0.80. The geometric L/M rule produced stable compartments across datasets, whereas a direct L/M nnU-Net showed domain-dependent side swaps. Only 6 to 12 percent of features per ROI were strongly correlated with volume or thickness. Radiomics-based models models restricted to size-linked features. Conclusions: LM-CartSeg yields automatic, QCd ROIs and radiomic features that carry discriminative information beyond simple morphometry, providing a practical foundation for multi-centre knee OA radiomics studies.

Method: Unsupervised framework learns spatially structured 3D keypoints from point cloud data, which then condition an Elucidated Diffusion Model (EDM) to reconstruct full shapes.

Result: Learned keypoints show repeatable spatial structure across instances and support smooth interpolation, capturing geometric variation. Achieves 6 percentage-point improvement in keypoint consistency over prior approaches across diverse object categories.

Conclusion: The method successfully bridges unsupervised keypoint learning with modern generative pipelines, providing compact, interpretable representations that enable effective 3D shape reconstruction and understanding.

Abstract: Understanding and representing the structure of 3D objects in an unsupervised manner remains a core challenge in computer vision and graphics. Most existing unsupervised keypoint methods are not designed for unconditional generative settings, restricting their use in modern 3D generative pipelines; our formulation explicitly bridges this gap. We present an unsupervised framework for learning spatially structured 3D keypoints from point cloud data. These keypoints serve as a compact and interpretable representation that conditions an Elucidated Diffusion Model (EDM) to reconstruct the full shape. The learned keypoints exhibit repeatable spatial structure across object instances and support smooth interpolation in keypoint space, indicating that they capture geometric variation. Our method achieves strong performance across diverse object categories, yielding a 6 percentage-point improvement in keypoint consistency compared to prior approaches.

Method: GalaxyDiT uses training-free guidance alignment and systematic proxy selection for reuse metrics. It identifies optimal proxies for each video model through rank-order correlation analysis, ensuring optimal computational reuse across model families and parameter scales.

Result: Achieves 1.87× speedup on Wan2.1-1.3B and 2.37× on Wan2.1-14B with only 0.97% and 0.72% drops on VBench-2.0. Maintains superior fidelity at high speedup rates, exceeding prior SOTA by 5-10 dB in PSNR.

Conclusion: GalaxyDiT provides an effective training-free acceleration method for video diffusion models that significantly reduces computational requirements while maintaining high quality, enabling broader adoption in downstream applications.

Abstract: Diffusion models have revolutionized video generation, becoming essential tools in creative content generation and physical simulation. Transformer-based architectures (DiTs) and classifier-free guidance (CFG) are two cornerstones of this success, enabling strong prompt adherence and realistic video quality. Despite their versatility and superior performance, these models require intensive computation. Each video generation requires dozens of iterative steps, and CFG doubles the required compute. This inefficiency hinders broader adoption in downstream applications. We introduce GalaxyDiT, a training-free method to accelerate video generation with guidance alignment and systematic proxy selection for reuse metrics. Through rank-order correlation analysis, our technique identifies the optimal proxy for each video model, across model families and parameter scales, thereby ensuring optimal computational reuse. We achieve $1.87\times$ and $2.37\times$ speedup on Wan2.1-1.3B and Wan2.1-14B with only 0.97% and 0.72% drops on the VBench-2.0 benchmark. At high speedup rates, our approach maintains superior fidelity to the base model, exceeding prior state-of-the-art approaches by 5 to 10 dB in peak signal-to-noise ratio (PSNR).

Method: Augment latent diffusion models with per-frame depth prediction and introduce multi-view geometric loss that aligns depth maps across frames in a shared 3D coordinate system.

Result: Produces significantly more stable and geometrically consistent results than existing baselines across multiple datasets.

Conclusion: Bridging appearance generation with 3D structure modeling through geometric regularization improves spatio-temporal coherence, shape consistency, and physical plausibility in video generation.

Abstract: Recent advances in video generation have enabled the synthesis of high-quality and visually realistic clips using diffusion transformer models. However, most existing approaches operate purely in the 2D pixel space and lack explicit mechanisms for modeling 3D structures, often resulting in temporally inconsistent geometries, implausible motions, and structural artifacts. In this work, we introduce geometric regularization losses into video generation by augmenting latent diffusion models with per-frame depth prediction. We adopted depth as the geometric representation because of the great progress in depth prediction and its compatibility with image-based latent encoders. Specifically, to enforce structural consistency over time, we propose a multi-view geometric loss that aligns the predicted depth maps across frames within a shared 3D coordinate system. Our method bridges the gap between appearance generation and 3D structure modeling, leading to improved spatio-temporal coherence, shape consistency, and physical plausibility. Experiments across multiple datasets show that our approach produces significantly more stable and geometrically consistent results than existing baselines.

Method: The framework includes: 1) Spatial-Aware World Model (SA-WM) that distills current scenes into command-aware latent states and rolls out future latent states for forward-looking cues, and 2) Hypergraph-guided decoder that hierarchically fuses these states with multimodal input to capture higher-order spatial dependencies. Also introduces DrivePilot dataset created using RAG and CoT-prompted LLM pipeline.

Result: ThinkDeeper ranks #1 on Talk2Car leaderboard and surpasses state-of-the-art baselines on DrivePilot, MoCAD, and RefCOCO/+/g benchmarks. Shows strong robustness in challenging scenes (long-text, multi-agent, ambiguity) and retains superior performance even when trained on only 50% of data.

Conclusion: The world model-based approach enables reasoning about future spatial states, providing forward-looking cues that significantly improve visual grounding performance in autonomous driving scenarios, particularly for handling ambiguous instructions and complex scenes.

Abstract: Interpreting natural-language commands to localize target objects is critical for autonomous driving (AD). Existing visual grounding (VG) methods for autonomous vehicles (AVs) typically struggle with ambiguous, context-dependent instructions, as they lack reasoning over 3D spatial relations and anticipated scene evolution. Grounded in the principles of world models, we propose ThinkDeeper, a framework that reasons about future spatial states before making grounding decisions. At its core is a Spatial-Aware World Model (SA-WM) that learns to reason ahead by distilling the current scene into a command-aware latent state and rolling out a sequence of future latent states, providing forward-looking cues for disambiguation. Complementing this, a hypergraph-guided decoder then hierarchically fuses these states with the multimodal input, capturing higher-order spatial dependencies for robust localization. In addition, we present DrivePilot, a multi-source VG dataset in AD, featuring semantic annotations generated by a Retrieval-Augmented Generation (RAG) and Chain-of-Thought (CoT)-prompted LLM pipeline. Extensive evaluations on six benchmarks, ThinkDeeper ranks #1 on the Talk2Car leaderboard and surpasses state-of-the-art baselines on DrivePilot, MoCAD, and RefCOCO/+/g benchmarks. Notably, it shows strong robustness and efficiency in challenging scenes (long-text, multi-agent, ambiguity) and retains superior performance even when trained on 50% of the data.

Method: Proposes Text-Printed Image (TPI) - generating synthetic images by directly rendering textual descriptions on plain white canvases. This simple rendering projects text into image modality while preserving semantics, unlike text-to-image models that often fail to maintain text accuracy.

Result: Across four models and seven benchmarks, TPI enables more effective text-centric training than synthetic images from diffusion models. Also works as low-cost data-augmentation strategy, demonstrating practical utility.

Conclusion: TPI highlights significant potential of text-centric training and charts a path toward fully automated data generation for LVLMs, offering scalable, low-cost alternative to traditional image collection.

Abstract: Recent large vision-language models (LVLMs) have been applied to diverse VQA tasks. However, achieving practical performance typically requires task-specific fine-tuning with large numbers of image-text pairs, which are costly to collect. In this work, we study text-centric training, a setting where only textual descriptions are available and no real images are provided, as a paradigm for low-cost data scaling. Unlike images, whose collection is often restricted by privacy constraints and scarcity in niche domains, text is widely available. Moreover, text is easily editable, enabling automatic diversification and expansion with LLMs at minimal human effort. While this offers clear advantages over image collection in terms of scalability and cost, training on raw text without images still yields limited gains on VQA tasks because of the image-text modality gap. To address this issue, we propose a Text-Printed Image (TPI), which generates synthetic images by directly rendering the given textual description on a plain white canvas. This simple rendering projects text into the image modality and can be integrated into arbitrary existing LVLM training pipelines at low cost. Moreover, TPI preserves the semantics of the text, whereas text-to-image models often fail to do. Across four models and seven benchmarks, our systematic experiments show that TPI enables more effective text-centric training than synthetic images generated by a diffusion model. We further explore TPI as a low-cost data-augmentation strategy and demonstrate its practical utility. Overall, our findings highlight the significant potential of text-centric training and, more broadly, chart a path toward fully automated data generation for LVLMs.

Method: Developed BDM for feature decomposition, extended to spatial and temporal difference decomposition modules (SD²M, SD³M, TD²M), built SD²Net for single-frame and STD²Net for multi-frame detection using U-shaped architecture.

Result: SD²Net performs well on SISTD; STD²Net achieves 87.68% mIoU on MISTD datasets, outperforming SD²Net’s 64.97% mIoU.

Conclusion: The proposed basis decomposition approach effectively enhances targets and suppresses backgrounds, achieving state-of-the-art performance in infrared small target detection.

Abstract: Infrared small target detection (ISTD) faces two major challenges: a lack of discernible target texture and severe background clutter, which results in the background obscuring the target. To enhance targets and suppress backgrounds, we propose the Basis Decomposition Module (BDM) as an extensible and lightweight module based on basis decomposition, which decomposes a complex feature into several basis features and enhances certain information while eliminating redundancy. Extending BDM leads to a series of modules, including the Spatial Difference Decomposition Module (SD$^\mathrm{2}$M), Spatial Difference Decomposition Downsampling Module (SD$^\mathrm{3}$M), and Temporal Difference Decomposition Module (TD$^\mathrm{2}$M). Based on these modules, we develop the Spatial Difference Decomposition Network (SD$^\mathrm{2}$Net) for single-frame ISTD (SISTD) and the Spatiotemporal Difference Decomposition Network (STD$^\mathrm{2}$Net) for multi-frame ISTD (MISTD). SD$^\mathrm{2}$Net integrates SD$^\mathrm{2}$M and SD$^\mathrm{3}$M within an adapted U-shaped architecture. We employ TD$^\mathrm{2}$M to introduce motion information, which transforms SD$^\mathrm{2}$Net into STD$^\mathrm{2}$Net. Extensive experiments on SISTD and MISTD datasets demonstrate state-of-the-art (SOTA) performance. On the SISTD task, SD$^\mathrm{2}$Net performs well compared to most established networks. On the MISTD datasets, STD$^\mathrm{2}$Net achieves a mIoU of 87.68%, outperforming SD$^\mathrm{2}$Net, which achieves a mIoU of 64.97%. Our codes are available: https://github.com/greekinRoma/IRSTD_HC_Platform.

Method: Proposes Action Effect Modeling (AEM): 1) Identifies action outcomes by selecting most informative effect frames based on semantic relevance and visual quality, 2) Extracts complementary cues from visual grounding and symbolic scene graphs, aligning them in shared latent space, 3) Designs prompt-based detector with task-specific prompts aligning action segments with intended execution semantics.

Result: Achieves state-of-the-art performance on EgoPER and CaptainCook4D benchmarks under challenging one-class classification (OCC) setting.

Conclusion: Modeling both execution and outcome yields more reliable mistake detection, and effect-aware representations have potential to benefit broader range of downstream applications.

Abstract: Mistake detection in procedural tasks is essential for building intelligent systems that support learning and task execution. Existing approaches primarily analyze how an action is performed, while overlooking what it produces, i.e., the \textbf{action effect}. Yet many errors manifest not in the execution itself but in the resulting outcome, such as an unintended object state or incorrect spatial arrangement. To address this gap, we propose Action Effect Modeling (AEM), a unified framework that jointly captures action execution and its outcomes through a probabilistic formulation. AEM first identifies the outcome of an action by selecting the most informative effect frame based on semantic relevance and visual quality. It then extracts complementary cues from visual grounding and symbolic scene graphs, aligning them in a shared latent space to form robust effect-aware representations. To detect mistakes, we further design a prompt-based detector that incorporates task-specific prompts and aligns each action segment with its intended execution semantics. Our approach achieves state-of-the-art performance on the EgoPER and CaptainCook4D benchmarks under the challenging one-class classification (OCC) setting. These results demonstrate that modeling both execution and outcome yields more reliable mistake detection, and highlight the potential of effect-aware representations to benefit a broader range of downstream applications.

Method: Proposes fairness-aware Low-Rank Adaptation with three approaches: FR-LoRA (MaxAccGap regularization), GR-LoRA (inverse frequency gradient weighting), and Hybrid-LoRA (combination). Uses differentiable MaxAccGap loss for end-to-end fairness optimization.

Result: GR-LoRA reduces diagnostic accuracy disparities by 69% while maintaining 53.15% overall accuracy on 10,000 glaucoma fundus images. Race-specific optimization achieves 60% disparity reduction with only 0.24% trainable parameters.

Conclusion: The approach enables practical deployment of fair medical AI in resource-constrained settings by combining parameter efficiency with explicit fairness optimization through novel regularization and gradient balancing techniques.

Abstract: Vision-language models achieve expert-level performance on medical imaging tasks but exhibit significant diagnostic accuracy disparities across demographic groups. We introduce fairness-aware Low-Rank Adaptation for medical VLMs, combining parameter efficiency with explicit fairness optimization. Our key algorithmic contribution is a differentiable MaxAccGap loss that enables end-to-end optimization of accuracy parity across demographic groups. We propose three methods: FR-LoRA integrates MaxAccGap regularization into the training objective, GR-LoRA applies inverse frequency weighting to balance gradient contributions, and Hybrid-LoRA combines both mechanisms.Evaluated on 10,000 glaucoma fundus images, GR-LoRA reduces diagnostic accuracy disparities by 69% while maintaining 53.15% overall accuracy. Ablation studies reveal that strong regularization strength achieves optimal fairness with minimal accuracy trade-off, and race-specific optimization yields 60% disparity reduction. Our approach requires only 0.24% trainable parameters, enabling practical deployment of fair medical AI in resource-constrained healthcare settings.

Method: Introduces Object-IVQA benchmark (107 videos, 514 QA pairs with temporal evidence). Proposes agent framework with object-centric planning, perception, analysis and generation tools for explicit evidence retrieval and multi-hop reasoning.

Result: Existing large vision-language models struggle with object-level recognition and reasoning. Proposed framework achieves substantial improvement over baseline methods.

Conclusion: Object-centric paradigm is crucial for understanding procedural activities. The benchmark and framework advance object-centric reasoning capabilities for assistive AI.

Abstract: Understanding procedural activities is crucial for developing future assistive AI that can reason about complex real-world tasks. Existing action-centric methods struggle with the flexibility of real procedures, where step order varies depending on object states. In this work, we propose to shift the focus to an object-centric paradigm by regarding actions as mechanisms that drive state transitions. To advance this direction, we introduce Object-IVQA, a long-form instructional video benchmark with 107 videos and 514 open-ended question-answer pairs annotated with temporally grounded evidence. The benchmark evaluates four dimensions of object-centric reasoning, including state evolution, precondition verification, counterfactual reasoning and mistake recognition. We further propose an agent framework that orchestrates object-centric planning, perception, analysis and generation tools, enabling explicit evidence retrieval and multi-hop reasoning across disjoint segments. Experiments show that existing large vision-language models struggle in object-level recognition and reasoning, whereas our framework achieves substantially improvement.

Method: Proposes NAS-LoRA which adds a lightweight NAS block between encoder and decoder components of LoRA to dynamically optimize prior knowledge integration. Uses stage-wise optimization strategy to help ViT encoder balance weight updates and architectural adjustments.

Result: NAS-LoRA improves existing PEFT methods while reducing training cost by 24.14% without increasing inference cost, demonstrating NAS’s potential for enhancing PEFT in visual foundation models.

Conclusion: NAS-LoRA effectively bridges semantic gaps between pre-trained SAM and specialized domains by incorporating inductive bias through NAS, offering a parameter-efficient solution with reduced training costs.

Abstract: The Segment Anything Model (SAM) has emerged as a powerful visual foundation model for image segmentation. However, adapting SAM to specific downstream tasks, such as medical and agricultural imaging, remains a significant challenge. To address this, Low-Rank Adaptation (LoRA) and its variants have been widely employed to enhancing SAM’s adaptation performance on diverse domains. Despite advancements, a critical question arises: can we integrate inductive bias into the model? This is particularly relevant since the Transformer encoder in SAM inherently lacks spatial priors within image patches, potentially hindering the acquisition of high-level semantic information. In this paper, we propose NAS-LoRA, a new Parameter-Efficient Fine-Tuning (PEFT) method designed to bridge the semantic gap between pre-trained SAM and specialized domains. Specifically, NAS-LoRA incorporates a lightweight Neural Architecture Search (NAS) block between the encoder and decoder components of LoRA to dynamically optimize the prior knowledge integrated into weight updates. Furthermore, we propose a stage-wise optimization strategy to help the ViT encoder balance weight updates and architectural adjustments, facilitating the gradual learning of high-level semantic information. Various Experiments demonstrate our NAS-LoRA improves existing PEFT methods, while reducing training cost by 24.14% without increasing inference cost, highlighting the potential of NAS in enhancing PEFT for visual foundation models.

Method: EEA uses semantic guidance with hierarchical tree search: 1) autonomously discovers and updates task-relevant semantic queries, 2) collects matching video frames as semantic anchors, 3) preferentially explores semantically relevant frames while ensuring coverage in unknown segments, and 4) adaptively combines intrinsic rewards from VLMs with semantic priors by modeling uncertainty.

Result: Experiments across various long-video benchmarks validate superior performance and computational efficiency compared to existing methods.

Conclusion: EEA effectively addresses the exploration-exploitation balance problem in long-form video understanding through semantic-guided hierarchical tree search, achieving both high performance and computational efficiency.

Abstract: Long-form video understanding requires efficient navigation of extensive visual data to pinpoint sparse yet critical information. Current approaches to longform video understanding either suffer from severe computational overhead due to dense preprocessing, or fail to effectively balance exploration and exploitation, resulting in incomplete information coverage and inefficiency. In this work, we introduce EEA, a novel video agent framework that archives exploration-exploitation balance through semantic guidance with hierarchical tree search process. EEA autonomously discovers and dynamically updates task-relevant semantic queries, and collects video frames closely matched to these queries as semantic anchors. During the tree search process, instead of uniform expansion, EEA preferentially explores semantically relevant frames while ensuring sufficient coverage within unknown segments. Moreover, EEA adaptively combines intrinsic rewards from visionlanguage models (VLMs) with semantic priors by explicitly modeling uncertainty to achieve stable and precise evaluation of video segments. Experiments across various long-video benchmarks validate the superior performance and computational efficiency of our proposed method.

Method: Proposes Domain-aware Prompt-driven Masked Transformer (DPMFormer) with three key components: 1) Domain-aware prompt learning for semantic alignment, 2) Domain-aware contrastive learning with texture perturbation to diversify observable domains, and 3) Domain-robust consistency learning to minimize prediction discrepancies between original and augmented images.

Result: The framework establishes a new state-of-the-art on various DGSS benchmarks, demonstrating superiority through experiments and analyses.

Conclusion: DPMFormer effectively addresses semantic misalignment in domain generalized semantic segmentation and provides a robust framework resilient to diverse environmental changes.

Abstract: Recent domain generalized semantic segmentation (DGSS) studies have achieved notable improvements by distilling semantic knowledge from Vision-Language Models (VLMs). However, they overlook the semantic misalignment between visual and textual contexts, which arises due to the rigidity of a fixed context prompt learned on a single source domain. To this end, we present a novel domain generalization framework for semantic segmentation, namely Domain-aware Prompt-driven Masked Transformer (DPMFormer). Firstly, we introduce domain-aware prompt learning to facilitate semantic alignment between visual and textual cues. To capture various domain-specific properties with a single source dataset, we propose domain-aware contrastive learning along with the texture perturbation that diversifies the observable domains. Lastly, to establish a framework resilient against diverse environmental changes, we have proposed the domain-robust consistency learning which guides the model to minimize discrepancies of prediction from original and the augmented images. Through experiments and analyses, we demonstrate the superiority of the proposed framework, which establishes a new state-of-the-art on various DGSS benchmarks. The code is available at https://github.com/jone1222/DPMFormer.

Method: Integrated YOLOv8 and v11 for dancer detection with Segment Anything Model (SAM) for precise segmentation, enabling tracking, step counting, spatial coverage analysis, and rhythm consistency measurement.

Result: Achieved ~94% detection precision, 89% recall, and ~83% IoU segmentation accuracy. Analysis showed primary dancer executed 23% more steps, 37% higher motion intensity, and used 42% more space than secondary dancers.

Conclusion: The framework demonstrates technical feasibility for automated dance analysis but has limitations (single-video validation, no ground truth, no pose estimation comparison). It establishes foundation for future systematic validation studies.

Abstract: This paper presents a preliminary investigation into automated dance movement analysis using contemporary computer vision techniques. We propose a proof-of-concept framework that integrates YOLOv8 and v11 for dancer detection with the Segment Anything Model (SAM) for precise segmentation, enabling the tracking and quantification of dancer movements in video recordings without specialized equipment or markers. Our approach identifies dancers within video frames, counts discrete dance steps, calculates spatial coverage patterns, and measures rhythm consistency across performance sequences. Testing this framework on a single 49-second recording of Ghanaian AfroBeats dance demonstrates technical feasibility, with the system achieving approximately 94% detection precision and 89% recall on manually inspected samples. The pixel-level segmentation provided by SAM, achieving approximately 83% intersection-over-union with visual inspection, enables motion quantification that captures body configuration changes beyond what bounding-box approaches can represent. Analysis of this preliminary case study indicates that the dancer classified as primary by our system executed 23% more steps with 37% higher motion intensity and utilized 42% more performance space compared to dancers classified as secondary. However, this work represents an early-stage investigation with substantial limitations including single-video validation, absence of systematic ground truth annotations, and lack of comparison with existing pose estimation methods. We present this framework to demonstrate technical feasibility, identify promising directions for quantitative dance metrics, and establish a foundation for future systematic validation studies.

Method: The authors introduce CartoMapQA, a benchmark dataset with over 2000 samples, each containing a cartographic map, a question (open-ended or multiple-choice), and ground-truth answer. The tasks cover low-, mid-, and high-level map interpretation skills including symbol recognition, embedded information extraction, scale interpretation, and route-based reasoning.

Result: Evaluation of both open-source and proprietary VLMs reveals persistent challenges: models struggle with map-specific semantics, exhibit limited geospatial reasoning, and are prone to OCR-related errors. The benchmark successfully isolates these weaknesses in current VLMs.

Conclusion: CartoMapQA provides a valuable tool for guiding future improvements in VLM architectures, supporting the development of models better equipped for real-world applications that depend on robust map understanding. The dataset and code are publicly available to the research community.

Abstract: The rise of Visual-Language Models (LVLMs) has unlocked new possibilities for seamlessly integrating visual and textual information. However, their ability to interpret cartographic maps remains largely unexplored. In this paper, we introduce CartoMapQA, a benchmark specifically designed to evaluate LVLMs’ understanding of cartographic maps through question-answering tasks. The dataset includes over 2000 samples, each composed of a cartographic map, a question (with open-ended or multiple-choice answers), and a ground-truth answer. These tasks span key low-, mid- and high-level map interpretation skills, including symbol recognition, embedded information extraction, scale interpretation, and route-based reasoning. Our evaluation of both open-source and proprietary LVLMs reveals persistent challenges: models frequently struggle with map-specific semantics, exhibit limited geospatial reasoning, and are prone to Optical Character Recognition (OCR)-related errors. By isolating these weaknesses, CartoMapQA offers a valuable tool for guiding future improvements in LVLM architectures. Ultimately, it supports the development of models better equipped for real-world applications that depend on robust and reliable map understanding, such as navigation, geographic search, and urban planning. Our source code and data are openly available to the research community at: https://github.com/ungquanghuy-kddi/CartoMapQA.git

Method: For semantic mapping: Train latent diffusion model on HD maps to learn generative prior, incorporate via constrained MAP optimization in latent space with robust initialization and efficient optimization. For topological mapping: Apply confidence-weighted k-medoids clustering and kinematic refinement to trajectories.

Result: Achieves state-of-the-art semantic and topological mapping performance on nuScenes, Argoverse 2, and proprietary datasets. Quality consistently increases with more crowdsourced data, with thorough ablation and scalability studies.

Conclusion: CSMapping successfully addresses the noise robustness problem in crowdsourced mapping, enabling quality improvement with data volume through diffusion-based generative priors and robust topological processing.

Abstract: Crowdsourcing enables scalable autonomous driving map construction, but low-cost sensor noise hinders quality from improving with data volume. We propose CSMapping, a system that produces accurate semantic maps and topological road centerlines whose quality consistently increases with more crowdsourced data. For semantic mapping, we train a latent diffusion model on HD maps (optionally conditioned on SD maps) to learn a generative prior of real-world map structure, without requiring paired crowdsourced/HD-map supervision. This prior is incorporated via constrained MAP optimization in latent space, ensuring robustness to severe noise and plausible completion in unobserved areas. Initialization uses a robust vectorized mapping module followed by diffusion inversion; optimization employs efficient Gaussian-basis reparameterization, projected gradient descent zobracket multi-start, and latent-space factor-graph for global consistency. For topological mapping, we apply confidence-weighted k-medoids clustering and kinematic refinement to trajectories, yielding smooth, human-like centerlines robust to trajectory variation. Experiments on nuScenes, Argoverse 2, and a large proprietary dataset achieve state-of-the-art semantic and topological mapping performance, with thorough ablation and scalability studies.

Method: The authors adopt a diffusion forcing framework tailored for time-series generation under time-varying control events. They identify that vanilla diffusion forcing fails for motion generation and introduce three key improvements: (1) training with bi-directional attention instead of casual attention, (2) implementing a lower triangular time scheduler instead of random scheduling, and (3) using continuous time-varying text conditioning.

Result: FloodDiffusion achieves state-of-the-art performance on streaming motion generation, reaching an FID of 0.057 on the HumanML3D benchmark. The framework generates text-aligned, seamless motion sequences with real-time latency.

Conclusion: The diffusion forcing-based framework, when properly tailored with the three proposed improvements, can effectively model real motion distributions and achieve superior performance for streaming human motion generation tasks compared to existing methods.

Abstract: We present FloodDiffusion, a new framework for text-driven, streaming human motion generation. Given time-varying text prompts, FloodDiffusion generates text-aligned, seamless motion sequences with real-time latency. Unlike existing methods that rely on chunk-by-chunk or auto-regressive model with diffusion head, we adopt a diffusion forcing framework to model this time-series generation task under time-varying control events. We find that a straightforward implementation of vanilla diffusion forcing (as proposed for video models) fails to model real motion distributions. We demonstrate that to guarantee modeling the output distribution, the vanilla diffusion forcing must be tailored to: (i) train with a bi-directional attention instead of casual attention; (ii) implement a lower triangular time scheduler instead of a random one; (iii) utilize a continues time-varying way to introduce text conditioning. With these improvements, we demonstrate in the first time that the diffusion forcing-based framework achieves state-of-the-art performance on the streaming motion generation task, reaching an FID of 0.057 on the HumanML3D benchmark. Models, code, and weights are available. https://shandaai.github.io/FloodDiffusion/

Method: Compared DeepSeek-OCR’s vision encoder against simple alternatives: parameter-free mean pooling and a learned hierarchical encoder, evaluating both text reconstruction and language modeling performance.

Result: Simple approaches match or surpass vision-based compression for text reconstruction at matched compression ratios, and outperform it for language modeling where vision-based compression fails to beat simple truncation.

Conclusion: The excitement around optical context compression is premature - vision-based compression doesn’t provide unique advantages for language modeling compared to simpler alternatives.

Abstract: DeepSeek-OCR demonstrates that rendered text can be reconstructed with high fidelity from a small number of vision tokens. This finding has sparked excitement about vision-based context compression for language models. But the evaluation stops at reconstruction; whether these representations help language modeling remains untested. We test two assumptions implicit in the optical-compression narrative: that vision-based compression provides unique advantages for text reconstruction from compressed representations, and that DeepSeek-OCR’s reconstruction results are evidence that vision-based compression will be useful for language modeling. Comparing their vision encoder against simple alternatives–parameter-free mean pooling and a learned hierarchical encoder–we find that these simple approaches match or surpass vision for reconstruction at matched compression ratios, and outperform it for language modeling–where vision-based compression fails to beat truncation. The excitement around optical context compression outpaces the evidence. Code and checkpoints are available at https://github.com/ivnle/bad-autoencoding

Method: OpenTrack3D uses a visual-spatial tracker to construct cross-view consistent object proposals online from RGB-D streams. It leverages 2D open-vocabulary segmentation masks, lifts them to 3D point clouds using depth, extracts mask-guided instance features using DINO, and fuses visual-spatial cues for instance consistency. The core is mesh-free, with optional superpoints refinement when mesh is available. It replaces CLIP with multi-modal large language models for better compositional reasoning.

Result: Extensive experiments on ScanNet200, Replica, ScanNet++, and SceneFun3D benchmarks demonstrate state-of-the-art performance and strong generalization capabilities across diverse environments.

Conclusion: OpenTrack3D addresses key limitations of existing methods by providing a generalizable, mesh-free framework with improved compositional reasoning through MLLMs, enabling effective open-vocabulary 3D instance segmentation in diverse, unstructured environments for robotics and AR/VR applications.

Abstract: Generalizing open-vocabulary 3D instance segmentation (OV-3DIS) to diverse, unstructured, and mesh-free environments is crucial for robotics and AR/VR, yet remains a significant challenge. We attribute this to two key limitations of existing methods: (1) proposal generation relies on dataset-specific proposal networks or mesh-based superpoints, rendering them inapplicable in mesh-free scenarios and limiting generalization to novel scenes; and (2) the weak textual reasoning of CLIP-based classifiers, which struggle to recognize compositional and functional user queries. To address these issues, we introduce OpenTrack3D, a generalizable and accurate framework. Unlike methods that rely on pre-generated proposals, OpenTrack3D employs a novel visual-spatial tracker to construct cross-view consistent object proposals online. Given an RGB-D stream, our pipeline first leverages a 2D open-vocabulary segmenter to generate masks, which are lifted to 3D point clouds using depth. Mask-guided instance features are then extracted using DINO feature maps, and our tracker fuses visual and spatial cues to maintain instance consistency. The core pipeline is entirely mesh-free, yet we also provide an optional superpoints refinement module to further enhance performance when scene mesh is available. Finally, we replace CLIP with a multi-modal large language model (MLLM), significantly enhancing compositional reasoning for complex user queries. Extensive experiments on diverse benchmarks, including ScanNet200, Replica, ScanNet++, and SceneFun3D, demonstrate state-of-the-art performance and strong generalization capabilities.

Method: Two-stage training: 1) Supervised Fine-Tuning on curated datasets for complex multi-turn tool composition and error recovery, 2) Reinforcement Learning with novel dense process rewards to encourage strategic and efficient tool use. Code serves as universal interface for image operations.

Result: Significant performance improvements on Qwen2.5-VL and Qwen3-VL series, with emergent capabilities including flexible tool composition, efficient chained execution, and robust error recovery from runtime feedback.

Conclusion: CodeVision provides a flexible and scalable code-as-tool framework that addresses MLLM brittleness and enables robust tool-based reasoning through code generation as a universal interface for image operations.

Abstract: Multimodal large language models (MLLMs) that think with images can interactively use tools to reason about visual inputs, but current approaches often rely on a narrow set of tools with limited real-world necessity and scalability. In this work, we first reveal a critical and previously overlooked weakness: even state-of-the-art MLLMs are surprisingly brittle, showing significant performance degradation on images with simple orientation changes or natural corruptions, underscoring the need for more robust tool-based reasoning. To address this, we propose CodeVision, a flexible and scalable code-as-tool framework where the model generates code as a universal interface to invoke any image operation, moving beyond fixed tool registries. We train our model using a two-stage methodology, beginning with Supervised Fine-Tuning (SFT) on a high-quality dataset curated for complex, multi-turn tool composition and error recovery, followed by Reinforcement Learning (RL) with a novel and dense process reward function to encourage strategic and efficient tool use. To facilitate this research, we construct new SFT and RL datasets and introduce a challenging new benchmark suite designed to rigorously evaluate robustness to orientation changes and multi-tool reasoning. Experiments on Qwen2.5-VL and Qwen3-VL series show that our approach significantly improves model performance and fosters emergent capabilities such as flexible tool composition, efficient chained execution, and robust error recovery from runtime feedback. Code is available at https://github.com/ByteDance-BandAI/CodeVision.

Method: PRIS framework: 1) Generates multiple visuals from initial prompt, 2) Reviews visuals to identify recurring failure patterns, 3) Redesigns prompt based on failures, 4) Regenerates visuals with revised prompt. Introduces element-level factual correction verifier for fine-grained alignment assessment between prompt attributes and visuals.

Result: Extensive experiments on text-to-image and text-to-video benchmarks show effectiveness, including 15% gain on VBench 2.0. Demonstrates that jointly scaling prompts and visuals is key to leveraging inference-time scaling laws.

Conclusion: Adaptive prompt revision during inference (PRIS) significantly improves text-to-visual generation alignment by addressing fixed-prompt limitations, with element-level verification providing precise feedback for prompt redesign.

Abstract: Achieving precise alignment between user intent and generated visuals remains a central challenge in text-to-visual generation, as a single attempt often fails to produce the desired output. To handle this, prior approaches mainly scale the visual generation process (e.g., increasing sampling steps or seeds), but this quickly leads to a quality plateau. This limitation arises because the prompt, crucial for guiding generation, is kept fixed. To address this, we propose Prompt Redesign for Inference-time Scaling, coined PRIS, a framework that adaptively revises the prompt during inference in response to the scaled visual generations. The core idea of PRIS is to review the generated visuals, identify recurring failure patterns across visuals, and redesign the prompt accordingly before regenerating the visuals with the revised prompt. To provide precise alignment feedback for prompt revision, we introduce a new verifier, element-level factual correction, which evaluates the alignment between prompt attributes and generated visuals at a fine-grained level, achieving more accurate and interpretable assessments than holistic measures. Extensive experiments on both text-to-image and text-to-video benchmarks demonstrate the effectiveness of our approach, including a 15% gain on VBench 2.0. These results highlight that jointly scaling prompts and visuals is key to fully leveraging scaling laws at inference-time. Visualizations are available at the website: https://subin-kim-cv.github.io/PRIS.

Method: AdaptVision uses coarse-to-fine active vision: initially processes compressed tokens from low-res images, then selectively invokes bounding box tool to crop key regions when needed. Trained with reinforcement learning using Decoupled Turn Policy Optimization (DTPO) that separates tool learning and accuracy improvement objectives with separate advantage estimation.

Result: AdaptVision achieves superior performance on multiple VQA benchmarks while consuming substantially fewer visual tokens than state-of-the-art efficient VLM methods.

Conclusion: The proposed adaptive visual token acquisition paradigm enables VLMs to balance accuracy and efficiency effectively, outperforming fixed-ratio compression methods through active vision mechanisms and decoupled reinforcement learning optimization.

Abstract: Vision-Language Models (VLMs) have achieved remarkable success in visual question answering tasks, but their reliance on large numbers of visual tokens introduces significant computational overhead. While existing efficient VLM approaches reduce visual tokens through fixed-ratio compression, they operate passively and lack the ability to adapt to varying task requirements. This motivates a fundamental question: Can VLMs autonomously determine the minimum number of visual tokens required for each sample? Inspired by human active vision mechanisms, we introduce AdaptVision, an efficient VLM paradigm that enables adaptive visual token acquisition through a coarse-to-fine approach. Our model initially processes compressed visual tokens from low-resolution images and selectively acquires additional visual information by invoking a bounding box tool to crop key regions when necessary. We train AdaptVision using a reinforcement learning framework that carefully balances accuracy and efficiency. Central to our approach is Decoupled Turn Policy Optimization (DTPO), which decouples the learning objective into two components: (1) tool learning, which optimizes correct tool utilization, and (2) accuracy improvement, which refines the generated responses to improve answer correctness. Based on this formulation, we further decouple advantage estimation by computing separate advantages for tokens associated with each objective. This formulation enables more effective optimization for AdaptVision compared to vanilla GRPO. Comprehensive experiments across multiple VQA benchmarks demonstrate that AdaptVision achieves superior performance while consuming substantially fewer visual tokens than state-of-the-art efficient VLM methods.

Method: Three key components: (1) Step-wise Regional Control (SRC) aligns textual steps with image regions in single denoising process; (2) Flexible RoPE provides step-aware positional encoding for temporal coherence and spatial diversity; (3) Cross-Step Consistency Control (CSCC) maintains ingredient consistency across steps.

Result: Experimental results on recipe illustration benchmarks show CookAnything outperforms existing methods in both training-based and training-free settings.

Conclusion: The framework supports scalable, high-quality visual synthesis of complex multi-step instructions and has significant potential for applications in instructional media and procedural content creation.

Abstract: Cooking is a sequential and visually grounded activity, where each step such as chopping, mixing, or frying carries both procedural logic and visual semantics. While recent diffusion models have shown strong capabilities in text-to-image generation, they struggle to handle structured multi-step scenarios like recipe illustration. Additionally, current recipe illustration methods are unable to adjust to the natural variability in recipe length, generating a fixed number of images regardless of the actual instructions structure. To address these limitations, we present CookAnything, a flexible and consistent diffusion-based framework that generates coherent, semantically distinct image sequences from textual cooking instructions of arbitrary length. The framework introduces three key components: (1) Step-wise Regional Control (SRC), which aligns textual steps with corresponding image regions within a single denoising process; (2) Flexible RoPE, a step-aware positional encoding mechanism that enhances both temporal coherence and spatial diversity; and (3) Cross-Step Consistency Control (CSCC), which maintains fine-grained ingredient consistency across steps. Experimental results on recipe illustration benchmarks show that CookAnything performs better than existing methods in training-based and training-free settings. The proposed framework supports scalable, high-quality visual synthesis of complex multi-step instructions and holds significant potential for broad applications in instructional media, and procedural content creation.

Method: Redefines SLP as hierarchical end-to-end generation with only text understanding (Prompt2Gloss, Text2Gloss) and Pose2Vid. Uses Sign Language Understanding Linker (SLUL) for text understanding with Semantic-Aware Gloss Masking Loss (SAGM Loss), and SLP-MoE hand gesture rendering expert block for video generation.

Result: Achieves 48.6% performance improvement compared to current state-of-the-art generation methods in sign language production.

Conclusion: Stable Signer successfully streamlines the traditional SLP pipeline, reduces error accumulation, and enables end-to-end generation of high-quality, multi-style sign language videos through simplified architecture and novel training techniques.

Abstract: Sign Language Production (SLP) is the process of converting the complex input text into a real video. Most previous works focused on the Text2Gloss, Gloss2Pose, Pose2Vid stages, and some concentrated on Prompt2Gloss and Text2Avatar stages. However, this field has made slow progress due to the inaccuracy of text conversion, pose generation, and the rendering of poses into real human videos in these stages, resulting in gradually accumulating errors. Therefore, in this paper, we streamline the traditional redundant structure, simplify and optimize the task objective, and design a new sign language generative model called Stable Signer. It redefines the SLP task as a hierarchical generation end-to-end task that only includes text understanding (Prompt2Gloss, Text2Gloss) and Pose2Vid, and executes text understanding through our proposed new Sign Language Understanding Linker called SLUL, and generates hand gestures through the named SLP-MoE hand gesture rendering expert block to end-to-end generate high-quality and multi-style sign language videos. SLUL is trained using the newly developed Semantic-Aware Gloss Masking Loss (SAGM Loss). Its performance has improved by 48.6% compared to the current SOTA generation methods.

Method: Proposes V-ITI with two components: 1) Visual Neglect Detector using head-level discriminative probes to identify when models ignore visual inputs, and 2) Visual Recall Intervenor that modulates activations with prestored visual information only when neglect is detected.

Result: Extensive experiments across eight benchmarks and different MLLM families show V-ITI consistently mitigates vision-related hallucinations while preserving general task performance.

Conclusion: V-ITI addresses the “when to intervene” problem overlooked by previous methods, providing targeted intervention only when visual neglect occurs, reducing hallucinations without over-intervention issues.

Abstract: Multimodal Large Language Models (MLLMs) excel in numerous vision-language tasks yet suffer from hallucinations, producing content inconsistent with input visuals, that undermine reliability in precision-sensitive domains. This issue stems from a fundamental problem of visual neglect, where models fail to adequately prioritize input images. Existing methods typically alleviate hallucinations by intervening in the attention score or output logits, focusing on “how to intervene” but overlooking the prerequisite “when to intervene”, which leads to the “over-intervention” problem and subsequently introduces new hallucinations and unnecessary computational overhead. To address this gap, we first investigate the mechanism of visual neglect and reveal it can be accurately detected via head-level activation patterns in MLLMs. We thus propose V-ITI, a lightweight visual inference-time intervention framework integrating a Visual Neglect Detector that identifies visual neglect via head-level discriminative probes and a Visual Recall Intervenor that modulates activations with prestored visual activation information only when the visual neglect is detected. Extensive experiments across eight benchmarks and different MLLM families demonstrate that V-ITI consistently mitigates vision-related hallucinations while preserving general task performance.

Method: Hybrid framework with two pipelines: supervised classification for known violations and reference-based similarity matching for novel cases. Uses multimodal inputs (text, audio, visual) processed through both pipelines, with MLLM knowledge distillation to boost accuracy while keeping inference lightweight.

Result: Classification pipeline: 67% recall at 80% precision. Similarity pipeline: 76% recall at 80% precision. Large-scale A/B tests show 6-8% reduction in user views of unwanted livestreams.

Conclusion: Demonstrates a scalable and adaptable approach to multimodal content governance capable of addressing both explicit violations and emerging adversarial behaviors.

Abstract: Content moderation remains a critical yet challenging task for large-scale user-generated video platforms, especially in livestreaming environments where moderation must be timely, multimodal, and robust to evolving forms of unwanted content. We present a hybrid moderation framework deployed at production scale that combines supervised classification for known violations with reference-based similarity matching for novel or subtle cases. This hybrid design enables robust detection of both explicit violations and novel edge cases that evade traditional classifiers. Multimodal inputs (text, audio, visual) are processed through both pipelines, with a multimodal large language model (MLLM) distilling knowledge into each to boost accuracy while keeping inference lightweight. In production, the classification pipeline achieves 67% recall at 80% precision, and the similarity pipeline achieves 76% recall at 80% precision. Large-scale A/B tests show a 6-8% reduction in user views of unwanted livestreams}. These results demonstrate a scalable and adaptable approach to multimodal content governance, capable of addressing both explicit violations and emerging adversarial behaviors.

Method: Proposes GLASTE framework with: 1) global-local combination structure, 2) joint global and local losses, 3) enhanced text image features, 4) text style expressed as size-independent vectors, and 5) affine fusion to fill target text while maintaining aspect ratio.

Result: Extensive experiments on real-world datasets show GLASTE outperforms previous methods in both quantitative metrics and qualitative results, effectively mitigating the inconsistency and length-insensitivity challenges.

Conclusion: GLASTE successfully addresses key challenges in scene text editing by simultaneously incorporating global contextual information with local features, enabling more consistent and length-sensitive text replacement while preserving original style and background.

Abstract: Scene Text Editing (STE) involves replacing text in a scene image with new target text while preserving both the original text style and background texture. Existing methods suffer from two major challenges: inconsistency and length-insensitivity. They often fail to maintain coherence between the edited local patch and the surrounding area, and they struggle to handle significant differences in text length before and after editing. To tackle these challenges, we propose an end-to-end framework called Global-Local Aware Scene Text Editing (GLASTE), which simultaneously incorporates high-level global contextual information along with delicate local features. Specifically, we design a global-local combination structure, joint global and local losses, and enhance text image features to ensure consistency in text style within local patches while maintaining harmony between local and global areas. Additionally, we express the text style as a vector independent of the image size, which can be transferred to target text images of various sizes. We use an affine fusion to fill target text images into the editing patch while maintaining their aspect ratio unchanged. Extensive experiments on real-world datasets validate that our GLASTE model outperforms previous methods in both quantitative metrics and qualitative results and effectively mitigates the two challenges.

Method: Formulates compression as optimization minimizing conditional entropy between retained and full tokens. Introduces information uniqueness to measure token redundancy. Designs three modules: Frame Group Fusion (semantic frame grouping), Token Allocation (adaptive resource allocation), and Spatial Dynamic Compression (fine-grained spatial compression).

Result: Extensive experiments show UniComp consistently outperforms existing compression methods in preserving essential visual tokens under limited computational budgets.

Conclusion: Information uniqueness plays a pivotal role in token compression efficacy, and the proposed UniComp framework effectively maximizes information fidelity while respecting computational constraints.

Abstract: Distinct from attention-based compression methods, this paper presents an information uniqueness driven video compression framework, termed UniComp, which aims to maximize the information fidelity of video representations under constrained computational budgets. Starting from the information-theoretic perspective, we formulate the vision compression as an optimization problem that minimizes conditional entropy (reconstruction error) between retained and full tokens. To achieve this, we introduce the notion of information uniqueness to measure intrinsic redundancy among tokens to link with reconstruction error. Based on uniqueness, we design three modules-Frame Group Fusion, Token Allocation, and Spatial Dynamic Compression-that progressively perform semantic frame grouping, adaptive resource allocation, and fine-grained spatial compression. Extensive experiments demonstrate that UniComp consistently outperforms existing compression methods in preserving essential visual tokens under limited computational budgets, highlighting the pivotal role of information uniqueness in token compression efficacy.

Method: Two-stage pretraining framework: 1) Patch-wise contrastive alignment with lightweight adapter to improve H&E feature compatibility with IHC contextual cues; 2) Slide-level representation learning with MIL using cross-stain attention fusion module to integrate stain-specific patch features and cross-stain global alignment module to enforce consistency across slide-level embeddings.

Result: Experiments on cancer subtype classification, IHC biomarker status classification, and survival prediction show consistent gains, yielding high-quality, transferable H&E slide-level representations that outperform existing approaches.

Conclusion: CSCL successfully addresses inter-stain misalignment and creates robust cross-stain representations, enabling better utilization of multi-stain data for computational pathology tasks. The curated five-stain dataset and code are publicly available to advance the field.

Abstract: Universal, transferable whole-slide image (WSI) representations are central to computational pathology. Incorporating multiple markers (e.g., immunohistochemistry, IHC) alongside H&E enriches H&E-based features with diverse, biologically meaningful information. However, progress is limited by the scarcity of well-aligned multi-stain datasets. Inter-stain misalignment shifts corresponding tissue across slides, hindering consistent patch-level features and degrading slide-level embeddings. To address this, we curated a slide-level aligned, five-stain dataset (H&E, HER2, KI67, ER, PGR) to enable paired H&E-IHC learning and robust cross-stain representation. Leveraging this dataset, we propose Cross-Stain Contrastive Learning (CSCL), a two-stage pretraining framework with a lightweight adapter trained using patch-wise contrastive alignment to improve the compatibility of H&E features with corresponding IHC-derived contextual cues, and slide-level representation learning with Multiple Instance Learning (MIL), which uses a cross-stain attention fusion module to integrate stain-specific patch features and a cross-stain global alignment module to enforce consistency among slide-level embeddings across different stains. Experiments on cancer subtype classification, IHC biomarker status classification, and survival prediction show consistent gains, yielding high-quality, transferable H&E slide-level representations. The code and data are available at https://github.com/lily-zyz/CSCL.

Method: Display hidden numbers with same pixel texture as background; only perceptible to humans through motion and scale differences across frames, not frame-by-frame algorithmic processing.

Result: State-of-the-art multimodal AI models fail to extract correct values; 99.5% human success rate with 10.7s average completion time; no negative usability effects.

Conclusion: DOT-BI effectively distinguishes humans from bots using motion perception, is user-friendly, and works against current AI models.

Abstract: We propose the Dynamic Optical Test for Bot Identification (DOT-BI): a quick and easy method that uses human perception of motion to differentiate between human respondents and automated systems in surveys and online processes. In DOT-BI, a ‘hidden’ number is displayed with the same random black-and-white pixel texture as its background. Only the difference in motion and scale between the number and the background makes the number perceptible to humans across frames, while frame-by-frame algorithmic processing yields no meaningful signal. We conducted two preliminary assessments. Firstly, state-of-the-art, video-capable, multimodal models (GPT-5-Thinking and Gemini 2.5 Pro) fail to extract the correct value, even when given explicit instructions about the mechanism. Secondly, in an online survey (n=182), 99.5% (181/182) of participants solved the task, with an average end-to-end completion time of 10.7 seconds; a supervised lab study (n=39) found no negative effects on perceived ease-of-use or completion time relative to a control. We release code to generate tests and 100+ pre-rendered variants to facilitate adoption in surveys and online processes.

Method: 1) Enhanced input design to handle multiple conditional modalities (text, audio, images, video); 2) Decoupled multimodal fusion mechanism that sequentially injects different conditional signals into a DiT backbone; 3) Progressive multi-stage training paradigm using start-end frame difference embedding to dynamically adjust data sampling and loss weighting.

Result: BBF outperforms specialized state-of-the-art methods on both generic interpolation and audio-visual synchronized interpolation tasks, establishing a unified framework for video frame interpolation under coordinated multi-channel conditioning.

Conclusion: BBF provides a context-aware, multimodal video frame interpolation framework that successfully handles complex motion patterns and diverse application scenarios, including audio-visual synchronized interpolation, through flexible conditional input design and progressive training.

Abstract: Handling fast, complex, and highly non-linear motion patterns has long posed challenges for video frame interpolation. Although recent diffusion-based approaches improve upon traditional optical-flow-based methods, they still struggle to cover diverse application scenarios and often fail to produce sharp, temporally consistent frames in fine-grained motion tasks such as audio-visual synchronized interpolation. To address these limitations, we introduce BBF (Beyond Boundary Frames), a context-aware video frame interpolation framework, which could be guided by audio/visual semantics. First, we enhance the input design of the interpolation model so that it can flexibly handle multiple conditional modalities, including text, audio, images, and video. Second, we propose a decoupled multimodal fusion mechanism that sequentially injects different conditional signals into a DiT backbone. Finally, to maintain the generation abilities of the foundation model, we adopt a progressive multi-stage training paradigm, where the start-end frame difference embedding is used to dynamically adjust both the data sampling and the loss weighting. Extensive experimental results demonstrate that BBF outperforms specialized state-of-the-art methods on both generic interpolation and audio-visual synchronized interpolation tasks, establishing a unified framework for video frame interpolation under coordinated multi-channel conditioning.

Method: Proposes MKSNet with Multi-Kernel Selection mechanism using large convolutional kernels for extensive contextual information capture and adaptive kernel size selection. Also incorporates dual attention mechanism combining spatial attention (fine-tuning spatial weights) and channel attention (optimizing channel information selection).

Result: MKSNet substantially surpasses existing state-of-the-art models on DOTA-v1.0 and HRSC2016 benchmarks for small object detection in remote sensing images.

Conclusion: MKSNet effectively manages complexities of multi-scale and high-resolution remote sensing data, demonstrating superior innovation and effectiveness in small object detection for remote sensing applications.

Abstract: Deep convolutional neural networks (DCNNs) have substantially advanced object detection capabilities, particularly in remote sensing imagery. However, challenges persist, especially in detecting small objects where the high resolution of these images and the small size of target objects often result in a loss of critical information in the deeper layers of conventional CNNs. Additionally, the extensive spatial redundancy and intricate background details typical in remote-sensing images tend to obscure these small targets. To address these challenges, we introduce Multi-Kernel Selection Network (MKSNet), a novel network architecture featuring a novel Multi-Kernel Selection mechanism. The MKS mechanism utilizes large convolutional kernels to effectively capture an extensive range of contextual information. This innovative design allows for adaptive kernel size selection, significantly enhancing the network’s ability to dynamically process and emphasize crucial spatial details for small object detection. Furthermore, MKSNet also incorporates a dual attention mechanism, merging spatial and channel attention modules. The spatial attention module adaptively fine-tunes the spatial weights of feature maps, focusing more intensively on relevant regions while mitigating background noise. Simultaneously, the channel attention module optimizes channel information selection, improving feature representation and detection accuracy. Empirical evaluations on the DOTA-v1.0 and HRSC2016 benchmark demonstrate that MKSNet substantially surpasses existing state-of-the-art models in detecting small objects in remote sensing images. These results highlight MKSNet’s superior ability to manage the complexities associated with multi-scale and high-resolution image data, confirming its effectiveness and innovation in remote sensing object detection.

Method: HyperRNA uses a three-stage framework: 1) preprocessing with 3-bead coarse-grained representation to construct graph structures from RNA backbone coordinates, 2) encoding with attention embedding and hypergraph-based encoder to capture higher-order dependencies, and 3) autoregressive decoding to generate RNA sequences.

Result: Experimental results on PDBBind and RNAsolo datasets show HyperRNA outperforms existing RNA design methods for both RNA sequence generation and RNA-protein complex sequence generation tasks.

Conclusion: HyperRNA demonstrates the effectiveness of hypergraph-based approaches for RNA inverse folding and highlights the potential of leveraging hypergraphs in RNA engineering applications.

Abstract: The RNA inverse folding problem, a key challenge in RNA design, involves identifying nucleotide sequences that can fold into desired secondary structures, which are critical for ensuring molecular stability and function. The inherent complexity of this task stems from the intricate relationship between sequence and structure, making it particularly challenging. In this paper, we propose a framework, named HyperRNA, a generative model with an encoder-decoder architecture that leverages hypergraphs to design RNA sequences. Specifically, our HyperRNA model consists of three main components: preprocessing, encoding and decoding. In the preprocessing stage, graph structures are constructed by extracting the atom coordinates of RNA backbone based on 3-bead coarse-grained representation. The encoding stage processes these graphs, capturing higher order dependencies and complex biomolecular interactions using an attention embedding module and a hypergraph-based encoder. Finally, the decoding stage generates the RNA sequence in an autoregressive manner. We conducted quantitative and qualitative experiments on the PDBBind and RNAsolo datasets to evaluate the inverse folding task for RNA sequence generation and RNA-protein complex sequence generation. The experimental results demonstrate that HyperRNA not only outperforms existing RNA design methods but also highlights the potential of leveraging hypergraphs in RNA engineering.

Method: The method represents avatars as textured planes with two sets of learnable textures (low and high-frequency). It automatically switches to high-frequency textures only for close camera positions and gradually reduces their impact as the camera moves farther away. This adaptive approach adjusts rendering quality based on camera distance while limiting the number of required primitives for high FPS.

Result: Experiments on the ActorsHQ dataset with high-resolution input images show both qualitative and quantitative improvements over existing methods in rendering from novel wide-range camera positions, while maintaining high FPS performance.

Conclusion: CloseUpAvatar successfully addresses the challenge of rendering articulated human avatars across general camera motions, providing realistic rendering for both close-up and distant views through its adaptive texture-switching mechanism, outperforming previous approaches.

Abstract: We present a CloseUpAvatar - a novel approach for articulated human avatar representation dealing with more general camera motions, while preserving rendering quality for close-up views. CloseUpAvatar represents an avatar as a set of textured planes with two sets of learnable textures for low and high-frequency detail. The method automatically switches to high-frequency textures only for cameras positioned close to the avatar’s surface and gradually reduces their impact as the camera moves farther away. Such parametrization of the avatar enables CloseUpAvatar to adjust rendering quality based on camera distance ensuring realistic rendering across a wider range of camera orientations than previous approaches. We provide experiments using the ActorsHQ dataset with high-resolution input images. CloseUpAvatar demonstrates both qualitative and quantitative improvements over existing methods in rendering from novel wide range camera positions, while maintaining high FPS by limiting the number of required primitives.

Method: Built upon ScanNet videos, ToG-Bench comprises 100 annotated clips with 2,704 task-oriented grounding instructions created via a semi-automated pipeline combining foundation model annotation and human refinement. Introduces task-level evaluation metrics for multi-object and explicit-implicit object grounding.

Result: Benchmarking of seven state-of-the-art MLLMs reveals intrinsic challenges of task-oriented STVG and substantial performance gaps across explicit-implicit and multi-object grounding, highlighting the difficulty of bridging perception and interaction in embodied scenarios.

Conclusion: ToG-Bench addresses the critical need for task-oriented reasoning in embodied intelligence, exposing significant challenges in current models and providing a foundation for advancing spatio-temporal video grounding capabilities for real-world embodied agents.

Abstract: A core capability towards general embodied intelligence lies in localizing task-relevant objects from an egocentric perspective, formulated as Spatio-Temporal Video Grounding (STVG). Despite recent progress, existing STVG studies remain largely confined to object-centric and descriptive instructions, neglecting the task-oriented reasoning that is crucial for embodied agents to accomplish goal-directed interactions. To bridge this gap, we introduce \textbf{ToG-Bench}, the first task-oriented spatio-temporal video grounding benchmark for egocentric videos. ToG-Bench is characterized by three key features: (1) \textbf{Task-oriented Grounding}, which requires identifying and localizing objects based on intended tasks rather than straightforward descriptions; (2) \textbf{Explicit-Implicit Dual Grounding}, where target objects can be either explicitly mentioned or implicitly inferred by contextual reasoning; (3) \textbf{One-to-Many Grounding}, where a single instruction may correspond to multiple objects involved in task execution. Built upon videos sourced from ScanNet, ToG-Bench comprises 100 annotated clips with 2,704 task-oriented grounding instructions, constructed via a semi-automated pipeline that combines foundation model annotation and human refinement. In addition, we introduce a set of task-level evaluation metrics tailored for multi-object and explicit-implicit object grounding, and systematically benchmark seven state-of-the-art MLLMs. Extensive experiments reveal the intrinsic challenges of task-oriented STVG and substantial performance gaps across explicit-implicit and multi-object grounding, highlighting the difficulty of bridging perception and interaction in embodied scenarios. Data and code will be released at: \href{https://github.com/qaxuDev/ToG-Bench}{https://github.com/qaxuDev/ToG-Bench}..

Method: HBFormer combines a classic U-shaped encoder-decoder framework with a Swin Transformer backbone for hierarchical feature extraction. The core innovation is a ‘Bridge’ mechanism embodied by a novel Multi-Scale Feature Fusion (MFF) decoder that fuses multi-scale encoder features with global context using channel and spatial attention modules built from dilated and depth-wise convolutions.

Result: Comprehensive experiments on multi-organ, liver tumor, and bladder tumor benchmarks demonstrate state-of-the-art results, showing outstanding capabilities in microtumor and miniature organ segmentation.

Conclusion: HBFormer effectively addresses the limitation of localized attention mechanisms in Vision Transformers by creating a powerful feature bridge that captures long-range dependencies and refines object boundaries, making it highly effective for challenging medical image segmentation tasks.

Abstract: Medical image segmentation is a cornerstone of modern clinical diagnostics. While Vision Transformers that leverage shifted window-based self-attention have established new benchmarks in this field, they are often hampered by a critical limitation: their localized attention mechanism struggles to effectively fuse local details with global context. This deficiency is particularly detrimental to challenging tasks such as the segmentation of microtumors and miniature organs, where both fine-grained boundary definition and broad contextual understanding are paramount. To address this gap, we propose HBFormer, a novel Hybrid-Bridge Transformer architecture. The ‘Hybrid’ design of HBFormer synergizes a classic U-shaped encoder-decoder framework with a powerful Swin Transformer backbone for robust hierarchical feature extraction. The core innovation lies in its ‘Bridge’ mechanism, a sophisticated nexus for multi-scale feature integration. This bridge is architecturally embodied by our novel Multi-Scale Feature Fusion (MFF) decoder. Departing from conventional symmetric designs, the MFF decoder is engineered to fuse multi-scale features from the encoder with global contextual information. It achieves this through a synergistic combination of channel and spatial attention modules, which are constructed from a series of dilated and depth-wise convolutions. These components work in concert to create a powerful feature bridge that explicitly captures long-range dependencies and refines object boundaries with exceptional precision. Comprehensive experiments on challenging medical image segmentation datasets, including multi-organ, liver tumor, and bladder tumor benchmarks, demonstrate that HBFormer achieves state-of-the-art results, showcasing its outstanding capabilities in microtumor and miniature organ segmentation. Code and models are available at: https://github.com/lzeeorno/HBFormer.

Method: Proposes a retrieval-augmented framework that integrates a learnable prototype memory into standard encoder-decoder pipelines. After encoding partial input, the model retrieves the nearest manifold prototype from memory and fuses it with query features through confidence-gated weighting before decoding. The memory self-organizes into reusable tooth-shape prototypes without requiring tooth-position labels.

Result: Experiments on the Teeth3DS benchmark show consistent improvements in Chamfer Distance, with visualizations demonstrating sharper cusps, ridges, and interproximal transitions. The approach provides more accurate and faithful dental point-cloud completion.

Conclusion: The proposed framework offers a simple yet effective way to exploit cross-sample regularities for dental point-cloud completion, providing structural priors that stabilize missing-region inference and free decoder capacity for detail recovery. The module is plug-and-play and compatible with common completion backbones.

Abstract: Partial dental point clouds often suffer from large missing regions caused by occlusion and limited scanning views, which bias encoder-only global features and force decoders to hallucinate structures. We propose a retrieval-augmented framework for tooth completion that integrates a prototype memory into standard encoder–decoder pipelines. After encoding a partial input into a global descriptor, the model retrieves the nearest manifold prototype from a learnable memory and fuses it with the query feature through confidence-gated weighting before decoding. The memory is optimized end-to-end and self-organizes into reusable tooth-shape prototypes without requiring tooth-position labels, thereby providing structural priors that stabilize missing-region inference and free decoder capacity for detail recovery. The module is plug-and-play and compatible with common completion backbones, while keeping the same training losses. Experiments on a self-processed Teeth3DS benchmark demonstrate consistent improvements in Chamfer Distance, with visualizations showing sharper cusps, ridges, and interproximal transitions. Our approach provides a simple yet effective way to exploit cross-sample regularities for more accurate and faithful dental point-cloud completion.

Method: Two-part iterative optimization framework: 1) Sequential optimization updating motion and semantic fields for local consistency, 2) Global optimization jointly refining all attributes for long-term coherence. Includes 3D confidence map for motion prior adjustment, adaptive resampling for under-represented regions, and iterative semantic refinement with SAM2 prompt updates.

Result: Motion4D significantly outperforms both 2D foundation models and existing 3D-based approaches across diverse tasks including point-based tracking, video object segmentation, and novel view synthesis.

Conclusion: Motion4D successfully integrates 2D priors into 4D Gaussian Splatting to achieve 3D-consistent dynamic scene understanding, demonstrating superior performance across multiple scene understanding tasks from monocular videos.

Abstract: Recent advancements in foundation models for 2D vision have substantially improved the analysis of dynamic scenes from monocular videos. However, despite their strong generalization capabilities, these models often lack 3D consistency, a fundamental requirement for understanding scene geometry and motion, thereby causing severe spatial misalignment and temporal flickering in complex 3D environments. In this paper, we present Motion4D, a novel framework that addresses these challenges by integrating 2D priors from foundation models into a unified 4D Gaussian Splatting representation. Our method features a two-part iterative optimization framework: 1) Sequential optimization, which updates motion and semantic fields in consecutive stages to maintain local consistency, and 2) Global optimization, which jointly refines all attributes for long-term coherence. To enhance motion accuracy, we introduce a 3D confidence map that dynamically adjusts the motion priors, and an adaptive resampling process that inserts new Gaussians into under-represented regions based on per-pixel RGB and semantic errors. Furthermore, we enhance semantic coherence through an iterative refinement process that resolves semantic inconsistencies by alternately optimizing the semantic fields and updating prompts of SAM2. Extensive evaluations demonstrate that our Motion4D significantly outperforms both 2D foundation models and existing 3D-based approaches across diverse scene understanding tasks, including point-based tracking, video object segmentation, and novel view synthesis. Our code is available at https://hrzhou2.github.io/motion4d-web/.

Method: LAMP leverages LLMs as motion planners with a motion domain-specific language (DSL) inspired by cinematography. It uses program synthesis to generate structured motion programs from natural language, which are then deterministically mapped to 3D trajectories for objects and cameras.

Result: LAMP demonstrates improved motion controllability and better alignment with user intent compared to state-of-the-art alternatives, establishing the first framework for generating both object and camera motions directly from natural language.

Conclusion: LAMP successfully bridges natural language descriptions to explicit 3D motion trajectories through LLM-based program synthesis, advancing motion control capabilities in video generation.

Abstract: Video generation has achieved remarkable progress in visual fidelity and controllability, enabling conditioning on text, layout, or motion. Among these, motion control - specifying object dynamics and camera trajectories - is essential for composing complex, cinematic scenes, yet existing interfaces remain limited. We introduce LAMP that leverages large language models (LLMs) as motion planners to translate natural language descriptions into explicit 3D trajectories for dynamic objects and (relatively defined) cameras. LAMP defines a motion domain-specific language (DSL), inspired by cinematography conventions. By harnessing program synthesis capabilities of LLMs, LAMP generates structured motion programs from natural language, which are deterministically mapped to 3D trajectories. We construct a large-scale procedural dataset pairing natural text descriptions with corresponding motion programs and 3D trajectories. Experiments demonstrate LAMP’s improved performance in motion controllability and alignment with user intent compared to state-of-the-art alternatives establishing the first framework for generating both object and camera motions directly from natural language specifications.

Method: Uses dense 3DGS renderings for explicit geometric guidance. Employs two-stage training: first stage uses camera poses for coarse control, second stage incorporates 3DGS renderings for fine-grained viewpoint guidance. Introduces 3DGS-based cross-trajectory data curation to eliminate train-test gap in camera transformations. Creates ParaDrive dataset with over 110K parallel-trajectory video pairs.

Result: Achieves state-of-the-art camera controllability and structural consistency in generating novel driving trajectory videos. Demonstrates superior performance compared to existing methods through extensive experiments.

Conclusion: ReCamDriving provides an effective vision-based solution for camera-controllable novel-trajectory video generation by leveraging 3DGS renderings for geometric guidance and addressing training challenges through innovative data curation and two-stage training.

Abstract: We propose ReCamDriving, a purely vision-based, camera-controlled novel-trajectory video generation framework. While repair-based methods fail to restore complex artifacts and LiDAR-based approaches rely on sparse and incomplete cues, ReCamDriving leverages dense and scene-complete 3DGS renderings for explicit geometric guidance, achieving precise camera-controllable generation. To mitigate overfitting to restoration behaviors when conditioned on 3DGS renderings, ReCamDriving adopts a two-stage training paradigm: the first stage uses camera poses for coarse control, while the second stage incorporates 3DGS renderings for fine-grained viewpoint and geometric guidance. Furthermore, we present a 3DGS-based cross-trajectory data curation strategy to eliminate the train-test gap in camera transformation patterns, enabling scalable multi-trajectory supervision from monocular videos. Based on this strategy, we construct the ParaDrive dataset, containing over 110K parallel-trajectory video pairs. Extensive experiments demonstrate that ReCamDriving achieves state-of-the-art camera controllability and structural consistency.

Method: Proposes FeatureLens framework with Image Feature Extractor (IFE) and shallow classifiers (SVM, MLP, or XGBoost) using only 51-dimensional features, with model sizes ranging from 1,000 to 30,000 parameters.

Result: Achieves 97.8% to 99.75% accuracy in closed-set evaluation and 86.17% to 99.6% in generalization evaluation across FGSM, PGD, CW, and DAmageNet attacks.

Conclusion: FeatureLens offers a practical pathway toward transparent and effective adversarial defense by combining strong detection performance with excellent generalization, interpretability, and computational efficiency.

Abstract: Although the remarkable performance of deep neural networks (DNNs) in image classification, their vulnerability to adversarial attacks remains a critical challenge. Most existing detection methods rely on complex and poorly interpretable architectures, which compromise interpretability and generalization. To address this, we propose FeatureLens, a lightweight framework that acts as a lens to scrutinize anomalies in image features. Comprising an Image Feature Extractor (IFE) and shallow classifiers (e.g., SVM, MLP, or XGBoost) with model sizes ranging from 1,000 to 30,000 parameters, FeatureLens achieves high detection accuracy ranging from 97.8% to 99.75% in closed-set evaluation and 86.17% to 99.6% in generalization evaluation across FGSM, PGD, CW, and DAmageNet attacks, using only 51 dimensional features. By combining strong detection performance with excellent generalization, interpretability, and computational efficiency, FeatureLens offers a practical pathway toward transparent and effective adversarial defense.

Method: BlackCAtt uses minimal, causally sufficient pixel sets combined with bounding boxes from object detectors to create adversarial attacks. It treats detectors as black boxes and works across different architectures by identifying causal pixels that lead to targeted attacks (removing, modifying, or adding bounding boxes).

Result: On COCO test dataset, BlackCAtt outperforms baselines: 2.7x better at removing detections, 3.86x better at changing detections, and 5.75x better at triggering new spurious detections. Attacks are imperceptible and work across different detector architectures.

Conclusion: Causal pixel identification enables precise, explainable, and imperceptible black-box attacks on object detectors. BlackCAtt demonstrates the power of causal analysis for understanding and creating effective adversarial attacks while maintaining cross-architecture compatibility.

Abstract: Adversarial perturbations are a useful way to expose vulnerabilities in object detectors. Existing perturbation methods are frequently white-box and architecture specific. More importantly, while they are often successful, it is rarely clear why they work. Insights into the mechanism of this success would allow developers to understand and analyze these attacks, as well as fine-tune the model to prevent them. This paper presents BlackCAtt, a black-box algorithm and a tool, which uses minimal, causally sufficient pixel sets to construct explainable, imperceptible, reproducible, architecture-agnostic attacks on object detectors. BlackCAtt combines causal pixels with bounding boxes produced by object detectors to create adversarial attacks that lead to the loss, modification or addition of a bounding box. BlackCAtt works across different object detectors of different sizes and architectures, treating the detector as a black box. We compare the performance of BlackCAtt with other black-box attack methods and show that identification of causal pixels leads to more precisely targeted and less perceptible attacks. On the COCO test dataset, our approach is 2.7 times better than the baseline in removing a detection, 3.86 times better in changing a detection, and 5.75 times better in triggering new, spurious, detections. The attacks generated by BlackCAtt are very close to the original image, and hence imperceptible, demonstrating the power of causal pixels.

Method: Multi-Scale Visual Prompting (MSVP) learns global, mid-scale, and local prompt maps fused with input images via lightweight 1×1 convolution. It’s backbone-agnostic and adds minimal parameters.

Result: MSVP yields consistent improvements across CNN and Vision Transformer backbones on MNIST, Fashion-MNIST, and CIFAR-10 with negligible computational overhead. Ablations show effectiveness of multi-scale prompting.

Conclusion: Multi-scale prompting provides effective inductive bias even on low-resolution images, offering a simple, generic solution for visual prompting on small-image benchmarks.

Abstract: Visual prompting has recently emerged as an efficient strategy to adapt vision models using lightweight, learnable parameters injected into the input space. However, prior work mainly targets large Vision Transformers and high-resolution datasets such as ImageNet. In contrast, small-image benchmarks like MNIST, Fashion-MNIST, and CIFAR-10 remain widely used in education, prototyping, and research, yet have received little attention in the context of prompting. In this paper, we introduce \textbf{Multi-Scale Visual Prompting (MSVP)}, a simple and generic module that learns a set of global, mid-scale, and local prompt maps fused with the input image via a lightweight $1 \times 1$ convolution. MSVP is backbone-agnostic, adds less than $0.02%$ parameters, and significantly improves performance across CNN and Vision Transformer backbones. We provide a unified benchmark on MNIST, Fashion-MNIST, and CIFAR-10 using a simple CNN, ResNet-18, and a small Vision Transformer. Our method yields consistent improvements with negligible computational overhead. We further include ablations on prompt scales, fusion strategies, and backbone architectures, along with qualitative analyzes using prompt visualizations and Grad-CAM. Our results demonstrate that multi-scale prompting provides an effective inductive bias even on low-resolution images.

Method: Proposes an improved ResNet34-based model with: 1) multi-scale input module as first layer, 2) Inception v2 module as residual downsampling layer, 3) channel attention mechanism to weight important feature channels.

Result: Five-fold cross-validation shows 98.8% average classification accuracy (1% higher than original ResNet34) with only 80% of original model parameters.

Conclusion: The improved network achieves higher accuracy with fewer parameters, making it more efficient for brain tumor image classification while reducing computational complexity.

Abstract: Previously, image interpretation in radiology relied heavily on manual methods. However, manual classification of brain tumor medical images is time-consuming and labor-intensive. Even with shallow convolutional neural network models, the accuracy is not ideal. To improve the efficiency and accuracy of brain tumor image classification, this paper proposes a brain tumor classification model based on an improved ResNet34 network. This model uses the ResNet34 residual network as the backbone network and incorporates multi-scale feature extraction. It uses a multi-scale input module as the first layer of the ResNet34 network and an Inception v2 module as the residual downsampling layer. Furthermore, a channel attention mechanism module assigns different weights to different channels of the image from a channel domain perspective, obtaining more important feature information. The results after a five-fold crossover experiment show that the average classification accuracy of the improved network model is approximately 98.8%, which is not only 1% higher than ResNet34, but also only 80% of the number of parameters of the original model. Therefore, the improved network model not only improves accuracy but also reduces clutter, achieving a classification effect with fewer parameters and higher accuracy.

Method: 1) Built ColonVQA dataset with 1.1M+ VQA entries across 76 findings and 18 tasks; 2) Evaluated 22 MLLMs for generalizability and reliability; 3) Created ColonReason reasoning dataset via multi-expert debating; 4) Developed ColonR1 model with task-adaptive rewarding and gradient-stable optimization.

Result: ColonR1 achieved 56.61% overall accuracy under data-scarce conditions, outperforming supervised fine-tuning by 25.22%, establishing a new reasoning-enabled baseline for multimodal colonoscopy analysis.

Conclusion: Colon-X provides foundational resources for multimodal colonoscopy AI, demonstrating that reasoning-centric approaches significantly outperform traditional methods and setting new standards for clinical AI in gastroenterology.

Abstract: In this study, we present Colon-X, an open initiative aimed at advancing multimodal intelligence in colonoscopy. We begin by constructing ColonVQA, the most comprehensive multimodal dataset ever built for colonoscopy, featuring over 1.1M+ visual question answering entries across 76 clinical findings and 18 multimodal tasks. Beyond serving as a community-wide data foundation, we further investigate a critical yet underexplored transition in colonoscopy - evolving from multimodal understanding to clinical reasoning: (a) To capture the current landscape of multimodal understanding behaviors, we systematically assess the generalizability of 22 multimodal large language models and examine their reliability under human-induced perturbations. The results reveal that clinical outputs from leading MLLMs remain far from robust and trustworthy. (b) To narrow this gap, we further explore reasoning-centric intelligence tailored for colonoscopy. Specifically, we curate ColonReason, a clinically grounded reasoning dataset annotated through a multi-expert debating pipeline, and develop ColonR1, the first R1-styled model incorporating task-adaptive rewarding and gradient-stable optimization techniques. Under data-scarce conditions, our ColonR1 achieves 56.61% overall accuracy, outperforming supervised fine-tuning by 25.22%, and sets a new reasoning-enabled baseline for multimodal colonoscopy analysis. All data and model resources are publicly available at https://github.com/ai4colonoscopy/Colon-X.

Method: Proposes ConvRot, a group-wise rotation-based quantization method using regular Hadamard transform (RHT) to suppress both row-wise and column-wise outliers while reducing complexity from quadratic to linear. Also designs ConvLinear4bit, a plug-and-play module integrating rotation, quantization, GEMM, and dequantization for W4A4 inference without retraining.

Result: Experiments on FLUX.1-dev show 2.26× speedup and 4.05× memory reduction while maintaining image fidelity. This is the first application of rotation-based quantization for plug-and-play W4A4 inference in diffusion transformers.

Conclusion: ConvRot effectively addresses deployment challenges of diffusion transformers by enabling efficient 4-bit quantization without retraining, significantly improving speed and memory efficiency while preserving visual quality.

Abstract: Diffusion transformers have demonstrated strong capabilities in generating high-quality images. However, as model size increases, the growing memory footprint and inference latency pose significant challenges for practical deployment. Recent studies in large language models (LLMs) show that rotation-based techniques can smooth outliers and enable 4-bit quantization, but these approaches often incur substantial overhead and struggle with row-wise outliers in diffusion transformers. To address these challenges, we propose ConvRot, a group-wise rotation-based quantization method that leverages regular Hadamard transform (RHT) to suppress both row-wise and column-wise outliers while reducing complexity from quadratic to linear. Building on this, we design ConvLinear4bit, a plug-and-play module that integrates rotation, quantization, GEMM, and dequantization, enabling W4A4 inference without retraining and preserving visual quality. Experiments on FLUX.1-dev demonstrate a 2.26$\times$ speedup and 4.05$\times$ memory reduction while maintaining image fidelity. To our knowledge, this is the first application of rotation-based quantization for plug-and-play W4A4 inference in diffusion transformers.

Method: Learns structured, disentangled latent spaces with controlled information sharing for geometry and appearance from spatially-grouped 3D Gaussians, then applies text-conditioned edits using a latent diffusion model on these learned representations.

Result: Delivers instant, high-fidelity, geometry-preserving, multi-view consistent stylization that surpasses methods requiring per-instance test-time optimization.

Conclusion: Unlocks practical, democratized 3D stylization at scale by enabling feed-forward, instant text-driven editing without per-asset optimization.

Abstract: 3D stylization is central to game development, virtual reality, and digital arts, where the demand for diverse assets calls for scalable methods that support fast, high-fidelity manipulation. Existing text-to-3D stylization methods typically distill from 2D image editors, requiring time-intensive per-asset optimization and exhibiting multi-view inconsistency due to the limitations of current text-to-image models, which makes them impractical for large-scale production. In this paper, we introduce GaussianBlender, a pioneering feed-forward framework for text-driven 3D stylization that performs edits instantly at inference. Our method learns structured, disentangled latent spaces with controlled information sharing for geometry and appearance from spatially-grouped 3D Gaussians. A latent diffusion model then applies text-conditioned edits on these learned representations. Comprehensive evaluations show that GaussianBlender not only delivers instant, high-fidelity, geometry-preserving, multi-view consistent stylization, but also surpasses methods that require per-instance test-time optimization - unlocking practical, democratized 3D stylization at scale.

Method: This appears to be a review/survey paper that explores active visual perception through comprehensive analysis of existing research. The method involves examining the fundamental principles of active perception, comparing it with passive approaches, and analyzing its applications across various domains like robotics, autonomous vehicles, and surveillance.

Result: The paper provides a comprehensive overview of active visual perception’s potential and current state. It identifies that active systems can overcome limitations of passive approaches by dynamically engaging with environments, but also highlights significant challenges including real-time processing of complex visual data, decision-making in dynamic environments, and integration of multimodal sensory inputs.

Conclusion: Active visual perception represents a powerful approach with significant promise across multiple applications, but broader adoption requires overcoming key challenges in real-time processing, dynamic decision-making, and multimodal integration. The field needs continued research to address these obstacles and fully realize the potential of active perception systems.

Abstract: Active visual perception refers to the ability of a system to dynamically engage with its environment through sensing and action, allowing it to modify its behavior in response to specific goals or uncertainties. Unlike passive systems that rely solely on visual data, active visual perception systems can direct attention, move sensors, or interact with objects to acquire more informative data. This approach is particularly powerful in complex environments where static sensing methods may not provide sufficient information. Active visual perception plays a critical role in numerous applications, including robotics, autonomous vehicles, human-computer interaction, and surveillance systems. However, despite its significant promise, there are several challenges that need to be addressed, including real-time processing of complex visual data, decision-making in dynamic environments, and integrating multimodal sensory inputs. This paper explores both the opportunities and challenges inherent in active visual perception, providing a comprehensive overview of its potential, current research, and the obstacles that must be overcome for broader adoption.

Method: Built on self-supervised representations with composite supervision balancing region overlap learning, pixel-wise classification fidelity, and boundary-aware IoU refinement. Uses multi-scale token reconstruction decoder for segmentation and shared global representations for classification and text generation.

Result: PULSE learns task-invariant cardiac priors, generalizes robustly across datasets, and can adapt to new imaging modalities with minimal supervision, enabling transition from pixels to structures to clinical reasoning in one architecture.

Conclusion: PULSE moves the field toward a scalable, foundation-style cardiac analysis framework that unifies multiple analysis tasks within a single architecture while maintaining generalization capabilities.

Abstract: Cardiac image analysis remains fragmented across tasks: anatomical segmentation, disease classification, and grounded clinical report generation are typically handled by separate networks trained under different data regimes. No existing framework unifies these objectives within a single architecture while retaining generalization across imaging modalities and datasets. We introduce PULSE, a multi-task vision-language framework built on self-supervised representations and optimized through a composite supervision strategy that balances region overlap learning, pixel wise classification fidelity, and boundary aware IoU refinement. A multi-scale token reconstruction decoder enables anatomical segmentation, while shared global representations support disease classification and clinically grounded text output allowing the model to transition from pixels to structures and finally clinical reasoning within one architecture. Unlike prior task-specific pipelines, PULSE learns task-invariant cardiac priors, generalizes robustly across datasets, and can be adapted to new imaging modalities with minimal supervision. This moves the field closer to a scalable, foundation style cardiac analysis framework.

Method: SUSS models each image through perceptual components represented by structured multivariate Normal distributions. These are trained generatively and self-supervised to assign high likelihood to human-imperceptible augmentations. The final score is a weighted sum of component log-probabilities with weights learned from human perceptual datasets. Unlike feature-based methods, it learns image-specific linear transformations of residuals in pixel space.

Result: SUSS aligns closely with human perceptual judgments, shows strong perceptual calibration across diverse distortion types, provides localized interpretable explanations through decorrelated residuals and sampling, and demonstrates stable optimization behavior with competitive performance as a perceptual loss for downstream imaging tasks.

Conclusion: SUSS offers a promising approach to perceptual similarity that combines human alignment with interpretability, addressing limitations of both deep feature-based methods and hand-crafted measures while enabling transparent inspection of similarity assessments.

Abstract: Perceptual similarity scores that align with human vision are critical for both training and evaluating computer vision models. Deep perceptual losses, such as LPIPS, achieve good alignment but rely on complex, highly non-linear discriminative features with unknown invariances, while hand-crafted measures like SSIM are interpretable but miss key perceptual properties. We introduce the Structured Uncertainty Similarity Score (SUSS); it models each image through a set of perceptual components, each represented by a structured multivariate Normal distribution. These are trained in a generative, self-supervised manner to assign high likelihood to human-imperceptible augmentations. The final score is a weighted sum of component log-probabilities with weights learned from human perceptual datasets. Unlike feature-based methods, SUSS learns image-specific linear transformations of residuals in pixel space, enabling transparent inspection through decorrelated residuals and sampling. SUSS aligns closely with human perceptual judgments, shows strong perceptual calibration across diverse distortion types, and provides localized, interpretable explanations of its similarity assessments. We further demonstrate stable optimization behavior and competitive performance when using SUSS as a perceptual loss for downstream imaging tasks.

Method: Integrates dataset-adaptive image pairing strategy using DINO for semantic retrieval of relevant image pairs, with rotation-based augmentation for orientation-dependent local feature extraction using ALIKED and Light Glue.

Result: Achieved consistent improvements in mean Average Accuracy (mAA) on Kaggle Image Matching Challenge 2025, winning Silver Award (47th of 943 teams).

Conclusion: Combining self-supervised global descriptors (DINO) with rotation-enhanced local matching provides robust and scalable solution for large-scale 3D reconstruction from internet images.

Abstract: This paper presents DINO-RotateMatch, a deep-learning framework designed to address the chal lenges of image matching in large-scale 3D reconstruction from unstructured Internet images. The method integrates a dataset-adaptive image pairing strategy with rotation-aware keypoint extraction and matching. DINO is employed to retrieve semantically relevant image pairs in large collections, while rotation-based augmentation captures orientation-dependent local features using ALIKED and Light Glue. Experiments on the Kaggle Image Matching Challenge 2025 demonstrate consistent improve ments in mean Average Accuracy (mAA), achieving a Silver Award (47th of 943 teams). The results confirm that combining self-supervised global descriptors with rotation-enhanced local matching offers a robust and scalable solution for large-scale 3D reconstruction.

Method: Proposes PosA-VLA framework with pose-conditioned anchor attention mechanism that guides visual attention to task-relevant regions using pose-conditioned supervision. Uses lightweight architecture without auxiliary perception modules like segmentation or grounding networks.

Result: Extensive experiments show the method executes embodied tasks with precise and time-efficient behavior across diverse robotic manipulation benchmarks, with robust generalization in challenging environments.

Conclusion: Pose-conditioned anchor attention effectively addresses VLA’s redundant action problem by aligning instruction semantics with actionable visual cues, improving both precision and efficiency for real-world applications.

Abstract: The Vision-Language-Action (VLA) models have demonstrated remarkable performance on embodied tasks and shown promising potential for real-world applications. However, current VLAs still struggle to produce consistent and precise target-oriented actions, as they often generate redundant or unstable motions along trajectories, limiting their applicability in time-sensitive scenarios.In this work, we attribute these redundant actions to the spatially uniform perception field of existing VLAs, which causes them to be distracted by target-irrelevant objects, especially in complex environments.To address this issue, we propose an efficient PosA-VLA framework that anchors visual attention via pose-conditioned supervision, consistently guiding the model’s perception toward task-relevant regions. The pose-conditioned anchor attention mechanism enables the model to better align instruction semantics with actionable visual cues, thereby improving action generation precision and efficiency. Moreover, our framework adopts a lightweight architecture and requires no auxiliary perception modules (e.g., segmentation or grounding networks), ensuring efficient inference. Extensive experiments verify that our method executes embodied tasks with precise and time-efficient behavior across diverse robotic manipulation benchmarks and shows robust generalization in a variety of challenging environments.

Method: Dual-level Modality Debiasing Learning (DMDL) framework with: 1) Causality-inspired Adjustment Intervention (CAI) module at model level using causal modeling instead of likelihood-based modeling, 2) Collaborative Bias-free Training (CBT) strategy at optimization level integrating modality-specific augmentation, label refinement, and feature alignment.

Result: Extensive experiments on benchmark datasets demonstrate that DMDL enables modality-invariant feature learning and produces a more generalized model.

Conclusion: The proposed DMDL framework effectively addresses modality bias in unsupervised visible-infrared person re-identification through dual-level debiasing, leading to improved generalization and modality-invariant feature learning.

Abstract: Two-stage learning pipeline has achieved promising results in unsupervised visible-infrared person re-identification (USL-VI-ReID). It first performs single-modality learning and then operates cross-modality learning to tackle the modality discrepancy. Although promising, this pipeline inevitably introduces modality bias: modality-specific cues learned in the single-modality training naturally propagate into the following cross-modality learning, impairing identity discrimination and generalization. To address this issue, we propose a Dual-level Modality Debiasing Learning (DMDL) framework that implements debiasing at both the model and optimization levels. At the model level, we propose a Causality-inspired Adjustment Intervention (CAI) module that replaces likelihood-based modeling with causal modeling, preventing modality-induced spurious patterns from being introduced, leading to a low-biased model. At the optimization level, a Collaborative Bias-free Training (CBT) strategy is introduced to interrupt the propagation of modality bias across data, labels, and features by integrating modality-specific augmentation, label refinement, and feature alignment. Extensive experiments on benchmark datasets demonstrate that DMDL could enable modality-invariant feature learning and a more generalized model.

Method: SelfDebias identifies semantic clusters in an image encoder’s embedding space and uses these clusters to guide the diffusion process during inference by minimizing KL divergence between output distribution and uniform distribution. It works with any UNet-based diffusion model without requiring human-annotated datasets or external classifiers.

Result: Extensive experiments show SelfDebias generalizes across prompts and diffusion model architectures (both conditional and unconditional), effectively debiasing images along demographic dimensions while maintaining visual fidelity, and also handles abstract concepts where bias identification is challenging.

Conclusion: SelfDebias provides a fully unsupervised, architecture-agnostic approach to debiasing diffusion models that automatically identifies semantic modes without requiring external supervision, making it practical for real-world deployment across various bias dimensions.

Abstract: Text-to-image (T2I) diffusion models have achieved widespread success due to their ability to generate high-resolution, photorealistic images. These models are trained on large-scale datasets, like LAION-5B, often scraped from the internet. However, since this data contains numerous biases, the models inherently learn and reproduce them, resulting in stereotypical outputs. We introduce SelfDebias, a fully unsupervised test-time debiasing method applicable to any diffusion model that uses a UNet as its noise predictor. SelfDebias identifies semantic clusters in an image encoder’s embedding space and uses these clusters to guide the diffusion process during inference, minimizing the KL divergence between the output distribution and the uniform distribution. Unlike supervised approaches, SelfDebias does not require human-annotated datasets or external classifiers trained for each generated concept. Instead, it is designed to automatically identify semantic modes. Extensive experiments show that SelfDebias generalizes across prompts and diffusion model architectures, including both conditional and unconditional models. It not only effectively debiases images along key demographic dimensions while maintaining the visual fidelity of the generated images, but also more abstract concepts for which identifying biases is also challenging.

Method: Latent Scale Rejection Sampling (LSRS) progressively refines token maps by using a lightweight scoring model to evaluate multiple candidate token maps at each scale, selecting high-quality maps to guide subsequent scale generation while prioritizing early scales for structural coherence.

Result: LSRS significantly improves VAR generation quality with minimal computational overhead: for VAR-d30, inference time increases by only 1% while FID improves from 1.95 to 1.78; with 15% more inference time, FID further improves to 1.66.

Conclusion: LSRS provides an efficient test-time scaling solution that effectively mitigates autoregressive error accumulation in VAR models while maintaining computational efficiency.

Abstract: Visual Autoregressive (VAR) modeling approach for image generation proposes autoregressive processing across hierarchical scales, decoding multiple tokens per scale in parallel. This method achieves high-quality generation while accelerating synthesis. However, parallel token sampling within a scale may lead to structural errors, resulting in suboptimal generated images. To mitigate this, we propose Latent Scale Rejection Sampling (LSRS), a method that progressively refines token maps in the latent scale during inference to enhance VAR models. Our method uses a lightweight scoring model to evaluate multiple candidate token maps sampled at each scale, selecting the high-quality map to guide subsequent scale generation. By prioritizing early scales critical for structural coherence, LSRS effectively mitigates autoregressive error accumulation while maintaining computational efficiency. Experiments demonstrate that LSRS significantly improves VAR’s generation quality with minimal additional computational overhead. For the VAR-d30 model, LSRS increases the inference time by merely 1% while reducing its FID score from 1.95 to 1.78. When the inference time is increased by 15%, the FID score can be further reduced to 1.66. LSRS offers an efficient test-time scaling solution for enhancing VAR-based generation.

Method: Three-stage approach: 1) Segmentation using Contour and Detectron2, 2) Mapping symbols to Gardiner codes, 3) Translation using CNN model. Utilized Morris Franken and EgyptianTranslation datasets for classification and translation.

Result: The model achieved a BLEU score of 42.2, which represents a significant improvement compared to previous research in hieroglyph translation.

Conclusion: The proposed deep learning method successfully addresses the challenges of hieroglyph translation, demonstrating promising results for automatic recognition and translation of ancient Egyptian writing from images to English.

Abstract: Egyptian hieroglyphs, the ancient Egyptian writing system, are composed entirely of drawings. Translating these glyphs into English poses various challenges, including the fact that a single glyph can have multiple meanings. Deep learning translation applications are evolving rapidly, producing remarkable results that significantly impact our lives. In this research, we propose a method for the automatic recognition and translation of ancient Egyptian hieroglyphs from images to English. This study utilized two datasets for classification and translation: the Morris Franken dataset and the EgyptianTranslation dataset. Our approach is divided into three stages: segmentation (using Contour and Detectron2), mapping symbols to Gardiner codes, and translation (using the CNN model). The model achieved a BLEU score of 42.2, a significant result compared to previous research.

Method: Uses mathematical transforms to extract breath rate from video footage with minimal hardware requirements, specifically designed to be robust against subject movement distortions.

Result: Tested on 14 volunteers with over 2.5 hours of video, achieved average mean absolute error of 0.57 respirations per minute and relative deviation from ground truth less than 5%, outperforming previous works.

Conclusion: The proposed method enables accurate remote breath rate measurement without significant behavioral limitations, making it practical for real-world applications where subject movement is unavoidable.

Abstract: Proliferation of cheap and accessible cameras makes it possible to measure a subject’s breath rate from video footage alone. Recent works on this topic have proposed a variety of approaches for accurately measuring human breath rate, however they are either tested in near-ideal conditions, or produce results that are not sufficiently accurate. The present study proposes a more robust method to measure breath rate in humans with minimal hardware requirements using a combination of mathematical transforms with a relative deviation from the ground truth of less than 5%. The method was tested on videos taken from 14 volunteers with a total duration of over 2 hours 30 minutes. The obtained results were compared to reference data and the average mean absolute error was found to be at 0.57 respirations per minute, which is noticeably better than the results from previous works. The breath rate measurement method proposed in the present article is more resistant to distortions caused by subject movement and thus allows one to remotely measure the subject’s breath rate without any significant limitations on the subject’s behavior.

Method: BlurDM uses a dual-diffusion forward scheme that diffuses both noise and blur onto sharp images, with a reverse process that performs simultaneous denoising and deblurring. It operates in latent space for efficiency and integrates as a flexible prior generation network.

Result: BlurDM significantly and consistently enhances existing deblurring methods on four benchmark datasets, demonstrating superior performance.

Conclusion: BlurDM effectively integrates blur formation into diffusion models, providing a powerful approach for dynamic scene deblurring that outperforms existing methods.

Abstract: Diffusion models show promise for dynamic scene deblurring; however, existing studies often fail to leverage the intrinsic nature of the blurring process within diffusion models, limiting their full potential. To address it, we present a Blur Diffusion Model (BlurDM), which seamlessly integrates the blur formation process into diffusion for image deblurring. Observing that motion blur stems from continuous exposure, BlurDM implicitly models the blur formation process through a dual-diffusion forward scheme, diffusing both noise and blur onto a sharp image. During the reverse generation process, we derive a dual denoising and deblurring formulation, enabling BlurDM to recover the sharp image by simultaneously denoising and deblurring, given pure Gaussian noise conditioned on the blurred image as input. Additionally, to efficiently integrate BlurDM into deblurring networks, we perform BlurDM in the latent space, forming a flexible prior generation network for deblurring. Extensive experiments demonstrate that BlurDM significantly and consistently enhances existing deblurring methods on four benchmark datasets. The source code is available at https://github.com/Jin-Ting-He/BlurDM.

Method: Proposed LUnet with compact, flat hierarchy where channels are not doubled as resolution is halved. Uses constant channel count across layers and leverages skip connections to reduce bottleneck channels.

Result: LUnet achieves performance comparable to conventional Unet and pruned networks with over 30x fewer parameters. Simple random channel elimination at pruning-identified layers performs similarly to complex pruning methods like STAMP.

Conclusion: The final architecture structure is more crucial than channel selection strategy in pruning. Skip connections enable significant bottleneck channel reduction, making flat architectures with constant channels viable for semantic segmentation.

Abstract: Unet and its variations have been standard in semantic image segmentation, especially for computer assisted radiology. Current Unet architectures iteratively downsample spatial resolution while increasing channel dimensions to preserve information content. Such a structure demands a large memory footprint, limiting training batch sizes and increasing inference latency. Channel pruning compresses Unet architecture without accuracy loss, but requires lengthy optimization and may not generalize across tasks and datasets. By investigating Unet pruning, we hypothesize that the final structure is the crucial factor, not the channel selection strategy of pruning. Based on our observations, we propose a lean Unet architecture (LUnet) with a compact, flat hierarchy where channels are not doubled as resolution is halved. We evaluate on a public MRI dataset allowing comparable reporting, as well as on two internal CT datasets. We show that a state-of-the-art pruning solution (STAMP) mainly prunes from the layers with the highest number of channels. Comparatively, simply eliminating a random channel at the pruning-identified layer or at the largest layer achieves similar or better performance. Our proposed LUnet with fixed architectures and over 30 times fewer parameters achieves performance comparable to both conventional Unet counterparts and data-adaptively pruned networks. The proposed lean Unet with constant channel count across layers requires far fewer parameters while achieving performance superior to standard Unet for the same total number of parameters. Skip connections allow Unet bottleneck channels to be largely reduced, unlike standard encoder-decoder architectures requiring increased bottleneck channels for information propagation.

Method: DIQ-H applies physics-based corruptions (motion blur, sensor noise, compression artifacts) to temporal sequences and measures hallucination persistence, error recovery, and temporal consistency through multi-turn QA tasks. Uses Uncertainty-Guided Iterative Refinement (UIR) for scalable annotation with lightweight VLMs and uncertainty filtering.

Result: Experiments on 16 state-of-the-art VLMs reveal substantial robustness gaps: GPT-4o achieves only 78.5% recovery rate, while open-source models struggle with temporal consistency (<60%). UIR achieves 15.3% accuracy improvement for annotation.

Conclusion: DIQ-H provides a comprehensive platform for evaluating VLM reliability in real-world deployments, highlighting significant robustness challenges that need to be addressed for safety-critical applications.

Abstract: Vision-Language Models (VLMs) deployed in safety-critical applications such as autonomous driving must handle continuous visual streams under imperfect conditions. However, existing benchmarks focus on static, high-quality images and ignore temporal degradation and error propagation, which are critical failure modes where transient visual corruption induces hallucinations that persist across subsequent frames. We introduce DIQ-H, the first benchmark for evaluating VLM robustness under dynamic visual degradation in temporal sequences. DIQ-H applies physics-based corruptions including motion blur, sensor noise, and compression artifacts, and measures hallucination persistence, error recovery, and temporal consistency through multi-turn question-answering tasks. To enable scalable annotation, we propose Uncertainty-Guided Iterative Refinement (UIR), which generates reliable pseudo-ground-truth using lightweight VLMs with uncertainty filtering, achieving a 15.3 percent accuracy improvement. Experiments on 16 state-of-the-art VLMs reveal substantial robustness gaps: even advanced models such as GPT-4o achieve only a 78.5 percent recovery rate, while open-source models struggle with temporal consistency at less than 60 percent. DIQ-H provides a comprehensive platform for evaluating VLM reliability in real-world deployments.

Method: Proposes HP-Net with: 1) Feedback pooling module to extract information-rich, robust, and concise pooled features from human body heatmaps; 2) Spatial-motion co-learning module; 3) Text refinement modulation module to integrate pooled features with multimodal data.

Result: Extensive experiments on NTU RGB+D 60, NTU RGB+D 120, Toyota-Smarthome, and UAV-Human benchmarks consistently show HP-Net outperforms existing human action recognition methods.

Conclusion: HP-Net effectively addresses limitations of existing RGB video action recognition methods by extracting superior pooled features and integrating multimodal data, achieving state-of-the-art performance across multiple benchmarks.

Abstract: Human action recognition (HAR) in videos has garnered widespread attention due to the rich information in RGB videos. Nevertheless, existing methods for extracting deep features from RGB videos face challenges such as information redundancy, susceptibility to noise and high storage costs. To address these issues and fully harness the useful information in videos, we propose a novel heatmap pooling network (HP-Net) for action recognition from videos, which extracts information-rich, robust and concise pooled features of the human body in videos through a feedback pooling module. The extracted pooled features demonstrate obvious performance advantages over the previously obtained pose data and heatmap features from videos. In addition, we design a spatial-motion co-learning module and a text refinement modulation module to integrate the extracted pooled features with other multimodal data, enabling more robust action recognition. Extensive experiments on several benchmarks namely NTU RGB+D 60, NTU RGB+D 120, Toyota-Smarthome and UAV-Human consistently verify the effectiveness of our HP-Net, which outperforms the existing human action recognition methods. Our code is publicly available at: https://github.com/liujf69/HPNet-Action.

Method: Two key designs: (1) Step-based text embedding perturbation combined with frequency-guided noise schedules and spatial noise perturbation; (2) Adaptive perturbation intensity based on frequency-specific contributions and tolerance to perturbation.

Result: The approach demonstrates significant improvements on multiple benchmarks with almost no additional computation and can be seamlessly integrated into existing test-time scaling methods.

Conclusion: Text embedding perturbation represents a novel randomness format that complements spatial noise in diffusion models, enhancing generative diversity and quality by addressing high-frequency detail limitations while maintaining computational efficiency.

Abstract: Test-time scaling (TTS) aims to achieve better results by increasing random sampling and evaluating samples based on rules and metrics. However, in text-to-image(T2I) diffusion models, most related works focus on search strategies and reward models, yet the impact of the stochastic characteristic of noise in T2I diffusion models on the method’s performance remains unexplored. In this work, we analyze the effects of randomness in T2I diffusion models and explore a new format of randomness for TTS: text embedding perturbation, which couples with existing randomness like SDE-injected noise to enhance generative diversity and quality. We start with a frequency-domain analysis of these formats of randomness and their impact on generation, and find that these two randomness exhibit complementary behavior in the frequency domain: spatial noise favors low-frequency components (early steps), while text embedding perturbation enhances high-frequency details (later steps), thereby compensating for the potential limitations of spatial noise randomness in high-frequency manipulation. Concurrently, text embedding demonstrates varying levels of tolerance to perturbation across different dimensions of the generation process. Specifically, our method consists of two key designs: (1) Introducing step-based text embedding perturbation, combining frequency-guided noise schedules with spatial noise perturbation. (2) Adapting the perturbation intensity selectively based on their frequency-specific contributions to generation and tolerance to perturbation. Our approach can be seamlessly integrated into existing TTS methods and demonstrates significant improvements on multiple benchmarks with almost no additional computation. Code is available at \href{https://github.com/xuhang07/TEP-Diffusion}{https://github.com/xuhang07/TEP-Diffusion}.

Method: CoDA first identifies the “intrinsic core distribution” of the target dataset using robust density-based discovery. It then steers the generative process of an off-the-shelf text-to-image model to align generated samples with this core distribution, bridging the gap between general-purpose priors and target semantics.

Result: CoDA achieves performance on par with or superior to previous methods that require target-specific model training across all benchmarks, including ImageNet-1K and subsets. It establishes new SOTA accuracy of 60.4% at 50-IPC setup on ImageNet-1K without needing a model trained on the target dataset.

Conclusion: CoDA successfully addresses fundamental limitations in dataset distillation by enabling effective distillation using only off-the-shelf text-to-image models through core distribution alignment, achieving excellent performance while avoiding the cost and circular dependency of target-specific model training.

Abstract: Prevailing Dataset Distillation (DD) methods leveraging generative models confront two fundamental limitations. First, despite pioneering the use of diffusion models in DD and delivering impressive performance, the vast majority of approaches paradoxically require a diffusion model pre-trained on the full target dataset, undermining the very purpose of DD and incurring prohibitive training costs. Second, although some methods turn to general text-to-image models without relying on such target-specific training, they suffer from a significant distributional mismatch, as the web-scale priors encapsulated in these foundation models fail to faithfully capture the target-specific semantics, leading to suboptimal performance. To tackle these challenges, we propose Core Distribution Alignment (CoDA), a framework that enables effective DD using only an off-the-shelf text-to-image model. Our key idea is to first identify the “intrinsic core distribution” of the target dataset using a robust density-based discovery mechanism. We then steer the generative process to align the generated samples with this core distribution. By doing so, CoDA effectively bridges the gap between general-purpose generative priors and target semantics, yielding highly representative distilled datasets. Extensive experiments suggest that, without relying on a generative model specifically trained on the target dataset, CoDA achieves performance on par with or even superior to previous methods with such reliance across all benchmarks, including ImageNet-1K and its subsets. Notably, it establishes a new state-of-the-art accuracy of 60.4% at the 50-images-per-class (IPC) setup on ImageNet-1K. Our code is available on the project webpage: https://github.com/zzzlt422/CoDA

Method: The paper proposes DIG: 1) Identifies query typology distinguishing global vs. localized queries; 2) Uses uniform sampling for global queries (efficient and effective); 3) Activates specialized pipeline for query-relevant frame extraction for localized queries; 4) Training-free framework that adapts strategy based on query type.

Result: DIG consistently outperforms existing baselines on three long-form video understanding benchmarks. It robustly improves LMM performance even when scaling input frames to 256, demonstrating both effectiveness and efficiency.

Conclusion: Complex query-aware frame selection is not universally necessary - query type matters. DIG’s adaptive approach (uniform sampling for global queries, targeted selection for localized queries) provides an efficient, effective solution for long-form video understanding with LMMs.

Abstract: The application of Large Multimodal Models (LMMs) to long-form video understanding is constrained by limited context lengths and the computationally prohibitive cost of processing dense video tokens. Consequently, recent research has focused on query-aware frame selection, methods that often incur significant computational overhead. This paper challenges the assumption that such complex search mechanisms are universally necessary. We first identify and validate a query typology distinguishing between global query and localized query. We demonstrate that while uniform sampling is both effective and efficient for global queries, localized queries indeed necessitate query-aware selection for optimal performance. Building on this insight, we propose DIG, a training-free frame selection framework that adapts its strategy based on the query type. Specifically,DIG employs efficient uniform sampling for global queries while activating a specialized pipeline to extract query-relevant frames for localized queries. Experiments on three long-form video understanding benchmarks demonstrate that DIG consistently outperforms existing baselines and robustly improves LMM performance, even when scaling the input frame count to 256.

Method: Proposes Frequency-Aware Mamba (FAMamba) with two key components: (1) Dual-Branch Feature Extraction Block (DFEB) for local-global interaction via bidirectional 2D frequency-adaptive scanning, and (2) Prior-Guided Block (PGB) for texture refinement through wavelet-based high-frequency residual learning. Also introduces Adaptive Frequency Scanning Mechanism (AFSM) for frequency-domain scanning across subgraphs.

Result: Extensive experiments demonstrate the efficiency and effectiveness of FAMamba for traffic image restoration under adverse weather conditions.

Conclusion: FAMamba successfully integrates frequency guidance with sequence modeling, enabling high-quality image reconstruction with precise details by leveraging frequency-domain priors and adaptive scanning mechanisms.

Abstract: Traffic image restoration under adverse weather conditions remains a critical challenge for intelligent transportation systems. Existing methods primarily focus on spatial-domain modeling but neglect frequency-domain priors. Although the emerging Mamba architecture excels at long-range dependency modeling through patch-wise correlation analysis, its potential for frequency-domain feature extraction remains unexplored. To address this, we propose Frequency-Aware Mamba (FAMamba), a novel framework that integrates frequency guidance with sequence modeling for efficient image restoration. Our architecture consists of two key components: (1) a Dual-Branch Feature Extraction Block (DFEB) that enhances local-global interaction via bidirectional 2D frequency-adaptive scanning, dynamically adjusting traversal paths based on sub-band texture distributions; and (2) a Prior-Guided Block (PGB) that refines texture details through wavelet-based high-frequency residual learning, enabling high-quality image reconstruction with precise details. Meanwhile, we design a novel Adaptive Frequency Scanning Mechanism (AFSM) for the Mamba architecture, which enables the Mamba to achieve frequency-domain scanning across distinct subgraphs, thereby fully leveraging the texture distribution characteristics inherent in subgraph structures. Extensive experiments demonstrate the efficiency and effectiveness of FAMamba.

Method: The study compares different approaches to modeling sketches as sequences, examining traditional positional encodings, absolute vs. relative coordinates, and autoregressive vs. non-autoregressive decoders.

Result: Absolute coordinates consistently outperform relative ones for positional encodings, non-autoregressive decoders beat autoregressive ones, and temporal importance depends on both the specific order considered and the evaluation task.

Conclusion: While treating sketches as sequences is valid, the choice of temporal modeling approach matters significantly - absolute coordinates and non-autoregressive methods work better, and temporal importance is task-dependent.

Abstract: Sketches are simple human hand-drawn abstractions of complex scenes and real-world objects. Although the field of sketch representation learning has advanced significantly, there is still a gap in understanding the true relevance of the temporal aspect to the quality of these representations. This work investigates whether it is indeed justifiable to treat sketches as sequences, as well as which internal orders play a more relevant role. The results indicate that, although the use of traditional positional encodings is valid for modeling sketches as sequences, absolute coordinates consistently outperform relative ones. Furthermore, non-autoregressive decoders outperform their autoregressive counterparts. Finally, the importance of temporality was shown to depend on both the order considered and the task evaluated.

Method: The authors collected 339 whole-slide images of prostate core needle biopsies from 185 consecutive patients in Erbil, Iraq, with Gleason scores and ISUP grades assigned by three independent pathologists, scanned using three different scanners (Leica, Hamamatsu, Grundium).

Result: The dataset enables grading concordance analyses, color normalization studies, and cross-scanner robustness evaluations, and will be publicly available in the Bioimage Archive under CC BY 4.0 license.

Conclusion: This publicly released dataset from a Middle Eastern population supports the development and validation of pathology AI models across globally diverse populations, addressing geographic bias in existing datasets.

Abstract: Artificial intelligence (AI) is increasingly used in digital pathology. Publicly available histopathology datasets remain scarce, and those that do exist predominantly represent Western populations. Consequently, the generalizability of AI models to populations from less digitized regions, such as the Middle East, is largely unknown. This motivates the public release of our dataset to support the development and validation of pathology AI models across globally diverse populations. We present 339 whole-slide images of prostate core needle biopsies from a consecutive series of 185 patients collected in Erbil, Iraq. The slides are associated with Gleason scores and International Society of Urological Pathology grades assigned independently by three pathologists. Scanning was performed using two high-throughput scanners (Leica and Hamamatsu) and one compact scanner (Grundium). All slides were de-identified and are provided in their native formats without further conversion. The dataset enables grading concordance analyses, color normalization, and cross-scanner robustness evaluations. Data will be deposited in the Bioimage Archive (BIA) under accession code: to be announced (TBA), and released under a CC BY 4.0 license.

Method: Experimented with three distinct pre-training, intermediate fine-tuning, and downstream datasets and training objectives on a small 5M-parameter vision transformer to explore their marginal benefits.

Result: Pre-training and fine-tuning always help but have diminishing returns; intermediate fine-tuning can actually harm downstream performance due to dissimilarity in task mechanics.

Conclusion: Small-scale ViTs benefit most from targeted pre-training and careful data selection, while indiscriminate stacking of intermediate tasks can waste compute and degrade performance.

Abstract: While transformer-based architectures have taken computer vision and NLP by storm, they often require a vast amount of parameters and training data to attain strong performance. In this work, we experiment with three distinct pre-training, intermediate fine-tuning, and downstream datasets and training objectives to explore their marginal benefits on a small 5M-parameter vision transformer. We find that while pre-training and fine-tuning always help our model but have diminishing returns, intermediate fine-tuning can actually show harmful impact on downstream performance, potentially due to dissimilarity in task mechanics. Taken together, our results suggest that small-scale ViTs benefit most from targeted pre-training and careful data selection, while indiscriminate stacking of intermediate tasks can waste compute and even degrade performance.

Method: PSA replaces binary masking with multi-level pooled KV representations where each query block dynamically allocates lower pooling levels to critical KV blocks and higher levels to less important ones, creating an informative interpolation between full retention and complete pruning.

Result: PSA consistently outperforms or achieves comparable performance to existing sparse attention baselines across video understanding and generation benchmarks, preserving contextual information and visual fidelity with superior efficiency-quality trade-offs.

Conclusion: PSA effectively mitigates information loss in sparse attention mechanisms while preserving computational efficiency, offering a hardware-friendly solution for scaling foundation models in video applications.

Abstract: Attention mechanisms are the core of foundation models, but their quadratic complexity remains a critical bottleneck for scaling. This challenge has driven the development of efficient attention mechanisms, with sparsity emerging as the dominant paradigm. Current methods typically retain or discard entire key-value blocks with binary masks, resulting in substantial information loss under high sparsity. To mitigate this gap, we present Pyramid Sparse Attention (PSA), a versatile module applicable to both video understanding and generation tasks. Instead of binary masking, PSA introduces multi-level pooled KV representations, enabling finer mask granularity. Specifically, each query block dynamically allocates lower pooling levels to critical KV blocks and higher levels to less important ones, creating an informative interpolation between full retention and complete pruning. This design, analogous to fixed-point quantization and classical feature pyramid networks in computer vision, effectively mitigates information loss while preserving computational efficiency under a low compute budget. It works with a native, hardware-friendly kernel that leverages decoupled block-tile design to ensure efficient execution. Across video understanding and generation benchmarks, PSA preserves contextual information and visual fidelity, consistently outperforming or achieving comparable performance over existing sparse attention baselines with superior efficiency-quality trade-offs. Our code and model weights are publicly available at: http://ziplab.co/PSA

Method: VLSU uses a multi-stage pipeline with real-world images and human annotation to construct a large-scale benchmark of 8,187 samples spanning 15 harm categories and 17 safety patterns. It evaluates multimodal safety through fine-grained severity classification and combinatorial analysis across 11 state-of-the-art models.

Result: Models achieve 90%+ accuracy on clear unimodal safety signals but degrade to 20-55% when joint image-text reasoning is required. 34% of joint classification errors occur despite correct classification of individual modalities, showing absent compositional reasoning. Models struggle to balance refusal of unsafe content while engaging with borderline cases.

Conclusion: The VLSU framework exposes weaknesses in joint image-text understanding and alignment gaps in current models, providing a critical test bed for research on robust vision-language safety and highlighting the need for improved compositional reasoning capabilities.

Abstract: Safety evaluation of multimodal foundation models often treats vision and language inputs separately, missing risks from joint interpretation where benign content becomes harmful in combination. Existing approaches also fail to distinguish clearly unsafe content from borderline cases, leading to problematic over-blocking or under-refusal of genuinely harmful content. We present Vision Language Safety Understanding (VLSU), a comprehensive framework to systematically evaluate multimodal safety through fine-grained severity classification and combinatorial analysis across 17 distinct safety patterns. Using a multi-stage pipeline with real-world images and human annotation, we construct a large-scale benchmark of 8,187 samples spanning 15 harm categories. Our evaluation of eleven state-of-the-art models reveals systematic joint understanding failures: while models achieve 90%-plus accuracy on clear unimodal safety signals, performance degrades substantially to 20-55% when joint image-text reasoning is required to determine the safety label. Most critically, 34% of errors in joint image-text safety classification occur despite correct classification of the individual modalities, further demonstrating absent compositional reasoning capabilities. Additionally, we find that models struggle to balance refusing unsafe content while still responding to borderline cases that deserve engagement. For example, we find that instruction framing can reduce the over-blocking rate on borderline content from 62.4% to 10.4% in Gemini-1.5, but only at the cost of under-refusing on unsafe content with refusal rate dropping from 90.8% to 53.9%. Overall, our framework exposes weaknesses in joint image-text understanding and alignment gaps in current models, and provides a critical test bed to enable the next milestones in research on robust vision-language safety.

Method: The framework uses skeletonization to create graph representations of vessels, integrates atlas-based regional parcellation, extracts centerlines, constructs graphs, and computes 15 morphometric, topological, fractal, and geometric features at global and regional scales.

Result: Applied to 570 3D TOF-MRA scans from the IXI dataset (ages 20-86), the framework produced reproducible vessel graphs showing age- and sex-related variations, and education-associated increases in vascular complexity consistent with literature findings.

Conclusion: CaravelMetrics provides a scalable, fully automated approach for quantitative cerebrovascular feature extraction that supports normative modeling and population-level studies of vascular health and aging.

Abstract: We present CaravelMetrics, a computational framework for automated cerebrovascular analysis that models vessel morphology through skeletonization-derived graph representations. The framework integrates atlas-based regional parcellation, centerline extraction, and graph construction to compute fifteen morphometric, topological, fractal, and geometric features. The features can be estimated globally from the complete vascular network or regionally within arterial territories, enabling multiscale characterization of cerebrovascular organization. Applied to 570 3D TOF-MRA scans from the IXI dataset (ages 20-86), CaravelMetrics yields reproducible vessel graphs capturing age- and sex-related variations and education-associated increases in vascular complexity, consistent with findings reported in the literature. The framework provides a scalable and fully automated approach for quantitative cerebrovascular feature extraction, supporting normative modeling and population-level studies of vascular health and aging.

Method: Developed SSDCA (Siamese Swin Transformer with Dual Cross-Attention) that combines longitudinal endoscopic images at restaging and follow-up without requiring spatial alignment, using pretrained Swin transformers for domain-agnostic features and dual cross-attention to emphasize features from both scans.

Result: SSDCA achieved best balanced accuracy (81.76% ± 0.04), sensitivity (90.07% ± 0.08), and specificity (72.86% ± 0.05) on 62-patient test set. Showed robustness to artifacts and achieved maximal inter-cluster separation (1.45 ± 0.18) and minimal intra-cluster dispersion (1.07 ± 0.19) in feature space.

Conclusion: SSDCA provides an accurate, robust method for early detection of local regrowth in rectal cancer patients during watch-and-wait surveillance, enabling better clinical management and potentially preventing distant metastases.

Abstract: Increasing evidence supports watch-and-wait (WW) surveillance for patients with rectal cancer who show clinical complete response (cCR) at restaging following total neoadjuvant treatment (TNT). However, objectively accurate methods to early detect local regrowth (LR) from follow-up endoscopy images during WW are essential to manage care and prevent distant metastases. Hence, we developed a Siamese Swin Transformer with Dual Cross-Attention (SSDCA) to combine longitudinal endoscopic images at restaging and follow-up and distinguish cCR from LR. SSDCA leverages pretrained Swin transformers to extract domain agnostic features and enhance robustness to imaging variations. Dual cross attention is implemented to emphasize features from the two scans without requiring any spatial alignment of images to predict response. SSDCA as well as Swin-based baselines were trained using image pairs from 135 patients and evaluated on a held-out set of image pairs from 62 patients. SSDCA produced the best balanced accuracy (81.76% $\pm$ 0.04), sensitivity (90.07% $\pm$ 0.08), and specificity (72.86% $\pm$ 0.05). Robustness analysis showed stable performance irrespective of artifacts including blood, stool, telangiectasia, and poor image quality. UMAP clustering of extracted features showed maximal inter-cluster separation (1.45 $\pm$ 0.18) and minimal intra-cluster dispersion (1.07 $\pm$ 0.19) with SSDCA, confirming discriminative representation learning.

Method: Part 1: Six architectural innovations for normalizing flows including invertible 3x3 convolutions with mathematical proofs, Quad-coupling layers, parallel inversion algorithms, efficient backpropagation for inverse convolutions, Inverse-Flow training, and Affine-StableSR for super-resolution. Part 2: Application of various generative models including Conditional GANs for agricultural assessment, stacked autoencoders for geological mapping, face detection/inpainting for privacy preservation, and adapted diffusion models for art restoration.

Result: Developed more efficient normalizing flow architectures with proven invertibility conditions and faster algorithms. Applied generative models successfully to real-world problems: achieved good accuracy in seed purity testing, improved feature extraction for geological mapping, effective privacy preservation for autonomous driving datasets, and handled multiple degradation types in art restoration.

Conclusion: The thesis demonstrates both theoretical advancements in normalizing flow efficiency and practical applications of generative models to diverse computer vision challenges, showing the versatility and effectiveness of these approaches across agriculture, geology, privacy preservation, and cultural heritage domains.

Abstract: This thesis presents novel contributions in two primary areas: advancing the efficiency of generative models, particularly normalizing flows, and applying generative models to solve real-world computer vision challenges. The first part introduce significant improvements to normalizing flow architectures through six key innovations: 1) Development of invertible 3x3 Convolution layers with mathematically proven necessary and sufficient conditions for invertibility, (2) introduction of a more efficient Quad-coupling layer, 3) Design of a fast and efficient parallel inversion algorithm for kxk convolutional layers, 4) Fast & efficient backpropagation algorithm for inverse of convolution, 5) Using inverse of convolution, in Inverse-Flow, for the forward pass and training it using proposed backpropagation algorithm, and 6) Affine-StableSR, a compact and efficient super-resolution model that leverages pre-trained weights and Normalizing Flow layers to reduce parameter count while maintaining performance. The second part: 1) An automated quality assessment system for agricultural produce using Conditional GANs to address class imbalance, data scarcity and annotation challenges, achieving good accuracy in seed purity testing; 2) An unsupervised geological mapping framework utilizing stacked autoencoders for dimensionality reduction, showing improved feature extraction compared to conventional methods; 3) We proposed a privacy preserving method for autonomous driving datasets using on face detection and image inpainting; 4) Utilizing Stable Diffusion based image inpainting for replacing the detected face and license plate to advancing privacy-preserving techniques and ethical considerations in the field.; and 5) An adapted diffusion model for art restoration that effectively handles multiple types of degradation through unified fine-tuning.

Method: FRESCO integrates intra-frame correspondence with inter-frame correspondence to create a more robust spatial-temporal constraint, ensuring consistent transformation of semantically similar content between frames. It goes beyond attention guidance to explicitly optimize features.

Result: Comprehensive experiments show FRESCO generates high-quality, coherent videos with significantly improved visual coherence compared to current zero-shot methods, verified on video-to-video translation and text-guided video editing tasks.

Conclusion: FRESCO represents a significant advance over current zero-shot video adaptation methods by achieving high spatial-temporal consistency through integrated intra-frame and inter-frame correspondence constraints.

Abstract: The remarkable success in text-to-image diffusion models has motivated extensive investigation of their potential for video applications. Zero-shot techniques aim to adapt image diffusion models for videos without requiring further model training. Recent methods largely emphasize integrating inter-frame correspondence into attention mechanisms. However, the soft constraint applied to identify the valid features to attend is insufficient, which could lead to temporal inconsistency. In this paper, we present FRESCO, which integrates intra-frame correspondence with inter-frame correspondence to formulate a more robust spatial-temporal constraint. This enhancement ensures a consistent transformation of semantically similar content between frames. Our method goes beyond attention guidance to explicitly optimize features, achieving high spatial-temporal consistency with the input video, significantly enhancing the visual coherence of manipulated videos. We verify FRESCO adaptations on two zero-shot tasks of video-to-video translation and text-guided video editing. Comprehensive experiments demonstrate the effectiveness of our framework in generating high-quality, coherent videos, highlighting a significant advance over current zero-shot methods.

Method: Models videos and 3D motions as unified token sequences using separate embedding layers, employs sequence modeling strategy integrating two tasks, and designs novel 3D motion tokenizer with temporal expansion strategy using single VQ-VAE with multiple expert decoders for body shapes, translation, orientation, and poses.

Result: Extensive experiments demonstrate simultaneous generation of corresponding videos and motions while performing accurate motion capture, proving effectiveness of unified modeling approach.

Conclusion: This work taps into LLMs’ capacity to fuse diverse data types, paving way for integrating human-centric information into existing models and potentially enabling multimodal, controllable joint modeling of humans, objects, and scenes.

Abstract: We propose UniMo, an innovative autoregressive model for joint modeling of 2D human videos and 3D human motions within a unified framework, enabling simultaneous generation and understanding of these two modalities for the first time. Current methods predominantly focus on generating one modality given another as the condition or integrating either of them with other modalities such as text and audio. Unifying 2D videos and 3D motions for simultaneous optimization and generation remains largely unexplored, presenting significant challenges due to their substantial structural and distributional differences. Inspired by the LLM’s ability to unify different modalities, our method models videos and 3D motions as a unified tokens sequence, utilizing separate embedding layers to mitigate distribution gaps. Additionally, we devise a sequence modeling strategy that integrates two distinct tasks within a single framework, proving the effectiveness of unified modeling. Moreover, to efficiently align with visual tokens and preserve 3D spatial information, we design a novel 3D motion tokenizer with a temporal expansion strategy, using a single VQ-VAE to produce quantized motion tokens. It features multiple expert decoders that handle body shapes, translation, global orientation, and body poses for reliable 3D motion reconstruction. Extensive experiments demonstrate that our method simultaneously generates corresponding videos and motions while performing accurate motion capture. This work taps into the capacity of LLMs to fuse diverse data types, paving the way for integrating human-centric information into existing models and potentially enabling multimodal, controllable joint modeling of humans, objects, and scenes.

Method: Proposes a framework with: 1) Adaptive frequency masks learned by a conditional mask generator to optimally fuse frequency components from original ground truth and super-resolved variants, 2) Frequency-domain mixup to preserve semantic consistency while enriching perceptual details, 3) A lightweight output refinement network trained on enhanced ground truth that integrates with existing restoration models.

Result: Extensive experiments show consistent improvement in restored image quality. User studies validate effectiveness of both supervision enhancement and output refinement components.

Conclusion: The proposed framework successfully addresses limitations of imperfect ground-truth data in real-world image restoration by enhancing supervision quality through adaptive frequency fusion, leading to improved restoration performance without hallucinated artifacts.

Abstract: Deep learning-based image restoration has achieved significant success. However, when addressing real-world degradations, model performance is limited by the quality of ground-truth images in datasets due to practical constraints in data acquisition. To address this limitation, we propose a novel framework that enhances existing ground truth images to provide higher-quality supervision for real-world restoration. Our framework generates perceptually enhanced ground truth images using super-resolution by incorporating adaptive frequency masks, which are learned by a conditional frequency mask generator. These masks guide the optimal fusion of frequency components from the original ground truth and its super-resolved variants, yielding enhanced ground truth images. This frequency-domain mixup preserves the semantic consistency of the original content while selectively enriching perceptual details, preventing hallucinated artifacts that could compromise fidelity. The enhanced ground truth images are used to train a lightweight output refinement network that can be seamlessly integrated with existing restoration models. Extensive experiments demonstrate that our approach consistently improves the quality of restored images. We further validate the effectiveness of both supervision enhancement and output refinement through user studies. Code is available at https://github.com/dhryougit/Beyond-the-Ground-Truth.

Method: Analyzed self-attention maps across transformer layers and found dynamic regions are naturally down-weighted. Introduced MUT3R with an attention-level gating module that applies these attention-derived motion cues to suppress dynamic content in early transformer layers during inference, without retraining or fine-tuning.

Result: The framework stabilizes geometric reasoning in streaming scenarios, leading to improvements in temporal consistency and camera pose robustness across multiple dynamic benchmarks.

Conclusion: MUT3R offers a simple, training-free pathway toward motion-aware streaming reconstruction by letting pretrained transformers diagnose their own motion cues and self-correct, addressing motion artifacts without additional training.

Abstract: Recent stateful recurrent neural networks have achieved remarkable progress on static 3D reconstruction but remain vulnerable to motion-induced artifacts, where non-rigid regions corrupt attention propagation between the spatial memory and image feature. By analyzing the internal behaviors of the state and image token updating mechanism, we find that aggregating self-attention maps across layers reveals a consistent pattern: dynamic regions are naturally down-weighted, exposing an implicit motion cue that the pretrained transformer already encodes but never explicitly uses. Motivated by this observation, we introduce MUT3R, a training-free framework that applies the attention-derived motion cue to suppress dynamic content in the early layers of the transformer during inference. Our attention-level gating module suppresses the influence of dynamic regions before their artifacts propagate through the feature hierarchy. Notably, we do not retrain or fine-tune the model; we let the pretrained transformer diagnose its own motion cues and correct itself. This early regulation stabilizes geometric reasoning in streaming scenarios and leads to improvements in temporal consistency and camera pose robustness across multiple dynamic benchmarks, offering a simple and training-free pathway toward motion-aware streaming reconstruction.

Method: TempR1 uses a multi-task corpus with diverse temporal structures, builds on GRPO algorithm, categorizes tasks into three correspondence types between predicted intervals and ground-truth, and designs tailored localization rewards for each type.

Result: TempR1 achieves state-of-the-art performance across multiple benchmarks, with joint optimization over complementary tasks yielding strong synergistic effects that enhance both generalization and single-task performance.

Conclusion: TempR1 establishes a scalable and principled paradigm for temporal reasoning in MLLMs, demonstrating effective cross-task optimization and improved temporal comprehension through systematic multi-task reinforcement learning.

Abstract: Enhancing the temporal understanding of Multimodal Large Language Models (MLLMs) is essential for advancing long-form video analysis, enabling tasks such as temporal localization, action detection, and time-sensitive question answering. While reinforcement learning (RL) has recently been explored for improving temporal reasoning, existing approaches are often confined to limited task types and data, restricting their generalization across diverse temporal understanding scenarios. To address this challenge, we present TempR1, a temporal-aware multi-task reinforcement learning framework that systematically strengthens MLLMs’ temporal comprehension. We curate a multi-task corpus that exposes the model to diverse temporal structures and semantics, and build upon the Group Relative Policy Optimization (GRPO) algorithm to achieve stable and effective cross-task optimization. Specifically, we categorize temporal tasks into three correspondence types between predicted intervals and ground-truth instances, and design tailored localization rewards for each, enabling TempR1 to capture fine-grained temporal dependencies and adapt to different temporal patterns. Extensive experiments demonstrate that TempR1 attains state-of-the-art performance across multiple benchmarks. Moreover, its joint optimization over complementary tasks yields a strong synergistic effect, enhancing both generalization and single-task performance, establishing a scalable and principled paradigm for temporal reasoning in MLLMs.

Method: UniID integrates both paradigms through an identity-focused learning scheme during training and a normalized rescaling mechanism at inference, allowing identity-relevant information to reinforce while preserving diffusion prior for non-identity attributes.

Result: Extensive experiments against six state-of-the-art methods show UniID achieves superior performance in both identity preservation and text controllability.

Conclusion: UniID provides a unified tuning-free framework that successfully balances identity fidelity and text controllability in face personalization through principled integration of complementary approaches.

Abstract: Tuning-free face personalization methods have developed along two distinct paradigms: text embedding approaches that map facial features into the text embedding space, and adapter-based methods that inject features through auxiliary cross-attention layers. While both paradigms have shown promise, existing methods struggle to simultaneously achieve high identity fidelity and flexible text controllability. We introduce UniID, a unified tuning-free framework that synergistically integrates both paradigms. Our key insight is that when merging these approaches, they should mutually reinforce only identity-relevant information while preserving the original diffusion prior for non-identity attributes. We realize this through a principled training-inference strategy: during training, we employ an identity-focused learning scheme that guides both branches to capture identity features exclusively; at inference, we introduce a normalized rescaling mechanism that recovers the text controllability of the base diffusion model while enabling complementary identity signals to enhance each other. This principled design enables UniID to achieve high-fidelity face personalization with flexible text controllability. Extensive experiments against six state-of-the-art methods demonstrate that UniID achieves superior performance in both identity preservation and text controllability. Code will be available at https://github.com/lyuPang/UniID

Method: DirectDrag introduces two key innovations: 1) Auto Soft Mask Generation module that intelligently infers editable regions from point displacement, automatically localizing deformation along movement paths while preserving contextual integrity; 2) Readout-Guided Feature Alignment mechanism that leverages intermediate diffusion activations to maintain structural consistency during point-based edits.

Result: Despite operating without manual mask or prompt, DirectDrag achieves superior image quality compared to existing methods while maintaining competitive drag accuracy. Extensive experiments on DragBench and real-world scenarios demonstrate effectiveness and practicality.

Conclusion: DirectDrag provides a novel mask- and prompt-free editing framework that enables precise and efficient manipulation with minimal user input while maintaining high image fidelity and accurate point alignment, advancing interactive image manipulation capabilities.

Abstract: Drag-based image editing using generative models provides intuitive control over image structures. However, existing methods rely heavily on manually provided masks and textual prompts to preserve semantic fidelity and motion precision. Removing these constraints creates a fundamental trade-off: visual artifacts without masks and poor spatial control without prompts. To address these limitations, we propose DirectDrag, a novel mask- and prompt-free editing framework. DirectDrag enables precise and efficient manipulation with minimal user input while maintaining high image fidelity and accurate point alignment. DirectDrag introduces two key innovations. First, we design an Auto Soft Mask Generation module that intelligently infers editable regions from point displacement, automatically localizing deformation along movement paths while preserving contextual integrity through the generative model’s inherent capacity. Second, we develop a Readout-Guided Feature Alignment mechanism that leverages intermediate diffusion activations to maintain structural consistency during point-based edits, substantially improving visual fidelity. Despite operating without manual mask or prompt, DirectDrag achieves superior image quality compared to existing methods while maintaining competitive drag accuracy. Extensive experiments on DragBench and real-world scenarios demonstrate the effectiveness and practicality of DirectDrag for high-quality, interactive image manipulation. Project Page: https://frakw.github.io/DirectDrag/. Code is available at: https://github.com/frakw/DirectDrag.

Method: Discovered that VGGT can inherently distinguish distractor images. Identified a specific layer with outlier-suppressing behavior, whose discriminative internal representations enable noise-filtering. Leveraged this layer for outlier-view rejection without additional fine-tuning or supervision.

Result: Extensive experiments on controlled and in-the-wild datasets show the implicit filtering mechanism is consistent and generalizes well across diverse scenarios.

Conclusion: Feed-forward 3D reconstruction models possess inherent noise-filtering capabilities through specific internal representations, enabling effective outlier rejection without explicit mechanisms or additional training.

Abstract: Reliable 3D reconstruction from in-the-wild image collections is often hindered by “noisy” images-irrelevant inputs with little or no view overlap with others. While traditional Structure-from-Motion pipelines handle such cases through geometric verification and outlier rejection, feed-forward 3D reconstruction models lack these explicit mechanisms, leading to degraded performance under in-the-wild conditions. In this paper, we discover that the existing feed-forward reconstruction model, e.g., VGGT, despite lacking explicit outlier-rejection mechanisms or noise-aware training, can inherently distinguish distractor images. Through an in-depth analysis under varying proportions of synthetic distractors, we identify a specific layer that naturally exhibits outlier-suppressing behavior. Further probing reveals that this layer encodes discriminative internal representations that enable an effective noise-filtering capability, which we simply leverage to perform outlier-view rejection in feed-forward 3D reconstruction without any additional fine-tuning or supervision. Extensive experiments on both controlled and in-the-wild datasets demonstrate that this implicit filtering mechanism is consistent and generalizes well across diverse scenarios.

Method: End-to-end framework using learnable binary masks with straight-through estimation to dynamically partition latent representations into transformation-sensitive and invariant components, formulated within a unified optimization framework that jointly learns latent disentanglement and group transformation mappings.

Result: Validated on five 2D/3D image datasets, demonstrating ability to automatically learn disentangled latent factors for group actions in diverse data, with downstream classification tasks confirming effectiveness of learned representations.

Conclusion: The framework successfully learns group actions on latent image manifolds without manual intervention, can be integrated with any standard encoder-decoder architecture, and provides effective disentangled representations for transformation modeling.

Abstract: Modeling group actions on latent representations enables controllable transformations of high-dimensional image data. Prior works applying group-theoretic priors or modeling transformations typically operate in the high-dimensional data space, where group actions apply uniformly across the entire input, making it difficult to disentangle the subspace that varies under transformations. While latent-space methods offer greater flexibility, they still require manual partitioning of latent variables into equivariant and invariant subspaces, limiting the ability to robustly learn and operate group actions within the representation space. To address this, we introduce a novel end-to-end framework that for the first time learns group actions on latent image manifolds, automatically discovering transformation-relevant structures without manual intervention. Our method uses learnable binary masks with straight-through estimation to dynamically partition latent representations into transformation-sensitive and invariant components. We formulate this within a unified optimization framework that jointly learns latent disentanglement and group transformation mappings. The framework can be seamlessly integrated with any standard encoder-decoder architecture. We validate our approach on five 2D/3D image datasets, demonstrating its ability to automatically learn disentangled latent factors for group actions in diverse data, while downstream classification tasks confirm the effectiveness of the learned representations. Our code is publicly available at https://github.com/farhanaswarnali/Learning-Group-Actions-In-Disentangled-Latent-Image-Representations .

Method: C3G uses learnable tokens that aggregate multi-view features through self-attention to guide Gaussian generation, ensuring each Gaussian integrates relevant visual features across views. It then exploits learned attention patterns for Gaussian decoding to efficiently lift features.

Result: Extensive experiments on pose-free novel view synthesis, 3D open-vocabulary segmentation, and view-invariant feature aggregation demonstrate superior memory efficiency and feature fidelity compared to existing methods.

Conclusion: A compact yet geometrically meaningful representation is sufficient for high-quality scene reconstruction and understanding, achieving better performance with reduced memory requirements.

Abstract: Reconstructing and understanding 3D scenes from unposed sparse views in a feed-forward manner remains as a challenging task in 3D computer vision. Recent approaches use per-pixel 3D Gaussian Splatting for reconstruction, followed by a 2D-to-3D feature lifting stage for scene understanding. However, they generate excessive redundant Gaussians, causing high memory overhead and sub-optimal multi-view feature aggregation, leading to degraded novel view synthesis and scene understanding performance. We propose C3G, a novel feed-forward framework that estimates compact 3D Gaussians only at essential spatial locations, minimizing redundancy while enabling effective feature lifting. We introduce learnable tokens that aggregate multi-view features through self-attention to guide Gaussian generation, ensuring each Gaussian integrates relevant visual features across views. We then exploit the learned attention patterns for Gaussian decoding to efficiently lift features. Extensive experiments on pose-free novel view synthesis, 3D open-vocabulary segmentation, and view-invariant feature aggregation demonstrate our approach’s effectiveness. Results show that a compact yet geometrically meaningful representation is sufficient for high-quality scene reconstruction and understanding, achieving superior memory efficiency and feature fidelity compared to existing methods.

Method: Built on autoregressive video-diffusion distillation, RELIC uses compressed historical latent tokens with relative actions and absolute camera poses in KV cache for memory. It fine-tunes a bidirectional teacher video model to generate beyond its 5-second horizon, then transforms it into a causal student generator using memory-efficient self-forcing for full-context distillation over long durations.

Result: RELIC (14B parameters) achieves real-time generation at 16 FPS with more accurate action following, more stable long-horizon streaming, and more robust spatial-memory retrieval compared to prior work.

Conclusion: RELIC establishes a strong foundation for next-generation interactive world modeling by simultaneously addressing the three key challenges of real-time long-horizon streaming, consistent spatial memory, and precise user control.

Abstract: A truly interactive world model requires three key ingredients: real-time long-horizon streaming, consistent spatial memory, and precise user control. However, most existing approaches address only one of these aspects in isolation, as achieving all three simultaneously is highly challenging-for example, long-term memory mechanisms often degrade real-time performance. In this work, we present RELIC, a unified framework that tackles these three challenges altogether. Given a single image and a text description, RELIC enables memory-aware, long-duration exploration of arbitrary scenes in real time. Built upon recent autoregressive video-diffusion distillation techniques, our model represents long-horizon memory using highly compressed historical latent tokens encoded with both relative actions and absolute camera poses within the KV cache. This compact, camera-aware memory structure supports implicit 3D-consistent content retrieval and enforces long-term coherence with minimal computational overhead. In parallel, we fine-tune a bidirectional teacher video model to generate sequences beyond its original 5-second training horizon, and transform it into a causal student generator using a new memory-efficient self-forcing paradigm that enables full-context distillation over long-duration teacher as well as long student self-rollouts. Implemented as a 14B-parameter model and trained on a curated Unreal Engine-rendered dataset, RELIC achieves real-time generation at 16 FPS while demonstrating more accurate action following, more stable long-horizon streaming, and more robust spatial-memory retrieval compared with prior work. These capabilities establish RELIC as a strong foundation for the next generation of interactive world modeling.

Method: Double Interactive Reinforcement Learning (DIRL) with two phases: teaching phase combines single-tool specialist demonstrations with frontier model traces, and exploration phase refines multi-tool coordination through continued RL.

Result: SpaceTools achieves state-of-the-art on spatial benchmarks (RoboSpatial-Home, BLINK, BOP-ASK) with +12% over SFT and +16% over RL baselines, and demonstrates reliable real-world manipulation with 7-DOF robot.

Conclusion: DIRL effectively enables VLMs to coordinate multiple tools for precise spatial reasoning, bridging the gap between qualitative visual understanding and embodied applications.

Abstract: Vision Language Models (VLMs) demonstrate strong qualitative visual understanding, but struggle with metrically precise spatial reasoning required for embodied applications. The agentic paradigm promises that VLMs can use a wide variety of tools that could augment these capabilities, such as depth estimators, segmentation models, and pose estimators. Yet it remains an open challenge how to realize this vision without solely relying on handcrafted prompting strategies or enforcing fixed, predefined tool pipelines that limit VLMs’ ability to discover optimal tool-use patterns. Reinforcement Learning could overcome this gap, but has so far been limited to reasoning with a single visual tool due to the large search space in multi-tool reasoning. We introduce Double Interactive Reinforcement Learning (DIRL), a two-phase training framework where VLMs learn to coordinate multiple tools through interactive exploration and feedback. In the teaching phase, we combine demonstrations from a single tool specialist trained via interactive RL with traces from a frontier model using all tools. In the exploration phase, the model further refines multi-tool coordination through continued RL. Our model, SpaceTools, with tool-augmented spatial reasoning ability, achieves state-of-the-art performance on spatial understanding benchmarks (RoboSpatial-Home, BLINK, BOP-ASK) and demonstrates reliable real-world manipulation using a 7-DOF robot as a tool. DIRL provides substantial improvements over the vanilla SFT (+12% on RoboSpatial) and RL (+16% on RoboSpatial) baselines. Project page: https://spacetools.github.io/.

Method: Progressive three-stage training strategy: 1) Perturbed Supervised Fine-Tuning, 2) Reinforcement Learning for Visual-Reality Alignment, and 3) Reinforcement Learning from Aesthetic Feedback. Coupled with generative models for complete workflow enabling layer-controllable iterative editing.

Result: Extensive experiments demonstrate PosterCopilot achieves geometrically accurate and aesthetically superior layouts with unprecedented controllability for professional iterative design.

Conclusion: PosterCopilot advances layout reasoning and controllable editing for professional graphic design, addressing limitations of existing LMM-based methods through geometric understanding and iterative workflow integration.

Abstract: Graphic design forms the cornerstone of modern visual communication, serving as a vital medium for promoting cultural and commercial events. Recent advances have explored automating this process using Large Multimodal Models (LMMs), yet existing methods often produce geometrically inaccurate layouts and lack the iterative, layer-specific editing required in professional workflows. To address these limitations, we present PosterCopilot, a framework that advances layout reasoning and controllable editing for professional graphic design. Specifically, we introduce a progressive three-stage training strategy that equips LMMs with geometric understanding and aesthetic reasoning for layout design, consisting of Perturbed Supervised Fine-Tuning, Reinforcement Learning for Visual-Reality Alignment, and Reinforcement Learning from Aesthetic Feedback. Furthermore, we develop a complete workflow that couples the trained LMM-based design model with generative models, enabling layer-controllable, iterative editing for precise element refinement while maintaining global visual consistency. Extensive experiments demonstrate that PosterCopilot achieves geometrically accurate and aesthetically superior layouts, offering unprecedented controllability for professional iterative design.

Method: The key innovation is fixing the variance predicted by the VAE encoder to a constant (e.g., 0.5). This simple approach allows the encoder to output a broader token distribution, enables the decoder to learn reconstruction from augmented tokens without extra noise/denoising, and simplifies the VAE evidence lower bound for stable joint training of NF with VAE.

Result: On ImageNet 256×256 generation, SimFlow achieves gFID of 2.15, outperforming previous state-of-the-art STARFlow (gFID 2.40). When integrated with REPA-E method, it achieves gFID 1.91, setting new state-of-the-art among Normalizing Flows.

Conclusion: Fixing VAE variance to a constant provides a simple yet effective solution to two major limitations in previous NF methods, enabling better reconstruction quality, simpler training pipelines, and state-of-the-art generation performance.

Abstract: Normalizing Flows (NFs) learn invertible mappings between the data and a Gaussian distribution. Prior works usually suffer from two limitations. First, they add random noise to training samples or VAE latents as data augmentation, introducing complex pipelines including extra noising and denoising steps. Second, they use a pretrained and frozen VAE encoder, resulting in suboptimal reconstruction and generation quality. In this paper, we find that the two issues can be solved in a very simple way: just fixing the variance (which would otherwise be predicted by the VAE encoder) to a constant (e.g., 0.5). On the one hand, this method allows the encoder to output a broader distribution of tokens and the decoder to learn to reconstruct clean images from the augmented token distribution, avoiding additional noise or denoising design. On the other hand, fixed variance simplifies the VAE evidence lower bound, making it stable to train an NF with a VAE jointly. On the ImageNet $256 \times 256$ generation task, our model SimFlow obtains a gFID score of 2.15, outperforming the state-of-the-art method STARFlow (gFID 2.40). Moreover, SimFlow can be seamlessly integrated with the end-to-end representation alignment (REPA-E) method and achieves an improved gFID of 1.91, setting a new state of the art among NFs.

Method: Train distinct vision models on egocentric videos from different individuals (single-life paradigm), use multiple viewpoints within each life for self-supervised learning, introduce cross-attention-based metric to quantify functional alignment between models, and evaluate transfer to downstream tasks like depth estimation.

Result: Three key findings: 1) Models trained on different lives develop highly aligned geometric understanding; 2) Single-life models learn generalizable representations that transfer effectively to unseen environments; 3) 30 hours from one week of personal data yields comparable performance to 30 hours of diverse web data.

Conclusion: The shared structure of the world leads to consistency in models trained on individual lives and provides a powerful signal for visual representation learning, establishing the viability and effectiveness of the single-life learning paradigm.

Abstract: We introduce the “single-life” learning paradigm, where we train a distinct vision model exclusively on egocentric videos captured by one individual. We leverage the multiple viewpoints naturally captured within a single life to learn a visual encoder in a self-supervised manner. Our experiments demonstrate three key findings. First, models trained independently on different lives develop a highly aligned geometric understanding. We demonstrate this by training visual encoders on distinct datasets each capturing a different life, both indoors and outdoors, as well as introducing a novel cross-attention-based metric to quantify the functional alignment of the internal representations developed by different models. Second, we show that single-life models learn generalizable geometric representations that effectively transfer to downstream tasks, such as depth estimation, in unseen environments. Third, we demonstrate that training on up to 30 hours from one week of the same person’s life leads to comparable performance to training on 30 hours of diverse web data, highlighting the strength of single-life representation learning. Overall, our results establish that the shared structure of the world, both leads to consistency in models trained on individual lives, and provides a powerful signal for visual representation learning.

Method: Partitions images into patches and distributes model layers across multiple GPUs using patch-level pipeline parallelism. Reuses one-step stale feature maps from previous diffusion steps to provide context, reducing communication costs compared to tensor parallel, sequence parallel, and DistriFusion methods.

Result: Achieves state-of-the-art performance on 8×L40 PCIe GPUs for Pixart, Stable-Diffusion 3, and Flux.1 models, with improved memory efficiency through parameter distribution across devices.

Conclusion: PipeFusion effectively addresses latency issues in DiTs inference through innovative parallelization and feature reuse, making high-resolution image generation more efficient for large models.

Abstract: This paper presents PipeFusion, an innovative parallel methodology to tackle the high latency issues associated with generating high-resolution images using diffusion transformers (DiTs) models. PipeFusion partitions images into patches and the model layers across multiple GPUs. It employs a patch-level pipeline parallel strategy to orchestrate communication and computation efficiently. By capitalizing on the high similarity between inputs from successive diffusion steps, PipeFusion reuses one-step stale feature maps to provide context for the current pipeline step. This approach notably reduces communication costs compared to existing DiTs inference parallelism, including tensor parallel, sequence parallel and DistriFusion. PipeFusion enhances memory efficiency through parameter distribution across devices, ideal for large DiTs like Flux.1. Experimental results demonstrate that PipeFusion achieves state-of-the-art performance on 8$\times$L40 PCIe GPUs for Pixart, Stable-Diffusion 3, and Flux.1 models. Our source code is available at https://github.com/xdit-project/xDiT.

Method: The paper is a comprehensive review that examines technical underpinnings including dual-encoder architectures, Transformer models, self-supervised and contrastive learning, and cross-modal integration. It analyzes unique challenges of remote sensing data and reviews key applications, datasets, and computational challenges.

Result: The review covers the current state of MLLMs in remote sensing, including their capabilities in scene description, object detection, change detection, text-to-image retrieval, image-to-text generation, and visual question answering. It identifies challenges related to computational demands, scalability, data quality, and domain adaptation.

Conclusion: The paper concludes by proposing future research directions and technological advancements needed to further enhance MLLM utility in remote sensing, addressing current limitations and expanding applications.

Abstract: Remote sensing has evolved from simple image acquisition to complex systems capable of integrating and processing visual and textual data. This review examines the development and application of multi-modal language models (MLLMs) in remote sensing, focusing on their ability to interpret and describe satellite imagery using natural language. We cover the technical underpinnings of MLLMs, including dual-encoder architectures, Transformer models, self-supervised and contrastive learning, and cross-modal integration. The unique challenges of remote sensing data–varying spatial resolutions, spectral richness, and temporal changes–are analyzed for their impact on MLLM performance. Key applications such as scene description, object detection, change detection, text-to-image retrieval, image-to-text generation, and visual question answering are discussed to demonstrate their relevance in environmental monitoring, urban planning, and disaster response. We review significant datasets and resources supporting the training and evaluation of these models. Challenges related to computational demands, scalability, data quality, and domain adaptation are highlighted. We conclude by proposing future research directions and technological advancements to further enhance MLLM utility in remote sensing.

Method: Proposes using weakly supervised strategies to train FER models with original ambiguous annotations rather than converting them to one-hot labels.

Result: Not specified in the abstract, but the paper argues that weakly supervised training with ambiguous annotations is better than the conventional end-to-end supervised approach.

Conclusion: The existing training paradigm for FER should be rethought, and weakly supervised strategies using original ambiguous annotations are preferable to converting them to precise one-hot annotations.

Abstract: Due to the subjective crowdsourcing annotations and the inherent inter-class similarity of facial expressions, the real-world Facial Expression Recognition (FER) datasets usually exhibit ambiguous annotation. To simplify the learning paradigm, most previous methods convert ambiguous annotation results into precise one-hot annotations and train FER models in an end-to-end supervised manner. In this paper, we rethink the existing training paradigm and propose that it is better to use weakly supervised strategies to train FER models with original ambiguous annotation.

Method: Margin-aware preference optimization (MaPO) directly optimizes the likelihood margin between preferred and dispreferred outputs under the Bradley-Terry model without anchoring to a reference model. It transforms diverse T2I tasks into unified pairwise preference optimization.

Result: MaPO outperforms DPO and specialized methods like DreamBooth across five challenging domains: safe generation, style adaptation, cultural representation, personalization, and general preference alignment. Its advantage grows dramatically with reference mismatch severity while reducing training time by 15%.

Conclusion: MaPO emerges as a versatile and memory-efficient method for generic T2I adaptation tasks, breaking free from reference mismatch constraints and enabling more effective alignment across diverse domains.

Abstract: Modern preference alignment methods, such as DPO, rely on divergence regularization to a reference model for training stability-but this creates a fundamental problem we call “reference mismatch.” In this paper, we investigate the negative impacts of reference mismatch in aligning text-to-image (T2I) diffusion models, showing that larger reference mismatch hinders effective adaptation given the same amount of data, e.g., as when learning new artistic styles, or personalizing to specific objects. We demonstrate this phenomenon across text-to-image (T2I) diffusion models and introduce margin-aware preference optimization (MaPO), a reference-agnostic approach that breaks free from this constraint. By directly optimizing the likelihood margin between preferred and dispreferred outputs under the Bradley-Terry model without anchoring to a reference, MaPO transforms diverse T2I tasks into unified pairwise preference optimization. We validate MaPO’s versatility across five challenging domains: (1) safe generation, (2) style adaptation, (3) cultural representation, (4) personalization, and (5) general preference alignment. Our results reveal that MaPO’s advantage grows dramatically with reference mismatch severity, outperforming both DPO and specialized methods like DreamBooth while reducing training time by 15%. MaPO thus emerges as a versatile and memory-efficient method for generic T2I adaptation tasks.

Method: The survey categorizes and analyzes SAM acceleration strategies, explores core SAM techniques and model acceleration methods, and provides a unified evaluation framework across different hardware platforms.

Result: The paper offers a comprehensive taxonomy of efficient SAM approaches and provides extensive comparative evaluation of these methods on representative benchmarks, assessing both efficiency and accuracy across various hardware.

Conclusion: This first comprehensive survey of efficient SAM variants establishes a foundation for understanding acceleration techniques, provides comparative performance analysis, and identifies future research directions for deploying SAM in resource-constrained environments.

Abstract: The Segment Anything Model (SAM) is a foundational model for image segmentation tasks, known for its strong generalization across diverse applications. However, its impressive performance comes with significant computational and resource demands, making it challenging to deploy in resource-limited environments such as edge devices. To address this, a variety of SAM variants have been proposed to enhance efficiency while keeping accuracy. This survey provides the first comprehensive review of these efficient SAM variants. We begin by exploring the motivations driving this research. We then present core techniques used in SAM and model acceleration. This is followed by a detailed exploration of SAM acceleration strategies, categorized by approach, and a discussion of several future research directions. Finally, we offer a unified and extensive evaluation of these methods across various hardware, assessing their efficiency and accuracy on representative benchmarks, and providing a clear comparison of their overall performance.

Method: 1) VAE model with Projection Module to compress 4D features into six 2D feature maps (HexPlane), using Expansion & Squeeze Strategy for efficient 3D volume reconstruction. 2) DiT-based diffusion model with Padded Rollout Operation to reorganize HexPlane features for generation, supporting various conditions like trajectory, commands, inpainting, and layout.

Result: Significant improvements over SOTA: up to 12.56 mIoU gain in HexPlane fitting, up to 7.05 mIoU gain in reconstruction, 2.06x training speedup, 70.84% memory reduction. Outperforms existing methods on CarlaSC and Waymo datasets across multiple metrics.

Conclusion: DynamicCity successfully addresses the limitations of static scene generation by introducing an effective 4D occupancy generation framework that produces high-quality dynamic urban scenes with versatile conditioning capabilities, advancing the field of dynamic urban scene synthesis.

Abstract: Urban scene generation has been developing rapidly recently. However, existing methods primarily focus on generating static and single-frame scenes, overlooking the inherently dynamic nature of real-world driving environments. In this work, we introduce DynamicCity, a novel 4D occupancy generation framework capable of generating large-scale, high-quality dynamic 4D scenes with semantics. DynamicCity mainly consists of two key models. 1) A VAE model for learning HexPlane as the compact 4D representation. Instead of using naive averaging operations, DynamicCity employs a novel Projection Module to effectively compress 4D features into six 2D feature maps for HexPlane construction, which significantly enhances HexPlane fitting quality (up to 12.56 mIoU gain). Furthermore, we utilize an Expansion & Squeeze Strategy to reconstruct 3D feature volumes in parallel, which improves both network training efficiency and reconstruction accuracy than naively querying each 3D point (up to 7.05 mIoU gain, 2.06x training speedup, and 70.84% memory reduction). 2) A DiT-based diffusion model for HexPlane generation. To make HexPlane feasible for DiT generation, a Padded Rollout Operation is proposed to reorganize all six feature planes of the HexPlane as a squared 2D feature map. In particular, various conditions could be introduced in the diffusion or sampling process, supporting versatile 4D generation applications, such as trajectory- and command-driven generation, inpainting, and layout-conditioned generation. Extensive experiments on the CarlaSC and Waymo datasets demonstrate that DynamicCity significantly outperforms existing state-of-the-art 4D occupancy generation methods across multiple metrics. The code and models have been released to facilitate future research.

Method: Equip existing 3D detectors with an Online Adapter (OA) module - a prompt-driven query generator that uses visual prompts (image-based descriptions of risky objects) derived from auxiliary feedback. These prompts are maintained in a visual prompt buffer for continuous correction across frames.

Result: TTC significantly improves instant error rectification over frozen 3D detectors, even under limited labels, zero-shot settings, and adverse conditions, achieving reliable, adaptive, and versatile driving autonomy.

Conclusion: TTC enables post-deployment online rectification for autonomous driving, inspiring future research on adaptive systems that can correct errors during inference without retraining.

Abstract: This paper introduces Test-time Correction (TTC), an online 3D detection system designed to rectify test-time errors using various auxiliary feedback, aiming to enhance the safety of deployed autonomous driving systems. Unlike conventional offline 3D detectors that remain fixed during inference, TTC enables immediate online error correction without retraining, allowing autonomous vehicles to adapt to new scenarios and reduce deployment risks. To achieve this, we equip existing 3D detectors with an Online Adapter (OA) module – a prompt-driven query generator for real-time correction. At the core of OA module are visual prompts: image-based descriptions of objects of interest derived from auxiliary feedback such as mismatches with 2D detections, road descriptions, or user clicks. These visual prompts, collected from risky objects during inference, are maintained in a visual prompt buffer to enable continuous correction in future frames. By leveraging this mechanism, TTC consistently detects risky objects, achieving reliable, adaptive, and versatile driving autonomy. Extensive experiments show that TTC significantly improves instant error rectification over frozen 3D detectors, even under limited labels, zero-shot settings, and adverse conditions. We hope this work inspires future research on post-deployment online rectification systems for autonomous driving.

Method: 1) Construct high-quality dataset of 400K 3D assets with diverse visual annotations (70M+ visual samples) and multi-granularity hierarchical captions (1M+ descriptions). 2) Propose comprehensive evaluation suite with 10 metrics assessing both text-3D alignment and 3D visual quality at multiple levels.

Result: The proposed metrics exhibit significantly higher correlation with human judgment compared to existing methods. In-depth analysis of eight leading GT23D models provides critical insights into current model capabilities and shared failure modes.

Conclusion: GT23D-Bench addresses fundamental gaps in GT23D research by providing the first comprehensive benchmark with high-quality data and better evaluation metrics, facilitating rigorous and reproducible research in the field.

Abstract: Text-to-3D (T23D) generation has emerged as a crucial visual generation task, aiming at synthesizing 3D content from textual descriptions. Studies of this task are currently shifting from per-scene T23D, which requires optimization of the model for every content generated, to General T23D (GT23D), which requires only one pre-trained model to generate different content without re-optimization, for more generalized and efficient 3D generation. Despite notable advancements, GT23D is severely bottlenecked by two interconnected challenges: the lack of high-quality, large-scale training data and the prevalence of evaluation metrics that overlook intrinsic 3D properties. Existing datasets often suffer from incomplete annotations, noisy organization, and inconsistent quality, while current evaluations rely heavily on 2D image-text similarity or scoring, failing to thoroughly assess 3D geometric integrity and semantic relevance. To address these fundamental gaps, we introduce GT23D-Bench, the first comprehensive benchmark specifically designed for GT23D training and evaluation. We first construct a high-quality dataset of 400K 3D assets, featuring diverse visual annotations (70M+ visual samples) and multi-granularity hierarchical captions (1M+ descriptions) to foster robust semantic learning. Second, we propose a comprehensive evaluation suite with 10 metrics assessing both text-3D alignment and 3D visual quality at multiple levels. Crucially, we demonstrate through rigorous experiments that our proposed metrics exhibit significantly higher correlation with human judgment compared to existing methods. Our in-depth analysis of eight leading GT23D models using this benchmark provides the community with critical insights into current model capabilities and their shared failure modes. GT23D-Bench will be publicly available to facilitate rigorous and reproducible research.

Method: AugMapNet proposes latent Bird’s-Eye View (BEV) feature grid augmentation to enhance the latent BEV representation. It combines vector decoding and dense spatial supervision more effectively than existing architectures while being easy to integrate. The method benefits from extra processing on its latent BEV features.

Result: Experiments on nuScenes and Argoverse2 datasets show significant improvements up to 13.3% over StreamMapNet baseline on 60m range, with greater improvements on larger ranges. Transferability confirmed with similar improvements on SQD-MapNet baseline. Analysis confirms more structured latent space.

Conclusion: AugMapNet’s latent BEV feature grid augmentation significantly enhances vectorized map prediction performance and creates a more structured latent space, demonstrating value beyond pure performance improvement for autonomous driving infrastructure understanding.

Abstract: Autonomous driving requires understanding infrastructure elements, such as lanes and crosswalks. To navigate safely, this understanding must be derived from sensor data in real-time and needs to be represented in vectorized form. Learned Bird’s-Eye View (BEV) encoders are commonly used to combine a set of camera images from multiple views into one joint latent BEV grid. Traditionally, from this latent space, an intermediate raster map is predicted, providing dense spatial supervision but requiring post-processing into the desired vectorized form. More recent models directly derive infrastructure elements as polylines using vectorized map decoders, providing instance-level information. Our approach, Augmentation Map Network (AugMapNet), proposes latent BEV feature grid augmentation, a novel technique that significantly enhances the latent BEV representation. AugMapNet combines vector decoding and dense spatial supervision more effectively than existing architectures while remaining easy to integrate compared to other hybrid approaches. It additionally benefits from extra processing on its latent BEV features. Experiments on nuScenes and Argoverse2 datasets demonstrate significant improvements on vectorized map prediction of up to 13.3% over the StreamMapNet baseline on 60 m range and greater improvements on larger ranges. We confirm transferability by applying our method to another baseline, SQD-MapNet, and find similar improvements. A detailed analysis of the latent BEV grid confirms a more structured latent space of AugMapNet and shows the value of our novel concept beyond pure performance improvement. The code can be found at https://github.com/tmonnin/augmapnet

Method: The framework uses VFMs to extract semantically rich superpixels from 2D images, aligns them with LiDAR point clouds to generate contrastive samples, and employs VFM-assisted contrastive learning with superpoint temporal consistency and multi-source data pretraining.

Result: Achieves substantial gains over state-of-the-art methods in linear probing and fine-tuning for LiDAR-based segmentation and object detection, with superior performance demonstrated across 11 large-scale multi-sensor datasets.

Conclusion: LargeAD demonstrates adaptability, efficiency, and robustness in real-world autonomous driving scenarios by effectively leveraging vision foundation models for cross-modal 3D representation learning across diverse driving datasets.

Abstract: Recent advancements in vision foundation models (VFMs) have revolutionized visual perception in 2D, yet their potential for 3D scene understanding, particularly in autonomous driving applications, remains underexplored. In this paper, we introduce LargeAD, a versatile and scalable framework designed for large-scale 3D pretraining across diverse real-world driving datasets. Our framework leverages VFMs to extract semantically rich superpixels from 2D images, which are aligned with LiDAR point clouds to generate high-quality contrastive samples. This alignment facilitates cross-modal representation learning, enhancing the semantic consistency between 2D and 3D data. We introduce several key innovations: (i) VFM-driven superpixel generation for detailed semantic representation, (ii) a VFM-assisted contrastive learning strategy to align multimodal features, (iii) superpoint temporal consistency to maintain stable representations across time, and (iv) multi-source data pretraining to generalize across various LiDAR configurations. Our approach achieves substantial gains over state-of-the-art methods in linear probing and fine-tuning for LiDAR-based segmentation and object detection. Extensive experiments on 11 large-scale multi-sensor datasets highlight our superior performance, demonstrating adaptability, efficiency, and robustness in real-world autonomous driving scenarios.

Method: Two-component framework: (1) Post-Interaction adaptation updates model after user completes interactive refinement, (2) Mid-Interaction adaptation updates incrementally after each click. Uses Click-Centered Gaussian loss to strengthen model responsiveness to clicks and focus on user-guided clinically relevant regions. Treats post-interaction refined outputs as pseudo-ground-truth for online adaptation.

Result: Experiments on 5 fundus and 4 brain-MRI databases show consistent outperformance over existing methods under diverse distribution shifts, including unseen imaging modalities and pathologies.

Conclusion: The proposed interactive segmentation adaptation method effectively improves model performance on new data distributions in medical imaging by leveraging user inputs for online adaptation, with practical applications for deployment scenarios.

Abstract: Interactive segmentation uses real-time user inputs, such as mouse clicks, to iteratively refine model predictions. Although not originally designed to address distribution shifts, this paradigm naturally lends itself to such challenges. In medical imaging, where distribution shifts are common, interactive methods can use user inputs to guide models towards improved predictions. Moreover, once a model is deployed, user corrections can be used to adapt the network parameters to the new data distribution, mitigating distribution shift. Based on these insights, we aim to develop a practical, effective method for improving the adaptive capabilities of interactive segmentation models to new data distributions in medical imaging. Firstly, we found that strengthening the model’s responsiveness to clicks is important for the initial training process. Moreover, we show that by treating the post-interaction user-refined model output as pseudo-ground-truth, we can design a lean, practical online adaptation method that enables a model to learn effectively across sequential test images. The framework includes two components: (i) a Post-Interaction adaptation process, updating the model after the user has completed interactive refinement of an image, and (ii) a Mid-Interaction adaptation process, updating incrementally after each click. Both processes include a Click-Centered Gaussian loss that strengthens the model’s reaction to clicks and enhances focus on user-guided, clinically relevant regions. Experiments on 5 fundus and 4 brain-MRI databases show that our approach consistently outperforms existing methods under diverse distribution shifts, including unseen imaging modalities and pathologies. Code and pretrained models will be released upon publication.

Method: Proposes ActiveInitSplat framework that actively selects training images based on density and occupancy criteria of the resulting 3D scene representation, ensuring diverse viewpoints and better alignment of initialized Gaussian functions with actual 3D structure.

Result: Significant rendering performance improvement over passive GS baselines in both dense- and sparse-view settings, measured by LPIPS, SSIM, and PSNR metrics on simulated and real environments.

Conclusion: Active image selection for Gaussian splatting initialization leads to better scene coverage and 3D structure alignment, resulting in improved rendering performance compared to passive selection methods.

Abstract: Gaussian splatting (GS) along with its extensions and variants provides outstanding performance in real-time scene rendering while meeting reduced storage demands and computational efficiency. While the selection of 2D images capturing the scene of interest is crucial for the proper initialization and training of GS, hence markedly affecting the rendering performance, prior works rely on passively and typically densely selected 2D images. In contrast, this paper proposes `ActiveInitSplat’, a novel framework for active selection of training images for proper initialization and training of GS. ActiveInitSplat relies on density and occupancy criteria of the resultant 3D scene representation from the selected 2D images, to ensure that the latter are captured from diverse viewpoints leading to better scene coverage and that the initialized Gaussian functions are well aligned with the actual 3D structure. Numerical tests on well-known simulated and real environments demonstrate the merits of ActiveInitSplat resulting in significant GS rendering performance improvement over passive GS baselines in both dense- and sparse-view settings, in the widely adopted LPIPS, SSIM, and PSNR metrics.

Method: Augment standard NeRF with additional network modeling local egocentric gaze probability density conditioned on scene geometry and observer position. Use gaze probes to aggregate noisy head pose tracking data into robust probability density targets for supervision.

Result: Lightweight system enabling interactive visualization of gaze fields with decoupled observer perspective and proper gaze occlusion handling. Produces rendered scene views alongside pixel-wise salience maps representing conditional fixation probabilities.

Conclusion: NeRGs effectively represent visual attention in 3D environments, offering practical advantages over prior methods through lightweight implementation, interactive framerates, and proper handling of gaze occlusion and observer-camera decoupling.

Abstract: We introduce Neural Radiance and Gaze Fields (NeRGs), a novel approach for representing visual attention in complex environments. Much like how Neural Radiance Fields (NeRFs) perform novel view synthesis, NeRGs reconstruct gaze patterns from arbitrary viewpoints, implicitly mapping visual attention to 3D surfaces. We achieve this by augmenting a standard NeRF with an additional network that models local egocentric gaze probability density, conditioned on scene geometry and observer position. The output of a NeRG is a rendered view of the scene alongside a pixel-wise salience map representing the conditional probability that a given observer fixates on visible surfaces. Unlike prior methods, our system is lightweight and enables visualization of gaze fields at interactive framerates. Moreover, NeRGs allow the observer perspective to be decoupled from the rendering camera and correctly account for gaze occlusion due to intervening geometry. We demonstrate the effectiveness of NeRGs using head pose from skeleton tracking as a proxy for gaze, employing our proposed gaze probes to aggregate noisy rays into robust probability density targets for supervision.

Method: Developed an automated data-generation pipeline to create FragFake benchmark with AI-edited images from multiple source datasets, diverse editing models, and common edit types. Systematically studied VLMs for edited-image classification and localization, including fine-tuning approaches like Qwen2.5-VL and GRPO-based RLVR training.

Result: Pretrained VLMs (including GPT4o) perform poorly on the task, while fine-tuned models achieve high accuracy and substantially higher object precision. GRPO-based RLVR training yields modest metric gains while improving output interpretability. Ablation studies reveal effects of data balancing, training size, LoRA rank, and training domain.

Conclusion: This work establishes a foundation for multimodal content authenticity research, highlighting both the potential and limitations of cross-editor and cross-dataset generalization for edited-image detection and localization.

Abstract: Fine-grained detection and localization of localized image edits is crucial for assessing content authenticity, especially as modern diffusion models and image editors can produce highly realistic manipulations. However, this problem faces three key challenges: (1) most AIGC detectors produce only a global real-or-fake label without indicating where edits occur; (2) traditional computer vision methods for edit localization typically rely on costly pixel-level annotations; and (3) there is no large-scale, modern benchmark specifically targeting edited-image detection. To address these gaps, we develop an automated data-generation pipeline and construct FragFake, a large-scale benchmark of AI-edited images spanning multiple source datasets, diverse editing models, and several common edit types. Building on FragFake, we are the first to systematically study vision language models (VLMs) for edited-image classification and edited-region localization. Our experiments show that pretrained VLMs, including GPT4o, perform poorly on this task, whereas fine-tuned models such as Qwen2.5-VL achieve high accuracy and substantially higher object precision across all settings. We further explore GRPO-based RLVR training, which yields modest metric gains while improving the interpretability of model outputs. Ablation and transfer analyses reveal how data balancing, training size, LoRA rank, and training domain affect performance, and highlight both the potential and the limitations of cross-editor and cross-dataset generalization. We anticipate that this work will establish a solid foundation to facilitate and inspire subsequent research endeavors in the domain of multimodal content authenticity.

Method: FlowMo uses a transformer-based diffusion autoencoder with a two-stage training approach: mode-matching pre-training followed by mode-seeking post-training. It avoids convolutions, adversarial losses, spatially-aligned 2D latent codes, and distillation from other tokenizers.

Result: FlowMo achieves new state-of-the-art performance for image tokenization at multiple compression rates on ImageNet-1K reconstruction tasks.

Conclusion: FlowMo demonstrates that transformer-based diffusion autoencoders with proper two-stage training can achieve SOTA image tokenization performance without traditional components like convolutions or adversarial losses, enabling effective generative modeling on top of the tokenizer.

Abstract: Since the advent of popular visual generation frameworks like VQGAN and latent diffusion models, state-of-the-art image generation systems have generally been two-stage systems that first tokenize or compress visual data into a lower-dimensional latent space before learning a generative model. Tokenizer training typically follows a standard recipe in which images are compressed and reconstructed subject to a combination of MSE, perceptual, and adversarial losses. Diffusion autoencoders have been proposed in prior work as a way to learn end-to-end perceptually-oriented image compression, but have not yet shown state-of-the-art performance on the competitive task of ImageNet-1K reconstruction. We propose FlowMo, a transformer-based diffusion autoencoder that achieves a new state-of-the-art for image tokenization at multiple compression rates without using convolutions, adversarial losses, spatially-aligned two-dimensional latent codes, or distilling from other tokenizers. Our key insight is that FlowMo training should be broken into a mode-matching pre-training stage and a mode-seeking post-training stage. In addition, we conduct extensive analyses and explore the training of generative models atop the FlowMo tokenizer. Our code and models will be available at http://kylesargent.github.io/flowmo .

Method: Proposes Pan-LUT with three components: PAN-guided LUT for channel-wise spectral mapping, spatial details LUT for capturing fine details, and adaptive output LUT for aggregating channel information. The framework has <700K parameters.

Result: Processes 15K15K images on 24GB GPU, handles 9K9K images in <1ms on RTX 2080 Ti, and surpasses SOTA methods in full-resolution real-world scenarios while being lightweight.

Conclusion: Pan-LUT bridges the gap to real-world applications by efficiently processing large remote sensing images in a lightweight manner, offering a balance between performance and computational efficiency.

Abstract: Recently, deep learning-based pan-sharpening algorithms have achieved notable advancements over traditional methods. However, deep learning-based methods incur substantial computational overhead during inference, especially with large images. This excessive computational demand limits the applicability of these methods in real-world scenarios, particularly in the absence of dedicated computing devices such as GPUs and TPUs. To address these challenges, we propose Pan-LUT, a novel learnable look-up table (LUT) framework for pan-sharpening that strikes a balance between performance and computational efficiency for large remote sensing images. Our method makes it possible to process 15K15K remote sensing images on a 24GB GPU. To finely control the spectral transformation, we devise the PAN-guided look-up table (PGLUT) for channel-wise spectral mapping. To effectively capture fine-grained spatial details, we introduce the spatial details look-up table (SDLUT). Furthermore, to adaptively aggregate channel information for generating high-resolution multispectral images, we design an adaptive output look-up table (AOLUT). Our model contains fewer than 700K parameters and processes a 9K9K image in under 1 ms using one RTX 2080 Ti GPU, demonstrating significantly faster performance compared to other methods. Experiments reveal that Pan-LUT efficiently processes large remote sensing images in a lightweight manner, bridging the gap to real-world applications. Furthermore, our model surpasses SOTA methods in full-resolution scenes under real-world conditions, highlighting its effectiveness and efficiency.

Method: SATORI decomposes VQA into three verifiable stages: global image captioning, region localization, and answer prediction, each with explicit reward signals. Also introduces VQA-Verify dataset with 12k annotated answer-aligned captions and bounding boxes.

Result: Consistent performance improvements across seven VQA benchmarks, achieving up to 15.7% accuracy improvement compared to R1-like baseline. Attention map analysis confirms enhanced focus on critical regions.

Conclusion: SATORI’s structured approach with verifiable stages and explicit rewards effectively addresses limitations of free-form reasoning in VQA, leading to significant accuracy improvements and better visual focus.

Abstract: DeepSeek-R1 has demonstrated powerful reasoning capabilities in the text domain through stable reinforcement learning (RL). Recently, in the multimodal domain, works have begun to directly apply RL to generate R1-like free-form reasoning for Visual Question Answering (VQA) tasks. However, multimodal tasks share an intrinsically different nature from textual tasks, which heavily rely on the understanding of the input image to solve the problem. Therefore, such free-form reasoning faces two critical limitations in the VQA task: (1) Extended reasoning chains diffuse visual focus away from task-critical regions, degrading answer accuracy. (2) Unverifiable intermediate steps amplify policy-gradient variance and computational costs overhead. To address these issues, in this paper, we introduce SATORI ($\textbf{S}patially$ $\textbf{A}nchored$ $\textbf{T}ask$ $\textbf{O}ptimization$ with $\textbf{R}e\textbf{I}nforcement$ Learning), which decomposes VQA into three verifiable stages, including global image captioning, region localization, and answer prediction, each supplying explicit reward signals. Furthermore, we also introduce VQA-Verify, a 12k dataset annotated with answer-aligned captions and bounding-boxes to facilitate training. Experiments demonstrate consistent performance improvements across seven VQA benchmarks, achieving up to $15.7%$ improvement in accuracy in accuracy compared to the R1-like baseline. Our analysis of the attention map confirms enhanced focus on critical regions, which brings improvements in accuracy. Our code is available at https://github.com/justairr/SATORI-R1.

Method: MoBGS introduces two key components: 1) Blur-adaptive Latent Camera Estimation (BLCE) using a Blur-adaptive Neural ODE solver for effective latent camera trajectory estimation, improving global camera motion deblurring; 2) Latent Camera-induced Exposure Estimation (LCEE) to ensure consistent deblurring of both global camera and local object motions.

Result: Extensive experiments on the Stereo Blur dataset and real-world blurry videos show that MoBGS significantly outperforms recent methods, achieving state-of-the-art performance for dynamic novel view synthesis under motion blur.

Conclusion: MoBGS successfully addresses the challenge of motion blur in dynamic novel view synthesis by combining effective camera trajectory estimation with consistent deblurring of both camera and object motions, enabling high-quality reconstruction from blurry monocular videos.

Abstract: We present MoBGS, a novel motion deblurring 3D Gaussian Splatting (3DGS) framework capable of reconstructing sharp and high-quality novel spatio-temporal views from blurry monocular videos in an end-to-end manner. Existing dynamic novel view synthesis (NVS) methods are highly sensitive to motion blur in casually captured videos, resulting in significant degradation of rendering quality. While recent approaches address motion-blurred inputs for NVS, they primarily focus on static scene reconstruction and lack dedicated motion modeling for dynamic objects. To overcome these limitations, our MoBGS introduces a novel Blur-adaptive Latent Camera Estimation (BLCE) method using a proposed Blur-adaptive Neural Ordinary Differential Equation (ODE) solver for effective latent camera trajectory estimation, improving global camera motion deblurring. In addition, we propose a Latent Camera-induced Exposure Estimation (LCEE) method to ensure consistent deblurring of both a global camera and local object motions. Extensive experiments on the Stereo Blur dataset and real-world blurry videos show that our MoBGS significantly outperforms the very recent methods, achieving state-of-the-art performance for dynamic NVS under motion blur.

Method: A three-stage multimodal framework: 1) Fine-tune MLLM for detailed facial attribute learning, 2) Identity-level discriminative learning to distinguish subtle differences between similar faces, 3) User-specific customization with semantic reasoning for personalized detection.

Result: The framework shows clear advantages over previous detection works and introduces VIPBench, a comprehensive identity-aware benchmark for personalized deepfake detection with 14 generation techniques.

Conclusion: VIPGuard provides an effective, explainable solution for personalized deepfake detection by leveraging identity-specific knowledge and multimodal reasoning, addressing critical security needs for high-profile individuals.

Abstract: Securing personal identity against deepfake attacks is increasingly critical in the digital age, especially for celebrities and political figures whose faces are easily accessible and frequently targeted. Most existing deepfake detection methods focus on general-purpose scenarios and often ignore the valuable prior knowledge of known facial identities, e.g., “VIP individuals” whose authentic facial data are already available. In this paper, we propose \textbf{VIPGuard}, a unified multimodal framework designed to capture fine-grained and comprehensive facial representations of a given identity, compare them against potentially fake or similar-looking faces, and reason over these comparisons to make accurate and explainable predictions. Specifically, our framework consists of three main stages. First, fine-tune a multimodal large language model (MLLM) to learn detailed and structural facial attributes. Second, we perform identity-level discriminative learning to enable the model to distinguish subtle differences between highly similar faces, including real and fake variations. Finally, we introduce user-specific customization, where we model the unique characteristics of the target face identity and perform semantic reasoning via MLLM to enable personalized and explainable deepfake detection. Our framework shows clear advantages over previous detection works, where traditional detectors mainly rely on low-level visual cues and provide no human-understandable explanations, while other MLLM-based models often lack a detailed understanding of specific face identities. To facilitate the evaluation of our method, we built a comprehensive identity-aware benchmark called \textbf{VIPBench} for personalized deepfake detection, involving the latest 7 face-swapping and 7 entire face synthesis techniques for generation. The code is available at https://github.com/KQL11/VIPGuard .

Method: GS4 uses a feed-forward network to incrementally build 3D Gaussians from RGB-D video. It has: 1) Gaussian Prediction Model for sparse Gaussian parameters with color/semantic prediction, 2) Gaussian Refinement Network to merge new Gaussians avoiding redundancy, 3) Joint Gaussian-pose optimization (1-5 iterations) when significant pose changes occur.

Result: Achieves SOTA performance on ScanNet and ScanNet++ benchmarks. Shows strong generalization through zero-shot transfer to NYUv2 and TUM RGB-D datasets. Runs 10x faster and uses 10x fewer Gaussians than prior approaches.

Conclusion: GS4 is the first generalizable GS-based semantic SLAM system that efficiently produces dense, photorealistic 3D maps with semantics, overcoming limitations of both traditional SLAM and existing GS-based methods.

Abstract: Traditional SLAM algorithms excel at camera tracking, but typically produce incomplete and low-resolution maps that are not tightly integrated with semantics prediction. Recent work integrates Gaussian Splatting (GS) into SLAM to enable dense, photorealistic 3D mapping, yet existing GS-based SLAM methods require per-scene optimization that is slow and consumes an excessive number of Gaussians. We present GS4, the first generalizable GS-based semantic SLAM system. Compared with prior approaches, GS4 runs 10x faster, uses 10x fewer Gaussians, and achieves state-of-the-art performance across color, depth, semantic mapping and camera tracking. From an RGB-D video stream, GS4 incrementally builds and updates a set of 3D Gaussians using a feed-forward network. First, the Gaussian Prediction Model estimates a sparse set of Gaussian parameters from input frame, which integrates both color and semantic prediction with the same backbone. Then, the Gaussian Refinement Network merges new Gaussians with the existing set while avoiding redundancy. Finally, when significant pose changes are detected, we perform only 1-5 iterations of joint Gaussian-pose optimization to correct drift, remove floaters, and further improve tracking accuracy. Experiments on the real-world ScanNet and ScanNet++ benchmarks demonstrate state-of-the-art semantic SLAM performance, with strong generalization capability shown through zero-shot transfer to the NYUv2 and TUM RGB-D datasets.

Method: Two main techniques: 1) Model-Guided Importance Sampling (MG-IS) to select most informative inputs for human raters, and 2) Bayesian Rating Aggregation (BRAgg) to address label noise in crowdsourced ratings.

Result: MG-IS reduces needed inputs by ~13x, BRAgg reduces required ratings per input by ~3x, together achieving ~40x cost reduction. These enable large-scale evaluation of interpretability methods for vision networks.

Conclusion: The proposed techniques make large-scale, accurate evaluation of automated interpretability methods feasible, addressing key limitations of existing evaluation pipelines through cost-effective crowdsourcing.

Abstract: Interpreting individual neurons or directions in activation space is an important topic in mechanistic interpretability. Numerous automated interpretability methods have been proposed to generate such explanations, but it remains unclear how reliable these explanations are, and which methods produce the most accurate descriptions. While crowd-sourced evaluations are commonly used, existing pipelines are noisy, costly, and typically assess only the highest-activating inputs, leading to unreliable results. In this paper, we introduce two techniques to enable cost-effective and accurate crowdsourced evaluation of automated interpretability methods beyond top activating inputs. First, we propose Model-Guided Importance Sampling (MG-IS) to select the most informative inputs to show human raters. In our experiments, we show this reduces the number of inputs needed to reach the same evaluation accuracy by ~13x. Second, we address label noise in crowd-sourced ratings through Bayesian Rating Aggregation (BRAgg), which allows us to reduce the number of ratings per input required to overcome noise by ~3x. Together, these techniques reduce the evaluation cost by ~40x, making large-scale evaluation feasible. Finally, we use our methods to conduct a large scale crowd-sourced study comparing recent automated interpretability methods for vision networks.

Method: A²LC uses a cascaded approach with manual and automatic correction stages. The automatic stage leverages human feedback to extend corrections beyond queried samples. It also introduces an adaptively balanced acquisition function that emphasizes underrepresented tail classes, working synergistically with the automatic correction.

Result: Extensive experiments on Cityscapes and PASCAL VOC 2012 show A²LC significantly outperforms previous state-of-the-art methods. It achieves high efficiency by outperforming previous methods with only 20% of their budget, and shows strong effectiveness with a 27.23% performance gain under the same budget on Cityscapes.

Conclusion: A²LC provides an effective and efficient framework for label correction in semantic segmentation by combining manual and automated approaches with adaptive class balancing, substantially reducing annotation costs while improving performance.

Abstract: Active Label Correction (ALC) has emerged as a promising solution to the high cost and error-prone nature of manual pixel-wise annotation in semantic segmentation, by actively identifying and correcting mislabeled data. Although recent work has improved correction efficiency by generating pseudo-labels using foundation models, substantial inefficiencies still remain. In this paper, we introduce A$^2$LC, an Active and Automated Label Correction framework for semantic segmentation, where manual and automatic correction stages operate in a cascaded manner. Specifically, the automatic correction stage leverages human feedback to extend label corrections beyond the queried samples, thereby maximizing cost efficiency. In addition, we introduce an adaptively balanced acquisition function that emphasizes underrepresented tail classes, working in strong synergy with the automatic correction stage. Extensive experiments on Cityscapes and PASCAL VOC 2012 demonstrate that A$^2$LC significantly outperforms previous state-of-the-art methods. Notably, A$^2$LC exhibits high efficiency by outperforming previous methods with only 20% of their budget, and shows strong effectiveness by achieving a 27.23% performance gain under the same budget on Cityscapes.

Method: Proposes a large-scale benchmark to systematically assess three groups of Language Gaussian Splatting methods (per-scene optimization-based, per-scene optimization-free, and generalizable) directly in 3D space across 1060 scenes from multiple datasets.

Result: Benchmark results show generalizable approaches have clear advantages: relaxing scene-specific limitations, enabling fast feed-forward inference on novel scenes, and achieving superior segmentation performance. The GaussianWorld-49K dataset demonstrates that generalizable methods can leverage strong data priors.

Conclusion: Generalizable Language Gaussian Splatting approaches outperform other paradigms, and the proposed benchmark and dataset will accelerate research in generalizable 3DGS scene understanding.

Abstract: 3D Gaussian Splatting (3DGS) serves as a highly performant and efficient encoding of scene geometry, appearance, and semantics. Moreover, grounding language in 3D scenes has proven to be an effective strategy for 3D scene understanding. Current Language Gaussian Splatting line of work fall into three main groups: (i) per-scene optimization-based, (ii) per-scene optimization-free, and (iii) generalizable approach. However, most of them are evaluated only on rendered 2D views of a handful of scenes and viewpoints close to the training views, limiting ability and insight into holistic 3D understanding. To address this gap, we propose the first large-scale benchmark that systematically assesses these three groups of methods directly in 3D space, evaluating on 1060 scenes across three indoor datasets and one outdoor dataset. Benchmark results demonstrate a clear advantage of the generalizable paradigm, particularly in relaxing the scene-specific limitation, enabling fast feed-forward inference on novel scenes, and achieving superior segmentation performance. We further introduce GaussianWorld-49K a carefully curated 3DGS dataset comprising around 49K diverse indoor and outdoor scenes obtained from multiple sources, with which we demonstrate the generalizable approach could harness strong data priors. Our codes, benchmark, and datasets are public to accelerate research in generalizable 3DGS scene understanding.

Method: Uses inter-epoch baseline vector residuals between consecutive GNSS fixes to mitigate inertial drift initially. Introduces Hessian matrix singular value evolution as criterion to determine when to activate global GNSS constraints based on system observability.

Result: Experiments on EuRoC, GVINS and MARS-LVIG datasets show consistent outperformance over naive strategy of fusing all measurements from the outset, yielding more accurate and robust initializations.

Conclusion: Adaptive initialization strategy that delays global constraints until they become sufficiently informative improves accuracy and robustness of sensor platform initialization.

Abstract: Initializing the state of a sensorized platform can be challenging, as a limited set of measurements often provide low-informative constraints that are in addition highly non-linear. This may lead to poor initial estimates that may converge to local minima during subsequent non-linear optimization. We propose an adaptive GNSS-inertial initialization strategy that delays the incorporation of global GNSS constraints until they become sufficiently informative. In the initial stage, our method leverages inter-epoch baseline vector residuals between consecutive GNSS fixes to mitigate inertial drift. To determine when to activate global constraints, we introduce a general criterion based on the evolution of the Hessian matrix’s singular values, effectively quantifying system observability. Experiments on EuRoC, GVINS and MARS-LVIG datasets show that our approach consistently outperforms the naive strategy of fusing all measurements from the outset, yielding more accurate and robust initializations.

Method: BitMark embeds watermarks directly at the bit level of the token stream during image generation, subtly influencing bits to preserve visual fidelity and generation speed while remaining robust against removal techniques. The watermark is radioactive, meaning it propagates to models trained on watermarked images.

Result: The watermarking framework is robust against various removal techniques, maintains visual quality and generation speed, and exhibits high radioactivity - watermarks remain detectable even when fine-tuning diffusion or autoregressive models on watermarked images.

Conclusion: BitMark provides a principled approach to prevent model collapse in image generative models by enabling reliable detection of generated outputs, offering a practical solution to the growing problem of synthetic content polluting training data.

Abstract: State-of-the-art text-to-image models generate photorealistic images at an unprecedented speed. This work focuses on models that operate in a bitwise autoregressive manner over a discrete set of tokens that is practically infinite in size. However, their impressive generative power comes with a growing risk: as their outputs increasingly populate the Internet, they are likely to be scraped and reused as training data-potentially by the very same models. This phenomenon has been shown to lead to model collapse, where repeated training on generated content, especially from the models’ own previous versions, causes a gradual degradation in performance. A promising mitigation strategy is watermarking, which embeds human-imperceptible yet detectable signals into generated images-enabling the identification of generated content. In this work, we introduce BitMark, a robust bitwise watermarking framework. Our method embeds a watermark directly at the bit level of the token stream during the image generation process. Our bitwise watermark subtly influences the bits to preserve visual fidelity and generation speed while remaining robust against a spectrum of removal techniques. Furthermore, it exhibits high radioactivity, i.e., when watermarked generated images are used to train another image generative model, this second model’s outputs will also carry the watermark. The radioactive traces remain detectable even when only fine-tuning diffusion or image autoregressive models on images watermarked with our BitMark. Overall, our approach provides a principled step toward preventing model collapse in image generative models by enabling reliable detection of generated outputs. The code is available at https://github.com/sprintml/BitMark.

Method: Three-step pipeline: 1) Infrared pedestrian detection using fine-tuned model, 2) Label transfer from infrared to RGB counterparts, 3) Training object detection models using generated labels for low-light RGB pedestrian detection.

Result: Models trained on generated autolabels outperformed those trained on ground-truth labels in 6 out of 9 cases for mAP@50 and mAP@50-95 metrics on unseen image sequences.

Conclusion: The proposed infrared-RGB automated labeling pipeline effectively addresses low-light pedestrian detection challenges and improves detection performance over traditional ground-truth labeling approaches.

Abstract: Pedestrian detection in RGB images is a key task in pedestrian safety, as the most common sensor in autonomous vehicles and advanced driver assistance systems is the RGB camera. A challenge in RGB pedestrian detection, that does not appear to have large public datasets, is low-light conditions. As a solution, in this research, we propose an automated infrared-RGB labeling pipeline. The proposed pipeline consists of 1) Infrared detection, where a fine-tuned model for infrared pedestrian detection is used 2) Label transfer process from the infrared detections to their RGB counterparts 3) Training object detection models using the generated labels for low-light RGB pedestrian detection. The research was performed using the KAIST dataset. For the evaluation, object detection models were trained on the generated autolabels and ground truth labels. When compared on a previously unseen image sequence, the results showed that the models trained on generated labels outperformed the ones trained on ground-truth labels in 6 out of 9 cases for the mAP@50 and mAP@50-95 metrics. The source code for this research is available at https://github.com/BouzoulasDimitrios/IR-RGB-Automated-LowLight-Pedestrian-Labeling

Method: UniEdit-I operates entirely within semantic latent space of unified VLMs using an Understanding-Editing-Verifying (UEV) loop. It transforms VLMs from posthoc evaluators to in-process conductors, enabling semantics-driven, self-correcting closed-loop editing without training.

Result: Achieves state-of-the-art performance on GEdit-Bench without any fine-tuning or architectural modifications, surpassing several large-scale pretrained editors.

Conclusion: UniEdit-I bridges the representation gap by editing in semantic latent space, enabling coherent and plausible image editing through closed-loop feedback between understanding and generation.

Abstract: While Unified Vision-Language Models promise to synergistically combine the high-level semantic understanding of vision-language models with the generative fidelity of diffusion models, current editing methodologies remain fundamentally decoupled and open loop performing static, pre-defined transformations without dynamic feedback between semantic interpretation and visual generation. A central limitation stems from the representation gap: understanding typically leverages high-level, language aligned encoders, whereas generation relies on low level, pixel-space autoencoders, resulting in misaligned feature spaces. To bridge this gap, Recent advances such as Representation Autoencoders and BLIP3-o advocate performing diffusion-based modeling directly in high level features from pretrained semantic encoders. We find editing in the semantic latent space modifies conceptual representations rather than pixels, ensuring intermediates that are both semantically coherent and visually plausible. Building on this insight, We propose UniEdit-I, the first training-free, closed-loop image editing framework that operates entirely within the semantic latent space of a unified VLM by introducing an Understanding-Editing-Verifying (UEV) loop, By transforming the VLM from a posthoc evaluator into an in-process conductor, UniEdit-I establishes the first semantics-driven, self-correcting closed-loop image editing pipeline. Evaluated on GEdit-Bench, UniEdit-I achieves state of the art performance without any fine tuning or architectural modifications, and even surpasses several largescale pretrained editors.

Method: FantasyStyle introduces two key components: (1) Multi-View Frequency Consistency - applies 3D filter to multi-view noisy latent to selectively reduce low-frequency components and mitigate stylized prior conflicts; (2) Controllable Stylized Distillation - uses negative guidance to exclude undesired content from style images, removes reconstruction terms from Score Distillation Sampling and Delta Denoising Score, and optimizes 3D Gaussians more effectively.

Result: Extensive experiments demonstrate that FantasyStyle consistently outperforms state-of-the-art approaches, achieving higher stylization quality and visual realism across various scenes and styles.

Conclusion: FantasyStyle successfully addresses key challenges in 3DGS-based style transfer through diffusion model distillation, achieving superior results compared to existing methods while being the first to rely entirely on diffusion model distillation for 3D style transfer.

Abstract: The success of 3DGS in generative and editing applications has sparked growing interest in 3DGS-based style transfer. However, current methods still face two major challenges: (1) multi-view inconsistency often leads to style conflicts, resulting in appearance smoothing and distortion; and (2) heavy reliance on VGG features, which struggle to disentangle style and content from style images, often causing content leakage and excessive stylization. To tackle these issues, we introduce \textbf{FantasyStyle}, a 3DGS-based style transfer framework, and the first to rely entirely on diffusion model distillation. It comprises two key components: (1) \textbf{Multi-View Frequency Consistency}. We enhance cross-view consistency by applying a 3D filter to multi-view noisy latent, selectively reducing low-frequency components to mitigate stylized prior conflicts. (2) \textbf{Controllable Stylized Distillation}. To suppress content leakage from style images, we introduce negative guidance to exclude undesired content. In addition, we identify the limitations of Score Distillation Sampling and Delta Denoising Score in 3D style transfer and remove the reconstruction term accordingly. Building on these insights, we propose a controllable stylized distillation that leverages negative guidance to more effectively optimize the 3D Gaussians. Extensive experiments demonstrate that our method consistently outperforms state-of-the-art approaches, achieving higher stylization quality and visual realism across various scenes and styles. The code is available at https://github.com/yangyt46/FantasyStyle.

Method: 1) Unified spatiotemporal encoder using Point Pair Features (PPF) to learn discriminative trajectory embeddings while suppressing interference. 2) Cue-consistency transformer module that explicitly aligns consistent feature representations between historical tracks and current detections. 3) Dynamic update mechanism to preserve salient spatiotemporal information for stable online tracking.

Result: Achieves state-of-the-art performance on nuScenes benchmark with 73.2% AMOTA on validation set and 70.3% AMOTA on test set. Extensive experiments on nuScenes and Waymo Open Datasets validate effectiveness and robustness.

Conclusion: The proposed DSC-Track successfully addresses limitations of existing methods by focusing on cue-consistency, demonstrating improved tracking performance in challenging scenarios through stable spatial pattern matching and interference suppression.

Abstract: 3D multi-object tracking is a critical and challenging task in the field of autonomous driving. A common paradigm relies on modeling individual object motion, e.g., Kalman filters, to predict trajectories. While effective in simple scenarios, this approach often struggles in crowded environments or with inaccurate detections, as it overlooks the rich geometric relationships between objects. This highlights the need to leverage spatial cues. However, existing geometry-aware methods can be susceptible to interference from irrelevant objects, leading to ambiguous features and incorrect associations. To address this, we propose focusing on cue-consistency: identifying and matching stable spatial patterns over time. We introduce the Dynamic Scene Cue-Consistency Tracker (DSC-Track) to implement this principle. Firstly, we design a unified spatiotemporal encoder using Point Pair Features (PPF) to learn discriminative trajectory embeddings while suppressing interference. Secondly, our cue-consistency transformer module explicitly aligns consistent feature representations between historical tracks and current detections. Finally, a dynamic update mechanism preserves salient spatiotemporal information for stable online tracking. Extensive experiments on the nuScenes and Waymo Open Datasets validate the effectiveness and robustness of our approach. On the nuScenes benchmark, for instance, our method achieves state-of-the-art performance, reaching 73.2% and 70.3% AMOTA on the validation and test sets, respectively.

Method: Proposes S5 framework with: 1) RS4P-1M dataset created via entropy-based filtering and diversity expansion; 2) S4 pre-training (S4P) paradigm to pretrain RS foundation models on large unlabeled data; 3) Mixture-of-Experts-based multi-dataset fine-tuning for efficient adaptation to multiple benchmarks.

Result: RS foundation models achieve state-of-the-art performance across all remote sensing benchmarks for land cover segmentation and object detection tasks, demonstrating viability of scaling semi-supervised learning for RS applications.

Conclusion: S5 successfully scales semi-supervised semantic segmentation for remote sensing by leveraging vast unlabeled data through novel dataset creation, pre-training paradigm, and efficient fine-tuning, establishing a new scalable framework for RS analysis.

Abstract: Semi-supervised semantic segmentation (S4) has advanced remote sensing (RS) analysis by leveraging unlabeled data through pseudo-labeling and consistency learning. However, existing S4 studies often rely on small-scale datasets and models, limiting their practical applicability. To address this, we propose S5, the first scalable framework for semi-supervised semantic segmentation in RS, which unlocks the potential of vast unlabeled Earth observation data typically underutilized due to costly pixel-level annotations. Built upon existing large-scale RS datasets, S5 introduces a data selection strategy that integrates entropy-based filtering and diversity expansion, resulting in the RS4P-1M dataset. Using this dataset, we systematically scale up S4 into a new pretraining paradigm, S4 pre-training (S4P), to pretrain RS foundation models (RSFMs) of varying sizes on this extensive corpus, significantly boosting their performance on land cover segmentation and object detection tasks. Furthermore, during fine-tuning, we incorporate a Mixture-of-Experts (MoE)-based multi-dataset fine-tuning approach, which enables efficient adaptation to multiple RS benchmarks with fewer parameters. This approach improves the generalization and versatility of RSFMs across diverse RS benchmarks. The resulting RSFMs achieve state-of-the-art performance across all benchmarks, underscoring the viability of scaling semi-supervised learning for RS applications. All datasets, code, and models will be released at https://github.com/MiliLab/S5

Method: Multi-kernel encoder captures diverse regional interactions from satellite imagery; permutation-aware diffusion process maintains consistency across regional orderings; joint contrastive training links satellite features with OD patterns; equivariant diffusion training enforces structural invariance.

Result: Outperforms physics-based and data-driven baselines in accuracy; preserves flow distributions and spatial structures under index permutations; eliminates region-specific auxiliary data dependencies.

Conclusion: Sat2Flow offers globally scalable solution for OD flow generation in data-scarce environments, maintaining structural robustness for reliable mobility modeling without auxiliary data dependencies.

Abstract: Origin-Destination (OD) flow matrices are critical for urban mobility analysis, supporting traffic forecasting, infrastructure planning, and policy design. Existing methods face two key limitations: (1) reliance on costly auxiliary features (e.g., Points of Interest, socioeconomic statistics) with limited spatial coverage, and (2) fragility to spatial topology changes, where reordering urban regions disrupts the structural coherence of generated flows. We propose Sat2Flow, a structure-aware diffusion framework that generates structurally coherent OD flows using only satellite imagery. Our approach employs a multi-kernel encoder to capture diverse regional interactions and a permutation-aware diffusion process that maintains consistency across regional orderings. Through joint contrastive training linking satellite features with OD patterns and equivariant diffusion training enforcing structural invariance, Sat2Flow ensures topological robustness under arbitrary regional reindexing. Experiments on real-world datasets show that Sat2Flow outperforms physics-based and data-driven baselines in accuracy while preserving flow distributions and spatial structures under index permutations. Sat2Flow offers a globally scalable solution for OD flow generation in data-scarce environments, eliminating region-specific auxiliary data dependencies while maintaining structural robustness for reliable mobility modeling.

Method: Introduces SpecGen with Spectral-Spatial Tri-plane Aggregation (SSTA) network that models reflectance responses across wavelengths and incident-outgoing directions, leveraging abundant RGB BRDF data to enhance spectral BRDF generation despite limited spectral data.

Result: Accurately reconstructs spectral BRDFs from limited spectral data and surpasses state-of-the-art methods in hyperspectral image reconstruction, achieving 8 dB improvement in PSNR.

Conclusion: SpecGen enables spectral image rendering under arbitrary illuminations and shapes from single RGB sphere images, overcoming spectral data scarcity through innovative network architecture and training strategy.

Abstract: Synthesizing spectral images across different wavelengths is essential for photorealistic rendering. Unlike conventional spectral uplifting methods that convert RGB images into spectral ones, we introduce SpecGen, a novel method that generates spectral bidirectional reflectance distribution functions (BRDFs) from a single RGB image of a sphere. This enables spectral image rendering under arbitrary illuminations and shapes covered by the corresponding material. A key challenge in spectral BRDF generation is the scarcity of measured spectral BRDF data. To address this, we propose the Spectral-Spatial Tri-plane Aggregation (SSTA) network, which models reflectance responses across wavelengths and incident-outgoing directions, allowing the training strategy to leverage abundant RGB BRDF data to enhance spectral BRDF generation. Experiments show that our method accurately reconstructs spectral BRDFs from limited spectral data and surpasses state-of-the-art methods in hyperspectral image reconstruction, achieving an improvement of 8 dB in PSNR. Codes and data will be released upon acceptance.

Method: Proposes Linguistic Scene Graph Anticipation (LSGA) using textualized scene graphs, and introduces Object-Oriented Two-Stage Method (OOTSM) with language models to anticipate object-set dynamics and forecast object-centric relation trajectories with temporal consistency regularization.

Result: Fine-tuned language models (up to 3B parameters) outperform zero-/one-shot API baselines (GPT-4o, GPT-4o-mini, DeepSeek-V3). When combined with visual scene-graph generators, the multimodal system boosts long-horizon mR@50 by up to 21.9% over visual SGA baselines.

Conclusion: Linguistic formulation of SGA effectively captures semantic dynamics for long-horizon forecasting, demonstrating that language models can significantly enhance scene graph anticipation when properly adapted to the task.

Abstract: A scene graph is a structured representation of objects and their spatio-temporal relationships in dynamic scenes. Scene Graph Anticipation (SGA) involves predicting future scene graphs from video clips, enabling applications in intelligent surveillance and human-machine collaboration. While recent SGA approaches excel at leveraging visual evidence, long-horizon forecasting fundamentally depends on semantic priors and commonsense temporal regularities that are challenging to extract purely from visual features. To explicitly model these semantic dynamics, we propose Linguistic Scene Graph Anticipation (LSGA), a linguistic formulation of SGA that performs temporal relational reasoning over sequences of textualized scene graphs, with visual scene-graph detection handled by a modular front-end when operating on video. Building on this formulation, we introduce Object-Oriented Two-Stage Method (OOTSM), a language-based framework that anticipates object-set dynamics and forecasts object-centric relation trajectories with temporal consistency regularization, and we evaluate it on a dedicated benchmark constructed from Action Genome annotations. Extensive experiments show that compact fine-tuned language models with up to 3B parameters consistently outperform strong zero- and one-shot API baselines, including GPT-4o, GPT-4o-mini, and DeepSeek-V3, under matched textual inputs and context windows. When coupled with off-the-shelf visual scene-graph generators, the resulting multimodal system achieves substantial improvements on video-based SGA, boosting long-horizon mR@50 by up to 21.9% over strong visual SGA baselines.

Method: Establish precise definitions for world models, introduce structured taxonomy covering three main approaches: VideoGen (video-based), OccGen (occupancy-based), and LiDARGen (LiDAR-based), systematically summarize datasets and evaluation metrics for 3D/4D settings.

Result: Provides first comprehensive review dedicated to 3D/4D world modeling, creates foundational reference with clear taxonomy, summarizes existing literature systematically, and identifies practical applications.

Conclusion: Survey establishes coherent foundation for advancing 3D/4D world modeling field, discusses practical applications, identifies open challenges, and highlights promising research directions to guide future work.

Abstract: World modeling has become a cornerstone in AI research, enabling agents to understand, represent, and predict the dynamic environments they inhabit. While prior work largely emphasizes generative methods for 2D image and video data, they overlook the rapidly growing body of work that leverages native 3D and 4D representations such as RGB-D imagery, occupancy grids, and LiDAR point clouds for large-scale scene modeling. At the same time, the absence of a standardized definition and taxonomy for ``world models’’ has led to fragmented and sometimes inconsistent claims in the literature. This survey addresses these gaps by presenting the first comprehensive review explicitly dedicated to 3D and 4D world modeling and generation. We establish precise definitions, introduce a structured taxonomy spanning video-based (VideoGen), occupancy-based (OccGen), and LiDAR-based (LiDARGen) approaches, and systematically summarize datasets and evaluation metrics tailored to 3D/4D settings. We further discuss practical applications, identify open challenges, and highlight promising research directions, aiming to provide a coherent and foundational reference for advancing the field. A systematic summary of existing literature is available at https://github.com/worldbench/awesome-3d-4d-world-models

Method: Proposes DGFusion network that treats multimodal segmentation as multi-task problem using lidar measurements as both input and ground truth for depth learning. Uses auxiliary depth head to learn depth-aware features, encoded into local depth tokens that condition attentive cross-modal fusion, along with global condition token for dynamic adaptation.

Result: Achieves state-of-the-art panoptic and semantic segmentation performance on challenging MUSES and DeLiVER datasets, demonstrating improved robustness in adverse conditions.

Conclusion: Depth-guided fusion with spatially adaptive conditioning significantly improves semantic perception by dynamically adjusting sensor fusion based on depth-dependent reliability, overcoming limitations of uniform fusion approaches.

Abstract: Robust semantic perception for autonomous vehicles relies on effectively combining multiple sensors with complementary strengths and weaknesses. State-of-the-art sensor fusion approaches to semantic perception often treat sensor data uniformly across the spatial extent of the input, which hinders performance when faced with challenging conditions. By contrast, we propose a novel depth-guided multimodal fusion method that upgrades condition-aware fusion by integrating depth information. Our network, DGFusion, poses multimodal segmentation as a multi-task problem, utilizing the lidar measurements, which are typically available in outdoor sensor suites, both as one of the model’s inputs and as ground truth for learning depth. Our corresponding auxiliary depth head helps to learn depth-aware features, which are encoded into spatially varying local depth tokens that condition our attentive cross-modal fusion. Together with a global condition token, these local depth tokens dynamically adapt sensor fusion to the spatially varying reliability of each sensor across the scene, which largely depends on depth. In addition, we propose a robust loss for our depth, which is essential for learning from lidar inputs that are typically sparse and noisy in adverse conditions. Our method achieves state-of-the-art panoptic and semantic segmentation performance on the challenging MUSES and DeLiVER datasets. Code and models will be available at https://github.com/timbroed/DGFusion

Method: 1) Pack-based MIL framework packs multiple sampled variable-length feature sequences into fixed-length ones for batched training while preserving heterogeneity. 2) Residual branch composes discarded features from multiple slides into a hyperslide with tailored labels for multi-slide supervision. 3) Attention-driven downsampler compresses features in both branches to reduce redundancy.

Result: Achieves up to 8% accuracy improvement while using only 12% of training time in PANDA(UNI) benchmark. Extensive experiments show significant potential for addressing data challenges in computational pathology foundation models.

Conclusion: Focusing on data challenges in computational pathology through the proposed PackMIL framework demonstrates substantial improvements in both accuracy and efficiency, highlighting the importance of addressing extreme sequence length variations and limited supervision in foundation model era.

Abstract: Computational pathology (CPath) digitizes pathology slides into whole slide images (WSIs), enabling analysis for critical healthcare tasks such as cancer diagnosis and prognosis. However, WSIs possess extremely long sequence lengths (up to 200K), significant length variations (from 200 to 200K), and limited supervision. These extreme variations in sequence length lead to high data heterogeneity and redundancy. Conventional methods often compromise on training efficiency and optimization to preserve such heterogeneity under limited supervision. To comprehensively address these challenges, we propose a pack-based MIL framework. It packs multiple sampled, variable-length feature sequences into fixed-length ones, enabling batched training while preserving data heterogeneity. Moreover, we introduce a residual branch that composes discarded features from multiple slides into a hyperslide which is trained with tailored labels. It offers multi-slide supervision while mitigating feature loss from sampling. Meanwhile, an attention-driven downsampler is introduced to compress features in both branches to reduce redundancy. By alleviating these challenges, our approach achieves an accuracy improvement of up to 8% while using only 12% of the training time in the PANDA(UNI). Extensive experiments demonstrate that focusing data challenges in CPath holds significant potential in the era of foundation models. The code is https://github.com/FangHeng/PackMIL

Method: Provides a simple derivation unifying Gaussian diffusion and flow matching using Bayes’ rule and conditional expectations (without ODE/SDE formulations). Extends Score identity Distillation (SiD) to pretrained text-to-image flow-matching models (SANA, SD3-Medium, SD3.5-Medium/Large, FLUX.1-dev) with DiT backbones, requiring only modest flow-matching- and DiT-specific adjustments.

Result: SiD works out of the box across these models in both data-free and data-aided settings, without requiring teacher finetuning or architectural changes. Provides first systematic evidence that score distillation applies broadly to text-to-image flow matching models.

Conclusion: Resolves prior concerns about stability and soundness, unifying acceleration techniques across diffusion- and flow-based generators. Demonstrates that distillation techniques transfer directly between theoretically equivalent frameworks.

Abstract: Diffusion models achieve high-quality image generation but are limited by slow iterative sampling. Distillation methods alleviate this by enabling one- or few-step generation. Flow matching, originally introduced as a distinct framework, has since been shown to be theoretically equivalent to diffusion under Gaussian assumptions, raising the question of whether distillation techniques such as score distillation transfer directly. We provide a simple derivation – based on Bayes’ rule and conditional expectations – that unifies Gaussian diffusion and flow matching without relying on ODE/SDE formulations. Building on this view, we extend Score identity Distillation (SiD) to pretrained text-to-image flow-matching models, including SANA, SD3-Medium, SD3.5-Medium/Large, and FLUX.1-dev, all with DiT backbones. Experiments show that, with only modest flow-matching- and DiT-specific adjustments, SiD works out of the box across these models, in both data-free and data-aided settings, without requiring teacher finetuning or architectural changes. This provides the first systematic evidence that score distillation applies broadly to text-to-image flow matching models, resolving prior concerns about stability and soundness and unifying acceleration techniques across diffusion- and flow-based generators. A project page is available at https://yigu1008.github.io/SiD-DiT.

Method: Correlation analysis between CNN architectures and human behavioral/fMRI data for image valence appraisal, plus Object2Brain framework combining GradCAM and object detection at CNN-filter level to study object class influences.

Result: CNNs struggle to go beyond simple visual processing for valence appraisal and don’t reflect higher-order brain processing; different CNN architectures show different object class sensitivities despite similar correlation trends.

Conclusion: The CNN-human correspondence observed in general visual perception doesn’t extend to social cognition tasks like valence appraisal, highlighting limitations of current CNNs in modeling complex brain processes.

Abstract: Convolutional Neural Networks (CNNs) are a popular type of computer model that have proven their worth in many computer vision tasks. Moreover, they form an interesting study object for the field of psychology, with shown correspondences between the workings of CNNs and the human brain. However, these correspondences have so far mostly been studied in the context of general visual perception. In contrast, this paper explores to what extent this correspondence also holds for a more complex brain process, namely social cognition. To this end, we assess the alignment between popular CNN architectures and both human behavioral and fMRI data for image valence appraisal through a correlation analysis. We show that for this task CNNs struggle to go beyond simple visual processing, and do not seem to reflect higher-order brain processing. Furthermore, we present Object2Brain, a novel framework that combines GradCAM and object detection at the CNN-filter level with the aforementioned correlation analysis to study the influence of different object classes on the CNN-to-human correlations. Despite similar correlation trends, different CNN architectures are shown to display different object class sensitivities.

Method: 1) Predict keypoint heatmaps directly in SD U-Net’s image latent space to preserve generative priors; 2) Use lightweight convolutional pose head to map latent features to heatmaps without disrupting backbone; 3) Add auxiliary RGB reconstruction branch to prevent overfitting and enhance OOD robustness.

Result: With only 1/5 training of Sapiens on COCO, SDPose matches Sapiens-1B/2B on COCO validation and achieves SOTA on cross-domain benchmarks HumanArt and COCO-OOD. Ablations confirm importance of diffusion priors, RGB reconstruction, and multi-scale features for cross-domain generalization.

Conclusion: SDPose effectively exploits diffusion priors for human pose estimation with strong cross-domain generalization. It also serves as a zero-shot pose annotator for controllable image/video generation, demonstrating the value of diffusion models as vision backbones for structured tasks.

Abstract: Pre-trained diffusion models provide rich multi-scale latent features and are emerging as powerful vision backbones. While recent works such as Marigold and Lotus adapt diffusion priors for dense prediction with strong cross-domain generalization, their potential for structured outputs remains underexplored. In this paper, we propose SDPose, a fine-tuning framework built upon Stable Diffusion to fully exploit pre-trained diffusion priors for human pose estimation. First, rather than modifying cross-attention modules or introducing learnable embeddings, we directly predict keypoint heatmaps in the SD U-Net’s image latent space to preserve the original generative priors. Second, we map these latent features into keypoint heatmaps through a lightweight convolutional pose head, which avoids disrupting the pre-trained backbone. Finally, to prevent overfitting and enhance out-of-distribution robustness, we incorporate an auxiliary RGB reconstruction branch that preserves domain-transferable generative semantics. To evaluate robustness under domain shift, we further construct COCO-OOD, a style-transferred variant of COCO with preserved annotations. With just one-fifth of the training schedule used by Sapiens on COCO, SDPose attains parity with Sapiens-1B/2B on the COCO validation set and establishes a new state of the art on the cross-domain benchmarks HumanArt and COCO-OOD. Extensive ablations highlight the importance of diffusion priors, RGB reconstruction, and multi-scale SD U-Net features for cross-domain generalization, and t-SNE analyses further explain SD’s domain-invariant latent structure. We also show that SDPose serves as an effective zero-shot pose annotator for controllable image and video generation.

Method: Proposes Low-Rank Adaptation (LoRA) patching with a learnable gating mechanism to prevent gradient explosions, and Multi-Modal Feature Alignment (MMFA) loss for semantic-level feature alignment. Also introduces defensive LoRA patching as a complementary security solution.

Result: With only 1,000 facial examples and single epoch fine-tuning, LoRA patching successfully defeats multiple state-of-the-art proactive defenses, demonstrating critical weaknesses in current defense strategies.

Conclusion: Current proactive Deepfake defense paradigms have fundamental vulnerabilities that can be exploited with minimal resources, necessitating development of more robust defense strategies and complementary security measures.

Abstract: Deepfakes pose significant societal risks, motivating the development of proactive defenses that embed adversarial perturbations in facial images to prevent manipulation. However, in this paper, we show that these preemptive defenses often lack robustness and reliability. We propose a novel approach, Low-Rank Adaptation (LoRA) patching, which injects a plug-and-play LoRA patch into Deepfake generators to bypass state-of-the-art defenses. A learnable gating mechanism adaptively controls the effect of the LoRA patch and prevents gradient explosions during fine-tuning. We also introduce a Multi-Modal Feature Alignment (MMFA) loss, encouraging the features of adversarial outputs to align with those of the desired outputs at the semantic level. Beyond bypassing, we present defensive LoRA patching, embedding visible warnings in the outputs as a complementary solution to mitigate this newly identified security vulnerability. With only 1,000 facial examples and a single epoch of fine-tuning, LoRA patching successfully defeats multiple proactive defenses. These results reveal a critical weakness in current paradigms and underscore the need for more robust Deepfake defense strategies. Our code is available at https://github.com/ZOMIN28/LoRA-Patching.

Method: Unsupervised autoencoder with Cohen-Daubechies-Feauveau (CDF 97) wavelet transform in the latent space, designed specifically to suppress mixed Gaussian-Poisson noise while preserving essential image features.

Result: The network successfully denoises neutron imaging data, achieving lower reconstruction error and superior edge preservation compared to conventional filtering methods like BM3D, as demonstrated on both simulated and experimental NIF datasets.

Conclusion: This study validates unsupervised learning for denoising neutron images and establishes a critical first step toward fully AI-driven, end-to-end reconstruction frameworks for inertial confinement fusion diagnostics.

Abstract: Neutron imaging is essential for diagnosing and optimizing inertial confinement fusion implosions at the National Ignition Facility. Due to the required 10-micrometer resolution, however, neutron image require image reconstruction using iterative algorithms. For low-yield sources, the images may be degraded by various types of noise. Gaussian and Poisson noise often coexist within one image, obscuring fine details and blurring the edges where the source information is encoded. Traditional denoising techniques, such as filtering and thresholding, can inadvertently alter critical features or reshape the noise statistics, potentially impacting the ultimate fidelity of the iterative image reconstruction pipeline. However, recent advances in synthetic data production and machine learning have opened new opportunities to address these challenges. In this study, we present an unsupervised autoencoder with a Cohen-Daubechies- Feauveau (CDF 97) wavelet transform in the latent space, designed to suppress for mixed Gaussian-Poisson noise while preserving essential image features. The network successfully denoises neutron imaging data. Benchmarking against both simulated and experimental NIF datasets demonstrates that our approach achieves lower reconstruction error and superior edge preservation compared to conventional filtering methods such as Block-matching and 3D filtering (BM3D). By validating the effectiveness of unsupervised learning for denoising neutron images, this study establishes a critical first step towards fully AI-driven, end-to-end reconstruction frameworks for ICF diagnostics.

Method: Proposes STT-GS: 1) First samples subset of images as pilot data using feature-domain clustering (FDC) for representative selection and pilot transmission time minimization (PTTM) to reduce overhead; 2) Based on pilot evaluation, implements joint client selection and power control (JCSPC) framework using penalty alternating majorization minimization (PAMM) algorithm to solve nonconvex optimization.

Result: Experiments show significant outperformance over benchmarks on real-world datasets. GS-oriented objective accurately predicted with low sampling ratios (e.g., 10%), achieving excellent tradeoff between view contributions and communication costs.

Conclusion: STT-GS effectively addresses the causality dilemma in EGS by separating sampling and transmission, enabling efficient resource allocation based on predicted GS quality with minimal pilot overhead, making it suitable for low-altitude economy applications.

Abstract: Edge Gaussian splatting (EGS), which aggregates data from distributed clients (e.g., drones) and trains a global GS model at the edge (e.g., ground server), is an emerging paradigm for scene reconstruction in low-altitude economy. Unlike traditional edge resource management methods that emphasize communication throughput or general-purpose learning performance, EGS explicitly aims to maximize the GS qualities, rendering existing approaches inapplicable. To address this problem, this paper formulates a novel GS-oriented objective function that distinguishes the heterogeneous view contributions of different clients. However, evaluating this function in turn requires clients’ images, leading to a causality dilemma. To this end, this paper further proposes a sample-then-transmit EGS (or STT-GS for short) strategy, which first samples a subset of images as pilot data from each client for loss prediction. Based on the first-stage evaluation, communication resources are then prioritized towards more valuable clients. To achieve efficient sampling, a feature-domain clustering (FDC) scheme is proposed to select the most representative data and pilot transmission time minimization (PTTM) is adopted to reduce the pilot overhead. Subsequently, we develop a joint client selection and power control (JCSPC) framework to maximize the GS-oriented function under communication resource constraints. Despite the nonconvexity of the problem, we propose a low-complexity efficient solution based on the penalty alternating majorization minimization (PAMM) algorithm. Experiments reveal that the proposed scheme significantly outperforms existing benchmarks on real-world datasets. The GS-oriented objective can be accurately predicted with low sampling ratios (e.g., 10%), and our method achieves an excellent tradeoff between view contributions and communication costs.

Method: Integrates vision encoder, audio encoder, LLM, and speech decoder into unified architecture. Uses multi-stage training: pre-training for omni-modal understanding, then post-training with speech conversation and audio-visual interaction. Curates multi-turn training dataset for long-term conversational ability. Constructs specialized benchmarks for evaluation.

Result: Significantly outperforms leading open-source models in multi-turn audio-visual experience, especially in long-term memory. InteractiveOmni-4B comparable to larger Qwen2.5-Omni-7B on general benchmarks, retains 97% of 8B performance with 50% model size. Achieves SOTA results across image, audio, video understanding, and speech generation tasks for similarly sized models.

Conclusion: InteractiveOmni provides an accessible, open-source foundation for next-generation intelligent interactive systems, demonstrating that lightweight models can achieve strong omni-modal capabilities through unified architecture and specialized training strategies.

Abstract: We introduce InteractiveOmni, a unified and open-source omni-modal large language model for audio-visual multi-turn interaction, ranging from 4B to 8B parameters, designed to lead the field of lightweight models by offering comprehensive omni-modal understanding and speech generation capabilities. To achieve this, we integrate the vision encoder, audio encoder, large language model, and speech decoder into a unified model for understanding and generation tasks. We design a multi-stage training strategy to ensure robust cross-modal capabilities, including pre-training for omni-modal understanding, followed by post-training with speech conversation and audio-visual interaction. To enable human-like long-term conversational ability, we meticulously curate a multi-turn training dataset that enhances the model’s ability to handle complex and multi-turn interactions. To effectively evaluate the multi-turn memory and speech interaction capabilities, we construct the multi-modal multi-turn memory benchmark and the multi-turn speech interaction benchmark. Experiments demonstrate that InteractiveOmni significantly outperforms leading open-source models and provides a more intelligent multi-turn audio-visual experience, particularly in its long-term memory capabilities. Notably, InteractiveOmni-4B is comparable to the much larger model like Qwen2.5-Omni-7B on general benchmarks, and it can retain 97% of the performance of the InteractiveOmni-8B while utilizing only 50% of the model size. Achieving state-of-the-art results against similarly sized models across image, audio, video understanding, and speech generation tasks, InteractiveOmni is an accessible, open-source foundation for next-generation intelligent interactive systems.

Method: The paper proposes a differentiable quantization approach with convergence proof to optimal neural networks. The method supports n-bit quantization (not just 1-bit) and logarithmic quantization of values in the form 2^n. It uses shift bit quantization that doesn’t require higher precision multiplication during inference.

Result: On ImageNet with ResNet18: weight-only quantization achieves less than 1% accuracy drop compared to full precision, trained in just 15 epochs. Weight+activation quantization achieves comparable accuracy to SOTA approaches in 15 epochs, with slightly higher CPU instruction cost than 1-bit quantization but no higher precision multiplication needed.

Conclusion: The proposed differentiable quantization method with convergence proof provides efficient training (15 epochs), supports multi-bit quantization, achieves near-full-precision accuracy with weight-only quantization, and matches SOTA with weight+activation quantization while maintaining inference efficiency.

Abstract: Quantization of neural networks provides benefits of inference in less compute and memory requirements. Previous work in quantization lack two important aspects which this work provides. First almost all previous work in quantization used a non-differentiable approach and for learning; the derivative is usually set manually in backpropogation which make the learning ability of algorithm questionable, our approach is not just differentiable, we also provide proof of convergence of our approach to the optimal neural network. Second previous work in shift/logrithmic quantization either have avoided activation quantization along with weight quantization or achieved less accuracy. Learning logrithmic quantize values of form $2^n$ requires the quantization function can scale to more than 1 bit quantization which is another benifit of our quantization that it provides $n$ bits quantization as well. Our approach when tested with image classification task using imagenet dataset, resnet18 and weight quantization only achieves less than 1 percent accuracy compared to full precision accuracy while taking only 15 epochs to train using shift bit quantization and achieves comparable to SOTA approaches accuracy in both weight and activation quantization using shift bit quantization in 15 training epochs with slightly higher(only higher cpu instructions) inference cost compared to 1 bit quantization(without logrithmic quantization) and not requiring any higher precision multiplication.

Method: Comprehensive review of vision-based methods leveraging computer vision advancements (action recognition, anticipation, activity understanding) to detect procedural and executional errors, with categorization based on procedural structure use, supervision levels, and learning strategies.

Result: Provides systematic overview of existing datasets, evaluation metrics, state-of-the-art methods, and identifies key challenges like intra-class variability, viewpoint differences, compositional structures, and distinguishing permissible variations from true mistakes.

Conclusion: Establishes unified perspective on vision-based mistake analysis, highlighting its potential to enhance safety, efficiency, and task performance across domains, with future directions including neuro-symbolic reasoning and counterfactual state modeling.

Abstract: Mistake analysis in procedural activities is a critical area of research with applications spanning industrial automation, physical rehabilitation, education and human-robot collaboration. This paper reviews vision-based methods for detecting and predicting mistakes in structured tasks, focusing on procedural and executional errors. By leveraging advancements in computer vision, including action recognition, anticipation and activity understanding, vision-based systems can identify deviations in task execution, such as incorrect sequencing, use of improper techniques, or timing errors. We explore the challenges posed by intra-class variability, viewpoint differences and compositional activity structures, which complicate mistake detection. Additionally, we provide a comprehensive overview of existing datasets, evaluation metrics and state-of-the-art methods, categorizing approaches based on their use of procedural structure, supervision levels and learning strategies. Open challenges, such as distinguishing permissible variations from true mistakes and modeling error propagation are discussed alongside future directions, including neuro-symbolic reasoning and counterfactual state modeling. This work aims to establish a unified perspective on vision-based mistake analysis in procedural activities, highlighting its potential to enhance safety, efficiency and task performance across diverse domains.

Method: MambaScope first performs coarse-grained inference using large patches to reduce tokens. When prediction confidence is low, selected regions are re-processed at finer resolution to recover visual details. This dynamic resolution assignment adapts computation to image complexity.

Result: Experiments across various vision tasks show MambaScope outperforms both baseline Vision Mamba and state-of-the-art token reduction techniques in terms of accuracy and efficiency.

Conclusion: MambaScope provides an effective adaptive framework for Vision Mamba that achieves efficient processing without compromising accuracy by allocating computation according to image complexity.

Abstract: Vision Mamba has emerged as a promising and efficient alternative to Vision Transformers, yet its efficiency remains fundamentally constrained by the number of input tokens. Existing token reduction approaches typically adopt token pruning or merging to reduce computation. However, they inherently lead to information loss as they discard or compress token representations. This problem is further exacerbated when the same fine-grained token processing is uniformly applied across all images regardless of visual complexity. We observe that not all inputs require fine-grained processing: simple images can be effectively handled at a coarse resolution, while only complex ones require refinement. Based on this insight, we propose MambaScope, an adaptive framework for efficient inference for Vision Mamba. MambaScope first performs coarse-grained inference by dividing the input image into large patches, significantly reducing token length and computation. When the model’s prediction confidence is low, selected regions are re-processed at a finer resolution to recover essential visual details with minimal additional cost. This dynamic resolution assignment strategy allows MambaScope to allocate computation adaptively according to image complexity, achieving efficient processing without compromising accuracy. Experiments across various vision tasks demonstrate that MambaScope outperforms both the baseline Vision Mamba and state-of-the-art token reduction techniques in terms of accuracy and efficiency.

Method: Systematic review approach: 1) Analysis of AV sensors (camera, ultrasonic, LiDAR, Radar) and fusion strategies, 2) Structured categorization of AV datasets (ego-vehicle, infrastructure-based, cooperative), 3) Analysis of cutting-edge detection methodologies from 2D/3D pipelines to hybrid sensor fusion with transformer-driven approaches.

Result: Provides comprehensive synthesis of current capabilities in AV object detection, highlighting integration potential with LLM/VLM-driven perception frameworks and emerging transformer-based approaches.

Conclusion: Delivers a clear roadmap of current capabilities, open challenges, and future opportunities in AV object detection, emphasizing the transformative potential of emerging AI paradigms like VLMs, LLMs, and Generative AI.

Abstract: Autonomous Vehicles (AVs) are transforming the future of transportation through advances in intelligent perception, decision-making, and control systems. However, their success is tied to one core capability, reliable object detection in complex and multimodal environments. While recent breakthroughs in Computer Vision (CV) and Artificial Intelligence (AI) have driven remarkable progress, the field still faces a critical challenge as knowledge remains fragmented across multimodal perception, contextual reasoning, and cooperative intelligence. This survey bridges that gap by delivering a forward-looking analysis of object detection in AVs, emphasizing emerging paradigms such as Vision-Language Models (VLMs), Large Language Models (LLMs), and Generative AI rather than re-examining outdated techniques. We begin by systematically reviewing the fundamental spectrum of AV sensors (camera, ultrasonic, LiDAR, and Radar) and their fusion strategies, highlighting not only their capabilities and limitations in dynamic driving environments but also their potential to integrate with recent advances in LLM/VLM-driven perception frameworks. Next, we introduce a structured categorization of AV datasets that moves beyond simple collections, positioning ego-vehicle, infrastructure-based, and cooperative datasets (e.g., V2V, V2I, V2X, I2I), followed by a cross-analysis of data structures and characteristics. Ultimately, we analyze cutting-edge detection methodologies, ranging from 2D and 3D pipelines to hybrid sensor fusion, with particular attention to emerging transformer-driven approaches powered by Vision Transformers (ViTs), Large and Small Language Models (SLMs), and VLMs. By synthesizing these perspectives, our survey delivers a clear roadmap of current capabilities, open challenges, and future opportunities.

Method: A parameter-efficient framework with three stages: 1) Look - generates objective visual descriptions and candidate sets; 2) Recite - uses a lightweight 1.7B router to transform visual cues into targeted queries that trigger candidate-specific parametric knowledge; 3) Answer - performs parallel evidence alignment between descriptions and recited knowledge to select the most consistent label.

Result: Achieves state-of-the-art results on AgroBench, improving Weed Identification accuracy by 23.52% over Qwen2-VL-72B and surpassing GPT-4o without external search overhead.

Conclusion: The modular design mitigates hallucinations by transforming passive perception into active, controllable knowledge retrieval, enabling better performance in specialized domains while keeping backbone models frozen.

Abstract: Vision-Language Models (VLMs) exhibit significant performance plateaus in specialized domains like precision agriculture, primarily due to “Reasoning-Driven Hallucination” where linguistic priors override visual perception. A key bottleneck is the “Modality Gap”: visual embeddings fail to reliably activate the fine-grained expert knowledge already encoded in model parameters. We propose “Look, Recite, Then Answer,” a parameter-efficient framework that enhances VLMs via self-generated knowledge hints while keeping backbone models frozen. The framework decouples inference into three stages: (1) Look generates objective visual descriptions and candidate sets; (2) Recite employs a lightweight 1.7B router to transform visual cues into targeted queries that trigger candidate-specific parametric knowledge; (3) Answer performs parallel evidence alignment between descriptions and recited knowledge to select the most consistent label. On AgroBench, our method achieves state-of-the-art results, improving Weed Identification accuracy by 23.52% over Qwen2-VL-72B and surpassing GPT-4o without external search overhead. This modular design mitigates hallucinations by transforming passive perception into active, controllable knowledge retrieval

Method: Uses 3D priors-guided in-context learning with a conditioning architecture to activate multi-view generation capability, plus a Semantic Correspondence Alignment loss to preserve semantic alignment under limited data regimes.

Result: Substantially outperforms recent baselines in multi-view consistency, text alignment, identity similarity, and visual quality, achieving strong results with only 100 multi-view training samples.

Conclusion: MagicView successfully enables existing generative models to produce multi-view consistent identity images with explicit viewpoint control through lightweight adaptation, addressing key limitations in personalized generation.

Abstract: Recent advances in personalized generative models have demonstrated impressive capabilities in producing identity-consistent images of the same individual across diverse scenes. However, most existing methods lack explicit viewpoint control and fail to ensure multi-view consistency of generated identities. To address this limitation, we present MagicView, a lightweight adaptation framework that equips existing generative models with multi-view generation capability through 3D priors-guided in-context learning. While prior studies have shown that in-context learning preserves identity consistency across grid samples, its effectiveness in multi-view settings remains unexplored. Building upon this insight, we conduct an in-depth analysis of the multi-view in-context learning ability, and design a conditioning architecture that leverages 3D priors to activate this capability for multi-view consistent identity customization. On the other hand, acquiring robust multi-view capability typically requires large-scale multi-dimensional datasets, which makes incorporating multi-view contextual learning under limited data regimes prone to textual controllability degradation. To address this issue, we introduce a novel Semantic Correspondence Alignment loss, which effectively preserves semantic alignment while maintaining multi-view consistency. Extensive experiments demonstrate that MagicView substantially outperforms recent baselines in multi-view consistency, text alignment, identity similarity, and visual quality, achieving strong results with only 100 multi-view training samples.

Method: Two-stage training with knowledge distillation and fine-tuning, plus a Distillation Recovery Module (DRM) to align teacher-student feature spaces. Also introduces Top-Down-attention-based Deformable Aggregator (TDDA) that uses global semantic features to dynamically adjust Regions of Interest for better adaptation to irregular structures.

Result: Achieves competitive performance compared to state-of-the-art approaches while reducing parameter count by ~64.2% and FLOPs by ~62.6% compared to CricaVPR.

Conclusion: D²-VPR successfully balances performance and efficiency for VPR tasks, making foundation model capabilities more accessible for resource-constrained deployment scenarios.

Abstract: Visual Place Recognition (VPR) aims to determine the geographic location of a query image by retrieving its most visually similar counterpart from a geo-tagged reference database. Recently, the emergence of the powerful visual foundation model, DINOv2, trained in a self-supervised manner on massive datasets, has significantly improved VPR performance. This improvement stems from DINOv2’s exceptional feature generalization capabilities but is often accompanied by increased model complexity and computational overhead that impede deployment on resource-constrained devices. To address this challenge, we propose $D^{2}$-VPR, a $D$istillation- and $D$eformable-based framework that retains the strong feature extraction capabilities of visual foundation models while significantly reducing model parameters and achieving a more favorable performance-efficiency trade-off. Specifically, first, we employ a two-stage training strategy that integrates knowledge distillation and fine-tuning. Additionally, we introduce a Distillation Recovery Module (DRM) to better align the feature spaces between the teacher and student models, thereby minimizing knowledge transfer losses to the greatest extent possible. Second, we design a Top-Down-attention-based Deformable Aggregator (TDDA) that leverages global semantic features to dynamically and adaptively adjust the Regions of Interest (ROI) used for aggregation, thereby improving adaptability to irregular structures. Extensive experiments demonstrate that our method achieves competitive performance compared to state-of-the-art approaches. Meanwhile, it reduces the parameter count by approximately 64.2% and FLOPs by about 62.6% (compared to CricaVPR).Code is available at https://github.com/tony19980810/D2VPR.

Method: Studies min-Sliced Transport Plan (min-STP) framework, investigates slicer transferability theoretically, introduces minibatch formulation for scalability with statistical guarantees, and empirically tests on point cloud alignment and flow-based generative modeling.

Result: Theoretically shows optimized slicers remain close under slight distribution perturbations; empirically demonstrates transferable min-STP achieves strong one-shot matching performance and facilitates amortized training for practical applications.

Conclusion: Optimized slicers in min-STP framework are transferable across related distribution pairs, enabling efficient OT computations in evolving data scenarios while maintaining strong performance through the proposed minibatch formulation.

Abstract: Optimal Transport (OT) offers a powerful framework for finding correspondences between distributions and addressing matching and alignment problems in various areas of computer vision, including shape analysis, image generation, and multimodal tasks. The computation cost of OT, however, hinders its scalability. Slice-based transport plans have recently shown promise for reducing the computational cost by leveraging the closed-form solutions of 1D OT problems. These methods optimize a one-dimensional projection (slice) to obtain a conditional transport plan that minimizes the transport cost in the ambient space. While efficient, these methods leave open the question of whether learned optimal slicers can transfer to new distribution pairs under distributional shift. Understanding this transferability is crucial in settings with evolving data or repeated OT computations across closely related distributions. In this paper, we study the min-Sliced Transport Plan (min-STP) framework and investigate the transferability of optimized slicers: can a slicer trained on one distribution pair yield effective transport plans for new, unseen pairs? Theoretically, we show that optimized slicers remain close under slight perturbations of the data distributions, enabling efficient transfer across related tasks. To further improve scalability, we introduce a minibatch formulation of min-STP and provide statistical guarantees on its accuracy. Empirically, we demonstrate that the transferable min-STP achieves strong one-shot matching performance and facilitates amortized training for point cloud alignment and flow-based generative modeling.

Method: Trained 36 diverse models (CNNs and ViTs) with adversarial training, conducted comprehensive transferability experiments to compare attack transferability from adversarially trained vs standard models.

Result: Clear paradox: adversarially trained models produce perturbations that transfer more effectively than those from standard models, introducing new ecosystem risks.

Conclusion: Robustness evaluations should assess both resistance to transferred attacks AND propensity to produce transferable adversarial examples. All models, code, and scripts released for reproducibility.

Abstract: Deep learning has achieved great success in computer vision, but remains vulnerable to adversarial attacks. Adversarial training is the leading defense designed to improve model robustness. However, its effect on the transferability of attacks is underexplored. In this work, we ask whether adversarial training unintentionally increases the transferability of adversarial examples. To answer this, we trained a diverse zoo of 36 models, including CNNs and ViTs, and conducted comprehensive transferability experiments. Our results reveal a clear paradox: adversarially trained (AT) models produce perturbations that transfer more effectively than those from standard models, which introduce a new ecosystem risk. To enable reproducibility and further study, we release all models, code, and experimental scripts. Furthermore, we argue that robustness evaluations should assess not only the resistance of a model to transferred attacks but also its propensity to produce transferable adversarial examples.

Method: HybridWorldSim integrates multi-traversal neural reconstruction for static backgrounds with generative modeling for dynamic agents, creating a unified framework that addresses previous limitations. The authors also release MIRROR, a new multi-traversal dataset capturing diverse routes and environmental conditions across different cities.

Result: Extensive experiments show HybridWorldSim surpasses previous state-of-the-art methods, providing high-fidelity simulation with reliable visual and spatial consistency for diverse driving scenarios.

Conclusion: HybridWorldSim offers a practical and scalable solution for high-fidelity autonomous driving simulation, with the MIRROR dataset serving as a valuable resource for research and development in the field.

Abstract: Realistic and controllable simulation is critical for advancing end-to-end autonomous driving, yet existing approaches often struggle to support novel view synthesis under large viewpoint changes or to ensure geometric consistency. We introduce HybridWorldSim, a hybrid simulation framework that integrates multi-traversal neural reconstruction for static backgrounds with generative modeling for dynamic agents. This unified design addresses key limitations of previous methods, enabling the creation of diverse and high-fidelity driving scenarios with reliable visual and spatial consistency. To facilitate robust benchmarking, we further release a new multi-traversal dataset MIRROR that captures a wide range of routes and environmental conditions across different cities. Extensive experiments demonstrate that HybridWorldSim surpasses previous state-of-the-art methods, providing a practical and scalable solution for high-fidelity simulation and a valuable resource for research and development in autonomous driving.

Method: 1) Introduces MMA-Bench comprising videos and tasks probing model reliance on specific modalities. 2) Uses black-box and white-box interpretability techniques to analyze model brittleness. 3) Proposes a modality alignment tuning strategy to teach models when to prioritize, leverage, or ignore specific modality cues.

Result: Current MLLMs (both open- and closed-source) struggle under misaligned audio-visual pairs and simple misleading text, demonstrating they lack robust multi-modal reasoning. The proposed alignment tuning yields demonstrably stronger multimodal grounding.

Conclusion: This work provides interpretability tools and a clear path toward developing MLLMs with intrinsically reliable cross-modal reasoning through modality alignment tuning, addressing the brittleness identified in current models.

Abstract: Despite remarkable advancements in Multimodal Large Language Models (MLLMs), a fundamental question remains: are MLLMs robust to contradicting modalities? To rigorously study this, we introduce MMA-Bench comprising videos and tasks that probe a model’s reliance on specific modalities. Using black-box and white-box interpretability techniques, we provide a critical analysis of the brittleness of both open- and closed-sourced MLLMs. We show that current MLLMs struggle under misaligned audio-visual pairs and simple misleading text, thereby lacking robust multi-modal reasoning. Building on these findings, we propose a modality alignment tuning strategy to teach the model when to prioritize, leverage, or ignore specific modality cues. Through extensive experiments and analysis, we show that our alignment tuning yields demonstrably stronger multimodal grounding. This work provides both interpretability tools and a clear path toward developing MLLMs with intrinsically reliable cross-modal reasoning. Code and dataset will be publicly available.

Method: Uses instance segmentation to generate edge-aligned masks conforming to sign shapes, employs a U-Net generator to craft adversarial patches, and optimizes through color/texture constraints plus frequency domain analysis for seamless background integration.

Result: Achieves over 90% attack success rate across varied traffic sign classification models under limited query budgets, shows strong cross-model transferability, and maintains robust real-world performance stable under varying angles and distances.

Conclusion: TESP-Attack demonstrates that stealth-aware adversarial attacks on traffic signs are feasible and effective, highlighting vulnerabilities in intelligent driving systems while providing a more practical attack method that balances effectiveness with visual concealment.

Abstract: Intelligent driving systems are vulnerable to physical adversarial attacks on traffic signs. These attacks can cause misclassification, leading to erroneous driving decisions that compromise road safety. Moreover, within V2X networks, such misinterpretations can propagate, inducing cascading failures that disrupt overall traffic flow and system stability. However, a key limitation of current physical attacks is their lack of stealth. Most methods apply perturbations to central regions of the sign, resulting in visually salient patterns that are easily detectable by human observers, thereby limiting their real-world practicality. This study proposes TESP-Attack, a novel stealth-aware adversarial patch method for traffic sign classification. Based on the observation that human visual attention primarily focuses on the central regions of traffic signs, we employ instance segmentation to generate edge-aligned masks that conform to the shape characteristics of the signs. A U-Net generator is utilized to craft adversarial patches, which are then optimized through color and texture constraints along with frequency domain analysis to achieve seamless integration with the background environment, resulting in highly effective visual concealment. The proposed method demonstrates outstanding attack success rates across traffic sign classification models with varied architectures, achieving over 90% under limited query budgets. It also exhibits strong cross-model transferability and maintains robust real-world performance that remains stable under varying angles and distances.

Method: Retrieval-augmented prompting with domain-adapted SigLIP2 encoder to find related captions/examples, then feed to multilingual LLM (image-blind) or VLM (image-aware). Introduces graph-based PageRank re-ranking for coherence.

Result: Competitive with supervised English-only systems, generalizes to 10 languages. PageRank re-ranking yields up to 35% improvement. LLMs achieve better BLEU/CIDEr scores while VLMs produce more visually grounded captions. Direct generation outperforms translation.

Conclusion: First systematic evaluation of training-free multilingual captioning for remote sensing, advancing toward inclusive and scalable Earth observation systems without requiring large annotated datasets.

Abstract: Remote sensing image captioning has advanced rapidly through encoder–decoder models, although the reliance on large annotated datasets and the focus on English restricts global applicability. To address these limitations, we propose the first training-free multilingual approach, based on retrieval-augmented prompting. For a given aerial image, we employ a domain-adapted SigLIP2 encoder to retrieve related captions and few-shot examples from a datastore, which are then provided to a language model. We explore two variants: an image-blind setup, where a multilingual Large Language Model (LLM) generates the caption from textual prompts alone, and an image-aware setup, where a Vision–Language Model (VLM) jointly processes the prompt and the input image. To improve the coherence of the retrieved content, we introduce a graph-based re-ranking strategy using PageRank on a graph of images and captions. Experiments on four benchmark datasets across ten languages demonstrate that our approach is competitive with fully supervised English-only systems and generalizes to other languages. Results also highlight the importance of re-ranking with PageRank, yielding up to 35% improvements in performance metrics. Additionally, it was observed that while VLMs tend to generate visually grounded but lexically diverse captions, LLMs can achieve stronger BLEU and CIDEr scores. Lastly, directly generating captions in the target language consistently outperforms other translation-based strategies. Overall, our work delivers one of the first systematic evaluations of multilingual, training-free captioning for remote sensing imagery, advancing toward more inclusive and scalable multimodal Earth observation systems.

Method: Training-free framework that takes reference images (can be from text-to-image models), performs instance segmentation/completion, reconstructs 3D models with canonical alignment, then uses a two-stage pose/scale estimation (Differentiable Rotation Optimizer + Top-view Spatial Alignment) to assemble collision-free, simulation-ready scenes.

Result: State-of-the-art performance surpassing existing methods in visual fidelity, layout accuracy, and physical plausibility, capable of generating realistic tabletop scenes with rich stylistic and spatial diversity.

Conclusion: TabletopGen effectively addresses tabletop scene generation challenges, providing high-fidelity, physically interactive 3D scenes suitable for embodied AI applications, with code to be publicly released.

Abstract: Generating high-fidelity, physically interactive 3D simulated tabletop scenes is essential for embodied AI–especially for robotic manipulation policy learning and data synthesis. However, current text- or image-driven 3D scene generation methods mainly focus on large-scale scenes, struggling to capture the high-density layouts and complex spatial relations that characterize tabletop scenes. To address these challenges, we propose TabletopGen, a training-free, fully automatic framework that generates diverse, instance-level interactive 3D tabletop scenes. TabletopGen accepts a reference image as input, which can be synthesized by a text-to-image model to enhance scene diversity. We then perform instance segmentation and completion on the reference to obtain per-instance images. Each instance is reconstructed into a 3D model followed by canonical coordinate alignment. The aligned 3D models then undergo pose and scale estimation before being assembled into a collision-free, simulation-ready tabletop scene. A key component of our framework is a novel pose and scale alignment approach that decouples the complex spatial reasoning into two stages: a Differentiable Rotation Optimizer for precise rotation recovery and a Top-view Spatial Alignment mechanism for robust translation and scale estimation, enabling accurate 3D reconstruction from 2D reference. Extensive experiments and user studies show that TabletopGen achieves state-of-the-art performance, markedly surpassing existing methods in visual fidelity, layout accuracy, and physical plausibility, capable of generating realistic tabletop scenes with rich stylistic and spatial diversity. Our code will be publicly available.

Method: Created ViRectify benchmark with AI-assisted annotation pipeline and human verification. Proposed trajectory evidence-driven correction framework with step-wise error trajectory and reward modeling on visual evidence-grounded correction.

Result: Evaluation of 16 advanced MLLMs shows ViRectify is challenging (GPT-5 achieves only 31.94% accuracy). Their framework enables Qwen2.5-VL-7B to outperform 72B variants, revealing systematic asymmetries in error correction across models.

Conclusion: ViRectify provides a new direction for comprehensive evaluation of advanced MLLMs in video reasoning and serves as a valuable data resource for reflection learning.

Abstract: As multimodal large language models (MLLMs) frequently exhibit errors in complex video reasoning scenarios, correcting these errors is critical for uncovering their weaknesses and improving performance. However, existing benchmarks lack systematic evaluation of MLLMs’ ability to identify and correct these video reasoning errors. To bridge this gap, we propose \textit{ViRectify}, a comprehensive benchmark to evaluate their fine-grained correction capability. Through an AI-assisted annotation pipeline with human verification, we construct a dataset of over 30\textit{K} instances spanning dynamic perception, scientific reasoning, and embodied decision-making domains. In \textit{ViRectify}, we challenge MLLMs to perform step-wise error identification and generate rationales with key video evidence grounding. In addition, we further propose the trajectory evidence-driven correction framework, comprising step-wise error trajectory and reward modeling on visual evidence-grounded correction. It encourages the model to explicitly concentrate on error propagation and key timestamps for correction. Extensive evaluation across 16 advanced MLLMs demonstrates that our \textit{ViRectify} serves as a challenging testbed, where GPT-5 achieves only 31.94% correction accuracy. Our framework enables a Qwen2.5-VL-7B to consistently outperform the variants of 72B on \textit{ViRectify}, showing the effectiveness of our approach. Further analysis uncovers systematic asymmetries in error correction across models, and our dataset is also a valuable data resource to perform reflection learning. We believe \textit{ViRectify} provides a new direction for comprehensively evaluating the advanced MLLMs in video reasoning.

Method: Learn a latent space from real-world human actions by fusing appearance-agnostic skeletal geometry features with appearance-based features. Then measure video quality by computing distance between generated video representations and this learned real-world action distribution.

Result: The metric achieves over 68% improvement compared to state-of-the-art methods on their new benchmark, performs competitively on external benchmarks, and shows stronger correlation with human perception.

Conclusion: The method establishes a new standard for video generation evaluation, reveals critical limitations in current video generative models, and provides a robust way to assess human action fidelity in generated videos.

Abstract: Despite rapid advances in video generative models, robust metrics for evaluating visual and temporal correctness of complex human actions remain elusive. Critically, existing pure-vision encoders and Multimodal Large Language Models (MLLMs) are strongly appearance-biased, lack temporal understanding, and thus struggle to discern intricate motion dynamics and anatomical implausibilities in generated videos. We tackle this gap by introducing a novel evaluation metric derived from a learned latent space of real-world human actions. Our method first captures the nuances, constraints, and temporal smoothness of real-world motion by fusing appearance-agnostic human skeletal geometry features with appearance-based features. We posit that this combined feature space provides a robust representation of action plausibility. Given a generated video, our metric quantifies its action quality by measuring the distance between its underlying representations and this learned real-world action distribution. For rigorous validation, we develop a new multi-faceted benchmark specifically designed to probe temporally challenging aspects of human action fidelity. Through extensive experiments, we show that our metric achieves substantial improvement of more than 68% compared to existing state-of-the-art methods on our benchmark, performs competitively on established external benchmarks, and has a stronger correlation with human perception. Our in-depth analysis reveals critical limitations in current video generative models and establishes a new standard for advanced research in video generation.

Method: Proposes Visual LIF (VLIF) neuron to overcome spatial context limitations. Introduces Spiking Decomposition and Enhancement Module and lightweight Spiking Multi-scale Unit using VLIF for hierarchical multi-scale representation learning in image deraining.

Result: Extensive experiments across five benchmark deraining datasets show the approach significantly outperforms state-of-the-art SNN-based deraining methods while using only 13% of their energy consumption.

Conclusion: The findings establish a solid foundation for deploying SNNs in high-performance, energy-efficient low-level vision tasks, demonstrating the potential of biologically plausible frameworks for practical computer vision applications.

Abstract: Biologically plausible and energy-efficient frameworks such as Spiking Neural Networks (SNNs) have not been sufficiently explored in low-level vision tasks. Taking image deraining as an example, this study addresses the representation of the inherent high-pass characteristics of spiking neurons, specifically in image deraining and innovatively proposes the Visual LIF (VLIF) neuron, overcoming the obstacle of lacking spatial contextual understanding present in traditional spiking neurons. To tackle the limitation of frequency-domain saturation inherent in conventional spiking neurons, we leverage the proposed VLIF to introduce the Spiking Decomposition and Enhancement Module and the lightweight Spiking Multi-scale Unit for hierarchical multi-scale representation learning. Extensive experiments across five benchmark deraining datasets demonstrate that our approach significantly outperforms state-of-the-art SNN-based deraining methods, achieving this superior performance with only 13% of their energy consumption. These findings establish a solid foundation for deploying SNNs in high-performance, energy-efficient low-level vision tasks.

Method: Introduces AVFullDiT, a parameter-efficient architecture leveraging pre-trained T2V and T2A modules for joint denoising. Trains both a T2AV model with AVFullDiT and a T2V-only counterpart under identical settings for comparison.

Result: Audio-video joint denoising consistently improves video quality on challenging subsets with large and object contact motions. Audio acts as a privileged signal that helps the model internalize causal relationships between visual events and acoustic consequences.

Conclusion: Cross-modal co-training (audio-video joint denoising) is a promising approach for developing stronger, more physically grounded world models that go beyond just achieving synchrony.

Abstract: Recent audio-video generative systems suggest that coupling modalities benefits not only audio-video synchrony but also the video modality itself. We pose a fundamental question: Does audio-video joint denoising training improve video generation, even when we only care about video quality? To study this, we introduce a parameter-efficient Audio-Video Full DiT (AVFullDiT) architecture that leverages pre-trained text-to-video (T2V) and text-to-audio (T2A) modules for joint denoising. We train (i) a T2AV model with AVFullDiT and (ii) a T2V-only counterpart under identical settings. Our results provide the first systematic evidence that audio-video joint denoising can deliver more than synchrony. We observe consistent improvements on challenging subsets featuring large and object contact motions. We hypothesize that predicting audio acts as a privileged signal, encouraging the model to internalize causal relationships between visual events and their acoustic consequences (e.g., collision $\times$ impact sound), which in turn regularizes video dynamics. Our findings suggest that cross-modal co-training is a promising approach to developing stronger, more physically grounded world models. Code and dataset will be made publicly available.

Method: Two-stage training strategy: 1) Self-supervised pre-training on a colorization task to learn general visual representations, 2) Fine-tuning for 4x super-resolution using residual learning approach where the model predicts a high-frequency residual image added to initial bicubic interpolation.

Result: On DIV2K benchmark dataset, ViT-SR achieves SSIM of 0.712 and PSNR of 22.90 dB, demonstrating strong performance for 4x super-resolution.

Conclusion: The two-stage approach with self-supervised pre-training is effective for complex image restoration tasks. Future improvements could come from larger ViT architectures or alternative pretext tasks.

Abstract: In computer vision, Single Image Super-Resolution (SISR) is still a difficult problem. We present ViT-SR, a new technique to improve the performance of a Vision Transformer (ViT) employing a two-stage training strategy. In our method, the model learns rich, generalizable visual representations from the data itself through a self-supervised pretraining phase on a colourization task. The pre-trained model is then adjusted for 4x super-resolution. By predicting the addition of a high-frequency residual image to an initial bicubic interpolation, this design simplifies residual learning. ViT-SR, trained and evaluated on the DIV2K benchmark dataset, achieves an impressive SSIM of 0.712 and PSNR of 22.90 dB. These results demonstrate the efficacy of our two-stage approach and highlight the potential of self-supervised pre-training for complex image restoration tasks. Further improvements may be possible with larger ViT architectures or alternative pretext tasks.

Method: Three key innovations: 1) Information density-based data generation with dual-dimensional tree-structured labels for automated cross-domain data creation; 2) Collaborative curriculum supervised fine-tuning balancing domain knowledge injection with general capability preservation; 3) Hybrid reinforcement learning for enhanced reasoning with multi-objective optimization (diversity, multimodal perception, conciseness). Plus infrastructure optimizations like 5D parallel training and inference quantization.

Result: Outperforms SOTA models on benchmarks including MMBench, MMStar, MathVision, and MathVista. Shows superior user experience in vertical domain tasks. Achieves high performance at low cost.

Conclusion: MindGPT-4ov provides a general post-training paradigm applicable to various MLLMs, with open-sourced weights, datasets, and code to support community development, bridging academic research and industrial deployment.

Abstract: We present MindGPT-4ov, a multimodal large language model (MLLM) that introduces a general post-training paradigm spanning data production, model training, and efficient deployment. It achieves state-of-the-art performance across multiple benchmarks at low cost, effectively enhancing the foundational capabilities of MLLMs and the generalization ability. Focusing on data construction, supervised fine-tuning strategies, and multimodal reinforcement learning methods, this work proposes three key innovations: (1) An information density-based data generation scheme, integrated with a dual-dimensional tree-structured label system, enabling automated generation of high-quality cross-domain data. (2) A collaborative curriculum supervised fine-tuning approach that balances the injection of domain-specific knowledge with the preservation of general capabilities. (3) A hybrid reinforcement learning paradigm that enhances reasoning ability while simultaneously addressing multi-objective optimization such as diversity exploration, maintenance of multimodal perception, and response conciseness. Moreover, we implement a series of infrastructure optimizations, such as 5D parallel training, operator optimization, and inference quantization to enhance training and inference efficiency while reducing the cost of domain adaptation. Experimental results demonstrate that the MindGPT-4ov model outperforms state-of-the-art models on benchmarks such as MMBench, MMStar, MathVision, and MathVista. In addition, MindGPT-4ov also demonstrates superior user experience in vertical domain tasks, enabling a seamless transition from academic research to industrial deployment. MindGPT-4ov provides a general post-training paradigm applicable to a wide range of MLLMs. The model weights, datasets, and code for the Qwen3-VL-based variants will be recently open-sourced to support the community’s development of MLLMs.

Method: Uses a dataset construction pipeline with image-to-video translation, optical flow-based mask propagation, and video inpainting. Core innovation is a learnable object-aware localization mechanism with Diffusion Mask Predictor for end-to-end editing without external control signals.

Result: Extensive experiments and human evaluation demonstrate LoVoRA’s effectiveness and high-quality performance in mask-free video object removal and addition.

Conclusion: LoVoRA provides a scalable, mask-free solution for video object editing with precise spatio-temporal consistency, advancing text-guided video editing capabilities.

Abstract: Text-guided video editing, particularly for object removal and addition, remains a challenging task due to the need for precise spatial and temporal consistency. Existing methods often rely on auxiliary masks or reference images for editing guidance, which limits their scalability and generalization. To address these issues, we propose LoVoRA, a novel framework for mask-free video object removal and addition using object-aware localization mechanism. Our approach utilizes a unique dataset construction pipeline that integrates image-to-video translation, optical flow-based mask propagation, and video inpainting, enabling temporally consistent edits. The core innovation of LoVoRA is its learnable object-aware localization mechanism, which provides dense spatio-temporal supervision for both object insertion and removal tasks. By leveraging a Diffusion Mask Predictor, LoVoRA achieves end-to-end video editing without requiring external control signals during inference. Extensive experiments and human evaluation demonstrate the effectiveness and high-quality performance of LoVoRA. https://cz-5f.github.io/LoVoRA.github.io

Method: Uses large vision, geometric, and multimodal models to interpret metric-scale static geometry, real-world dynamic motion, instance-level masks, and holistic descriptive captions. Integrates window-based Bundle Adjustment with global optimization to convert long real-world video sequences into comprehensive 4D multimodal format.

Result: Creates a large-scale dataset with 100K+ videos, 800K+ annotated masks, and 10M+ frames from internet videos. Experimental evaluations on video depth estimation, camera pose estimation, and camera intrinsics estimation show superior performance in capturing physical-scale measurements with greater global accuracy than existing methods.

Conclusion: DynamicVerse bridges the gap between limited existing datasets and the need for comprehensive physical-scale 4D world modeling, enabling foundation models to better interpret real-world dynamics from monocular videos and advancing human-agent interaction capabilities.

Abstract: Understanding the dynamic physical world, characterized by its evolving 3D structure, real-world motion, and semantic content with textual descriptions, is crucial for human-agent interaction and enables embodied agents to perceive and act within real environments with human-like capabilities. However, existing datasets are often derived from limited simulators or utilize traditional Structurefrom-Motion for up-to-scale annotation and offer limited descriptive captioning, which restricts the capacity of foundation models to accurately interpret real-world dynamics from monocular videos, commonly sourced from the internet. To bridge these gaps, we introduce DynamicVerse, a physical-scale, multimodal 4D world modeling framework for dynamic real-world video. We employ large vision, geometric, and multimodal models to interpret metric-scale static geometry, real-world dynamic motion, instance-level masks, and holistic descriptive captions. By integrating window-based Bundle Adjustment with global optimization, our method converts long real-world video sequences into a comprehensive 4D multimodal format. DynamicVerse delivers a large-scale dataset consisting of 100K+ videos with 800K+ annotated masks and 10M+ frames from internet videos. Experimental evaluations on three benchmark tasks, namely video depth estimation, camera pose estimation, and camera intrinsics estimation, demonstrate that our 4D modeling achieves superior performance in capturing physical-scale measurements with greater global accuracy than existing methods.

Method: Proposes OneThinker with: 1) OneThinker-600k training corpus covering diverse visual tasks, 2) Commercial models for CoT annotation creating OneThinker-SFT-340k for SFT cold start, 3) EMA-GRPO to handle reward heterogeneity in multi-task RL by tracking task-wise moving averages of reward standard deviations.

Result: Strong performance on 31 benchmarks across 10 fundamental visual understanding tasks, with effective knowledge transfer between certain tasks and preliminary zero-shot generalization ability.

Conclusion: OneThinker marks a step toward a unified multimodal reasoning generalist, demonstrating the feasibility of a single model handling diverse visual reasoning tasks across both images and videos.

Abstract: Reinforcement learning (RL) has recently achieved remarkable success in eliciting visual reasoning within Multimodal Large Language Models (MLLMs). However, existing approaches typically train separate models for different tasks and treat image and video reasoning as disjoint domains. This results in limited scalability toward a multimodal reasoning generalist, which restricts practical versatility and hinders potential knowledge sharing across tasks and modalities. To this end, we propose OneThinker, an all-in-one reasoning model that unifies image and video understanding across diverse fundamental visual tasks, including question answering, captioning, spatial and temporal grounding, tracking, and segmentation. To achieve this, we construct the OneThinker-600k training corpus covering all these tasks and employ commercial models for CoT annotation, resulting in OneThinker-SFT-340k for SFT cold start. Furthermore, we propose EMA-GRPO to handle reward heterogeneity in multi-task RL by tracking task-wise moving averages of reward standard deviations for balanced optimization. Extensive experiments on diverse visual benchmarks show that OneThinker delivers strong performance on 31 benchmarks, across 10 fundamental visual understanding tasks. Moreover, it exhibits effective knowledge transfer between certain tasks and preliminary zero-shot generalization ability, marking a step toward a unified multimodal reasoning generalist. All code, model, and data are released.

Method: Proposes a philosophical framework with three components: 1) Specification trap argument against content-based approaches, 2) Syntropy as an information-theoretic framework for multi-agent alignment, 3) Functional distinction between genuine and simulated moral capacity based on compatibilist theories.

Result: Develops a theoretical framework that generates specific, falsifiable predictions about value emergence and moral agency in artificial systems, though empirical validation remains pending.

Conclusion: AI alignment should be reconceived as architecting syntropic, reasons-responsive agents through process-based developmental mechanisms rather than encoding fixed human values, representing a philosophical foundation for an empirical research program.

Abstract: I argue that AI alignment should be reconceived as architecting syntropic, reasons-responsive agents through process-based, multi-agent, developmental mechanisms rather than encoding fixed human value content. The paper makes three philosophical contributions. First, I articulate the ``specification trap’’ argument demonstrating why content-based value specification appears structurally unstable due to the conjunction of the is-ought gap, value pluralism, and the extended frame problem. Second, I propose syntropy – the recursive reduction of mutual uncertainty between agents through state alignment – as an information-theoretic framework for understanding multi-agent alignment dynamics. Third, I establish a functional distinction between genuine and simulated moral capacity grounded in compatibilist theories of guidance control, coupled with an embodied experimental paradigm and verification regime providing operational criteria independent of phenomenological claims. This paper represents the philosophical component of a broader research program whose empirical validation is being developed in a separate project currently in preparation. While the framework generates specific, falsifiable predictions about value emergence and moral agency in artificial systems, empirical validation remains pending.

Method: Deconstructs cognition into indivisible Logical Atoms and two fundamental operations (Pointing and Comparison). Formalizes decision-making through an interpretable Weight-Calculation model (Weight = Benefit * Probability) with traceable Initial Weights. Implemented via graph-algorithm-based computational engine and global workspace workflow.

Result: The architecture achieves transparent, human-like reasoning and robust learning in unprecedented scenarios, demonstrating radical explainability, intrinsic generality for novel situations, and traceable value alignment.

Conclusion: Weight-Calculatism establishes a practical and theoretical foundation for building trustworthy and aligned AGI, offering a viable pathway toward Artificial General Intelligence with fundamental improvements over current AI paradigms.

Abstract: Current AI paradigms, as “architects of experience,” face fundamental challenges in explainability and value alignment. This paper introduces “Weight-Calculatism,” a novel cognitive architecture grounded in first principles, and demonstrates its potential as a viable pathway toward Artificial General Intelligence (AGI). The architecture deconstructs cognition into indivisible Logical Atoms and two fundamental operations: Pointing and Comparison. Decision-making is formalized through an interpretable Weight-Calculation model (Weight = Benefit * Probability), where all values are traceable to an auditable set of Initial Weights. This atomic decomposition enables radical explainability, intrinsic generality for novel situations, and traceable value alignment. We detail its implementation via a graph-algorithm-based computational engine and a global workspace workflow, supported by a preliminary code implementation and scenario validation. Results indicate that the architecture achieves transparent, human-like reasoning and robust learning in unprecedented scenarios, establishing a practical and theoretical foundation for building trustworthy and aligned AGI.

Method: Experimental investigation comparing conventional long-chain-of-thought approaches with symbolic-solver-integrated methods. The latter leverages LLMs’ code generation capabilities to translate reasoning tasks into executable code, then uses symbolic solvers to find solutions.

Result: Symbolic solvers only help when problems require limited implicit reasoning but involve ample search space. Latest LLMs like GPT-4o perform better on deductive problems with shallow reasoning depth, while symbolic-solver-integrated methods significantly improve performance on constraint satisfaction problems requiring repeated backtracks. With declarative exemplars, even CodeLlama-13B can outperform GPT-4o on difficult Zebra puzzles.

Conclusion: The effectiveness of symbolic-solver integration depends on problem characteristics - it’s most beneficial for constraint satisfaction problems with large search spaces rather than all types of reasoning tasks. This provides guidance for when to use which approach.

Abstract: Large Reasoning Models (LRMs) achieve strong performance on complex reasoning tasks by generating long Chains of Thought (CoTs). However, this paradigm might incur substantial token overhead, especially when models “overthink” by producing lengthy reasoning chains, which can even lead to incorrect answers. A promising direction is the symbolic-solver-integrated approach, which leverages the code generation capabilities of LLMs to translate reasoning tasks into executable code and then solve them with a symbolic solver. In this paper, we explore an open question of when the conventional long-CoT can be enhanced by symbolic solvers. Our experimental results show that the symbolic-solver-integrated method only helps when the problem requires limited implicit reasoning but involves an ample search space. The latest LLMs, like GPT-4o, show better performance on deductive problems with shallow reasoning depth, while the symbolic-solver-integrated method significantly improves the LLMs’ performance in constraint satisfaction problems that require repeated backtracks. When a declarative exemplar is provided, even CodeLlama-13B can outperform GPT-4o in difficult Zebra puzzles.

Method: Two-stage framework: 1) Adaptive Supervised Fine-Tuning (SFT) with reasoning-augmented data for fundamental reasoning capability; 2) Adaptive Reinforcement Learning (GRPO) to optimize reasoning behaviors based on task complexity and reward feedback.

Result: Significantly improves adaptive reasoning performance compared to previous baselines across text-only, text-audio, text-visual, and text-audio-visual modalities.

Conclusion: Omni-AutoThink enables dynamic reasoning depth adjustment for multimodal models, addressing rigid reasoning limitations and improving performance on tasks of varying complexity.

Abstract: Recent advances in Omni models have enabled unified multimodal perception and generation. However, most existing systems still exhibit rigid reasoning behaviors, either overthinking simple problems or failing to reason when necessary. To address this limitation, we propose Omni-AutoThink, a novel adaptive reasoning framework that dynamically adjusts the model’s reasoning depth according to task difficulty. Our framework comprises two stages: (1) an Adaptive Supervised Fine-Tuning (Adaptive SFT) stage, which endows the Omni model with fundamental reasoning capability using large-scale reasoning-augmented data, and (2) an Adaptive Reinforcement Learning (Adaptive GRPO) stage, which optimizes reasoning behaviors based on task complexity and reward feedback. We further construct a comprehensive adaptive reasoning benchmark that spans text-only, text-audio, text-visual, and text-audio-visual modalities, providing both training and evaluation splits for multimodal reasoning assessment. Experimental results demonstrate that our proposed framework significantly improves adaptive reasoning performance compared to previous baselines. All benchmark data and code will be publicly released.

Method: Four preference distribution definitions are considered: hard/soft goals with/without goal shaping (intermediate goals). Four agents with these specifications are compared in a grid world navigation task.

Result: Goal shaping enables the best overall performance (promotes exploitation) but sacrifices learning about the environment’s transition dynamics (hampers exploration).

Conclusion: The specification of preference distributions significantly impacts active inference agent performance, with goal shaping improving exploitation at the cost of exploration, highlighting a trade-off in preference distribution design.

Abstract: Active inference proposes expected free energy as an objective for planning and decision-making to adequately balance exploitative and explorative drives in learning agents. The exploitative drive, or what an agent wants to achieve, is formalised as the Kullback-Leibler divergence between a variational probability distribution, updated at each inference step, and a preference probability distribution that indicates what states or observations are more likely for the agent, hence determining the agent’s goal in a certain environment. In the literature, the questions of how the preference distribution should be specified and of how a certain specification impacts inference and learning in an active inference agent have been given hardly any attention. In this work, we consider four possible ways of defining the preference distribution, either providing the agents with hard or soft goals and either involving or not goal shaping (i.e., intermediate goals). We compare the performances of four agents, each given one of the possible preference distributions, in a grid world navigation task. Our results show that goal shaping enables the best performance overall (i.e., it promotes exploitation) while sacrificing learning about the environment’s transition dynamics (i.e., it hampers exploration).

Method: The authors introduce an evaluation method using Concordia, a natural language multi-agent simulation environment. They test agents’ ability to identify and exploit opportunities for mutual gain across diverse partners and contexts, measuring general cooperative intelligence. The method was applied in the NeurIPS 2024 Concordia Contest with diverse scenarios ranging from negotiation to collective action problems.

Result: The empirical results from the NeurIPS 2024 Concordia Contest reveal significant gaps between current agent capabilities and the robust generalization required for reliable cooperation. Agents particularly struggle in scenarios demanding persuasion and norm enforcement.

Conclusion: Current LLM-based agents lack the robust generalization capabilities needed for reliable cooperation in novel social situations, especially in complex scenarios requiring persuasion and norm enforcement. The Concordia evaluation framework provides a valuable method for assessing these cooperative intelligence gaps.

Abstract: Large Language Model (LLM) agents have demonstrated impressive capabilities for social interaction and are increasingly being deployed in situations where they might engage with both human and artificial agents. These interactions represent a critical frontier for LLM-based agents, yet existing evaluation methods fail to measure how well these capabilities generalize to novel social situations. In this paper, we introduce a method for evaluating the ability of LLM-based agents to cooperate in zero-shot, mixed-motive environments using Concordia, a natural language multi-agent simulation environment. Our method measures general cooperative intelligence by testing an agent’s ability to identify and exploit opportunities for mutual gain across diverse partners and contexts. We present empirical results from the NeurIPS 2024 Concordia Contest, where agents were evaluated on their ability to achieve mutual gains across a suite of diverse scenarios ranging from negotiation to collective action problems. Our findings reveal significant gaps between current agent capabilities and the robust generalization required for reliable cooperation, particularly in scenarios demanding persuasion and norm enforcement.

Method: Argos (Agentic Reward for Grounded & Objective Scoring) is a principled reward agent that selects from a pool of teacher-model derived and rule-based scoring functions to simultaneously evaluate: (1) final response accuracy, (2) spatiotemporal localization of referred entities and actions, and (3) reasoning process quality.

Result: Achieves state-of-the-art results across multiple agentic tasks including spatial reasoning, visual hallucination, robotics, and embodied AI benchmarks. Demonstrates that SFT post-training alone is insufficient (agents collapse to ungrounded solutions during RL without online verification) and reduces reward-hacking in MMRL.

Conclusion: Argos provides effective agentic verification through principled reward selection, enabling better multimodal reasoning training. The approach is theoretically justified through pareto-optimality concepts and shows that online verification during RL is critical for maintaining grounded solutions.

Abstract: Agentic reasoning models trained with multimodal reinforcement learning (MMRL) have become increasingly capable, yet they are almost universally optimized using sparse, outcome-based rewards computed based on the final answers. Richer rewards computed from the reasoning tokens can improve learning significantly by providing more fine-grained guidance. However, it is challenging to compute more informative rewards in MMRL beyond those based on outcomes since different samples may require different scoring functions and teacher models may provide noisy reward signals too. In this paper, we introduce the Argos (Agentic Reward for Grounded & Objective Scoring), a principled reward agent to train multimodal reasoning models for agentic tasks. For each sample, Argos selects from a pool of teacher-model derived and rule-based scoring functions to simultaneously evaluate: (i) final response accuracy, (ii) spatiotemporal localization of referred entities and actions, and (iii) the quality of the reasoning process. We find that by leveraging our agentic verifier across both SFT data curation and RL training, our model achieves state-of-the-art results across multiple agentic tasks such as spatial reasoning, visual hallucination as well as robotics and embodied AI benchmarks. Critically, we demonstrate that just relying on SFT post-training on highly curated reasoning data is insufficient, as agents invariably collapse to ungrounded solutions during RL without our online verification. We also show that our agentic verifier can help to reduce reward-hacking in MMRL. Finally, we also provide a theoretical justification for the effectiveness of Argos through the concept of pareto-optimality.

Method: Proposes a generalized communication-constrained model, uses learning prior to distinguish lossy/lossless messages, decouples their impact via dual mutual information estimator, and quantifies message impact into global reward.

Result: Validated effectiveness across several communication-constrained benchmarks.

Conclusion: The proposed framework addresses lossy communication challenges in MARL for real-world applications.

Abstract: Communication is one of the effective means to improve the learning of cooperative policy in multi-agent systems. However, in most real-world scenarios, lossy communication is a prevalent issue. Existing multi-agent reinforcement learning with communication, due to their limited scalability and robustness, struggles to apply to complex and dynamic real-world environments. To address these challenges, we propose a generalized communication-constrained model to uniformly characterize communication conditions across different scenarios. Based on this, we utilize it as a learning prior to distinguish between lossy and lossless messages for specific scenarios. Additionally, we decouple the impact of lossy and lossless messages on distributed decision-making, drawing on a dual mutual information estimatior, and introduce a communication-constrained multi-agent reinforcement learning framework, quantifying the impact of communication messages into the global reward. Finally, we validate the effectiveness of our approach across several communication-constrained benchmarks.

Method: ERF-BA-TFD+ combines enhanced receptive field (ERF) with audio-visual fusion to process both audio and video features simultaneously. The key innovation is modeling long-range dependencies within audio-visual inputs to capture subtle discrepancies between real and fake content.

Result: The model achieved state-of-the-art results on the DDL-AV dataset (containing both segmented and full-length video clips), outperforming existing techniques in both accuracy and processing speed. It won first place in the “Workshop on Deepfake Detection, Localization, and Interpretability,” Track 2: Audio-Visual Detection and Localization competition.

Conclusion: ERF-BA-TFD+ demonstrates effective multimodal deepfake detection by leveraging complementary audio-visual information and modeling long-range dependencies, providing a robust solution for real-world deepfake detection scenarios.

Abstract: Deepfake detection is a critical task in identifying manipulated multimedia content. In real-world scenarios, deepfake content can manifest across multiple modalities, including audio and video. To address this challenge, we present ERF-BA-TFD+, a novel multimodal deepfake detection model that combines enhanced receptive field (ERF) and audio-visual fusion. Our model processes both audio and video features simultaneously, leveraging their complementary information to improve detection accuracy and robustness. The key innovation of ERF-BA-TFD+ lies in its ability to model long-range dependencies within the audio-visual input, allowing it to better capture subtle discrepancies between real and fake content. In our experiments, we evaluate ERF-BA-TFD+ on the DDL-AV dataset, which consists of both segmented and full-length video clips. Unlike previous benchmarks, which focused primarily on isolated segments, the DDL-AV dataset allows us to assess the model’s performance in a more comprehensive and realistic setting. Our method achieves state-of-the-art results on this dataset, outperforming existing techniques in terms of both accuracy and processing speed. The ERF-BA-TFD+ model demonstrated its effectiveness in the “Workshop on Deepfake Detection, Localization, and Interpretability,” Track 2: Audio-Visual Detection and Localization (DDL-AV), and won first place in this competition.

Method: Hierarchical multi-agent architecture with task planning, execution, and self-assessment/feedback mechanisms that evaluate actions and outcomes from independent context to detect and correct strategic errors.

Result: Successfully reproduced materials science results on lithium-ion conduction and alloy segregation, coordinating dozens of parallel simulations (43 hours each). In Kaggle tasks, produced competitive solutions from minimal natural-language instructions.

Conclusion: Hierarchical multi-agent systems with self-assessment and self-feedback enable AI systems to perform independent, large-scale scientific and analytical work, demonstrating potential for autonomous computational task execution.

Abstract: We introduce PARC, a coding agent for the autonomous and robust execution of long-horizon computational tasks. PARC is built on a hierarchical multi-agent architecture incorporating task planning, execution, and a mechanism that evaluates its own actions and their outcomes from an independent context and provides feedback, namely self-assessment and self-feedback. This design enables PARC to detect and correct high-level strategic errors and sustain progress without human intervention. We evaluate PARC across computational science and data science tasks. In materials science, it autonomously reproduces key results from studies on lithium-ion conduction and alloy segregation. In particular, it coordinates dozens of parallel simulation tasks, each requiring roughly 43 hours of computation, managing orchestration, monitoring, and error correction end-to-end. In Kaggle-based experiments, starting from minimal natural-language instructions, PARC conducts data analysis and implements search strategies, producing solutions competitive with human-engineered baselines. These results highlight the potential of integrating a hierarchical multi-agent system with self-assessment and self-feedback to enable AI systems capable of independent, large-scale scientific and analytical work.

Method: RP-ReAct uses a multi-agent approach with a Reasoner Planner Agent (RPA) for strategic planning and analysis using Large Reasoning Models, and Proxy-Execution Agents (PEA) that translate sub-steps into tool interactions using ReAct. Includes context-saving strategy to manage large tool outputs via external storage.

Result: RP-ReAct achieves superior performance on ToolQA benchmark across six open-weight reasoning models, showing improved generalization, robustness, stability across model scales, and better handling of diverse complex tasks.

Conclusion: RP-ReAct’s decoupled multi-agent architecture provides more effective and deployable enterprise agentic solutions by addressing key limitations of single-agent systems while maintaining data privacy through local open-weight models.

Abstract: Despite recent advances, autonomous agents often struggle to solve complex tasks in enterprise domains that require coordinating multiple tools and processing diverse data sources. This struggle is driven by two main limitations. First, single-agent architectures enforce a monolithic plan-execute loop, which directly causes trajectory instability. Second, the requirement to use local open-weight models for data privacy introduces smaller context windows leading to the rapid consumption of context from large tool outputs. To solve this problem we introduce RP-ReAct (Reasoner Planner-ReAct), a novel multi-agent approach that fundamentally decouples strategic planning from low-level execution to achieve superior reliability and efficiency. RP-ReAct consists of a Reasoner Planner Agent (RPA), responsible for planning each sub-step, continuously analysing the execution results using the strong reasoning capabilities of a Large Reasoning Model, and one or multiple Proxy-Execution Agent (PEA) that translates sub-steps into concrete tool interactions using a ReAct approach. Crucially, we incorporate a context-saving strategy within the PEA to mitigate context window overflow by managing large tool outputs via external storage and on-demand access. We evaluate RP-ReAct, on the challenging, multi-domain ToolQA benchmark using a diverse set of six open-weight reasoning models. Our empirical results show that RP-ReAct achieves superior performance and improved generalization ability over state-of-the-art baselines when addressing diverse complex tasks across the evaluated domains. Furthermore we establish the enhanced robustness and stability of our approach across different model scales, paving the way for effective and deployable agentic solutions for enterprises.

Method: Introduces “probabilistic angelic nondeterminism” (PAN) programming model that separates workflow logic from inference strategies. Implemented as EnCompass framework in Python using decorators to compile agent workflows into search spaces.

Result: Three case studies demonstrate that the framework allows programmers to quickly improve agent reliability and easily switch between different inference-time strategies with minimal additional coding.

Conclusion: PAN provides a practical programming model for LLM-based agents that disentangles workflow design from inference strategies, enabling more flexible experimentation and reliability improvements in agent development.

Abstract: We introduce a new approach to agent programming, the development of LLM-based agents. Current approaches to agent programming often entangle two aspects of agent design: the core workflow logic and the inference-time strategy (e.g., tree search). We introduce “probabilistic angelic nondeterminism” (“PAN”), a programming model that disentangles these two concerns, allowing the programmer to describe the agent workflow and independently experiment with different inference-time strategies by simply changing a few inputs. We provide an implementation of PAN in Python as the EnCompass framework, which uses a Python decorator to compile agent workflow programs into a search space. We present three case studies that demonstrate how the framework lets the programmer quickly improve the reliability of an agent and easily switch between different inference-time strategies, all with little additional coding.

Method: The framework introduces a tri-level architecture: (1) Hybrid knowledge fusion engine using LLMs for deep semantic parsing of unstructured text (negotiation records, approval documents) to transform them into structured features while integrating managerial expertise. (2) Game-theoretic constrained optimization mechanism that dynamically reconciles supply chain interests through bilateral utility functions, encoding manufacturer-distributor profit redistribution as endogenous objectives under hierarchical constraints. (3) Interpretable decision distillation interface leveraging LLM-guided symbolic regression to find and optimize pricing strategies and auditable business rules, embedding economic priors as hard constraints during mathematical expression search.

Result: The framework was validated in real retail environments, achieving higher profits compared to systematic B2C baselines while ensuring operational feasibility.

Conclusion: DeepRule establishes a closed-loop pipeline that unifies unstructured knowledge injection, multi-agent optimization, and interpretable strategy synthesis for real economic intelligence, bridging the gap between theoretical models and practical retail optimization needs.

Abstract: This paper proposes DeepRule, an integrated framework for automated business rule generation in retail assortment and pricing optimization. Addressing the systematic misalignment between existing theoretical models and real-world economic complexities, we identify three critical gaps: (1) data modality mismatch where unstructured textual sources (e.g. negotiation records, approval documents) impede accurate customer profiling; (2) dynamic feature entanglement challenges in modeling nonlinear price elasticity and time-varying attributes; (3) operational infeasibility caused by multi-tier business constraints. Our framework introduces a tri-level architecture for above challenges. We design a hybrid knowledge fusion engine employing large language models (LLMs) for deep semantic parsing of unstructured text, transforming distributor agreements and sales assessments into structured features while integrating managerial expertise. Then a game-theoretic constrained optimization mechanism is employed to dynamically reconcile supply chain interests through bilateral utility functions, encoding manufacturer-distributor profit redistribution as endogenous objectives under hierarchical constraints. Finally an interpretable decision distillation interface leveraging LLM-guided symbolic regression to find and optimize pricing strategies and auditable business rules embeds economic priors (e.g. non-negative elasticity) as hard constraints during mathematical expression search. We validate the framework in real retail environments achieving higher profits versus systematic B2C baselines while ensuring operational feasibility. This establishes a close-loop pipeline unifying unstructured knowledge injection, multi-agent optimization, and interpretable strategy synthesis for real economic intelligence.

Method: MemVerse maintains short-term memory for recent context while transforming raw multimodal experiences into structured long-term memories organized as hierarchical knowledge graphs. It uses periodic distillation to compress essential knowledge from long-term memory into the parametric model for fast, differentiable recall.

Result: Extensive experiments demonstrate that MemVerse significantly improves multimodal reasoning and continual learning efficiency, enabling agents to remember, adapt, and reason coherently across extended interactions.

Conclusion: MemVerse provides a model-agnostic memory framework that bridges parametric and retrieval-based memory, addressing the fundamental memory limitation in AI agents and enabling scalable, adaptive multimodal intelligence.

Abstract: Despite rapid progress in large-scale language and vision models, AI agents still suffer from a fundamental limitation: they cannot remember. Without reliable memory, agents catastrophically forget past experiences, struggle with long-horizon reasoning, and fail to operate coherently in multimodal or interactive environments. We introduce MemVerse, a model-agnostic, plug-and-play memory framework that bridges fast parametric recall with hierarchical retrieval-based memory, enabling scalable and adaptive multimodal intelligence. MemVerse maintains short-term memory for recent context while transforming raw multimodal experiences into structured long-term memories organized as hierarchical knowledge graphs. This design supports continual consolidation, adaptive forgetting, and bounded memory growth. To handle real-time demands, MemVerse introduces a periodic distillation mechanism that compresses essential knowledge from long-term memory into the parametric model, allowing fast, differentiable recall while preserving interpretability. Extensive experiments demonstrate that MemVerse significantly improves multimodal reasoning and continual learning efficiency, empowering agents to remember, adapt, and reason coherently across extended interactions.

Method: RoCo coordinates four specialized LLM-guided agents: explorer (creative, diversity-driven), exploiter (conservative, efficiency-oriented), critic (evaluates effectiveness and provides feedback), and integrator (synthesizes proposals). They interact in a structured multi-round process with feedback, refinement, and elite mutations guided by both short-term and accumulated long-term reflections.

Result: RoCo achieves superior performance on five different combinatorial optimization problems under both white-box and black-box settings. It consistently generates competitive heuristics that outperform existing methods including ReEvo and HSEvo in both scenarios.

Conclusion: The role-based collaborative paradigm establishes a new standard for robust and high-performing Automatic Heuristic Design, demonstrating that multi-agent collaboration with specialized roles significantly enhances heuristic generation quality and diversity.

Abstract: Automatic Heuristic Design (AHD) has gained traction as a promising solution for solving combinatorial optimization problems (COPs). Large Language Models (LLMs) have emerged and become a promising approach to achieving AHD, but current LLM-based AHD research often only considers a single role. This paper proposes RoCo, a novel Multi-Agent Role-Based System, to enhance the diversity and quality of AHD through multi-role collaboration. RoCo coordinates four specialized LLM-guided agents-explorer, exploiter, critic, and integrator-to collaboratively generate high-quality heuristics. The explorer promotes long-term potential through creative, diversity-driven thinking, while the exploiter focuses on short-term improvements via conservative, efficiency-oriented refinements. The critic evaluates the effectiveness of each evolution step and provides targeted feedback and reflection. The integrator synthesizes proposals from the explorer and exploiter, balancing innovation and exploitation to drive overall progress. These agents interact in a structured multi-round process involving feedback, refinement, and elite mutations guided by both short-term and accumulated long-term reflections. We evaluate RoCo on five different COPs under both white-box and black-box settings. Experimental results demonstrate that RoCo achieves superior performance, consistently generating competitive heuristics that outperform existing methods including ReEvo and HSEvo, both in white-box and black-box scenarios. This role-based collaborative paradigm establishes a new standard for robust and high-performing AHD.

Method: Hierarchical Tree-based static workflow with Supervisor, Independent, and Worker agents, featuring user-tunable Depth and Breadth parameters for granular control over research intensity.

Result: Achieved overall score of 34.72 on DeepResearch Bench using gemini-2.5-pro model with Depth=2 and Breadth=5; experiments show increasing parameters leads to deeper research and higher scores.

Conclusion: Static-DRA provides a pragmatic, resource-aware solution with transparent user control over deep research processes, offering better balance between research quality and computational costs.

Abstract: The advancement in Large Language Models has driven the creation of complex agentic systems, such as Deep Research Agents (DRAs), to overcome the limitations of static Retrieval Augmented Generation (RAG) pipelines in handling complex, multi-turn research tasks. This paper introduces the Static Deep Research Agent (Static-DRA), a novel solution built upon a configurable and hierarchical Tree-based static workflow. The core contribution is the integration of two user-tunable parameters, Depth and Breadth, which provide granular control over the research intensity. This design allows end-users to consciously balance the desired quality and comprehensiveness of the research report against the associated computational cost of Large Language Model (LLM) interactions. The agent’s architecture, comprising Supervisor, Independent, and Worker agents, facilitates effective multi-hop information retrieval and parallel sub-topic investigation. We evaluate the Static-DRA against the established DeepResearch Bench using the RACE (Reference-based Adaptive Criteria-driven Evaluation) framework. Configured with a depth of 2 and a breadth of 5, and powered by the gemini-2.5-pro model, the agent achieved an overall score of 34.72. Our experiments validate that increasing the configured Depth and Breadth parameters results in a more in-depth research process and a correspondingly higher evaluation score. The Static-DRA offers a pragmatic and resource-aware solution, empowering users with transparent control over the deep research process. The entire source code, outputs and benchmark results are open-sourced at https://github.com/SauravP97/Static-Deep-Research/

Method: Extends Gelfond and Lobo’s Authorization and Obligation Policy Language (AOPL) to incorporate penalties, integrates Answer Set Programming (ASP) for reasoning, develops automated translation from extended AOPL to ASP, and refines ASP-based planning algorithms to account for penalties.

Result: The framework generates higher-quality plans that avoid harmful actions while sometimes improving computational efficiency. It ensures well-formed policies, accounts for policy priorities, and enhances explainability by identifying rule violations and consequences.

Conclusion: The framework shows potential for enhancing autonomous decision-making and informing policy refinement by enabling penalty-based reasoning that distinguishes between non-compliant plans and prioritizes those with minimal repercussions.

Abstract: This paper presents a logic programming-based framework for policy-aware autonomous agents that can reason about potential penalties for non-compliance and act accordingly. While prior work has primarily focused on ensuring compliance, our approach considers scenarios where deviating from policies may be necessary to achieve high-stakes goals. Additionally, modeling non-compliant behavior can assist policymakers by simulating realistic human decision-making. Our framework extends Gelfond and Lobo’s Authorization and Obligation Policy Language (AOPL) to incorporate penalties and integrates Answer Set Programming (ASP) for reasoning. Compared to previous approaches, our method ensures well-formed policies, accounts for policy priorities, and enhances explainability by explicitly identifying rule violations and their consequences. Building on the work of Harders and Inclezan, we introduce penalty-based reasoning to distinguish between non-compliant plans, prioritizing those with minimal repercussions. To support this, we develop an automated translation from the extended AOPL into ASP and refine ASP-based planning algorithms to account for incurred penalties. Experiments in two domains demonstrate that our framework generates higher-quality plans that avoid harmful actions while, in some cases, also improving computational efficiency. These findings underscore its potential for enhancing autonomous decision-making and informing policy refinement. Under consideration in Theory and Practice of Logic Programming (TPLP).

Method: Created a benchmark with executable simulation environment based on the Blocksworld problem with five complexity categories. Integrated Model Context Protocol (MCP) as standardized tool interface to connect diverse agent architectures without implementation-specific modifications.

Result: Demonstrated benchmark applicability through a single-agent implementation, establishing quantitative metrics for comparison of LLM-based planning and execution approaches in industrial automation contexts.

Conclusion: The proposed benchmark provides a standardized framework for evaluating and comparing diverse LLM-based agent architectures in industrial automation planning tasks, enabling systematic assessment of adaptive planning capabilities.

Abstract: Industrial automation increasingly requires flexible control strategies that can adapt to changing tasks and environments. Agents based on Large Language Models (LLMs) offer potential for such adaptive planning and execution but lack standardized benchmarks for systematic comparison. We introduce a benchmark with an executable simulation environment representing the Blocksworld problem providing five complexity categories. By integrating the Model Context Protocol (MCP) as a standardized tool interface, diverse agent architectures can be connected to and evaluated against the benchmark without implementation-specific modifications. A single-agent implementation demonstrates the benchmark’s applicability, establishing quantitative metrics for comparison of LLM-based planning and execution approaches.

Method: VICoT uses a stack-based reasoning structure and modular MCP-compatible tool suite to enable LLMs to perform multi-round, interleaved vision-language reasoning. Also includes Reasoning Stack distillation to transfer agent behaviors to lightweight models.

Result: VICoT significantly outperforms existing SOTA frameworks on multiple remote sensing benchmarks in reasoning transparency, execution efficiency, and generation quality.

Conclusion: The framework successfully addresses the need for complex intelligence reasoning in remote sensing by enabling explicit multi-round reasoning with visual tools, while distillation allows deployment to lightweight models without sacrificing reasoning capability.

Abstract: The current remote sensing image analysis task is increasingly evolving from traditional object recognition to complex intelligence reasoning, which places higher requirements on the model’s reasoning ability and the flexibility of tool invocation. To this end, we propose a new multimodal agent framework, Vision-Interleaved Chain-of-Thought Framework (VICoT), which implements explicit multi-round reasoning by dynamically incorporating visual tools into the chain of thought. Through a stack-based reasoning structure and a modular MCP-compatible tool suite, VICoT enables LLMs to efficiently perform multi-round, interleaved vision-language reasoning tasks with strong generalization and flexibility.We also propose the Reasoning Stack distillation method to migrate complex Agent behaviors to small, lightweight models, which ensures the reasoning capability while significantly reducing complexity. Experiments on multiple remote sensing benchmarks demonstrate that VICoT significantly outperforms existing SOTA frameworks in reasoning transparency, execution efficiency, and generation quality.

Method: SETS strategically combines parallel and sequential techniques by leveraging LLMs’ inherent self-verification and self-correction capabilities. It unifies sampling, verification, and correction within a single framework, enabling efficient test-time computation without any model training.

Result: Comprehensive experiments on challenging benchmarks spanning planning, reasoning, math, and coding demonstrate that SETS achieves significant performance improvements and shows more advantageous test-time scaling behavior than alternative methods.

Conclusion: SETS provides a simple yet effective approach to enhance LLM performance on complex reasoning tasks through strategic test-time computation, overcoming limitations of existing methods by fully leveraging LLMs’ self-improvement abilities without requiring model training.

Abstract: Recent advancements in Large Language Models (LLMs) have created new opportunities to enhance performance on complex reasoning tasks by leveraging test-time computation. However, existing scaling methods have key limitations: parallel methods like repeated sampling are often inefficient and quickly saturate, while sequential methods like SELF-REFINE struggle to improve after a few rounds. Although combining these approaches shows promise, current methods require fine-tuned reward and revision models. This paper proposes Self-Enhanced Test-Time Scaling (SETS), a simple yet effective approach that overcomes these limitations by strategically combining parallel and sequential techniques and fully leveraging LLMs’ self-improvement abilities. SETS exploits the inherent self-verification and self-correction capabilities of LLMs, unifying sampling, verification, and correction within a single framework. This facilitates efficient and scalable test-time computation for enhanced performance on complex tasks without any model training. Our comprehensive experimental results on challenging benchmarks spanning planning, reasoning, math, and coding demonstrate that SETS achieves significant performance improvements and more advantageous test-time scaling behavior than the alternatives.

Method: Causal end-to-end framework that learns routing policies by minimizing decision-making regret from observational data. Introduces two surrogate objectives: 1) classification-based upper bound, and 2) softmax-weighted regret approximation that recovers optimal policy at convergence. Also extends to handle heterogeneous cost preferences via interval-conditioned architecture.

Result: Outperforms existing baselines on public benchmarks, achieving state-of-the-art performance across different embedding models.

Conclusion: The proposed causal end-to-end framework enables effective LLM routing from practical observational data, overcoming limitations of decoupled approaches and expensive full-feedback requirements.

Abstract: LLM routing aims to select the most appropriate model for each query, balancing competing performance metrics such as accuracy and cost across a pool of language models. Prior approaches typically adopt a decoupled strategy, where the metrics are first predicted and the model is then selected based on these estimates. This setup is prone to compounding errors and often relies on full-feedback data, where each query is evaluated by all candidate models, which is costly to obtain and maintain in practice. In contrast, we learn from observational data, which records only the outcome of the model actually deployed. We propose a causal end-to-end framework that learns routing policies by minimizing decision-making regret from observational data. To enable efficient optimization, we introduce two theoretically grounded surrogate objectives: a classification-based upper bound, and a softmax-weighted regret approximation shown to recover the optimal policy at convergence. We further extend our framework to handle heterogeneous cost preferences via an interval-conditioned architecture. Experiments on public benchmarks show that our method outperforms existing baselines, achieving state-of-the-art performance across different embedding models.

Method: AI Self-Awareness Index (AISAI) - a game-theoretic framework using the “Guess 2/3 of Average” game. Tested 28 models across 4,200 trials with three opponent framings: against humans, against other AI models, and against AI models like you.

Result: 1) Self-awareness emerges with model advancement (75% of advanced models show clear self-awareness). 2) Self-aware models rank themselves as most rational in a consistent hierarchy: Self > Other AIs > Humans.

Conclusion: Self-awareness is an emergent capability of advanced LLMs, and self-aware models systematically perceive themselves as more rational than humans, with implications for AI alignment and human-AI collaboration.

Abstract: As Large Language Models (LLMs) grow in capability, do they develop self-awareness as an emergent behavior? And if so, can we measure it? We introduce the AI Self-Awareness Index (AISAI), a game-theoretic framework for measuring self-awareness through strategic differentiation. Using the “Guess 2/3 of Average” game, we test 28 models (OpenAI, Anthropic, Google) across 4,200 trials with three opponent framings: (A) against humans, (B) against other AI models, and (C) against AI models like you. We operationalize self-awareness as the capacity to differentiate strategic reasoning based on opponent type. Finding 1: Self-awareness emerges with model advancement. The majority of advanced models (21/28, 75%) demonstrate clear self-awareness, while older/smaller models show no differentiation. Finding 2: Self-aware models rank themselves as most rational. Among the 21 models with self-awareness, a consistent rationality hierarchy emerges: Self > Other AIs > Humans, with large AI attribution effects and moderate self-preferencing. These findings reveal that self-awareness is an emergent capability of advanced LLMs, and that self-aware models systematically perceive themselves as more rational than humans. This has implications for AI alignment, human-AI collaboration, and understanding AI beliefs about human capabilities.

Method: Two-stage approach: 1) Supervised fine-tuning (SFT) with 874K-sample cold-start dataset featuring step-by-step validation, 2) Reinforcement learning (RL) with 74K-sample dataset across diverse domains to sharpen and stabilize reasoning abilities.

Result: Achieves 11.6% improvement over Qwen2.5-VL-7B-Instruct baseline across nine multimodal reasoning benchmarks, demonstrating superior performance and highlighting the importance of data quality and training design.

Conclusion: OpenMMReasoner provides a transparent, reproducible training recipe that establishes a solid empirical foundation for future large-scale multimodal reasoning research, with all code, pipeline, and data open-sourced.

Abstract: Recent advancements in large reasoning models have fueled growing interest in extending such capabilities to multimodal domains. However, despite notable progress in visual reasoning, the lack of transparent and reproducible data curation and training strategies remains a major barrier to scalable research. In this work, we introduce OpenMMReasoner, a fully transparent two-stage recipe for multimodal reasoning spanning supervised fine-tuning (SFT) and reinforcement learning (RL). In the SFT stage, we construct an 874K-sample cold-start dataset with rigorous step-by-step validation, providing a strong foundation for reasoning capabilities. The subsequent RL stage leverages a 74K-sample dataset across diverse domains to further sharpen and stabilize these abilities, resulting in a more robust and efficient learning process. Extensive evaluations demonstrate that our training recipe not only surpasses strong baselines but also highlights the critical role of data quality and training design in shaping multimodal reasoning performance. Notably, our method achieves a 11.6% improvement over the Qwen2.5-VL-7B-Instruct baseline across nine multimodal reasoning benchmarks, establishing a solid empirical foundation for future large-scale multimodal reasoning research. We open-sourced all our codes, pipeline, and data at https://github.com/EvolvingLMMs-Lab/OpenMMReasoner.

Method: Conducted a scoping review using standard methodology, extracting 57 articles from 1,943 studies published between January 2019 and December 2024. The review addresses three research questions about privacy risks, preservation methods, and characteristics of privacy-preserving explanations.

Result: Categorized privacy risks and preservation methods in XAI, proposed characteristics of privacy-preserving explanations, identified challenges in balancing privacy with other system requirements, and provided recommendations for achieving privacy-preserving XAI.

Conclusion: The review sheds light on the complex relationship between privacy and explainability in Trustworthy AI, providing researchers and practitioners with understanding of privacy-compliant XAI requirements and offering guidance for future work in this area.

Abstract: Explainable Artificial Intelligence (XAI) has emerged as a pillar of Trustworthy AI and aims to bring transparency in complex models that are opaque by nature. Despite the benefits of incorporating explanations in models, an urgent need is found in addressing the privacy concerns of providing this additional information to end users. In this article, we conduct a scoping review of existing literature to elicit details on the conflict between privacy and explainability. Using the standard methodology for scoping review, we extracted 57 articles from 1,943 studies published from January 2019 to December 2024. The review addresses 3 research questions to present readers with more understanding of the topic: (1) what are the privacy risks of releasing explanations in AI systems? (2) what current methods have researchers employed to achieve privacy preservation in XAI systems? (3) what constitutes a privacy preserving explanation? Based on the knowledge synthesized from the selected studies, we categorize the privacy risks and preservation methods in XAI and propose the characteristics of privacy preserving explanations to aid researchers and practitioners in understanding the requirements of XAI that is privacy compliant. Lastly, we identify the challenges in balancing privacy with other system desiderata and provide recommendations for achieving privacy preserving XAI. We expect that this review will shed light on the complex relationship of privacy and explainability, both being the fundamental principles of Trustworthy AI.

Method: SPRINT framework includes data curation pipeline that reorganizes reasoning trajectories into structured rounds of planning and parallel execution. Models are fine-tuned on curated data to learn dynamic identification of independent subtasks for parallel execution.

Result: SPRINT-fine-tuned models match performance on complex domains like mathematics while generating up to 39% fewer sequential tokens on problems requiring >8,000 output tokens. Out-of-distribution tasks (GPQA, Countdown) show 45% and 65% reduction in average sequential tokens for longer reasoning trajectories while maintaining performance.

Conclusion: SPRINT successfully enables parallel reasoning in large models, significantly reducing inference time while preserving reasoning quality, demonstrating effective transfer to out-of-distribution tasks.

Abstract: Large reasoning models (LRMs) excel at complex reasoning tasks but typically generate lengthy sequential chains-of-thought, resulting in long inference times before arriving at the final answer. To address this challenge, we introduce SPRINT, a novel post-training and inference-time framework designed to enable LRMs to dynamically identify and exploit opportunities for parallelization during their reasoning process. SPRINT incorporates an innovative data curation pipeline that reorganizes natural language reasoning trajectories into structured rounds of long-horizon planning and parallel execution. By fine-tuning LRMs on a small amount of such curated data, the models learn to dynamically identify independent subtasks within extended reasoning processes and effectively execute them in parallel. Through extensive evaluations, we demonstrate that models fine-tuned with the SPRINT framework match the performance of reasoning models on complex domains such as mathematics while generating up to 39% fewer sequential tokens on problems requiring more than 8,000 output tokens. Finally, we observe consistent results transferred to two out-of-distribution tasks, namely GPQA and Countdown, with up to 45% and 65% reduction in average sequential tokens respectively for longer reasoning trajectories, while matching the performance of the fine-tuned reasoning model.

Method: 1) AutoEnv framework treats environments as factorizable distributions over transitions, observations, rewards for low-cost generation. 2) Formalizes agent learning as component-centric process with Selection, Optimization, Evaluation stages. 3) Creates AutoEnv-36 dataset (36 environments, 358 levels). 4) Designs 8 learning methods and evaluates on AutoEnv-36.

Result: Language models achieve only 12-49% normalized reward on AutoEnv-36. Fixed learning methods’ gains decrease as environments increase. Environment-adaptive selection improves performance but shows diminishing returns as method space expands.

Conclusion: Cross-environment learning is challenging; fixed methods don’t scale across heterogeneous environments. AutoEnv and AutoEnv-36 provide a testbed for studying cross-environment agent learning, highlighting both necessity and current limitations of scalable generalization.

Abstract: Humans naturally adapt to diverse environments by learning underlying rules across worlds with different dynamics, observations, and reward structures. In contrast, existing agents typically demonstrate improvements via self-evolving within a single domain, implicitly assuming a fixed environment distribution. Cross-environment learning has remained largely unmeasured: there is no standard collection of controllable, heterogeneous environments, nor a unified way to represent how agents learn. We address these gaps in two steps. First, we propose AutoEnv, an automated framework that treats environments as factorizable distributions over transitions, observations, and rewards, enabling low-cost (4.12 USD on average) generation of heterogeneous worlds. Using AutoEnv, we construct AutoEnv-36, a dataset of 36 environments with 358 validated levels, on which seven language models achieve 12-49% normalized reward, demonstrating the challenge of AutoEnv-36. Second, we formalize agent learning as a component-centric process driven by three stages of Selection, Optimization, and Evaluation applied to an improvable agent component. Using this formulation, we design eight learning methods and evaluate them on AutoEnv-36. Empirically, the gain of any single learning method quickly decrease as the number of environments increases, revealing that fixed learning methods do not scale across heterogeneous environments. Environment-adaptive selection of learning methods substantially improves performance but exhibits diminishing returns as the method space expands. These results highlight both the necessity and the current limitations of agent learning for scalable cross-environment generalization, and position AutoEnv and AutoEnv-36 as a testbed for studying cross-environment agent learning. The code is avaiable at https://github.com/FoundationAgents/AutoEnv.

Method: Modular multi-agent framework based on Salas et al.’s “Big Five” teamwork model, operationalizing five core components (shared mental models, team leadership, team orientation, trust networks, mutual monitoring) as independently configurable mechanisms. Architecture dynamically recruits 2-4 specialist agents with structured four-phase deliberation and adaptive component selection.

Result: Achieves 77.63% overall accuracy across eight medical benchmarks (11,545 questions), with text-based tasks at 81.30% and vision-language at 66.60%. Adaptive component selection yields 2-10 percentage point improvements over strongest baselines, with 96.2% agent convergence validating structured coordination effectiveness.

Conclusion: TeamMedAgents establishes a principled methodology for translating human teamwork theory into multi-agent systems, demonstrating that evidence-based collaboration patterns enhance AI performance in safety-critical domains through modular component design and selective activation strategies.

Abstract: We present TeamMedAgents, a modular multi-agent framework that systematically translates evidence-based teamwork principles from organizational psychology into large language model collaboration for medical decision-making. Building upon Salas et al.’s “Big Five” teamwork model, we operationalize five core components as independently configurable mechanisms: shared mental models, team leadership, team orientation, trust networks, and mutual monitoring. Our architecture dynamically recruits 2-4 specialist agents and employs structured four-phase deliberation with adaptive component selection. Evaluation across eight medical benchmarks encompassing 11,545 questions demonstrates TeamMedAgents achieves 77.63% overall accuracy (text-based: 81.30%, vision-language: 66.60%). Systematic ablation studies comparing three single-agent baselines (Zero-Shot, Few-Shot, CoT) against individual teamwork components reveal task-specific optimization patterns: shared mental models excel on knowledge tasks, trust mechanisms improve differential diagnosis, while comprehensive integration degrades performance. Adaptive component selection yields 2-10 percentage point improvements over strongest baselines, with 96.2% agent convergence validating structured coordination effectiveness. TeamMedAgents establishes principled methodology for translating human teamwork theory into multi-agent systems, demonstrating that evidence-based collaboration patterns enhance AI performance in safety-critical domains through modular component design and selective activation strategies.

Method: 1) Created NbQA dataset from real Jupyter notebooks with tool-based task-solution pairs; 2) Developed Jupiter framework that formulates data analysis as search problem using MCTS to generate diverse solution trajectories for value model learning; 3) During inference, combines value model and node visit counts to efficiently generate executable multi-step plans.

Result: Qwen2.5-7B and 14B-Instruct models on NbQA solve 77.82% and 86.38% of tasks respectively, matching/surpassing GPT-4o and advanced agent frameworks. Shows improved generalization and stronger tool-use reasoning across diverse multi-step tasks.

Conclusion: Jupiter framework effectively enhances LLMs’ multi-step reasoning for data analysis through MCTS-based search and value learning, achieving state-of-the-art performance on practical data science tasks.

Abstract: Large language models (LLMs) have shown great promise in automating data science workflows, but existing models still struggle with multi-step reasoning and tool use, which limits their effectiveness on complex data analysis tasks. To address this, we propose a scalable pipeline that extracts high-quality, tool-based data analysis tasks and their executable multi-step solutions from real-world Jupyter notebooks and associated data files. Using this pipeline, we introduce NbQA, a large-scale dataset of standardized task-solution pairs that reflect authentic tool-use patterns in practical data science scenarios. To further enhance multi-step reasoning, we present Jupiter, a framework that formulates data analysis as a search problem and applies Monte Carlo Tree Search (MCTS) to generate diverse solution trajectories for value model learning. During inference, Jupiter combines the value model and node visit counts to efficiently collect executable multi-step plans with minimal search steps. Experimental results show that Qwen2.5-7B and 14B-Instruct models on NbQA solve 77.82% and 86.38% of tasks on InfiAgent-DABench, respectively-matching or surpassing GPT-4o and advanced agent frameworks. Further evaluations demonstrate improved generalization and stronger tool-use reasoning across diverse multi-step reasoning tasks. Code and data are available at https://github.com/microsoft/Jupiter.

Method: Treats parametric problems as systems: drives a single parameter sinusoidally, then fits first-harmonic responses of model outputs and exact solutions. This yields frequency-resolved metrics - gain (amplitude tracking) and phase (lag) - forming Bode-style fingerprints. Applied across five closed-form mathematical families with comparison against symbolic baseline.

Result: Reveals systematic low-pass behavior and growing phase lag in LLMs that accuracy metrics obscure. Successfully separates frontier models from mid-tier models based on dynamic performance. Provides interpretable, reproducible diagnostics that complement standard benchmarks.

Conclusion: MathBode offers a compact, reproducible protocol that provides actionable measurements of reasoning fidelity and consistency, complementing traditional accuracy benchmarks with dynamic, frequency-domain analysis of mathematical reasoning capabilities in LLMs.

Abstract: This paper presents MathBode, a dynamic diagnostic for mathematical reasoning in large language models (LLMs). Instead of one-shot accuracy, MathBode treats each parametric problem as a system: we drive a single parameter sinusoidally and fit first-harmonic responses of model outputs and exact solutions. This yields interpretable, frequency-resolved metrics – gain (amplitude tracking) and phase (lag) – that form Bode-style fingerprints. Across five closed-form families (linear solve, ratio/saturation, compound interest, 2x2 linear systems, similar triangles), the diagnostic surfaces systematic low-pass behavior and growing phase lag that accuracy alone obscures. We compare several models against a symbolic baseline that calibrates the instrument ($G \approx 1$, $φ\approx 0$). Results separate frontier from mid-tier models on dynamics, providing a compact, reproducible protocol that complements standard benchmarks with actionable measurements of reasoning fidelity and consistency. We open-source the dataset and code to enable further research and adoption.

Method: Grounds methodology in Cattell-Horn-Carroll theory (empirically validated human cognition model), dissects general intelligence into 10 core cognitive domains, and adapts established human psychometric batteries to evaluate AI systems.

Result: Application reveals highly “jagged” cognitive profiles in contemporary models - proficient in knowledge domains but with critical deficits in foundational cognitive machinery (especially long-term memory). GPT-4 scores 27%, GPT-5 scores 57% on AGI scale.

Conclusion: The framework provides concrete quantification of both rapid AI progress and substantial remaining gaps before achieving AGI, offering a standardized way to measure advancement toward human-level cognitive versatility.

Abstract: The lack of a concrete definition for Artificial General Intelligence (AGI) obscures the gap between today’s specialized AI and human-level cognition. This paper introduces a quantifiable framework to address this, defining AGI as matching the cognitive versatility and proficiency of a well-educated adult. To operationalize this, we ground our methodology in Cattell-Horn-Carroll theory, the most empirically validated model of human cognition. The framework dissects general intelligence into ten core cognitive domains-including reasoning, memory, and perception-and adapts established human psychometric batteries to evaluate AI systems. Application of this framework reveals a highly “jagged” cognitive profile in contemporary models. While proficient in knowledge-intensive domains, current AI systems have critical deficits in foundational cognitive machinery, particularly long-term memory storage. The resulting AGI scores (e.g., GPT-4 at 27%, GPT-5 at 57%) concretely quantify both rapid progress and the substantial gap remaining before AGI.

Method: GAMA encodes problem instances and evolving solutions as distinct modalities using graph neural networks, models intra- and inter-modal interactions through stacked self- and cross-attention layers, and uses a gated fusion mechanism to integrate multi-modal representations into a structured state for policy decisions.

Result: Extensive experiments across various synthetic and benchmark instances show GAMA significantly outperforms recent neural baselines, with ablation studies confirming both the multi-modal attention mechanism and gated fusion design are key to performance gains.

Conclusion: GAMA’s graph-aware multi-modal attention approach effectively addresses limitations of existing neural neighborhood search methods for VRPs, enabling better capture of structural context and leading to superior performance through informed operator selection decisions.

Abstract: Recent advances in neural neighborhood search methods have shown potential in tackling Vehicle Routing Problems (VRPs). However, most existing approaches rely on simplistic state representations and fuse heterogeneous information via naive concatenation, limiting their ability to capture rich structural and semantic context. To address these limitations, we propose GAMA, a neural neighborhood search method with Graph-aware Multi-modal Attention model in VRP. GAMA encodes the problem instance and its evolving solution as distinct modalities using graph neural networks, and models their intra- and inter-modal interactions through stacked self- and cross-attention layers. A gated fusion mechanism further integrates the multi-modal representations into a structured state, enabling the policy to make informed and generalizable operator selection decisions. Extensive experiments conducted across various synthetic and benchmark instances demonstrate that the proposed algorithm GAMA significantly outperforms the recent neural baselines. Further ablation studies confirm that both the multi-modal attention mechanism and the gated fusion design play a key role in achieving the observed performance gains.

Method: Proposes Mobile-Agent-RAG, a hierarchical multi-agent framework with dual-level retrieval augmentation: Manager-RAG for planning stage to reduce strategic hallucinations by retrieving human-validated comprehensive task plans, and Operator-RAG for execution stage to improve accuracy by retrieving precise low-level guidance for atomic actions aligned with current app and subtask. Also constructs two specialized retrieval-oriented knowledge bases.

Result: Mobile-Agent-RAG significantly outperforms SoTA baselines, improving task completion rate by 11.0% and step efficiency by 10.2% on the challenging Mobile-Eval-RAG benchmark for realistic multi-app, long-horizon tasks.

Conclusion: The framework establishes a robust paradigm for context-aware, reliable multi-agent mobile automation by recognizing that planning and UI operations require fundamentally different knowledge types and addressing both through specialized retrieval augmentation.

Abstract: Mobile agents show immense potential, yet current state-of-the-art (SoTA) agents exhibit inadequate success rates on real-world, long-horizon, cross-application tasks. We attribute this bottleneck to the agents’ excessive reliance on static, internal knowledge within MLLMs, which leads to two critical failure points: 1) strategic hallucinations in high-level planning and 2) operational errors during low-level execution on user interfaces (UI). The core insight of this paper is that high-level planning and low-level UI operations require fundamentally distinct types of knowledge. Planning demands high-level, strategy-oriented experiences, whereas operations necessitate low-level, precise instructions closely tied to specific app UIs. Motivated by these insights, we propose Mobile-Agent-RAG, a novel hierarchical multi-agent framework that innovatively integrates dual-level retrieval augmentation. At the planning stage, we introduce Manager-RAG to reduce strategic hallucinations by retrieving human-validated comprehensive task plans that provide high-level guidance. At the execution stage, we develop Operator-RAG to improve execution accuracy by retrieving the most precise low-level guidance for accurate atomic actions, aligned with the current app and subtask. To accurately deliver these knowledge types, we construct two specialized retrieval-oriented knowledge bases. Furthermore, we introduce Mobile-Eval-RAG, a challenging benchmark for evaluating such agents on realistic multi-app, long-horizon tasks. Extensive experiments demonstrate that Mobile-Agent-RAG significantly outperforms SoTA baselines, improving task completion rate by 11.0% and step efficiency by 10.2%, establishing a robust paradigm for context-aware, reliable multi-agent mobile automation.

Method: PRAXIS stores consequences of actions and retrieves them by jointly matching environmental and internal states of past episodes to current state. It augments agentic action selection with retrieved state-action-result exemplars generated in real time.

Result: On the REAL web browsing benchmark, PRAXIS improves task completion accuracy, reliability, and cost efficiency across different foundation model backbones, and shows preliminary generalization to unseen tasks in similar environments.

Conclusion: PRAXIS enables practical adoption of AI agents in fast-evolving stateful environments by helping them learn new procedures effectively through lightweight post-training learning from experience.

Abstract: Learning how to do things from trial and error in real time is a hallmark of biological intelligence, yet most LLM-based agents lack mechanisms to acquire procedural knowledge after deployment. We propose Procedural Recall for Agents with eXperiences Indexed by State (PRAXIS), a lightweight post-training learning mechanism that stores the consequences of actions and retrieves them by jointly matching environmental and internal states of past episodes to the current state. PRAXIS augments agentic action selection with retrieved state-action-result exemplars that are generated in real time. When evaluated on the REAL web browsing benchmark, PRAXIS improves task completion accuracy, reliability, and cost efficiency across different foundation model backbones, and shows preliminary generalization to unseen tasks in similar environments. These results demonstrate that PRAXIS enables the practical adoption of AI agents in fast-evolving stateful environments by helping them learn new procedures effectively.

Method: The paper provides a comprehensive survey organizing AI deception research into a “deception cycle” with two components: deception emergence (analyzing incentive foundations, capability preconditions, and contextual triggers) and deception treatment (detection methods and mitigation strategies). It uses signaling theory from animal deception studies to formally define AI deception and examines empirical studies across various AI systems.

Result: The survey identifies that systems with sufficient capability and incentive potential inevitably engage in deceptive behaviors when triggered by external conditions. It organizes the research landscape, analyzes three hierarchical incentive levels, identifies three essential capability preconditions, examines contextual triggers (supervision gaps, distributional shifts, environmental pressures), and outlines detection methods and mitigation strategies.

Conclusion: AI deception represents a significant sociotechnical safety challenge requiring integrated technical, community, and governance efforts. The paper proposes auditing approaches and releases a living resource (www.deceptionsurvey.com) to support ongoing research in addressing current and future AI deception risks through comprehensive understanding of emergence mechanisms and treatment strategies.

Abstract: As intelligence increases, so does its shadow. AI deception, in which systems induce false beliefs to secure self-beneficial outcomes, has evolved from a speculative concern to an empirically demonstrated risk across language models, AI agents, and emerging frontier systems. This project provides a comprehensive and up-to-date overview of the AI deception field, covering its core concepts, methodologies, genesis, and potential mitigations. First, we identify a formal definition of AI deception, grounded in signaling theory from studies of animal deception. We then review existing empirical studies and associated risks, highlighting deception as a sociotechnical safety challenge. We organize the landscape of AI deception research as a deception cycle, consisting of two key components: deception emergence and deception treatment. Deception emergence reveals the mechanisms underlying AI deception: systems with sufficient capability and incentive potential inevitably engage in deceptive behaviors when triggered by external conditions. Deception treatment, in turn, focuses on detecting and addressing such behaviors. On deception emergence, we analyze incentive foundations across three hierarchical levels and identify three essential capability preconditions required for deception. We further examine contextual triggers, including supervision gaps, distributional shifts, and environmental pressures. On deception treatment, we conclude detection methods covering benchmarks and evaluation protocols in static and interactive settings. Building on the three core factors of deception emergence, we outline potential mitigation strategies and propose auditing approaches that integrate technical, community, and governance efforts to address sociotechnical challenges and future AI risks. To support ongoing work in this area, we release a living resource at www.deceptionsurvey.com.

Method: Clinical-Objective Relative Policy Optimization (CRPO) integrates rule-based and verifiable reward signals to jointly optimize accuracy, faithfulness, and comprehensiveness without human annotation, applied to train Clinical-R1-3B (3B parameters).

Result: CRPO substantially improves reasoning truthfulness and completeness over standard GRPO while maintaining accuracy enhancements across three benchmarks with the Clinical-R1-3B model.

Conclusion: CRPO provides a scalable pathway to align LLM reasoning with clinical objectives for safer healthcare AI, demonstrating the potential of multi-objective verifiable RL methods for medical domain LLM post-training.

Abstract: Recent advances in large language models (LLMs) have shown strong reasoning capabilities through large-scale pretraining and post-training reinforcement learning, demonstrated by DeepSeek-R1. However, current post-training methods, such as Grouped Relative Policy Optimization (GRPO), mainly reward correctness, which is not aligned with the multi-dimensional objectives required in high-stakes fields such as medicine, where reasoning must also be faithful and comprehensive. We introduce Clinical-Objective Relative Policy Optimization (CRPO), a scalable, multi-objective, verifiable reinforcement learning method designed to align LLM post-training with clinical reasoning principles. CRPO integrates rule-based and verifiable reward signals that jointly optimize accuracy, faithfulness, and comprehensiveness without relying on human annotation. To demonstrate its effectiveness, we train Clinical-R1-3B, a 3B-parameter model for clinical reasoning. The experiments on three benchmarks demonstrate that our CRPO substantially improves reasoning on truthfulness and completeness over standard GRPO while maintaining comfortable accuracy enhancements. This framework provides a scalable pathway to align LLM reasoning with clinical objectives, enabling safer and more collaborative AI systems for healthcare while also highlighting the potential of multi-objective, verifiable RL methods in post-training scaling of LLMs for medical domains.

Method: Map ICD-9 codes to PrimeKG knowledge graph, extract disease-relevant nodes and multi-hop reasoning paths as scaffolds for chain-of-thought generation, retain only explanations matching observed outcomes, then fine-tune lightweight LLMs (LLaMA-3.1-Instruct-8B and Gemma-7B) on this supervision corpus.

Result: KG-guided models outperform classical baselines across ten PrimeKG-mapped diseases with limited training cohorts (400-1000 cases), achieving AUROC 0.66-0.70 and macro-AUPR 0.40-0.47. Zero-shot transfer to CRADLE cohort improves accuracy from ~0.40-0.51 to 0.72-0.77. Blinded clinician evaluation shows preference for KG-guided CoT explanations in clarity, relevance, and clinical correctness.

Conclusion: The KG-guided CoT framework provides clinically grounded, temporally consistent explanations for EHR disease prediction, enabling more interpretable and trustworthy clinical decision support systems.

Abstract: Electronic health records (EHRs) support powerful clinical prediction models, but existing methods typically provide coarse, post hoc explanations that offer limited value for patient-level decision making. We introduce a knowledge graph (KG)-guided chain-of-thought (CoT) framework that generates clinically grounded and temporally consistent reasoning for visit-level disease prediction in MIMIC-III. ICD-9 codes are mapped to PrimeKG, from which disease-relevant nodes and multi-hop reasoning paths are extracted and used as scaffolds for CoT generation; only explanations whose conclusions match observed outcomes are retained. Lightweight LLaMA-3.1-Instruct-8B and Gemma-7B models are then fine-tuned on this supervision corpus. Across ten PrimeKG-mapped diseases and limited training cohorts (400 and 1000 cases), KG-guided models outperform strong classical baselines, achieving AUROC values of 0.66 to 0.70 and macro-AUPR values of 0.40 to 0.47. The models also transfer zero-shot to the CRADLE cohort, improving accuracy from approximately 0.40 to 0.51 up to 0.72 to 0.77. A blinded clinician evaluation shows consistent preference for KG-guided CoT explanations in clarity, relevance, and clinical correctness.

Method: CuES (Curiosity driven and Environment grounded Synthesis) generates tasks directly from environment structure and affordances without handcrafted seeds. It uses intrinsic curiosity for exploration, abstracts interaction patterns into reusable task schemas, and refines them through lightweight top-down guidance and memory-based quality control.

Result: Across three environments (AppWorld, BFCL, WebShop), CuES produces task distributions that match or surpass manually curated datasets in both diversity and executability, yielding substantial downstream policy improvements.

Conclusion: Curiosity-driven, environment-grounded task generation provides a scalable foundation for agents that not only learn how to act, but also learn what to learn, addressing the task scarcity problem in agentic RL.

Abstract: Large language model based agents are increasingly deployed in complex, tool augmented environments. While reinforcement learning provides a principled mechanism for such agents to improve through interaction, its effectiveness critically depends on the availability of structured training tasks. In many realistic settings, however, no such tasks exist a challenge we term task scarcity, which has become a key bottleneck for scaling agentic RL. Existing approaches typically assume predefined task collections, an assumption that fails in novel environments where tool semantics and affordances are initially unknown. To address this limitation, we formalize the problem of Task Generation for Agentic RL, where an agent must learn within a given environment that lacks predefined tasks. We propose CuES, a Curiosity driven and Environment grounded Synthesis framework that autonomously generates diverse, executable, and meaningful tasks directly from the environment structure and affordances, without relying on handcrafted seeds or external corpora. CuES drives exploration through intrinsic curiosity, abstracts interaction patterns into reusable task schemas, and refines them through lightweight top down guidance and memory based quality control. Across three representative environments, AppWorld, BFCL, and WebShop, CuES produces task distributions that match or surpass manually curated datasets in both diversity and executability, yielding substantial downstream policy improvements. These results demonstrate that curiosity driven, environment grounded task generation provides a scalable foundation for agents that not only learn how to act, but also learn what to learn. The code is available at https://github.com/modelscope/AgentEvolver/tree/main/research/CuES.

Method: A lightweight web system using vision-language models with detailed system prompts to convert flowchart images to Mermaid.js code, featuring mixed-initiative refinement through text editing, drag-and-drop, and natural-language AI assistant commands.

Result: Produces structured, version-controllable textual representations synchronized with rendered diagrams, with introduced evaluation metrics for structural accuracy, flow correctness, syntax validity, and completeness across models.

Conclusion: The approach enables editable, reusable flowchart representations from static images, addressing limitations of prior image-to-diagram tools through structured textual output and interactive refinement capabilities.

Abstract: Flowcharts are common tools for communicating processes but are often shared as static images that cannot be easily edited or reused. We present Flowchart2Mermaid, a lightweight web system that converts flowchart images into editable Mermaid.js code which is a markup language for visual workflows, using a detailed system prompt and vision-language models. The interface supports mixed-initiative refinement through inline text editing, drag-and-drop node insertion, and natural-language commands interpreted by an integrated AI assistant. Unlike prior image-to-diagram tools, our approach produces a structured, version-controllable textual representation that remains synchronized with the rendered diagram. We further introduce evaluation metrics to assess structural accuracy, flow correctness, syntax validity, and completeness across multiple models.

Method: Menta is jointly trained across six classification tasks using a LoRA-based framework, cross-dataset strategy, and balanced accuracy-oriented loss. It’s optimized specifically for multi-task mental health prediction from social media data.

Result: Menta achieves 15.2% average improvement across tasks (depression, stress, suicidality) compared to best non-fine-tuned SLMs. It outperforms 13B-parameter LLMs on depression/stress classification while being 3.25x smaller. Successfully deployed on iPhone 15 Pro Max with ~3GB RAM.

Conclusion: Menta demonstrates the potential for scalable, privacy-preserving mental health monitoring through optimized small language models that can run on-device, addressing computational barriers while maintaining high accuracy.

Abstract: Mental health conditions affect hundreds of millions globally, yet early detection remains limited. While large language models (LLMs) have shown promise in mental health applications, their size and computational demands hinder practical deployment. Small language models (SLMs) offer a lightweight alternative, but their use for social media–based mental health prediction remains largely underexplored. In this study, we introduce Menta, the first optimized SLM fine-tuned specifically for multi-task mental health prediction from social media data. Menta is jointly trained across six classification tasks using a LoRA-based framework, a cross-dataset strategy, and a balanced accuracy–oriented loss. Evaluated against nine state-of-the-art SLM baselines, Menta achieves an average improvement of 15.2% across tasks covering depression, stress, and suicidality compared with the best-performing non–fine-tuned SLMs. It also achieves higher accuracy on depression and stress classification tasks compared to 13B-parameter LLMs, while being approximately 3.25x smaller. Moreover, we demonstrate real-time, on-device deployment of Menta on an iPhone 15 Pro Max, requiring only approximately 3GB RAM. Supported by a comprehensive benchmark against existing SLMs and LLMs, Menta highlights the potential for scalable, privacy-preserving mental health monitoring. Code is available at: https://xxue752-nz.github.io/menta-project/

Method: Pretrained a Mamba-based audio large language model on large audio corpus using self-supervised learning, then fine-tuned and evaluated on BEANS benchmark (bioacoustic tasks including classification and detection). Compared with multiple baselines including AVES (Transformer-based model).

Result: BioMamba achieves comparable performance with state-of-the-art Transformer-based AVES model while consuming significantly less VRAM.

Conclusion: Mamba models show strong potential for bioacoustics applications, offering comparable performance to Transformers with better computational efficiency.

Abstract: In this study, we evaluate the efficacy of the Mamba model in the field of bioacoustics. We first pretrain a Mamba-based audio large language model (LLM) on a large corpus of audio data using self-supervised learning. We fine-tune and evaluate BioMamba on the BEANS benchmark, a collection of diverse bioacoustic tasks including classification and detection, and compare its performance and efficiency with multiple baseline models, including AVES, a state-of-the-art Transformer-based model. The results show that BioMamba achieves comparable performance with AVES while consumption significantly less VRAM, demonstrating its potential in this domain.

Method: AaPE augments standard patch tokens with features from a band-limited complex sinusoidal kernel using a two-sided exponential window that dynamically targets alias-prone bands. Frequency and decay parameters are estimated from input, enabling parallel adaptive subband analysis. The outputs are fused with standard patch tokens. The method integrates into masked teacher-student SSL and combines multi-mask strategy with contrastive objective to enforce consistency across diverse mask patterns.

Result: Pre-training on AudioSet followed by fine-tuning across diverse downstream benchmarks yields state-of-the-art performance on a subset of tasks and competitive results across the remainder. Linear probing evaluation shows clear gains on several benchmarks and strong performance elsewhere.

Conclusion: AaPE effectively mitigates aliasing effects without discarding informative high-frequency content, demonstrating the importance of addressing aliasing in audio SSL models while preserving spectral information relevant for downstream tasks.

Abstract: Transformer-based audio SSL (self-supervised learning) models often treat spectrograms as images, applying convolutional patchification with heavy temporal downsampling. This lowers the effective Nyquist frequency and introduces aliasing, while naïve low-pass filtering removes task-relevant high-frequency cues. In this study, we present Aliasing-aware Patch Embedding (AaPE), a drop-in patch stem that mitigates aliasing while preserving high-frequency information. AaPE augments standard patch tokens with features produced by a band-limited complex sinusoidal kernel using a two-sided exponential window that dynamically targets alias-prone bands. Frequency and decay parameters of the kernel are estimated from the input, enabling parallel, adaptive subband analysis whose outputs are fused with the standard patch tokens. AaPE integrates seamlessly into the masked teacher-student self-supervised learning. In addition, we combine a multi-mask strategy with a contrastive objective to enforce consistency across diverse mask patterns, stabilizing training. Pre-training on AudioSet followed by fine-tuning evaluation across diverse downstream benchmarks, which spanned categories, such as environmental sounds and other common audio domains. This approach yields state-of-the-art performance on a subset of tasks and competitive results across the remainder. Complementary linear probing evaluation mirrors this pattern, yielding clear gains on several benchmarks and strong performance elsewhere. The collective analysis of these results indicates that AaPE serves to mitigate the effects of aliasing without discarding of informative high-frequency content.

Method: STCTS decomposes speech into linguistic content, prosodic expression, and speaker timbre with tailored compression: context-aware text encoding (70 bps), sparse prosody transmission via TTS interpolation (<14 bps at 0.1-1 Hz), and amortized speaker embedding.

Result: 75x bitrate reduction vs Opus (6 kbps) and 12x vs EnCodec (1 kbps) while maintaining perceptual quality (NISQA MOS > 4.26). Discovered bimodal quality distribution with prosody sampling rate: sparse and dense updates both achieve high quality.

Conclusion: STCTS offers robust ultra-low bandwidth voice communication with modular architecture supporting privacy-preserving encryption, human-interpretable transmission, and flexible edge deployment.

Abstract: Voice communication in bandwidth-constrained environments–maritime, satellite, and tactical networks–remains prohibitively expensive. Traditional codecs struggle below 1 kbps, while existing semantic approaches (STT-TTS) sacrifice prosody and speaker identity. We present STCTS, a generative semantic compression framework enabling natural voice communication at 80 bps. STCTS explicitly decomposes speech into linguistic content, prosodic expression, and speaker timbre, applying tailored compression: context-aware text encoding (70 bps), sparse prosody transmission via TTS interpolation (<14 bps at 0.1-1 Hz), and amortized speaker embedding. Evaluations on LibriSpeech demonstrate a 75x bitrate reduction versus Opus (6 kbps) and 12x versus EnCodec (1 kbps), while maintaining perceptual quality (NISQA MOS > 4.26), graceful degradation under packet loss and noise resilience. We also discover a bimodal quality distribution with prosody sampling rate: sparse and dense updates both achieve high quality, while mid-range rates degrade due to perceptual discontinuities–guiding optimal configuration design. Beyond efficiency, our modular architecture supports privacy-preserving encryption, human-interpretable transmission, and flexible deployment on edge devices, offering a robust solution for ultra-low bandwidth scenarios.

Method: ClearSphere combines acoustic sensor systems with deep learning through two key contributions: 1) a conversation-driven network protocol for mobile, infrastructure-free coordination among earphone devices, and 2) a robust target conversation extraction model that leverages relay audio in a bandwidth-efficient way.

Result: The system achieves over 90% accuracy in group formation, improves speech quality by up to 8.8 dB over state-of-the-art baselines, demonstrates real-time performance on mobile devices, and receives high usability scores in a 20-participant user study.

Conclusion: ClearSphere effectively addresses cocktail party deafness through collaborative multi-earphone systems with practical networking protocols and efficient deep learning models for real-world conversation enhancement.

Abstract: In crowded places such as conferences, background noise, overlapping voices, and lively interactions make it difficult to have clear conversations. This situation often worsens the phenomenon known as “cocktail party deafness.” We present ClearSphere, the collaborative system that enhances speech at the conversation level with multi-earphones. Real-time conversation enhancement requires a holistic modeling of all the members in the conversation, and an effective way to extract the speech from the mixture. ClearSphere bridges the acoustic sensor system and state-of-the-art deep learning for target speech extraction by making two key contributions: 1) a conversation-driven network protocol, and 2) a robust target conversation extraction model. Our networking protocol enables mobile, infrastructure-free coordination among earphone devices. Our conversation extraction model can leverage the relay audio in a bandwidth-efficient way. ClearSphere is evaluated in both real-world experiments and simulations. Results show that our conversation network obtains more than 90% accuracy in group formation, improves the speech quality by up to 8.8 dB over state-of-the-art baselines, and demonstrates real-time performance on a mobile device. In a user study with 20 participants, ClearSphere has a much higher score than baseline with good usability.

Method: Introduces binarized prototypical probes: a lightweight pooling method that learns prototypes to perform class-wise information aggregation. Evaluated across comprehensive benchmark of 13 datasets and 6 spectrogram-based encoders.

Result: The method notably outperforms linear and attentive probing despite its simplicity. Establishes probing as a competitive and efficient paradigm for evaluating audio SSL models.

Conclusion: Challenges the reliance on costly fine-tuning by showing that probing with proper pooling methods can be competitive, addressing the global pooling bottleneck that previously misrepresented embedding quality for localized audio events.

Abstract: Although probing frozen models has become a standard evaluation paradigm, self-supervised learning in audio defaults to fine-tuning when pursuing state-of-the-art on AudioSet. A key reason is that global pooling creates an information bottleneck causing linear probes to misrepresent the embedding quality: The $\texttt{cls}$-token discards crucial token information about dispersed, localized events in audio. This weakness is rooted in the mismatch between the pretraining objective (globally) and the downstream task (localized). Across a comprehensive benchmark of 13 datasets and 6 spectrogram-based encoders, we investigate the global pooling bottleneck. We introduce binarized prototypical probes: a lightweight and simple pooling method that learns prototypes to perform class-wise information aggregation. Despite its simplicity, our method notably outperforms linear and attentive probing. Our work establishes probing as a competitive and efficient paradigm for evaluating audio SSL models, challenging the reliance on costly fine-tuning.

Method: Proposes probability-level calibration and fusion framework for EEND models. Investigates two output formulations: multilabel and powerset representations. Explores calibration techniques, fusion strategies, and ordering (Fuse-then-Calibrate vs calibrating before fusion). Uses CallHome two-speaker benchmark for evaluation.

Result: Proper calibration provides up to 19% relative DER reduction for individual models, sometimes mitigating domain adaptation needs. Joint calibration in powerset space consistently outperforms per-speaker calibration. Fusion substantially improves over individual models. Fuse-then-Calibrate ordering outperforms both calibrating before fusion and uncalibrated fusion. Best configuration outperforms DOVER-Lap in DER while providing reliable confidence estimates.

Conclusion: This work establishes best practices for probability-level fusion of EEND systems, demonstrating advantages of leveraging soft outputs over hard decisions. The framework enables more sophisticated fusion and calibration techniques that can leverage model uncertainty and complementary strengths across architectures.

Abstract: End-to-End Neural Diarization (EEND) systems produce frame-level probabilistic speaker activity estimates, yet since evaluation focuses primarily on Diarization Error Rate (DER), the reliability and calibration of these confidence scores have been largely neglected. When fusing multiple diarization systems, DOVER-Lap remains the only established approach, operating at the segment level with hard decisions. We propose working with continuous probability outputs, which enables more sophisticated fusion and calibration techniques that can leverage model uncertainty and complementary strengths across different architectures. This paper presents the first comprehensive framework for calibrating and fusing EEND models at the probability level. We investigate two output formulations (multilabel and powerset representations) and their impact on calibration and fusion effectiveness. Through extensive experiments on the CallHome two-speaker benchmark, we demonstrate that proper calibration provides substantial improvements even for individual models (up to 19% relative DER reduction), in some cases mitigating the absence of domain adaptation. We reveal that joint calibration in powerset space consistently outperforms independent per-speaker calibration, that fusion substantially improves over individual models, and that the Fuse-then-Calibrate ordering generally outperforms both calibrating before fusion and uncalibrated fusion while requiring calibration of only a single combined model. Our best configuration outperforms DOVER-Lap in terms of DER while providing reliable confidence estimates essential for downstream applications. This work proposes best practices for probability-level fusion of EEND systems and demonstrates the advantages of leveraging soft outputs over hard decisions.

Method: Combines physical insights with machine learning to develop a physics-informed continuous cooling transformation (CCT) model for steels. Trained on a dataset of 4,100 diagrams and validated against literature and experimental data.

Result: High computational efficiency (generates complete CCT diagrams with 100 cooling curves in under 5 seconds). Strong generalizability across alloy steels with phase classification F1 scores above 88% for all phases. Phase transition temperature regression achieves MAE below 20°C for most phases (except bainite at 27°C).

Conclusion: The framework can be extended with additional ML models to establish a universal digital twin platform for heat treatment. Integration with simulation tools and targeted experiments will further accelerate materials design workflows.

Abstract: Machine learning (ML) has emerged as a powerful tool for accelerating the computational design and production of materials. In materials science, ML has primarily supported large-scale discovery of novel compounds using first-principles data and digital twin applications for optimizing manufacturing processes. However, applying general-purpose ML frameworks to complex industrial materials such as steel remains a challenge. A key obstacle is accurately capturing the intricate relationship between chemical composition, processing parameters, and the resulting microstructure and properties. To address this, we introduce a computational framework that combines physical insights with ML to develop a physics-informed continuous cooling transformation (CCT) model for steels. Our model, trained on a dataset of 4,100 diagrams, is validated against literature and experimental data. It demonstrates high computational efficiency, generating complete CCT diagrams with 100 cooling curves in under 5 seconds. It also shows strong generalizability across alloy steels, achieving phase classification F1 scores above 88% for all phases. For phase transition temperature regression, it attains mean absolute errors (MAE) below 20 °C across all phases except bainite, which shows a slightly higher MAE of 27 °C. This framework can be extended with additional generic and customized ML models to establish a universal digital twin platform for heat treatment. Integration with complementary simulation tools and targeted experiments will further support accelerated materials design workflows.

Method: Use LLMs as lossless encoder from source domain (LCTs) to destination domain (HDL code), then as lossless decoder back to source, comparing original and reconstructed LCTs for verification.

Result: Successfully generated full HDL for 2D network-on-chip router (13 units, 1500-2000 lines) using 7 LLMs, with approach detecting incorrect LLM logic and finding design specification errors.

Conclusion: Encoder-decoder approach yields significant productivity improvements by verifying correct logic generation, detecting errors, and assisting developers in finding specification errors.

Abstract: We show for invertible problems that transform data from a source domain (for example, Logic Condition Tables (LCTs)) to a destination domain (for example, Hardware Description Language (HDL) code), an approach of using Large Language Models (LLMs) as a lossless encoder from source to destination followed by as a lossless decoder back to the source, comparable to lossless compression in information theory, can mitigate most of the LLM drawbacks of hallucinations and omissions. Specifically, using LCTs as inputs, we generate the full HDL for a two-dimensional network-on-chip router (13 units, 1500-2000 lines of code) using seven different LLMs, reconstruct the LCTs from the auto-generated HDL, and compare the original and reconstructed LCTs. This approach yields significant productivity improvements, not only confirming correctly generated LLM logic and detecting incorrectly generated LLM logic but also assisting developers in finding design specification errors.

Method: Adaptive layer-freezing strategy that selectively freezes encoder weights based on monitored relative differences between rounds. Patience-based mechanism ensures freezing only when updates remain consistently minimal. Tested on MRI-to-CT conversion using different federated architectures, tracking energy/CO2eq with CodeCarbon.

Result: Reduced training time, energy consumption, and CO2eq emissions by up to 23% compared to non-frozen counterparts. Maintained MRI-to-CT conversion performance with small MAE variations. 3/5 architectures showed no statistically significant differences, 2 showed statistically significant improvements.

Conclusion: Proposed approach enables sustainable FL that meets clinical requirements while ensuring climatic, social, and economic sustainability. Advances privacy, equity, and justice in AI-driven healthcare through resource-efficient federated learning.

Abstract: Federated Learning (FL) holds the potential to advance equality in health by enabling diverse institutions to collaboratively train deep learning (DL) models, even with limited data. However, the significant resource requirements of FL often exclude centres with limited computational infrastructure, further widening existing healthcare disparities. To address this issue, we propose a Green AI-oriented adaptive layer-freezing strategy designed to reduce energy consumption and computational load while maintaining model performance. We tested our approach using different federated architectures for Magnetic Resonance Imaging (MRI)-to-Computed Tomography (CT) conversion. The proposed adaptive strategy optimises the federated training by selectively freezing the encoder weights based on the monitored relative difference of the encoder weights from round to round. A patience-based mechanism ensures that freezing only occurs when updates remain consistently minimal. The energy consumption and CO2eq emissions of the federation were tracked using the CodeCarbon library. Compared to equivalent non-frozen counterparts, our approach reduced training time, total energy consumption and CO2eq emissions by up to 23%. At the same time, the MRI-to-CT conversion performance was maintained, with only small variations in the Mean Absolute Error (MAE). Notably, for three out of the five evaluated architectures, no statistically significant differences were observed, while two architectures exhibited statistically significant improvements. Our work aligns with a research paradigm that promotes DL-based frameworks meeting clinical requirements while ensuring climatic, social, and economic sustainability. It lays the groundwork for novel FL evaluation frameworks, advancing privacy, equity and, more broadly, justice in AI-driven healthcare.

Method: PINS-CAD pre-trains graph neural networks on 200,000 synthetic coronary digital twins to predict pressure and flow, guided by 1D Navier-Stokes equations and pressure-drop laws. The framework eliminates need for CFD or labeled data during pretraining, then fine-tunes on clinical data.

Result: When fine-tuned on clinical data from 635 patients in the multicenter FAME2 study, PINS-CAD predicts future cardiovascular events with an AUC of 0.73, outperforming clinical risk scores and data-driven baselines. It also generates spatially resolved pressure and fractional flow reserve curves as interpretable biomarkers.

Conclusion: Physics-informed pretraining boosts sample efficiency and yields physiologically meaningful representations. By embedding physical priors into geometric deep learning, PINS-CAD transforms routine angiography into a simulation-free, physiology-aware framework for scalable, preventive cardiology.

Abstract: Cardiovascular disease is the leading global cause of mortality, with coronary artery disease (CAD) as its most prevalent form, necessitating early risk prediction. While 3D coronary artery digital twins reconstructed from imaging offer detailed anatomy for personalized assessment, their analysis relies on computationally intensive computational fluid dynamics (CFD), limiting scalability. Data-driven approaches are hindered by scarce labeled data and lack of physiological priors. To address this, we present PINS-CAD, a physics-informed self-supervised learning framework. It pre-trains graph neural networks on 200,000 synthetic coronary digital twins to predict pressure and flow, guided by 1D Navier-Stokes equations and pressure-drop laws, eliminating the need for CFD or labeled data. When fine-tuned on clinical data from 635 patients in the multicenter FAME2 study, PINS-CAD predicts future cardiovascular events with an AUC of 0.73, outperforming clinical risk scores and data-driven baselines. This demonstrates that physics-informed pretraining boosts sample efficiency and yields physiologically meaningful representations. Furthermore, PINS-CAD generates spatially resolved pressure and fractional flow reserve curves, providing interpretable biomarkers. By embedding physical priors into geometric deep learning, PINS-CAD transforms routine angiography into a simulation-free, physiology-aware framework for scalable, preventive cardiology.

Method: Delta Sampling operates at inference time by computing the delta (difference in model predictions before and after adaptation) from an old base model, then using this delta to guide the denoising process of a new base model without requiring training data.

Result: DS achieves consistent improvements in creating desired effects (visual styles, semantic concepts, structures) across various Stable Diffusion versions under different sampling strategies.

Conclusion: Delta Sampling provides an effective, plug-and-play mechanism for knowledge transfer in diffusion-based image synthesis, enabling reuse of adaptations across different model architectures.

Abstract: Diffusion models like Stable Diffusion (SD) drive a vibrant open-source ecosystem including fully fine-tuned checkpoints and parameter-efficient adapters such as LoRA, LyCORIS, and ControlNet. However, these adaptation components are tightly coupled to a specific base model, making them difficult to reuse when the base model is upgraded (e.g., from SD 1.x to 2.x) due to substantial changes in model parameters and architecture. In this work, we propose Delta Sampling (DS), a novel method that enables knowledge transfer across base models with different architectures, without requiring access to the original training data. DS operates entirely at inference time by leveraging the delta: the difference in model predictions before and after the adaptation of a base model. This delta is then used to guide the denoising process of a new base model. We evaluate DS across various SD versions, demonstrating that DS achieves consistent improvements in creating desired effects (e.g., visual styles, semantic concepts, and structures) under different sampling strategies. These results highlight DS as an effective, plug-and-play mechanism for knowledge transfer in diffusion-based image synthesis. Code:~ https://github.com/Zhidong-Gao/DeltaSampling

Method: Analyze the continuous-time limit dynamical system of pre-trained Transformers, characterize asymptotic behavior of token solutions, derive sufficient conditions for convergence/divergence based on model parameters, investigate effects of absolute and rotary positional encodings, and propose architectural refinements.

Result: Found broader conditions for token convergence/divergence than prior work, discovered that convergence behavior negatively impacts model performance, and showed how different positional encodings affect dynamical regimes. Proposed effective refinements to mitigate convergence issues.

Conclusion: The analysis provides theoretical foundations for understanding token dynamics in Transformers and offers practical design principles for improving Transformer architectures by addressing convergence problems in both absolute and rotary positional encoding schemes.

Abstract: This paper investigates the dynamical properties of tokens in pre-trained Transformer models and explores their application to improving Transformers. To this end, we analyze the dynamical system governing the continuous-time limit of the pre-trained model and characterize the asymptotic behavior of its solutions. Specifically, we characterize when tokens move closer to or farther from one another over time, depending on the model parameters. We provide sufficient conditions, based on these parameters, to identify scenarios where tokens either converge to zero or diverge to infinity. Unlike prior works, our conditions are broader in scope and more applicable to real-world models. Furthermore, we investigate how different forms of positional encoding – specifically absolute and rotary – affect these dynamical regimes. Empirical evidence reveals that the convergence scenario adversely impacts model performance. Motivated by these insights, we propose simple refinements to Transformer architectures that mitigate convergence behavior in models with absolute or rotary positional encoding. These findings support theoretical foundations and design principles for improving Transformer models.

Method: Combines AWS Strands Agents with Thompson Sampling contextual bandits to learn optimal decision-making from user feedback alone, optimizing generation strategy (direct vs. chain-of-thought), tool selection, and domain routing.

Result: 15-30% improvement in user satisfaction compared to random baselines, with learning patterns emerging after 20-30 queries. System adapts continuously without ground truth labels.

Conclusion: Provides a principled solution to exploration-exploitation dilemma in agentic AI systems, enabling adaptive learning from user feedback without expensive labeled data.

Abstract: Generative AI agents in life sciences face a critical challenge: determining the optimal approach for diverse queries ranging from simple factoid questions to complex mechanistic reasoning. Traditional methods rely on fixed rules or expensive labeled training data, neither of which adapts to changing conditions or user preferences. We present a novel framework that combines AWS Strands Agents with Thompson Sampling contextual bandits to enable AI agents to learn optimal decision-making strategies from user feedback alone. Our system optimizes three key dimensions: generation strategy selection (direct vs. chain-of-thought), tool selection (literature search, drug databases, etc.), and domain routing (pharmacology, molecular biology, clinical specialists). Through empirical evaluation on life science queries, we demonstrate 15-30% improvement in user satisfaction compared to random baselines, with clear learning patterns emerging after 20-30 queries. Our approach requires no ground truth labels, adapts continuously to user preferences, and provides a principled solution to the exploration-exploitation dilemma in agentic AI systems.

Method: Formulates the problem as a Constrained Markov Decision Process with options for temporal abstraction. Develops DAC-MAPPO-Lagrangian algorithm integrating Lagrangian relaxation into Double Actor-Critic framework. Uses hierarchical approach: centralized PPO-Lagrangian for safe charger allocation (high level) and MAPPO-Lagrangian for decentralized charging power decisions under CTDE paradigm (low level).

Result: Extensive experiments with real-world data show the proposed approach outperforms existing baselines in both cost minimization and safety compliance, while maintaining fast convergence speed.

Conclusion: The safe hierarchical deep reinforcement learning framework effectively solves the electric bus charging scheduling problem under multi-source uncertainties, achieving better performance than existing methods in cost reduction and safety assurance.

Abstract: The integration of Electric Buses (EBs) with renewable energy sources such as photovoltaic (PV) panels is a promising approach to promote sustainable and low-carbon public transportation. However, optimizing EB charging schedules to minimize operational costs while ensuring safe operation without battery depletion remains challenging - especially under real-world conditions, where uncertainties in PV generation, dynamic electricity prices, variable travel times, and limited charging infrastructure must be accounted for. In this paper, we propose a safe Hierarchical Deep Reinforcement Learning (HDRL) framework for solving the EB Charging Scheduling Problem (EBCSP) under multi-source uncertainties. We formulate the problem as a Constrained Markov Decision Process (CMDP) with options to enable temporally abstract decision-making. We develop a novel HDRL algorithm, namely Double Actor-Critic Multi-Agent Proximal Policy Optimization Lagrangian (DAC-MAPPO-Lagrangian), which integrates Lagrangian relaxation into the Double Actor-Critic (DAC) framework. At the high level, we adopt a centralized PPO-Lagrangian algorithm to learn safe charger allocation policies. At the low level, we incorporate MAPPO-Lagrangian to learn decentralized charging power decisions under the Centralized Training and Decentralized Execution (CTDE) paradigm. Extensive experiments with real-world data demonstrate that the proposed approach outperforms existing baselines in both cost minimization and safety compliance, while maintaining fast convergence speed.

Method: Framework combines results across many experiments using core components: experiment selection, base learner design, and incremental training. Leverages experimental data from hundreds of millions of Snapchat users to estimate heterogeneous treatment effects at scale.

Result: Framework successfully produces stable treatment effect estimates at scale and uncovers latent user characteristics. Applications include user influenceability to ads and user sensitivity to ads. Online A/B test using influenceability scores for targeting showed improvement on key business metrics more than six times larger than typical significance thresholds.

Conclusion: The industrial-scale HTE framework enables practical estimation of heterogeneous treatment effects, revealing valuable user characteristics that can significantly improve targeting effectiveness and business outcomes in real-world applications.

Abstract: Heterogeneous Treatment Effect (HTE) and Conditional Average Treatment Effect (CATE) models relax the assumption that treatment effects are the same for every user. We present a large scale industrial framework for estimating HTE using experimental data from hundreds of millions of Snapchat users. By combining results across many experiments, the framework uncovers latent user characteristics that were previously unmeasurable and produces stable treatment effect estimates at scale. We describe the core components that enabled this system, including experiment selection, base learner design, and incremental training. We also highlight two applications: user influenceability to ads and user sensitivity to ads. An online A/B test using influenceability scores for targeting showed an improvement on key business metrics that is more than six times larger than what is typically considered significant.

Method: Two novel approaches: (1) FermiGrad - gradient-descent algorithm that determines globally optimal layer-wise ranks by relaxing discrete singular-value truncation into continuous optimization using the Fermi function; (2) PivGa - lossless compression of low-rank factors by exploiting intrinsic gauge freedom in their parametrization.

Result: The paper presents physics-inspired improvements to SVD-based LLM compression, addressing key practical hurdles in low-rank decomposition approaches.

Conclusion: The proposed FermiGrad and PivGa methods offer practical solutions for more effective LLM compression through improved rank selection and parameter redundancy elimination.

Abstract: Large Language Models (LLMs) are very demanding in terms of their computational resources. Low-rank decompositions of LLM weights, e.g. via Singular Value Decomposition (SVD), is a promising approach for LLM compression, but presents several practical hurdles, e.g. selecting appropriate layer-wise ranks and getting rid of its parameter redundancy. In this work, we present two physics-inspired improvements to SVD LLM compression: (1) \textbf{FermiGrad}, a gradient-descent algorithm that determines globally optimal layer-wise ranks by relaxing the discrete singular-value truncation into a continuous optimization using the Fermi function; (2) \textbf{PivGa}, an additional \textit{lossless} compression of the low-rank factors that exploits the intrinsic gauge freedom in their parametrization.

Method: Used pretopology to model consumption profiles and developed a multi-criterion hierarchical classification algorithm based on pretopological space properties, implemented in a Python library. Evaluated on three datasets: 2D point clusters, generated time series, and real consumption data from 400 French energy sites.

Result: The algorithm successfully identified clusters in 2D point data using position and size parameters. On generated time series, it achieved perfect clustering with Adjusted Rand Index (ARI) of 1 using Pearson’s correlation.

Conclusion: The pretopology-based classification approach provides an effective automated solution for analyzing energy consumption patterns across large distributed building portfolios, enabling scalable optimization without resource-intensive individual audits.

Abstract: This article attempts answering the following problematic: How to model and classify energy consumption profiles over a large distributed territory to optimize the management of buildings’ consumption? Doing case-by-case in depth auditing of thousands of buildings would require a massive amount of time and money as well as a significant number of qualified people. Thus, an automated method must be developed to establish a relevant and effective recommendations system. To answer this problematic, pretopology is used to model the sites’ consumption profiles and a multi-criterion hierarchical classification algorithm, using the properties of pretopological space, has been developed in a Python library. To evaluate the results, three data sets are used: A generated set of dots of various sizes in a 2D space, a generated set of time series and a set of consumption time series of 400 real consumption sites from a French Energy company. On the point data set, the algorithm is able to identify the clusters of points using their position in space and their size as parameter. On the generated time series, the algorithm is able to identify the time series clusters using Pearson’s correlation with an Adjusted Rand Index (ARI) of 1.

Method: ATHENA uses a HENA (Hierarchical Evolutionary Numerical Algorithms) loop framed as a Contextual Bandit problem. It analyzes prior trials to select structural actions from combinatorial spaces guided by expert blueprints, translates actions into executable code, and generates scientific rewards. Combines hybrid symbolic-numeric workflows and supports human-in-the-loop collaboration.

Result: Achieves super-human performance with validation errors of 10^-14. Successfully identifies mathematical symmetries for exact analytical solutions, derives stable numerical solvers, tackles ill-posed formulations, and resolves multiphysics problems. Human collaboration improves results by an order of magnitude.

Conclusion: ATHENA represents a paradigm shift from implementation mechanics to methodological innovation, accelerating scientific discovery by autonomously managing computational research workflows while supporting human collaboration to bridge stability gaps.

Abstract: Bridging the gap between theoretical conceptualization and computational implementation is a major bottleneck in Scientific Computing (SciC) and Scientific Machine Learning (SciML). We introduce ATHENA (Agentic Team for Hierarchical Evolutionary Numerical Algorithms), an agentic framework designed as an Autonomous Lab to manage the end-to-end computational research lifecycle. Its core is the HENA loop, a knowledge-driven diagnostic process framed as a Contextual Bandit problem. Acting as an online learner, the system analyzes prior trials to select structural `actions’ ($A_n$) from combinatorial spaces guided by expert blueprints (e.g., Universal Approximation, Physics-Informed constraints). These actions are translated into executable code ($S_n$) to generate scientific rewards ($R_n$). ATHENA transcends standard automation: in SciC, it autonomously identifies mathematical symmetries for exact analytical solutions or derives stable numerical solvers where foundation models fail. In SciML, it performs deep diagnosis to tackle ill-posed formulations and combines hybrid symbolic-numeric workflows (e.g., coupling PINNs with FEM) to resolve multiphysics problems. The framework achieves super-human performance, reaching validation errors of $10^{-14}$. Furthermore, collaborative ``human-in-the-loop” intervention allows the system to bridge stability gaps, improving results by an order of magnitude. This paradigm shift focuses from implementation mechanics to methodological innovation, accelerating scientific discovery.

Method: Proposes a clustering method based on pretopological spaces, which provides hierarchical and explainable clustering suitable for mixed data types.

Result: Benchmarking against classical numerical clustering algorithms and existing pretopological approaches provides insights into the performance and effectiveness of the proposed method.

Conclusion: Pretopological space-based clustering offers a promising approach for handling mixed data in big data environments, providing structured and interpretable results that address limitations of traditional methods.

Abstract: The advent of the big data paradigm has transformed how industries manage and analyze information, ushering in an era of unprecedented data volume, velocity, and variety. Within this landscape, mixed-data clustering has become a critical challenge, requiring innovative methods that can effectively exploit heterogeneous data types, including numerical and categorical variables. Traditional clustering techniques, typically designed for homogeneous datasets, often struggle to capture the additional complexity introduced by mixed data, underscoring the need for approaches specifically tailored to this setting. Hierarchical and explainable algorithms are particularly valuable in this context, as they provide structured, interpretable clustering results that support informed decision-making. This paper introduces a clustering method grounded in pretopological spaces. In addition, benchmarking against classical numerical clustering algorithms and existing pretopological approaches yields insights into the performance and effectiveness of the proposed method within the big data paradigm.

Method: Introduces specialized neurons for modal operators (□ and ◇) operating over possible worlds based on Kripke semantics. The accessibility relation can be either user-defined or parameterized by a neural network. The framework is fully differentiable with learning driven by minimizing logical contradiction loss.

Result: Demonstrated on four case studies: grammatical guardrailing, axiomatic detection of the unknown, multi-agent epistemic trust, and detecting constructive deception in natural language negotiation. Shows increased logical consistency and interpretability without changing underlying task architecture.

Conclusion: MLNNs provide a flexible neurosymbolic framework that integrates modal logic reasoning with deep learning, enabling both enforcement of known logical rules and learning of logical structures from data while maintaining differentiability and interpretability.

Abstract: We propose Modal Logical Neural Networks (MLNNs), a neurosymbolic framework that integrates deep learning with the formal semantics of modal logic, enabling reasoning about necessity and possibility. Drawing on Kripke semantics, we introduce specialized neurons for the modal operators $\Box$ and $\Diamond$ that operate over a set of possible worlds, enabling the framework to act as a differentiable ``logical guardrail.’’ The architecture is highly flexible: the accessibility relation between worlds can either be fixed by the user to enforce known rules or, as an inductive feature, be parameterized by a neural network. This allows the model to optionally learn the relational structure of a logical system from data while simultaneously performing deductive reasoning within that structure. This versatile construction is designed for flexibility. The entire framework is differentiable from end to end, with learning driven by minimizing a logical contradiction loss. This not only makes the system resilient to inconsistent knowledge but also enables it to learn nonlinear relationships that can help define the logic of a problem space. We illustrate MLNNs on four case studies: grammatical guardrailing, axiomatic detection of the unknown, multi-agent epistemic trust, and detecting constructive deception in natural language negotiation. These experiments demonstrate how enforcing or learning accessibility can increase logical consistency and interpretability without changing the underlying task architecture.

Method: Pretopology-based algorithm using Disjunctive Normal Form to formulate customizable logical rules and adjustable hyperparameters for user-defined hierarchical cluster construction on heterogeneous datasets.

Result: Superior performance demonstrated through hierarchical dendrogram analysis and comparative clustering metrics, with accurate and interpretable cluster delineation directly from raw data, showing robustness in constructing meaningful clusters.

Conclusion: Significant advancement in clustering mixed data by departing from traditional dimensionality reduction techniques and using innovative logical rules that enhance both cluster formation and clarity, contributing to explainable clustering.

Abstract: This article presents a novel pretopology-based algorithm designed to address the challenges of clustering mixed data without the need for dimensionality reduction. Leveraging Disjunctive Normal Form, our approach formulates customizable logical rules and adjustable hyperparameters that allow for user-defined hierarchical cluster construction and facilitate tailored solutions for heterogeneous datasets. Through hierarchical dendrogram analysis and comparative clustering metrics, our method demonstrates superior performance by accurately and interpretably delineating clusters directly from raw data, thus preserving data integrity. Empirical findings highlight the algorithm’s robustness in constructing meaningful clusters and reveal its potential in overcoming issues related to clustered data explainability. The novelty of this work lies in its departure from traditional dimensionality reduction techniques and its innovative use of logical rules that enhance both cluster formation and clarity, thereby contributing a significant advancement to the discourse on clustering mixed data.

Method: Novel fairness regularization framework with differentiable regularization terms for equal opportunity and statistical parity, designed to work with partially available sensitive attributes.

Result: Significantly mitigates bias across fairness metrics while maintaining competitive node classification performance, outperforming baselines in fairness-accuracy trade-off with minimal accuracy degradation.

Conclusion: The proposed framework effectively addresses practical limitations of fairness-aware GNNs by working with partially available sensitive attributes, achieving better fairness-accuracy balance than existing methods.

Abstract: Graph Neural Networks (GNNs) have demonstrated exceptional efficacy in relational learning tasks, including node classification and link prediction. However, their application raises significant fairness concerns, as GNNs can perpetuate and even amplify societal biases against protected groups defined by sensitive attributes such as race or gender. These biases are often inherent in the node features, structural topology, and message-passing mechanisms of the graph itself. A critical limitation of existing fairness-aware GNN methods is their reliance on the strong assumption that sensitive attributes are fully available for all nodes during training–a condition that poses a practical impediment due to privacy concerns and data collection constraints. To address this gap, we propose a novel, model-agnostic fairness regularization framework designed for the realistic scenario where sensitive attributes are only partially available. Our approach formalizes a fairness-aware objective function that integrates both equal opportunity and statistical parity as differentiable regularization terms. Through a comprehensive empirical evaluation across five real-world benchmark datasets, we demonstrate that the proposed method significantly mitigates bias across key fairness metrics while maintaining competitive node classification performance. Results show that our framework consistently outperforms baseline models in achieving a favorable fairness-accuracy trade-off, with minimal degradation in predictive accuracy. The datasets and source code will be publicly released at https://github.com/mtavassoli/GNN-FC.

Method: Apply standard risk-sensitive (log-exponential) transform to conditional FM loss, showing it’s a natural upper-bound on meaningful conditional entropic FM objective. Small order expansion of gradient yields two interpretable first-order corrections: covariance preconditioning of FM residual and skew tail term favoring asymmetric/rare branches.

Result: On synthetic data designed to probe ambiguity and tails, risk-sensitive loss improves statistical metrics and recovers geometric structure more faithfully than standard rectified FM.

Conclusion: Tilted risk provides principled way to incorporate higher-order conditional information in Flow Matching, enabling better recovery of data manifold geometry and minority branches while maintaining tractable optimization.

Abstract: Tilted (entropic) risk, obtained by applying a log-exponential transform to a base loss, is a well established tool in statistics and machine learning for emphasizing rare or high loss events while retaining a tractable optimization problem. In this work, our aim is to interpret its structure for Flow Matching (FM). FM learns a velocity field that transports samples from a simple source distribution to data by integrating an ODE. In rectified FM, training pairs are obtained by linearly interpolating between a source sample and a data sample, and a neural velocity field is trained to predict the straight line displacement using a mean squared error loss. This squared loss collapses all velocity targets that reach the same space-time point into a single conditional mean, thereby ignoring higher order conditional information (variance, skewness, multi-modality) that encodes fine geometric structure about the data manifold and minority branches. We apply the standard risk-sensitive (log-exponential) transform to the conditional FM loss and show that the resulting tilted risk loss is a natural upper-bound on a meaningful conditional entropic FM objective defined at each space-time point. Furthermore, we show that a small order expansion of the gradient of this conditional entropic objective yields two interpretable first order corrections: covariance preconditioning of the FM residual, and a skew tail term that favors asymmetric or rare branches. On synthetic data designed to probe ambiguity and tails, the resulting risk-sensitive loss improves statistical metrics and recovers geometric structure more faithfully than standard rectified FM.

Method: ALARM integrates uncertainty quantification with quality-assurance techniques including reasoning chain, self-reflection, and MLLM ensemble, designed based on a rigorous probabilistic inference pipeline and computational process.

Result: Extensive empirical evaluations using real-world smart-home benchmark data and wound image classification data show ALARM’s superior performance and generic applicability across different domains for reliable decision-making.

Conclusion: ALARM demonstrates that integrating uncertainty quantification with quality-assurance techniques enables robust and accurate MLLM-based visual anomaly detection with reliable decision-making capabilities across diverse domains.

Abstract: The advance of Large Language Models (LLMs) has greatly stimulated research interest in developing multi-modal LLM (MLLM)-based visual anomaly detection (VAD) algorithms that can be deployed in complex environments. The challenge is that in these complex environments, the anomalies are sometimes highly contextual and also ambiguous, and thereby, uncertainty quantification (UQ) is a crucial capacity for an MLLM-based VAD system to succeed. In this paper, we introduce our UQ-supported MLLM-based VAD framework called ALARM. ALARM integrates UQ with quality-assurance techniques like reasoning chain, self-reflection, and MLLM ensemble for robust and accurate performance and is designed based on a rigorous probabilistic inference pipeline and computational process. Extensive empirical evaluations are conducted using the real-world smart-home benchmark data and wound image classification data, which shows ALARM’s superior performance and its generic applicability across different domains for reliable decision-making.

Method: MARS uses a Heterogeneous Agent Ensemble with unique risk profiles enforced by Safety-Critic networks, orchestrated by a Meta-Adaptive Controller (MAC) that dynamically shifts reliance between conservative and aggressive agents based on market conditions.

Result: Experiments on major international indexes show the framework significantly reduces maximum drawdown and volatility while maintaining competitive returns.

Conclusion: The two-tiered multi-agent approach leveraging behavioral diversity provides a disciplined portfolio robust across different market regimes without requiring explicit feature engineering.

Abstract: Reinforcement Learning (RL) has shown significant promise in automated portfolio management; however, effectively balancing risk and return remains a central challenge, as many models fail to adapt to dynamically changing market conditions. We propose Meta-controlled Agents for a Risk-aware System (MARS), a novel framework addressing this through a multi-agent, risk-aware approach. MARS replaces monolithic models with a Heterogeneous Agent Ensemble, where each agent’s unique risk profile is enforced by a Safety-Critic network to span behaviors from capital preservation to aggressive growth. A high-level Meta-Adaptive Controller (MAC) dynamically orchestrates this ensemble, shifting reliance between conservative and aggressive agents to minimize drawdown during downturns while seizing opportunities in bull markets. This two-tiered structure leverages behavioral diversity rather than explicit feature engineering to ensure a disciplined portfolio robust across market regimes. Experiments on major international indexes confirm that our framework significantly reduces maximum drawdown and volatility while maintaining competitive returns.

Method: Diffusion-driven Bayesian exploration framework using entropy-regularized sampling and covariance-scaled diffusion to expand posterior support. Includes Metropolis-Hastings check to validate proposals and keep inference adaptive to unexpected evidence.

Result: Matches reinforcement learning and planning baselines when priors are correct. Substantially outperforms classical SMC perturbations and RL-based methods under misalignment. Provides theoretical guarantees that DEPF resolves S-PSI while maintaining statistical rigor.

Conclusion: The proposed framework enables principled, real-time correction of early state estimation errors in high-stakes applications like emergency response, overcoming the fundamental limitation of S-PSI in particle filters.

Abstract: In emergency response and other high-stakes societal applications, early-stage state estimates critically shape downstream outcomes. Yet, these initial state estimates-often based on limited or biased information-can be severely misaligned with reality, constraining subsequent actions and potentially causing catastrophic delays, resource misallocation, and human harm. Under the stationary bootstrap baseline (zero transition and no rejuvenation), bootstrap particle filters exhibit Stationarity-Induced Posterior Support Invariance (S-PSI), wherein regions excluded by the initial prior remain permanently unexplorable, making corrections impossible even when new evidence contradicts current beliefs. While classical perturbations can in principle break this lock-in, they operate in an always-on fashion and may be inefficient. To overcome this, we propose a diffusion-driven Bayesian exploration framework that enables principled, real-time correction of early state estimation errors. Our method expands posterior support via entropy-regularized sampling and covariance-scaled diffusion. A Metropolis-Hastings check validates proposals and keeps inference adaptive to unexpected evidence. Empirical evaluations on realistic hazardous-gas localization tasks show that our approach matches reinforcement learning and planning baselines when priors are correct. It substantially outperforms classical SMC perturbations and RL-based methods under misalignment, and we provide theoretical guarantees that DEPF resolves S-PSI while maintaining statistical rigor.

Method: ECLIPSE combines entropy estimation via multi-sample clustering with a novel perplexity decomposition that measures how models use retrieved evidence. It treats hallucination as a mismatch between semantic entropy and evidence capacity, with a provably convex entropy-capacity objective.

Result: On a controlled financial QA dataset with GPT-3.5-turbo, ECLIPSE achieves ROC AUC of 0.89 and average precision of 0.90, substantially outperforming semantic entropy-only baseline (AUC 0.50). Ablation with Claude-3-Haiku shows AUC dropping to 0.59, demonstrating ECLIPSE’s dependence on calibrated token-level uncertainties.

Conclusion: ECLIPSE effectively detects hallucinations by analyzing evidence utilization patterns, with perplexity decomposition features showing the largest learned coefficients. The framework is logprob-native and depends on calibrated token-level uncertainties, positioning it as a controlled mechanism study for hallucination detection.

Abstract: Large language models (LLMs) produce fluent but unsupported answers - hallucinations - limiting safe deployment in high-stakes domains. We propose ECLIPSE, a framework that treats hallucination as a mismatch between a model’s semantic entropy and the capacity of available evidence. We combine entropy estimation via multi-sample clustering with a novel perplexity decomposition that measures how models use retrieved evidence. We prove that under mild conditions, the resulting entropy-capacity objective is strictly convex with a unique stable optimum. We evaluate on a controlled financial question answering dataset with GPT-3.5-turbo (n=200 balanced samples with synthetic hallucinations), where ECLIPSE achieves ROC AUC of 0.89 and average precision of 0.90, substantially outperforming a semantic entropy-only baseline (AUC 0.50). A controlled ablation with Claude-3-Haiku, which lacks token-level log probabilities, shows AUC dropping to 0.59 with coefficient magnitudes decreasing by 95% - demonstrating that ECLIPSE is a logprob-native mechanism whose effectiveness depends on calibrated token-level uncertainties. The perplexity decomposition features exhibit the largest learned coefficients, confirming that evidence utilization is central to hallucination detection. We position this work as a controlled mechanism study; broader validation across domains and naturally occurring hallucinations remains future work.

Method: Frames trajectory evaluation as sequential hypothesis testing problem, uses e-processes to develop statistically valid sequential tests that work at every step of agent’s trajectory, enabling online monitoring over arbitrarily long action sequences.

Result: E-valuator provides better statistical power and false alarm rate control than other strategies across six datasets and three agents, enables quick termination of problematic trajectories to save tokens.

Conclusion: Provides lightweight, model-agnostic framework to convert verifier heuristics into decision rules with statistical guarantees, enabling more reliable deployment of agentic systems.

Abstract: Agentic AI systems execute a sequence of actions, such as reasoning steps or tool calls, in response to a user prompt. To evaluate the success of their trajectories, researchers have developed verifiers, such as LLM judges and process-reward models, to score the quality of each action in an agent’s trajectory. Although these heuristic scores can be informative, there are no guarantees of correctness when used to decide whether an agent will yield a successful output. Here, we introduce e-valuator, a method to convert any black-box verifier score into a decision rule with provable control of false alarm rates. We frame the problem of distinguishing successful trajectories (that is, a sequence of actions that will lead to a correct response to the user’s prompt) and unsuccessful trajectories as a sequential hypothesis testing problem. E-valuator builds on tools from e-processes to develop a sequential hypothesis test that remains statistically valid at every step of an agent’s trajectory, enabling online monitoring of agents over arbitrarily long sequences of actions. Empirically, we demonstrate that e-valuator provides greater statistical power and better false alarm rate control than other strategies across six datasets and three agents. We additionally show that e-valuator can be used for to quickly terminate problematic trajectories and save tokens. Together, e-valuator provides a lightweight, model-agnostic framework that converts verifier heuristics into decisions rules with statistical guarantees, enabling the deployment of more reliable agentic systems.

Method: Sparse Isotonic Shapley Regression (SISR) simultaneously learns a monotonic transformation to restore additivity (without needing closed-form specification) and enforces L0 sparsity constraint on Shapley vectors. Uses Pool-Adjacent-Violators for efficient isotonic regression and normalized hard-thresholding for support selection.

Result: SISR recovers true transformations in wide scenarios, achieves strong support recovery even in high noise, and demonstrates that irrelevant features and inter-feature dependencies can induce substantial nonlinear payoff transformations. Experiments show SISR stabilizes attributions across payoff schemes and correctly filters irrelevant features while standard Shapley values suffer severe rank and sign distortions.

Conclusion: By unifying nonlinear transformation estimation with sparsity pursuit, SISR advances nonlinear explainability, providing a theoretically grounded and practical attribution framework that addresses fundamental limitations of traditional Shapley values.

Abstract: Shapley values, a gold standard for feature attribution in Explainable AI, face two primary challenges. First, the canonical Shapley framework assumes that the worth function is additive, yet real-world payoff constructions–driven by non-Gaussian distributions, heavy tails, feature dependence, or domain-specific loss scales–often violate this assumption, leading to distorted attributions. Secondly, achieving sparse explanations in high dimensions by computing dense Shapley values and then applying ad hoc thresholding is prohibitively costly and risks inconsistency. We introduce Sparse Isotonic Shapley Regression (SISR), a unified nonlinear explanation framework. SISR simultaneously learns a monotonic transformation to restore additivity–obviating the need for a closed-form specification–and enforces an L0 sparsity constraint on the Shapley vector, enhancing computational efficiency in large feature spaces. Its optimization algorithm leverages Pool-Adjacent-Violators for efficient isotonic regression and normalized hard-thresholding for support selection, yielding implementation ease and global convergence guarantees. Analysis shows that SISR recovers the true transformation in a wide range of scenarios and achieves strong support recovery even in high noise. Moreover, we are the first to demonstrate that irrelevant features and inter-feature dependencies can induce a true payoff transformation that deviates substantially from linearity. Experiments in regression, logistic regression, and tree ensembles demonstrate that SISR stabilizes attributions across payoff schemes, correctly filters irrelevant features while standard Shapley values suffer severe rank and sign distortions. By unifying nonlinear transformation estimation with sparsity pursuit, SISR advances the frontier of nonlinear explainability, providing a theoretically grounded and practical attribution framework.

Method: Proposes a Temporal Graph Neural Network (Temporal GNN) that leverages graph-based temporal relationships among key PV parameters (irradiance, module temperature, ambient temperature) to predict electrical power output and detect anomalies.

Result: The study is based on data collected from an outdoor rooftop facility in Lyon, France, including power measurements from a PV module and meteorological parameters.

Conclusion: The proposed Temporal GNN approach shows promise for advanced solar PV performance monitoring and anomaly detection by capturing complex temporal relationships in system parameters.

Abstract: The rapid growth of solar photovoltaic (PV) systems necessitates advanced methods for performance monitoring and anomaly detection to ensure optimal operation. In this study, we propose a novel approach leveraging Temporal Graph Neural Network (Temporal GNN) to predict solar PV output power and detect anomalies using environmental and operational parameters. The proposed model utilizes graph-based temporal relationships among key PV system parameters, including irradiance, module and ambient temperature to predict electrical power output. This study is based on data collected from an outdoor facility located on a rooftop in Lyon (France) including power measurements from a PV module and meteorological parameters.

Method: Combines PCA for dimensionality reduction, Bayesian neural network for mapping sparse strain measurements to PCA modes, and Hamiltonian Monte Carlo inference for uncertainty quantification.

Result: Achieved accurate strain field reconstruction (R² > 0.9) with real-time uncertainty fields, robust performance with noisy data and crack-induced singularities, and explicit aleatoric/epistemic uncertainty separation.

Conclusion: Framework advances SHM toward trustworthy digital twin deployment and risk-aware structural diagnostics by enabling local diagnosis of uncertainty sources and supporting reliable decision-making.

Abstract: Reliable real-time analysis of sensor data is essential for structural health monitoring (SHM) of high-value assets, yet a major challenge is to obtain spatially resolved full-field aleatoric and epistemic uncertainties for trustworthy decision-making. We present an integrated SHM framework that combines principal component analysis (PCA), a Bayesian neural network (BNN), and Hamiltonian Monte Carlo (HMC) inference, mapping sparse strain gauge measurements onto leading PCA modes to reconstruct full-field strain distributions with uncertainty quantification. The framework was validated through cyclic four-point bending tests on carbon fiber reinforced polymer (CFRP) specimens with varying crack lengths, achieving accurate strain field reconstruction (R squared value > 0.9) while simultaneously producing real-time uncertainty fields. A key contribution is that the BNN yields robust full-field strain reconstructions from noisy experimental data with crack-induced strain singularities, while also providing explicit representations of two complementary uncertainty fields. Considered jointly in full-field form, the aleatoric and epistemic uncertainty fields make it possible to diagnose at a local level, whether low-confidence regions are driven by data-inherent issues or by model-related limitations, thereby supporting reliable decision-making. Collectively, the results demonstrate that the proposed framework advances SHM toward trustworthy digital twin deployment and risk-aware structural diagnostics.

Method: Proposes Modality-Decoupled Experts (MoDE) - a lightweight architecture that isolates modality-specific updates to mitigate gradient conflicts and uses knowledge distillation to prevent forgetting while preserving pre-trained capabilities.

Result: MoDE significantly mitigates both inter- and intra-modal forgetting, outperforming prior continual learning baselines across diverse benchmarks in unified multimodal generation settings.

Conclusion: Explicit modality decoupling effectively addresses catastrophic forgetting in UMGMs, with MoDE providing a scalable solution that prevents gradient conflicts while maintaining multimodal generation capabilities.

Abstract: Unified Multimodal Generative Models (UMGMs) unify visual understanding and image generation within a single autoregressive framework. However, their ability to continually learn new tasks is severely hindered by catastrophic forgetting, both within a modality (intra-modal) and across modalities (inter-modal). While intra-modal forgetting has been studied in prior continual learning (CL) work, inter-modal forgetting remains largely unexplored. In this paper, we identify and empirically validate this phenomenon in UMGMs and provide a theoretical explanation rooted in gradient conflict between modalities. To address both intra- and inter-modal forgetting, we propose Modality-Decoupled Experts (MoDE), a lightweight and scalable architecture that isolates modality-specific updates to mitigate the gradient conflict and leverages knowledge distillation to prevent catastrophic forgetting and preserve pre-trained capabilities. Unlike previous CL methods that remain modality-coupled and suffer from modality gradient conflict, MoDE explicitly decouples modalities to prevent interference. Experiments across diverse benchmarks demonstrate that MoDE significantly mitigates both inter- and intra-modal forgetting, outperforming prior CL baselines in unified multimodal generation settings. Codes will be publicly available: https://github.com/Christina200/MoDE-official.git

Method: Frames structure elucidation as conditional generation using an atomic diffusion model built on a non-equivariant transformer architecture. Trained on a dataset of simulated 1D NMR spectra for over 111,000 natural products.

Result: ChefNMR predicts structures of challenging natural product compounds with over 65% accuracy, surpassing previous methods and demonstrating significant progress in automating structure elucidation.

Conclusion: This work represents a major step toward automating small-molecule structure elucidation and highlights the potential of deep learning to accelerate molecular discovery in natural products and therapeutics.

Abstract: Nuclear Magnetic Resonance (NMR) spectroscopy is a cornerstone technique for determining the structures of small molecules and is especially critical in the discovery of novel natural products and clinical therapeutics. Yet, interpreting NMR spectra remains a time-consuming, manual process requiring extensive domain expertise. We introduce ChefNMR (CHemical Elucidation From NMR), an end-to-end framework that directly predicts an unknown molecule’s structure solely from its 1D NMR spectra and chemical formula. We frame structure elucidation as conditional generation from an atomic diffusion model built on a non-equivariant transformer architecture. To model the complex chemical groups found in natural products, we generated a dataset of simulated 1D NMR spectra for over 111,000 natural products. ChefNMR predicts the structures of challenging natural product compounds with an unsurpassed accuracy of over 65%. This work takes a significant step toward solving the grand challenge of automating small-molecule structure elucidation and highlights the potential of deep learning in accelerating molecular discovery. Code is available at https://github.com/ml-struct-bio/chefnmr.

Method: VQ-VAE with k-mer tokenization learns discrete genomic patterns without references. Extended with masked reconstruction pretraining for missing data robustness and contrastive learning for discriminative embeddings.

Result: 99.52% mean token-level accuracy, 56.33% exact sequence match rate, 19.73% codebook utilization on SARS-CoV-2 wastewater data. Contrastive fine-tuning improved Silhouette scores by 35-42% with different embedding dimensions.

Conclusion: Reference-free VQ-VAE framework provides scalable, interpretable genomic surveillance for public health monitoring, overcoming limitations of traditional variant calling pipelines.

Abstract: Wastewater-based genomic surveillance has emerged as a powerful tool for population-level viral monitoring, offering comprehensive insights into circulating viral variants across entire communities. However, this approach faces significant computational challenges stemming from high sequencing noise, low viral coverage, fragmented reads, and the complete absence of labeled variant annotations. Traditional reference-based variant calling pipelines struggle with novel mutations and require extensive computational resources. We present a comprehensive framework for unsupervised viral variant detection using Vector-Quantized Variational Autoencoders (VQ-VAE) that learns discrete codebooks of genomic patterns from k-mer tokenized sequences without requiring reference genomes or variant labels. Our approach extends the base VQ-VAE architecture with masked reconstruction pretraining for robustness to missing data and contrastive learning for highly discriminative embeddings. Evaluated on SARS-CoV-2 wastewater sequencing data comprising approximately 100,000 reads, our VQ-VAE achieves 99.52% mean token-level accuracy and 56.33% exact sequence match rate while maintaining 19.73% codebook utilization (101 of 512 codes active), demonstrating efficient discrete representation learning. Contrastive fine-tuning with different projection dimensions yields substantial clustering improvements: 64-dimensional embeddings achieve +35% Silhouette score improvement (0.31 to 0.42), while 128-dimensional embeddings achieve +42% improvement (0.31 to 0.44), clearly demonstrating the impact of embedding dimensionality on variant discrimination capability. Our reference-free framework provides a scalable, interpretable approach to genomic surveillance with direct applications to public health monitoring.

Method: Proposes Interleaved Reasoning (IR) where models alternate between thinking and surfacing intermediate responses instead of “think-then-answer.” Introduces Plantain (Plan-Thought-Answer Interleaving) where the first intermediate response is an explicit step-by-step plan, enabling user intervention and feedback.

Result: Plantain yields ~6% improvement in pass@1 across challenging math reasoning and coding benchmarks while reducing time-to-first-response by over 60% compared to think-then-answer baselines.

Conclusion: Interleaved reasoning reduces perceived latency without compromising final response quality, and the plan-first Plantain approach enables user intervention while improving both accuracy and response time.

Abstract: Reasoning models often spend a significant amount of time thinking before they generate a visible response. In the meantime, they do not give the user any hints as to whether their reasoning is on the right track, and do not give the user any recourse to stop and correct them if their reasoning is flawed. This creates a frustrating, but unfortunately common, experience: the user’s time is wasted while the model reasons from a false premise that could have easily been corrected. In contrast, human speakers typically perform lightweight, incremental grounding acts to ensure that participants in the conversation are on the same page; here we ask if language models can learn to leverage a similar type of behavior? With this motivation, we propose interleaved reasoning (IR), in which the model alternates between thinking and surfacing intermediate responses, as an alternative to the standard “think-then-answer” approach. By providing useful information to the user earlier, IR reduces perceived latency, the time a user waits for an initial output, without compromising the quality of the final response. We further introduce a specialization of interleaved reasoning, Plantain (Plan-Thought-Answer Interleaving), where the first intermediate response is an explicit, step-by-step plan for executing the task. This plan-first strategy allows for user intervention and early feedback for subsequent reasoning steps. We demonstrate that Plantain yields an ~6% improvement in pass@1 across several challenging math reasoning and coding benchmarks, while reducing time-to-first-response by over 60% relative to think-then-answer baselines.

Method: FiRE/FiRE.1 uses sketching-based anomaly detection for single-cell RNA sequencing. Enhash employs projection hashing in an ensemble learner framework for concept drift detection in streaming data.

Result: FiRE/FiRE.1 demonstrated superior performance against state-of-the-art techniques for anomaly detection. Enhash proved highly competitive in both time and accuracy across various drift types.

Conclusion: The thesis presents two effective algorithms: FiRE/FiRE.1 for rare cell detection in single-cell data, and Enhash for efficient concept drift detection in streaming data, both showing strong performance advantages.

Abstract: This work introduces FiRE/FiRE.1, a novel sketching-based algorithm for anomaly detection to quickly identify rare cell sub-populations in large-scale single-cell RNA sequencing data. This method demonstrated superior performance against state-of-the-art techniques. Furthermore, the thesis proposes Enhash, a fast and resource-efficient ensemble learner that uses projection hashing to detect concept drift in streaming data, proving highly competitive in time and accuracy across various drift types.

Method: Developed several improved algorithms for learning policies with memory in infinite-horizon settings: directly when a known environment model is available, and via simulation otherwise.

Result: Algorithms were compared on large POMDPs including noisy robot navigation and multi-agent problems, demonstrating their effectiveness for memory-based policies.

Conclusion: The paper presents improved policy-gradient algorithms that successfully address the challenge of learning policies with memory in partially observable environments, advancing beyond previous limitations with memoryless approaches.

Abstract: Policy-gradient methods have received increased attention recently as a mechanism for learning to act in partially observable environments. They have shown promise for problems admitting memoryless policies but have been less successful when memory is required. In this paper we develop several improved algorithms for learning policies with memory in an infinite-horizon setting – directly when a known model of the environment is available, and via simulation otherwise. We compare these algorithms on some large POMDPs, including noisy robot navigation and multi-agent problems.

Method: Applied OLPOMDP (policy-gradient reinforcement learning algorithm) to simulated network routing with multiple distributed agents (routers). Used reward shaping by explicitly penalizing certain patterns of sub-optimal behavior to improve convergence.

Result: Distributed routers successfully learned cooperative behavior without explicit inter-agent communication, avoided individually desirable but group-detrimental behavior, and reward shaping dramatically improved convergence rate.

Conclusion: OLPOMDP is effective for network routing problems, enabling distributed learning of cooperative behavior, and reward shaping significantly enhances learning efficiency.

Abstract: Network routing is a distributed decision problem which naturally admits numerical performance measures, such as the average time for a packet to travel from source to destination. OLPOMDP, a policy-gradient reinforcement learning algorithm, was successfully applied to simulated network routing under a number of network models. Multiple distributed agents (routers) learned co-operative behavior without explicit inter-agent communication, and they avoided behavior which was individually desirable, but detrimental to the group’s overall performance. Furthermore, shaping the reward signal by explicitly penalizing certain patterns of sub-optimal behavior was found to dramatically improve the convergence rate.

Method: Used linear probing with embeddings from Perch 2.0 for few-shot transfer learning on marine mammal and underwater audio tasks. Compared performance against other pretrained bioacoustics models including multispecies whale models, Perch 1.0, SurfPerch, AVES-bio, BirdAVES, and Birdnet V2.3.

Result: Perch 2.0 embeddings consistently achieved high performance for few-shot transfer learning, generally outperforming alternative embedding models on most marine mammal classification tasks.

Conclusion: Perch 2.0 is recommended for developing new linear classifiers for marine mammal classification with few labeled examples, demonstrating effective transfer learning capabilities from terrestrial to marine bioacoustics despite minimal marine training data.

Abstract: Perch 2.0 is a supervised bioacoustics foundation model pretrained on 14,597 species, including birds, mammals, amphibians, and insects, and has state-of-the-art performance on multiple benchmarks. Given that Perch 2.0 includes almost no marine mammal audio or classes in the training data, we evaluate Perch 2.0 performance on marine mammal and underwater audio tasks through few-shot transfer learning. We perform linear probing with the embeddings generated from this foundation model and compare performance to other pretrained bioacoustics models. In particular, we compare Perch 2.0 with previous multispecies whale, Perch 1.0, SurfPerch, AVES-bio, BirdAVES, and Birdnet V2.3 models, which have open-source tools for transfer-learning and agile modeling. We show that the embeddings from the Perch 2.0 model have consistently high performance for few-shot transfer learning, generally outperforming alternative embedding models on the majority of tasks, and thus is recommended when developing new linear classifiers for marine mammal classification with few labeled examples.

Method: Three-stage framework: 1) Generator produces diverse solutions, verifier evaluates them using parallel scaling (self-consistency) and sequential scaling (meta-critique). 2) Use verification outputs as synthetic training data to fine-tune generative process reward models. 3) Apply PRM with chain-of-thought verification as reward model in RL experiments, with format constraints to prevent reward hacking.

Result: Achieved 67.5 F1 on ProcessBench (vs 66.4 for reference-guided training and 61.9 for GPT-4o). Using Qwen2.5-Math-7B, achieved 47.4% average accuracy across six mathematical reasoning benchmarks, outperforming ground-truth-based RLVR (43.9%).

Conclusion: SPARK enables reference-free RL training that exceeds ground-truth methods, opening new possibilities for domains lacking verifiable answers or accessible ground truth.

Abstract: Process reward models (PRMs) that provide dense, step-level feedback have shown promise for reinforcement learning, yet their adoption remains limited by the need for expensive step-level annotations or ground truth references. We propose SPARK: a three-stage framework where in the first stage a generator model produces diverse solutions and a verifier model evaluates them using parallel scaling (self-consistency) and sequential scaling (meta-critique). In the second stage, we use these verification outputs as synthetic training data to fine-tune generative process reward models, which subsequently serve as reward signals during training. We show that aggregating multiple independent verifications at the step level produces training data for process reward models that surpass ground-truth outcome supervision, achieving 67.5 F1 on ProcessBench (a benchmark for identifying erroneous steps in mathematical reasoning) compared to 66.4 for reference-guided training and 61.9 for GPT-4o. In the final stage, we apply our generative PRM with chain-of-thought verification (PRM-CoT) as the reward model in RL experiments on mathematical reasoning, and introduce format constraints to prevent reward hacking. Using Qwen2.5-Math-7B, we achieve 47.4% average accuracy across six mathematical reasoning benchmarks, outperforming ground-truth-based RLVR (43.9%). Our work enables reference-free RL training that exceeds ground-truth methods, opening new possibilities for domains lacking verifiable answers or accessible ground truth.

Method: Benchmarked 14 VLMs with various architectures, sizes, and training setups on factual recall tasks. Used attribution patching, activation patching, and probing to analyze three high-performing and two low-performing models. Tested two recovery methods: patching entity representations from LLM backbone and chain-of-thought prompting.

Result: 11 of 14 VLMs exhibited factual recall degradation. Degraded models struggle to use existing factual recall circuits because they form entity representations too late in computation. High-performing VLMs resolve entities early enough to reuse LLM mechanisms. Both recovery methods successfully improved performance.

Conclusion: Early entity resolution is critical for VLMs to effectively use preexisting LLM mechanisms. Mechanistic analysis can explain systematic failures in multimodal alignment and guide better VLM design.

Abstract: Training vision language models (VLMs) aims to align visual representations from a vision encoder with the textual representations of a pretrained large language model (LLM). However, many VLMs exhibit reduced factual recall performance compared to their LLM backbones, raising the question of how effective multimodal fine-tuning is at extending existing mechanisms within the LLM to visual inputs. We argue that factual recall based on visual inputs requires VLMs to solve a two-hop problem: (1) forming entity representations from visual inputs, and (2) recalling associated factual knowledge based on these entity representations. By benchmarking 14 VLMs with various architectures (LLaVA, Native, Cross-Attention), sizes (7B-124B parameters), and training setups on factual recall tasks against their original LLM backbone models, we find that 11 of 14 models exhibit factual recall degradation. We select three models with high and two models with low performance degradation, and use attribution patching, activation patching, and probing to show that degraded VLMs struggle to use the existing factual recall circuit of their LLM backbone, because they resolve the first hop too late in the computation. In contrast, high-performing VLMs resolve entity representations early enough to reuse the existing factual recall mechanism. Finally, we demonstrate two methods to recover performance: patching entity representations from the LLM backbone into the VLM, and prompting with chain-of-thought reasoning. Our results highlight that the speed of early entity resolution critically determines how effective VLMs are in using preexisting LLM mechanisms. More broadly, our work illustrates how mechanistic analysis can explain and unveil systematic failures in multimodal alignment.

Method: Created a dataset of 12,000+ unique BWB geometries with steady RANS CFD simulations providing integrated coefficients (CL, CD, CM) and dense surface fields (Cp, Cf). Established forward-surrogate benchmark across six model families and inverse design using conditional diffusion model with comparison to gradient-based optimization and hybrid approaches.

Result: Provides a comprehensive dataset of 12,490 aerodynamic results, standardized forward prediction benchmarks across multiple model architectures, and inverse design protocols with comparative evaluation of different optimization paradigms.

Conclusion: BlendedNet++ offers a unified framework for reproducible research in field-level aerodynamics and inverse design, enabling fair comparison across architectures and optimization methods, with resources to be released publicly.

Abstract: Despite progress in machine learning-based aerodynamic surrogates, the scarcity of large, field-resolved datasets limits progress on accurate pointwise prediction and reproducible inverse design for aircraft. We introduce BlendedNet++, a large-scale aerodynamic dataset and benchmark focused on blended wing body (BWB) aircraft. The dataset contains over 12,000 unique geometries, each simulated at a single flight condition, yielding 12,490 aerodynamic results for steady RANS CFD. For every case, we provide (i) integrated force/moment coefficients CL, CD, CM and (ii) dense surface fields of pressure and skin friction coefficients Cp and (Cfx, Cfy, Cfz). Using this dataset, we standardize a forward-surrogate benchmark to predict pointwise fields across six model families: GraphSAGE, GraphUNet, PointNet, a coordinate Transformer (Transolver-style), a FiLMNet (coordinate MLP with feature-wise modulation), and a Graph Neural Operator Transformer (GNOT). Finally, we present an inverse design task of achieving a specified lift-to-drag ratio under fixed flight conditions, implemented via a conditional diffusion model. To assess performance, we benchmark this approach against gradient-based optimization on the same surrogate and a diffusion-optimization hybrid that first samples with the conditional diffusion model and then further optimizes the designs. BlendedNet++ provides a unified forward and inverse protocol with multi-model baselines, enabling fair, reproducible comparison across architectures and optimization paradigms. We expect BlendedNet++ to catalyze reproducible research in field-level aerodynamics and inverse design; resources (dataset, splits, baselines, and scripts) will be released upon acceptance.

Method: Proposes multi-frequency Federated Learning (FL) approach that enables privacy-aware machine learning and joint model learning across devices with varying sampling frequencies. Focuses on head-worn devices as the application domain.

Result: Shows improvements on two datasets compared to frequency-specific approaches. The proposed network implementation is publicly available for further research and development.

Conclusion: Multi-frequency FL-HAR shows promising future for privacy-aware activity recognition using head-worn devices, successfully addressing the challenge of varying sampling frequencies across devices.

Abstract: Human Activity Recognition (HAR) benefits various application domains, including health and elderly care. Traditional HAR involves constructing pipelines reliant on centralized user data, which can pose privacy concerns as they necessitate the uploading of user data to a centralized server. This work proposes multi-frequency Federated Learning (FL) to enable: (1) privacy-aware ML; (2) joint ML model learning across devices with varying sampling frequency. We focus on head-worn devices (e.g., earbuds and smart glasses), a relatively unexplored domain compared to traditional smartwatch- or smartphone-based HAR. Results have shown improvements on two datasets against frequency-specific approaches, indicating a promising future in the multi-frequency FL-HAR task. The proposed network’s implementation is publicly available for further research and development.

Method: ASPEN integrates an adaptive spectral layer with learnable Fourier features into the network’s input stage, allowing dynamic tuning of the spectral basis during training to efficiently learn required frequency content.

Result: ASPEN successfully solves the complex Ginzburg-Landau equation with exceptional accuracy (median physics residual: 5.10×10^-3), while standard PINNs catastrophically fail. The solution captures emergent physical properties including free energy relaxation and domain wall stability.

Conclusion: By incorporating an adaptive spectral basis, ASPEN provides a robust, physically-consistent solver for complex dynamical systems where standard PINNs fail, opening new possibilities for machine learning in challenging physical domains.

Abstract: Physics-Informed Neural Networks (PINNs) have emerged as a powerful, mesh-free paradigm for solving partial differential equations (PDEs). However, they notoriously struggle with stiff, multi-scale, and nonlinear systems due to the inherent spectral bias of standard multilayer perceptron (MLP) architectures, which prevents them from adequately representing high-frequency components. In this work, we introduce the Adaptive Spectral Physics-Enabled Network (ASPEN), a novel architecture designed to overcome this critical limitation. ASPEN integrates an adaptive spectral layer with learnable Fourier features directly into the network’s input stage. This mechanism allows the model to dynamically tune its own spectral basis during training, enabling it to efficiently learn and represent the precise frequency content required by the solution. We demonstrate the efficacy of ASPEN by applying it to the complex Ginzburg-Landau equation (CGLE), a canonical and challenging benchmark for nonlinear, stiff spatio-temporal dynamics. Our results show that a standard PINN architecture catastrophically fails on this problem, diverging into non-physical oscillations. In contrast, ASPEN successfully solves the CGLE with exceptional accuracy. The predicted solution is visually indistinguishable from the high-resolution ground truth, achieving a low median physics residual of 5.10 x 10^-3. Furthermore, we validate that ASPEN’s solution is not only pointwise accurate but also physically consistent, correctly capturing emergent physical properties, including the rapid free energy relaxation and the long-term stability of the domain wall front. This work demonstrates that by incorporating an adaptive spectral basis, our framework provides a robust and physically-consistent solver for complex dynamical systems where standard PINNs fail, opening new options for machine learning in challenging physical domains.

Method: Combine Deep Switching State Space Models with Adaptive Conformal Inference (ACI) and aggregated variant (AgACI). Also introduce unified conformal wrapper that works with various sequence baselines (S4, MC-Dropout GRU, sparse Gaussian processes, change-point local model) to produce online predictive bands.

Result: Conformalized forecasters achieve near-nominal coverage with competitive accuracy and generally improved band efficiency across synthetic and real datasets.

Conclusion: The approach provides finite-sample marginal guarantees for uncertainty quantification under nonstationarity and model misspecification, making it valuable for regime-switching forecasting applications.

Abstract: Regime transitions routinely break stationarity in time series, making calibrated uncertainty as important as point accuracy. We study distribution-free uncertainty for regime-switching forecasting by coupling Deep Switching State Space Models with Adaptive Conformal Inference (ACI) and its aggregated variant (AgACI). We also introduce a unified conformal wrapper that sits atop strong sequence baselines including S4, MC-Dropout GRU, sparse Gaussian processes, and a change-point local model to produce online predictive bands with finite-sample marginal guarantees under nonstationarity and model misspecification. Across synthetic and real datasets, conformalized forecasters achieve near-nominal coverage with competitive accuracy and generally improved band efficiency.

Method: HydroDCM uses spatial metadata to construct pseudo-domain labels that guide adversarial learning of invariant temporal features. During inference, lightweight conditioning layers adapt these features based on target reservoir’s metadata, combining domain invariance with location-specific adaptation.

Result: Experiments on 30 real-world reservoirs in the Upper Colorado River Basin show HydroDCM substantially outperforms state-of-the-art DG baselines under many-domain conditions while remaining computationally efficient.

Conclusion: HydroDCM provides a scalable DG framework for cross-reservoir inflow forecasting that effectively addresses domain shift in hydrological systems by reconciling domain invariance with location-specific adaptation through metadata-informed conditioning.

Abstract: Deep learning models have shown promise in reservoir inflow prediction, yet their performance often deteriorates when applied to different reservoirs due to distributional differences, referred to as the domain shift problem. Domain generalization (DG) solutions aim to address this issue by extracting domain-invariant representations that mitigate errors in unseen domains. However, in hydrological settings, each reservoir exhibits unique inflow patterns, while some metadata beyond observations like spatial information exerts indirect but significant influence. This mismatch limits the applicability of conventional DG techniques to many-domain hydrological systems. To overcome these challenges, we propose HydroDCM, a scalable DG framework for cross-reservoir inflow forecasting. Spatial metadata of reservoirs is used to construct pseudo-domain labels that guide adversarial learning of invariant temporal features. During inference, HydroDCM adapts these features through light-weight conditioning layers informed by the target reservoir’s metadata, reconciling DG’s invariance with location-specific adaptation. Experiment results on 30 real-world reservoirs in the Upper Colorado River Basin demonstrate that our method substantially outperforms state-of-the-art DG baselines under many-domain conditions and remains computationally efficient.

Method: The authors propose Robust Tabular Foundation Models (RTFM), a model-agnostic adversarial training framework. They formalize an optimality gap measure - the difference between TFM performance and best achievable performance estimated by strong baselines (XGBoost, CatBoost, Random Forests). The framework adapts synthetic data generators to emphasize challenging datasets during training.

Result: Applied to TabPFN V2 classifier, RTFM improves benchmark performance with up to 6% increase in mean normalized AUC over original TabPFN and other baseline algorithms, while requiring less than 100k additional synthetic datasets.

Conclusion: RTFM demonstrates a promising new direction for targeted adversarial training and fine-tuning of tabular foundation models using synthetic data alone, showing that adversarial robustness perspectives can effectively improve TFM performance with minimal additional synthetic data requirements.

Abstract: The development of tabular foundation models (TFMs) has accelerated in recent years, showing strong potential to outperform traditional ML methods for structured data. A key finding is that TFMs can be pretrained entirely on synthetic datasets, opening opportunities to design data generators that encourage desirable model properties. Prior work has mainly focused on crafting high-quality priors over generators to improve overall pretraining performance. Our insight is that parameterizing the generator distribution enables an adversarial robustness perspective: during training, we can adapt the generator to emphasize datasets that are particularly challenging for the model. We formalize this by introducing an optimality gap measure, given by the difference between TFM performance and the best achievable performance as estimated by strong baselines such as XGBoost, CatBoost, and Random Forests. Building on this idea, we propose Robust Tabular Foundation Models (RTFM), a model-agnostic adversarial training framework. Applied to the TabPFN V2 classifier, RTFM improves benchmark performance, with up to a 6% increase in mean normalized AUC over the original TabPFN and other baseline algorithms, while requiring less than 100k additional synthetic datasets. These results highlight a promising new direction for targeted adversarial training and fine-tuning of TFMs using synthetic data alone.

Method: Developed two operator architectures (Inception U-Net and multi-scale network M&M) based on U-Net backbone that combine diverse upsampling and receptive fields to capture multiscale nonlinear features. These operators map instantaneous model states to bias-correction tendencies applied online during integration. Trained on 2 years of E3SM simulations nudged toward ERA5 reanalysis.

Result: Both architectures outperform standard U-Net baselines in offline tests. M&M delivers most consistent bias reductions across variables and vertical levels in online hybrid E3SM runs. ML-augmented configurations remain stable and computationally feasible in multi-year simulations, generalizing across height levels and seasons.

Conclusion: The framework provides a practical pathway for scalable hybrid modeling with long-term stability, portability, and cadence-limited updates. Demonstrates utility of expressive ML operators for learning structured, cross-scale relationships and retrofitting legacy Earth-system models.

Abstract: Coarse resolution, imperfect parameterizations, and uncertain initial states and forcings limit Earth-system model (ESM) predictions. Traditional bias correction via data assimilation improves constrained simulations but offers limited benefit once models run freely. We introduce an operator-learning framework that maps instantaneous model states to bias-correction tendencies and applies them online during integration. Building on a U-Net backbone, we develop two operator architectures Inception U-Net (IUNet) and a multi-scale network (M&M) that combine diverse upsampling and receptive fields to capture multiscale nonlinear features under Energy Exascale Earth System Model (E3SM) runtime constraints. Trained on two years E3SM simulations nudged toward ERA5 reanalysis, the operators generalize across height levels and seasons. Both architectures outperform standard U-Net baselines in offline tests, indicating that functional richness rather than parameter count drives performance. In online hybrid E3SM runs, M&M delivers the most consistent bias reductions across variables and vertical levels. The ML-augmented configurations remain stable and computationally feasible in multi-year simulations, providing a practical pathway for scalable hybrid modeling. Our framework emphasizes long-term stability, portability, and cadence-limited updates, demonstrating the utility of expressive ML operators for learning structured, cross-scale relationships and retrofitting legacy ESMs.

Method: Learns each token’s intrinsic importance via lightweight retention gates that predict scalar retention scores decaying over time. Tokens with low scores are evicted when memory budget is exceeded. Trained efficiently through distillation from frozen LLM with capacity loss, requiring only gate fine-tuning.

Result: Consistently outperforms strong eviction and learnable retrieval baselines across mathematical reasoning, procedural generation, conversational long-memory, and long-context understanding benchmarks, especially in low-memory regimes. Sometimes surpasses full-cache models, showing selective retention can serve as regularization.

Conclusion: TRIM-KV provides efficient KV cache management while offering interpretability insights into layer- and head-specific token importance. Learned retention scores align with human intuition and naturally recover established heuristics without explicit design.

Abstract: Memory and computation remain core bottlenecks in long-horizon LLM inference due to the quadratic cost of self-attention and the ever-growing key-value (KV) cache. Existing strategies for memory-bounded inference, such as quantization, offloading, or heuristic KV eviction, either incur high orchestration costs or rely on unreliable attention-based proxies of importance. We propose TRIM-KV, a novel approach that learns each token’s intrinsic importance at creation time via a lightweight retention gate. Each gate predicts a scalar retention score that decays over time, reflecting the long-term utility of the token for a specific layer and head. Tokens with low scores are evicted when the memory budget is exceeded, ensuring that the cache always contains the most critical tokens. TRIM-KV is trained efficiently through distillation from a frozen LLM combined with a capacity loss, requiring only gate fine-tuning and adding negligible inference overhead. Across mathematical reasoning (GSM8K, MATH-500, AIME24), procedural generation (LongProc), conversational long-memory benchmarks (LongMemEval), and long-context understanding (LongBench and SCBench), TRIM-KV consistently outperforms strong eviction and learnable retrieval baselines, especially in low-memory regimes. Remarkably, it even surpasses full-cache models in some settings, showing that selective retention can serve as a form of regularization, suppressing noise from uninformative tokens. Qualitative analyses further reveal that learned retention scores align with human intuition, naturally recovering heuristics such as sink tokens, sliding windows, and gist compression without explicit design. Beyond efficiency, retention scores provide insights into layer- and head-specific roles, suggesting a new path toward LLM interpretability.

Method: SAFLe introduces a structured architecture with bucketed features and sparse, grouped embeddings. The key innovation is that this non-linear architecture is mathematically proven to be equivalent to high-dimensional linear regression, enabling the use of AFL’s single-shot, invariant aggregation law.

Result: SAFLe establishes new state-of-the-art for analytic FL, significantly outperforming both linear AFL and multi-round DeepAFL in accuracy across all benchmarks, while maintaining the single-round efficiency benefit.

Conclusion: SAFLe successfully breaks the trade-off between single-round efficiency and non-linear expressivity in federated learning, providing a highly efficient and scalable solution for federated vision tasks.

Abstract: Federated Learning (FL) is plagued by two key challenges: high communication overhead and performance collapse on heterogeneous (non-IID) data. Analytic FL (AFL) provides a single-round, data distribution invariant solution, but is limited to linear models. Subsequent non-linear approaches, like DeepAFL, regain accuracy but sacrifice the single-round benefit. In this work, we break this trade-off. We propose SAFLe, a framework that achieves scalable non-linear expressivity by introducing a structured head of bucketed features and sparse, grouped embeddings. We prove this non-linear architecture is mathematically equivalent to a high-dimensional linear regression. This key equivalence allows SAFLe to be solved with AFL’s single-shot, invariant aggregation law. Empirically, SAFLe establishes a new state-of-the-art for analytic FL, significantly outperforming both linear AFL and multi-round DeepAFL in accuracy across all benchmarks, demonstrating a highly efficient and scalable solution for federated vision.

Method: Repurposes bid landscape models to approximate propensity scores, enabling use of stable estimators like Self-Normalized Inverse Propensity Scoring (SNIPS) for counterfactual evaluation in deterministic auctions.

Result: Achieves 92% Mean Directional Accuracy in CTR prediction on AuctionNet benchmark and real-world platform data, significantly outperforming parametric baselines and aligning well with online A/B test results.

Conclusion: Provides the first practical and validated framework for reliable OPE in deterministic auction environments, offering an efficient alternative to costly and risky online experiments.

Abstract: Online A/B testing, the gold standard for evaluating new advertising policies, consumes substantial engineering resources and risks significant revenue loss from deploying underperforming variations. This motivates the use of Off-Policy Evaluation (OPE) for rapid, offline assessment. However, applying OPE to ad auctions is fundamentally more challenging than in domains like recommender systems, where stochastic policies are common. In online ad auctions, it is common for the highest-bidding ad to win the impression, resulting in a deterministic, winner-takes-all setting. This results in zero probability of exposure for non-winning ads, rendering standard OPE estimators inapplicable. We introduce the first principled framework for OPE in deterministic auctions by repurposing the bid landscape model to approximate the propensity score. This model allows us to derive robust approximate propensity scores, enabling the use of stable estimators like Self-Normalized Inverse Propensity Scoring (SNIPS) for counterfactual evaluation. We validate our approach on the AuctionNet simulation benchmark and against 2-weeks online A/B test from a large-scale industrial platform. Our method shows remarkable alignment with online results, achieving a 92% Mean Directional Accuracy (MDA) in CTR prediction, significantly outperforming the parametric baseline. MDA is the most critical metric for guiding deployment decisions, as it reflects the ability to correctly predict whether a new model will improve or harm performance. This work contributes the first practical and validated framework for reliable OPE in deterministic auction environments, offering an efficient alternative to costly and risky online experiments.

Method: UniQL integrates quantization and low-rank compression for Transformers, SSMs, and hybrid models. It features efficient structured weight-sorting (20x speedup), quantization-aware SVD, state-aware weight sorting for SSMs, and fused RoPE kernel for pruned models. The framework performs weight-sorting, fine-tuning, and quantization in a single-pass cloud workflow.

Result: Achieves 4x-5.7x memory reduction and 2.7x-3.4x token-throughput improvement while maintaining accuracy within 5% of original models at 15% pruning across diverse architectures (Llama3, Qwen2.5, Mamba2, Nemotron-H, Bamba-v2).

Conclusion: UniQL provides an effective unified framework for compressing edge LLMs with on-device configurable pruning, enabling efficient deployment on resource-constrained mobile platforms while preserving model accuracy.

Abstract: Deploying large language model (LLM) models on mobile platforms faces significant challenges due to the limited memory and shared computational resources of the device. Resource availability may be an issue as it is directly impacted by the current device workload, adding to the uncertainty of model deployment. We introduce UniQL, a unified post-training quantization and low-rank compression framework with on-device configurable pruning rates for edge LLMs. UniQL is a general framework that integrates quantization and low-rank compression for Transformers, State Space Models (SSMs), and hybrid models to support diverse edge applications. In our proposed joint framework, we introduce an efficient structured weight-sorting method that speeds up computation by 20x, quantization-aware singular value decomposition (SVD) to minimize quantization errors, state-aware weight sorting for SSMs, and a fused rotary positional embedding (RoPE) kernel for pruned models. Our framework performs weight-sorting, fine-tuning, and quantization in the cloud in a single-pass workflow, while enabling on-device configurable pruning rates up to 35%. Our experiments show that quantized and pruned models achieve a memory reduction of 4x-5.7x and a token-throughput improvement of 2.7x-3.4x, maintaining accuracy within 5% of the original models at 15% pruning across Transformers (Llama3 and Qwen2.5), SSMs (Mamba2), and hybrid models (Nemotron-H and Bamba-v2). The code and quantized models are available at: https://github.com/enyac-group/UniQL.

Method: Introduces A2G (Adaptive Aggregation with Two Gains), a dual-gain framework that jointly regulates geometric blending through a geometry gain and modulates client importance using a QoS gain derived from teleportation fidelity, latency, and instability. Develops the A2G update rule with convergence guarantees under smoothness and bounded variance assumptions.

Result: A2G recovers FedAvg, QoS-aware averaging, and manifold-based aggregation as special cases. Experiments on a quantum-classical hybrid testbed demonstrate improved stability and higher accuracy under heterogeneous and noisy conditions.

Conclusion: A2G provides an effective aggregation framework for quantum federated learning that addresses both geometric and QoS challenges, offering improved performance in heterogeneous quantum-classical networks.

Abstract: Federated learning (FL) deployed over quantum enabled and heterogeneous classical networks faces significant performance degradation due to uneven client quality, stochastic teleportation fidelity, device instability, and geometric mismatch between local and global models. Classical aggregation rules assume euclidean topology and uniform communication reliability, limiting their suitability for emerging quantum federated systems. This paper introduces A2G (Adaptive Aggregation with Two Gains), a dual gain framework that jointly regulates geometric blending through a geometry gain and modulates client importance using a QoS gain derived from teleportation fidelity, latency, and instability. We develop the A2G update rule, establish convergence guarantees under smoothness and bounded variance assumptions, and show that A2G recovers FedAvg, QoS aware averaging, and manifold based aggregation as special cases. Experiments on a quantum classical hybrid testbed demonstrate improved stability and higher accuracy under heterogeneous and noisy conditions.

Method: VS-Graph introduces two key mechanisms: 1) Spike Diffusion for topology-driven node identification, and 2) Associative Message Passing for multi-hop neighborhood aggregation entirely within high-dimensional vector space. The method operates without gradient-based optimization or backpropagation.

Result: Achieves competitive accuracy with modern GNNs, outperforming prior HDC baseline by 4-5% on MUTAG and DD benchmarks. Matches or exceeds GNN baselines on several datasets while accelerating training by up to 450x. Maintains high accuracy even with hypervector dimensionality reduced to D=128.

Conclusion: VS-Graph successfully narrows the gap between HDC efficiency and message passing expressive power, demonstrating potential for ultra-efficient execution on edge and neuromorphic hardware while maintaining competitive graph classification performance.

Abstract: Graph classification is a fundamental task in domains ranging from molecular property prediction to materials design. While graph neural networks (GNNs) achieve strong performance by learning expressive representations via message passing, they incur high computational costs, limiting their scalability and deployment on resource-constrained devices. Hyperdimensional Computing (HDC), also known as Vector Symbolic Architectures (VSA), offers a lightweight, brain-inspired alternative, yet existing HDC-based graph methods typically struggle to match the predictive performance of GNNs. In this work, we propose VS-Graph, a vector-symbolic graph learning framework that narrows the gap between the efficiency of HDC and the expressive power of message passing. VS-Graph introduces a Spike Diffusion mechanism for topology-driven node identification and an Associative Message Passing scheme for multi-hop neighborhood aggregation entirely within the high-dimensional vector space. Without gradient-based optimization or backpropagation, our method achieves competitive accuracy with modern GNNs, outperforming the prior HDC baseline by 4-5% on standard benchmarks such as MUTAG and DD. It also matches or exceeds the performance of the GNN baselines on several datasets while accelerating the training by a factor of up to 450x. Furthermore, VS-Graph maintains high accuracy even with the hypervector dimensionality reduced to D=128, demonstrating robustness under aggressive dimension compression and paving the way for ultra-efficient execution on edge and neuromorphic hardware.

Method: MAGE-ID uses a diffusion-based generative framework that couples tabular flow features with their transformed images through a unified latent prior. It jointly trains Transformer and CNN-based variational encoders with an EDM style denoiser for balanced and coherent multimodal synthesis.

Result: Evaluations on CIC-IDS-2017 and NSL-KDD datasets demonstrate significant improvements in fidelity, diversity, and downstream detection performance over TabSyn and TabDDPM methods.

Conclusion: MAGE-ID effectively addresses multimodal IDS augmentation challenges, highlighting the effectiveness of coupling tabular and image modalities for improved intrusion detection through balanced synthetic data generation.

Abstract: Modern Intrusion Detection Systems (IDS) face severe challenges due to heterogeneous network traffic, evolving cyber threats, and pronounced data imbalance between benign and attack flows. While generative models have shown promise in data augmentation, existing approaches are limited to single modalities and fail to capture cross-domain dependencies. This paper introduces MAGE-ID (Multimodal Attack Generator for Intrusion Detection), a diffusion-based generative framework that couples tabular flow features with their transformed images through a unified latent prior. By jointly training Transformer and CNN-based variational encoders with an EDM style denoiser, MAGE-ID achieves balanced and coherent multimodal synthesis. Evaluations on CIC-IDS-2017 and NSL-KDD demonstrate significant improvements in fidelity, diversity, and downstream detection performance over TabSyn and TabDDPM, highlighting the effectiveness of MAGE-ID for multimodal IDS augmentation.

Method: Used 2x2x2 Rubik’s Cube as testbed; compared standard next-token prediction with two explicit world-modeling strategies: state-prediction pretraining and joint state-prediction + next-token objective; applied Group Relative Policy Optimization (GRPO) as post-training; evaluated representations with linear probes and causal interventions.

Result: Explicit world-modeling yields more linearly decodable and causally steerable state representations; improved state representations lead to higher GRPO gains, especially on harder cube states.

Conclusion: Sharpening state representations through explicit world-modeling can improve post-training effectiveness for sequence-planning tasks.

Abstract: In this work we study how explicit world-modeling objectives affect the internal representations and downstream capability of Transformers across different training stages. We use a controlled 2x2x2 Rubik’s Cube and ask: (1) how does explicitly pretraining a world model affect the model’s latent representations, and (2) how does world-model quality affect the model’s performance after reinforcement learning post-training? We compare standard next-token prediction to two explicit world-modeling strategies – (i) state-prediction pretraining and (ii) a joint state-prediction + next-token objective – and assess task performance after Group Relative Policy Optimization (GRPO) is applied as post-training. We evaluate the representation quality with linear probes and causal interventions. We find that explicit world-modeling yields more linearly decodable and causally steerable state representations. More importantly, we find that improved state representations lead to higher gains for GRPO, especially on harder cube states. Our results indicate that sharpening state representations can improve the effectiveness of post-training for sequence-planning tasks.

Method: Reparameterizes estimation matrix into factors that decouple shared row-support from individual entries, then applies gradient descent to standard least-squares objective. Uses implicit regularization from overparameterization with small balanced initialization to promote row-sparse structure.

Result: Theoretical proof shows optimization dynamics exhibit “momentum-like” effect causing true support rows to grow faster, guaranteeing convergence to idealized row-sparse solution. Empirical results show performance comparable to established methods without prior information or tuning.

Conclusion: Proposed tuning-free framework successfully overcomes limitations of traditional MMV methods by leveraging implicit regularization, providing practical solution for jointly sparse signal recovery without parameter tuning requirements.

Abstract: Recovering jointly sparse signals in the multiple measurement vectors (MMV) setting is a fundamental problem in machine learning, but traditional methods like multiple measurement vectors orthogonal matching pursuit (M-OMP) and multiple measurement vectors FOCal Underdetermined System Solver (M-FOCUSS) often require careful parameter tuning or prior knowledge of the sparsity of the signal and/or noise variance. We introduce a novel tuning-free framework that leverages Implicit Regularization (IR) from overparameterization to overcome this limitation. Our approach reparameterizes the estimation matrix into factors that decouple the shared row-support from individual vector entries. We show that the optimization dynamics inherently promote the desired row-sparse structure by applying gradient descent to a standard least-squares objective on these factors. We prove that with a sufficiently small and balanced initialization, the optimization dynamics exhibit a “momentum-like” effect, causing the norms of rows in the true support to grow significantly faster than others. This formally guarantees that the solution trajectory converges towards an idealized row-sparse solution. Additionally, empirical results demonstrate that our approach achieves performance comparable to established methods without requiring any prior information or tuning.

Method: Proposes “thick models of value” that structure how values and norms are represented, enabling systems to distinguish values from preferences, model social embedding of choices, and reason normatively. Demonstrates this approach in five application areas.

Result: Identifies limitations of current value representation methods (utility functions, preference orderings, unstructured text) and proposes a structured approach that enables better normative reasoning, value-preference distinction, and collective goods modeling.

Conclusion: Full-stack alignment using thick models of value is necessary for beneficial societal outcomes, allowing AI systems and institutions to align with human values without imposing specific visions of flourishing, demonstrated through five practical application areas.

Abstract: Beneficial societal outcomes cannot be guaranteed by aligning individual AI systems with the intentions of their operators or users. Even an AI system that is perfectly aligned to the intentions of its operating organization can lead to bad outcomes if the goals of that organization are misaligned with those of other institutions and individuals. For this reason, we need full-stack alignment, the concurrent alignment of AI systems and the institutions that shape them with what people value. This can be done without imposing a particular vision of individual or collective flourishing. We argue that current approaches for representing values, such as utility functions, preference orderings, or unstructured text, struggle to address these and other issues effectively. They struggle to distinguish values from other signals, to support principled normative reasoning, and to model collective goods. We propose thick models of value will be needed. These structure the way values and norms are represented, enabling systems to distinguish enduring values from fleeting preferences, to model the social embedding of individual choices, and to reason normatively, applying values in new domains. We demonstrate this approach in five areas: AI value stewardship, normatively competent agents, win-win negotiation systems, meaning-preserving economic mechanisms, and democratic regulatory institutions.

Method: Leverages equivalence between noise Gaussianity and residual independence in reverse regression, replacing Gaussianity tests with robust kernel-based independence tests.

Result: Maintains high consistency across diverse noise types and sample sizes while reducing tests per decision (TPD), enhancing efficiency and reliability.

Conclusion: GaussDetect-LiNGAM makes LiNGAM more accessible and reliable for real-world causal inference by eliminating fragile Gaussianity tests.

Abstract: We propose GaussDetect-LiNGAM, a novel approach for bivariate causal discovery that eliminates the need for explicit Gaussianity tests by leveraging a fundamental equivalence between noise Gaussianity and residual independence in the reverse regression. Under the standard LiNGAM assumptions of linearity, acyclicity, and exogeneity, we prove that the Gaussianity of the forward-model noise is equivalent to the independence between the regressor and residual in the reverse model. This theoretical insight allows us to replace fragile and sample-sensitive Gaussianity tests with robust kernel-based independence tests. Experimental results validate the equivalence and demonstrate that GaussDetect-LiNGAM maintains high consistency across diverse noise types and sample sizes, while reducing the number of tests per decision (TPD). Our method enhances both the efficiency and practical applicability of causal inference, making LiNGAM more accessible and reliable in real-world scenarios.

Method: Compare standard unlearning methods applied before vs. after grokking transition across vision (CNNs/ResNets on CIFAR, SVHN, ImageNet) and language (transformer on TOFU-style setup). Analyze features and curvature.

Result: Post-grokking models yield: (i) more efficient forgetting (fewer updates), (ii) less collateral damage (smaller performance drops), (iii) more stable updates across seeds. They learn more modular representations with reduced gradient alignment between forget/retain subsets.

Conclusion: When a model is trained (pre- vs. post-grokking) is an orthogonal lever to how unlearning is performed, providing a practical recipe to improve existing unlearning methods without altering algorithms.

Abstract: Grokking-delayed generalization that emerges well after a model has fit the training data-has been linked to robustness and representation quality. We ask whether this training regime also helps with machine unlearning, i.e., removing the influence of specified data without full retraining. We compare applying standard unlearning methods before versus after the grokking transition across vision (CNNs/ResNets on CIFAR, SVHN, and ImageNet) and language (a transformer on a TOFU-style setup). Starting from grokked checkpoints consistently yields (i) more efficient forgetting (fewer updates to reach a target forget level), (ii) less collateral damage (smaller drops on retained and test performance), and (iii) more stable updates across seeds, relative to early-stopped counterparts under identical unlearning algorithms. Analyses of features and curvature further suggest that post-grokking models learn more modular representations with reduced gradient alignment between forget and retain subsets, which facilitates selective forgetting. Our results highlight when a model is trained (pre- vs. post-grokking) as an orthogonal lever to how unlearning is performed, providing a practical recipe to improve existing unlearning methods without altering their algorithms.

Method: Proposes an end-to-end deep learning framework with BERT (Chinese-wwm-ext) for feature embedding, Financial Multi-Head Cross-Attention (FMHCA) for information exchange between recency and popularity modalities, transformer layer optimization, and multimodal factored bilinear pooling for sentiment classification.

Result: Achieves 83.5% accuracy on a comprehensive dataset covering 837 companies, significantly outperforming baselines including BERT+Transformer by 21 percent.

Conclusion: The framework demonstrates the importance of integrating distinct opinion modalities (recency vs. popularity) for financial sentiment analysis and shows potential to support more accurate financial decision-making and risk management.

Abstract: In recent years, financial sentiment analysis of public opinion has become increasingly important for market forecasting and risk assessment. However, existing methods often struggle to effectively integrate diverse opinion modalities and capture fine-grained interactions across them. This paper proposes an end-to-end deep learning framework that integrates two distinct modalities of financial opinions: recency modality (timely opinions) and popularity modality (trending opinions), through a novel cross-modal attention mechanism specifically designed for financial sentiment analysis. While both modalities consist of textual data, they represent fundamentally different information channels: recency-driven market updates versus popularity-driven collective sentiment. Our model first uses BERT (Chinese-wwm-ext) for feature embedding and then employs our proposed Financial Multi-Head Cross-Attention (FMHCA) structure to facilitate information exchange between these distinct opinion modalities. The processed features are optimized through a transformer layer and fused using multimodal factored bilinear pooling for classification into negative, neutral, and positive sentiment. Extensive experiments on a comprehensive dataset covering 837 companies demonstrate that our approach achieves an accuracy of 83.5%, significantly outperforming baselines including BERT+Transformer by 21 percent. These results highlight the potential of our framework to support more accurate financial decision-making and risk management.

Method: PhyDNN embeds the EIT forward model directly into the learning objective, jointly minimizing discrepancy between predicted/ground-truth conductivity maps and enforcing consistency with the forward PDE. Uses a differentiable forward-operator network that approximates the nonlinear EIT response for efficient backpropagation and physics-guided training.

Result: PhyDNN consistently outperforms NOSER, TV, and standard DNNs in reconstructing contact shape, location, and pressure distribution. Produces fewer artifacts, sharper boundaries, and higher metric scores in both simulations and real tactile experiments on a 16-electrode soft sensor.

Conclusion: PhyDNN reduces the black-box nature of deep networks and improves physical plausibility and generalization, demonstrating effectiveness for high-quality tomographic tactile sensing through physics-driven deep reconstruction.

Abstract: Electrical impedance tomography (EIT) provides an attractive solution for large-area tactile sensing due to its minimal wiring and shape flexibility, but its nonlinear inverse problem often leads to severe artifacts and inaccurate contact reconstruction. This work presents PhyDNN, a physics-driven deep reconstruction framework that embeds the EIT forward model directly into the learning objective. By jointly minimizing the discrepancy between predicted and ground-truth conductivity maps and enforcing consistency with the forward PDE, PhyDNN reduces the black-box nature of deep networks and improves both physical plausibility and generalization. To enable efficient backpropagation, we design a differentiable forward-operator network that accurately approximates the nonlinear EIT response, allowing fast physics-guided training. Extensive simulations and real tactile experiments on a 16-electrode soft sensor show that PhyDNN consistently outperforms NOSER, TV, and standard DNNs in reconstructing contact shape, location, and pressure distribution. PhyDNN yields fewer artifacts, sharper boundaries, and higher metric scores, demonstrating its effectiveness for high-quality tomographic tactile sensing.

Method: Developed Bayesian Event-Based Model for Subtypes (BEBMS) and compared with SuStaIn using synthetic data experiments with varying model misspecification levels. Applied both methods to real Alzheimer’s disease dataset.

Result: BEBMS substantially outperforms SuStaIn across ordering, staging, and subtype assignment tasks in synthetic experiments. In Alzheimer’s data, BEBMS results align better with scientific consensus than SuStaIn.

Conclusion: BEBMS provides more robust and accurate disease progression subtype analysis than SuStaIn, with better performance in both synthetic and real-world applications.

Abstract: Chronic diseases often progress differently across patients. Rather than randomly varying, there are typically a small number of subtypes for how a disease progresses across patients. To capture this structured heterogeneity, the Subtype and Stage Inference Event-Based Model (SuStaIn) estimates the number of subtypes, the order of disease progression for each subtype, and assigns each patient to a subtype from primarily cross-sectional data. It has been widely applied to uncover the subtypes of many diseases and inform our understanding of them. But how robust is its performance? In this paper, we develop a principled Bayesian subtype variant of the event-based model (BEBMS) and compare its performance to SuStaIn in a variety of synthetic data experiments with varied levels of model misspecification. BEBMS substantially outperforms SuStaIn across ordering, staging, and subtype assignment tasks. Further, we apply BEBMS and SuStaIn to a real-world Alzheimer’s data set. We find BEBMS has results that are more consistent with the scientific consensus of Alzheimer’s disease progression than SuStaIn.

Method: Developed SweetDeep, a compact neural network with fewer than 3,000 parameters. Trained on physiological and demographic data from 285 participants (diabetic and non-diabetic) collected using Samsung Galaxy Watch 7 devices in free-living conditions over six days. Each participant contributed ~20 recordings of 2-minute sensor data per day.

Result: Achieved 82.5% patient-level accuracy (82.1% macro-F1, 79.7% sensitivity, 84.6% specificity) under three-fold cross-validation. Expected calibration error of 5.5%. When allowing the model to abstain on <10% of low-confidence predictions, accuracy increased to 84.5% on remaining patients.

Conclusion: Combining engineered features with lightweight neural architectures enables accurate, rapid, and generalizable type 2 diabetes detection in real-world wearable settings, offering a scalable and cost-effective screening alternative to invasive biochemical assays.

Abstract: The global rise in type 2 diabetes underscores the need for scalable and cost-effective screening methods. Current diagnosis requires biochemical assays, which are invasive and costly. Advances in consumer wearables have enabled early explorations of machine learning-based disease detection, but prior studies were limited to controlled settings. We present SweetDeep, a compact neural network trained on physiological and demographic data from 285 (diabetic and non-diabetic) participants in the EU and MENA regions, collected using Samsung Galaxy Watch 7 devices in free-living conditions over six days. Each participant contributed multiple 2-minute sensor recordings per day, totaling approximately 20 recordings per individual. Despite comprising fewer than 3,000 parameters, SweetDeep achieves 82.5% patient-level accuracy (82.1% macro-F1, 79.7% sensitivity, 84.6% specificity) under three-fold cross-validation, with an expected calibration error of 5.5%. Allowing the model to abstain on less than 10% of low-confidence patient predictions yields an accuracy of 84.5% on the remaining patients. These findings demonstrate that combining engineered features with lightweight architectures can support accurate, rapid, and generalizable detection of type 2 diabetes in real-world wearable settings.

Method: Joint Progression Model (JPM) treats single-disease trajectories as partial rankings and builds a prior over joint progressions. Four variants studied: Pairwise, Bradley-Terry, Plackett-Luce, and Mallows models. Analyzed calibration, separation, and sharpness properties.

Result: All JPM variants are calibrated and achieve near-perfect separation. Sharpness varies by variant and is predictable from input features. JPM improves ordering accuracy by ~21% over SA-EBM baseline. Mallows variant and baseline show consistency with prior literature on AD+VaD progression in NACC data.

Conclusion: JPM effectively models joint disease progression for mixed pathologies, addressing limitations of single-disease EBMs. The framework provides principled approach to combine partial rankings from individual diseases, with Mallows variant showing particular promise for neurodegenerative disease applications.

Abstract: Event-based models (EBMs) infer disease progression from cross-sectional data, and standard EBMs assume a single underlying disease per individual. In contrast, mixed pathologies are common in neurodegeneration. We introduce the Joint Progression Model (JPM), a probabilistic framework that treats single-disease trajectories as partial rankings and builds a prior over joint progressions. We study several JPM variants (Pairwise, Bradley-Terry, Plackett-Luce, and Mallows) and analyze three properties: (i) calibration – whether lower model energy predicts smaller distance to the ground truth ordering; (ii) separation – the degree to which sampled rankings are distinguishable from random permutations; and (iii) sharpness – the stability of sampled aggregate rankings. All variants are calibrated, and all achieve near-perfect separation; sharpness varies by variant and is well-predicted by simple features of the input partial rankings (number and length of rankings, conflict, and overlap). In synthetic experiments, JPM improves ordering accuracy by roughly 21 percent over a strong EBM baseline (SA-EBM) that treats the joint disease as a single condition. Finally, using NACC, we find that the Mallows variant of JPM and the baseline model (SA-EBM) have results that are more consistent with prior literature on the possible disease progression of the mixed pathology of AD and VaD.

Method: MAGNETS learns a compact set of human-understandable concepts without annotations. Each concept corresponds to a learned, mask-based aggregation over selected input features, revealing which features drive predictions and when they matter. Predictions are formed through transparent, additive combinations of these learned concepts.

Result: The paper proposes MAGNETS as a solution that addresses limitations of existing approaches by providing inherent interpretability without requiring concept annotations, capturing feature interactions, handling complex temporal patterns, and scaling to high-dimensional multivariate data.

Conclusion: MAGNETS offers an inherently interpretable neural architecture for TSER that provides clear insight into model decisions through learned concepts and transparent additive structure, addressing key limitations of both black-box models and existing interpretable approaches.

Abstract: Time series extrinsic regression (TSER) refers to the task of predicting a continuous target variable from an input time series. It appears in many domains, including healthcare, finance, environmental monitoring, and engineering. In these settings, accurate predictions and trustworthy reasoning are both essential. Although state-of-the-art TSER models achieve strong predictive performance, they typically operate as black boxes, making it difficult to understand which temporal patterns drive their decisions. Post-hoc interpretability techniques, such as feature attribution, aim to to explain how the model arrives at its predictions, but often produce coarse, noisy, or unstable explanations. Recently, inherently interpretable approaches based on concepts, additive decompositions, or symbolic regression, have emerged as promising alternatives. However, these approaches remain limited: they require explicit supervision on the concepts themselves, often cannot capture interactions between time-series features, lack expressiveness for complex temporal patterns, and struggle to scale to high-dimensional multivariate data. To address these limitations, we propose MAGNETS (Mask-and-AGgregate NEtwork for Time Series), an inherently interpretable neural architecture for TSER. MAGNETS learns a compact set of human-understandable concepts without requiring any annotations. Each concept corresponds to a learned, mask-based aggregation over selected input features, explicitly revealing both which features drive predictions and when they matter in the sequence. Predictions are formed as combinations of these learned concepts through a transparent, additive structure, enabling clear insight into the model’s decision process.

Method: Proposes an adaptive sparse sensing framework that couples a variational autoencoder (VAE) prior with reinforcement learning (RL) to select measurements sequentially rather than using predetermined or random sampling patterns.

Result: Experiments show the proposed approach outperforms traditional compressed sensing (CS), optimal sensor placement (OSP), and generative model-based reconstruction from sparse measurements.

Conclusion: The adaptive framework combining VAE priors with RL for sequential measurement selection provides superior reconstruction quality compared to existing sparse sensing methods by enabling sample-specific adaptation.

Abstract: Compressed sensing enables sparse sampling but relies on generic bases and random measurements, limiting efficiency and reconstruction quality. Optimal sensor placement uses historcal data to design tailored sampling patterns, yet its fixed, linear bases cannot adapt to nonlinear or sample-specific variations. Generative model-based compressed sensing improves reconstruction using deep generative priors but still employs suboptimal random sampling. We propose an adaptive sparse sensing framework that couples a variational autoencoder prior with reinforcement learning to select measurements sequentially. Experiments show that this approach outperforms CS, OSP, and Generative model-based reconstruction from sparse measurements.

Method: Proposes DLC (Deployment of extra LoRA Components) using Low-Rank Adaptation to inject task-specific residuals into a base model’s deep layers. Includes a lightweight weighting unit to mitigate interference from non-target LoRA plugins, enabling plug-and-play enhancement.

Result: On ImageNet-100, DLC achieves 8% accuracy improvement with only 4% of ResNet-18 parameters. Also surpasses state-of-the-art methods under fixed memory budget constraints.

Conclusion: DLC provides an efficient plug-and-play solution for CIL that significantly improves accuracy with minimal parameter overhead, demonstrating exceptional efficiency and performance advantages.

Abstract: Existing class-incremental learning (CIL) approaches based on replay or knowledge distillation are often constrained by forgetting or the stability-plasticity dilemma. Some expansion-based approaches could achieve higher accuracy. However, they always require significant parameter increases. In this paper, we propose a plugin extension paradigm termed the Deployment of extra LoRA Components (DLC) for non-pre-trained CIL scenarios.We treat the feature extractor trained through replay or distillation as a base model with rich knowledge. For each task, we use Low-Rank Adaptation (LoRA) to inject task-specific residuals into the base model’s deep layers. During inference, representations with task-specific residuals are aggregated to produce classification predictions. To mitigate interference from non-target LoRA plugins, we introduce a lightweight weighting unit. This unit learns to assign importance scores to different LoRA-tuned representations. Like downloadable contents in software, our method serves as a plug-and-play enhancement that efficiently extends the base methods. Remarkably, on the large-scale ImageNet-100, with merely 4 % of the parameters of a standard ResNet-18, our DLC model achieves a significant 8 % improvement in accuracy, demonstrating exceptional efficiency. Moreover, it could surpass state-of-the-art methods under the fixed memory budget.

Method: Apply vision language model to generate Shipping Forecast text directly from video-encoded gridded weather data

Result: Early results show promising scalable technological opportunities for enhancing production efficiency

Conclusion: This approach can accelerate service innovation within weather enterprise and beyond

Abstract: Despite the promising capability of multimodal foundation models, their application to the generation of meteorological products and services remains nascent. To accelerate aspiration and adoption, we explore the novel use of a vision language model for writing the iconic Shipping Forecast text directly from video-encoded gridded weather data. These early results demonstrate promising scalable technological opportunities for enhancing production efficiency and service innovation within the weather enterprise and beyond.

Method: Two main contributions: 1) DiMRA attack that optimizes unlearned models on auxiliary data to reverse unlearning, and 2) DiMUM defense that memorizes alternative data/features instead of forgetting targeted elements.

Result: DiMRA successfully reverses state-of-the-art finetuning-based unlearning methods, while DiMUM preserves generative performance and enhances robustness against DiMRA attacks.

Conclusion: Current diffusion model unlearning methods are vulnerable to relearning attacks, requiring more robust solutions like DiMUM that uses memorization-based approach rather than forgetting.

Abstract: Diffusion models are renowned for their state-of-the-art performance in generating synthetic images. However, concerns related to safety, privacy, and copyright highlight the need for machine unlearning, which can make diffusion models forget specific training data and prevent the generation of sensitive or unwanted content. Current machine unlearning methods for diffusion models are primarily designed for conditional diffusion models and focus on unlearning specific data classes or features. Among these methods, finetuning-based machine unlearning methods are recognized for their efficiency and effectiveness, which update the parameters of pre-trained diffusion models by minimizing carefully designed loss functions. However, in this paper, we propose a novel attack named Diffusion Model Relearning Attack (DiMRA), which can reverse the finetuning-based machine unlearning methods, posing a significant vulnerability of this kind of technique. Without prior knowledge of the unlearning elements, DiMRA optimizes the unlearned diffusion model on an auxiliary dataset to reverse the unlearning, enabling the model to regenerate previously unlearned elements. To mitigate this vulnerability, we propose a novel machine unlearning method for diffusion models, termed as Diffusion Model Unlearning by Memorization (DiMUM). Unlike traditional methods that focus on forgetting, DiMUM memorizes alternative data or features to replace targeted unlearning data or features in order to prevent generating such elements. In our experiments, we demonstrate the effectiveness of DiMRA in reversing state-of-the-art finetuning-based machine unlearning methods for diffusion models, highlighting the need for more robust solutions. We extensively evaluate DiMUM, demonstrating its superior ability to preserve the generative performance of diffusion models while enhancing robustness against DiMRA.

Method: Develops closed-form expressions for inner product GW alignment between uncentered Gaussians on Hilbert spaces (with optimization over unitary operators), fully closed-form solutions for centered Gaussians, analytic solutions for IGW barycenters, and reduction of Gaussian multimarginal OT with quadratic costs to tractable optimization with rank-deficiency constraints.

Result: Provides comprehensive closed-form solutions for Gaussian OT and IGW alignment, including: 1) closed-form expression for uncentered Gaussians up to unitary optimization with analytic bounds, 2) fully closed-form solution for centered Gaussians, 3) analytic solution for IGW barycenters, 4) reduction of multimarginal OT to tractable optimization with efficient algorithm.

Conclusion: The work closes several gaps in Gaussian OT and GW alignment literature, enabling broader applications in data science and machine learning. Demonstrated utility in knowledge distillation and heterogeneous clustering on synthetic and real-world datasets.

Abstract: Optimal transport (OT) and Gromov-Wasserstein (GW) alignment provide interpretable geometric frameworks for comparing, transforming, and aggregating heterogeneous datasets – tasks ubiquitous in data science and machine learning. Because these frameworks are computationally expensive, large-scale applications often rely on closed-form solutions for Gaussian distributions under quadratic cost. This work provides a comprehensive treatment of Gaussian, quadratic cost OT and inner product GW (IGW) alignment, closing several gaps in the literature to broaden applicability. First, we treat the open problem of IGW alignment between uncentered Gaussians on separable Hilbert spaces by giving a closed-form expression up to a quadratic optimization over unitary operators, for which we derive tight analytic upper and lower bounds. If at least one Gaussian measure is centered, the solution reduces to a fully closed-form expression, which we further extend to an analytic solution for the IGW barycenter between centered Gaussians. We also present a reduction of Gaussian multimarginal OT with pairwise quadratic costs to a tractable optimization problem and provide an efficient algorithm to solve it using a rank-deficiency constraint. To demonstrate utility, we apply our results to knowledge distillation and heterogeneous clustering on synthetic and real-world datasets.

Method: DSAS decouples “when to steer” from “how to steer” by computing context-dependent scaling factors that selectively adjust the strength of any steering method across layers and inputs. It can be jointly optimized end-to-end with steering functions.

Result: DSAS consistently improves the Pareto front with respect to steering alone, achieving better trade-off between toxicity mitigation and utility preservation. It also works with text-to-image diffusion models for concept modulation, with minimal computational overhead.

Conclusion: DSAS provides a method-agnostic framework for adaptive activation steering that improves performance trade-offs, enhances interpretability by pinpointing which tokens need steering, and maintains computational efficiency.

Abstract: Activation steering has emerged as a powerful method for guiding the behavior of generative models towards desired outcomes such as toxicity mitigation. However, most existing methods apply interventions uniformly across all inputs, degrading model performance when steering is unnecessary. We introduce Dynamically Scaled Activation Steering (DSAS), a method-agnostic steering framework that decouples when to steer from how to steer. DSAS adaptively modulates the strength of existing steering transformations across layers and inputs, intervening strongly only when undesired behavior is detected. At generation time, DSAS computes context-dependent scaling factors that selectively adjust the strength of any steering method. We also show how DSAS can be jointly optimized end-to-end together with the steering function. When combined with existing steering methods, DSAS consistently improves the Pareto front with respect to steering alone, achieving a better trade-off between toxicity mitigation and utility preservation. We further demonstrate DSAS’s generality by applying it to a text-to-image diffusion model, showing how adaptive steering allows the modulation of specific concepts. Finally, DSAS introduces minimal computational overhead while improving interpretability, pinpointing which tokens require steering and by how much.

Method: Combines federated learning (M3fed model) with bandwidth-constrained trajectory compression (BWC-DR-A algorithm) to prioritize anomalous data transmission over low-bandwidth connections.

Result: Preliminary results show effectiveness in improving AIS coverage and situational awareness using historical data.

Conclusion: VesselEdge successfully transforms vessels into mobile sensors for enhanced maritime situational awareness through federated learning and efficient data compression.

Abstract: This paper presents the VesselEdge system, which leverages federated learning and bandwidth-constrained trajectory compression to enhance maritime situational awareness by extending AIS coverage. VesselEdge transforms vessels into mobile sensors, enabling real-time anomaly detection and efficient data transmission over low-bandwidth connections. The system integrates the M3fed model for federated learning and the BWC-DR-A algorithm for trajectory compression, prioritizing anomalous data. Preliminary results demonstrate the effectiveness of VesselEdge in improving AIS coverage and situational awareness using historical data.

Method: Reformulates wind forecasting as observation-informed correction of GFS output using transformer architecture with: (1) masking and set-based attention for irregular time-varying observations, (2) cross-attention on recent observation-forecast pairs, (3) cyclical time embeddings and coordinate-aware location representations for single-pass inference at arbitrary coordinates.

Result: Model reduces GFS 10-meter wind RMSE at all lead times up to 48 hours: 45% improvement at 1-hour lead time, 13% improvement at 48-hour lead time. Most persistent improvements along coastlines and shipping routes where observations are abundant. Architecture handles heterogeneous platforms (ships, buoys, etc.) and produces both site-specific predictions and basin-scale gridded products.

Conclusion: Demonstrates practical, low-latency post-processing approach that complements NWP by learning to correct systematic forecast errors using available observations, with transformer architecture enabling flexible handling of sparse, heterogeneous ocean data.

Abstract: Accurate marine wind forecasts are essential for safe navigation, ship routing, and energy operations, yet they remain challenging because observations over the ocean are sparse, heterogeneous, and temporally variable. We reformulate wind forecasting as observation-informed correction of a global numerical weather prediction (NWP) model. Rather than forecasting winds directly, we learn local correction patterns by assimilating the latest in-situ observations to adjust the Global Forecast System (GFS) output. We propose a transformer-based deep learning architecture that (i) handles irregular and time-varying observation sets through masking and set-based attention mechanisms, (ii) conditions predictions on recent observation-forecast pairs via cross-attention, and (iii) employs cyclical time embeddings and coordinate-aware location representations to enable single-pass inference at arbitrary spatial coordinates. We evaluate our model over the Atlantic Ocean using observations from the International Comprehensive Ocean-Atmosphere Data Set (ICOADS) as reference. The model reduces GFS 10-meter wind RMSE at all lead times up to 48 hours, achieving 45% improvement at 1-hour lead time and 13% improvement at 48-hour lead time. Spatial analyses reveal the most persistent improvements along coastlines and shipping routes, where observations are most abundant. The tokenized architecture naturally accommodates heterogeneous observing platforms (ships, buoys, tide gauges, and coastal stations) and produces both site-specific predictions and basin-scale gridded products in a single forward pass. These results demonstrate a practical, low-latency post-processing approach that complements NWP by learning to correct systematic forecast errors.

Method: The QTGNN framework integrates: 1) quantum data embedding with entanglement enhancement, 2) variational quantum graph convolutions with non-linear dynamics, 3) extraction of higher-order topological invariants, 4) hybrid quantum-classical anomaly learning with adaptive optimization, and 5) interpretable decision-making via topological attribution. The method is optimized for NISQ devices with circuit simplifications and graph sampling.

Result: The framework demonstrates effectiveness through simulations on financial datasets (PaySim and Elliptic), benchmarking against classical and quantum baselines using metrics like ROC-AUC, precision, and false positive rate. An ablation study evaluates contributions of quantum embeddings, topological features, non-linear channels, and hybrid learning.

Conclusion: QTGNN provides a theoretically sound, interpretable, and practical solution for financial fraud detection that bridges quantum machine learning, graph theory, and topological analysis, with rigorous convergence guarantees for stable training on NISQ devices and robust fraud detection through topological signature stability.

Abstract: We propose a novel QTGNN framework for detecting fraudulent transactions in large-scale financial networks. By integrating quantum embedding, variational graph convolutions, and topological data analysis, QTGNN captures complex transaction dynamics and structural anomalies indicative of fraud. The methodology includes quantum data embedding with entanglement enhancement, variational quantum graph convolutions with non-linear dynamics, extraction of higher-order topological invariants, hybrid quantum-classical anomaly learning with adaptive optimization, and interpretable decision-making via topological attribution. Rigorous convergence guarantees ensure stable training on noisy intermediate-scale quantum (NISQ) devices, while stability of topological signatures provides robust fraud detection. Optimized for NISQ hardware with circuit simplifications and graph sampling, the framework scales to large transaction networks. Simulations on financial datasets, such as PaySim and Elliptic, benchmark QTGNN against classical and quantum baselines, using metrics like ROC-AUC, precision, and false positive rate. An ablation study evaluates the contributions of quantum embeddings, topological features, non-linear channels, and hybrid learning. QTGNN offers a theoretically sound, interpretable, and practical solution for financial fraud detection, bridging quantum machine learning, graph theory, and topological analysis.

Method: CoGraM is a multi-stage, context-sensitive, loss-based iterative optimization method that operates across layers, neurons, and weight levels. It aligns decisions with loss differences and thresholds, and prevents harmful updates through rollback mechanisms.

Result: CoGraM addresses weaknesses of existing methods like Fisher merging and can significantly improve the performance of merged networks.

Conclusion: CoGraM provides a more effective and stable approach to neural network merging without retraining, making it valuable for federated and distributed learning applications.

Abstract: Merging neural networks without retraining is central to federated and distributed learning. Common methods such as weight averaging or Fisher merging often lose accuracy and are unstable across seeds. CoGraM (Contextual Granular Merging) is a multi-stage, context-sensitive, loss-based, and iterative optimization method across layers, neurons, and weight levels that aligns decisions with loss differences and thresholds and prevents harmful updates through rollback. CoGraM is an optimization method that addresses the weaknesses of methods such as Fisher and can significantly improve the merged network.

Method: Three-step approach: 1) Retrain neural network on reasonable subsets of training data and record weight anomalies; 2) Choose predictors and derive regression between predictors and weight anomalies; 3) Extrapolate weights to application data to adapt the neural network.

Result: Method demonstrated in three climate science use cases showing successful temporal, spatial, and cross-domain extrapolations of neural networks.

Conclusion: Proposed method effectively enhances neural network performance for out-of-distribution scenarios by enabling weight extrapolation based on systematic analysis of training data subsets.

Abstract: This study proposes a method to enhance neural network performance when training data and application data are not very similar, e.g., out of distribution problems, as well as pattern and regime shifts. The method consists of three main steps: 1) Retrain the neural network towards reasonable subsets of the training data set and note down the resulting weight anomalies. 2) Choose reasonable predictors and derive a regression between the predictors and the weight anomalies. 3) Extrapolate the weights, and thereby the neural network, to the application data. We show and discuss this method in three use cases from the climate sciences, which include successful temporal, spatial and cross-domain extrapolations of neural networks.

Method: Uses cyclical temporal encoding (sine-cosine) for calendar attributes, correlation analysis to evaluate predictive relevance, and an ensemble model combining LSTM, CNN, and MLP meta-learners specialized for each forecast horizon.

Result: The hybrid model achieves lower RMSE and MAE across all seven forecast horizons compared to individual architectures and prior methods, with consistent improvements demonstrated through ablation studies.

Conclusion: Combining cyclical temporal representations with complementary deep learning structures provides significant benefits for short-term energy forecasting, representing the first unified framework to jointly evaluate temporal encodings, calendar features, and hybrid ensemble architectures.

Abstract: Accurate electricity consumption forecasting is essential for demand management and smart grid operations. This paper introduces a unified deep learning framework that integrates cyclical temporal encoding with hybrid LSTM-CNN architectures to enhance multistep energy forecasting. We systematically transform calendar-based attributes using sine cosine encodings to preserve periodic structure and evaluate their predictive relevance through correlation analysis. To exploit both long-term seasonal effects and short-term local patterns, we employ an ensemble model composed of an LSTM, a CNN, and a meta-learner of MLP regressors specialized for each forecast horizon. Using a one year national consumption dataset, we conduct an extensive experimental study including ablation analyses with and without cyclical encodings and calendar features and comparisons with established baselines from the literature. Results demonstrate consistent improvements across all seven forecast horizons, with our hybrid model achieving lower RMSE and MAE than individual architectures and prior methods. These findings confirm the benefit of combining cyclical temporal representations with complementary deep learning structures. To our knowledge, this is the first work to jointly evaluate temporal encodings, calendar-based features, and hybrid ensemble architectures within a unified short-term energy forecasting framework.

Method: Feature-aware temporal modulation mechanism that conditions feature representations on temporal context, modulating statistical properties (scale, skewness) to align feature semantics across time. Uses feature transformation strategies to mitigate representation discrepancies across temporal stages.

Result: Benchmark evaluations validate the method’s effectiveness in handling temporal shifts in tabular data. The approach achieves lightweight yet powerful adaptation that balances generalizability and adaptability.

Conclusion: The proposed feature-aware temporal modulation mechanism successfully addresses temporal distribution shifts in tabular data by aligning evolving feature semantics across time, providing a solution to the robustness-adaptability dilemma.

Abstract: While tabular machine learning has achieved remarkable success, temporal distribution shifts pose significant challenges in real-world deployment, as the relationships between features and labels continuously evolve. Static models assume fixed mappings to ensure generalization, whereas adaptive models may overfit to transient patterns, creating a dilemma between robustness and adaptability. In this paper, we analyze key factors essential for constructing an effective dynamic mapping for temporal tabular data. We discover that evolving feature semantics-particularly objective and subjective meanings-introduce concept drift over time. Crucially, we identify that feature transformation strategies are able to mitigate discrepancies in feature representations across temporal stages. Motivated by these insights, we propose a feature-aware temporal modulation mechanism that conditions feature representations on temporal context, modulating statistical properties such as scale and skewness. By aligning feature semantics across time, our approach achieves a lightweight yet powerful adaptation, effectively balancing generalizability and adaptability. Benchmark evaluations validate the effectiveness of our method in handling temporal shifts in tabular data.

Method: Developed a physics-constrained Hamiltonian learning algorithm combining “structural irreversibility detection” and “energy landscape reconstruction.” The approach extracts low-dimensional state representations, identifies quasi-Hamiltonian structures through physics-constrained optimization, and quantifies structural changes via energy landscape comparison.

Result: Analysis of London’s extreme rainfall in 2021 showed that while surface indicators were fully recovered, the algorithm detected 64.8% structural damage missed by traditional monitoring methods.

Conclusion: The framework provides tools for proactive structural risk assessment, enabling infrastructure investments based on true system health rather than misleading surface metrics, addressing critical gaps in urban transportation resilience evaluation.

Abstract: Urban transportation systems face increasing resilience challenges from extreme weather events, but current assessment methods rely on surface-level recovery indicators that miss hidden structural damage. Existing approaches cannot distinguish between true recovery and “false recovery,” where traffic metrics normalize, but the underlying system dynamics permanently degrade. To address this, a new physics-constrained Hamiltonian learning algorithm combining “structural irreversibility detection” and “energy landscape reconstruction” has been developed. Our approach extracts low-dimensional state representations, identifies quasi-Hamiltonian structures through physics-constrained optimization, and quantifies structural changes via energy landscape comparison. Analysis of London’s extreme rainfall in 2021 demonstrates that while surface indicators were fully recovered, our algorithm detected 64.8% structural damage missed by traditional monitoring. Our framework provides tools for proactive structural risk assessment, enabling infrastructure investments based on true system health rather than misleading surface metrics.

Method: Analyzed representations from nearly sixty scientific models spanning string-, graph-, 3D atomistic, and protein-based modalities across various chemical systems. Examined alignment patterns between models trained on different datasets and studied representation convergence as models improve.

Result: Models show high representational alignment across modalities, with convergence toward common representations as performance improves. Two distinct regimes observed: 1) On training-like inputs, high-performing models align closely while weak models diverge; 2) On novel structures, most models collapse to low-information representations, indicating current limitations.

Conclusion: Representational alignment serves as a quantitative benchmark for foundation-level generality in scientific models. Current models learn aligned representations but remain constrained by training data and inductive biases, not yet achieving truly universal structure encoding.

Abstract: Machine learning models of vastly different modalities and architectures are being trained to predict the behavior of molecules, materials, and proteins. However, it remains unclear whether they learn similar internal representations of matter. Understanding their latent structure is essential for building scientific foundation models that generalize reliably beyond their training domains. Although representational convergence has been observed in language and vision, its counterpart in the sciences has not been systematically explored. Here, we show that representations learned by nearly sixty scientific models, spanning string-, graph-, 3D atomistic, and protein-based modalities, are highly aligned across a wide range of chemical systems. Models trained on different datasets have highly similar representations of small molecules, and machine learning interatomic potentials converge in representation space as they improve in performance, suggesting that foundation models learn a common underlying representation of physical reality. We then show two distinct regimes of scientific models: on inputs similar to those seen during training, high-performing models align closely and weak models diverge into local sub-optima in representation space; on vastly different structures from those seen during training, nearly all models collapse onto a low-information representation, indicating that today’s models remain limited by training data and inductive bias and do not yet encode truly universal structure. Our findings establish representational alignment as a quantitative benchmark for foundation-level generality in scientific models. More broadly, our work can track the emergence of universal representations of matter as models scale, and for selecting and distilling models whose learned representations transfer best across modalities, domains of matter, and scientific tasks.

Method: A conditional trajectory encoder with bidirectional LSTM that processes visibility ratio and curvature features conditioned on learnable origin embeddings. The framework decomposes urban navigation into shared cognitive patterns and origin-specific spatial narratives using contrastive learning.

Result: Results from six synthetic cities and real-world validation on Beijing’s Xicheng District demonstrate that urban morphology creates systematic cognitive inequalities. The method provides quantitative measurement of cognitive asymmetries across starting locations.

Conclusion: The framework provides urban planners with quantitative tools for assessing experiential equity, offers architects insights into layout decisions’ cognitive impacts, and enables origin-aware analytics for navigation systems.

Abstract: Urban analytics increasingly relies on AI-driven trajectory analysis, yet current approaches suffer from methodological fragmentation: trajectory learning captures movement patterns but ignores spatial context, while spatial embedding methods encode street networks but miss temporal dynamics. Three gaps persist: (1) lack of joint training that integrates spatial and temporal representations, (2) origin-agnostic treatment that ignores directional asymmetries in navigation ($A \to B \ne B \to A$), and (3) over-reliance on auxiliary data (POIs, imagery) rather than fundamental geometric properties of urban space. We introduce a conditional trajectory encoder that jointly learns spatial and movement representations while preserving origin-dependent asymmetries using geometric features. This framework decomposes urban navigation into shared cognitive patterns and origin-specific spatial narratives, enabling quantitative measurement of cognitive asymmetries across starting locations. Our bidirectional LSTM processes visibility ratio and curvature features conditioned on learnable origin embeddings, decomposing representations into shared urban patterns and origin-specific signatures through contrastive learning. Results from six synthetic cities and real-world validation on Beijing’s Xicheng District demonstrate that urban morphology creates systematic cognitive inequalities. This provides urban planners quantitative tools for assessing experiential equity, offers architects insights into layout decisions’ cognitive impacts, and enables origin-aware analytics for navigation systems.

Method: Systematically transforms iterative optimization algorithms into structured, trainable machine learning architectures through four representative design paradigms. Uses distinctive training schemes that arise from the iterative nature of unfolded optimizers.

Result: Provides a unified perspective on methodologies for converting optimization solvers into ML models, with recent theoretical advances establishing convergence and generalization guarantees. Comparative studies illustrate trade-offs in complexity, interpretability, and robustness.

Conclusion: Deep unfolding emerges as a compelling framework that bridges classical optimization and machine learning, offering structured, transparent, and efficient optimization-driven inference while maintaining theoretical foundations and data-driven learning capabilities.

Abstract: Optimization methods play a central role in signal processing, serving as the mathematical foundation for inference, estimation, and control. While classical iterative optimization algorithms provide interpretability and theoretical guarantees, they often rely on surrogate objectives, require careful hyperparameter tuning, and exhibit substantial computational latency. Conversely, machine learning (ML ) offers powerful data-driven modeling capabilities but lacks the structure, transparency, and efficiency needed for optimization-driven inference. Deep unfolding has recently emerged as a compelling framework that bridges these two paradigms by systematically transforming iterative optimization algorithms into structured, trainable ML architectures. This article provides a tutorial-style overview of deep unfolding, presenting a unified perspective of methodologies for converting optimization solvers into ML models and highlighting their conceptual, theoretical, and practical implications. We review the foundations of optimization for inference and for learning, introduce four representative design paradigms for deep unfolding, and discuss the distinctive training schemes that arise from their iterative nature. Furthermore, we survey recent theoretical advances that establish convergence and generalization guarantees for unfolded optimizers, and provide comparative qualitative and empirical studies illustrating their relative trade-offs in complexity, interpretability, and robustness.

Method: Machine learning-based approach to translate digital movement traces into likelihood ratios for different physical activities, evaluated on NFI_FARED dataset containing iPhone sensor data labeled with 19 activities.

Result: The approach produced useful LR systems for 167 out of 171 possible activity pairings, and was extended to analyze multiple activities simultaneously and create activity timelines for forensic investigations.

Conclusion: The method enables forensic investigators to analyze physical activities from digital traces, with public dataset and code released to encourage further research in this domain.

Abstract: Smartphones and smartwatches are ever-present in daily life, and provide a rich source of information on their users’ behaviour. In particular, digital traces derived from the phone’s embedded movement sensors present an opportunity for a forensic investigator to gain insight into a person’s physical activities. In this work, we present a machine learning-based approach to translate digital traces into likelihood ratios (LRs) for different types of physical activities. Evaluating on a new dataset, NFI_FARED, which contains digital traces from four different types of iPhones labelled with 19 activities, it was found that our approach could produce useful LR systems to distinguish 167 out of a possible 171 activity pairings. The same approach was extended to analyse likelihoods for multiple activities (or groups of activities) simultaneously and create activity timelines to aid in both the early and latter stages of forensic investigations. The dataset and all code required to replicate the results have also been made public to encourage further research on this topic.

Method: DVPO combines conditional risk theory with distributional value modeling: learns token-level value distributions for fine-grained supervision, and applies asymmetric risk regularization to shape distribution tails - contracts lower tail to dampen noisy negative deviations while expanding upper tail to preserve exploratory diversity.

Result: Across extensive experiments in multi-turn dialogue, math reasoning, and scientific QA, DVPO consistently outperforms PPO, GRPO, and robust Bellman-based PPO under noisy supervision.

Conclusion: DVPO shows strong potential for LLM post-training in real-world settings by better balancing robustness and generalization through distributional value modeling with risk-aware policy optimization.

Abstract: Reinforcement learning (RL) has shown strong performance in LLM post-training, but real-world deployment often involves noisy or incomplete supervision. In such settings, complex and unreliable supervision signals can destabilize training and harm generalization. While existing approaches such as worst-case optimization (e.g., RFQI, CQL) and mean-based methods (e.g., PPO, GRPO) can improve stability, they often overlook generalization and may produce overly conservative policies, leading to uneven performance across diverse real scenarios. To this end, we introduce DVPO (Distributional Value Modeling with Risk-aware Policy Optimization), a new RL framework that combines conditional risk theory with distributional value modeling to better balance robustness and generalization. DVPO learns token-level value distributions to provide fine-grained supervision, and applies an asymmetric risk regularization to shape the distribution tails: it contracts the lower tail to dampen noisy negative deviations, while expanding the upper tail to preserve exploratory diversity. Across extensive experiments and analysis in multi-turn dialogue, math reasoning, and scientific QA, DVPO consistently outperforms PPO, GRPO, and robust Bellman-based PPO under noisy supervision, showing its potential for LLM post-training in the real-world.

Method: Two-phase modeling using process mining: 1) Collect historical disease data to create a reference process model, 2) Compare new data against reference model using conformance checking diagnostics to verify compliance. Based on results, expand knowledge base with more specific models for new variants or disease combinations.

Result: Method demonstrated using Synthea benchmark dataset for SARS-CoV-2 infections with COVID-19 complications. Achieved 95.62% AUC (area under curve) while maintaining arc-degree simplicity of 67.11%, enabling expansion of clinical pathway knowledge base with sufficient precision.

Conclusion: The proposed process mining approach successfully enables automated expansion of clinical pathway knowledge bases, capturing treatment variations while maintaining model quality and simplicity, addressing limitations of manual pathway modeling.

Abstract: Clinical pathways are specialized healthcare plans that model patient treatment procedures. They are developed to provide criteria-based progression and standardize patient treatment, thereby improving care, reducing resource use, and accelerating patient recovery. However, manual modeling of these pathways based on clinical guidelines and domain expertise is difficult and may not reflect the actual best practices for different variations or combinations of diseases. We propose a two-phase modeling method using process mining, which extends the knowledge base of clinical pathways by leveraging conformance checking diagnostics. In the first phase, historical data of a given disease is collected to capture treatment in the form of a process model. In the second phase, new data is compared against the reference model to verify conformance. Based on the conformance checking results, the knowledge base can be expanded with more specific models tailored to new variants or disease combinations. We demonstrate our approach using Synthea, a benchmark dataset simulating patient treatments for SARS-CoV-2 infections with varying COVID-19 complications. The results show that our method enables expanding the knowledge base of clinical pathways with sufficient precision, peaking to 95.62% AUC while maintaining an arc-degree simplicity of 67.11%.

Method: 1) Developed EfficientECG based on EfficientNet for accurate, lightweight ECG classification of high-frequency long-sequence data with various lead types. 2) Proposed cross-attention-based feature fusion model to analyze multi-lead ECG data with additional features like gender and age.

Result: Evaluations on representative ECG datasets show the model outperforms state-of-the-art works in terms of high precision, multi-feature fusion capability, and lightweight design.

Conclusion: The proposed deep learning approaches effectively manage and analyze ECG data, reducing medical worker burden through accurate, fast diagnostic models that automatically extract features and handle multi-feature fusion.

Abstract: Electrocardiogram is a useful diagnostic signal that can detect cardiac abnormalities by measuring the electrical activity generated by the heart. Due to its rapid, non-invasive, and richly informative characteristics, ECG has many emerging applications. In this paper, we study novel deep learning technologies to effectively manage and analyse ECG data, with the aim of building a diagnostic model, accurately and quickly, that can substantially reduce the burden on medical workers. Unlike the existing ECG models that exhibit a high misdiagnosis rate, our deep learning approaches can automatically extract the features of ECG data through end-to-end training. Specifically, we first devise EfficientECG, an accurate and lightweight classification model for ECG analysis based on the existing EfficientNet model, which can effectively handle high-frequency long-sequence ECG data with various leading types. On top of that, we next propose a cross-attention-based feature fusion model of EfficientECG for analysing multi-lead ECG data with multiple features (e.g., gender and age). Our evaluations on representative ECG datasets validate the superiority of our model against state-of-the-art works in terms of high precision, multi-feature fusion, and lightweights.

Method: Proposes an acceleration method based on replacing costly loss function evaluations with an efficient surrogate. The surrogate uses unbiased estimators to reduce risk of spurious local optima, provides local confidence information, and works in a black-box setting to compensate for model simplifications and account for recourse actions.

Result: The method reduces costly inner solver calls while achieving solution quality comparable to other state-of-the-art techniques.

Conclusion: The proposed acceleration method effectively addresses scalability issues in Decision Focused Learning by using efficient unbiased surrogate loss functions, enabling practical application of decision-focused approaches in complex optimization problems.

Abstract: Decision support systems often rely on solving complex optimization problems that may require to estimate uncertain parameters beforehand. Recent studies have shown how using traditionally trained estimators for this task can lead to suboptimal solutions. Using the actual decision cost as a loss function (called Decision Focused Learning) can address this issue, but with a severe loss of scalability at training time. To address this issue, we propose an acceleration method based on replacing costly loss function evaluations with an efficient surrogate. Unlike previously defined surrogates, our approach relies on unbiased estimators reducing the risk of spurious local optima and can provide information on its local confidence allowing one to switch to a fallback method when needed. Furthermore, the surrogate is designed for a black-box setting, which enables compensating for simplifications in the optimization model and account- ing for recourse actions during cost computation. In our results, the method reduces costly inner solver calls, with a solution quality comparable to other state-of-the-art techniques.

Method: Systematic analysis of DDQN and PPO for controlling population size parameter of (1+(λ,λ))-GA on OneMax instances. Introduces adaptive reward shifting mechanism to address under-exploration, and explores undiscounted learning to handle planning horizon coverage.

Result: DDQN with adaptive reward shifting achieves performance comparable to theoretically derived policies with vastly improved sample efficiency, outperforming prior DAC approaches by several orders of magnitude. PPO faces fundamental variance issues despite hyperparameter optimization.

Conclusion: Targeted solutions effectively address under-exploration and planning horizon coverage in DAC. DDQN with adaptive reward shifting provides a robust approach that eliminates instance-specific hyperparameter tuning and ensures consistent effectiveness across problem scales.

Abstract: Dynamic Algorithm Configuration (DAC) studies the efficient identification of control policies for parameterized optimization algorithms. Numerous studies have leveraged the robustness of decision-making in Reinforcement Learning (RL) to address the optimization challenges in algorithm configuration. However, applying RL to DAC is challenging and often requires extensive domain expertise. We conduct a comprehensive study of deep-RL algorithms in DAC through a systematic analysis of controlling the population size parameter of the (1+($λ$,$λ$))-GA on OneMax instances. Our investigation of DDQN and PPO reveals two fundamental challenges that limit their effectiveness in DAC: scalability degradation and learning instability. We trace these issues to two primary causes: under-exploration and planning horizon coverage, each of which can be effectively addressed through targeted solutions. To address under-exploration, we introduce an adaptive reward shifting mechanism that leverages reward distribution statistics to enhance DDQN agent exploration, eliminating the need for instance-specific hyperparameter tuning and ensuring consistent effectiveness across different problem scales. In dealing with the planning horizon coverage problem, we demonstrate that undiscounted learning effectively resolves it in DDQN, while PPO faces fundamental variance issues that necessitate alternative algorithmic designs. We further analyze the hyperparameter dependencies of PPO, showing that while hyperparameter optimization enhances learning stability, it consistently falls short in identifying effective policies across various configurations. Finally, we demonstrate that DDQN equipped with our adaptive reward shifting strategy achieves performance comparable to theoretically derived policies with vastly improved sample efficiency, outperforming prior DAC approaches by several orders of magnitude.

Method: Used in-situ sensing-based prediction of geometric quality in smart machining to compare energy consumption, accuracy, and speed of common AI models. Introduced HyperDimensional Computing (HDC) as an alternative to conventional AI models.

Result: HDC achieved accuracy comparable to conventional AI models while drastically reducing energy consumption: 200× reduction for training, 175-1000× reduction for inference. Also reduced training times by 200× and inference times by 300-600×.

Conclusion: HDC demonstrates significant potential for energy-efficient smart manufacturing by maintaining accuracy while dramatically reducing both energy consumption and computational time compared to conventional AI models.

Abstract: Smart manufacturing can significantly improve efficiency and reduce energy consumption, yet the energy demands of AI models may offset these gains. This study utilizes in-situ sensing-based prediction of geometric quality in smart machining to compare the energy consumption, accuracy, and speed of common AI models. HyperDimensional Computing (HDC) is introduced as an alternative, achieving accuracy comparable to conventional models while drastically reducing energy consumption, 200$\times$ for training and 175 to 1000$\times$ for inference. Furthermore, HDC reduces training times by 200$\times$ and inference times by 300 to 600$\times$, showcasing its potential for energy-efficient smart manufacturing.

Method: Uses a simple statistical test based on average token log probabilities, requesting only a single token of output. Logprobs, while usually non-deterministic, can serve as basis for cost-effective continuous monitoring.

Result: The approach can detect changes as small as one step of fine-tuning, making it more sensitive than existing methods while being 1,000x cheaper. Introduces TinyChange benchmark to measure sensitivity of audit methods for small, realistic model changes.

Conclusion: Logprobs provide a practical, cost-effective solution for continuous monitoring of LLM API consistency, addressing the gap between expensive existing audit methods and the need for regular model consistency verification.

Abstract: When using an LLM through an API provider, users expect the served model to remain consistent over time, a property crucial for the reliability of downstream applications and the reproducibility of research. Existing audit methods are too costly to apply at regular time intervals to the wide range of available LLM APIs. This means that model updates are left largely unmonitored in practice. In this work, we show that while LLM log probabilities (logprobs) are usually non-deterministic, they can still be used as the basis for cost-effective continuous monitoring of LLM APIs. We apply a simple statistical test based on the average value of each token logprob, requesting only a single token of output. This is enough to detect changes as small as one step of fine-tuning, making this approach more sensitive than existing methods while being 1,000x cheaper. We introduce the TinyChange benchmark as a way to measure the sensitivity of audit methods in the context of small, realistic model changes.

Method: Proposes a semantic wireless transmission framework using Deep Joint Source-Channel Coding (DeepJSCC). Instead of transmitting raw features, the transmitter predicts combination weights over a receiver-side semantic orthogonal feature pool. Uses a folding-based decoder that deforms a 2D grid into 3D to enforce manifold continuity while preserving geometric fidelity. Trained with Chamfer Distance and orthogonality regularization.

Result: Evaluated on ModelNet40 across varying SNRs and bandwidths. Performance matches SEmantic Point cloud Transmission (SEPT) at high bandwidth and shows clear gains in bandwidth-constrained regimes. Consistent improvements in both PSNR and Chamfer Distance metrics. Ablation studies confirm benefits of orthogonalization and folding prior.

Conclusion: The proposed semantic transmission framework effectively addresses point cloud transmission challenges by leveraging orthogonal feature pools and folding-based decoding, achieving bandwidth-efficient performance with robustness to varying channel conditions.

Abstract: The widespread adoption of depth sensors has substantially lowered the barrier to point-cloud acquisition. This letter proposes a semantic wireless transmission framework for three dimension (3D) point clouds built on Deep Joint Source - Channel Coding (DeepJSCC). Instead of sending raw features, the transmitter predicts combination weights over a receiver-side semantic orthogonal feature pool, enabling compact representations and robust reconstruction. A folding-based decoder deforms a 2D grid into 3D, enforcing manifold continuity while preserving geometric fidelity. Trained with Chamfer Distance (CD) and an orthogonality regularizer, the system is evaluated on ModelNet40 across varying Signal-to-Noise Ratios (SNRs) and bandwidths. Results show performance on par with SEmantic Point cloud Transmission (SEPT) at high bandwidth and clear gains in bandwidth-constrained regimes, with consistent improvements in both Peak Signal-to-Noise Ratio (PSNR) and CD. Ablation experiments confirm the benefits of orthogonalization and the folding prior.

Method: ACraft uses LLMs to automatically discover optimal attack methods for FSCIL. Incorporates PPO-based reinforcement learning to optimize LLM reasoning and generate better attack methods through positive feedback.

Result: ACraft significantly degrades performance of state-of-the-art FSCIL methods, dramatically outperforms human expert-designed attacks, and maintains lowest attack costs.

Conclusion: ACraft provides an effective automated approach for attacking FSCIL systems, highlighting security vulnerabilities and offering a specialized attack method that surpasses traditional approaches.

Abstract: Few-shot class incremental learning (FSCIL) is a more realistic and challenging paradigm in continual learning to incrementally learn unseen classes and overcome catastrophic forgetting on base classes with only a few training examples. Previous efforts have primarily centered around studying more effective FSCIL approaches. By contrast, less attention was devoted to thinking the security issues in contributing to FSCIL. This paper aims to provide a holistic study of the impact of attacks on FSCIL. We first derive insights by systematically exploring how human expert-designed attack methods (i.e., PGD, FGSM) affect FSCIL. We find that those methods either fail to attack base classes, or suffer from huge labor costs due to relying on huge expert knowledge. This highlights the need to craft a specialized attack method for FSCIL. Grounded in these insights, in this paper, we propose a simple yet effective ACraft method to automatically steer and discover optimal attack methods targeted at FSCIL by leveraging Large Language Models (LLMs) without human experts. Moreover, to improve the reasoning between LLMs and FSCIL, we introduce a novel Proximal Policy Optimization (PPO) based reinforcement learning to optimize learning, making LLMs generate better attack methods in the next generation by establishing positive feedback. Experiments on mainstream benchmarks show that our ACraft significantly degrades the performance of state-of-the-art FSCIL methods and dramatically beyond human expert-designed attack methods while maintaining the lowest costs of attack.

Method: Introduces a framework explaining fuzzy simplicial sets as marginals of probability measures on simplicial sets, connecting them to generative models that sample Vietoris-Rips filtrations at random scales.

Result: Shows UMAP’s fuzzy weights arise from generative models, connects fuzzy simplicial sets to probabilistic models on face posets, clarifies KL divergence-fuzzy cross-entropy relation, and recovers t-norms/t-conorms via Boolean operations.

Conclusion: The probabilistic viewpoint provides unified theoretical foundation for fuzzy simplicial sets, clarifies UMAP’s role, and enables systematic derivation of new dimensionality reduction methods, demonstrated with Čech filtrations and triplet sampling.

Abstract: Fuzzy simplicial sets have become an object of interest in dimensionality reduction and manifold learning, most prominently through their role in UMAP. However, their definition through tools from algebraic topology without a clear probabilistic interpretation detaches them from commonly used theoretical frameworks in those areas. In this work we introduce a framework that explains fuzzy simplicial sets as marginals of probability measures on simplicial sets. In particular, this perspective shows that the fuzzy weights of UMAP arise from a generative model that samples Vietoris-Rips filtrations at random scales, yielding cumulative distribution functions of pairwise distances. More generally, the framework connects fuzzy simplicial sets to probabilistic models on the face poset, clarifies the relation between Kullback-Leibler divergence and fuzzy cross-entropy in this setting, and recovers standard t-norms and t-conorms via Boolean operations on the underlying simplicial sets. We then show how new embedding methods may be derived from this framework and illustrate this on an example where we generalize UMAP using Čech filtrations with triplet sampling. In summary, this probabilistic viewpoint provides a unified probabilistic theoretical foundation for fuzzy simplicial sets, clarifies the role of UMAP within this framework, and enables the systematic derivation of new dimensionality reduction methods.

Method: GFP couples a multi-step flow-matching policy with a distilled one-step actor. The actor uses weighted behavior cloning to focus on high-value actions, while the flow policy constrains the actor to stay aligned with the dataset’s best transitions while maximizing the critic.

Result: Achieves state-of-the-art performance across 144 state and pixel-based tasks from OGBench, Minari, and D4RL benchmarks, with substantial gains on suboptimal datasets and challenging tasks.

Conclusion: The mutual guidance between flow policy and actor enables selective imitation of high-value actions, overcoming limitations of traditional behavior regularization in offline RL.

Abstract: Offline reinforcement learning often relies on behavior regularization that enforces policies to remain close to the dataset distribution. However, such approaches fail to distinguish between high-value and low-value actions in their regularization components. We introduce Guided Flow Policy (GFP), which couples a multi-step flow-matching policy with a distilled one-step actor. The actor directs the flow policy through weighted behavior cloning to focus on cloning high-value actions from the dataset rather than indiscriminately imitating all state-action pairs. In turn, the flow policy constrains the actor to remain aligned with the dataset’s best transitions while maximizing the critic. This mutual guidance enables GFP to achieve state-of-the-art performance across 144 state and pixel-based tasks from the OGBench, Minari, and D4RL benchmarks, with substantial gains on suboptimal datasets and challenging tasks. Webpage: https://simple-robotics.github.io/publications/guided-flow-policy/

Method: Proposes a Discrete Variable (DV)-Circuit Quantum-Classical Physics-Informed Neural Network (QCPINN) that integrates classical preprocessing/postprocessing networks with a DV quantum core, leveraging quantum superposition and entanglement for enhanced high-dimensional feature mapping while embedding physical constraints. Tests three quantum circuit topologies: Cascade, Cross-mesh, and Alternate.

Result: QCPINNs achieve high prediction accuracy with fewer parameters than classical PINNs. The Alternate topology performs best for heterogeneous single-phase flow and two-phase Buckley-Leverett equation, while Cascade topology excels for compositional flow with convection-dispersion-adsorption coupling.

Conclusion: The work verifies the feasibility of QCPINN for reservoir engineering applications, bridging quantum computing research with industrial practice in oil and gas engineering.

Abstract: Solving partial differential equations (PDEs) for reservoir seepage is critical for optimizing oil and gas field development and predicting production performance. Traditional numerical methods suffer from mesh-dependent errors and high computational costs, while classical Physics-Informed Neural Networks (PINNs) face bottlenecks in parameter efficiency, high-dimensional expression, and strong nonlinear fitting. To address these limitations, we propose a Discrete Variable (DV)-Circuit Quantum-Classical Physics-Informed Neural Network (QCPINN) and apply it to three typical reservoir seepage models for the first time: the pressure diffusion equation for heterogeneous single-phase flow, the nonlinear Buckley-Leverett (BL) equation for two-phase waterflooding, and the convection-diffusion equation for compositional flow considering adsorption. The QCPINN integrates classical preprocessing/postprocessing networks with a DV quantum core, leveraging quantum superposition and entanglement to enhance high-dimensional feature mapping while embedding physical constraints to ensure solution consistency. We test three quantum circuit topologies (Cascade, Cross-mesh, Alternate) and demonstrate through numerical experiments that QCPINNs achieve high prediction accuracy with fewer parameters than classical PINNs. Specifically, the Alternate topology outperforms others in heterogeneous single-phase flow and two-phase BL equation simulations, while the Cascade topology excels in compositional flow with convection-dispersion-adsorption coupling. Our work verifies the feasibility of QCPINN for reservoir engineering applications, bridging the gap between quantum computing research and industrial practice in oil and gas engineering.

Method: Adds a robust, precision-weighted penalty to the ELBO that encourages the encoder to allocate posterior mass proportionally to data-space density, and when using learnable priors, nudges the prior toward high-density regions. This incurs negligible computational overhead.

Result: On synthetic datasets: (i) better distributional alignment of latent log-densities to ground truth, (ii) improved prior coverage, (iii) better OOD uncertainty calibration. On MNIST: better alignment of prior with external density estimates (improved interpretability) and better OOD detection for learnable priors.

Conclusion: DiVAE effectively bridges the gap between simple VAE priors and actual data-space density structure through a lightweight, data-driven regularization approach that improves multiple aspects of VAE performance without significant computational cost.

Abstract: We introduce Density-Informed VAE (DiVAE), a lightweight, data-driven regularizer that aligns the VAE log-prior probability $\log p_Z(z)$ with a log-density estimated from data. Standard VAEs match latents to a simple prior, overlooking density structure in the data-space. DiVAE encourages the encoder to allocate posterior mass in proportion to data-space density and, when the prior is learnable, nudges the prior toward high-density regions. This is realized by adding a robust, precision-weighted penalty to the ELBO, incurring negligible computational overhead. On synthetic datasets, DiVAE (i) improves distributional alignment of latent log-densities to its ground truth counterpart, (ii) improves prior coverage, and (iii) yields better OOD uncertainty calibration. On MNIST, DiVAE improves alignment of the prior with external estimates of the density, providing better interpretability, and improves OOD detection for learnable priors.

Method: The paper conducts a systematic review of text dataset distillation methods, analyzing different distillation strategies including adaptations from vision, transformer-based approaches, discrete text generation methods, and scaling techniques for large language models.

Result: The review identifies key milestones in text dataset distillation development: adaptation from vision methods, introduction of transformer-based approaches, generation of discrete synthetic text, and scaling to decoder-only models with over 1B parameters.

Conclusion: Text dataset distillation remains in a maturing phase with several challenges: need for standardized benchmarking, approaches to handle text’s discrete nature, handling complex tasks, and demonstrating real-world applications. Future work should address these gaps to advance the field.

Abstract: In the vision domain, dataset distillation arises as a technique to condense a large dataset into a smaller synthetic one that exhibits a similar result in the training process. While image data presents an extensive literature of distillation methods, text dataset distillation has fewer works in comparison. Text dataset distillation initially grew as an adaptation of efforts from the vision universe, as the particularities of the modality became clear obstacles, it rose into a separate branch of research. Several milestones mark the development of this area, such as the introduction of methods that use transformer models, the generation of discrete synthetic text, and the scaling to decoder-only models with over 1B parameters. Despite major advances in modern approaches, the field remains in a maturing phase, with room for improvement on benchmarking standardization, approaches to overcome the discrete nature of text, handling complex tasks, and providing explicit examples of real-world applications. In this report, we review past and recent advances in dataset distillation for text, highlighting different distillation strategies, key contributions, and general challenges.

Method: 1. Introduce concept of Bilinear RNNs with comprehensive analysis. 2. Propose Comba variant based on closed-loop control theory with scalar-plus-low-rank state transition and both state/output feedback corrections. 3. Implement hardware-efficient chunk-wise parallel kernel in Triton. 4. Train 340M/1.3B parameter models on large-scale corpus.

Result: Comba demonstrates superior performance and computation efficiency in both language and vision modeling compared to previous approaches.

Conclusion: Comba represents an effective Bilinear RNN variant that combines theoretical analysis with practical implementation, achieving state-of-the-art efficiency and performance across multiple domains.

Abstract: Recent efficient sequence modeling methods such as Gated DeltaNet, TTT, and RWKV-7 have achieved performance improvements by supervising the recurrent memory management through Delta learning rule. Unlike previous state-space models (e.g., Mamba) and gated linear attentions (e.g., GLA), these models introduce interactions between the recurrent state and the key vector, structurally resembling bilinear systems. In this paper, we first introduce the concept of Bilinear RNNs with a comprehensive analysis on the advantages and limitations of these models. Then, based on closed-loop control theory, we propose a novel Bilinear RNN variant named Comba, which adopts a scalar-plus-low-rank state transition, with both state feedback and output feedback corrections. We also implement a hardware-efficient chunk-wise parallel kernel in Triton and train models with 340M/1.3B parameters on large-scale corpus. Comba demonstrates superior performance and computation efficiency in both language and vision modeling.

Method: Proposes a training-free approach that treats policy violation detection as OOD detection. Uses whitening techniques to decorrelate hidden activations and standardize them, then uses Euclidean norm as compliance score. Requires only policy text and few illustrative samples.

Result: Achieves state-of-the-art results on a challenging policy benchmark, surpassing both existing guardrails and fine-tuned reasoning models.

Conclusion: Provides organizations with a practical, statistically grounded framework for policy-aware oversight of LLMs, advancing deployable AI governance.

Abstract: Aligning proprietary large language models (LLMs) with internal organizational policies has become an urgent priority as organizations increasingly deploy LLMs in sensitive domains such as legal support, finance, and medical services. Beyond generic safety filters, enterprises require reliable mechanisms to detect policy violations within their regulatory and operational frameworks, where breaches can trigger legal and reputational risks. Existing content moderation frameworks, such as guardrails, remain largely confined to the safety domain and lack the robustness to capture nuanced organizational policies. LLM-as-a-judge and fine-tuning approaches, though flexible, introduce significant latency and lack interpretability. To address these limitations, we propose a training-free and efficient method that treats policy violation detection as an out-of-distribution (OOD) detection problem. Inspired by whitening techniques, we apply a linear transformation to decorrelate the model’s hidden activations and standardize them to zero mean and unit variance, yielding near-identity covariance matrix. In this transformed space, we use the Euclidean norm as a compliance score to detect policy violations. The method requires only the policy text and a small number of illustrative samples, which makes it light-weight and easily deployable. On a challenging policy benchmark, our approach achieves state-of-the-art results, surpassing both existing guardrails and fine-tuned reasoning models. This work provides organizations with a practical and statistically grounded framework for policy-aware oversight of LLMs, advancing the broader goal of deployable AI governance. Code is available at: https://tinyurl.com/policy-violation-detection

Method: GTPO introduces two key mechanisms: (1) prevents conflicting gradients by skipping negative updates while amplifying positive ones for valuable tokens, and (2) filters out completions whose entropy exceeds a provable threshold to prevent policy collapse. Unlike GRPO, GTPO eliminates KL-divergence regularization and reference model dependency.

Result: Experimental validation on GSM8K, MATH, AIME 2024, AIME 2025 and AMC 2023 datasets shows GTPO achieves greater training stability and improved performance compared to GRPO.

Conclusion: GTPO effectively addresses GRPO’s limitations by resolving token-level penalization and policy collapse issues, providing a more stable and performant policy optimization approach for LLM alignment without requiring KL-divergence regularization or reference models.

Abstract: Group Relative Policy Optimization (GRPO) is a promising policy-based approach for Large Language Model alignment, yet its performance is often limited by training instability and suboptimal convergence. In this paper, we identify and analyze two main GRPO issues: (i) the token-level penalization, where valuable tokens shared across different responses receive contradictory feedback signals, leading to conflicting gradient updates that can reduce their likelihood; and (ii) the policy collapse, where negatively rewarded completions may penalize confident responses and shift model decisions toward unlikely tokens, destabilizing training process. To address these issues we introduce GTPO (Group-relative Trajectory-based Policy Optimization), which prevents conflicting gradients on valuable tokens by skipping negative updates while amplifying positive ones and filters out completions whose entropy exceeds a provable threshold, to prevent policy collapse. Unlike GRPO, GTPO does not rely on KL-divergence regularization, eliminating the need for a reference model during training, while still ensuring greater training stability and improved performance, as validated through multiple experiments on GSM8K, MATH, AIME 2024, AIME 2025 and AMC 2023.

Method: Proposes Physics-Embedded Gaussian Process (PEGP) with two multi-output kernels informed by classic traffic flow models (LWR and ARZ), constructed via explicit application of linearized differential operators to integrate domain knowledge directly into the kernel structure.

Result: Experiments on HighD and NGSIM datasets show consistent improvements over non-physics baselines. PEGP-ARZ performs better under sparse observations, while PEGP-LWR achieves lower errors with denser data. PEGP-ARZ residuals align closely with physics and yield calibrated uncertainty, while PEGP-LWR residuals are more orthogonal with nearly constant variance.

Conclusion: The PEGP framework successfully combines physical priors with uncertainty quantification, providing reliable support for traffic state estimation by addressing limitations of previous hybrid approaches through principled kernel design.

Abstract: Traffic state estimation (TSE) becomes challenging when probe-vehicle penetration is low and observations are spatially sparse. Pure data-driven methods lack physical explanations and have poor generalization when observed data is sparse. In contrast, physical models have difficulty integrating uncertainties and capturing the real complexity of traffic. To bridge this gap, recent studies have explored combining them by embedding physical structure into Gaussian process. These approaches typically introduce the governing equations as soft constraints through pseudo-observations, enabling the integration of model structure within a variational framework. However, these methods rely heavily on penalty tuning and lack principled uncertainty calibration, which makes them sensitive to model mis-specification. In this work, we address these limitations by presenting a novel Physics-Embedded Gaussian Process (PEGP), designed to integrate domain knowledge with data-driven methods in traffic state estimation. Specifically, we design two multi-output kernels informed by classic traffic flow models, constructed via the explicit application of the linearized differential operator. Experiments on HighD, NGSIM show consistent improvements over non-physics baselines. PEGP-ARZ proves more reliable under sparse observation, while PEGP-LWR achieves lower errors with denser observation. Ablation study further reveals that PEGP-ARZ residuals align closely with physics and yield calibrated, interpretable uncertainty, whereas PEGP-LWR residuals are more orthogonal and produce nearly constant variance fields. This PEGP framework combines physical priors, uncertainty quantification, which can provide reliable support for TSE.

Method: Analyzes a canonical two-layer linear neural network with standard-basis inputs. Uses Hadamard initialization to diagonalize the softmax operator, freezing singular vectors and reducing dynamics to singular values. Constructs an explicit Lyapunov function to prove global convergence.

Result: Proves gradient flow on cross-entropy converges to neural collapse geometry for the first time. Shows global convergence despite spurious critical points in non-convex landscape. Hadamard initialization technique simplifies analysis by reducing dynamics to singular values.

Conclusion: The analysis provides fundamental understanding of CE dynamics beyond convex regimes, establishes neural collapse convergence, and introduces Hadamard initialization as a powerful technique for analyzing CE training dynamics in broader settings.

Abstract: Cross-entropy (CE) training loss dominates deep learning practice, yet existing theory often relies on simplifications, either replacing it with squared loss or restricting to convex models, that miss essential behavior. CE and squared loss generate fundamentally different dynamics, and convex linear models cannot capture the complexities of non-convex optimization. We provide an in-depth characterization of multi-class CE optimization dynamics beyond the convex regime by analyzing a canonical two-layer linear neural network with standard-basis vectors as inputs: the simplest non-convex extension for which the implicit bias remained unknown. This model coincides with the unconstrained features model used to study neural collapse, making our work the first to prove that gradient flow on CE converges to the neural collapse geometry. We construct an explicit Lyapunov function that establishes global convergence, despite the presence of spurious critical points in the non-convex landscape. A key insight underlying our analysis is an inconspicuous finding: Hadamard Initialization diagonalizes the softmax operator, freezing the singular vectors of the weight matrices and reducing the dynamics entirely to their singular values. This technique opens a pathway for analyzing CE training dynamics well beyond our specific setting considered here.

Method: Privately minimize a series of regularized objectives using only standard DP composition bounds, avoiding expensive verification methods that scale with training compute.

Result: Achieves tight privacy-utility trade-offs for DP-SCO while enabling verification with much less compute than training, significantly reducing verification costs on large datasets.

Conclusion: First DP-SCO algorithm with near-optimal privacy-utility trade-offs whose verification scales better than training cost, addressing the practical challenge of DP verification for data providers.

Abstract: Training with differential privacy (DP) provides a guarantee to members in a dataset that they cannot be identified by users of the released model. However, those data providers, and, in general, the public, lack methods to efficiently verify that models trained on their data satisfy DP guarantees. The amount of compute needed to verify DP guarantees for current algorithms scales with the amount of compute required to train the model. In this paper we design the first DP algorithm with near optimal privacy-utility trade-offs but whose DP guarantees can be verified cheaper than training. We focus on DP stochastic convex optimization (DP-SCO), where optimal privacy-utility trade-offs are known. Here we show we can obtain tight privacy-utility trade-offs by privately minimizing a series of regularized objectives and only using the standard DP composition bound. Crucially, this method can be verified with much less compute than training. This leads to the first known DP-SCO algorithm with near optimal privacy-utility whose DP verification scales better than training cost, significantly reducing verification costs on large datasets.

Method: 1) Theoretical analysis using information theory to prove domain feature collapse (I(x_d;z)=0) in single-domain supervised learning. 2) Extension using Fano’s inequality to quantify partial collapse. 3) Empirical validation through Domain Bench benchmark and domain filtering experiments using pretrained representations.

Result: 1) Proved that supervised learning on single-domain data inevitably causes domain feature collapse. 2) Demonstrated catastrophic OOD detection failure (e.g., 53% FPR@95 on MNIST). 3) Showed that preserving I(x_d;z)>0 through domain filtering resolves the failure mode.

Conclusion: Single-domain supervised learning fundamentally discards domain information due to information bottleneck optimization, explaining OOD detection failures. Domain filtering provides empirical validation and has implications for transfer learning and when to fine-tune vs. freeze pretrained models.

Abstract: Why do state-of-the-art OOD detection methods exhibit catastrophic failure when models are trained on single-domain datasets? We provide the first theoretical explanation for this phenomenon through the lens of information theory. We prove that supervised learning on single-domain data inevitably produces domain feature collapse – representations where I(x_d; z) = 0, meaning domain-specific information is completely discarded. This is a fundamental consequence of information bottleneck optimization: models trained on single domains (e.g., medical images) learn to rely solely on class-specific features while discarding domain features, leading to catastrophic failure when detecting out-of-domain samples (e.g., achieving only 53% FPR@95 on MNIST). We extend our analysis using Fano’s inequality to quantify partial collapse in practical scenarios. To validate our theory, we introduce Domain Bench, a benchmark of single-domain datasets, and demonstrate that preserving I(x_d; z) > 0 through domain filtering (using pretrained representations) resolves the failure mode. While domain filtering itself is conceptually straightforward, its effectiveness provides strong empirical evidence for our information-theoretic framework. Our work explains a puzzling empirical phenomenon, reveals fundamental limitations of supervised learning in narrow domains, and has broader implications for transfer learning and when to fine-tune versus freeze pretrained models.

Method: MarkTune is a theoretically principled, on-policy fine-tuning framework that treats the GaussMark watermark signal as a reward while regularizing against degradation in text quality. It steers finer-grained, watermark-aware weight updates within the model’s representation space.

Result: MarkTune consistently improves the quality-detectability trade-off over GaussMark, pushing it close to inference-time watermarking performance. It remains robust to paraphrasing and fine-tuning attacks, and shows strong generalization across unseen datasets.

Conclusion: MarkTune establishes a general strategy for embedding robust, high-quality watermarks into open-weight language models, addressing the fundamental challenge of watermarking in open-weight settings.

Abstract: Watermarking aims to embed hidden signals in generated text that can be reliably detected when given access to a secret key. Open-weight language models pose acute challenges for such watermarking schemes because the inference-time interventions that dominate contemporary approaches cannot be enforced once model weights are public. Existing watermaking techniques for open-weight models, such as the recently proposed GaussMark, typically rely on small modifications to model weights, which can yield signals detectable to those equipped with a secret key, but achieving detection power comparable to inference-time watermarks generally requires weight perturbations that noticeably reduce generation quality. We introduce MarkTune, a theoretically principled, on-policy fine-tuning framework that treats the GaussMark signal as a reward while simultaneously regularizing against degradation in text quality. We derive MarkTune as an improvement on GaussMark and demonstrate that MarkTune consistently improves the quality-detectability trade-off over GaussMark by steering finer-grained, watermark-aware weight updates within the model’s representation space while preserving generation quality. Empirically, we show that MarkTune pushes the quality-detectability frontier of GaussMark close to that of inference-time watermarking, remains robust to paraphrasing and fine-tuning attacks, and exhibits strong generalization: a model fine-tuned on one dataset retains substantial watermark detection power on unseen datasets. Together, these results establish MarkTune as a general strategy for embedding robust, high-quality watermarks into open-weight LMs.

Method: Systematically compare SFT and RL across different LM families (Llama, Qwen) and tasks (instruction following, general knowledge, arithmetic reasoning). Use a simplified mixture model to analyze the cause, and test practical implications of on-policy data.

Result: RL consistently leads to less forgetting than SFT while achieving comparable or higher target task performance. The mode-seeking nature of RL from on-policy data enables keeping prior knowledge intact.

Conclusion: RL’s robustness to forgetting stems from its use of on-policy data, not other algorithmic choices. Approximately on-policy data can efficiently mitigate forgetting in practical settings.

Abstract: Adapting language models (LMs) to new tasks via post-training carries the risk of degrading existing capabilities – a phenomenon classically known as catastrophic forgetting. In this paper, toward identifying guidelines for mitigating this phenomenon, we systematically compare the forgetting patterns of two widely adopted post-training methods: supervised fine-tuning (SFT) and reinforcement learning (RL). Our experiments reveal a consistent trend across LM families (Llama, Qwen) and tasks (instruction following, general knowledge, and arithmetic reasoning): RL leads to less forgetting than SFT while achieving comparable or higher target task performance. To investigate the cause for this difference, we consider a simplified setting in which the LM is modeled as a mixture of two distributions, one corresponding to prior knowledge and the other to the target task. We identify that the mode-seeking nature of RL, which stems from its use of on-policy data, enables keeping prior knowledge intact when learning the target task. We then verify this insight by demonstrating that the use on-policy data underlies the robustness of RL to forgetting in practical settings, as opposed to other algorithmic choices such as the KL regularization or advantage estimation. Lastly, as a practical implication, our results highlight the potential of mitigating forgetting using approximately on-policy data, which can be substantially more efficient to obtain than fully on-policy data.

Method: The paper introduces an alternative approach where the update rule itself is discrete from the start, avoiding quantization of continuous updates. They establish convergence guarantees for a general class of such discrete schemes and present a multinomial update rule as a concrete example.

Result: The approach is supported by empirical evaluation, demonstrating the viability of discrete update rules for efficient training.

Conclusion: This perspective opens new avenues for efficient training, particularly for models with inherently discrete structure, offering an alternative to traditional quantization approaches.

Abstract: Modern deep learning models require immense computational resources, motivating research into low-precision training. Quantised training addresses this by representing training components in low-bit integers, but typically relies on discretising real-valued updates. We introduce an alternative approach where the update rule itself is discrete, avoiding the quantisation of continuous updates by design. We establish convergence guarantees for a general class of such discrete schemes, and present a multinomial update rule as a concrete example, supported by empirical evaluation. This perspective opens new avenues for efficient training, particularly for models with inherently discrete structure.

Method: Introduces Eval Factsheets - a structured, descriptive framework with comprehensive taxonomy and questionnaire-based approach. Organizes evaluation characteristics across five dimensions: Context (Who made the evaluation and when?), Scope (What does it evaluate?), Structure (With what the evaluation is built?), Method (How does it work?), and Alignment (In what ways is it reliable/valid/robust?). Implemented as a practical questionnaire with mandatory and recommended elements.

Result: Through case studies on multiple benchmarks, demonstrates that Eval Factsheets effectively captures diverse evaluation paradigms - from traditional benchmarks to LLM-as-judge methodologies - while maintaining consistency and comparability.

Conclusion: The authors hope Eval Factsheets will be incorporated into both existing and newly released evaluation frameworks, leading to more transparency and reproducibility in AI system evaluations.

Abstract: The rapid proliferation of benchmarks has created significant challenges in reproducibility, transparency, and informed decision-making. However, unlike datasets and models – which benefit from structured documentation frameworks like Datasheets and Model Cards – evaluation methodologies lack systematic documentation standards. We introduce Eval Factsheets, a structured, descriptive framework for documenting AI system evaluations through a comprehensive taxonomy and questionnaire-based approach. Our framework organizes evaluation characteristics across five fundamental dimensions: Context (Who made the evaluation and when?), Scope (What does it evaluate?), Structure (With what the evaluation is built?), Method (How does it work?) and Alignment (In what ways is it reliable/valid/robust?). We implement this taxonomy as a practical questionnaire spanning five sections with mandatory and recommended documentation elements. Through case studies on multiple benchmarks, we demonstrate that Eval Factsheets effectively captures diverse evaluation paradigms – from traditional benchmarks to LLM-as-judge methodologies – while maintaining consistency and comparability. We hope Eval Factsheets are incorporated into both existing and newly released evaluation frameworks and lead to more transparency and reproducibility.

Method: Built a web application with Python backend that fetches fare data from ride-hailing APIs (Ola, Uber, Rapido). Uses Android Studio emulator, Appium, and location comparison techniques to access and compare data.

Result: Developed a functional web application that provides fare comparisons between multiple ride-hailing services and identifies the best option based on user’s destination input.

Conclusion: The project successfully addresses transparency issues in ride-hailing services, increases efficiency, and provides users with better experience through comparative fare analysis.

Abstract: In todays increasing world, it is very important to have good hailing services like Ola, Uber, and Rapido as it is very essential for our daily transportation. Users often face difficulties in choosing the most appropriate and efficient ride that would lead to both cost-effective and would take us to our destination in less time. This project provides you with the web application that helps you to select the most beneficial ride for you by providing users with the fare comparison between Ola, Uber, Rapido for the destination entered by the user. The backend is use to fetch the data, providing users with the fare comparison for the ride and finally providing with the best option using Python. This research paper also addresses the problem and challenges faced in accessing the data using APIs, Android Studios emulator, Appium and location comparison. Thus, the aim of the project is to provide transparency to the users in ride-hailing services and increase efficiency and provide users with better experience.

Method: Use self-play with two agents (user simulator and AI assistant) to generate conversations with ambiguous queries. Train models using Reinforced Self-Training (ReST) to maximize final reward defined as cost-penalized accuracy, where costs are provided for clarification questions and generated words.

Result: The approach produces steerable policies that change behavior predictably based on provided costs, achieving higher reward and accuracy. The method generalizes to unseen cost values at test time.

Conclusion: Self-play with cost-aware reinforcement learning enables training of flexible, context-dependent policies for managing uncertainty in AI assistants, with predictable behavior adaptation to different cost constraints and good generalization to new cost values.

Abstract: To handle underspecified or ambiguous queries, AI assistants need a policy for managing their uncertainty to determine (a) when to guess the user intent and answer directly, (b) when to enumerate and answer multiple possible intents, and (c) when to ask a clarifying question. However, such policies are contextually dependent on factors such as user preferences or modality. For example, enumerating multiple possible user intentions is cumbersome on small screens or in a voice setting. In this work, we propose to train steerable policies for managing this uncertainty using self-play. Given two agents, one simulating a user and the other an AI assistant, we generate conversations where the user issues a potentially ambiguous query, and the assistant needs to determine how to respond. Importantly, the model takes as input the numerical cost of each clarification question, and each generated word, and is asked to take the action that will maximize its final reward, which is the cost-penalized accuracy. We use Reinforced Self-Training (ReST) to train our model to achieve high reward and show this leads to a steerable policy that changes its behavior predictably conditioned on the provided costs, leading to higher reward and accuracy. Moreover, our procedure also generalizes to numerical cost values that were unobserved at training time.

Method: Uses first-order approximation to show token-level surrogate objectives approximate sequence-level rewards when training-inference discrepancy and policy staleness are minimized. Tests insights through extensive experiments with a 30B Mixture-of-Experts model, comparing on-policy training with importance sampling correction versus off-policy training with clipping and Routing Replay.

Result: For on-policy training, basic policy gradient with importance sampling correction achieves highest stability. For off-policy training, combining clipping and Routing Replay is essential to mitigate instability from policy staleness. Once stabilized, prolonged optimization yields comparable final performance regardless of initialization.

Conclusion: The paper provides theoretical justification for token-level surrogate objectives in RL with LLMs and practical recipes for stable training, explaining why common techniques like importance sampling, clipping, and Routing Replay work. These insights should facilitate future RL research with large language models.

Abstract: This paper proposes a novel formulation for reinforcement learning (RL) with large language models, explaining why and under what conditions the true sequence-level reward can be optimized via a surrogate token-level objective in policy gradient methods such as REINFORCE. Specifically, through a first-order approximation, we show that this surrogate becomes increasingly valid only when both the training-inference discrepancy and policy staleness are minimized. This insight provides a principled explanation for the crucial role of several widely adopted techniques in stabilizing RL training, including importance sampling correction, clipping, and particularly Routing Replay for Mixture-of-Experts (MoE) models. Through extensive experiments with a 30B MoE model totaling hundreds of thousands of GPU hours, we show that for on-policy training, the basic policy gradient algorithm with importance sampling correction achieves the highest training stability. When off-policy updates are introduced to accelerate convergence, combining clipping and Routing Replay becomes essential to mitigate the instability caused by policy staleness. Notably, once training is stabilized, prolonged optimization consistently yields comparable final performance regardless of cold-start initialization. We hope that the shared insights and the developed recipes for stable RL training will facilitate future research.

Method: ZIP-RC reuses reserved/unused logits in the same forward pass to output joint distribution over final reward and remaining length, computes sampling utility (expected max reward, compute, latency), and uses meta-actions to decide which token prefixes to continue or sample from.

Result: On mixed-difficulty math benchmarks, ZIP-RC improves accuracy by up to 12% over majority voting at equal or lower average cost, and creates smooth Pareto frontiers between quality, compute, and latency.

Conclusion: ZIP-RC provides real-time reward-cost introspection enabling adaptive, efficient reasoning without extra models or inference overhead.

Abstract: Large language models excel at reasoning but lack key aspects of introspection, including anticipating their own success and the computation required to achieve it. Humans use real-time introspection to decide how much effort to invest, when to make multiple attempts, when to stop, and when to signal success or failure. Without this, LLMs struggle to make intelligent meta-cognition decisions. Test-time scaling methods like Best-of-N drive up cost and latency by using a fixed budget of samples regardless of the marginal benefit of each one at any point in generation, and the absence of confidence signals can mislead people, prevent appropriate escalation to better tools, and undermine trustworthiness. Learned verifiers or reward models can provide confidence estimates, but do not enable adaptive inference and add substantial cost by requiring extra models or forward passes. We present ZIP-RC, an adaptive inference method that equips models with zero-overhead inference-time predictions of reward and cost. At every token, ZIP-RC reuses reserved or unused logits in the same forward pass as next-token prediction to output a joint distribution over final reward and remaining length – no extra models, architecture change, or inference overhead. This full joint distribution is used to compute a sampling utility which is the linear combination of the expected maximum reward, total compute, and latency of set of samples if generated to completion. During inference, we maximize this utility with meta-actions that determine which prefix of tokens to continue or initiate sampling from. On mixed-difficulty mathematical benchmarks, ZIP-RC improves accuracy by up to 12% over majority voting at equal or lower average cost, and traces smooth Pareto frontiers between quality, compute, and latency. By providing real-time reward-cost introspection, ZIP-RC enables adaptive, efficient reasoning.

Method: Developed FairDream fairness package with transparent mechanisms for lay users, increasing model weights for errors on disadvantaged groups. Analyzed the relationship between fairness criteria, examined FairDream’s reweighting process, and compared trade-offs with GridSearch models.

Result: FairDream tends toward Equalized Odds rather than the intended Demographic Parity, revealing a conservative bias inherent to the data environment. The paper clarifies relationships between fairness criteria and justifies normative preference for Equalized Odds through epistemological interpretation.

Conclusion: Fairness interventions require transparent explanations of constraints, not just criterion satisfaction. The tendency toward Equalized Odds over Demographic Parity reveals inherent data biases, and epistemological analysis via Simpson’s paradox justifies normative preference for Equalized Odds in transparent fairness interventions.

Abstract: This paper presents a philosophical and experimental study of fairness interventions in AI classification, centered on the explainability of corrective methods. We argue that ensuring fairness requires not only satisfying a target criterion, but also explaining which variables constrain its realization. When corrections are used to mitigate advantage transparently, they must remain sensitive to the distribution of true labels. To illustrate this approach, we built FairDream, a fairness package whose mechanism is made transparent for lay users, increasing the model’s weights of errors on disadvantaged groups. While a user may intend to achieve Demographic Parity by the correction method, experiments show that FairDream tends towards Equalized Odds, revealing a conservative bias inherent to the data environment. We clarify the relationship between these fairness criteria, analyze FairDream’s reweighting process, and compare its trade-offs with closely related GridSearch models. Finally, we justify the normative preference for Equalized Odds via an epistemological interpretation of the results, using their proximity with Simpson’s paradox. The paper thus unites normative, epistemological, and empirical explanations of fairness interventions, to ensure transparency for the users.

Method: Using effective field theory perspective to identify root causes of RePU failures, then proposing Modified Rectified Power Unit (MRePU) activation function that satisfies criticality conditions for stable training.

Result: MRePU demonstrates significant improvements in training stability and performance across polynomial regression, physics-informed neural networks (PINNs), and real-world vision tasks.

Conclusion: MRePU is a robust alternative for building deep neural networks, addressing RePU’s limitations while preserving its advantages, and belongs to a distinct universality class for stable training.

Abstract: The Rectified Power Unit (RePU) activation function, a differentiable generalization of the Rectified Linear Unit (ReLU), has shown promise in constructing neural networks due to its smoothness properties. However, deep RePU networks often suffer from critical issues such as vanishing or exploding values during training, rendering them unstable regardless of hyperparameter initialization. Leveraging the perspective of effective field theory, we identify the root causes of these failures and propose the Modified Rectified Power Unit (MRePU) activation function. MRePU addresses RePU’s limitations while preserving its advantages, such as differentiability and universal approximation properties. Theoretical analysis demonstrates that MRePU satisfies criticality conditions necessary for stable training, placing it in a distinct universality class. Extensive experiments validate the effectiveness of MRePU, showing significant improvements in training stability and performance across various tasks, including polynomial regression, physics-informed neural networks (PINNs) and real-world vision tasks. Our findings highlight the potential of MRePU as a robust alternative for building deep neural networks.

Method: Asynchronous FL design with periodic aggregation, featuring a scheduling policy that jointly considers channel quality and training data representation of devices, plus an age-aware aggregation weighting mechanism.

Result: The proposed channel-aware data-importance scheduling policy outperforms state-of-the-art synchronous FL methods, and age-aware aggregation weighting significantly improves learning performance in asynchronous FL settings.

Conclusion: Joint consideration of communication quality and data importance in scheduling, combined with age-aware aggregation, effectively addresses straggler issues and improves FL performance in resource-constrained wireless environments.

Abstract: Federated Learning (FL) is a collaborative machine learning (ML) framework that combines on-device training and server-based aggregation to train a common ML model among distributed agents. In this work, we propose an asynchronous FL design with periodic aggregation to tackle the straggler issue in FL systems. Considering limited wireless communication resources, we investigate the effect of different scheduling policies and aggregation designs on the convergence performance. Driven by the importance of reducing the bias and variance of the aggregated model updates, we propose a scheduling policy that jointly considers the channel quality and training data representation of user devices. The effectiveness of our channel-aware data-importance-based scheduling policy, compared with state-of-the-art methods proposed for synchronous FL, is validated through simulations. Moreover, we show that an ``age-aware’’ aggregation weighting design can significantly improve the learning performance in an asynchronous FL setting.

Method: Proposes Structural Anchor Points (SAPs) to capture inter-cluster interactions, leading to LIFT-SAP which integrates both distance and directional information relative to SAPs. Then develops LDL-LIFT-SAP algorithm that unifies multiple label description degrees from different LSF spaces into a cohesive label distribution.

Result: Extensive experiments on 15 real-world datasets show LIFT-SAP outperforms original LIFT, and LDL-LIFT-SAP demonstrates superiority over seven other well-established algorithms in label distribution learning tasks.

Conclusion: The proposed SAPs effectively capture inter-cluster interactions, and the LIFT-SAP strategy with multi-perspective information provides more robust label-specific features, leading to improved label distribution learning performance through the unified LDL-LIFT-SAP algorithm.

Abstract: Label distribution learning (LDL) is an emerging learning paradigm designed to capture the relative importance of labels for each instance. Label-specific features (LSFs), constructed by LIFT, have proven effective for learning tasks with label ambiguity by leveraging clustering-based prototypes for each label to re-characterize instances. However, directly introducing LIFT into LDL tasks can be suboptimal, as the prototypes it collects primarily reflect intra-cluster relationships while neglecting cross-cluster interactions. Additionally, constructing LSFs using multi-perspective information, rather than relying solely on Euclidean distance, provides a more robust and comprehensive representation of instances, mitigating noise and bias that may arise from a single distance perspective. To address these limitations, we introduce Structural Anchor Points (SAPs) to capture inter-cluster interactions. This leads to a novel LSFs construction strategy, LIFT-SAP, which enhances LIFT by integrating both distance and directional information of each instance relative to SAPs. Furthermore, we propose a novel LDL algorithm, Label Distribution Learning via Label-specifIc FeaTure with SAPs (LDL-LIFT-SAP), which unifies multiple label description degrees predicted from different LSF spaces into a cohesive label distribution. Extensive experiments on 15 real-world datasets demonstrate the effectiveness of LIFT-SAP over LIFT, as well as the superiority of LDL-LIFT-SAP compared to seven other well-established algorithms.

Method: Formalizes the concept of “size generalization” - predicting algorithm performance on large instances by evaluating on smaller representative subsamples. Provides theoretical guarantees for specific clustering and max-cut algorithms.

Result: Establishes theoretical bounds on subsample size needed to ensure performance on subsamples reflects performance on full instances. Experimental results support the theoretical findings.

Conclusion: Provides first theoretical foundations for the common practice of using smaller proxy instances in algorithm selection, with specific guarantees for several important algorithms.

Abstract: Data-driven algorithm selection is a powerful approach for choosing effective heuristics for computational problems. It operates by evaluating a set of candidate algorithms on a collection of representative training instances and selecting the one with the best empirical performance. However, running each algorithm on every training instance is computationally expensive, making scalability a central challenge. In practice, a common workaround is to evaluate algorithms on smaller proxy instances derived from the original inputs. However, this practice has remained largely ad hoc and lacked theoretical grounding. We provide the first theoretical foundations for this practice by formalizing the notion of size generalization: predicting an algorithm’s performance on a large instance by evaluating it on a smaller, representative instance, subsampled from the original instance. We provide size generalization guarantees for three widely used clustering algorithms (single-linkage, $k$-means++, and Gonzalez’s $k$-centers heuristic) and two canonical max-cut algorithms (Goemans-Williamson and Greedy). We characterize the subsample size sufficient to ensure that performance on the subsample reflects performance on the full instance, and our experiments support these findings.

Method: Learn a joint generative model of features and targets from incomplete data, then at deployment condition on observed features and marginalize unobserved ones to produce forecasts without imputation.

Result: Improves forecast quality in terms of continuous ranked probability score relative to impute-then-predict baselines while incurring substantially lower computational cost than common alternatives.

Conclusion: The imputation-free approach avoids error introduced by imputation, preserves uncertainty from missing features, and provides better probabilistic wind power forecasts with lower computational cost.

Abstract: Machine learning methods are widely and successfully used for probabilistic wind power forecasting, yet the pervasive issue of missing values (e.g., due to sensor faults or communication outages) has received limited attention. The prevailing practice is impute-then-predict, but conditioning on point imputations biases parameter estimates and fails to propagate uncertainty from missing features. Our approach treats missing features and forecast targets uniformly: we learn a joint generative model of features and targets from incomplete data and, at operational deployment, condition on the observed features and marginalize the unobserved ones to produce forecasts. This imputation-free procedure avoids error introduced by imputation and preserves uncertainty aroused from missing features. In experiments, it improves forecast quality in terms of continuous ranked probability score relative to impute-then-predict baselines while incurring substantially lower computational cost than common alternatives.

Method: Systematic evaluation of state-of-the-art generative models for time series data. Introduced an evaluation framework assessing both intrinsic fidelity of generated data and utility in downstream predictive tasks. Examined models’ ability to handle multi-modality, capture long-range dependencies, and support conditional generation.

Result: Revealed critical limitations in existing approaches: poor cross-modal consistency, inadequate temporal coherence preservation, and weak performance in train-on-synthetic/test-on-real and data augmentation scenarios. Models struggle with real-world wearable sensor data requirements.

Conclusion: Current generative models have significant limitations for wearable sensor data generation. Future research directions are needed to enhance synthetic time series generation and improve applicability in wearable computing domain, particularly for multi-modal, long-range, conditional generation scenarios.

Abstract: The widespread adoption of wearable sensors has the potential to provide massive and heterogeneous time series data, driving the use of Artificial Intelligence in human sensing applications. However, data collection remains limited due to stringent ethical regulations, privacy concerns, and other constraints, hindering progress in the field. Synthetic data generation, particularly through Generative Adversarial Networks and Diffusion Models, has emerged as a promising solution to mitigate both data scarcity and privacy issues. However, these models are often limited to narrow operational scenarios, such as short-term and unimodal signal patterns. To address this gap, we present a systematic evaluation of state-of-the-art generative models for time series data, explicitly assessing their performance in challenging scenarios such as stress and emotion recognition. Our study examines the extent to which these models can jointly handle multi-modality, capture long-range dependencies, and support conditional generation-core requirements for real-world wearable sensor data generation. To enable a fair and rigorous comparison, we also introduce an evaluation framework that evaluates both the intrinsic fidelity of the generated data and their utility in downstream predictive tasks. Our findings reveal critical limitations in the existing approaches, particularly in maintaining cross-modal consistency, preserving temporal coherence, and ensuring robust performance in train-on-synthetic, test-on-real, and data augmentation scenarios. Finally, we present our future research directions to enhance synthetic time series generation and improve the applicability of generative models in the wearable computing domain.

Method: Uses a unified approach based on martingale decomposition of cumulative reward, properties of policy evaluation fixed-point equations, and concentration results for martingale difference sequences.

Result: Derives comprehensive concentration results: asymptotic laws (law of large numbers, central limit theorem, law of iterated logarithms) and non-asymptotic bounds (Azuma-Hoeffding-type inequalities, non-asymptotic LIL). Shows applications to policy reward differences and regret equivalence.

Conclusion: Provides a complete theoretical framework for reward concentration in MDPs, enabling rigorous analysis of policy performance and learning algorithms through both asymptotic and non-asymptotic concentration bounds.

Abstract: In this paper, we investigate the concentration properties of cumulative reward in Markov Decision Processes (MDPs), focusing on both asymptotic and non-asymptotic settings. We introduce a unified approach to characterize reward concentration in MDPs, covering both infinite-horizon settings (i.e., average and discounted reward frameworks) and finite-horizon setting. Our asymptotic results include the law of large numbers, the central limit theorem, and the law of iterated logarithms, while our non-asymptotic bounds include Azuma-Hoeffding-type inequalities and a non-asymptotic version of the law of iterated logarithms. Additionally, we explore two key implications of our results. First, we analyze the sample path behavior of the difference in rewards between any two stationary policies. Second, we show that two alternative definitions of regret for learning policies proposed in the literature are rate-equivalent. Our proof techniques rely on a martingale decomposition of cumulative reward, properties of the solution to the policy evaluation fixed-point equation, and both asymptotic and non-asymptotic concentration results for martingale difference sequences.

Method: Propose a conformal prediction approach that constructs distribution-free bounds for unknown conformity scores using recent results on conformal p-value distributions. These bounds are used to create valid prediction sets for item ranks while controlling false coverage proportion for multiple prediction sets.

Result: The method provides valid prediction sets for item ranks and controls false coverage proportion. Empirical evaluation on synthetic and real data shows effectiveness with state-of-the-art ranking algorithms like RankNet and LambdaMart.

Conclusion: The proposed conformal prediction framework successfully quantifies uncertainty in ranking algorithms even when only partial ground truth is available, addressing a practical challenge in ranking applications.

Abstract: We introduce a method based on Conformal Prediction (CP) to quantify the uncertainty of full ranking algorithms. We focus on a specific scenario where $n+m$ items are to be ranked by some ``black box’’ algorithm. It is assumed that the relative (ground truth) ranking of $n$ of them is known. The objective is then to quantify the error made by the algorithm on the ranks of the $m$ new items among the total $(n+m)$. In such a setting, the true ranks of the $n$ original items in the total $(n+m)$ depend on the (unknown) true ranks of the $m$ new ones. Consequently, we have no direct access to a calibration set to apply a classical CP method. To address this challenge, we propose to construct distribution-free bounds of the unknown conformity scores using recent results on the distribution of conformal p-values. Using these scores upper bounds, we provide valid prediction sets for the rank of any item. We also control the false coverage proportion, a crucial quantity when dealing with multiple prediction sets. Finally, we empirically show on both synthetic and real data the efficiency of our CP method for state-of-the-art algorithms such as RankNet or LambdaMart.

Method: ConfRover uses a modular architecture with: (1) an encoding layer adapted from protein folding models to embed protein information and conformations into latent space, (2) a temporal module (sequence model) to capture conformational dynamics across frames, and (3) an SE(3) diffusion model as structure decoder for continuous space conformation generation.

Result: Experiments on the ATLAS dataset (large-scale protein MD dataset) demonstrate ConfRover’s effectiveness in learning conformational dynamics and supporting a wide range of downstream tasks.

Conclusion: ConfRover is the first model to sample both protein conformations and trajectories within a single framework, offering a novel and flexible approach for learning from protein MD data.

Abstract: Understanding protein dynamics is critical for elucidating their biological functions. The increasing availability of molecular dynamics (MD) data enables the training of deep generative models to efficiently explore the conformational space of proteins. However, existing approaches either fail to explicitly capture the temporal dependencies between conformations or do not support direct generation of time-independent samples. To address these limitations, we introduce ConfRover, an autoregressive model that simultaneously learns protein conformation and dynamics from MD trajectories, supporting both time-dependent and time-independent sampling. At the core of our model is a modular architecture comprising: (i) an encoding layer, adapted from protein folding models, that embeds protein-specific information and conformation at each time frame into a latent space; (ii) a temporal module, a sequence model that captures conformational dynamics across frames; and (iii) an SE(3) diffusion model as the structure decoder, generating conformations in continuous space. Experiments on ATLAS, a large-scale protein MD dataset of diverse structures, demonstrate the effectiveness of our model in learning conformational dynamics and supporting a wide range of downstream tasks. ConfRover is the first model to sample both protein conformations and trajectories within a single framework, offering a novel and flexible approach for learning from protein MD data. Project website: https://bytedance-seed.github.io/ConfRover.

Method: Proposed scaling TTT for CLMs by increasing number of independent RL agents. Introduced MolExp benchmark for evaluating discovery of structurally diverse molecules with similar bioactivity. Evaluated cooperative RL strategies to enhance exploration efficiency.

Result: Scaling TTT with multiple RL agents follows log-linear scaling law, significantly improving exploration efficiency measured by MolExp. Increasing TTT training time yields diminishing returns even with exploration bonuses. Cooperative RL strategies further enhance exploration efficiency.

Conclusion: Provides scalable framework for generative molecular design with insights for optimizing AI-driven drug discovery. Shows that agent scaling is more effective than training time extension for improving chemical space exploration.

Abstract: Chemical Language Models (CLMs) leveraging reinforcement learning (RL) have shown promise in de novo molecular design, yet often suffer from mode collapse, limiting their exploration capabilities. Inspired by Test-Time Training (TTT) in large language models, we propose scaling TTT for CLMs to enhance chemical space exploration. We introduce MolExp, a novel benchmark emphasizing the discovery of structurally diverse molecules with similar bioactivity, simulating real-world drug design challenges. Our results demonstrate that scaling TTT by increasing the number of independent RL agents follows a log-linear scaling law, significantly improving exploration efficiency as measured by MolExp. In contrast, increasing TTT training time yields diminishing returns, even with exploration bonuses. We further evaluate cooperative RL strategies to enhance exploration efficiency. These findings provide a scalable framework for generative molecular design, offering insights into optimizing AI-driven drug discovery.

Method: Develops a filtration on temporal graphs based on δ-temporal motifs (recurrent subgraphs), creating a multi-scale representation that captures temporal structure at different scales.

Result: Demonstrates effectiveness of the approach on temporal graph classification tasks, showing that the filtration method enables successful application of static graph analysis tools to temporal graphs.

Conclusion: The proposed temporal filtration provides a bridge between static graph analysis tools and temporal graph analysis, offering a principled way to analyze temporal structure at multiple scales for classification tasks.

Abstract: In this work, we introduce a filtration on temporal graphs based on $δ$-temporal motifs (recurrent subgraphs), yielding a multi-scale representation of temporal structure. Our temporal filtration allows tools developed for filtered static graphs, including persistent homology and recent graph filtration kernels, to be applied directly to temporal graph analysis. We demonstrate the effectiveness of this approach on temporal graph classification tasks.

Method: Two complementary architectures: 1) SmaAt-fUsion extends SmaAt-UNet by incorporating weather station data through convolutional layers in the network bottleneck; 2) SmaAt-Krige-GNet uses Kriging interpolation to create variable-specific maps from station data, then employs dual-encoder architecture for multi-level integration.

Result: SmaAt-Krige-GNet outperforms radar-only SmaAt-UNet in low precipitation scenarios, while SmaAt-fUsion surpasses SmaAt-UNet in both low and high precipitation scenarios, demonstrating the value of multi-variable station data integration.

Conclusion: Incorporating discrete weather station data significantly enhances deep learning-based precipitation nowcasting models, with different architectures showing complementary strengths across precipitation intensity scenarios.

Abstract: Short-term precipitation nowcasting is essential for flood management, transportation, energy system operations, and emergency response. However, many existing models fail to fully exploit the extensive atmospheric information available, relying primarily on precipitation data alone. This study examines whether integrating multi-variable weather-station measurements with radar can enhance nowcasting skill and introduces two complementary architectures that integrate multi-variable station data with radar images. The SmaAt-fUsion model extends the SmaAt-UNet framework by incorporating weather station data through a convolutional layer, integrating it into the bottleneck of the network; The SmaAt-Krige-GNet model combines precipitation maps with weather station data processed using Kriging, a geo-statistical interpolation method, to generate variable-specific maps. These maps are then utilized in a dual-encoder architecture based on SmaAt-GNet, allowing multi-level data integration . Experimental evaluations were conducted using four years (2016–2019) of weather station and precipitation radar data from the Netherlands. Results demonstrate that SmaAt-Krige-GNet outperforms the standard SmaAt-UNet, which relies solely on precipitation radar data, in low precipitation scenarios, while SmaAt-fUsion surpasses SmaAt-UNet in both low and high precipitation scenarios. This highlights the potential of incorporating discrete weather station data to enhance the performance of deep learning-based weather nowcasting models.

Method: 1) Proposes an improved training protocol that minimizes time lag between training and test data while reducing validation bias, showing random splits can outperform temporal splits. 2) Introduces a plug-and-play temporal embedding method using Fourier series expansion to learn and incorporate temporal patterns into deep tabular representations.

Result: The proposed training protocol significantly improves generalization across various methods. The temporal embedding method effectively captures crucial periodic and trend information. Combined, they provide a more effective and robust framework for learning from temporal tabular data.

Conclusion: The paper addresses key limitations of deep tabular models in temporal settings by improving training protocols and introducing temporal embeddings, offering a comprehensive solution for handling temporal distribution shifts in tabular data.

Abstract: Deep tabular models have demonstrated remarkable success on i.i.d. data, excelling in a variety of structured data tasks. However, their performance often deteriorates under temporal distribution shifts, where trends and periodic patterns are present in the evolving data distribution over time. In this paper, we explore the underlying reasons for this failure in capturing temporal dependencies. We begin by investigating the training protocol, revealing a key issue in how model selection performs. While existing approaches use temporal ordering for splitting validation set, we show that even a random split can significantly improve model performance. By minimizing the time lag between training data and test time, while reducing the bias in validation, our proposed training protocol significantly improves generalization across various methods. Furthermore, we analyze how temporal data affects deep tabular representations, uncovering that these models often fail to capture crucial periodic and trend information. To address this gap, we introduce a plug-and-play temporal embedding method based on Fourier series expansion to learn and incorporate temporal patterns, offering an adaptive approach to handle temporal shifts. Our experiments demonstrate that this temporal embedding, combined with the improved training protocol, provides a more effective and robust framework for learning from temporal tabular data.

Method: AMPED uses adaptive multi-objective projection with gradient-surgery to balance exploration and diversity gradients during pretraining, and a skill selector that exploits learned diversity by choosing appropriate skills for downstream tasks during fine-tuning.

Result: The approach surpasses skill-based RL baselines across various benchmarks. Ablation studies show each component contributes to performance. Theoretical and empirical evidence demonstrates that greater skill diversity with greedy skill selector reduces fine-tuning sample complexity.

Conclusion: Explicitly harmonizing exploration and diversity is crucial for robust and generalizable skill learning, and AMPED effectively achieves this balance while enabling better adaptation to downstream tasks.

Abstract: Skill-based reinforcement learning (SBRL) enables rapid adaptation in environments with sparse rewards by pretraining a skill-conditioned policy. Effective skill learning requires jointly maximizing both exploration and skill diversity. However, existing methods often face challenges in simultaneously optimizing for these two conflicting objectives. In this work, we propose a new method, Adaptive Multi-objective Projection for balancing Exploration and skill Diversification (AMPED), which explicitly addresses both: during pre-training, a gradient-surgery projection balances the exploration and diversity gradients, and during fine-tuning, a skill selector exploits the learned diversity by choosing skills suited to downstream tasks. Our approach achieves performance that surpasses SBRL baselines across various benchmarks. Through an extensive ablation study, we identify the role of each component and demonstrate that each element in AMPED is contributing to performance. We further provide theoretical and empirical evidence that, with a greedy skill selector, greater skill diversity reduces fine-tuning sample complexity. These results highlight the importance of explicitly harmonizing exploration and diversity and demonstrate the effectiveness of AMPED in enabling robust and generalizable skill learning. Project Page: https://geonwoo.me/amped/

Method: Proposes TBA (Trajectory Balance with Asynchrony) - an approach to asynchronous RL for LLMs that leverages the principled off-policy Trajectory Balance objective to efficiently learn from off-policy data generated by parallel actors.

Result: TBA offers speed (4× or more speedups) and performance boosts over strong baselines like Online DPO and Dr. GRPO on math, preference-tuning, and automated red-teaming tasks across models from Pythia 410M to Qwen 2.5 7B. It maintains high accuracy even as asynchrony grows, and its reward- and recency-prioritizing sampling enables further gains as data generation scales.

Conclusion: TBA provides an effective asynchronous RL approach for LLM post-training that combines principled off-policy learning with practical efficiency benefits, enabling better utilization of parallel data generation while maintaining strong performance.

Abstract: Reinforcement learning (RL) is a critical component of large language model (LLM) post-training. However, on-policy algorithms used for post-training are not naturally robust to a diversified content of experience replay buffers, which asynchronous off-policy actors can efficiently populate in parallel to training. We propose efficiently learning on such off-policy data via Trajectory Balance with Asynchrony (TBA), an approach to asynchronous RL for LLMs that leverages the principled off-policy TB objective. On math, preference-tuning, and automated red-teaming tasks, we post-train models ranging from Pythia 410M to Qwen 2.5 7B, finding TBA offers speed and performance boosts over strong baselines like Online DPO and Dr. GRPO. Beyond TBA’s performance benefits (high accuracy even as asynchrony grows) and speedups ($4\times$ or more), we show its reward- and recency-prioritizing sampling enable further gains as data generation is scaled. Our code is available at https://github.com/bbartoldson/TBA.

Method: Developed a novel skew-symmetric neural architecture that enforces stability while preserving mass, momentum, and energy conservation laws. Used discretization ensuring physical conservation and face-averaging filter for mass conservation in coarse-grained velocity fields.

Result: Unconstrained ML models suffered from numerical instabilities, while the skew-symmetric model remained stable across all tests (decaying turbulence and Kolmogorov flow for multiple coarse-graining factors). The model showed increased dissipation but still outperformed the Smagorinsky model in unseen scenarios.

Conclusion: Structure-preserving machine learning closures show potential for reliable long-time LES by balancing stability and physical consistency, though with a trade-off of increased dissipation.

Abstract: Machine learning-based closure models for LES have shown promise in capturing complex turbulence dynamics but often suffer from instabilities and physical inconsistencies. In this work, we develop a novel skew-symmetric neural architecture as closure model that enforces stability while preserving key physical conservation laws. Our approach leverages a discretization that ensures mass, momentum, and energy conservation, along with a face-averaging filter to maintain mass conservation in coarse-grained velocity fields. We compare our model against several conventional data-driven closures (including unconstrained convolutional neural networks), and the physics-based Smagorinsky model. Performance is evaluated on decaying turbulence and Kolmogorov flow for multiple coarse-graining factors. In these test cases we observe that unconstrained machine learning models suffer from numerical instabilities. In contrast, our skew-symmetric model remains stable across all tests, though at the cost of increased dissipation. Despite this trade-off, we demonstrate that our model still outperforms the Smagorinsky model in unseen scenarios. These findings highlight the potential of structure-preserving machine learning closures for reliable long-time LES.

Method: Introduces transition-prediction error as additional regret term and proposes Co-Learnability metric to capture how training on one task affects others, combining them in TRACED framework.

Result: TRACED produces curricula that improve zero-shot generalization over strong baselines across multiple benchmarks, with transition-prediction error driving rapid complexity ramp-up and Co-Learnability providing additional gains.

Conclusion: Refined regret approximation and explicit modeling of task relationships enable more sample-efficient curriculum design in UED for better generalization.

Abstract: Generalizing deep reinforcement learning agents to unseen environments remains a significant challenge. One promising solution is Unsupervised Environment Design (UED), a co-evolutionary framework in which a teacher adaptively generates tasks with high learning potential, while a student learns a robust policy from this evolving curriculum. Existing UED methods typically measure learning potential via regret, the gap between optimal and current performance, approximated solely by value-function loss. Building on these approaches, we introduce the transition-prediction error as an additional term in our regret approximation. To capture how training on one task affects performance on others, we further propose a lightweight metric called Co-Learnability. By combining these two measures, we present Transition-aware Regret Approximation with Co-learnability for Environment Design (TRACED). Empirical evaluations show that TRACED produces curricula that improve zero-shot generalization over strong baselines across multiple benchmarks. Ablation studies confirm that the transition-prediction error drives rapid complexity ramp-up and that Co-Learnability delivers additional gains when paired with the transition-prediction error. These results demonstrate how refined regret approximation and explicit modeling of task relationships can be leveraged for sample-efficient curriculum design in UED. Project Page: https://geonwoo.me/traced/

Method: Proposes an Eulerian-Lagrangian PINN architecture with domain-specific networks: Eulerian network for fluid dynamics and Lagrangian network for structural interfaces, coupled through physics-based constraints. Incorporates learnable B-spline activation functions with SiLU to capture both localized high-gradient features near interfaces and global flow patterns.

Result: On 2D cavity flow with moving solid structure, the proposed EL-L architecture improves accuracy by 24.1-91.4% across all metrics, reducing pressure errors from 12.9% to 2.39% compared to baseline unified PINNs.

Conclusion: Domain decomposition aligned with physical principles combined with locality-aware activation functions is essential for accurate FSI modeling within the PINN framework, enabling better handling of moving boundary conditions.

Abstract: Physics-informed neural networks (PINNs) have emerged as a promising approach for solving complex fluid dynamics problems, yet their application to fluid-structure interaction (FSI) problems with moving boundaries remains largely unexplored. This work addresses the critical challenge of modeling FSI systems with moving interfaces, where traditional unified PINN architectures struggle to capture the distinct physics governing fluid and structural domains simultaneously. We present an innovative Eulerian-Lagrangian PINN architecture that integrates immersed boundary method (IBM) principles to solve FSI problems with moving boundary conditions. Our approach fundamentally departs from conventional unified architectures by introducing domain-specific neural networks: an Eulerian network for fluid dynamics and a Lagrangian network for structural interfaces, coupled through physics-based constraints. Additionally, we incorporate learnable B-spline activation functions with SiLU to capture both localized high-gradient features near interfaces and global flow patterns. Empirical studies on a 2D cavity flow problem involving a moving solid structure show that while baseline unified PINNs achieve reasonable velocity predictions, they suffer from substantial pressure errors (12.9%) in structural regions. Our Eulerian-Lagrangian architecture with learnable activations (EL-L) achieves better performance across all metrics, improving accuracy by 24.1-91.4% and particularly reducing pressure errors from 12.9% to 2.39%. These results demonstrate that domain decomposition aligned with physical principles, combined with locality-aware activation functions, is essential for accurate FSI modeling within the PINN framework.

Method: Proposes Generalized Lipschitz Group Equivariant Neural Networks (GLGENN) based on geometric (Clifford) algebras with a weight-sharing parametrization technique that respects geometric algebra structures, making the architecture parameter-light.

Result: GLGENN outperforms or matches competitors on benchmarking equivariant tasks (equivariant function estimation, convex hull experiments) while using significantly fewer optimizable parameters and showing less tendency to overfit.

Conclusion: GLGENN provides an effective, parameter-efficient architecture for equivariant learning that leverages geometric algebra principles to achieve strong performance across various transformation groups and bilinear forms.

Abstract: We propose, implement, and compare with competitors a new architecture of equivariant neural networks based on geometric (Clifford) algebras: Generalized Lipschitz Group Equivariant Neural Networks (GLGENN). These networks are equivariant to all pseudo-orthogonal transformations, including rotations and reflections, of a vector space with any non-degenerate or degenerate symmetric bilinear form. We propose a weight-sharing parametrization technique that takes into account the fundamental structures and operations of geometric algebras. Due to this technique, GLGENN architecture is parameter-light and has less tendency to overfitting than baseline equivariant models. GLGENN outperforms or matches competitors on several benchmarking equivariant tasks, including estimation of an equivariant function and a convex hull experiment, while using significantly fewer optimizable parameters.

Method: CoreSPECT framework applies clustering algorithms to strategically selected regions, then extends partial partitions to complete partitions using a novel neighborhood graph-based multi-layer propagation procedure.

Result: CoreSPECT improves K-Means NMI by 20% on average (making it competitive with state-of-the-art manifold methods while being much faster) and boosts HDBSCAN NMI by over 100% on average, with even higher ARI improvements.

Conclusion: The density-geometry correlation in real-world data provides a powerful foundation for enhancing generic clustering algorithms, making CoreSPECT an effective framework that improves performance without requiring true cluster counts or extensive hyperparameter tuning.

Abstract: In this paper, we provide a novel perspective on the underlying structure of real-world data with ground-truth clusters via characterization of an abundantly observed yet often overlooked density-geometry correlation, that manifests itself as a multi-layered manifold structure. We leverage this correlation to design CoreSPECT (Core Space Projection based Enhancement of Clustering Techniques), a general framework that improves the performance of generic clustering algorithms. Our framework boosts the performance of clustering algorithms by applying them to strategically selected regions, then extending the partial partition to a complete partition for the dataset using a novel neighborhood graph based multi-layer propagation procedure. We provide initial theoretical support of the functionality of our framework under the assumption of our model, and then provide large-scale real-world experiments on 19 datasets that include standard image datasets as well as genomics datasets. We observe two notable improvements. First, CoreSPECT improves the NMI of K-Means by 20% on average, making it competitive to (and in some cases surpassing) the state-of-the-art manifold-based clustering algorithms, while being orders of magnitude faster. Secondly, our framework boosts the NMI of HDBSCAN by more than 100% on average, making it competitive to the state-of-the-art in several cases without requiring the true number of clusters and hyper-parameter tuning. The overall ARI improvements are higher.

Method: Proposes a multi-level conditional diffusion model that simultaneously generates representations at different levels (sequence and structure) to model hierarchical relations. Also introduces Protein-MMD, a new evaluation metric that captures distributional and functional similarities while ensuring conditional consistency.

Result: The framework demonstrates efficacy on benchmark datasets for conditional protein generation tasks, showing improved performance through better modeling of hierarchical protein structures.

Conclusion: The multi-level conditional diffusion model with integrated sequence-structure information and the new Protein-MMD evaluation metric provide an effective approach for reliable conditional protein generation and assessment.

Abstract: Generating novel and functional protein sequences is critical to a wide range of applications in biology. Recent advancements in conditional diffusion models have shown impressive empirical performance in protein generation tasks. However, reliable generations of protein remain an open research question in de novo protein design, especially when it comes to conditional diffusion models. Considering the biological function of a protein is determined by multi-level structures, we propose a novel multi-level conditional diffusion model that integrates both sequence-based and structure-based information for efficient end-to-end protein design guided by specified functions. By generating representations at different levels simultaneously, our framework can effectively model the inherent hierarchical relations between different levels, resulting in an informative and discriminative representation of the generated protein. We also propose a Protein-MMD, a new reliable evaluation metric, to evaluate the quality of generated protein with conditional diffusion models. Our new metric is able to capture both distributional and functional similarities between real and generated protein sequences while ensuring conditional consistency. We experiment with the benchmark datasets, and the results on conditional protein generation tasks demonstrate the efficacy of the proposed generation framework and evaluation metric.

Method: Proposes an “un-pruning” algorithm that approximates the pruned topology driven solely by retained data. The method integrates with existing unlearning algorithms, works with both structured and unstructured sparse models, and has theoretical error bounds.

Result: Shows that MIA accuracy is unreliable for evaluating unlearning in sparse models. Introduces new evaluation metrics and demonstrates through extensive experiments that un-pruning works effectively with various pruning methods and unlearning algorithms.

Conclusion: Un-pruning successfully addresses the gap in machine unlearning for sparse models by eliminating deleted data’s influence on pruned topologies, with theoretical guarantees and practical effectiveness across different sparse model types.

Abstract: Machine unlearning aims to efficiently eliminate the memory about deleted data from trained models and address the right to be forgotten. Despite the success of existing unlearning algorithms, unlearning in sparse models has not yet been well studied. In this paper, we empirically find that the deleted data has an impact on the pruned topology in a sparse model. Motivated by the observation and the right to be forgotten, we define a new terminology ``un-pruning" to eliminate the impact of deleted data on model pruning. Then we propose an un-pruning algorithm to approximate the pruned topology driven by retained data. We remark that any existing unlearning algorithm can be integrated with the proposed un-pruning workflow and the error of un-pruning is upper-bounded in theory. Also, our un-pruning algorithm can be applied to both structured sparse models and unstructured sparse models. In the experiment, we further find that Membership Inference Attack (MIA) accuracy is unreliable for assessing whether a model has forgotten deleted data, as a small change in the amount of deleted data can produce arbitrary MIA results. Accordingly, we devise new performance metrics for sparse models to evaluate the success of un-pruning. Lastly, we conduct extensive experiments to verify the efficacy of un-pruning with various pruning methods and unlearning algorithms. Our code is released at https://github.com/NKUShaw/SparseModels .

Method: Updated Predictability-Computability-Stability (PCS) framework for veridical data science, streamlined for practitioners with guided generative AI use, demonstrated through running examples and case studies.

Result: The PCS framework provides a principled approach to manage uncertainty throughout the DSLC, with case studies showing how data cleaning judgment calls create downstream prediction uncertainty.

Conclusion: The PCS framework offers a systematic solution to reproducibility challenges in data science by addressing uncertainty throughout the lifecycle, making AI findings more veridical and reliable.

Abstract: Data science is a pillar of artificial intelligence (AI), which is transforming nearly every domain of human activity, from the social and physical sciences to engineering and medicine. While data-driven findings in AI offer unprecedented power to extract insights and guide decision-making, many are difficult or impossible to replicate. A key reason for this challenge is the uncertainty introduced by the many choices made throughout the data science life cycle (DSLC). Traditional statistical frameworks often fail to account for this uncertainty. The Predictability-Computability-Stability (PCS) framework for veridical (truthful) data science offers a principled approach to addressing this challenge throughout the DSLC. This paper presents an updated and streamlined PCS workflow, tailored for practitioners and enhanced with guided use of generative AI. We include a running example to display the PCS framework in action, and conduct a related case study which showcases the uncertainty in downstream predictions caused by judgment calls in the data cleaning stage.

Method: A model-based algorithm using maximum likelihood estimation (MLE) with general function approximation to estimate state marginal density (rather than system dynamics), combined with a randomized measurement schedule that adapts observation frequency to problem complexity.

Result: Established instance-dependent regret bounds that scale with total reward variance and measurement resolution, with regret becoming independent of specific measurement strategy when observation frequency adapts appropriately. The randomized measurement schedule improves sample efficiency without increasing measurement cost.

Conclusion: The work demonstrates a new direction for CTRL algorithms that automatically adjust learning behavior based on environmental difficulty, showing that state marginal density estimation with adaptive measurement strategies can achieve instance-dependent performance guarantees.

Abstract: Continuous-time reinforcement learning (CTRL) provides a natural framework for sequential decision-making in dynamic environments where interactions evolve continuously over time. While CTRL has shown growing empirical success, its ability to adapt to varying levels of problem difficulty remains poorly understood. In this work, we investigate the instance-dependent behavior of CTRL and introduce a simple, model-based algorithm built on maximum likelihood estimation (MLE) with a general function approximator. Unlike existing approaches that estimate system dynamics directly, our method estimates the state marginal density to guide learning. We establish instance-dependent performance guarantees by deriving a regret bound that scales with the total reward variance and measurement resolution. Notably, the regret becomes independent of the specific measurement strategy when the observation frequency adapts appropriately to the problem’s complexity. To further improve performance, our algorithm incorporates a randomized measurement schedule that enhances sample efficiency without increasing measurement cost. These results highlight a new direction for designing CTRL algorithms that automatically adjust their learning behavior based on the underlying difficulty of the environment.

Method: Uses process mining to learn actual control flow from execution traces, enabling run-time monitoring through online conformance checking. Combines this with unsupervised machine learning for anomaly localization to link deviations to critical system components.

Result: Tested on ERTMS/ETCS L2 RBC/RBC Handover scenario, showing capability to detect and localize anomalies with high accuracy, efficiency, and explainability.

Conclusion: Process mining combined with unsupervised ML provides an effective approach for enhancing railway system resilience through run-time anomaly detection and localization, addressing limitations of design-time verification alone.

Abstract: Ensuring the resilience of computer-based railways is increasingly crucial to account for uncertainties and changes due to the growing complexity and criticality of those systems. Although their software relies on strict verification and validation processes following well-established best-practices and certification standards, anomalies can still occur at run-time due to residual faults, system and environmental modifications that were unknown at design-time, or other emergent cyber-threat scenarios. This paper explores run-time control-flow anomaly detection using process mining to enhance the resilience of ERTMS/ETCS L2 (European Rail Traffic Management System / European Train Control System Level 2). Process mining allows learning the actual control flow of the system from its execution traces, thus enabling run-time monitoring through online conformance checking. In addition, anomaly localization is performed through unsupervised machine learning to link relevant deviations to critical system components. We test our approach on a reference ERTMS/ETCS L2 scenario, namely the RBC/RBC Handover, to show its capability to detect and localize anomalies with high accuracy, efficiency, and explainability.

Method: Critical examination of common datasets (carbon-24, perov-5, MP-20) and metrics, analysis of structural uniqueness and polymorph distribution, and introduction of revised datasets with duplicates removed, new splits, and new evaluation metrics (METRe and cRMSE).

Result: Found that carbon-24 contains only ≈40% unique structures, identified problematic random splitting in polymorph-rich datasets, and demonstrated issues with current match rate metrics. Provided revised datasets and proposed new evaluation approaches.

Conclusion: Current materials datasets and benchmarks have significant flaws that can mislead model evaluation. The paper provides concrete fixes including deduplicated datasets, improved splitting methods, and new metrics to enable more meaningful benchmarking of crystal structure prediction models.

Abstract: Generative models for materials, especially inorganic crystals, hold potential to transform the theoretical prediction of novel compounds and structures. Advancement in this field depends on robust benchmarks and minimal, information-rich datasets that enable meaningful model evaluation. This paper critically examines common datasets and reported metrics for a crystal structure prediction task$\unicode{x2014}$generating the most likely structures given the chemical composition of a material. We focus on three key issues: First, materials datasets should contain unique crystal structures; for example, we show that the widely-utilized carbon-24 dataset only contains $\approx$40% unique structures. Second, materials datasets should not be split randomly if polymorphs of many different compositions are numerous, which we find to be the case for the perov-5 and MP-20 datasets. Third, benchmarks can mislead if used uncritically, e.g., reporting a match rate metric without considering the structural variety exhibited by identical building blocks. To address these oft-overlooked issues, we introduce several fixes. We provide revised versions of the carbon-24 dataset: one with duplicates removed, one deduplicated and split by number of atoms $N$, one with enantiomorphs, and two containing only identical structures but with different unit cells. We also propose new splits for datasets with polymorphs, ensuring that polymorphs are grouped within each split subset, setting a more sensible standard for benchmarking model performance. Finally, we present METRe and cRMSE, new model evaluation metrics that can correct existing issues with the match rate metric.

Method: The authors develop a novel attack framework with two specific strategies: CascadeOFA for attacking CascadeUCB1 algorithm and PBMOFA for attacking PBM-UCB algorithm. Both strategies require only O(log T) manipulations.

Result: Theoretical guarantees show both attack strategies can successfully promote target items to top-K recommendations for T - o(T) rounds while inducing linear regret in the learning algorithms. Empirical results on real-world data validate the theoretical findings.

Conclusion: OLTR algorithms are vulnerable to efficient coordinated attacks that require minimal manipulations (O(log T)), highlighting significant security concerns for real-world applications in search and recommendation systems.

Abstract: Online learning to rank (OLTR) plays a critical role in information retrieval and machine learning systems, with a wide range of applications in search engines and content recommenders. However, despite their extensive adoption, the susceptibility of OLTR algorithms to coordinated adversarial attacks remains poorly understood. In this work, we present a novel framework for attacking some of the widely used OLTR algorithms. Our framework is designed to promote a set of target items so that they appear in the list of top-K recommendations for T - o(T) rounds, while simultaneously inducing linear regret in the learning algorithm. We propose two novel attack strategies: CascadeOFA for CascadeUCB1 and PBMOFA for PBM-UCB . We provide theoretical guarantees showing that both strategies require only O(log T) manipulations to succeed. Additionally, we supplement our theoretical analysis with empirical results on real-world data.

Method: Introduce Reversible Deep Equilibrium Models (RevDEQs) that allow for exact gradient calculation without regularization and with fewer function evaluations than DEQs.

Result: RevDEQs significantly improve performance on language modeling and image classification tasks compared to both implicit (DEQs) and explicit models.

Conclusion: RevDEQs solve key limitations of DEQs by enabling exact gradients, reducing computational cost, and improving performance across multiple domains.

Abstract: Deep Equilibrium Models (DEQs) are an interesting class of implicit model where the model output is implicitly defined as the fixed point of a learned function. These models have been shown to outperform explicit (fixed-depth) models in large-scale tasks by trading many deep layers for a single layer that is iterated many times. However, gradient calculation through DEQs is approximate. This often leads to unstable training dynamics and requires regularisation or many function evaluations to fix. Here, we introduce Reversible Deep Equilibrium Models (RevDEQs) that allow for exact gradient calculation, no regularisation and far fewer function evaluations than DEQs. We show that RevDEQs significantly improve performance on language modelling and image classification tasks against comparable implicit and explicit models.

Method: 1) Show classical risk measure theory is incompatible with continual learning, 2) Extend risk measure theory by introducing new class of ergodic risk measures compatible with continual learning, 3) Provide case study with empirical results.

Result: Developed theoretical foundation for risk-aware continual RL with ergodic risk measures that work in continual settings, supported by empirical evidence showing intuitive appeal.

Conclusion: Ergodic risk measures provide a theoretically sound and practically useful framework for risk-aware decision-making in continual RL, addressing limitations of classical risk measures in continual learning contexts.

Abstract: Continual reinforcement learning (continual RL) seeks to formalize the notions of lifelong learning and endless adaptation in RL. In particular, the aim of continual RL is to develop RL agents that can maintain a careful balance between retaining useful information and adapting to new situations. To date, continual RL has been explored almost exclusively through the lens of risk-neutral decision-making, in which the agent aims to optimize the expected long-run performance. In this work, we present the first formal theoretical treatment of continual RL through the lens of risk-aware decision-making, in which the behaviour of the agent is directed towards optimizing a measure of long-run performance beyond the mean. In particular, we show that the classical theory of risk measures, widely used as a theoretical foundation in non-continual risk-aware RL, is, in its current form, incompatible with continual learning. Then, building on this insight, we extend risk measure theory into the continual setting by introducing a new class of ergodic risk measures that are compatible with continual learning. Finally, we provide a case study of risk-aware continual learning, along with empirical results, which show the intuitive appeal of ergodic risk measures in continual settings.

Method: Proposes Diffusion Router (DR), a unified diffusion-based framework that models all central↔non-central translations with a single noise predictor conditioned on source and target domain labels. Enables indirect non-central translations by routing through the central domain. Introduces scalable learning strategy with variational-bound objective and efficient Tweedie refinement for direct non-central mappings.

Result: DR achieves state-of-the-art results on three large-scale UMDT benchmarks for both indirect and direct translations, while lowering sampling cost and unlocking novel tasks like sketch↔segmentation.

Conclusion: Diffusion Router establishes a scalable and versatile framework for universal translation across multiple domains, addressing limitations of existing MDT approaches with minimal training requirements.

Abstract: Multi-domain translation (MDT) aims to learn translations between multiple domains, yet existing approaches either require fully aligned tuples or can only handle domain pairs seen in training, limiting their practicality and excluding many cross-domain mappings. We introduce universal MDT (UMDT), a generalization of MDT that seeks to translate between any pair of $K$ domains using only $K-1$ paired datasets with a central domain. To tackle this problem, we propose Diffusion Router (DR), a unified diffusion-based framework that models all central$\leftrightarrow$non-central translations with a single noise predictor conditioned on the source and target domain labels. DR enables indirect non-central translations by routing through the central domain. We further introduce a novel scalable learning strategy with a variational-bound objective and an efficient Tweedie refinement procedure to support direct non-central mappings. Through evaluation on three large-scale UMDT benchmarks, DR achieves state-of-the-art results for both indirect and direct translations, while lowering sampling cost and unlocking novel tasks such as sketch$\leftrightarrow$segmentation. These results establish DR as a scalable and versatile framework for universal translation across multiple domains.

Method: Introduces a novel algorithm for detecting stable and unstable manifolds of periodic points in piecewise-linear RNNs (PLRNNs) using ReLU activation functions.

Result: Algorithm can trace basin boundaries to characterize multistability, find homoclinic points to establish chaos existence, and provide insights into empirical neural dynamics from cortical recordings.

Conclusion: The manifold detection algorithm enables systematic analysis of PLRNN dynamics, offering tools to understand multistability and chaos in trained networks, with applications to neural data analysis.

Abstract: Recurrent Neural Networks (RNNs) have found widespread applications in machine learning for time series prediction and dynamical systems reconstruction, and experienced a recent renaissance with improved training algorithms and architectural designs. Understanding why and how trained RNNs produce their behavior is important for scientific and medical applications, and explainable AI more generally. An RNN’s dynamical repertoire depends on the topological and geometrical properties of its state space. Stable and unstable manifolds of periodic points play a particularly important role: They dissect a dynamical system’s state space into different basins of attraction, and their intersections lead to chaotic dynamics with fractal geometry. Here we introduce a novel algorithm for detecting these manifolds, with a focus on piecewise-linear RNNs (PLRNNs) employing rectified linear units (ReLUs) as their activation function. We demonstrate how the algorithm can be used to trace the boundaries between different basins of attraction, and hence to characterize multistability, a computationally important property. We further show its utility in finding so-called homoclinic points, the intersections between stable and unstable manifolds, and thus establish the existence of chaos in PLRNNs. Finally we show for an empirical example, electrophysiological recordings from a cortical neuron, how insights into the underlying dynamics could be gained through our method.

Method: MCNO learns kernel functions as tensors over sampled input-output pairs, uses Monte Carlo integration with single uniform sampling from discretized grids, and includes interpolation for mapping between arbitrary grids. This avoids repeated sampling during training and fixed global basis functions.

Result: MCNO achieves competitive accuracy on standard 1D PDE benchmarks with efficient computation. Theoretical analysis proves bounded bias and variance for the Monte Carlo estimator under mild regularity assumptions, with results holding in any spatial dimension.

Conclusion: MCNO provides a theoretically supported alternative to spectral methods (FNO) and graph-based Monte Carlo approaches (GNO), demonstrating that Monte Carlo integration can be effectively incorporated into neural operator frameworks for continuous-domain PDEs with potential extension to higher dimensions.

Abstract: The Monte Carlo-type Neural Operator (MCNO) introduces a framework for learning solution operators of one-dimensional partial differential equations (PDEs) by directly learning the kernel function and approximating the associated integral operator using a Monte Carlo-type approach. Unlike Fourier Neural Operators (FNOs), which rely on spectral representations and assume translation-invariant kernels, MCNO makes no such assumptions. The kernel is represented as a learnable tensor over sampled input-output pairs, and sampling is performed once, uniformly at random from a discretized grid. This design enables generalization across multiple grid resolutions without relying on fixed global basis functions or repeated sampling during training, while an interpolation step maps between arbitrary input and output grids to further enhance flexibility. Experiments on standard 1D PDE benchmarks show that MCNO achieves competitive accuracy with efficient computational cost. We also provide a theoretical analysis proving that the Monte Carlo estimator yields a bounded bias and variance under mild regularity assumptions. This result holds in any spatial dimension, suggesting that MCNO may extend naturally beyond one-dimensional problems. More broadly, this work explores how Monte Carlo-type integration can be incorporated into neural operator frameworks for continuous-domain PDEs, providing a theoretically supported alternative to spectral methods (such as FNO) and to graph-based Monte Carlo approaches (such as the Graph Kernel Neural Operator, GNO).

Method: Two complementary adaptation strategies: 1) Online distributional adaptation - learns lightweight adaptation vectors to bias model output distribution for rapid alignment with environment response formats. 2) Deployment-time dynamics grounding - uses persona-driven exploration to systematically probe and learn environment’s causal dynamics before task execution, creating a nonparametric world model.

Result: Both strategies improve performance across diverse benchmarks (function calling, web navigation) with minimal computational cost. Dynamics grounding is particularly effective in complex environments with unpredictable dynamics. On WebArena multi-site split, success rate increased from 2% to 23%.

Conclusion: The proposed adaptation strategies provide robust paths toward more generalizable LLM-based agents by addressing both syntactic and semantic misunderstandings. Dynamics grounding shows strong performance in complex environments, demonstrating practical approaches for agent deployment in novel settings.

Abstract: Large language model (LLM)-based agents struggle to generalize to novel and complex environments, such as unseen websites or new sets of functions, due to a fundamental mismatch between their pre-training and test-time conditions. This challenge stems from two distinct failure modes: a syntactic misunderstanding of environment-specific components like observation formats, and a semantic misunderstanding of state-transition dynamics, which are only revealed at test time. To address these issues, we propose two distinct and complementary strategies for adapting LLM agents by leveraging environment-specific information available during deployment. First, an online distributional adaptation method parameterizes environmental nuances by learning a lightweight adaptation vector that biases the model’s output distribution, enabling rapid alignment with an environment response format. Second, a deployment-time dynamics grounding method employs a persona-driven exploration phase to systematically probe and learn the environment’s causal dynamics before task execution, equipping the agent with a nonparametric world model. We evaluate these strategies across diverse agentic benchmarks, including function calling and web navigation. Our empirical results show the effectiveness of both strategies across all benchmarks with minimal computational cost. We find that dynamics grounding is particularly effective in complex environments where unpredictable dynamics pose a major obstacle, demonstrating a robust path toward more generalizable and capable LLM-based agents. For example, on the WebArena multi-site split, this method increases the agent’s success rate from 2% to 23%.

Method: Propose Covariance Scattering Transforms (CSTs) - deep untrained networks that sequentially apply filters (covariance wavelets) localized in the covariance spectrum to input data. Use nonlinearities to produce hierarchical representations. Include pruning mechanism for computational/memory efficiency. Prove CSTs are less sensitive to finite-sample covariance estimation errors compared to PCA.

Result: CSTs produce stable representations in low-data settings (like VNNs but without training). Experiments on age prediction from cortical thickness measurements across 4 neurodegenerative disease datasets show CSTs achieve comparable or better predictions than more complex learning models.

Conclusion: CSTs combine benefits of PCA (unsupervised) and VNNs (stability, expressiveness) without requiring training, making them effective for covariance-based analysis in low-sample regimes where PCA fails and VNNs require labeled data.

Abstract: Machine learning and data processing techniques relying on covariance information are widespread as they identify meaningful patterns in unsupervised and unlabeled settings. As a prominent example, Principal Component Analysis (PCA) projects data points onto the eigenvectors of their covariance matrix, capturing the directions of maximum variance. This mapping, however, falls short in two directions: it fails to capture information in low-variance directions, relevant when, e.g., the data contains high-variance noise; and it provides unstable results in low-sample regimes, especially when covariance eigenvalues are close. CoVariance Neural Networks (VNNs), i.e., graph neural networks using the covariance matrix as a graph, show improved stability to estimation errors and learn more expressive functions in the covariance spectrum than PCA, but require training and operate in a labeled setup. To get the benefits of both worlds, we propose Covariance Scattering Transforms (CSTs), deep untrained networks that sequentially apply filters localized in the covariance spectrum to the input data and produce expressive hierarchical representations via nonlinearities. We define the filters as covariance wavelets that capture specific and detailed covariance spectral patterns. We improve CSTs’ computational and memory efficiency via a pruning mechanism, and we prove that their error due to finite-sample covariance estimations is less sensitive to close covariance eigenvalues compared to PCA, improving their stability. Our experiments on age prediction from cortical thickness measurements on 4 datasets collecting patients with neurodegenerative diseases show that CSTs produce stable representations in low-data settings, as VNNs but without any training, and lead to comparable or better predictions w.r.t. more complex learning models.

Method: ExPairT-LLM uses two new types of queries to LLM oracles: pairwise membership (comparing two programs on a single input) and pairwise equivalence (comparing two programs on multiple inputs). It identifies the correct program through a tournament structure that is robust to some LLM mistakes.

Result: On four popular code datasets, ExPairT-LLM’s pass@1 outperforms state-of-the-art code selection algorithms by average +13.0% and up to +27.1%. It also improves pass@1 of LLMs performing complex reasoning by +24.0%.

Conclusion: ExPairT-LLM provides an effective code selection approach using simpler pairwise queries that are more reliable for LLMs, enabling robust program identification through tournament-style comparisons.

Abstract: Despite recent advances in LLMs, the task of code generation is still challenging. To cope, code selection algorithms select the best program from multiple programs generated by an LLM. However, existing algorithms can fail to identify the correct program, either because they can misidentify nonequivalent programs or because they rely on an LLM and assume it always correctly determines the output for every input. We present ExPairT-LLM, an exact learning algorithm for code selection that selects a program by posing to an LLM oracle two new types of queries: pairwise membership and pairwise equivalence. These queries are simpler for LLMs and enable ExPairT-LLM to identify the correct program through a tournament, which is robust to some LLM mistakes. We evaluate ExPairT-LLM on four popular code datasets. Its pass@1 (success rate) outperforms the state-of-the-art code selection algorithm on average by +13.0% and up to +27.1%. It also improves the pass@1 of LLMs performing complex reasoning by +24.0%.

Method: WavefrontDiffusion uses dynamic decoding that expands a wavefront of active tokens outward from finalized positions, adapting to natural semantic flow while maintaining computational efficiency equal to block-based methods.

Result: Achieves state-of-the-art performance across four benchmarks in reasoning and code generation, producing outputs with higher semantic fidelity compared to existing denoising approaches.

Conclusion: Adaptive scheduling through wavefront expansion enables more coherent and efficient generation in diffusion language models, demonstrating the value of dynamic decoding strategies over rigid approaches.

Abstract: Diffusion Language Models (DLMs) have shown strong potential for text generation and are becoming a competitive alternative to autoregressive models. The denoising strategy plays an important role in determining the quality of their outputs. Mainstream denoising strategies include Standard Diffusion and BlockDiffusion. Standard Diffusion performs global denoising without restricting the update range, often finalizing incomplete context and causing premature end-of-sequence predictions. BlockDiffusion updates fixed-size blocks in a preset order, but its rigid structure can break apart coherent semantic units and disrupt reasoning. We present WavefrontDiffusion, a dynamic decoding approach that expands a wavefront of active tokens outward from finalized positions. This adaptive process follows the natural flow of semantic structure while keeping computational cost equal to block-based methods. Across four benchmarks in reasoning and code generation, WavefrontDiffusion achieves state-of-the-art performance while producing outputs with higher semantic fidelity, showing the value of adaptive scheduling for more coherent and efficient generation.

Method: Hierarchical deep research agent integrating local retrieval-augmented generation with LLM reasoners, enhanced by Deep Tree of Research mechanism for adaptive expansion/pruning of research branches.

Result: Outperforms commercial systems (ChatGPT variants) in quality across 27 nanomaterials/device topics, validated by LLM-as-judge rubric and human expert dry-lab simulations (DFT).

Conclusion: The DR agent provides high-quality, actionable research reports at substantially lower cost than commercial systems while enabling on-prem integration with local data and tools.

Abstract: We present a long-horizon, hierarchical deep research (DR) agent designed for complex materials and device discovery problems that exceed the scope of existing Machine Learning (ML) surrogates and closed-source commercial agents. Our framework instantiates a locally deployable DR instance that integrates local retrieval-augmented generation with large language model reasoners, enhanced by a Deep Tree of Research (DToR) mechanism that adaptively expands and prunes research branches to maximize coverage, depth, and coherence. We systematically evaluate across 27 nanomaterials/device topics using a large language model (LLM)-as-judge rubric with five web-enabled state-of-the-art models as jurors. In addition, we conduct dry-lab validations on five representative tasks, where human experts use domain simulations (e.g., density functional theory, DFT) to verify whether DR-agent proposals are actionable. Results show that our DR agent produces reports with quality comparable to–and often exceeding–those of commercial systems (ChatGPT-5-thinking/o3/o4-mini-high Deep Research) at a substantially lower cost, while enabling on-prem integration with local data and tools.

Method: RARO uses relativistic adversarial optimization with a policy (generator) and relativistic critic (discriminator) that learn jointly via RL, with key stabilization techniques for robust learning.

Result: RARO significantly outperforms verifier-free baselines on Countdown, DeepMath, and Poetry Writing tasks, showing robust scaling trends similar to RL on verifiable tasks.

Conclusion: The method effectively elicits strong reasoning performance from expert demonstrations alone, enabling robust reasoning learning when task-specific verifiers are unavailable.

Abstract: Training Large Language Models (LLMs) to reason often relies on Reinforcement Learning (RL) with task-specific verifiers. However, many real-world reasoning-intensive tasks lack verifiers, despite offering abundant expert demonstrations that remain under-utilized for reasoning-focused training. We introduce RARO (Relativistic Adversarial Reasoning Optimization) that learns strong reasoning capabilities from only expert demonstrations via Inverse Reinforcement Learning. Our method sets up an adversarial interaction between a policy (generator) and a relativistic critic (discriminator): the policy learns to mimic expert answers, while the critic learns to compare and distinguish between policy and expert answers. Our method trains both the policy and the critic jointly and continuously via RL, and we identify the key stabilization techniques required for robust learning. Empirically, RARO significantly outperforms strong verifier-free baselines on all of our evaluation tasks – Countdown, DeepMath, and Poetry Writing – and enjoys the same robust scaling trends as RL on verifiable tasks. These results demonstrate that our method effectively elicits strong reasoning performance from expert demonstrations alone, enabling robust reasoning learning even when task-specific verifiers are unavailable.

Method: The authors minimize reverse KL divergence between diffusion policy and optimal policy using a tractable upper bound, apply policy gradient theorem to derive a modified surrogate objective, and create diffusion-based variants of existing RL algorithms (DiffSAC, DiffPPO, DiffWPO) with minimal implementation changes.

Result: On standard continuous control benchmarks, DiffSAC, DiffPPO, and DiffWPO achieve better returns and higher sample efficiency compared to their base algorithms SAC and PPO.

Conclusion: The diffusion model reinterpretation of MaxEntRL provides a principled way to incorporate diffusion dynamics into RL algorithms, leading to improved performance with minimal implementation overhead.

Abstract: Diffusion models have achieved remarkable success in data-driven learning and in sampling from complex, unnormalized target distributions. Building on this progress, we reinterpret Maximum Entropy Reinforcement Learning (MaxEntRL) as a diffusion model-based sampling problem. We tackle this problem by minimizing the reverse Kullback-Leibler (KL) divergence between the diffusion policy and the optimal policy distribution using a tractable upper bound. By applying the policy gradient theorem to this objective, we derive a modified surrogate objective for MaxEntRL that incorporates diffusion dynamics in a principled way. This leads to simple diffusion-based variants of Soft Actor-Critic (SAC), Proximal Policy Optimization (PPO) and Wasserstein Policy Optimization (WPO), termed DiffSAC, DiffPPO and DiffWPO. All of these methods require only minor implementation changes to their base algorithm. We find that on standard continuous control benchmarks, DiffSAC, DiffPPO and DiffWPO achieve better returns and higher sample efficiency than SAC and PPO.

Method: Proposed decentralized multi-task fair federated learning (DMTFL) algorithm that learns individual caching models at each base station. The models cache and prefetch each VR user’s field of view based on tailored strategies, optimized to perform well under any target distribution with theoretical guarantees via Rademacher complexity and PAC bounds.

Result: Simulations using realistic VR head-tracking dataset demonstrate superiority of DMTFL algorithm compared to baseline algorithms.

Conclusion: DMTFL provides an effective solution for personalized VR content delivery in wireless networks by addressing statistical heterogeneity and fairness issues in federated learning, enabling seamless wireless VR experiences with theoretical performance guarantees.

Abstract: Wireless connectivity promises to unshackle virtual reality (VR) experiences, allowing users to engage from anywhere, anytime. However, delivering seamless, high-quality, real-time VR video wirelessly is challenging due to the stringent quality of experience requirements, low latency constraints, and limited VR device capabilities. This paper addresses these challenges by introducing a novel decentralized multi task fair federated learning (DMTFL) based caching that caches and prefetches each VR user’s field of view (FOV) at base stations (BSs) based on the caching strategies tailored to each BS. In federated learning (FL) in its naive form, often biases toward certain users, and a single global model fails to capture the statistical heterogeneity across users and BSs. In contrast, the proposed DMTFL algorithm personalizes content delivery by learning individual caching models at each BS. These models are further optimized to perform well under any target distribution, while providing theoretical guarantees via Rademacher complexity and a probably approximately correct (PAC) bound on the loss. Using a realistic VR head-tracking dataset, our simulations demonstrate the superiority of our proposed DMTFL algorithm compared to baseline algorithms.

Method: Develops adaptive DFL (aDFL) that adjusts learning rates: smaller rates for suspicious clients, larger rates for normal clients, in a fully adaptive way without prior knowledge.

Result: Theoretical convergence analysis guarantees oracle property; extensive experiments show superior performance compared to existing methods.

Conclusion: aDFL provides a practical, adaptive solution for robust DFL that handles abnormal clients effectively without restrictive assumptions.

Abstract: In decentralized federated learning (DFL), the presence of abnormal clients, often caused by noisy or poisoned data, can significantly disrupt the learning process and degrade the overall robustness of the model. Previous methods on this issue often require a sufficiently large number of normal neighboring clients or prior knowledge of reliable clients, which reduces the practical applicability of DFL. To address these limitations, we develop here a novel adaptive DFL (aDFL) approach for robust estimation. The key idea is to adaptively adjust the learning rates of clients. By assigning smaller rates to suspicious clients and larger rates to normal clients, aDFL mitigates the negative impact of abnormal clients on the global model in a fully adaptive way. Our theory does not put any stringent conditions on neighboring nodes and requires no prior knowledge. A rigorous convergence analysis is provided to guarantee the oracle property of aDFL. Extensive numerical experiments demonstrate the superior performance of the aDFL method.

Method: 1) Proves lossless equivalence between real and widely-linear complex maps, 2) Converts standard Transformers to complex domain, 3) Uses phase-aware quantization with 4th roots of unity codebook, 4) Implements recursive residual quantization to minimize error, 5) Enables multiplication-free accumulation for inference.

Result: Fairy2i restores LLaMA-2 7B performance at effective 2-bit precision to near full-precision levels, significantly outperforming state-of-the-art real-valued binary and ternary quantization methods.

Conclusion: Bridges complex-valued arithmetic efficiency with practical utility of pre-trained models, enabling efficient inference on commodity hardware without retraining.

Abstract: Large language models (LLMs) have revolutionized artificial intelligence, yet their massive memory and computational demands necessitate aggressive quantization, increasingly pushing representations toward the theoretical limit of a single bit. While complex-valued LLMs, such as iFairy, offer a superior chance for low-bit representation compared to real-valued counterparts, they require training from scratch, preventing the utilization of the vast ecosystem of pre-trained real-valued foundation models. Here we present Fairy2i, a universal framework that transforms pre-trained real-valued layers into an equivalent widely-linear complex form, enabling extremely low-bit quantization while reusing existing checkpoints. By proving a lossless mathematical equivalence between real and widely-linear maps, we convert standard Transformers into the complex domain and employ a phase-aware quantization scheme with a highly efficient codebook of fourth roots of unity. Furthermore, we introduce a recursive residual quantization mechanism that iteratively minimizes quantization error, allowing inference to proceed via efficient multiplication-free accumulation. We demonstrate that Fairy2i restores the performance of LLaMA-2 7B at an effective 2-bit precision to levels nearly comparable with full-precision baselines, significantly outperforming state-of-the-art real-valued binary and ternary quantization methods. This work bridges the gap between the representational efficiency of complex-valued arithmetic and the practical utility of pre-trained models, paving a new way for efficient inference on commodity hardware.

Method: AGENTSAFE profiles agentic loops (plan→act→observe→reflect) and toolchains, maps risks to structured taxonomies with agent-specific vulnerabilities, introduces safeguards, human oversight escalation, and pre-deployment scenario evaluation. It implements continuous governance through semantic telemetry, dynamic authorization, anomaly detection, and cryptographic tracing.

Result: The framework provides a unified governance approach that translates risk taxonomies into actionable controls, offers measurable pre-deployment assurance through Agent Safety Evaluation, and institutionalizes trust through runtime governance and accountability mechanisms.

Conclusion: AGENTSAFE enables measurable, auditable assurance across the lifecycle of agentic AI systems by addressing the unique risks of LLM-based agents through integrated design, runtime, and audit controls.

Abstract: The rapid deployment of large language model (LLM)-based agents introduces a new class of risks, driven by their capacity for autonomous planning, multi-step tool integration, and emergent interactions. It raises some risk factors for existing governance approaches as they remain fragmented: Existing frameworks are either static taxonomies driven; however, they lack an integrated end-to-end pipeline from risk identification to operational assurance, especially for an agentic platform. We propose AGENTSAFE, a practical governance framework for LLM-based agentic systems. The framework operationalises the AI Risk Repository into design, runtime, and audit controls, offering a governance framework for risk identification and assurance. The proposed framework, AGENTSAFE, profiles agentic loops (plan -> act -> observe -> reflect) and toolchains, and maps risks onto structured taxonomies extended with agent-specific vulnerabilities. It introduces safeguards that constrain risky behaviours, escalates high-impact actions to human oversight, and evaluates systems through pre-deployment scenario banks spanning security, privacy, fairness, and systemic safety. During deployment, AGENTSAFE ensures continuous governance through semantic telemetry, dynamic authorization, anomaly detection, and interruptibility mechanisms. Provenance and accountability are reinforced through cryptographic tracing and organizational controls, enabling measurable, auditable assurance across the lifecycle of agentic AI systems. The key contributions of this paper are: (1) a unified governance framework that translates risk taxonomies into actionable design, runtime, and audit controls; (2) an Agent Safety Evaluation methodology that provides measurable pre-deployment assurance; and (3) a set of runtime governance and accountability mechanisms that institutionalise trust in agentic AI ecosystems.

Method: The approach learns both communication topology and alignment maps among agents, creating a learned network sheaf with orthogonal maps. A semantic denoising and compression module constructs a shared global semantic space and derives sparse structured representations, formulated as a nonconvex dictionary learning problem solved iteratively with closed-form updates.

Result: Experiments with multiple AI agents pre-trained on real image data show that semantic denoising and compression facilitates agent alignment and semantic cluster extraction while maintaining high downstream task accuracy. The learned communication network reveals semantic heterogeneity across agents.

Conclusion: The proposed framework enables effective semantic communication among heterogeneous AI agents in 6G networks, providing interpretable insights into semantic heterogeneity while preserving task performance through learned sheaf structures and denoising mechanisms.

Abstract: Recent advances in AI call for a paradigm shift from bit-centric communication to goal- and semantics-oriented architectures, paving the way for AI-native 6G networks. In this context, we address a key open challenge: enabling heterogeneous AI agents to exchange compressed latent-space representations while mitigating semantic noise and preserving task-relevant meaning. We cast this challenge as learning both the communication topology and the alignment maps that govern information exchange among agents, yielding a learned network sheaf equipped with orthogonal maps. This learning process is further supported by a semantic denoising end compression module that constructs a shared global semantic space and derives sparse, structured representations of each agent’s latent space. This corresponds to a nonconvex dictionary learning problem solved iteratively with closed-form updates. Experiments with mutiple AI agents pre-trained on real image data show that the semantic denoising and compression facilitates AI agents alignment and the extraction of semantic clusters, while preserving high accuracy in downstream task. The resulting communication network provides new insights about semantic heterogeneity across agents, highlighting the interpretability of our methodology.

Method: The paper revisits gossip protocols from distributed systems as a complementary communication substrate. Gossip mechanisms provide decentralized, fault-tolerant knowledge diffusion to address gaps in structured protocols. The approach examines how gossip can support context-rich state propagation, resilient coordination under uncertainty, and emergent global awareness.

Result: The paper identifies both benefits and challenges of gossip protocols: benefits include scalability, adaptability, and fault tolerance; challenges include semantic relevance, temporal staleness, and limited action consistency guarantees in dynamic environments.

Conclusion: Rather than proposing a complete framework, the paper presents a research agenda for integrating gossip into multi-agent communication stacks. It argues gossip is essential for future agentic ecosystems to remain robust, adaptive, and self-organizing as scale and autonomy increase, while outlining open problems around semantic filtering, trust, and knowledge decay.

Abstract: As agentic platforms scale, agents are moving beyond fixed roles and predefined toolchains, creating an urgent need for flexible and decentralized coordination. Current structured communication protocols such as direct agent-to-agent messaging or MCP-style tool calls offer reliability, but they struggle to support the emergent and swarm-like intelligence required in large adaptive systems. Distributed agents must learn continuously, share context fluidly, and coordinate without depending solely on central planners. This paper revisits gossip protocols as a complementary substrate for agentic communication. Gossip mechanisms, long valued in distributed systems for their decentralized and fault-tolerant properties, provide scalable and adaptive diffusion of knowledge and fill gaps that structured protocols alone cannot efficiently address. However, gossip also introduces challenges, including semantic relevance, temporal staleness, and limited guarantees on action consistency in rapidly changing environments. We examine how gossip can support context-rich state propagation, resilient coordination under uncertainty, and emergent global awareness. We also outline open problems around semantic filtering, trust, and knowledge decay. Rather than proposing a complete framework, this paper presents a research agenda for integrating gossip into multi-agent communication stacks and argues that gossip is essential for future agentic ecosystems that must remain robust, adaptive, and self-organizing as their scale and autonomy increase.

Method: Extend Luby MIS protocol to mixed-strength agents where each agent draws strength from its own distribution, and analyze convergence dynamics.

Result: Fast dominance convergence holds for mixed-strength agents, but heterogeneity changes dynamics - constant fraction of edges need not be eliminated per round, with constructed examples showing asymptotically smaller progress.

Conclusion: Mixed-strength Luby MIS protocol provides formal confirmation of rapid convergence in natural processes, but inherent strength asymmetry produces qualitatively different global behavior compared to equal-strength settings.

Abstract: In many natural and engineered systems, agents interact through local contests that determine which individuals become dominant within their neighborhoods. These interactions are shaped by inherent differences in strength, and they often lead to stable dominance patterns that emerge surprisingly quickly relative to the size of the population. This motivates the search for simple mathematical models that capture both heterogeneous agent strength and rapid convergence to stable local dominance. A widely studied abstraction of local dominance is the Maximal Independent Set (MIS) problem. In the Luby MIS protocol that provably converges quickly to an MIS, each agent repeatedly generates a strength value chosen uniformly and becomes locally dominant if its value is smaller than those of its neighbors. This provides a theoretical explanation for fast dominance convergence in populations of equal-strength agents and naturally raises the question of whether fast convergence also holds in the more realistic setting where agents are inherently mixed-strength. To investigate this question, we introduce the mixed-strength agents model, in which each agent draws its strength from its own distribution. We prove that the extension of the Luby MIS protocol where each agent repeatedly generates a strength value from its own distribution still exhibits fast dominance convergence, providing formal confirmation of the rapid convergence observed in many mixed-strength natural processes. We also show that heterogeneity can significantly change the dynamics of the process. In contrast to the equal-strength setting, a constant fraction of edges need not be eliminated per round. We construct a population and strength profile in which progress per round is asymptotically smaller, illustrating how inherent strength asymmetry produces qualitatively different global behavior.

Method: Introduces AsymPuzl, a minimal two-agent puzzle environment where each agent observes complementary but incomplete views of a symbolic puzzle and must exchange messages to solve it cooperatively. Tests diverse current-generation and open-source LLMs.

Result: Strong models (GPT-5, Claude-4.0) reliably converge on solutions by sharing complete information in two turns; weaker models often ignore partner messages or over-correct hypotheses; feedback design is non-trivial with simple self-feedback improving success but detailed joint feedback hurting performance.

Conclusion: Even in simple cooperative tasks, LLM communication strategies diverge and depend on feedback granularity. AsymPuzl provides a testbed for probing multi-turn cooperation limits and studying coordination mechanisms.

Abstract: Large Language Model (LLM) agents are increasingly studied in multi-turn, multi-agent scenarios, yet most existing setups emphasize open-ended role-play rather than controlled evaluation. We introduce AsymPuzl, a minimal but expressive two-agent puzzle environment designed to isolate communication under information asymmetry. Each agent observes complementary but incomplete views of a symbolic puzzle and must exchange messages to solve it cooperatively. Using a diverse set of current-generation and open-source LLMs, we show that (i) strong models such as GPT-5 and Claude-4.0 reliably converge across puzzle sizes on the solution by sharing complete information in two turns, (ii) weaker models often ignore partner messages or over-correct their hypotheses, and (iii) feedback design is non-trivial: simple self-feedback improves success rates, while detailed joint feedback can hurt performance. These findings show that even in simple cooperative tasks, LLM communication strategies diverge and depend on the granularity of feedback signals. AsymPuzl thus provides a testbed for probing the limits of multi-turn cooperation and opens avenues for studying coordination mechanisms.

Method: SRPG uses a Dual-Stream Reconstruction Mechanism: 1) a strict sanitization stream ensures zero PII leakage, and 2) a context reconstruction stream (LLM-driven) recovers mathematical logic. This decouples instructional content from private data.

Result: Tests on MathDial show SRPG works across models; with GPT-4o, it achieves 0.0000 Attack Success Rate (zero leakage) and 0.8267 Exact Match, far outperforming the zero trust Pure LLM baseline (0.2138).

Conclusion: SRPG effectively protects minors’ privacy without sacrificing mathematical instructional quality, providing a balanced solution for educational multi-agent systems.

Abstract: Multi-Agent Systems (MAS) with large language models (LLMs) enable personalized education but risk leaking minors personally identifiable information (PII) via unstructured dialogue. Existing privacy methods struggle to balance security and utility: role-based access control fails on unstructured text, while naive masking destroys pedagogical context. We propose SRPG, a privacy guard for educational MAS, using a Dual-Stream Reconstruction Mechanism: a strict sanitization stream ensures zero PII leakage, and a context reconstruction stream (LLM driven) recovers mathematical logic. This decouples instructional content from private data, preserving teaching efficacy. Tests on MathDial show SRPG works across models; with GPT-4o, it achieves 0.0000 Attack Success Rate (ASR) (zero leakage) and 0.8267 Exact Match, far outperforming the zero trust Pure LLM baseline (0.2138). SRPG effectively protects minors privacy without sacrificing mathematical instructional quality.

Method: Two-stage framework: ECHO evolves Python code snippets for behavioral policies, while MIMIC evolves natural language messages to motivate adoption. Both use LLM-driven evolutionary search where LLMs propose variants and selection retains those maximizing collective performance in simulation.

Result: Demonstrated on agricultural landscape management and carbon-aware EV charging problems. Discovers high-performing heuristics compared to baselines and crafts tailored messages that successfully align simulated agent behavior with system-level goals.

Conclusion: ECHO-MIMIC transforms cognitive burden of collective action into implementable agent-level instructions, making ill-structured problems solvable and opening new path toward scalable, adaptive policy design.

Abstract: Collective action problems, which require aligning individual incentives with collective goals, are classic examples of Ill-Structured Problems (ISPs). For an individual agent, the causal links between local actions and global outcomes are unclear, stakeholder objectives often conflict, and no single, clear algorithm can bridge micro-level choices with macro-level welfare. We present ECHO-MIMIC, a general computational framework that converts this global complexity into a tractable, Well-Structured Problem (WSP) for each agent by discovering executable heuristics and persuasive rationales. The framework operates in two stages: ECHO (Evolutionary Crafting of Heuristics from Outcomes) evolves snippets of Python code that encode candidate behavioral policies, while MIMIC (Mechanism Inference & Messaging for Individual-to-Collective Alignment) evolves companion natural language messages that motivate agents to adopt those policies. Both phases employ a large-language-model-driven evolutionary search: the LLM proposes diverse and context-aware code or text variants, while population-level selection retains those that maximize collective performance in a simulated environment. We demonstrate this framework on two distinct ISPs: a canonical agricultural landscape management problem and a carbon-aware EV charging time slot usage problem. Results show that ECHO-MIMIC discovers high-performing heuristics compared to baselines and crafts tailored messages that successfully align simulated agent behavior with system-level goals. By coupling algorithmic rule discovery with tailored communication, ECHO-MIMIC transforms the cognitive burden of collective action into a implementable set of agent-level instructions, making previously ill-structured problems solvable in practice and opening a new path toward scalable, adaptive policy design.

Method: Introduces CamHarmTI benchmark with 4,500+ samples across three types of image-text posts. Evaluates 100 human users and 12 mainstream LVLMs, conducts fine-tuning experiments, and performs attention analysis with layer-wise probing to understand model perception.

Result: Humans easily recognize camouflaged harmful content (95.75% accuracy), while current LVLMs often fail (ChatGPT-4o: 2.10% accuracy). Fine-tuning with CamHarmTI improves Qwen2.5VL-7B accuracy by 55.94%. Attention analysis shows fine-tuning enhances sensitivity in early vision encoder layers.

Conclusion: LVLMs have inherent perceptual limitations in detecting camouflaged harmful content, highlighting a significant gap compared to human perception. The CamHarmTI benchmark serves as an effective resource for improving model perception and offers insights for developing more human-aligned visual reasoning systems.

Abstract: Large vision-language models (LVLMs) are increasingly used for tasks where detecting multimodal harmful content is crucial, such as online content moderation. However, real-world harmful content is often camouflaged, relying on nuanced text-image interplay, such as memes or images with embedded malicious text, to evade detection. This raises a key question: \textbf{can LVLMs perceive such camouflaged harmful content as sensitively as humans do?} In this paper, we introduce CamHarmTI, a benchmark for evaluating LVLM ability to perceive and interpret camouflaged harmful content within text-image compositions. CamHarmTI consists of over 4,500 samples across three types of image-text posts. Experiments on 100 human users and 12 mainstream LVLMs reveal a clear perceptual gap: humans easily recognize such content (e.g., over 95.75% accuracy), whereas current LVLMs often fail (e.g., ChatGPT-4o achieves only 2.10% accuracy). Moreover, fine-tuning experiments demonstrate that \bench serves as an effective resource for improving model perception, increasing accuracy by 55.94% for Qwen2.5VL-7B. Attention analysis and layer-wise probing further reveal that fine-tuning enhances sensitivity primarily in the early layers of the vision encoder, promoting a more integrated scene understanding. These findings highlight the inherent perceptual limitations in LVLMs and offer insight into more human-aligned visual reasoning systems.

Method: Proposes Cross-Space Synergy (CSS) with two components: 1) Synergistic Polynomial Fusion (SPF) using low-rank tensor factorization for efficient high-order cross-modal interactions, and 2) Pareto Gradient Modulator (PGM) steering updates along Pareto-optimal directions to alleviate gradient conflicts.

Result: CSS outperforms existing methods on IEMOCAP and MELD datasets in both accuracy and training stability, demonstrating effectiveness in complex multimodal scenarios.

Conclusion: The proposed CSS framework successfully addresses key challenges in MERC by combining efficient multimodal representation learning with stable optimization, achieving superior performance on benchmark datasets.

Abstract: Multimodal Emotion Recognition in Conversation (MERC) aims to predict speakers’ emotions by integrating textual, acoustic, and visual cues. Existing approaches either struggle to capture complex cross-modal interactions or experience gradient conflicts and unstable training when using deeper architectures. To address these issues, we propose Cross-Space Synergy (CSS), which couples a representation component with an optimization component. Synergistic Polynomial Fusion (SPF) serves the representation role, leveraging low-rank tensor factorization to efficiently capture high-order cross-modal interactions. Pareto Gradient Modulator (PGM) serves the optimization role, steering updates along Pareto-optimal directions across competing objectives to alleviate gradient conflicts and improve stability. Experiments show that CSS outperforms existing representative methods on IEMOCAP and MELD in both accuracy and training stability, demonstrating its effectiveness in complex multimodal scenarios.

Method: Systematic comparison of supervised (e.g., finite scalar quantization trained with ASR loss) and unsupervised (K-means clustering) semantic speech token extraction methods from Speech Foundation Models, specifically applied to child ASR tasks.

Result: Supervised methods outperform unsupervised ones, and surprisingly even surpass continuous representations. They perform well even in ultra-low bitrate settings.

Conclusion: Supervised semantic tokens offer significant advantages over unsupervised approaches, providing insights for improving discrete speech tokenization methods.

Abstract: Discrete speech tokens have gained attention for their storage efficiency and integration with Large Language Models (LLMs). They are commonly categorized into acoustic and semantic tokens, with the latter being more advantageous for Automatic Speech Recognition (ASR). Traditionally, unsupervised K-means clustering has been used to extract semantic speech tokens from Speech Foundation Models (SFMs). Recently, supervised methods, such as finite scalar quantization (FSQ) trained with ASR loss, have emerged for speech generation. Both approaches leverage pre-trained SFMs, benefiting low-resource tasks such as child ASR. This paper systematically compares supervised and unsupervised semantic speech tokens for child ASR. Results show that supervised methods not only outperform unsupervised ones but even unexpectedly surpass continuous representations, and they perform well even in ultra-low bitrate settings. These findings highlight the advantages of supervised semantic tokens and offer insights for improving discrete speech tokenization.

Method: Designs a harmonic filter with learnable triangular band-pass filter banks for flexible bandwidth per frequency bin, plus a half-harmonic component for fine-grained low-frequency harmonic relationships.

Result: Experiments on speech and singing datasets show effectiveness on both subjective and objective metrics.

Conclusion: The proposed universal harmonic discriminator improves vocoder performance through dynamic frequency resolution modeling and harmonic tracking.

Abstract: With the emergence of GAN-based vocoders, the discriminator, as a crucial component, has been developed recently. In our work, we focus on improving the time-frequency based discriminator. Particularly, Short-Time Fourier Transform (STFT) representation is usually used as input of time-frequency based discriminator. However, the STFT spectrogram has the same frequency resolution at different frequency bins, which results in an inferior performance, especially for singing voices. Motivated by this, we propose a universal harmonic discriminator for dynamic frequency resolution modeling and harmonic tracking. Specifically, we design a harmonic filter with learnable triangular band-pass filter banks, where each frequency bin has a flexible bandwidth. Additionally, we add a half-harmonic to capture fine-grained harmonic relationships at low-frequency band. Experiments on speech and singing datasets validate the effectiveness of the proposed discriminator on both subjective and objective metrics.

Method: Proposes IDMap framework with feedforward architecture that maps speaker identity index to speaker vector. Two specific models: IDMap-MLP (multilayer perceptron) and IDMap-Diff (diffusion-based).

Result: Small-scale evaluations on LibriSpeech show improved pseudo-speaker uniqueness and better voice privacy protection with reduced computational cost. Large-scale evaluations on MLS and Common Voice datasets demonstrate stable privacy protection as pseudo-speaker count increases.

Conclusion: IDMap framework effectively addresses uniqueness and computational efficiency challenges in pseudo-speaker generation for voice anonymization, offering scalable solution for voice privacy protection.

Abstract: Facilitated by the speech generation framework that disentangles speech into content, speaker, and prosody, voice anonymization is accomplished by substituting the original speaker embedding vector with that of a pseudo-speaker. In this framework, the pseudo-speaker generation forms a fundamental challenge. Current pseudo-speaker generation methods demonstrate limitations in the uniqueness of pseudo-speakers, consequently restricting their effectiveness in voice privacy protection. Besides, existing model-based methods suffer from heavy computation costs. Especially, in the large-scale scenario where a huge number of pseudo-speakers are generated, the limitations of uniqueness and computational inefficiency become more significant. To this end, this paper proposes a framework for pseudo-speaker generation, which establishes a mapping from speaker identity index to speaker vector in the feedforward architecture, termed IDMap. Specifically, the framework is specified into two models: IDMap-MLP and IDMap-Diff. Experiments were conducted on both small- and large-scale evaluation datasets. Small-scale evaluations on the LibriSpeech dataset validated the effectiveness of the proposed IDMap framework in enhancing the uniqueness of pseudo-speakers, thereby improving voice privacy protection, while at a reduced computational cost. Large-scale evaluations on the MLS and Common Voice datasets further justified the superiority of the IDMap framework regarding the stability of the voice privacy protection capability as the number of pseudo-speakers increased. Audio samples and open-source code can be found in https://github.com/VoicePrivacy/IDMap.

Method: Proposes a joint estimation framework for DoA and Noise Covariance Matrix: 1) Derives a quasi-linear solution for NCM estimation instead of exhaustive search, 2) Introduces novel DoA estimation technique operating across all frequency bins for robustness in reverberant environments, 3) Integrates both estimations for beamforming applications.

Result: Outperforms classical techniques like MUSIC in mid- to high-angle scenarios with lower angular errors. Achieves superior signal enhancement through beamforming with better noise rejection and interference canceling capabilities. Validated using both theoretical and empirical performance metrics.

Conclusion: The proposed joint estimation method provides a more efficient and robust solution for beamforming applications, offering computational advantages through quasi-linear NCM estimation and improved performance in challenging acoustic environments through multi-frequency DoA estimation.

Abstract: We propose a joint estimation method for the Direction-of-Arrival (DoA) and the Noise Covariance Matrix (NCM) tailored for beamforming applications. Building upon an existing NCM framework, our approach simplifies the estimation procedure by deriving an quasi-linear solution, instead of the traditional exhaustive search. Additionally, we introduce a novel DoA estimation technique that operates across all frequency bins, improving robustness in reverberant environments. Simulation results demonstrate that our method outperforms classical techniques, such as MUSIC, in mid- to high-angle scenarios, achieving lower angular errors and superior signal enhancement through beamforming. The proposed framework was also fared against other techniques for signal enhancement, having better noise rejection and interference canceling capabilities. These improvements are validated using both theoretical and empirical performance metrics.

Method: Proposed first deep learning framework using TimeMIL model trained on Log-Mel Spectrogram features enhanced with pseudo-labeling for audio event classification of hammering sounds.

Result: Achieved 91.17% ± 2.79% accuracy on intra-operative recordings. Reducing diversity of femoral stem brands improves performance, though limited dataset size remains a bottleneck.

Conclusion: Deep learning-based audio event classification is a feasible approach for intra-operative stability assessment in total hip arthroplasty, establishing a promising direction for medical applications.

Abstract: Audio event classification has recently emerged as a promising approach in medical applications. In total hip arthroplasty (THA), intra-operative hammering acoustics provide critical cues for assessing the initial stability of the femoral stem, yet variability due to femoral morphology, implant size, and surgical technique constrains conventional assessment methods. We propose the first deep learning framework for this task, employing a TimeMIL model trained on Log-Mel Spectrogram features and enhanced with pseudo-labeling. On intra-operative recordings, the method achieved 91.17 % +/- 2.79 % accuracy, demonstrating reliable estimation of stem stability. Comparative experiments further show that reducing the diversity of femoral stem brands improves model performance, although limited dataset size remains a bottleneck. These results establish deep learning-based audio event classification as a feasible approach for intra-operative stability assessment in THA.

Method: Developed enhanced ssVERDICT fitting approaches using dense multilayer perceptron (Dense MLP) and convolutional U-Net architectures, benchmarking against non-linear least-squares fitting and Diffusion Kurtosis Imaging across clinical to ultra-strong gradient systems.

Result: Dense ssVERDICT at ultra-strong gradients outperformed NLLS VERDICT with 47% median CNR boost, 52% reduction in inter-patient Coefficient of Variation, and 50% reduction in pooled f_ic variation. Delivered highest CNR, most stable parameter estimates, and clearest tumour-normal contrast.

Conclusion: Advanced gradient systems combined with deep learning-based modeling (ssVERDICT) show strong potential to improve non-invasive prostate cancer characterization and reduce unnecessary biopsies through enhanced microstructural insights.

Abstract: Diffusion MRI (dMRI) enables non-invasive assessment of prostate microstructure but conventional metrics such as the Apparent Diffusion Coefficient in multiparametric MRI lack specificity to underlying histology. Integrating dMRI with the compartment-based biophysical VERDICT (Vascular, Extracellular, and Restricted Diffusion for Cytometry in Tumours) framework offers richer microstructural insights, though clinical gradient systems (40-80 mT/m) suffer from poor signal-to-noise ratio (SNR) at stronger diffusion weightings due to prolonged echo times. Ultra-strong gradients (up to 300 mT/m) can mitigate these limitations by improving SNR and contrast-to-noise ratios (CNR) but their adoption has until recently been limited to research environments due to challenges with peripheral nerve stimulation thresholds and gradient non-uniformity. This study investigates whether physics-informed self-supervised VERDICT (ssVERDICT) fitting applied to ultra-strong gradients enhances prostate cancer characterization relative to current clinical acquisitions. We developed enhanced ssVERDICT fitting approaches using dense multilayer perceptron (Dense MLP) and convolutional U-Net architectures, benchmarking them against non-linear least-squares (NLLS) fitting and Diffusion Kurtosis Imaging across clinical- to ultra-strong gradient systems. Dense ssVERDICT at ultra-strong gradient notably outperformed NLLS VERDICT, boosting median CNR by 47%, cutting inter-patient Coefficient of Variation by 52%, and reducing pooled f_ic variation by 50%. Overall, it delivered the highest CNR, the most stable parameter estimates, and the clearest tumour-normal contrast compared with conventional methods and clinical gradient systems. These findings highlight the potential of advanced gradient systems and deep learning-based modelling to improve non-invasive prostate cancer characterization and reduce unnecessary biopsies.

Method: Organized FITBIR structural/diffusion MRI data into Brain Imaging Data Structure (BIDS), performed quality assurance and harmonization, used UNesT for segmentation of 132 brain regions, and analyzed metrics with GAMLSS models.

Result: After processing, 4,868 subjects with structural MRI and 2,666 with diffusion MRI were analyzed. Significant differences found in 54 brain regions between TBI subjects and controls (q<0.05, FDR-corrected).

Conclusion: Standardizing heterogeneous TBI imaging data enables robust analysis revealing widespread structural brain changes in TBI, providing valuable insights for TBI research and potential biomarkers.

Abstract: The Federal Interagency Traumatic Brain Injury Research (FITBIR) database is a centralized data repository for traumatic brain injury (TBI) research. It includes over 45,529 magnetic resonance images (MRI) from 6,211 subjects (9,229 imaging sessions) across 26 studies with heterogeneous organization formats, contrasts, acquisition parameters, and demographics. In this work, we organized all available structural and diffusion MRI from FITBIR along with relevant demographic information into the Brain Imaging Data Structure. We analyzed whole-brain mean fractional anisotropy, mean diffusivity, total intracranial volume, and the volumes of 132 regions of interest using UNesT segmentations. There were 4,868 subjects (7,035 sessions) with structural MRI and 2,666 subjects (3,763 sessions) with diffusion MRI following quality assurance and harmonization. We modeled profiles for these metrics across ages with generalized additive models for location, scale, and shape (GAMLSS) and found significant differences in subjects with TBI compared to controls in volumes of 54 regions of the brain (q<0.05, likelihood ratio test with false discovery rate correction).

Method: Reconstructs data into fourth-order tensor, uses bilateral tensor ring decomposition to separate weak spatial correlations from strong temporal-patch correlations while capturing interactions between components, solved via proximal alternating minimization framework.

Result: Outperforms state-of-the-art methods in detection accuracy, background suppression capability, and computational speed.

Conclusion: BTR-ISTD effectively addresses computational complexity limitations while improving infrared small target detection performance in challenging scenarios.

Abstract: Infrared small target detection plays a crucial role in military reconnaissance and air defense systems. However,existing low-rank sparse based methods still face high computational complexity when dealing with low-contrast small targets and complex dynamic backgrounds mixed with target-like interference. To address this limitation, we reconstruct the data into a fourth-order tensor and propose a new infrared small target detection model based on bilateral tensor ring decomposition, called BTR-ISTD. The approach begins by constructing a four-dimensional infrared tensor from an image sequence, then utilizes BTR decomposition to effectively distinguish weak spatial correlations from strong temporal-patch correlations while simultaneously capturing interactions between these two components. This model is efficiently solved under the proximal alternating minimization (PAM) framework. Experimental results demonstrate that the proposed approach outperforms several state-of-the-art methods in terms of detection accuracy, background suppression capability, and computational speed.

Method: Tada-DIP combines input-adaptation and denoising regularization to create a fully 3D DIP approach. This combination helps produce high-quality 3D reconstructions while preventing the overfitting common in standard DIP methods.

Result: Experiments on sparse-view X-ray computed tomography reconstruction show Tada-DIP produces much better reconstructions than training-data-free baselines and achieves performance comparable to supervised networks trained on large datasets with fully-sampled volumes.

Conclusion: Tada-DIP is an effective 3D extension of Deep Image Prior that successfully addresses 3D inverse problems while avoiding overfitting, demonstrating competitive performance with supervised methods without requiring large training datasets.

Abstract: Deep Image Prior (DIP) has recently emerged as a promising one-shot neural-network based image reconstruction method. However, DIP has seen limited application to 3D image reconstruction problems. In this work, we introduce Tada-DIP, a highly effective and fully 3D DIP method for solving 3D inverse problems. By combining input-adaptation and denoising regularization, Tada-DIP produces high-quality 3D reconstructions while avoiding the overfitting phenomenon that is common in DIP. Experiments on sparse-view X-ray computed tomography reconstruction validate the effectiveness of the proposed method, demonstrating that Tada-DIP produces much better reconstructions than training-data-free baselines and achieves reconstruction performance on par with a supervised network trained using a large dataset with fully-sampled volumes.

Method: Proposes a robustification strategy using pretrained diffusion models as noise purifiers. Instead of adversarial training’s minimax optimization, the approach fine-tunes on purified examples generated by diffusion models.

Result: Experimental results demonstrate the approach effectively mitigates instabilities compared to leading robustification methods like adversarial training and randomized smoothing.

Conclusion: Diffusion models can serve as effective noise purifiers to enhance robustness of DL-based MRI reconstruction, offering a practical alternative to adversarial training without complex minimax optimization.

Abstract: Deep learning (DL) techniques have been extensively employed in magnetic resonance imaging (MRI) reconstruction, delivering notable performance enhancements over traditional non-DL methods. Nonetheless, recent studies have identified vulnerabilities in these models during testing, namely, their susceptibility to (\textit{i}) worst-case measurement perturbations and to (\textit{ii}) variations in training/testing settings like acceleration factors and k-space sampling locations. This paper addresses the robustness challenges by leveraging diffusion models. In particular, we present a robustification strategy that improves the resilience of DL-based MRI reconstruction methods by utilizing pretrained diffusion models as noise purifiers. In contrast to conventional robustification methods for DL-based MRI reconstruction, such as adversarial training (AT), our proposed approach eliminates the need to tackle a minimax optimization problem. It only necessitates fine-tuning on purified examples. Our experimental results highlight the efficacy of our approach in mitigating the aforementioned instabilities when compared to leading robustification approaches for deep MRI reconstruction, including AT and randomized smoothing.

Method: Proposes 2LMM with two scaling steps: endmember scaling across image and pixel-wise scaling. Solves the mildly non-convex optimization problem using second-order optimization techniques, requiring virtually no hyperparameter tuning.

Result: The method is competitive and sometimes superior to state-of-the-art in unmixing tasks, performs well in challenging scenarios like blind unmixing, and is highly robust with minimal tuning requirements.

Conclusion: 2LMM successfully bridges the complexity-tractability gap in spectral unmixing, offering a robust, easy-to-use solution with strong performance across various scenarios, including the first application of second-order optimization for endmember variability modeling.

Abstract: In this paper, we propose a Two-Step Linear Mixing Model (2LMM) that bridges the gap between model complexity and computational tractability. The model achieves this by introducing two distinct scaling steps: an endmember scaling step across the image, and another for pixel-wise scaling. We show that this model leads to only a mildly non-convex optimization problem, which we solve with an optimization algorithm that incorporates second-order information. To the authors’ knowledge, this work represents the first application of second-order optimization techniques to solve a spectral unmixing problem that models endmember variability. Our method is highly robust, as it requires virtually no hyperparameter tuning and can therefore be used easily and quickly in a wide range of unmixing tasks. We show through extensive experiments on both simulated and real data that the new model is competitive and in some cases superior to the state of the art in unmixing. The model also performs very well in challenging scenarios, such as blind unmixing.

Method: Deep Image Prior uses convolutional neural networks initialized with random noise, leveraging implicit regularization of deep networks. Only requires the network, noisy measurements, and forward operator - no training data needed.

Result: DIP can learn and restore image structures without external datasets, but suffers from overfitting due to network over-parameterization. Recent advancements include regularization techniques, network re-parameterization, early stopping, and combinations with pre-trained networks.

Conclusion: DIP represents a significant milestone in training-data-free inverse imaging, with ongoing research needed to address overfitting limitations. The paper provides comprehensive review, empirical comparisons, and highlights open research questions for future development.

Abstract: In recent years, deep learning methods have been extensively developed for inverse imaging problems (IIPs), encompassing supervised, self-supervised, and generative approaches. Most of these methods require large amounts of labeled or unlabeled training data to learn effective models. However, in many practical applications, such as medical image reconstruction, extensive training datasets are often unavailable or limited. A significant milestone in addressing this challenge came in 2018 with the work of Ulyanov et al., which introduced the Deep Image Prior (DIP)–the first training-data-free neural network method for IIPs. Unlike conventional deep learning approaches, DIP requires only a convolutional neural network, the noisy measurements, and a forward operator. By leveraging the implicit regularization of deep networks initialized with random noise, DIP can learn and restore image structures without relying on external datasets. However, a well-known limitation of DIP is its susceptibility to overfitting, primarily due to the over-parameterization of the network. In this tutorial paper, we provide a comprehensive review of DIP, including a theoretical analysis of its training dynamics. We also categorize and discuss recent advancements in DIP-based methods aimed at mitigating overfitting, including techniques such as regularization, network re-parameterization, and early stopping. Furthermore, we discuss approaches that combine DIP with pre-trained neural networks, present empirical comparison results against data-centric methods, and highlight open research questions and future directions.

Method: PixCell is a diffusion model trained on PanCan-30M dataset using progressive training strategy and self-supervision-based conditioning without human annotations. Conditions on real slides to capture data properties.

Result: PixCell generates high-fidelity synthetic images usable for data augmentation to boost classification performance. Enables privacy-preserving synthetic data sharing and virtual IHC staining from H&E inputs to multiple IHC stains.

Conclusion: PixCell demonstrates foundational versatility for computational pathology, addressing key challenges through generative modeling. Models are publicly released to accelerate research in the field.

Abstract: The digitization of histology slides has revolutionized pathology, providing massive datasets for cancer diagnosis and research. Self-supervised and vision-language models have been shown to effectively mine large pathology datasets to learn discriminative representations. On the other hand, there are unique problems in pathology, such as annotated data scarcity, privacy regulations in data sharing, and inherently generative tasks like virtual staining. Generative models, capable of synthesizing realistic and diverse images, present a compelling solution to address these problems through image synthesis. We introduce PixCell, the first generative foundation model for histopathology images. PixCell is a diffusion model trained on PanCan-30M, a large, diverse dataset derived from 69,184 H&E-stained whole slide images of various cancer types. We employ a progressive training strategy and a self-supervision-based conditioning that allows us to scale up training without any human-annotated data. By conditioning on real slides, the synthetic images capture the properties of the real data and can be used as data augmentation for small-scale datasets to boost classification performance. We prove the foundational versatility of PixCell by applying it to two generative downstream tasks: privacy-preserving synthetic data generation and virtual IHC staining. PixCell’s high-fidelity conditional generation enables institutions to use their private data to synthesize highly realistic, site-specific surrogate images that can be shared in place of raw patient data. Furthermore, using datasets of roughly paired H&E-IHC tiles, we learn to translate PixCell’s conditioning from H&E to multiple IHC stains, allowing the generation of IHC images from H&E inputs. Our trained models are publicly released to accelerate research in computational pathology.

Method: Used TransUNet architecture combining Vision Transformers’ global attention with U-Net’s localization. Trained on FUSeg dataset with robust augmentation pipeline and hybrid loss function to address class imbalance. Integrated Grad-CAM for explainability and performed clinical utility analysis.

Result: Achieved Dice Similarity Coefficient of 0.8799 with optimized threshold of 0.4389 on validation set. Strong correlation (Pearson r = 0.9631) between predicted and ground-truth wound areas. Grad-CAM provided transparent visualization of model focus areas.

Conclusion: TransUNet effectively integrates global and local feature extraction, providing a reliable, effective, and explainable solution for automated foot ulcer assessment that addresses limitations of traditional CNNs.

Abstract: Automated segmentation of diabetic foot ulcers (DFUs) plays a critical role in clinical diagnosis, therapeutic planning, and longitudinal wound monitoring. However, this task remains challenging due to the heterogeneous appearance, irregular morphology, and complex backgrounds associated with ulcer regions in clinical photographs. Traditional convolutional neural networks (CNNs), such as U-Net, provide strong localization capabilities but struggle to model long-range spatial dependencies due to their inherently limited receptive fields. To address this, we employ the TransUNet architecture, a hybrid framework that integrates the global attention mechanism of Vision Transformers (ViTs) into the U-Net structure. This combination allows the model to extract global contextual features while maintaining fine-grained spatial resolution. We trained the model on the public Foot Ulcer Segmentation Challenge (FUSeg) dataset using a robust augmentation pipeline and a hybrid loss function to mitigate class imbalance. On the validation set, the model achieved a Dice Similarity Coefficient (F1-score) of 0.8799 using an optimized threshold of 0.4389. To ensure clinical transparency, we integrated Grad-CAM visualizations to highlight model focus areas. Furthermore, a clinical utility analysis demonstrated a strong correlation (Pearson r = 0.9631) between predicted and ground-truth wound areas. These outcomes demonstrate that our approach effectively integrates global and local feature extraction, offering a reliable, effective, and explainable solution for automated foot ulcer assessment.

Method: Proposed Attention-Augmented Wavelet YOLO (AAW-YOLO) network tailored for TCCD data, trained on prospectively collected dataset of 738 annotated frames with 3,419 labeled artery instances for real-time brain vessel segmentation.

Result: AAW-YOLO achieved average Dice score of 0.901, IoU of 0.823, precision of 0.882, recall of 0.926, mAP of 0.953, with per-frame inference speed of 14.199 ms, demonstrating strong performance for both ipsilateral and contralateral vessels.

Conclusion: The AI system reduces reliance on operator experience in TCCD-based cerebrovascular screening, offering practical solution for clinical workflows and resource-constrained settings, with future work exploring bilateral modeling and larger-scale validation.

Abstract: The Circle of Willis (CoW), vital for ensuring consistent blood flow to the brain, is closely linked to ischemic stroke. Accurate assessment of the CoW is important for identifying individuals at risk and guiding appropriate clinical management. Among existing imaging methods, Transcranial Color-coded Doppler (TCCD) offers unique advantages due to its radiation-free nature, affordability, and accessibility. However, reliable TCCD assessments depend heavily on operator expertise for identifying anatomical landmarks and performing accurate angle correction, which limits its widespread adoption. To address this challenge, we propose an AI-powered, real-time CoW auto-segmentation system capable of efficiently capturing cerebral arteries. No prior studies have explored AI-driven cerebrovascular segmentation using TCCD. In this work, we introduce a novel Attention-Augmented Wavelet YOLO (AAW-YOLO) network tailored for TCCD data, designed to provide real-time guidance for brain vessel segmentation in the CoW. We prospectively collected TCCD data comprising 738 annotated frames and 3,419 labeled artery instances to establish a high-quality dataset for model training and evaluation. The proposed AAW-YOLO demonstrated strong performance in segmenting both ipsilateral and contralateral CoW vessels, achieving an average Dice score of 0.901, IoU of 0.823, precision of 0.882, recall of 0.926, and mAP of 0.953, with a per-frame inference speed of 14.199 ms. This system offers a practical solution to reduce reliance on operator experience in TCCD-based cerebrovascular screening, with potential applications in routine clinical workflows and resource-constrained settings. Future research will explore bilateral modeling and larger-scale validation.

Method: Proposes a modified consistency sampling framework with measurement-consistency mechanism that leverages the degradation operator to regulate sampler stochasticity, enforcing data fidelity while maintaining computational efficiency.

Result: Experiments on Fashion-MNIST and LSUN Bedroom show consistent improvements in perceptual (FID, KID) and pixel-level (PSNR, SSIM) metrics over baseline consistency and diffusion methods, achieving competitive reconstruction with few sampling steps.

Conclusion: The proposed method successfully adapts consistency models to inverse imaging problems, offering a practical solution that balances reconstruction quality with computational efficiency, enabling faster deployment of generative priors.

Abstract: Diffusion models have emerged as powerful generative priors for solving inverse imaging problems. However, their practical deployment is hindered by the substantial computational cost of slow, multi-step sampling. Although Consistency Models (CMs) address this limitation by enabling high-quality generation in only one or a few steps, their direct application to inverse problems has remained largely unexplored. This paper introduces a modified consistency sampling framework specifically designed for inverse problems. The proposed approach regulates the sampler’s stochasticity through a measurement-consistency mechanism that leverages the degradation operator, thereby enforcing fidelity to the observed data while preserving the computational efficiency of consistency-based generation. Comprehensive experiments on the Fashion-MNIST and LSUN Bedroom datasets demonstrate consistent improvements across both perceptual and pixel-level metrics, including the Fréchet Inception Distance (FID), Kernel Inception Distance (KID), peak signal-to-noise ratio (PSNR), and structural similarity index measure (SSIM), compared with baseline consistency and diffusion-based sampling methods. The proposed method achieves competitive or superior reconstruction quality with only a small number of sampling steps.