Method: Developed Risk-Concealment Attacks (RCA) - a multi-turn framework that iteratively conceals regulatory risks to provoke regulatory-violating responses. Created FIN-Bench benchmark for systematic evaluation of LLM safety in financial contexts.

Result: RCA achieved 93.18% average attack success rate across nine mainstream LLMs, including 98.28% on GPT-4.1 and 97.56% on OpenAI o1, demonstrating severe vulnerabilities.

Conclusion: Current alignment techniques have critical gaps in financial domains, highlighting urgent need for stronger moderation mechanisms and domain-aware LLM alignment approaches.

Abstract: Large Language Models (LLMs) are increasingly integrated into financial applications, yet existing red-teaming research primarily targets harmful content, largely neglecting regulatory risks. In this work, we aim to investigate the vulnerability of financial LLMs through red-teaming approaches. We introduce Risk-Concealment Attacks (RCA), a novel multi-turn framework that iteratively conceals regulatory risks to provoke seemingly compliant yet regulatory-violating responses from LLMs. To enable systematic evaluation, we construct FIN-Bench, a domain-specific benchmark for assessing LLM safety in financial contexts. Extensive experiments on FIN-Bench demonstrate that RCA effectively bypasses nine mainstream LLMs, achieving an average attack success rate (ASR) of 93.18%, including 98.28% on GPT-4.1 and 97.56% on OpenAI o1. These findings reveal a critical gap in current alignment techniques and underscore the urgent need for stronger moderation mechanisms in financial domains. We hope this work offers practical insights for advancing robust and domain-aware LLM alignment.

Method: Extract activations after question reading but before token generation, train linear probes to predict answer correctness across three model families (7B-70B parameters), and test generalization on diverse knowledge datasets.

Result: Linear probes trained on trivia questions successfully predict correctness in-distribution and out-of-distribution, outperforming black-box baselines and verbal confidence. Predictive power saturates in intermediate layers, but generalization fails on mathematical reasoning tasks. The same direction also captures confidence when models respond “I don’t know”.

Conclusion: LLMs develop internal self-assessment capabilities mid-computation that can predict answer correctness across domains, except mathematical reasoning, providing essential insights into model internals and complementing previous work on truthfulness probes.

Abstract: Do large language models (LLMs) anticipate when they will answer correctly? To study this, we extract activations after a question is read but before any tokens are generated, and train linear probes to predict whether the model’s forthcoming answer will be correct. Across three open-source model families ranging from 7 to 70 billion parameters, projections on this “in-advance correctness direction” trained on generic trivia questions predict success in distribution and on diverse out-of-distribution knowledge datasets, outperforming black-box baselines and verbalised predicted confidence. Predictive power saturates in intermediate layers, suggesting that self-assessment emerges mid-computation. Notably, generalisation falters on questions requiring mathematical reasoning. Moreover, for models responding “I don’t know”, doing so strongly correlates with the probe score, indicating that the same direction also captures confidence. By complementing previous results on truthfulness and other behaviours obtained with probes and sparse auto-encoders, our work contributes essential findings to elucidate LLM internals.

Method: Position paper with case study of GlossLM (multilingual IGT generation model) and small-scale user study with three documentary linguists to evaluate real-world usability.

Result: Despite strong metric-based performance, the system fails to meet core usability needs in documentation contexts, revealing issues with model constraints, label standardization, segmentation, and personalization.

Conclusion: Integrating User-Centered Design principles not only produces more effective tools but also surfaces richer, more relevant research directions for computational morphology in language documentation.

Abstract: Computational morphology has the potential to support language documentation through tasks like morphological segmentation and the generation of Interlinear Glossed Text (IGT). However, our research outputs have seen limited use in real-world language documentation settings. This position paper situates the disconnect between computational morphology and language documentation within a broader misalignment between research and practice in NLP and argues that the field risks becoming decontextualized and ineffectual without systematic integration of User-Centered Design (UCD). To demonstrate how principles from UCD can reshape the research agenda, we present a case study of GlossLM, a state-of-the-art multilingual IGT generation model. Through a small-scale user study with three documentary linguists, we find that despite strong metric based performance, the system fails to meet core usability needs in real documentation contexts. These insights raise new research questions around model constraints, label standardization, segmentation, and personalization. We argue that centering users not only produces more effective tools, but surfaces richer, more relevant research directions

Method: Analyzed entropy neurons in autoregressive transformer models, examined their impact on output entropy and token ranking, and performed ablation studies to observe changes in generation behavior.

Result: Entropy neurons are responsible for suppressing context copying across various LLMs, and their ablation leads to significant changes in the generation process when contextual and parametric information conflict.

Conclusion: These findings enhance our understanding of LLM internal dynamics when processing conflicting information, revealing the specific role of entropy neurons in context suppression mechanisms.

Abstract: The behavior of Large Language Models (LLMs) when facing contextual information that conflicts with their internal parametric knowledge is inconsistent, with no generally accepted explanation for the expected outcome distribution. Recent work has identified in autoregressive transformer models a class of neurons – called entropy neurons – that produce a significant effect on the model output entropy while having an overall moderate impact on the ranking of the predicted tokens. In this paper, we investigate the preliminary claim that these neurons are involved in inhibiting context copying behavior in transformers by looking at their role in resolving conflicts between contextual and parametric information. We show that entropy neurons are responsible for suppressing context copying across a range of LLMs, and that ablating them leads to a significant change in the generation process. These results enhance our understanding of the internal dynamics of LLMs when handling conflicting information.

Method: Proposes a gloss-free latent diffusion model that generates sign language sequences from noisy latent codes and spoken text through non-autoregressive iterative denoising. Also designs a cross-modal signing aligner to learn a shared latent space between visual and textual content.

Result: Achieves state-of-the-art performance on PHOENIX14T and How2Sign datasets, demonstrating effective gloss-free sign language generation with improved accuracy and contextual relevance.

Conclusion: The proposed Text2SignDiff method successfully eliminates the need for gloss annotations in sign language production, providing a more flexible and generalizable approach that bridges communication gaps for deaf communities.

Abstract: Sign language production (SLP) aims to translate spoken language sentences into a sequence of pose frames in a sign language, bridging the communication gap and promoting digital inclusion for deaf and hard-of-hearing communities. Existing methods typically rely on gloss, a symbolic representation of sign language words or phrases that serves as an intermediate step in SLP. This limits the flexibility and generalization of SLP, as gloss annotations are often unavailable and language-specific. Therefore, we present a novel diffusion-based generative approach - Text2Sign Diffusion (Text2SignDiff) for gloss-free SLP. Specifically, a gloss-free latent diffusion model is proposed to generate sign language sequences from noisy latent sign codes and spoken text jointly, reducing the potential error accumulation through a non-autoregressive iterative denoising process. We also design a cross-modal signing aligner that learns a shared latent space to bridge visual and textual content in sign and spoken languages. This alignment supports the conditioned diffusion-based process, enabling more accurate and contextually relevant sign language generation without gloss. Extensive experiments on the commonly used PHOENIX14T and How2Sign datasets demonstrate the effectiveness of our method, achieving the state-of-the-art performance.

Method: Proposed EthosAgents approach designed to simulate diverse perspectives and values through a lightweight, generalizable pluralistic alignment method.

Result: Empirically demonstrated advancement in pluralistic alignment across seven varying-sized open and closed models for all three modes, showing effectiveness in health-related contexts.

Conclusion: Health-related pluralism requires adaptable and normatively aware approaches, offering insights for better respecting diversity in high-stakes domains beyond healthcare.

Abstract: As large language models are increasingly deployed in sensitive domains such as healthcare, ensuring their outputs reflect the diverse values and perspectives held across populations is critical. However, existing alignment approaches, including pluralistic paradigms like Modular Pluralism, often fall short in the health domain, where personal, cultural, and situational factors shape pluralism. Motivated by the aforementioned healthcare challenges, we propose a first lightweight, generalizable, pluralistic alignment approach, EthosAgents, designed to simulate diverse perspectives and values. We empirically show that it advances the pluralistic alignment for all three modes across seven varying-sized open and closed models. Our findings reveal that health-related pluralism demands adaptable and normatively aware approaches, offering insights into how these models can better respect diversity in other high-stakes domains.

Method: Proposes Struct-Bench framework requiring users to represent dataset structure as Context-Free Grammars (CFGs), includes 5 real-world and 2 synthetic datasets annotated with CFGs, and provides reference implementations of metrics and a leaderboard.

Result: The benchmark demonstrates significant challenges for state-of-the-art DP synthetic data generation methods and provides a standardized evaluation platform for researchers to benchmark privacy-preserving methods.

Conclusion: Struct-Bench offers a comprehensive evaluation framework that enables systematic improvement of synthetic data quality for structured datasets with natural language components, as demonstrated through a case study improving Private Evolution’s performance.

Abstract: Differentially private (DP) synthetic data generation is a promising technique for utilizing private datasets that otherwise cannot be exposed for model training or other analytics. While much research literature has focused on generating private unstructured text and image data, in enterprise settings, structured data (e.g., tabular) is more common, often including natural language fields or components. Existing synthetic data evaluation techniques (e.g., FID) struggle to capture the structural properties and correlations of such datasets. In this work, we propose Struct-Bench, a framework and benchmark for evaluating synthetic datasets derived from structured datasets that contain natural language data. The Struct-Bench framework requires users to provide a representation of their dataset structure as a Context-Free Grammar (CFG). Our benchmark comprises 5 real-world and 2 synthetically generated datasets, each annotated with CFGs. We show that these datasets demonstrably present a great challenge even for state-of-the-art DP synthetic data generation methods. Struct-Bench also includes reference implementations of different metrics and a leaderboard, thereby providing researchers a standardized evaluation platform to benchmark and investigate privacy-preserving synthetic data generation methods. Further, we also present a case study showing how to use Struct-Bench to improve the synthetic data quality of Private Evolution (PE) on structured data. The benchmark and the leaderboard have been publicly made available at https://struct-bench.github.io.

Method: Examines three main components: (1) retrieval mechanisms (sparse, dense, hybrid approaches), (2) text structuring techniques (taxonomy construction, hierarchical classification, information extraction), and (3) integration methods with LLMs (prompt-based methods, reasoning frameworks, knowledge embedding).

Result: Provides a comprehensive overview of RAS methods, identifies technical challenges in retrieval efficiency, structure quality, and knowledge integration, and highlights research opportunities in multimodal retrieval, cross-lingual structures, and interactive systems.

Conclusion: RAS Augmented Generation offers a promising framework to enhance LLM capabilities by integrating dynamic retrieval with structured knowledge representations, providing researchers and practitioners with insights into current methods and future directions.

Abstract: Large Language Models (LLMs) have revolutionized natural language processing with their remarkable capabilities in text generation and reasoning. However, these models face critical challenges when deployed in real-world applications, including hallucination generation, outdated knowledge, and limited domain expertise. Retrieval And Structuring (RAS) Augmented Generation addresses these limitations by integrating dynamic information retrieval with structured knowledge representations. This survey (1) examines retrieval mechanisms including sparse, dense, and hybrid approaches for accessing external knowledge; (2) explore text structuring techniques such as taxonomy construction, hierarchical classification, and information extraction that transform unstructured text into organized representations; and (3) investigate how these structured representations integrate with LLMs through prompt-based methods, reasoning frameworks, and knowledge embedding techniques. It also identifies technical challenges in retrieval efficiency, structure quality, and knowledge integration, while highlighting research opportunities in multimodal retrieval, cross-lingual structures, and interactive systems. This comprehensive overview provides researchers and practitioners with insights into RAS methods, applications, and future directions.

Method: Recorded two groups of participants twice - before and after acoustic improvements in a seminar room at Graz University of Technology, comparing communication outcomes.

Result: Findings indicate that room acoustic interventions improve communicative success in hybrid meetings, though results did not reach statistical significance due to small sample size.

Conclusion: Acoustic design interventions show clear potential for enhancing communication in hybrid meeting spaces, highlighting the importance of considering room acoustics alongside internet connectivity in hybrid meeting room design.

Abstract: Since the COVID-19 pandemic in 2020, universities and companies have increasingly integrated hybrid features into their meeting spaces, or even created dedicated rooms for this purpose. While the importance of a fast and stable internet connection is often prioritized, the acoustic design of seminar rooms is frequently overlooked. Poor acoustics, particularly excessive reverberation, can lead to issues such as misunderstandings, reduced speech intelligibility or cognitive and vocal fatigue. This pilot study investigates whether room acoustic interventions in a seminar room at Graz University of Technology support better communication in hybrid meetings. For this purpose, we recorded two groups of persons twice, once before and once after improving the acoustics of the room. Our findings – despite not reaching statistical significance due to the small sample size - indicate clearly that our spatial interventions improve communicative success in hybrid meetings. To make the paper accessible also for readers from the speech communication community, we explain room acoustics background, relevant for the interpretation of our results.

Method: Starts with limited human-generated domain questions, systematically expands them using LLM, then dynamically retrieves domain-relevant resources to generate accurate answers for each augmented question.

Result: Enhances both diversity and quality of SFT datasets, leading to measurable improvements in LLM performance within specialized domains, and facilitates model editing tasks.

Conclusion: SearchInstruct provides an effective automated solution for generating high-quality domain-specific instruction datasets, improving LLM fine-tuning while enabling efficient model updates and community adoption through open-source release.

Abstract: Supervised Fine-Tuning (SFT) is essential for training large language models (LLMs), significantly enhancing critical capabilities such as instruction following and in-context learning. Nevertheless, creating suitable training datasets tailored for specific domains remains challenging due to unique domain constraints and data scarcity. In this paper, we propose SearchInstruct, an innovative method explicitly designed to construct high quality instruction datasets for SFT. Our approach begins with a limited set of domain specific, human generated questions, which are systematically expanded using a large language model. Subsequently, domain relevant resources are dynamically retrieved to generate accurate and contextually appropriate answers for each augmented question. Experimental evaluation demonstrates that SearchInstruct enhances both the diversity and quality of SFT datasets, leading to measurable improvements in LLM performance within specialized domains. Additionally, we show that beyond dataset generation, the proposed method can also effectively facilitate tasks such as model editing, enabling efficient updates to existing models. To facilitate reproducibility and community adoption, we provide full implementation details, the complete set of generated instruction response pairs, and the source code in a publicly accessible Git repository: https://github.com/mostafaamiri/SearchInstruct

Method: Compared mBERT, XLM, XLM-RoBERTa, RemBERT, and mT5 on fake-vs-true classification using PolyTruth Disinfo Corpus - 60,486 statement pairs across 25+ languages covering 5 language families and diverse topics.

Result: Performance variations observed: RemBERT achieved better overall accuracy, particularly in low-resource languages, while mBERT and XLM showed limitations with scarce training data.

Conclusion: Findings reveal both potential and current limitations of AI systems for multilingual disinformation detection, with dataset publicly available to encourage further research advancement.

Abstract: Disinformation spreads rapidly across linguistic boundaries, yet most AI models are still benchmarked only on English. We address this gap with a systematic comparison of five multilingual transformer models: mBERT, XLM, XLM-RoBERTa, RemBERT, and mT5 on a common fake-vs-true machine learning classification task. While transformer-based language models have demonstrated notable success in detecting disinformation in English, their effectiveness in multilingual contexts still remains up for debate. To facilitate evaluation, we introduce PolyTruth Disinfo Corpus, a novel corpus of 60,486 statement pairs (false claim vs. factual correction) spanning over twenty five languages that collectively cover five language families and a broad topical range from politics, health, climate, finance, and conspiracy, half of which are fact-checked disinformation claims verified by an augmented MindBugs Discovery dataset. Our experiments revealed performance variations. Models such as RemBERT achieved better overall accuracy, particularly excelling in low-resource languages, whereas models like mBERT and XLM exhibit considerable limitations when training data is scarce. We provide a discussion of these performance patterns and implications for real-world deployment. The dataset is publicly available on our GitHub repository to encourage further experimentation and advancement. Our findings illuminate both the potential and the current limitations of AI systems for multilingual disinformation detection.

Method: Evaluated models on three tasks (mode identification, maximum likelihood estimation, sample generation) by prompting them with observations from probability distributions and queries about joint distributions or conditionals.

Result: Clear performance gap between smaller and larger models, with larger models showing stronger inference and surprising sample generation capabilities. Notable limitations include 60% performance degradation with increased context length and sensitivity to notation variations.

Conclusion: The study provides detailed understanding of LLMs’ probabilistic reasoning abilities and identifies key directions for future improvement, highlighting both capabilities and limitations.

Abstract: Despite widespread success in language understanding and generation, large language models (LLMs) exhibit unclear and often inconsistent behavior when faced with tasks that require probabilistic reasoning. In this work, we present the first comprehensive study of the reasoning capabilities of LLMs over explicit discrete probability distributions. Given observations from a probability distribution, we evaluate models on three carefully designed tasks, mode identification, maximum likelihood estimation, and sample generation, by prompting them to provide responses to queries about either the joint distribution or its conditionals. These tasks thus probe a range of probabilistic skills, including frequency analysis, marginalization, and generative behavior. Through comprehensive empirical evaluations, we demonstrate that there exists a clear performance gap between smaller and larger models, with the latter demonstrating stronger inference and surprising capabilities in sample generation. Furthermore, our investigations reveal notable limitations, including sensitivity to variations in the notation utilized to represent probabilistic outcomes and performance degradation of over 60% as context length increases. Together, our results provide a detailed understanding of the probabilistic reasoning abilities of LLMs and identify key directions for future improvement.

Method: Created GeoExplain dataset with panorama-geocoordinate-explanation tuples and developed SightSense - a multimodal, multilevel reasoning method that leverages hierarchical visual information and external knowledge for prediction and explanation generation.

Result: The proposed method demonstrates outstanding performance in the GeoGuess task, effectively handling the challenge of reasoning across hierarchical visual information and geographic knowledge.

Conclusion: GeoGuess serves as a valuable benchmark for evaluating multimodal reasoning capabilities, particularly the ability to integrate hierarchical visual clues with external knowledge for complex real-world tasks like geographic location identification.

Abstract: Multimodal reasoning is a process of understanding, integrating and inferring information across different data modalities. It has recently attracted surging academic attention as a benchmark for Artificial Intelligence (AI). Although there are various tasks for evaluating multimodal reasoning ability, they still have limitations. Lack of reasoning on hierarchical visual clues at different levels of granularity, e.g., local details and global context, is of little discussion, despite its frequent involvement in real scenarios. To bridge the gap, we introduce a novel and challenging task for multimodal reasoning, namely GeoGuess. Given a street view image, the task is to identify its location and provide a detailed explanation. A system that succeeds in GeoGuess should be able to detect tiny visual clues, perceive the broader landscape, and associate with vast geographic knowledge. Therefore, GeoGuess would require the ability to reason between hierarchical visual information and geographic knowledge. In this work, we establish a benchmark for GeoGuess by introducing a specially curated dataset GeoExplain which consists of panoramas-geocoordinates-explanation tuples. Additionally, we present a multimodal and multilevel reasoning method, namely SightSense which can make prediction and generate comprehensive explanation based on hierarchy of visual information and external knowledge. Our analysis and experiments demonstrate their outstanding performance in GeoGuess.

Method: Automated pipeline for MCQA creation including PDF parsing, semantic chunking, question generation, and model evaluation. Uses retrieval-augmented generation with paper-derived semantic chunks and reasoning traces distilled from GPT-4.1.

Result: Generated 16,000+ MCQs from 22,000 radiation/cancer biology papers. Reasoning-trace retrieval consistently improved performance, enabling several small models (1.1B-14B parameters) to surpass GPT-4 on the 2023 Astro Radiation and Cancer Biology exam.

Conclusion: The framework successfully creates scalable scientific benchmarks, and reasoning-trace retrieval is an effective technique for enhancing small language model performance on specialized scientific domains.

Abstract: As scientific knowledge grows at an unprecedented pace, evaluation benchmarks must evolve to reflect new discoveries and ensure language models are tested on current, diverse literature. We propose a scalable, modular framework for generating multiple-choice question-answering (MCQA) benchmarks directly from large corpora of scientific papers. Our pipeline automates every stage of MCQA creation, including PDF parsing, semantic chunking, question generation, and model evaluation. As a case study, we generate more than 16,000 MCQs from 22,000 open-access articles in radiation and cancer biology. We then evaluate a suite of small language models (1.1B-14B parameters) on these questions, comparing baseline accuracy with retrieval-augmented generation (RAG) from paper-derived semantic chunks and from reasoning traces distilled from GPT-4.1. We find that reasoning-trace retrieval consistently improves performance on both synthetic and expert-annotated benchmarks, enabling several small models to surpass GPT-4 on the 2023 Astro Radiation and Cancer Biology exam.

Method: RECAP (Reflect-Extract-Calibrate-Align-Produce) framework uses structured emotional reasoning with transparent appraisal-theoretic stages and per-dimension Likert signals to produce nuanced, auditable responses without model retraining.

Result: Across EmoBench, SECEU, and EQ-Bench, RECAP improves emotional reasoning by 22-28% on 8B models and 10-13% on larger models over zero-shot baselines. Clinician evaluations confirm superior empathetic communication.

Conclusion: RECAP demonstrates that modular, theory-grounded prompting can systematically enhance emotional intelligence in medical AI while preserving the accountability required for clinical deployment.

Abstract: Large language models in healthcare often miss critical emotional cues, delivering medically sound but emotionally flat advice. This is especially problematic in clinical contexts where patients are distressed and vulnerable, and require empathic communication to support safety, adherence, and trust. We present RECAP (Reflect-Extract-Calibrate-Align-Produce), an inference-time framework that adds structured emotional reasoning without retraining. By decomposing empathy into transparent appraisal-theoretic stages and exposing per-dimension Likert signals, RECAP produces nuanced, auditable responses. Across EmoBench, SECEU, and EQ-Bench, RECAP improves emotional reasoning by 22-28% on 8B models and 10-13% on larger models over zero-shot baselines. Clinician evaluations further confirm superior empathetic communication. RECAP shows that modular, theory-grounded prompting can systematically enhance emotional intelligence in medical AI while preserving the accountability required for deployment.

Method: Propose Judge Q method that trains soft tokens appended to input sequences. Only tunes the embedding layer at low cost. Soft tokens’ attention maps align with actual decoded tokens to capture global information for better KV importance evaluation.

Result: Under same eviction budget, shows less performance degradation. Improves LongBench by ~1 point and RULER by over 3 points. Validated on Llama-3.1-8B-Instruct and Mistral-7B-Instruct-v0.3 models.

Conclusion: The method effectively captures global information for KV cache eviction, maintains decoding quality, and can be easily integrated into existing models with minimal training overhead.

Abstract: Large language models (LLMs) utilize key-value (KV) cache to store historical information during sequence processing. The size of KV cache grows linearly as the length of the sequence extends, which seriously affects memory usage and decoding efficiency. Current methods for KV cache eviction typically utilize the last window from the pre-filling phase as queries to compute the KV importance scores for eviction. Although this scheme is simple to implement, it tends to overly focus on local information, potentially leading to the neglect or omission of crucial global information. To mitigate this issue, we propose Judge Q, a novel training method which incorporates a soft token list. This method only tunes the model’s embedding layer at a low training cost. By concatenating the soft token list at the end of the input sequence, we train these tokens’ attention map to the original input sequence to align with that of the actual decoded tokens. In this way, the queries corresponding to the soft tokens can effectively capture global information and better evaluate the importance of the keys and values within the KV cache, thus maintaining decoding quality when KV cache is evicted. Under the same eviction budget, our method exhibits less performance degradation compared to existing eviction approaches. We validate our approach through experiments conducted on models such as Llama-3.1-8B-Instruct and Mistral-7B-Instruct-v0.3, using benchmarks including LongBench, RULER, and Needle-in-a-Haystack. Results indicate an improvement of approximately 1 point on the LongBench and over 3 points on RULER. This proposed methodology can be seamlessly integrated into existing open-source models with minimal training overhead, thereby enhancing performance in KV cache eviction scenarios.

Method: Constructed an evaluation dataset of spoken queries by native Korean speakers, evaluated various multilingual ASR model configurations, and implemented a text-to-speech-based data augmentation method to improve recognition of specialized terms.

Result: The text-to-speech data augmentation method successfully improved recognition of specialized weather terminology while maintaining general-domain performance, addressing the identified limitations in domain-specific term recognition.

Conclusion: The work provides a foundation for future ASR advancements in the Korean weather forecasting domain through the created dataset, comprehensive evaluations, and effective augmentation technique.

Abstract: This paper explores integrating Automatic Speech Recognition (ASR) into natural language query systems to improve weather forecasting efficiency for Korean meteorologists. We address challenges in developing ASR systems for the Korean weather domain, specifically specialized vocabulary and Korean linguistic intricacies. To tackle these issues, we constructed an evaluation dataset of spoken queries recorded by native Korean speakers. Using this dataset, we assessed various configurations of a multilingual ASR model family, identifying performance limitations related to domain-specific terminology. We then implemented a simple text-to-speech-based data augmentation method, which improved the recognition of specialized terms while maintaining general-domain performance. Our contributions include creating a domain-specific dataset, comprehensive ASR model evaluations, and an effective augmentation technique. We believe our work provides a foundation for future advancements in ASR for the Korean weather forecasting domain.

Method: Proposed SEEED - Soft Clustering Extended Encoder-Based Error Detection with enhanced Soft Nearest Neighbor Loss and Label-Based Sample Ranking for better contrastive representation learning.

Result: Outperforms adapted baselines (GPT-4o, Phi-4) across multiple error-annotated dialogue datasets, improving unknown error detection accuracy by up to 8 points with strong generalization to unknown intent detection.

Conclusion: The framework effectively addresses limitations of current LLMs in error detection, providing robust performance for identifying unspecified errors in conversational AI systems.

Abstract: Although LLM-based conversational agents demonstrate strong fluency and coherence, they still produce undesirable behaviors (errors) that are challenging to prevent from reaching users during deployment. Recent research leverages large language models (LLMs) to detect errors and guide response-generation models toward improvement. However, current LLMs struggle to identify errors not explicitly specified in their instructions, such as those arising from updates to the response-generation model or shifts in user behavior. In this work, we introduce Automated Error Discovery, a framework for detecting and defining errors in conversational AI, and propose SEEED (Soft Clustering Extended Encoder-Based Error Detection), as an encoder-based approach to its implementation. We enhance the Soft Nearest Neighbor Loss by amplifying distance weighting for negative samples and introduce Label-Based Sample Ranking to select highly contrastive examples for better representation learning. SEEED outperforms adapted baselines – including GPT-4o and Phi-4 – across multiple error-annotated dialogue datasets, improving the accuracy for detecting unknown errors by up to 8 points and demonstrating strong generalization to unknown intent detection.

Method: Curated multi-source benchmark from Cochrane systematic reviews and clinical guidelines, tested GPT-4o-mini and GPT-5 models, analyzed performance patterns across sources, and evaluated retrieval-augmented prompting using gold-source abstracts and PubMed results.

Result: Accuracy highest on structured guidelines (90%), lower on narrative/systematic reviews (60-70%). Strong correlation between accuracy and citation count. Retrieval-augmented prompting with gold-source abstracts improved accuracy to 0.79, while PubMed abstracts showed limited benefit (0.23 accuracy).

Conclusion: LLMs show promise but have limitations for evidence-based clinical QA. Source clarity and targeted retrieval drive performance more than model size. Retrieval-augmented prompting improves factual accuracy, and stratified evaluation by specialty/question type is essential.

Abstract: Large Language Models (LLMs) have demonstrated substantial progress in biomedical and clinical applications, motivating rigorous evaluation of their ability to answer nuanced, evidence-based questions. We curate a multi-source benchmark drawing from Cochrane systematic reviews and clinical guidelines, including structured recommendations from the American Heart Association and narrative guidance used by insurers. Using GPT-4o-mini and GPT-5, we observe consistent performance patterns across sources and clinical domains: accuracy is highest on structured guideline recommendations (90%) and lower on narrative guideline and systematic review questions (60–70%). We also find a strong correlation between accuracy and the citation count of the underlying systematic reviews, where each doubling of citations is associated with roughly a 30% increase in the odds of a correct answer. Models show moderate ability to reason about evidence quality when contextual information is supplied. When we incorporate retrieval-augmented prompting, providing the gold-source abstract raises accuracy on previously incorrect items to 0.79; providing top 3 PubMed abstracts (ranked by semantic relevance) improves accuracy to 0.23, while random abstracts reduce accuracy (0.10, within temperature variation). These effects are mirrored in GPT-4o-mini, underscoring that source clarity and targeted retrieval – not just model size – drive performance. Overall, our results highlight both the promise and current limitations of LLMs for evidence-based clinical question answering. Retrieval-augmented prompting emerges as a useful strategy to improve factual accuracy and alignment with source evidence, while stratified evaluation by specialty and question type remains essential to understand current knowledge access and to contextualize model performance.

Method: GAPrune uses Fisher Information to measure parameter importance and general-domain gradient alignment to assess parameter behavior, combining these signals through Domain Alignment Importance (DAI) scoring to identify parameters that are either less important for domain tasks or create conflicts between domain and general objectives.

Result: At 50% sparsity with one-shot pruning, GAPrune maintains performance within 2.5% of dense models. With 100-step retraining, it achieves +4.51% improvement on FinMTEB and +1.73% on ChemTEB, outperforming all baselines.

Conclusion: Principled pruning strategies can achieve both model compression and enhanced domain specialization, providing a new approach for developing efficient domain-specific embedding models.

Abstract: Domain-specific embedding models have shown promise for applications that require specialized semantic understanding, such as coding agents and financial retrieval systems, often achieving higher performance gains than general models. However, state-of-the-art embedding models are typically based on LLMs, which contain billions of parameters, making deployment challenging in resource-constrained environments. Model compression through pruning offers a promising solution, but existing pruning methods treat all parameters uniformly, failing to distinguish between general semantic representations and domain-specific patterns, leading to suboptimal pruning decisions. Thus, we propose GAPrune, a pruning framework that addresses this challenge by considering both domain importance and preserving general linguistic foundation. Our method uses Fisher Information to measure importance and general-domain gradient alignment to assess parameter behavior, then combines these signals using our Domain Alignment Importance (DAI) scoring. Lower DAI scores indicate that the parameter is either less important for the domain task or creates conflicts between domain and general objectives. Experiments on two domain benchmarks, FinMTEB and ChemTEB, show that GAPrune maintains performance within 2.5% of dense models in one-shot pruning at 50% sparsity, while outperforming all baselines. With retraining in 100 steps, GAPrune achieves +4.51% improvement on FinMTEB and +1.73% on ChemTEB, demonstrating that our pruning strategy not only preserves but enhances domain-specific capabilities. Our findings demonstrate that principled pruning strategies can achieve model compression and enhanced domain specialization, providing the research community with a new approach for development.

Method: Used electroencephalography (EEG) to compare neural processing of humor from AI versus human sources, analyzing behavioral ratings and neurophysiological responses including N400 and LPP components.

Result: Participants rated AI and human humor as equally funny behaviorally, but neurophysiologically: AI humor showed smaller N400 (reduced cognitive effort), larger LPP (greater surprise/emotional response), and improving patterns over time (decreasing N400, increasing LPP) unlike human humor which showed habituation.

Conclusion: The brain responds to AI humor with surprisingly positive and intense reactions, demonstrating how cognitive adaptation to AI’s language patterns can lead to intensified emotional reward and challenging algorithm aversion in humor contexts.

Abstract: As AI companions become capable of human-like communication, including telling jokes, understanding how people cognitively and emotionally respond to AI humor becomes increasingly important. This study used electroencephalography (EEG) to compare how people process humor from AI versus human sources. Behavioral analysis revealed that participants rated AI and human humor as comparably funny. However, neurophysiological data showed that AI humor elicited a smaller N400 effect, suggesting reduced cognitive effort during the processing of incongruity. This was accompanied by a larger Late Positive Potential (LPP), indicating a greater degree of surprise and emotional response. This enhanced LPP likely stems from the violation of low initial expectations regarding AI’s comedic capabilities. Furthermore, a key temporal dynamic emerged: human humor showed habituation effects, marked by an increasing N400 and a decreasing LPP over time. In contrast, AI humor demonstrated increasing processing efficiency and emotional reward, with a decreasing N400 and an increasing LPP. This trajectory reveals how the brain can dynamically update its predictive model of AI capabilities. This process of cumulative reinforcement challenges “algorithm aversion” in humor, as it demonstrates how cognitive adaptation to AI’s language patterns can lead to an intensified emotional reward. Additionally, participants’ social attitudes toward AI modulated these neural responses, with higher perceived AI trustworthiness correlating with enhanced emotional engagement. These findings indicate that the brain responds to AI humor with surprisingly positive and intense reactions, highlighting humor’s potential for fostering genuine engagement in human-AI social interaction.

Method: PREMem extracts fine-grained memory fragments categorized into factual, experiential, and subjective information, then establishes explicit relationships between memory items across sessions to capture evolution patterns like extensions, transformations, and implications.

Result: Experiments show significant performance improvements across all model sizes, with smaller models achieving results comparable to much larger baselines while maintaining effectiveness even with constrained token budgets.

Conclusion: By performing reasoning during pre-storage rather than response generation, PREMem creates enriched memory representations while reducing computational demands during interactions, enabling more efficient and scalable conversational AI systems.

Abstract: Effective long-term memory in conversational AI requires synthesizing information across multiple sessions. However, current systems place excessive reasoning burden on response generation, making performance significantly dependent on model sizes. We introduce PREMem (Pre-storage Reasoning for Episodic Memory), a novel approach that shifts complex reasoning processes from inference to memory construction. PREMem extracts fine-grained memory fragments categorized into factual, experiential, and subjective information; it then establishes explicit relationships between memory items across sessions, capturing evolution patterns like extensions, transformations, and implications. By performing this reasoning during pre-storage rather than when generating a response, PREMem creates enriched representations while reducing computational demands during interactions. Experiments show significant performance improvements across all model sizes, with smaller models achieving results comparable to much larger baselines while maintaining effectiveness even with constrained token budgets. Code and dataset are available at https://github.com/sangyeop-kim/PREMem.

Method: Cross-linguistic approach using probabilities to assess interpretive likelihood, with human similarity scores to quantify how closely LLMs emulate human performance across language groups.

Result: Most LLMs prefer surface scope interpretations (aligning with human tendencies), while only some differentiate between English and Chinese in inverse scope preferences. Model architecture, scale, and pre-training data language background significantly influence human approximation.

Conclusion: LLMs show notable potential to align with human quantifier scope interpretations, but their approximation varies based on technical factors, with overall preference for surface scope mirroring human tendencies.

Abstract: Sentences with multiple quantifiers often lead to interpretive ambiguities, which can vary across languages. This study adopts a cross-linguistic approach to examine how large language models (LLMs) handle quantifier scope interpretation in English and Chinese, using probabilities to assess interpretive likelihood. Human similarity (HS) scores were used to quantify the extent to which LLMs emulate human performance across language groups. Results reveal that most LLMs prefer the surface scope interpretations, aligning with human tendencies, while only some differentiate between English and Chinese in the inverse scope preferences, reflecting human-similar patterns. HS scores highlight variability in LLMs’ approximation of human behavior, but their overall potential to align with humans is notable. Differences in model architecture, scale, and particularly models’ pre-training data language background, significantly influence how closely LLMs approximate human quantifier scope interpretations.

Method: Structurally separates content and form, generates section-wise note content conditioned on DP medical terms with separate DP constraints, and uses a DP quality maximizer to select high-quality outputs.

Result: Produces synthetic notes with statistical properties closely aligned with real clinical notes. Classification models trained on synthetic data perform comparably to those trained on real data.

Conclusion: Term2Note achieves substantial improvements over existing DP text generation methods with fewer assumptions, making it a viable privacy-preserving alternative for clinical data.

Abstract: Training data is fundamental to the success of modern machine learning models, yet in high-stakes domains such as healthcare, the use of real-world training data is severely constrained by concerns over privacy leakage. A promising solution to this challenge is the use of differentially private (DP) synthetic data, which offers formal privacy guarantees while maintaining data utility. However, striking the right balance between privacy protection and utility remains challenging in clinical note synthesis, given its domain specificity and the complexity of long-form text generation. In this paper, we present Term2Note, a methodology to synthesise long clinical notes under strong DP constraints. By structurally separating content and form, Term2Note generates section-wise note content conditioned on DP medical terms, with each governed by separate DP constraints. A DP quality maximiser further enhances synthetic notes by selecting high-quality outputs. Experimental results show that Term2Note produces synthetic notes with statistical properties closely aligned with real clinical notes, demonstrating strong fidelity. In addition, multi-label classification models trained on these synthetic notes perform comparably to those trained on real data, confirming their high utility. Compared to existing DP text generation baselines, Term2Note achieves substantial improvements in both fidelity and utility while operating under fewer assumptions, suggesting its potential as a viable privacy-preserving alternative to using sensitive clinical notes.

Method: Developed CultureSynth framework with (1) hierarchical multilingual cultural taxonomy covering 12 primary/130 secondary topics, and (2) RAG-based methodology using factual knowledge to synthesize culturally relevant question-answer pairs.

Result: Created CultureSynth-7 benchmark with 19,360 entries (4,149 manually verified) across 7 languages. Evaluation of 14 LLMs showed performance stratification led by ChatGPT-4o-Latest and Qwen2.5-72B-Instruct, with 3B-parameter threshold needed for basic cultural competence, architectural biases in knowledge processing, and significant geographic disparities.

Conclusion: CultureSynth provides a scalable framework for developing culturally aware AI systems while reducing manual annotation reliance, enabling better assessment of LLM cultural competence across diverse contexts.

Abstract: Cultural competence, defined as the ability to understand and adapt to multicultural contexts, is increasingly vital for large language models (LLMs) in global environments. While several cultural benchmarks exist to assess LLMs' cultural competence, current evaluations suffer from fragmented taxonomies, domain specificity, and heavy reliance on manual data annotation. To address these limitations, we introduce CultureSynth, a novel framework comprising (1) a comprehensive hierarchical multilingual cultural taxonomy covering 12 primary and 130 secondary topics, and (2) a Retrieval-Augmented Generation (RAG)-based methodology leveraging factual knowledge to synthesize culturally relevant question-answer pairs. The CultureSynth-7 synthetic benchmark contains 19,360 entries and 4,149 manually verified entries across 7 languages. Evaluation of 14 prevalent LLMs of different sizes reveals clear performance stratification led by ChatGPT-4o-Latest and Qwen2.5-72B-Instruct. The results demonstrate that a 3B-parameter threshold is necessary for achieving basic cultural competence, models display varying architectural biases in knowledge processing, and significant geographic disparities exist across models. We believe that CultureSynth offers a scalable framework for developing culturally aware AI systems while reducing reliance on manual annotation\footnote{Benchmark is available at https://github.com/Eyr3/CultureSynth.}.

Method: Uses lightweight ontology design, formal pattern modeling, and large language models to convert normative principles into RDF templates for extracting ESG events from news content and building structured knowledge graphs.

Result: Creates a scalable and transparent framework that links reported ESG incidents to specific international sustainability principles, enabling better interpretation of non-compliance.

Conclusion: The approach provides an effective solution for aligning unstructured ESG news data with principle-based normative frameworks through structured knowledge representation.

Abstract: The growing importance of environmental, social, and governance data in regulatory and investment contexts has increased the need for accurate, interpretable, and internationally aligned representations of non-financial risks, particularly those reported in unstructured news sources. However, aligning such controversy-related data with principle-based normative frameworks, such as the United Nations Global Compact or Sustainable Development Goals, presents significant challenges. These frameworks are typically expressed in abstract language, lack standardized taxonomies, and differ from the proprietary classification systems used by commercial data providers. In this paper, we present a semi-automatic method for constructing structured knowledge representations of environmental, social, and governance events reported in the news. Our approach uses lightweight ontology design, formal pattern modeling, and large language models to convert normative principles into reusable templates expressed in the Resource Description Framework. These templates are used to extract relevant information from news content and populate a structured knowledge graph that links reported incidents to specific framework principles. The result is a scalable and transparent framework for identifying and interpreting non-compliance with international sustainability guidelines.

Method: Two-stage approach: 1) Fine-tuning a large language model on benchmark datasets curated for spotlight generation, 2) Alignment via Direct Preference Optimization (DPO) to enhance quality.

Result: The resulting model demonstrates precise identification of key elements, improved readability, and significantly enhanced engagement value of original documents.

Conclusion: Spotlight represents an effective paradigm shift from comprehensive summarization to engaging narrative generation, successfully boosting reader engagement through selective emphasis of compelling content.

Abstract: In this paper, we introduce Spotlight, a novel paradigm for information extraction that produces concise, engaging narratives by highlighting the most compelling aspects of a document. Unlike traditional summaries, which prioritize comprehensive coverage, spotlights selectively emphasize intriguing content to foster deeper reader engagement with the source material. We formally differentiate spotlights from related constructs and support our analysis with a detailed benchmarking study using new datasets curated for this work. To generate high-quality spotlights, we propose a two-stage approach: fine-tuning a large language model on our benchmark data, followed by alignment via Direct Preference Optimization (DPO). Our comprehensive evaluation demonstrates that the resulting model not only identifies key elements with precision but also enhances readability and boosts the engagement value of the original document.

Method: Created synthetic dataset by augmenting real headlines with predefined clickbait strategies. Two-stage framework: 1) Detection using fine-tuned BERT vs LLMs (GPT-4.0, Gemini 2.4 Flash) with zero/few-shot prompting, 2) Strategy attribution using dedicated BERT classifier.

Result: Developed an explainable clickbait analysis system that not only detects but also explains the specific manipulation tactics used in clickbait headlines.

Conclusion: This work advances transparent AI systems for combating manipulative media content, with the dataset shared publicly for research community use.

Abstract: The proliferation of clickbait headlines poses significant challenges to the credibility of information and user trust in digital media. While recent advances in machine learning have improved the detection of manipulative content, the lack of explainability limits their practical adoption. This paper presents a model for explainable clickbait detection that not only identifies clickbait titles but also attributes them to specific linguistic manipulation strategies. We introduce a synthetic dataset generated by systematically augmenting real news headlines using a predefined catalogue of clickbait strategies. This dataset enables controlled experimentation and detailed analysis of model behaviour. We present a two-stage framework for automatic clickbait analysis comprising detection and tactic attribution. In the first stage, we compare a fine-tuned BERT classifier with large language models (LLMs), specifically GPT-4.0 and Gemini 2.4 Flash, under both zero-shot prompting and few-shot prompting enriched with illustrative clickbait headlines and their associated persuasive tactics. In the second stage, a dedicated BERT-based classifier predicts the specific clickbait strategies present in each headline. This work advances the development of transparent and trustworthy AI systems for combating manipulative media content. We share the dataset with the research community at https://github.com/LLM-HITCS25S/ClickbaitTacticsDetection

Method: Created a dataset of 350 Reddit samples with images, user text, and emotion categories. Designed hierarchical tasks with 6 multiple-choice and 1 open-ended question per sample, progressing from perception (basic visual elements) to cognition (scene reasoning, intent understanding, empathy). Used AI assistance (Claude 4) and manual verification for annotation quality.

Result: The benchmark provides a structured framework to assess MLLMs’ emotion understanding capabilities across different difficulty levels, from basic perception to advanced cognitive tasks requiring empathy and contextual integration.

Conclusion: EmoBench-Reddit addresses the gap in evaluating multimodal emotion understanding and provides a comprehensive benchmark for assessing MLLMs’ ability to handle complex subjective human emotions through hierarchical task design.

Abstract: With the rapid advancement of Multimodal Large Language Models (MLLMs), they have demonstrated exceptional capabilities across a variety of vision-language tasks. However, current evaluation benchmarks predominantly focus on objective visual question answering or captioning, inadequately assessing the models' ability to understand complex and subjective human emotions. To bridge this gap, we introduce EmoBench-Reddit, a novel, hierarchical benchmark for multimodal emotion understanding. The dataset comprises 350 meticulously curated samples from the social media platform Reddit, each containing an image, associated user-provided text, and an emotion category (sad, humor, sarcasm, happy) confirmed by user flairs. We designed a hierarchical task framework that progresses from basic perception to advanced cognition, with each data point featuring six multiple-choice questions and one open-ended question of increasing difficulty. Perception tasks evaluate the model’s ability to identify basic visual elements (e.g., colors, objects), while cognition tasks require scene reasoning, intent understanding, and deep empathy integrating textual context. We ensured annotation quality through a combination of AI assistance (Claude 4) and manual verification.

Method: Fluid Benchmarking uses item response theory to map LM performance into latent ability space and implements dynamic item selection similar to computerized adaptive testing in education, adapting evaluation to each LM’s capability level.

Result: Fluid Benchmarking achieves superior performance across four dimensions - efficiency, validity, variance, and saturation. It demonstrates higher validity and less variance on MMLU with fifty times fewer items compared to random sampling and other baselines.

Conclusion: LM benchmarking can be substantially improved by moving beyond static evaluation to adaptive approaches like Fluid Benchmarking, which combines item response theory for increased validity and dynamic item selection for reduced variance.

Abstract: Language model (LM) benchmarking faces several challenges: comprehensive evaluations are costly, benchmarks often fail to measure the intended capabilities, and evaluation quality can degrade due to labeling errors and benchmark saturation. Although various strategies have been proposed to mitigate these issues, they tend to address individual aspects in isolation, neglecting broader questions about overall evaluation quality. Here, we introduce Fluid Benchmarking, a new evaluation approach that advances LM benchmarking across multiple dimensions. Inspired by psychometrics, Fluid Benchmarking is based on the insight that the relative value of benchmark items depends on an LM’s capability level, suggesting that evaluation should adapt to each LM. Methodologically, Fluid Benchmarking estimates an item response model based on existing LM evaluation results and uses the inferred quantities to select evaluation items dynamically, similar to computerized adaptive testing in education. In our experiments, we compare Fluid Benchmarking against the common practice of random item sampling as well as more sophisticated baselines, including alternative methods grounded in item response theory. We examine four dimensions – efficiency, validity, variance, and saturation – and find that Fluid Benchmarking achieves superior performance in all of them (e.g., higher validity and less variance on MMLU with fifty times fewer items). Our analysis shows that the two components of Fluid Benchmarking have distinct effects: item response theory, used to map performance into a latent ability space, increases validity, while dynamic item selection reduces variance. Overall, our results suggest that LM benchmarking can be substantially improved by moving beyond static evaluation.

Method: Created PACT dataset using LLMs with three personality profiles (Argumentation, Preference, Buying Style) for tourism negotiations. Conducted comparative experiments between pre-trained and fine-tuned LLMs for the PAN-DG task.

Result: Automatic and manual evaluations confirmed high-quality dialogues in PACT dataset. Fine-tuned LLMs effectively generated personality-driven rational responses during negotiations, outperforming pre-trained models.

Conclusion: PACT dataset successfully enhances personalization and reasoning in negotiation dialogue systems, establishing a foundation for future research in personality-driven argumentation-based negotiation.

Abstract: Integrating argumentation mechanisms into negotiation dialogue systems improves conflict resolution through exchanges of arguments and critiques. Moreover, incorporating personality attributes enhances adaptability by aligning interactions with individuals’ preferences and styles. To advance these capabilities in negotiation dialogue systems, we propose a novel Personality-driven Argumentation-based Negotiation Dialogue Generation (PAN-DG) task. To support this task, we introduce PACT, a dataset of Personality-driven Argumentation-based negotiation Conversations for Tourism sector. This dataset, generated using Large Language Models (LLMs), features three distinct personality profiles, viz. Argumentation Profile, Preference Profile, and Buying Style Profile to simulate a variety of negotiation scenarios involving diverse personalities. Thorough automatic and manual evaluations indicate that the dataset comprises high-quality dialogues. Further, we conduct comparative experiments between pre-trained and fine-tuned LLMs for the PAN-DG task. Multi-dimensional evaluation demonstrates that the fine-tuned LLMs effectively generate personality-driven rational responses during negotiations. This underscores the effectiveness of PACT in enhancing personalization and reasoning capabilities in negotiation dialogue systems, thereby establishing a foundation for future research in this domain.

Method: Used Qwen-3 (8B) model with data from U.S. presidential debates, testing six fallacy types. Compared two Chain-of-Thought frameworks (Pragma-Dialectics and Periodic Table of Arguments) against baseline prompts under three input conditions: text-only, text with context, and text with context plus audio-based emotional tone metadata.

Result: Theoretical prompting improved interpretability but context and emotional tone metadata often lowered performance. Emotional tone metadata biased the model toward labeling statements as Appeal to Emotion fallacies, worsening logical reasoning. Basic prompts frequently outperformed enhanced ones.

Conclusion: Attention dilution from added inputs may worsen fallacy classification in LLMs, suggesting that simpler prompts without additional context or emotional metadata are often more effective for logical reasoning tasks.

Abstract: This study investigates how context and emotional tone metadata influence large language model (LLM) reasoning and performance in fallacy classification tasks, particularly within political debate settings. Using data from U.S. presidential debates, we classify six fallacy types through various prompting strategies applied to the Qwen-3 (8B) model. We introduce two theoretically grounded Chain-of-Thought frameworks: Pragma-Dialectics and the Periodic Table of Arguments, and evaluate their effectiveness against a baseline prompt under three input settings: text-only, text with context, and text with both context and audio-based emotional tone metadata. Results suggest that while theoretical prompting can improve interpretability and, in some cases, accuracy, the addition of context and especially emotional tone metadata often leads to lowered performance. Emotional tone metadata biases the model toward labeling statements as \textit{Appeal to Emotion}, worsening logical reasoning. Overall, basic prompts often outperformed enhanced ones, suggesting that attention dilution from added inputs may worsen rather than improve fallacy classification in LLMs.

Method: Automated construction of prompts with emojis to express toxic intent subtly, tested across 5 languages on 7 LLMs including jailbreak tasks, followed by model-level interpretations spanning semantic cognition, sequence generation, and tokenization analysis.

Result: Prompts with emojis easily induce toxicity generation in LLMs, with emojis acting as heterogeneous semantic channels that bypass safety mechanisms, and pre-training corpus analysis reveals correlations with emoji-related data pollution.

Conclusion: Emojis present a significant vulnerability in LLM safety mechanisms, serving as unexpected triggers for toxic content generation that requires attention in model development and safety protocols.

Abstract: Emojis are globally used non-verbal cues in digital communication, and extensive research has examined how large language models (LLMs) understand and utilize emojis across contexts. While usually associated with friendliness or playfulness, it is observed that emojis may trigger toxic content generation in LLMs. Motivated by such a observation, we aim to investigate: (1) whether emojis can clearly enhance the toxicity generation in LLMs and (2) how to interpret this phenomenon. We begin with a comprehensive exploration of emoji-triggered LLM toxicity generation by automating the construction of prompts with emojis to subtly express toxic intent. Experiments across 5 mainstream languages on 7 famous LLMs along with jailbreak tasks demonstrate that prompts with emojis could easily induce toxicity generation. To understand this phenomenon, we conduct model-level interpretations spanning semantic cognition, sequence generation and tokenization, suggesting that emojis can act as a heterogeneous semantic channel to bypass the safety mechanisms. To pursue deeper insights, we further probe the pre-training corpus and uncover potential correlation between the emoji-related data polution with the toxicity generation behaviors. Supplementary materials provide our implementation code and data. (Warning: This paper contains potentially sensitive contents)

Method: Text2Mem defines a compact yet expressive operation set with JSON-based schema instances, includes parser for typed operation objects, validator for correctness, and adapters for backend mapping with integrated model services.

Result: The design ensures safety, determinism, and portability across heterogeneous backends through unified execution contract, with planned Text2Mem Bench benchmark for systematic evaluation.

Conclusion: Text2Mem establishes the first standardized foundation for memory control in agents, providing reliable execution from natural language commands.

Abstract: Large language model agents increasingly depend on memory to sustain long horizon interaction, but existing frameworks remain limited. Most expose only a few basic primitives such as encode, retrieve, and delete, while higher order operations like merge, promote, demote, split, lock, and expire are missing or inconsistently supported. Moreover, there is no formal and executable specification for memory commands, leaving scope and lifecycle rules implicit and causing unpredictable behavior across systems. We introduce Text2Mem, a unified memory operation language that provides a standardized pathway from natural language to reliable execution. Text2Mem defines a compact yet expressive operation set aligned with encoding, storage, and retrieval. Each instruction is represented as a JSON based schema instance with required fields and semantic invariants, which a parser transforms into typed operation objects with normalized parameters. A validator ensures correctness before execution, while adapters map typed objects either to a SQL prototype backend or to real memory frameworks. Model based services such as embeddings or summarization are integrated when required. All results are returned through a unified execution contract. This design ensures safety, determinism, and portability across heterogeneous backends. We also outline Text2Mem Bench, a planned benchmark that separates schema generation from backend execution to enable systematic evaluation. Together, these components establish the first standardized foundation for memory control in agents.

Method: Tuned five LLMs with three corpora under four privacy levels, assessed text length, grammatical correctness, lexical diversity, and utility in downstream classification tasks (book genre recognition and cause of death recognition).

Result: Stronger privacy constraints led to texts that are shorter by at least 77%, less grammatically correct by at least 9%, less diverse by at least 10% in bi-gram diversity, and reduced accuracy in downstream classification tasks.

Conclusion: DP fine-tuning significantly degrades text quality and utility, which may limit the usefulness of synthetic data generated for privacy-preserving applications.

Abstract: Ensuring user privacy by synthesizing data from large language models (LLMs) tuned under differential privacy (DP) has become popular recently. However, the impact of DP fine-tuned LLMs on the quality of the language and the utility of the texts they produce has not been investigated. In this work, we tune five LLMs with three corpora under four levels of privacy and assess the length, the grammatical correctness, and the lexical diversity of the text outputs they produce. We also probe the utility of the synthetic outputs in downstream classification tasks such as book genre recognition based on book descriptions and cause of death recognition based on verbal autopsies. The results indicate that LLMs tuned under stronger privacy constrains produce texts that are shorter by at least 77 %, that are less grammatically correct by at least 9 %, and are less diverse by at least 10 % in bi-gram diversity. Furthermore, the accuracy they reach in downstream classification tasks decreases, which might be detrimental to the usefulness of the generated synthetic data.

Method: Optimal Brain Restoration (OBR) uses second-order Hessian objective with surrogate approximation and group error compensation to align pruning and quantization.

Result: Enables W4A4KV4 quantization with 50% sparsity, achieving 4.72x speedup and 6.4x memory reduction vs FP16-dense baseline.

Conclusion: OBR successfully addresses conflicting requirements of joint quantization-sparsity compression through error compensation, enabling aggressive LLM compression.

Abstract: Recent advances in Large Language Model (LLM) compression, such as quantization and pruning, have achieved notable success. However, as these techniques gradually approach their respective limits, relying on a single method for further compression has become increasingly challenging. In this work, we explore an alternative solution by combining quantization and sparsity. This joint approach, though promising, introduces new difficulties due to the inherently conflicting requirements on weight distributions: quantization favors compact ranges, while pruning benefits from high variance. To attack this problem, we propose Optimal Brain Restoration (OBR), a general and training-free framework that aligns pruning and quantization by error compensation between both. OBR minimizes performance degradation on downstream tasks by building on a second-order Hessian objective, which is then reformulated into a tractable problem through surrogate approximation and ultimately reaches a closed-form solution via group error compensation. Experiments show that OBR enables aggressive W4A4KV4 quantization with 50% sparsity on existing LLMs, and delivers up to 4.72x speedup and 6.4x memory reduction compared to the FP16-dense baseline.

Method: Built on PubMedBERT, RAT integrates random sampling mechanisms with adversarial training principles to enhance generalization and robustness while reducing computational requirements.

Result: RAT demonstrates superior performance compared to baseline models across various BioIE tasks while significantly reducing computational costs.

Conclusion: RAT offers a balanced solution for biomedical NLP, providing both improved model performance and computational efficiency, making it a transformative framework for BioIE applications.

Abstract: We introduce random adversarial training (RAT), a novel framework successfully applied to biomedical information extraction (BioIE) tasks. Building on PubMedBERT as the foundational architecture, our study first validates the effectiveness of conventional adversarial training in enhancing pre-trained language models’ performance on BioIE tasks. While adversarial training yields significant improvements across various performance metrics, it also introduces considerable computational overhead. To address this limitation, we propose RAT as an efficiency solution for biomedical information extraction. This framework strategically integrates random sampling mechanisms with adversarial training principles, achieving dual objectives: enhanced model generalization and robustness while significantly reducing computational costs. Through comprehensive evaluations, RAT demonstrates superior performance compared to baseline models in BioIE tasks. The results highlight RAT’s potential as a transformative framework for biomedical natural language processing, offering a balanced solution to the model performance and computational efficiency.

Method: The authors analyze and distill 58 different prompt engineering techniques from the 2025 Prompt Report to focus on 6 core approaches: zero-shot, few-shot, thought generation, ensembling, self-criticism, and decomposition. They provide detailed recommendations on prompt structure, address common pitfalls, and examine context limitations and agentic tools across major AI platforms.

Result: The paper delivers actionable guidelines for structuring prompts effectively, identifies and addresses common issues like multi-turn degradation and hallucinations, and provides platform-specific analysis of tools like Claude Code and Deep Research capabilities across OpenAI, Google, Anthropic, and Perplexity.

Conclusion: Prompt engineering should augment rather than replace established research practices, and systematic application of core techniques can transition researchers from opportunistic prompting to effective, low-friction practices that contribute to higher quality life sciences research outcomes.

Abstract: Developing effective prompts demands significant cognitive investment to generate reliable, high-quality responses from Large Language Models (LLMs). By deploying case-specific prompt engineering techniques that streamline frequently performed life sciences workflows, researchers could achieve substantial efficiency gains that far exceed the initial time investment required to master these techniques. The Prompt Report published in 2025 outlined 58 different text-based prompt engineering techniques, highlighting the numerous ways prompts could be constructed. To provide actionable guidelines and reduce the friction of navigating these various approaches, we distil this report to focus on 6 core techniques: zero-shot, few-shot approaches, thought generation, ensembling, self-criticism, and decomposition. We breakdown the significance of each approach and ground it in use cases relevant to life sciences, from literature summarization and data extraction to editorial tasks. We provide detailed recommendations for how prompts should and shouldn’t be structured, addressing common pitfalls including multi-turn conversation degradation, hallucinations, and distinctions between reasoning and non-reasoning models. We examine context window limitations, agentic tools like Claude Code, while analyzing the effectiveness of Deep Research tools across OpenAI, Google, Anthropic and Perplexity platforms, discussing current limitations. We demonstrate how prompt engineering can augment rather than replace existing established individual practices around data processing and document editing. Our aim is to provide actionable guidance on core prompt engineering principles, and to facilitate the transition from opportunistic prompting to an effective, low-friction systematic practice that contributes to higher quality research.

Method: Multi-stage filtering approach using three language models on 3.01M web-crawled questions, followed by GPT-4o refinement and human validation to create 441 high-quality Korean question-answer pairs with cultural elements.

Result: Best model achieved 83.22% accuracy while weakest reached 59.86%, with models particularly struggling with culturally specific scenarios containing traditional Korean elements like kimchi, hanbok, and kimchi refrigerators.

Conclusion: Ko-PIQA serves as both a benchmark for Korean language models and a foundation for more inclusive commonsense reasoning research, highlighting the importance of culturally diverse datasets.

Abstract: Physical commonsense reasoning datasets like PIQA are predominantly English-centric and lack cultural diversity. We introduce Ko-PIQA, a Korean physical commonsense reasoning dataset that incorporates cultural context. Starting from 3.01 million web-crawled questions, we employed a multi-stage filtering approach using three language models to identify 11,553 PIQA-style questions. Through GPT-4o refinement and human validation, we obtained 441 high-quality question-answer pairs. A key feature of Ko-PIQA is its cultural grounding: 19.7% of questions contain culturally specific elements like traditional Korean foods (kimchi), clothing (hanbok), and specialized appliances (kimchi refrigerators) that require culturally-aware reasoning beyond direct translation. We evaluate seven language models on Ko-PIQA, with the best model achieving 83.22% accuracy while the weakest reaches only 59.86%, demonstrating significant room for improvement. Models particularly struggle with culturally specific scenarios, highlighting the importance of culturally diverse datasets. Ko-PIQA serves as both a benchmark for Korean language models and a foundation for more inclusive commonsense reasoning research. The dataset and code will be publicly available.

Method: For multiple-choice QA: Gemini 2.5 Flash with few-shot prompting, dataset preprocessing, and ensemble of three prompt configurations. For open-ended QA: unified prompt with role-playing as Arabic medical expert, few-shot examples, and post-processing for concise responses.

Result: Achieved 2nd place in both Sub-Task 1 (multiple-choice QA) and Sub-Task 2 (open-ended QA) in the AraHealthQA-2025 shared task, demonstrating strong performance across standard, biased, fill-in-the-blank, patient-doctor Q&A, GEC, and paraphrased question variants.

Conclusion: The Gemini 2.5 Flash model with carefully designed prompting strategies, ensemble methods, and role-playing techniques is highly effective for Arabic health question answering tasks, showing robust performance across diverse clinical question formats and contexts.

Abstract: We present our systems for Track 2 (General Arabic Health QA, MedArabiQ) of the AraHealthQA-2025 shared task, where our methodology secured 2nd place in both Sub-Task 1 (multiple-choice question answering) and Sub-Task 2 (open-ended question answering) in Arabic clinical contexts. For Sub-Task 1, we leverage the Gemini 2.5 Flash model with few-shot prompting, dataset preprocessing, and an ensemble of three prompt configurations to improve classification accuracy on standard, biased, and fill-in-the-blank questions. For Sub-Task 2, we employ a unified prompt with the same model, incorporating role-playing as an Arabic medical expert, few-shot examples, and post-processing to generate concise responses across fill-in-the-blank, patient-doctor Q&A, GEC, and paraphrased variants.

Method: Used multiple non-transformer architectures (PA-LSTM, C-GCN, AGGCN) and transformer architectures (BERT, RoBERTa, R-BERT) evaluated on TACRED, TACREV, and RE-TACRED datasets with traditional metrics like micro F1, varying sentence lengths, and different training dataset percentages.

Result: Transformer-based models achieved significantly higher micro F1 scores (80-90%) compared to non-transformer models (64-67%), demonstrating superior performance across different evaluation scenarios.

Conclusion: Transformer architectures provide substantial performance improvements in relation extraction tasks, and the paper also reviews the research journey in supervised relation classification and discusses the current role of large language models in this domain.

Abstract: In the era of large language model, relation extraction (RE) plays an important role in information extraction through the transformation of unstructured raw text into structured data (Wadhwa et al., 2023). In this paper, we systematically compare the performance of deep supervised learning approaches without transformers and those with transformers. We used a series of non-transformer architectures such as PA-LSTM(Zhang et al., 2017), C-GCN(Zhang et al., 2018), and AGGCN(attention guide GCN)(Guo et al., 2019), and a series of transformer architectures such as BERT, RoBERTa, and R-BERT(Wu and He, 2019). Our comparison included traditional metrics like micro F1, as well as evaluations in different scenarios, varying sentence lengths, and different percentages of the dataset for training. Our experiments were conducted on TACRED, TACREV, and RE-TACRED. The results show that transformer-based models outperform non-transformer models, achieving micro F1 scores of 80-90% compared to 64-67% for non-transformer models. Additionally, we briefly review the research journey in supervised relation classification and discuss the role and current status of large language models (LLMs) in relation extraction.

Method: Proposed Layer-Selective LoRA (LayRA) that adds Low-Rank Adapters to selected initial and final layers while freezing intermediate layers, based on insights about language encoding patterns in multilingual models.

Result: LayRA provides the best tradeoff between preserving existing language capabilities and learning new languages, outperforming existing approaches like standard LoRA, and enables instruction following without target language data.

Conclusion: LayRA is an effective method for continually expanding multilingual models to new languages while mitigating catastrophic forgetting, with practical applications for model adaptation without access to original training data.

Abstract: Multilingual language models are trained on a fixed set of languages, and to support new languages, the models need to be retrained from scratch. This is an expensive endeavor and is often infeasible, as model developers tend not to release their pre-training data. Naive approaches, such as continued pretraining, suffer from catastrophic forgetting; however, mitigation strategies like experience replay cannot be applied due to the lack of original pretraining data. In this work, we investigate the problem of continually adding new languages to a multilingual model, assuming access to pretraining data in only the target languages. We explore multiple approaches to address this problem and propose Layer-Selective LoRA (LayRA), which adds Low-Rank Adapters (LoRA) to selected initial and final layers while keeping the rest of the model frozen. LayRA builds on two insights: (1) LoRA reduces forgetting, and (2) multilingual models encode inputs in the source language in the initial layers, reason in English in intermediate layers, and translate back to the source language in final layers. We experiment with adding multiple combinations of Galician, Swahili, and Urdu to pretrained language models and evaluate each method on diverse multilingual tasks. We find that LayRA provides the overall best tradeoff between preserving models’ capabilities in previously supported languages, while being competitive with existing approaches such as LoRA in learning new languages. We also demonstrate that using model arithmetic, the adapted models can be equipped with strong instruction following abilities without access to any instruction tuning data in the target languages.

Method: Used compressed transformer models (DistilRoBERTa, DistilBERT, MiniLM, ELECTRA, TinyBERT) with Llama-3-8B for annotation, employing stratified 5-fold cross-validation on Twitter, Reddit, and news media data.

Result: DistilRoBERTa achieved highest F1-score (0.8292), TinyBERT showed best efficiency, and MiniLM demonstrated best cross-platform consistency. News data showed inflated performance due to class imbalance.

Conclusion: Compressed models perform competitively, challenging the need for larger models, with platform-specific trade-offs identified for scalable sentiment classification in urban planning.

Abstract: This study presents the first multi-platform sentiment analysis of public opinion on the 15-minute city concept across Twitter, Reddit, and news media. Using compressed transformer models and Llama-3-8B for annotation, we classify sentiment across heterogeneous text domains. Our pipeline handles long-form and short-form text, supports consistent annotation, and enables reproducible evaluation. We benchmark five models (DistilRoBERTa, DistilBERT, MiniLM, ELECTRA, TinyBERT) using stratified 5-fold cross-validation, reporting F1-score, AUC, and training time. DistilRoBERTa achieved the highest F1 (0.8292), TinyBERT the best efficiency, and MiniLM the best cross-platform consistency. Results show News data yields inflated performance due to class imbalance, Reddit suffers from summarization loss, and Twitter offers moderate challenge. Compressed models perform competitively, challenging assumptions that larger models are necessary. We identify platform-specific trade-offs and propose directions for scalable, real-world sentiment classification in urban planning discourse.

Method: Ingests data through Bluesky’s API, applies transformer-based models to annotate user-generated content, and produces structured outputs that drive a dynamic dashboard for visualizing patterns in emotion, activity, and conversation topics.

Result: Built entirely on free-tier infrastructure, achieves low operational cost and high accessibility while providing analyzable outputs for monitoring discourse patterns.

Conclusion: CognitiveSky offers a transparent, extensible tool that bridges large language models with decentralized networks, enabling applications across domains like mental health monitoring, disinformation detection, crisis response, and civic sentiment analysis.

Abstract: The emergence of decentralized social media platforms presents new opportunities and challenges for real-time analysis of public discourse. This study introduces CognitiveSky, an open-source and scalable framework designed for sentiment, emotion, and narrative analysis on Bluesky, a federated Twitter or X.com alternative. By ingesting data through Bluesky’s Application Programming Interface (API), CognitiveSky applies transformer-based models to annotate large-scale user-generated content and produces structured and analyzable outputs. These summaries drive a dynamic dashboard that visualizes evolving patterns in emotion, activity, and conversation topics. Built entirely on free-tier infrastructure, CognitiveSky achieves both low operational cost and high accessibility. While demonstrated here for monitoring mental health discourse, its modular design enables applications across domains such as disinformation detection, crisis response, and civic sentiment analysis. By bridging large language models with decentralized networks, CognitiveSky offers a transparent, extensible tool for computational social science in an era of shifting digital ecosystems.

Method: Uses fine-tuned large vision language models for contextualized embeddings, distributional attention mechanism for token-level weighting, and reconstruction objective for semantic alignment across modalities.

Result: Outperforms unimodal and multimodal baselines on six benchmarks, achieves 2.61 average LLM score, and demonstrates effectiveness in few-shot retrieval and capturing visually grounded semantics.

Conclusion: CEMTM provides an effective solution for coherent and interpretable topic modeling from multimodal documents while efficiently handling multiple images without repeated encoding.

Abstract: We introduce CEMTM, a context-enhanced multimodal topic model designed to infer coherent and interpretable topic structures from both short and long documents containing text and images. CEMTM builds on fine-tuned large vision language models (LVLMs) to obtain contextualized embeddings, and employs a distributional attention mechanism to weight token-level contributions to topic inference. A reconstruction objective aligns topic-based representations with the document embedding, encouraging semantic consistency across modalities. Unlike existing approaches, CEMTM can process multiple images per document without repeated encoding and maintains interpretability through explicit word-topic and document-topic distributions. Extensive experiments on six multimodal benchmarks show that CEMTM consistently outperforms unimodal and multimodal baselines, achieving a remarkable average LLM score of 2.61. Further analysis shows its effectiveness in downstream few-shot retrieval and its ability to capture visually grounded semantics in complex domains such as scientific articles.

Method: Proposed two techniques: 1) Reversed Training with Joint Inference to improve QA Template method, 2) Iterative Document Generation to eliminate hallucinations in source text generation.

Result: Reversed Training improves QA Template performance, while Iterative Document Generation eliminates hallucinations and boosts coreference resolution accuracy.

Conclusion: Integration of these methods provides an effective and robust solution for LLM-based coreference resolution.

Abstract: Coreference Resolution (CR) is crucial for many NLP tasks, but existing LLMs struggle with hallucination and under-performance. In this paper, we investigate the limitations of existing LLM-based approaches to CR-specifically the Question-Answering (QA) Template and Document Template methods and propose two novel techniques: Reversed Training with Joint Inference and Iterative Document Generation. Our experiments show that Reversed Training improves the QA Template method, while Iterative Document Generation eliminates hallucinations in the generated source text and boosts coreference resolution. Integrating these methods and techniques offers an effective and robust solution to LLM-based coreference resolution.

Method: Two approaches: zero-shot prompting with instruction-tuned LLMs and supervised fine-tuning using parameter-efficient LoRA. Evidence selection strategies included full-document input and top-k sentence filtering using BM25 and MiniLM.

Result: The best-performing model (LLaMA fine-tuned with LoRA) achieved strong performance on the English validation set, but showed a notable performance drop on the test set, indicating generalization challenges.

Conclusion: Evidence granularity and model adaptation are crucial for robust numerical fact verification, highlighting the need for better generalization approaches in fact-checking systems.

Abstract: This paper presents our system for Task 3 of the CLEF 2025 CheckThat! Lab, which focuses on verifying numerical and temporal claims using retrieved evidence. We explore two complementary approaches: zero-shot prompting with instruction-tuned large language models (LLMs) and supervised fine-tuning using parameter-efficient LoRA. To enhance evidence quality, we investigate several selection strategies, including full-document input and top-k sentence filtering using BM25 and MiniLM. Our best-performing model LLaMA fine-tuned with LoRA achieves strong performance on the English validation set. However, a notable drop in the test set highlights a generalization challenge. These findings underscore the importance of evidence granularity and model adaptation for robust numerical fact verification.

Method: The approach uses fine-tuned Small Language Models (SLMs) for the 13 supervised (high-resource) languages and Large Language Model (LLM) prompting for the 7 zero-shot languages without training data.

Result: The system achieved podium positions (top three) in 15 of 20 languages, including second place in 8 languages (5 of which were zero-shot). For Portuguese, it achieved an average METEOR score of 0.5290 (third place).

Conclusion: The results demonstrate the effectiveness of combining SLM fine-tuning for supervised languages with LLM prompting for zero-shot scenarios, with particularly strong performance in zero-shot languages showing the power of LLM-based strategies for cross-lingual claim normalization.

Abstract: Claim normalization, the transformation of informal social media posts into concise, self-contained statements, is a crucial step in automated fact-checking pipelines. This paper details our submission to the CLEF-2025 CheckThat! Task~2, which challenges systems to perform claim normalization across twenty languages, divided into thirteen supervised (high-resource) and seven zero-shot (no training data) tracks. Our approach, leveraging fine-tuned Small Language Models (SLMs) for supervised languages and Large Language Model (LLM) prompting for zero-shot scenarios, achieved podium positions (top three) in fifteen of the twenty languages. Notably, this included second-place rankings in eight languages, five of which were among the seven designated zero-shot languages, underscoring the effectiveness of our LLM-based zero-shot strategy. For Portuguese, our initial development language, our system achieved an average METEOR score of 0.5290, ranking third. All implementation artifacts, including inference, training, evaluation scripts, and prompt configurations, are publicly available at https://github.com/ju-resplande/checkthat2025_normalization.

Method: Used mt5-based encoder and Qwen decoder models, augmented training data with automatically translated English data for low-resource languages, and incorporated additional linguistic features from previous shared task entries.

Result: Achieved macro-accuracy score of 71.28 on the discourse relation classification task.

Conclusion: The combination of different model architectures, data augmentation techniques, and linguistic features proved effective for discourse relation classification, with error analysis providing insights for future improvements.

Abstract: This paper presents DeDisCo, Georgetown University’s entry in the DISRPT 2025 shared task on discourse relation classification. We test two approaches, using an mt5-based encoder and a decoder based approach using the openly available Qwen model. We also experiment on training with augmented dataset for low-resource languages using matched data translated automatically from English, as well as using some additional linguistic features inspired by entries in previous editions of the Shared Task. Our system achieves a macro-accuracy score of 71.28, and we provide some interpretation and error analysis for our results.

Method: Two approaches: 1) using attention weights to measure context token influence on target token, and 2) leveraging integrated gradients method for interpretable measurement of semantic influence, both incorporating semantic similarity between original and substituted sentences.

Result: Experiments on LS07 and SWORDS datasets show both approaches improve ranking performance compared to existing methods.

Conclusion: The proposed attention-based and integrated gradients approaches effectively address the challenge of modeling bidirectional semantic influence in lexical substitution ranking, leading to improved performance.

Abstract: A key subtask in lexical substitution is ranking the given candidate words. A common approach is to replace the target word with a candidate in the original sentence and feed the modified sentence into a model to capture semantic differences before and after substitution. However, effectively modeling the bidirectional influence of candidate substitution on both the target word and its context remains challenging. Existing methods often focus solely on semantic changes at the target position or rely on parameter tuning over multiple evaluation metrics, making it difficult to accurately characterize semantic variation. To address this, we investigate two approaches: one based on attention weights and another leveraging the more interpretable integrated gradients method, both designed to measure the influence of context tokens on the target token and to rank candidates by incorporating semantic similarity between the original and substituted sentences. Experiments on the LS07 and SWORDS datasets demonstrate that both approaches improve ranking performance.

Method: Evaluated seven state-of-the-art Large Vision Language Models (LVLMs) as overhearers on a corpus of human conversations during collaborative object-matching tasks, testing their ability to learn from repeated interactions.

Result: Current LVLMs perform poorly on this task and show no consistent performance improvement even after overhearing multiple conversations between the same participants repeating the same task.

Conclusion: The task remains challenging for current LVLMs, highlighting limitations in their ability to integrate conversational context and learn from repeated interactions. The released corpus and code aim to facilitate future research in this area.

Abstract: During spontaneous conversations, speakers collaborate on novel referring expressions, which they can then re-use in subsequent conversations. Understanding such referring expressions is an important ability for an embodied agent, so that it can carry out tasks in the real world. This requires integrating and understanding language, vision, and conversational interaction. We study the capabilities of seven state-of-the-art Large Vision Language Models (LVLMs) as overhearers to a corpus of spontaneous conversations between pairs of human discourse participants engaged in a collaborative object-matching task. We find that such a task remains challenging for current LVLMs and they all fail to show a consistent performance improvement as they overhear more conversations from the same discourse participants repeating the same task for multiple rounds. We release our corpus and code for reproducibility and to facilitate future research.

Method: Two-stage fusion-based generation: first assesses candidate response styles, then optimizes and integrates them through instance-level fusion to reduce stylistic variation while maintaining quality.

Result: Experimental results across multiple datasets show the approach consistently outperforms baselines in both response quality and stylistic uniformity.

Conclusion: The proposed method effectively addresses the gap in maintaining stylistic consistency for crisis communication systems, providing more trustworthy automated responses.

Abstract: In response to the urgent need for effective communication with crisis-affected populations, automated responses driven by language models have been proposed to assist in crisis communications. A critical yet often overlooked factor is the consistency of response style, which could affect the trust of affected individuals in responders. Despite its importance, few studies have explored methods for maintaining stylistic consistency across generated responses. To address this gap, we propose a novel metric for evaluating style consistency and introduce a fusion-based generation approach grounded in this metric. Our method employs a two-stage process: it first assesses the style of candidate responses and then optimizes and integrates them at the instance level through a fusion process. This enables the generation of high-quality responses while significantly reducing stylistic variation between instances. Experimental results across multiple datasets demonstrate that our approach consistently outperforms baselines in both response quality and stylistic uniformity.

Method: Created PeruMedQA dataset (8,380 multiple-choice questions from 2018-2025), used zero-shot prompting with 8 medical LLMs, and fine-tuned medgemma-4b-it using PEFT/LoRA on all questions except 2025 test set.

Result: medgemma-27b-text-it achieved >90% accuracy in several cases. Smaller LLMs (<10B parameters) performed poorly (<60% accuracy). Fine-tuned medgemma-4b-it outperformed all smaller LLMs and rivaled a 70B parameter model.

Conclusion: For Spanish medical AI applications in countries with similar epidemiological profiles to Peru, use medgemma-27b-text-it or fine-tuned medgemma-4b-it rather than smaller LLMs.

Abstract: BACKGROUND: Medical large language models (LLMS) have demonstrated remarkable performance in answering medical examinations. However, the extent to which this high performance is transferable to medical questions in Spanish and from a Latin American country remains unexplored. This knowledge is crucial as LLM-based medical applications gain traction in Latin America. AIMS: to build a dataset of questions from medical examinations taken by Peruvian physicians pursuing specialty training; to fine-tune a LLM on this dataset; to evaluate and compare the performance in terms of accuracy between vanilla LLMs and the fine-tuned LLM. METHODS: We curated PeruMedQA, a multiple-choice question-answering (MCQA) datasets containing 8,380 questions spanning 12 medical domains (2018-2025). We selected eight medical LLMs including medgemma-4b-it and medgemma-27b-text-it, and developed zero-shot task-specific prompts to answer the questions appropriately. We employed parameter-efficient fine tuning (PEFT)and low-rant adaptation (LoRA) to fine-tune medgemma-4b-it utilizing all questions except those from 2025 (test set). RESULTS: medgemma-27b-text-it outperformed all other models, achieving a proportion of correct answers exceeding 90% in several instances. LLMs with <10 billion parameters exhibited <60% of correct answers, while some exams yielded results <50%. The fine-tuned version of medgemma-4b-it emerged victorious agains all LLMs with <10 billion parameters and rivaled a LLM with 70 billion parameters across various examinations. CONCLUSIONS: For medical AI application and research that require knowledge bases from Spanish-speaking countries and those exhibiting similar epidemiological profiles to Peru’s, interested parties should utilize medgemma-27b-text-it or a fine-tuned version of medgemma-4b-it.

Method: Controlled-Literacy framework combining retrieval-augmented generation (RAG) with reinforcement learning (RL) to retrieve knowledge aligned with specific health literacy levels and optimize counterspeech using reward functions that incorporate both subjective user preferences and objective readability metrics.

Result: Experiment results show that Controlled-Literacy outperforms baseline methods by generating more accessible and user-preferred counterspeech tailored to different health literacy levels.

Conclusion: This research contributes to more equitable and impactful public health communication by improving the accessibility and comprehension of counterspeech to health misinformation through literacy-level adaptation.

Abstract: Health misinformation spreading online poses a significant threat to public health. Researchers have explored methods for automatically generating counterspeech to health misinformation as a mitigation strategy. Existing approaches often produce uniform responses, ignoring that the health literacy level of the audience could affect the accessibility and effectiveness of counterspeech. We propose a Controlled-Literacy framework using retrieval-augmented generation (RAG) with reinforcement learning (RL) to generate tailored counterspeech adapted to different health literacy levels. In particular, we retrieve knowledge aligned with specific health literacy levels, enabling accessible and factual information to support generation. We design a reward function incorporating subjective user preferences and objective readability-based rewards to optimize counterspeech to the target health literacy level. Experiment results show that Controlled-Literacy outperforms baselines by generating more accessible and user-preferred counterspeech. This research contributes to more equitable and impactful public health communication by improving the accessibility and comprehension of counterspeech to health misinformation

Method: Compared antonymy with three other semantic relations across parts of speech using robust co-occurrence metrics.

Result: Antonymy is distinctive in three respects: high co-occurrence strength, preferred linear order, and occurrence within short spans.

Conclusion: Antonymy has unique co-occurrence patterns that distinguish it from other semantic relations, with all results made available online.

Abstract: Antonymy has long received particular attention in lexical semantics. Previous studies have shown that antonym pairs frequently co-occur in text, across genres and parts of speech, more often than would be expected by chance. However, whether this co-occurrence pattern is distinctive of antonymy remains unclear, due to a lack of comparison with other semantic relations. This work fills the gap by comparing antonymy with three other relations across parts of speech using robust co-occurrence metrics. We find that antonymy is distinctive in three respects: antonym pairs co-occur with high strength, in a preferred linear order, and within short spans. All results are available online.

Method: HARP decomposes LLM hidden state space into semantic and reasoning subspaces using SVD on Unembedding layer parameters, then projects hidden states onto reasoning subspace basis vectors for feature extraction.

Result: HARP achieves state-of-the-art performance with 92.8% AUROC on TriviaQA, outperforming previous best method by 7.5%, while reducing feature dimension to ~5% of original and filtering noise.

Conclusion: The reasoning subspace projection approach provides enhanced robustness and superior hallucination detection performance, making HARP an effective solution for reliable LLM deployment in critical applications.

Abstract: Hallucinations in Large Language Models (LLMs) pose a major barrier to their reliable use in critical decision-making. Although existing hallucination detection methods have improved accuracy, they still struggle with disentangling semantic and reasoning information and maintaining robustness. To address these challenges, we propose HARP (Hallucination detection via reasoning subspace projection), a novel hallucination detection framework. HARP establishes that the hidden state space of LLMs can be decomposed into a direct sum of a semantic subspace and a reasoning subspace, where the former encodes linguistic expression and the latter captures internal reasoning processes. Moreover, we demonstrate that the Unembedding layer can disentangle these subspaces, and by applying Singular Value Decomposition (SVD) to its parameters, the basis vectors spanning the semantic and reasoning subspaces are obtained. Finally, HARP projects hidden states onto the basis vectors of the reasoning subspace, and the resulting projections are then used as input features for hallucination detection in LLMs. By using these projections, HARP reduces the dimension of the feature to approximately 5% of the original, filters out most noise, and achieves enhanced robustness. Experiments across multiple datasets show that HARP achieves state-of-the-art hallucination detection performance; in particular, it achieves an AUROC of 92.8% on TriviaQA, outperforming the previous best method by 7.5%.

Method: Proposed HiCBench with manually annotated multi-level chunking points and synthesized evidence-dense QA pairs. Developed HiChunk framework using fine-tuned LLMs for multi-level document structuring combined with Auto-Merge retrieval algorithm.

Result: HiCBench effectively evaluates different chunking methods across the RAG pipeline. HiChunk achieves better chunking quality with reasonable time consumption, enhancing overall RAG system performance.

Conclusion: The proposed HiCBench benchmark and HiChunk framework successfully address document chunking evaluation challenges and improve retrieval quality in RAG systems through multi-level structuring and optimized retrieval algorithms.

Abstract: Retrieval-Augmented Generation (RAG) enhances the response capabilities of language models by integrating external knowledge sources. However, document chunking as an important part of RAG system often lacks effective evaluation tools. This paper first analyzes why existing RAG evaluation benchmarks are inadequate for assessing document chunking quality, specifically due to evidence sparsity. Based on this conclusion, we propose HiCBench, which includes manually annotated multi-level document chunking points, synthesized evidence-dense quetion answer(QA) pairs, and their corresponding evidence sources. Additionally, we introduce the HiChunk framework, a multi-level document structuring framework based on fine-tuned LLMs, combined with the Auto-Merge retrieval algorithm to improve retrieval quality. Experiments demonstrate that HiCBench effectively evaluates the impact of different chunking methods across the entire RAG pipeline. Moreover, HiChunk achieves better chunking quality within reasonable time consumption, thereby enhancing the overall performance of RAG systems.

Method: Proposes D²HScore framework that measures: 1) Intra-Layer Dispersion - semantic diversity of token representations within each layer, and 2) Inter-Layer Drift - progressive transformation of key token representations across layers, using attention signals for token selection.

Result: Extensive experiments across five open-source LLMs and five benchmarks show D²HScore consistently outperforms existing training-free baselines for hallucination detection.

Conclusion: The proposed architecture-aware approach provides an effective, interpretable, and lightweight solution for hallucination detection without requiring training or labeled data, capturing both horizontal and vertical representation dynamics during inference.

Abstract: Although large Language Models (LLMs) have achieved remarkable success, their practical application is often hindered by the generation of non-factual content, which is called “hallucination”. Ensuring the reliability of LLMs' outputs is a critical challenge, particularly in high-stakes domains such as finance, security, and healthcare. In this work, we revisit hallucination detection from the perspective of model architecture and generation dynamics. Leveraging the multi-layer structure and autoregressive decoding process of LLMs, we decompose hallucination signals into two complementary dimensions: the semantic breadth of token representations within each layer, and the semantic depth of core concepts as they evolve across layers. Based on this insight, we propose \textbf{D$^2$HScore (Dispersion and Drift-based Hallucination Score)}, a training-free and label-free framework that jointly measures: (1) \textbf{Intra-Layer Dispersion}, which quantifies the semantic diversity of token representations within each layer; and (2) \textbf{Inter-Layer Drift}, which tracks the progressive transformation of key token representations across layers. To ensure drift reflects the evolution of meaningful semantics rather than noisy or redundant tokens, we guide token selection using attention signals. By capturing both the horizontal and vertical dynamics of representation during inference, D$^2$HScore provides an interpretable and lightweight proxy for hallucination detection. Extensive experiments across five open-source LLMs and five widely used benchmarks demonstrate that D$^2$HScore consistently outperforms existing training-free baselines.

Method: Comprehensive examination of studies since 2020, focusing on transformer-based models (BERT, T5, GPT) across three aspects: data, models, and tasks including data sources, fine-tuning strategies, and domain applications.

Result: Identified substantial issues including missing data in critical domains (e.g., health), code-mixing challenges, and lack of standardized evaluation benchmarks tailored to South Asian linguistic and cultural nuances.

Conclusion: Calls for raising awareness, targeted data curation, unified benchmarks, and equitable representation of South Asian languages in NLP to facilitate model development and address current gaps.

Abstract: Rapid developments of large language models have revolutionized many NLP tasks for English data. Unfortunately, the models and their evaluations for low-resource languages are being overlooked, especially for languages in South Asia. Although there are more than 650 languages in South Asia, many of them either have very limited computational resources or are missing from existing language models. Thus, a concrete question to be answered is: Can we assess the current stage and challenges to inform our NLP community and facilitate model developments for South Asian languages? In this survey, we have comprehensively examined current efforts and challenges of NLP models for South Asian languages by retrieving studies since 2020, with a focus on transformer-based models, such as BERT, T5, & GPT. We present advances and gaps across 3 essential aspects: data, models, & tasks, such as available data sources, fine-tuning strategies, & domain applications. Our findings highlight substantial issues, including missing data in critical domains (e.g., health), code-mixing, and lack of standardized evaluation benchmarks. Our survey aims to raise awareness within the NLP community for more targeted data curation, unify benchmarks tailored to cultural and linguistic nuances of South Asia, and encourage an equitable representation of South Asian languages. The complete list of resources is available at: https://github.com/trust-nlp/LM4SouthAsia-Survey.

Method: Dual-layered analytical framework using Bloom’s Taxonomy (six cognitive levels) and dialogue act categorization (eleven types) to analyze 7,077 user utterances in TALKA, a generative AI-powered Hakka language chatbot.

Result: Findings show generative AI chatbots effectively support language learning by aligning dialogue acts with cognitive intentions, helping learners express themselves confidently and connect with cultural identity through various functions like translations, cultural inquiries, and creative language use.

Conclusion: AI-mediated dialogue facilitates cognitive development, pragmatic negotiation, and socio-cultural affiliation in low-resource language learners, offering new insights for technology-supported language preservation and educational practice.

Abstract: With many endangered languages at risk of disappearing, efforts to preserve them now rely more than ever on using technology alongside culturally informed teaching strategies. This study examines user behaviors in TALKA, a generative AI-powered chatbot designed for Hakka language engagement, by employing a dual-layered analytical framework grounded in Bloom’s Taxonomy of cognitive processes and dialogue act categorization. We analyzed 7,077 user utterances, each carefully annotated according to six cognitive levels and eleven dialogue act types. These included a variety of functions, such as asking for information, requesting translations, making cultural inquiries, and using language creatively. Pragmatic classifications further highlight how different types of dialogue acts–such as feedback, control commands, and social greetings–align with specific cognitive intentions. The results suggest that generative AI chatbots can support language learning in meaningful ways–especially when they are designed with an understanding of how users think and communicate. They may also help learners express themselves more confidently and connect with their cultural identity. The TALKA case provides empirical insights into how AI-mediated dialogue facilitates cognitive development in low-resource language learners, as well as pragmatic negotiation and socio-cultural affiliation. By focusing on AI-assisted language learning, this study offers new insights into how technology can support language preservation and educational practice.

Method: Proposes SpanEIT framework with span-based representations for entities and sentiment phrases, bidirectional attention for fine-grained interactions, graph attention network for syntactic relations, and coreference-aware memory module for entity consistency.

Result: Experiments on FSAD, BARU, and IMDB datasets show SpanEIT outperforms state-of-the-art transformer and hybrid baselines in accuracy and F1 scores, with ablation studies validating the approach’s effectiveness.

Conclusion: SpanEIT demonstrates strong potential for fine-grained sentiment analysis applications like social media monitoring and customer feedback analysis, effectively addressing the complexities of entity-level sentiment classification.

Abstract: Entity-level sentiment classification involves identifying the sentiment polarity linked to specific entities within text. This task poses several challenges: effectively modeling the subtle and complex interactions between entities and their surrounding sentiment expressions; capturing dependencies that may span across sentences; and ensuring consistent sentiment predictions for multiple mentions of the same entity through coreference resolution. Additionally, linguistic phenomena such as negation, ambiguity, and overlapping opinions further complicate the analysis. These complexities make entity-level sentiment classification a difficult problem, especially in real-world, noisy textual data. To address these issues, we propose SpanEIT, a novel framework integrating dynamic span interaction and graph-aware memory mechanisms for enhanced entity-sentiment relational modeling. SpanEIT builds span-based representations for entities and candidate sentiment phrases, employs bidirectional attention for fine-grained interactions, and uses a graph attention network to capture syntactic and co-occurrence relations. A coreference-aware memory module ensures entity-level consistency across documents. Experiments on FSAD, BARU, and IMDB datasets show SpanEIT outperforms state-of-the-art transformer and hybrid baselines in accuracy and F1 scores. Ablation and interpretability analyses validate the effectiveness of our approach, underscoring its potential for fine-grained sentiment analysis in applications like social media monitoring and customer feedback analysis.

Method: Developed HalluDetect, an LLM-based hallucination detection system, and benchmarked five chatbot architectures including AgentBot to identify the most effective approach.

Result: HalluDetect achieved 69% F1 score (25.44% improvement over baselines). AgentBot minimized hallucinations to 0.4159 per turn with 96.13% token accuracy.

Conclusion: Optimized inference strategies can significantly improve factual accuracy in LLMs, providing a scalable framework for hallucination mitigation that generalizes to high-risk domains beyond consumer law.

Abstract: Large Language Models (LLMs) are widely used in industry but remain prone to hallucinations, limiting their reliability in critical applications. This work addresses hallucination reduction in consumer grievance chatbots built using LLaMA 3.1 8B Instruct, a compact model frequently used in industry. We develop HalluDetect, an LLM-based hallucination detection system that achieves an F1 score of 69% outperforming baseline detectors by 25.44%. Benchmarking five chatbot architectures, we find that out of them, AgentBot minimizes hallucinations to 0.4159 per turn while maintaining the highest token accuracy (96.13%), making it the most effective mitigation strategy. Our findings provide a scalable framework for hallucination mitigation, demonstrating that optimized inference strategies can significantly improve factual accuracy. While applied to consumer law, our approach generalizes to other high-risk domains, enhancing trust in LLM-driven assistants. We will release the code and dataset

Method: Proposed AesBiasBench benchmark with three subtasks (Aesthetic Perception, Assessment, Empathy) and structured metrics (IFD, NRD, AAS) to evaluate 19 MLLMs including proprietary and open-source models.

Result: Smaller models exhibit stronger stereotype biases, larger models align more closely with human preferences, and incorporating identity information often exacerbates bias, especially in emotional judgments.

Conclusion: Identity-aware evaluation frameworks are crucial for subjective vision-language tasks to address demographic biases in MLLM aesthetic assessments.

Abstract: Multimodal Large Language Models (MLLMs) are increasingly applied in Personalized Image Aesthetic Assessment (PIAA) as a scalable alternative to expert evaluations. However, their predictions may reflect subtle biases influenced by demographic factors such as gender, age, and education. In this work, we propose AesBiasBench, a benchmark designed to evaluate MLLMs along two complementary dimensions: (1) stereotype bias, quantified by measuring variations in aesthetic evaluations across demographic groups; and (2) alignment between model outputs and genuine human aesthetic preferences. Our benchmark covers three subtasks (Aesthetic Perception, Assessment, Empathy) and introduces structured metrics (IFD, NRD, AAS) to assess both bias and alignment. We evaluate 19 MLLMs, including proprietary models (e.g., GPT-4o, Claude-3.5-Sonnet) and open-source models (e.g., InternVL-2.5, Qwen2.5-VL). Results indicate that smaller models exhibit stronger stereotype biases, whereas larger models align more closely with human preferences. Incorporating identity information often exacerbates bias, particularly in emotional judgments. These findings underscore the importance of identity-aware evaluation frameworks in subjective vision-language tasks.

Method: Created a pilot dataset of 125 ethically charged mental health scenarios with structured fields including decision options, expert reasoning, expected behavior, real-world impact, and multi-stakeholder viewpoints.

Result: Developed EthicsMH dataset that enables evaluation of both decision accuracy and explanation quality while aligning with professional norms in mental health practice.

Conclusion: EthicsMH provides a foundational framework and seed resource for community expansion to foster development of AI systems capable of handling delicate mental health decisions responsibly.

Abstract: The deployment of large language models (LLMs) in mental health and other sensitive domains raises urgent questions about ethical reasoning, fairness, and responsible alignment. Yet, existing benchmarks for moral and clinical decision-making do not adequately capture the unique ethical dilemmas encountered in mental health practice, where confidentiality, autonomy, beneficence, and bias frequently intersect. To address this gap, we introduce Ethical Reasoning in Mental Health (EthicsMH), a pilot dataset of 125 scenarios designed to evaluate how AI systems navigate ethically charged situations in therapeutic and psychiatric contexts. Each scenario is enriched with structured fields, including multiple decision options, expert-aligned reasoning, expected model behavior, real-world impact, and multi-stakeholder viewpoints. This structure enables evaluation not only of decision accuracy but also of explanation quality and alignment with professional norms. Although modest in scale and developed with model-assisted generation, EthicsMH establishes a task framework that bridges AI ethics and mental health decision-making. By releasing this dataset, we aim to provide a seed resource that can be expanded through community and expert contributions, fostering the development of AI systems capable of responsibly handling some of society’s most delicate decisions.

Method: Construct news-domain-specific knowledge graph, use Monte Carlo Tree Search to decompose complex news and verify step by step, extract and update new knowledge from verified real news texts and reasoning paths.

Result: DYNAMO achieves the best performance on two real-world datasets.

Conclusion: The proposed model effectively solves key problems of ensuring new knowledge authenticity and deeply mining news semantics through dynamic knowledge updates and integration with large language models.

Abstract: As the Internet and social media evolve rapidly, distinguishing credible news from a vast amount of complex information poses a significant challenge. Due to the suddenness and instability of news events, the authenticity labels of news can potentially shift as events develop, making it crucial for fake news detection to obtain the latest event updates. Existing methods employ retrieval-augmented generation to fill knowledge gaps, but they suffer from issues such as insufficient credibility of retrieved content and interference from noisy information. We propose a dynamic knowledge update-driven model for fake news detection (DYNAMO), which leverages knowledge graphs to achieve continuous updating of new knowledge and integrates with large language models to fulfill dual functions: news authenticity detection and verification of new knowledge correctness, solving the two key problems of ensuring the authenticity of new knowledge and deeply mining news semantics. Specifically, we first construct a news-domain-specific knowledge graph. Then, we use Monte Carlo Tree Search to decompose complex news and verify them step by step. Finally, we extract and update new knowledge from verified real news texts and reasoning paths. Experimental results demonstrate that DYNAMO achieves the best performance on two real-world datasets.

Method: CoachMe analyzes differences between learner’s motion and reference motions under temporal and physical aspects, enabling domain-knowledge learning and coach-like thinking process to identify movement errors and provide corrective feedback.

Result: CoachMe outperforms GPT-4o by 31.6% in G-Eval on figure skating and by 58.3% on boxing, providing high-quality instructions that elaborate on errors and improvement methods rather than just coach-like tone without critical information.

Conclusion: CoachMe effectively adapts to specific sports by learning from general movements and leveraging limited data, demonstrating strong performance in generating precise, informative motion instructions for sports like skating and boxing.

Abstract: Motion instruction is a crucial task that helps athletes refine their technique by analyzing movements and providing corrective guidance. Although recent advances in multimodal models have improved motion understanding, generating precise and sport-specific instruction remains challenging due to the highly domain-specific nature of sports and the need for informative guidance. We propose CoachMe, a reference-based model that analyzes the differences between a learner’s motion and a reference under temporal and physical aspects. This approach enables both domain-knowledge learning and the acquisition of a coach-like thinking process that identifies movement errors effectively and provides feedback to explain how to improve. In this paper, we illustrate how CoachMe adapts well to specific sports such as skating and boxing by learning from general movements and then leveraging limited data. Experiments show that CoachMe provides high-quality instructions instead of directions merely in the tone of a coach but without critical information. CoachMe outperforms GPT-4o by 31.6% in G-Eval on figure skating and by 58.3% on boxing. Analysis further confirms that it elaborates on errors and their corresponding improvement methods in the generated instructions. You can find CoachMe here: https://motionxperts.github.io/

Method: Domain-specific stateful agentic system with planner-executor-responder architecture that infers user intent, detects document modality, and dynamically orchestrates tools for robust, traceable reasoning while avoiding tool misuse.

Result: Evaluation on curated DoP dataset shows improved robustness across different formats and languages, demonstrating scalable structured data extraction for regulated workflows.

Conclusion: The proposed agentic system provides a scalable solution for handling structural diversity in regulated document workflows, offering traceable reasoning and preventing common issues like hallucinations and tool misuse.

Abstract: Declaration of Performance (DoP) documents, mandated by EU regulation, certify the performance of construction products. While some of their content is standardized, DoPs vary widely in layout, language, schema, and format, posing challenges for automated key-value pair extraction (KVP) and question answering (QA). Existing static or LLM-only IE pipelines often hallucinate and fail to adapt to this structural diversity. Our domain-specific, stateful agentic system addresses these challenges through a planner-executor-responder architecture. The system infers user intent, detects document modality, and orchestrates tools dynamically for robust, traceable reasoning while avoiding tool misuse or execution loops. Evaluation on a curated DoP dataset demonstrates improved robustness across formats and languages, offering a scalable solution for structured data extraction in regulated workflows.

Method: Created UXPID dataset with 7130 artificially synthesized user feedback branches from industrial automation forums, using LLM to systematically analyze and annotate each JSON record for UX insights, sentiment, severity, and topic classifications.

Result: A comprehensive dataset that facilitates research in user requirements, UX analysis, and AI-driven feedback processing, particularly useful where privacy restrictions limit access to real data.

Conclusion: UXPID enables training and evaluation of transformer-based models for technical forum analysis tasks like issue detection, sentiment analysis, and requirements extraction, addressing the gap in accessible industrial feedback data.

Abstract: Customer feedback in industrial forums reflect a rich but underexplored source of insight into real-world product experience. These publicly shared discussions offer an organic view of user expectations, frustrations, and success stories shaped by the specific contexts of use. Yet, harnessing this information for systematic analysis remains challenging due to the unstructured and domain-specific nature of the content. The lack of structure and specialized vocabulary makes it difficult for traditional data analysis techniques to accurately interpret, categorize, and quantify the feedback, thereby limiting its potential to inform product development and support strategies. To address these challenges, this paper presents the User eXperience Perception Insights Dataset (UXPID), a collection of 7130 artificially synthesized and anonymized user feedback branches extracted from a public industrial automation forum. Each JavaScript object notation (JSON) record contains multi-post comments related to specific hardware and software products, enriched with metadata and contextual conversation data. Leveraging a large language model (LLM), each branch is systematically analyzed and annotated for UX insights, user expectations, severity and sentiment ratings, and topic classifications. The UXPID dataset is designed to facilitate research in user requirements, user experience (UX) analysis, and AI-driven feedback processing, particularly where privacy and licensing restrictions limit access to real-world data. UXPID supports the training and evaluation of transformer-based models for tasks such as issue detection, sentiment analysis, and requirements extraction in the context of technical forums.

Method: Created a manually annotated dataset of medication questions labeled by clinical risk factors. Tested six traditional ML classifiers using TF-IDF features and three state-of-the-art LLM-based classification approaches leveraging deep contextual understanding.

Result: The study demonstrates the potential of both classical machine learning methods and modern LLM approaches to support real-time triage and alert systems for detecting critical health questions in digital health spaces.

Conclusion: The publicly released dataset and benchmark encourage further research at the intersection of patient-generated data, NLP, and early warning systems for critical health events, showing promise for automated detection of medication-related risks.

Abstract: Online medical forums are a rich and underutilized source of insight into patient concerns, especially regarding medication use. Some of the many questions users pose may signal confusion, misuse, or even the early warning signs of a developing health crisis. Detecting these critical questions that may precede severe adverse events or life-threatening complications is vital for timely intervention and improving patient safety. This study introduces a novel annotated dataset of medication-related questions extracted from online forums. Each entry is manually labelled for criticality based on clinical risk factors. We benchmark the performance of six traditional machine learning classifiers using TF-IDF textual representations, alongside three state-of-the-art large language model (LLM)-based classification approaches that leverage deep contextual understanding. Our results highlight the potential of classical and modern methods to support real-time triage and alert systems in digital health spaces. The curated dataset is made publicly available to encourage further research at the intersection of patient-generated data, natural language processing, and early warning systems for critical health events. The dataset and benchmark are available at: https://github.com/Dvora-coder/LLM-Medication-QA-Risk-Classifier-MediGuard.

Method: Created a synthetic dataset simulating patient self-descriptions with varying linguistic noise, fuzzy language, and layperson terminology. Fine-tuned and evaluated BERT-based and T5 models using this benchmark.

Result: Developed the NDB benchmark with clinically consistent scenarios annotated with ground-truth diagnoses, spanning communication clarity levels to reflect diverse real-world reporting styles.

Conclusion: The NDB dataset enables stress-testing and comparison of LLMs’ diagnostic capabilities under realistic linguistic conditions, supporting reproducibility and future healthcare AI research.

Abstract: The widespread adoption of large language models (LLMs) in healthcare raises critical questions about their ability to interpret patient-generated narratives, which are often informal, ambiguous, and noisy. Existing benchmarks typically rely on clean, structured clinical text, offering limited insight into model performance under realistic conditions. In this work, we present a novel synthetic dataset designed to simulate patient self-descriptions characterized by varying levels of linguistic noise, fuzzy language, and layperson terminology. Our dataset comprises clinically consistent scenarios annotated with ground-truth diagnoses, spanning a spectrum of communication clarity to reflect diverse real-world reporting styles. Using this benchmark, we fine-tune and evaluate several state-of-the-art models (LLMs), including BERT-based and encoder-decoder T5 models. To support reproducibility and future research, we release the Noisy Diagnostic Benchmark (NDB), a structured dataset of noisy, synthetic patient descriptions designed to stress-test and compare the diagnostic capabilities of large language models (LLMs) under realistic linguistic conditions. We made the benchmark available for the community: https://github.com/lielsheri/PatientSignal

Method: Three-component system: (1) multi-step evidence retrieval module, (2) timeline construction module, and (3) fulfillment filtering module that captures evolving pledge fulfillment and produces structured, interpretable timelines.

Result: Evaluated in collaboration with professional fact-checkers in real-world workflows, demonstrating effectiveness in retrieving relevant evidence and reducing human verification effort.

Conclusion: PledgeTracker successfully addresses the limitations of existing methods by providing a structured approach to track political pledge fulfillment through event timeline construction, making the process more efficient and interpretable for fact-checking professionals.

Abstract: Political pledges reflect candidates’ policy commitments, but tracking their fulfilment requires reasoning over incremental evidence distributed across multiple, dynamically updated sources. Existing methods simplify this task into a document classification task, overlooking its dynamic, temporal and multi-document nature. To address this issue, we introduce \textsc{PledgeTracker}, a system that reformulates pledge verification into structured event timeline construction. PledgeTracker consists of three core components: (1) a multi-step evidence retrieval module; (2) a timeline construction module and; (3) a fulfilment filtering module, allowing the capture of the evolving nature of pledge fulfilment and producing interpretable and structured timelines. We evaluate PledgeTracker in collaboration with professional fact-checkers in real-world workflows, demonstrating its effectiveness in retrieving relevant evidence and reducing human verification effort.

Method: SCDTour orders and merges interpretable axes based on (a) semantic similarity between axes in the embedding space and (b) the degree to which each axis contributes to semantic change, producing a refined set of word senses.

Result: Experimental results show SCDTour preserves semantic change detection performance while maintaining high interpretability, achieving comparable or improved performance against original full-dimensional embeddings.

Conclusion: SCDTour effectively balances interpretability and performance in semantic change detection, enabling meaningful interpretation of semantic shifts through a small number of refined axes.

Abstract: In Semantic Change Detection (SCD), it is a common problem to obtain embeddings that are both interpretable and high-performing. However, improving interpretability often leads to a loss in the SCD performance, and vice versa. To address this problem, we propose SCDTour, a method that orders and merges interpretable axes to alleviate the performance degradation of SCD. SCDTour considers both (a) semantic similarity between axes in the embedding space, as well as (b) the degree to which each axis contributes to semantic change. Experimental results show that SCDTour preserves performance in semantic change detection while maintaining high interpretability. Moreover, agglomerating the sorted axes produces a more refined set of word senses, which achieves comparable or improved performance against the original full-dimensional embeddings in the SCD task. These findings demonstrate that SCDTour effectively balances interpretability and SCD performance, enabling meaningful interpretation of semantic shifts through a small number of refined axes. Source code is available at https://github.com/LivNLP/svp-tour .

Method: Proposes MOOM - a dual-branch memory plugin using literary theory: one branch summarizes plot conflicts across time scales, another extracts user character profiles, plus a forgetting mechanism based on competition-inhibition theory.

Result: MOOM outperforms all state-of-the-art memory extraction methods, requires fewer large language model invocations, and maintains controllable memory capacity. Also presents ZH-4O dataset with 600-turn average dialogues.

Conclusion: The proposed MOOM framework effectively addresses uncontrolled memory growth in ultra-long role-playing dialogues through literary theory-inspired dual-branch architecture and forgetting mechanisms, demonstrating superior performance with reduced computational costs.

Abstract: Memory extraction is crucial for maintaining coherent ultra-long dialogues in human-robot role-playing scenarios. However, existing methods often exhibit uncontrolled memory growth. To address this, we propose MOOM, the first dual-branch memory plugin that leverages literary theory by modeling plot development and character portrayal as core storytelling elements. Specifically, one branch summarizes plot conflicts across multiple time scales, while the other extracts the user’s character profile. MOOM further integrates a forgetting mechanism, inspired by the ``competition-inhibition’’ memory theory, to constrain memory capacity and mitigate uncontrolled growth. Furthermore, we present ZH-4O, a Chinese ultra-long dialogue dataset specifically designed for role-playing, featuring dialogues that average 600 turns and include manually annotated memory information. Experimental results demonstrate that MOOM outperforms all state-of-the-art memory extraction methods, requiring fewer large language model invocations while maintaining a controllable memory capacity.

Method: Uses the PerspAct system integrating ReAct paradigm with LLMs, evaluated through an extended director task to assess GPT’s ability to generate developmentally-consistent narratives and their impact on collaborative performance.

Result: GPT produces developmentally-consistent narratives before tasks but often shifts to more advanced stages during interaction. Higher developmental stages enhance collaborative effectiveness, while earlier stages show more variable outcomes in complex contexts.

Conclusion: Integration of embodied perspective taking and language in LLMs shows promise for modeling developmental dynamics, and evaluating internal speech during combined linguistic-embodied tasks is crucial.

Abstract: Language and embodied perspective taking are essential for human collaboration, yet few computational models address both simultaneously. This work investigates the PerspAct system [1], which integrates the ReAct (Reason and Act) paradigm with Large Language Models (LLMs) to simulate developmental stages of perspective taking, grounded in Selman’s theory [2]. Using an extended director task, we evaluate GPT’s ability to generate internal narratives aligned with specified developmental stages, and assess how these influence collaborative performance both qualitatively (action selection) and quantitatively (task efficiency). Results show that GPT reliably produces developmentally-consistent narratives before task execution but often shifts towards more advanced stages during interaction, suggesting that language exchanges help refine internal representations. Higher developmental stages generally enhance collaborative effectiveness, while earlier stages yield more variable outcomes in complex contexts. These findings highlight the potential of integrating embodied perspective taking and language in LLMs to better model developmental dynamics and stress the importance of evaluating internal speech during combined linguistic and embodied tasks.

Method: The paper draws conceptual parallels between quantum uncertainty principles and authorship detection, analyzing current methods (stylometry, watermarking, neural classifiers) and their inherent limitations through theoretical analysis.

Result: The analysis shows that enhancing detection accuracy often disrupts text authenticity, creating a fundamental trade-off where perfect detection becomes theoretically impossible when AI text closely mimics human writing.

Conclusion: The AI-text detection challenge reflects a deeper, unavoidable tension in language itself, not just a technological problem requiring better tools, with significant implications for authorship, ethics, and policy.

Abstract: As large language models (LLMs) become more advanced, it is increasingly difficult to distinguish between human-written and AI-generated text. This paper draws a conceptual parallel between quantum uncertainty and the limits of authorship detection in natural language. We argue that there is a fundamental trade-off: the more confidently one tries to identify whether a text was written by a human or an AI, the more one risks disrupting the text’s natural flow and authenticity. This mirrors the tension between precision and disturbance found in quantum systems. We explore how current detection methods–such as stylometry, watermarking, and neural classifiers–face inherent limitations. Enhancing detection accuracy often leads to changes in the AI’s output, making other features less reliable. In effect, the very act of trying to detect AI authorship introduces uncertainty elsewhere in the text. Our analysis shows that when AI-generated text closely mimics human writing, perfect detection becomes not just technologically difficult but theoretically impossible. We address counterarguments and discuss the broader implications for authorship, ethics, and policy. Ultimately, we suggest that the challenge of AI-text detection is not just a matter of better tools–it reflects a deeper, unavoidable tension in the nature of language itself.

Method: Analyzed cultural sensitivity of LLMs through English-Japanese workplace email translations using three prompting strategies: naive translation prompts, audience-targeted prompts specifying cultural background, and instructional prompts with explicit Japanese communication norms guidance. Used mixed-methods study analyzing culture-specific language patterns and native speaker evaluations of tone appropriateness.

Result: Culturally-tailored prompting can improve cultural fit in translations, showing that explicit cultural guidance enhances LLM performance in culturally appropriate communication.

Conclusion: Recommendations are provided for designing culturally inclusive LLMs in multilingual settings, emphasizing the importance of culturally-aware prompting strategies for effective cross-cultural communication.

Abstract: Large language models (LLMs) are increasingly used in everyday communication, including multilingual interactions across different cultural contexts. While LLMs can now generate near-perfect literal translations, it remains unclear whether LLMs support culturally appropriate communication. In this paper, we analyze the cultural sensitivity of different LLM designs when applied to English-Japanese translations of workplace e-mails. Here, we vary the prompting strategies: (1) naive “just translate” prompts, (2) audience-targeted prompts specifying the recipient’s cultural background, and (3) instructional prompts with explicit guidance on Japanese communication norms. Using a mixed-methods study, we then analyze culture-specific language patterns to evaluate how well translations adapt to cultural norms. Further, we examine the appropriateness of the tone of the translations as perceived by native speakers. We find that culturally-tailored prompting can improve cultural fit, based on which we offer recommendations for designing culturally inclusive LLMs in multilingual settings.

Method: Pairs a lightweight draft VLM with a large target model, using dynamic tree-based verification that adaptively expands and prunes speculative branches based on draft confidence.

Result: Achieves up to 3.28× faster decoding on LLaVA-1.5 (7B, 13B) models with no loss in generation quality on MS COCO out-of-domain images.

Conclusion: Presents a lossless acceleration framework using dynamic tree-structured speculative decoding that enables practical real-time multimodal assistants and is suitable for resource-constrained deployment.

Abstract: Vision-Language Models (VLMs) enable powerful multimodal reasoning but suffer from slow autoregressive inference, limiting their deployment in real-time applications. We introduce Spec-LLaVA, a system that applies speculative decoding to accelerate VLMs without sacrificing output quality. Spec-LLaVA pairs a lightweight draft VLM with a large target model: the draft speculates future tokens, which the target verifies in parallel, allowing multiple tokens to be generated per step. To maximize efficiency, we design a dynamic tree-based verification algorithm that adaptively expands and prunes speculative branches using draft model confidence. On MS COCO out-of-domain images, Spec-LLaVA achieves up to 3.28$\times$ faster decoding on LLaVA-1.5 (7B, 13B) with no loss in generation quality. This work presents a lossless acceleration framework for VLMs using dynamic tree-structured speculative decoding, opening a path toward practical real-time multimodal assistants. Importantly, the lightweight draft model design makes the framework amenable to resource-constrained or on-device deployment settings.

Method: Proposed a training framework for outcome-based reward models using data synthesized from permissively licensed, open-weight LLMs. Trained models ranging from 1.7B to 14B parameters and evaluated across seven out-of-domain benchmarks.

Result: The trained models consistently outperform general-purpose baselines, achieving up to 25% average improvement in downstream task performance and enabling data-efficient fine-tuning through reward-guided filtering.

Conclusion: Domain-specific reward modeling for tool use is necessary and effective, with the proposed framework significantly improving performance over general-purpose reward models in tool-calling scenarios.

Abstract: As large language models (LLMs) increasingly interact with external tools, reward modeling for tool use has become a critical yet underexplored area. Existing reward models, trained primarily on natural language outputs, struggle to evaluate tool-based reasoning and execution. To quantify this gap, we introduce FC-RewardBench, the first benchmark designed to systematically assess reward models’ performance in tool-calling scenarios. Our analysis shows that current reward models often miss key signals of effective tool use, highlighting the need for domain-specific modeling. To address this, we propose a training framework for outcome-based reward models using data synthesized from permissively licensed, open-weight LLMs. We train models ranging from 1.7B to 14B parameters and evaluate them across seven out-of-domain benchmarks. These models consistently outperform general-purpose baselines, achieving up to 25% average improvement in downstream task performance and enabling data-efficient fine-tuning through reward-guided filtering.

Method: Proposed a multi-bias framework at a domain-agnostic level, then formulated specialized approaches using sentiment-based biases and query expansion for sentiment explanation problems.

Result: Achieved experimental results that outperform baseline models on a real-world proprietary sentiment-aware QFS dataset.

Conclusion: The multi-bias framework effectively bridges the query-document gap and specialized sentiment approaches enhance performance in sentiment explanation tasks.

Abstract: Constructive analysis of feedback from clients often requires determining the cause of their sentiment from a substantial amount of text documents. To assist and improve the productivity of such endeavors, we leverage the task of Query-Focused Summarization (QFS). Models of this task are often impeded by the linguistic dissonance between the query and the source documents. We propose and substantiate a multi-bias framework to help bridge this gap at a domain-agnostic, generic level; we then formulate specialized approaches for the problem of sentiment explanation through sentiment-based biases and query expansion. We achieve experimental results outperforming baseline models on a real-world proprietary sentiment-aware QFS dataset.

Method: Used iterative automatic post-editing with Large Language Models, generating successive adaptations until readability and similarity metrics indicated optimal refinement.

Result: Achieved first place in Plain Language adaptation and second place in Easy Read adaptation based on average official metrics in the CLEARS challenge.

Conclusion: Iterative LLM post-editing with metric-based stopping criteria is an effective approach for text simplification tasks, demonstrating strong performance in Spanish language adaptation.

Abstract: We describe Vicomtech’s participation in the CLEARS challenge on text adaptation to Plain Language and Easy Read in Spanish. Our approach features automatic post-editing of different types of initial Large Language Model adaptations, where successive adaptations are generated iteratively until readability and similarity metrics indicate that no further adaptation refinement can be successfully performed. Taking the average of all official metrics, our submissions achieved first and second place in Plain language and Easy Read adaptation, respectively.

Method: Used linear discriminant analysis to identify distinct, orthogonal directions in residual space for verb tense and aspect. Demonstrated causal control through concept steering across three generation tasks, and conducted a case study to investigate steering parameters.

Result: Found that models encode tense and aspect in structurally organized, human-like ways. Identified that steering strength, location, and duration are crucial parameters for reducing undesirable side effects like topic shift and degeneration during generation.

Conclusion: While LLMs encode grammatical features in human-like structural patterns, effective control of these features during generation requires careful manual tuning or automated optimization due to sensitivity to multiple steering parameters.

Abstract: Large language models (LLMs) are able to generate grammatically well-formed text, but how do they encode their syntactic knowledge internally? While prior work has focused largely on binary grammatical contrasts, in this work, we study the representation and control of two multidimensional hierarchical grammar phenomena - verb tense and aspect - and for each, identify distinct, orthogonal directions in residual space using linear discriminant analysis. Next, we demonstrate causal control over both grammatical features through concept steering across three generation tasks. Then, we use these identified features in a case study to investigate factors influencing effective steering in multi-token generation. We find that steering strength, location, and duration are crucial parameters for reducing undesirable side effects such as topic shift and degeneration. Our findings suggest that models encode tense and aspect in structurally organized, human-like ways, but effective control of such features during generation is sensitive to multiple factors and requires manual tuning or automated optimization.

Method: Uses teacher-student framework to align self-supervised speech encoder with language-agnostic text encoder representations. Updated SAMU-XLSR with stronger teacher text model and better initial speech encoder.

Result: Achieves highly competitive performance on multilingual and multimodal semantic tasks. Model integrated into SpeechBrain toolkit and publicly released.

Conclusion: Provides new insights into how semantics are captured in semantically aligned speech encoders and offers an open-source solution for multilingual speech-text representation learning.

Abstract: This paper introduces SENSE (Shared Embedding for N-lingual Speech and tExt), an open-source solution inspired by the SAMU-XLSR framework and conceptually similar to Meta AI’s SONAR models. These approaches rely on a teacher-student framework to align a self-supervised speech encoder with the language-agnostic continuous representations of a text encoder at the utterance level. We describe how the original SAMU-XLSR method has been updated by selecting a stronger teacher text model and a better initial speech encoder. The source code for training and using SENSE models has been integrated into the SpeechBrain toolkit, and the first SENSE model we trained has been publicly released. We report experimental results on multilingual and multimodal semantic tasks, where our SENSE model achieves highly competitive performance. Finally, this study offers new insights into how semantics are captured in such semantically aligned speech encoders.

Method: Evaluated six systems (three traditional NLP tools: NLTK, spaCy, Stanza; three LLMs: Gemini-1.5-flash, DeepSeek-V3, Qwen-3-4B) on a manually annotated benchmark dataset of 119 tokens covering five entity types using F1-score metrics.

Result: LLMs generally outperformed conventional tools, especially for context-sensitive entities like person names. Gemini achieved the highest average F1-score. Traditional systems like Stanza showed greater consistency for structured tags (LOCATION, DATE). Variability observed among LLMs in handling temporal expressions and multi-word organizations.

Conclusion: While LLMs offer improved contextual understanding, traditional NLP tools remain competitive for specific structured tasks, suggesting the need for task-specific model selection rather than universal LLM adoption.

Abstract: This pilot study presents a small-scale but carefully annotated benchmark of Named Entity Recognition (NER) performance across six systems: three non-LLM NLP tools (NLTK, spaCy, Stanza) and three general-purpose large language models (LLMs: Gemini-1.5-flash, DeepSeek-V3, Qwen-3-4B). The dataset contains 119 tokens covering five entity types (PERSON, LOCATION, ORGANIZATION, DATE, TIME). We evaluated each system’s output against the manually annotated gold standard dataset using F1-score. The results show that LLMs generally outperform conventional tools in recognizing context-sensitive entities like person names, with Gemini achieving the highest average F1-score. However, traditional systems like Stanza demonstrate greater consistency in structured tags such as LOCATION and DATE. We also observed variability among LLMs, particularly in handling temporal expressions and multi-word organizations. Our findings highlight that while LLMs offer improved contextual understanding, traditional tools remain competitive in specific tasks, informing model selection.

Method: Train BEST-RQ models on 4.5k hours of unlabeled ATC data for self-supervised pre-training, then fine-tune on supervised ATC data. Propose chunked attention and dynamic convolutions for streaming ASR with low-latency inference.

Result: Domain-adapted pre-training substantially improves performance on ATC benchmarks, significantly reducing word error rates compared to general-purpose models like w2v-BERT 2.0 and HuBERT. Streaming approach further improves WER under tight latency constraints.

Conclusion: Specializing SSL representations for ATC data is an effective approach for developing more accurate and efficient ASR systems suitable for real-world operational settings in safety-critical aviation applications.

Abstract: In this study, we investigate the benefits of domain-specific self-supervised pre-training for both offline and streaming ASR in Air Traffic Control (ATC) environments. We train BEST-RQ models on 4.5k hours of unlabeled ATC data, then fine-tune on a smaller supervised ATC set. To enable real-time processing, we propose using chunked attention and dynamic convolutions, ensuring low-latency inference. We compare these in-domain SSL models against state-of-the-art, general-purpose speech encoders such as w2v-BERT 2.0 and HuBERT. Results show that domain-adapted pre-training substantially improves performance on standard ATC benchmarks, significantly reducing word error rates when compared to models trained on broad speech corpora. Furthermore, the proposed streaming approach further improves word error rate under tighter latency constraints, making it particularly suitable for safety-critical aviation applications. These findings highlight that specializing SSL representations for ATC data is a practical path toward more accurate and efficient ASR systems in real-world operational settings.

Method: Proposes Guess-Think-Answer framework: model produces provisional guess (cross-entropy loss), reflects on it, then generates final answer with RL rewards shaping both output and structure format. Uses loss masking and gradient constraints to mitigate training signal conflicts.

Result: Substantially accelerates convergence while outperforming both standalone SFT and RL baselines on four text classification benchmarks.

Conclusion: GTA successfully combines SFT efficiency with RL capability gains in a unified training paradigm, achieving both faster convergence and higher performance ceiling than pure approaches.

Abstract: In natural language processing tasks, pure reinforcement learning (RL) fine-tuning methods often suffer from inefficient exploration and slow convergence; while supervised fine-tuning (SFT) methods, although efficient in training, have limited performance ceiling and less solid theoretical foundation compared to RL. To address efficiency-capability trade-off, we propose the Guess-Think-Answer (GTA) framework that combines the efficiency of SFT with the capability gains of RL in a unified training paradigm. GTA works by having the model first produce a provisional guess (optimized via cross-entropy loss), then reflect on this guess before generating the final answer, with RL rewards shaping both the final output and the format of the entire GTA structure. This hybrid approach achieves both faster convergence than pure RL and higher performance ceiling than pure SFT. To mitigate gradient conflicts between the two training signals, we employ loss masking and gradient constraints. Empirical results on four text classification benchmarks demonstrate that GTA substantially accelerates convergence while outperforming both standalone SFT and RL baselines.

Method: Two-stage framework: (1) server-side prompt generator trained for domain-specific capabilities, (2) user-side gradient-free optimization to tailor soft prompts for individual tasks using only a single customized vector per task.

Result: Superior performance demonstrated in commonsense reasoning, medical, and financial domains compared to baselines, with advantages in task-agnostic processing and privacy preservation.

Conclusion: CBP-Tuning provides an effective solution for bidirectional privacy preservation while enabling efficient local customization of black-box LLMs, making personalized adaptation scalable and privacy-conscious.

Abstract: The high costs of customizing large language models (LLMs) fundamentally limit their adaptability to user-specific needs. Consequently, LLMs are increasingly offered as cloud-based services, a paradigm that introduces critical limitations: providers struggle to support personalized customization at scale, while users face privacy risks when exposing sensitive data. To address this dual challenge, we propose Customized Black-box Prompt Tuning (CBP-Tuning), a novel framework that facilitates efficient local customization while preserving bidirectional privacy. Specifically, we design a two-stage framework: (1) a prompt generator trained on the server-side to capture domain-specific and task-agnostic capabilities, and (2) user-side gradient-free optimization that tailors soft prompts for individual tasks. This approach eliminates the need for users to access model weights or upload private data, requiring only a single customized vector per task while achieving effective adaptation. Furthermore, the evaluation of CBP-Tuning in the commonsense reasoning, medical and financial domain settings demonstrates superior performance compared to baselines, showcasing its advantages in task-agnostic processing and privacy preservation.

Method: Two approaches: (1) Supervised fine-tuning of transformer encoders (EuroBERT, XLM-RoBERTa, German-BERT) on monolingual and machine-translated data; (2) Zero-shot prompting using LLMs (o3-mini for rule-based labeling, gpt-4.1-mini for contrastive rewriting and comparative reasoning).

Result: Achieved 1st place in Italian monolingual subtask (F1=0.8104 vs baseline 0.6941). XLM-RoBERTa ranked 3rd in Romanian zero-shot (F1=0.7917 vs 0.6461 baseline). Reliable multilingual performance and improvements in Greek. German-BERT performed competitively. Some challenges in Ukrainian and Polish zero-shot settings.

Conclusion: Transformer fine-tuning and LLM prompting are effective for multilingual subjectivity detection, with strong results in several languages, though low-resource cross-lingual scenarios remain challenging and require further improvement.

Abstract: This notebook reports the XplaiNLP submission to the CheckThat! 2025 shared task on multilingual subjectivity detection. We evaluate two approaches: (1) supervised fine-tuning of transformer encoders, EuroBERT, XLM-RoBERTa, and German-BERT, on monolingual and machine-translated training data; and (2) zero-shot prompting using two LLMs: o3-mini for Annotation (rule-based labelling) and gpt-4.1-mini for DoubleDown (contrastive rewriting) and Perspective (comparative reasoning). The Annotation Approach achieves 1st place in the Italian monolingual subtask with an F_1 score of 0.8104, outperforming the baseline of 0.6941. In the Romanian zero-shot setting, the fine-tuned XLM-RoBERTa model obtains an F_1 score of 0.7917, ranking 3rd and exceeding the baseline of 0.6461. The same model also performs reliably in the multilingual task and improves over the baseline in Greek. For German, a German-BERT model fine-tuned on translated training data from typologically related languages yields competitive performance over the baseline. In contrast, performance in the Ukrainian and Polish zero-shot settings falls slightly below the respective baselines, reflecting the challenge of generalization in low-resource cross-lingual scenarios.

Method: Systematic analysis and reformulation of existing pun benchmarks, comprehensive human evaluation across recent LLMs, and robustness analysis.

Result: Subtle changes in puns are sufficient to mislead LLMs, revealing their shallow understanding compared to human interpretation.

Conclusion: LLMs face significant robustness challenges in processing puns and lack the nuanced grasp that humans possess for this form of wordplay humor.

Abstract: Puns are a form of humorous wordplay that exploits polysemy and phonetic similarity. While LLMs have shown promise in detecting puns, we show in this paper that their understanding often remains shallow, lacking the nuanced grasp typical of human interpretation. By systematically analyzing and reformulating existing pun benchmarks, we demonstrate how subtle changes in puns are sufficient to mislead LLMs. Our contributions include comprehensive and nuanced pun detection benchmarks, human evaluation across recent LLMs, and an analysis of the robustness challenges these models face in processing puns.

Method: Inspired by RAG, the framework reformulates LLM role-playing as a text retrieval problem, leveraging curated reference demonstrations to condition LLM responses. It introduces token-level ROUGE metrics (IOO and IOR) to measure improvisation and demonstration utilization.

Result: The method incorporates 35% more tokens from reference demonstrations during hostile interactions. Across 453 interactions, models were consistently judged more authentic and remained in-character more often than zero-shot and ICL methods.

Conclusion: RAGs-to-Riches provides a scalable strategy for building robust, human-aligned LLM role-playing frameworks that maintain character authenticity in challenging scenarios.

Abstract: Role-playing Large language models (LLMs) are increasingly deployed in high-stakes domains such as healthcare, education, and governance, where failures can directly impact user trust and well-being. A cost effective paradigm for LLM role-playing is few-shot learning, but existing approaches often cause models to break character in unexpected and potentially harmful ways, especially when interacting with hostile users. Inspired by Retrieval-Augmented Generation (RAG), we reformulate LLM role-playing into a text retrieval problem and propose a new prompting framework called RAGs-to-Riches, which leverages curated reference demonstrations to condition LLM responses. We evaluate our framework with LLM-as-a-judge preference voting and introduce two novel token-level ROUGE metrics: Intersection over Output (IOO) to quantity how much an LLM improvises and Intersection over References (IOR) to measure few-shot demonstrations utilization rate during the evaluation tasks. When simulating interactions with a hostile user, our prompting strategy incorporates in its responses during inference an average of 35% more tokens from the reference demonstrations. As a result, across 453 role-playing interactions, our models are consistently judged as being more authentic, and remain in-character more often than zero-shot and in-context Learning (ICL) methods. Our method presents a scalable strategy for building robust, human-aligned LLM role-playing frameworks.

Method: Uses textual tasks converted to speech via voice cloning text-to-speech model to test models, quantifying capability preservation across text and speech modalities.

Result: The benchmark provides a way to measure how well language understanding is preserved when models process speech input instead of text input.

Conclusion: C3T offers a standardized method to assess cross-modal performance and fairness of speech-aware language models, ensuring they work equitably across different input modalities and speaker types.

Abstract: The paper presents C3T (Cross-modal Capabilities Conservation Test), a new benchmark for assessing the performance of speech-aware large language models. The benchmark utilizes textual tasks and a voice cloning text-to-speech model to quantify the extent to which language understanding capabilities are preserved when the model is accessed via speech input. C3T quantifies the fairness of the model for different categories of speakers and its robustness across text and speech modalities.

Method: Proposed a kernel-regression perspective, proving Bayesian inference on in-context prompts asymptotically becomes kernel regression. Empirically investigated attention and hidden features in LLMs during ICL to match kernel regression behaviors.

Result: Found that LLMs’ attention and hidden features during in-context learning exhibit behaviors consistent with kernel regression. The theory explains several ICL phenomena including why similar demonstrations help, sensitivity to output formats, and benefits of in-distribution samples.

Conclusion: Transformer-based language models accomplish in-context learning through mechanisms that can be understood as kernel regression, providing a theoretical framework that explains multiple observed ICL phenomena and offering insights into LLM capabilities.

Abstract: Large language models (LLMs) have initiated a paradigm shift in transfer learning. In contrast to the classic pretraining-then-finetuning procedure, in order to use LLMs for downstream prediction tasks, one only needs to provide a few demonstrations, known as in-context examples, without adding more or updating existing model parameters. This in-context learning (ICL) capability of LLMs is intriguing, and it is not yet fully understood how pretrained LLMs acquire such capabilities. In this paper, we investigate the reason why a transformer-based language model can accomplish in-context learning after pre-training on a general language corpus by proposing a kernel-regression perspective of understanding LLMs’ ICL bahaviors when faced with in-context examples. More concretely, we first prove that Bayesian inference on in-context prompts can be asymptotically understood as kernel regression $\hat y = \sum_i y_i K(x, x_i)/\sum_i K(x, x_i)$ as the number of in-context demonstrations grows. Then, we empirically investigate the in-context behaviors of language models. We find that during ICL, the attention and hidden features in LLMs match the behaviors of a kernel regression. Finally, our theory provides insights into multiple phenomena observed in the ICL field: why retrieving demonstrative samples similar to test samples can help, why ICL performance is sensitive to the output formats, and why ICL accuracy benefits from selecting in-distribution and representative samples. Code and resources are publicly available at https://github.com/Glaciohound/Explain-ICL-As-Kernel-Regression.

Method: Four distinct approaches using summarization and augmentation techniques with five pre-trained language models, including translating English news articles and summarizing content to handle BERT token limitations.

Result: BanglaBERT Base with augmentation achieved 96% accuracy on first test dataset, BanglaBERT with summarized augmented articles achieved 97% on second dataset, and mBERT Base achieved 86% on third generalization test dataset.

Conclusion: The research demonstrates the effectiveness of summarization and augmentation techniques for Bengali fake news detection, with models showing high accuracy across multiple test datasets.

Abstract: With the rise of social media and online news sources, fake news has become a significant issue globally. However, the detection of fake news in low resource languages like Bengali has received limited attention in research. In this paper, we propose a methodology consisting of four distinct approaches to classify fake news articles in Bengali using summarization and augmentation techniques with five pre-trained language models. Our approach includes translating English news articles and using augmentation techniques to curb the deficit of fake news articles. Our research also focused on summarizing the news to tackle the token length limitation of BERT based models. Through extensive experimentation and rigorous evaluation, we show the effectiveness of summarization and augmentation in the case of Bengali fake news detection. We evaluated our models using three separate test datasets. The BanglaBERT Base model, when combined with augmentation techniques, achieved an impressive accuracy of 96% on the first test dataset. On the second test dataset, the BanglaBERT model, trained with summarized augmented news articles achieved 97% accuracy. Lastly, the mBERT Base model achieved an accuracy of 86% on the third test dataset which was reserved for generalization performance evaluation. The datasets and implementations are available at https://github.com/arman-sakif/Bengali-Fake-News-Detection

Method: Survey and analysis of recent research efforts, presenting taxonomies of hallucination phenomena, evaluation benchmarks, and existing mitigation approaches.

Result: Systematic categorization of LLM hallucination types and comprehensive review of current detection and mitigation techniques.

Conclusion: Hallucination remains a significant challenge for LLM reliability, requiring continued research into better detection, explanation, and mitigation strategies.

Abstract: While large language models (LLMs) have demonstrated remarkable capabilities across a range of downstream tasks, a significant concern revolves around their propensity to exhibit hallucinations: LLMs occasionally generate content that diverges from the user input, contradicts previously generated context, or misaligns with established world knowledge. This phenomenon poses a substantial challenge to the reliability of LLMs in real-world scenarios. In this paper, we survey recent efforts on the detection, explanation, and mitigation of hallucination, with an emphasis on the unique challenges posed by LLMs. We present taxonomies of the LLM hallucination phenomena and evaluation benchmarks, analyze existing approaches aiming at mitigating LLM hallucination, and discuss potential directions for future research.

Method: Combined word embedding similarity (WE) and compositional semantics (CS) approaches to generate an ensemble of lexicons, evaluated on manually annotated data across in-domain and out-of-domain scenarios.

Result: Produced a robust combined liberty lexicon that effectively captures complex liberty expressions across different platforms, though the task complexity requires combined knowledge approaches.

Conclusion: The developed lexicon improves representation of liberty concepts in learning systems and demonstrates the need for knowledge-combining approaches to handle the complexity of moral value expressions.

Abstract: The moral value of liberty is a central concept in our inference system when it comes to taking a stance towards controversial social issues such as vaccine hesitancy, climate change, or the right to abortion. Here, we propose a novel Liberty lexicon evaluated on more than 3,000 manually annotated data both in in- and out-of-domain scenarios. As a result of this evaluation, we produce a combined lexicon that constitutes the main outcome of this work. This final lexicon incorporates information from an ensemble of lexicons that have been generated using word embedding similarity (WE) and compositional semantics (CS). Our key contributions include enriching the liberty annotations, developing a robust liberty lexicon for broader application, and revealing the complexity of expressions related to liberty across different platforms. Through the evaluation, we show that the difficulty of the task calls for designing approaches that combine knowledge, in an effort of improving the representations of learning systems.

Method: Conducted comprehensive survey and taxonomy of over 70 metrics. Proposed EnsembEval - a linear regression-based ensemble method that combines diverse metrics. Trained on one human ratings dataset and tested on five additional datasets.

Result: Found that vast majority of studies rely on only five popular metrics which are weakly correlated with human ratings. EnsembEval demonstrated improved correlation across multiple datasets, showing potential of leveraging diverse metrics.

Conclusion: There is significant room for improvement in image captioning evaluation by using diverse metrics rather than relying on a few popular ones. Ensemble methods like EnsembEval can enhance correlation with human judgment.

Abstract: The task of image captioning has recently been gaining popularity, and with it the complex task of evaluating the quality of image captioning models. In this work, we present the first survey and taxonomy of over 70 different image captioning metrics and their usage in hundreds of papers, specifically designed to help users select the most suitable metric for their needs. We find that despite the diversity of proposed metrics, the vast majority of studies rely on only five popular metrics, which we show to be weakly correlated with human ratings. We hypothesize that combining a diverse set of metrics can enhance correlation with human ratings. As an initial step, we demonstrate that a linear regression-based ensemble method, which we call EnsembEval, trained on one human ratings dataset, achieves improved correlation across five additional datasets, showing there is a lot of room for improvement by leveraging a diverse set of metrics.

Method: GER extracts scenario-specific guidance from a teacher LLM, stores scenario-guidance pairs in a structured library, and retrieves relevant guidance at inference time to integrate into the student’s prompt. Can use synthetic data and human conversational logs.

Result: GER’s guidance-based coaching outperforms both example output-based fine-tuning and non-customized guidance baselines, and generalizes across different contexts and student models.

Conclusion: GER provides an effective framework for knowledge distillation that balances performance and cost, with potential extensibility to coach human service agents and easy auditing/updating capabilities.

Abstract: Enterprises deploying LLMs for goal-oriented dialogs, such as customer service, face a critical trade-off between performance, control, and cost. Proprietary models like GPT-4 offer strong performance but are costly and cannot be self-hosted, raising security and privacy concerns. Open-source alternatives offer flexibility and lower token costs but lag in performance. We introduce Guidance Elicitation and Retrieval (GER), a prompt-based knowledge distillation framework where a high-performance teacher LLM coaches a lower-performance student without modifying the student’s parameters. GER extracts tactical guidance for a wide range of dialog scenarios from the teacher and stores these scenario-guidance pairs in a structured library. At inference time, the student retrieves the relevant guidance and integrates it into its prompt. While GER training can be bootstrapped entirely with synthetic data, its modular design lets it seamlessly augment the synthetic data with human conversational logs. In addition, the modular design enables easy auditing and updating of the guidance library as new scenarios and constraints emerge. Experiments show GER’s guidance-based coaching outperforms both example output based fine-tuning and non-customized guidance baselines, and generalizes across other contexts and student models. The GER framework is potentially extensible to coach human service agents.

Method: Two-stage fine-tuning on comprehensive corpus of over 3,000,000 genomics, proteomics, and medical genetics terms from validated datasets and scientific publications.

Result: GP-GPT outperforms state-of-the-art LLMs (Llama2, Llama3, GPT-4) in domain-specific tasks, demonstrating proficiency in medical genetics information retrieval and genomics analysis tasks.

Conclusion: GP-GPT shows potential to enhance genetic disease relation research and facilitate accurate genomics analysis, with subtle changes in bio-factor representations suggesting opportunities for advancing gene-phenotype research using LLMs.

Abstract: Pre-trained large language models(LLMs) have attracted increasing attention in biomedical domains due to their success in natural language processing. However, the complex traits and heterogeneity of multi-sources genomics data pose significant challenges when adapting these models to the bioinformatics and biomedical field. To address these challenges, we present GP-GPT, the first specialized large language model for genetic-phenotype knowledge representation and genomics relation analysis. Our model is fine-tuned in two stages on a comprehensive corpus composed of over 3,000,000 terms in genomics, proteomics, and medical genetics, derived from multiple large-scale validated datasets and scientific publications. GP-GPT demonstrates proficiency in accurately retrieving medical genetics information and performing common genomics analysis tasks, such as genomics information retrieval and relationship determination. Comparative experiments across domain-specific tasks reveal that GP-GPT outperforms state-of-the-art LLMs, including Llama2, Llama3 and GPT-4. These results highlight GP-GPT’s potential to enhance genetic disease relation research and facilitate accurate and efficient analysis in the fields of genomics and medical genetics. Our investigation demonstrated the subtle changes of bio-factor entities’ representations in the GP-GPT, which suggested the opportunities for the application of LLMs to advancing gene-phenotype research.

Method: Proposed Response Tuning (RT) which removes instructions and only uses response data to establish response distributions, comparing it against traditional instruction tuning with instruction-response pairs.

Result: RT models trained solely on responses performed comparably to instruction-tuned models across various instructions, could reject unsafe queries using response-only safety policies, and showed similar capabilities in in-context learning settings.

Conclusion: Pre-trained LLMs possess extensive inherent instruction-following capabilities that can be activated through response-only training, challenging the necessity of explicit instruction-response supervision.

Abstract: Instruction tuning – supervised fine-tuning using instruction-response pairs – is a key step in making pre-trained large language models (LLMs) instructable. Meanwhile, LLMs perform multitask learning during their pre-training, acquiring extensive knowledge and capabilities. We hypothesize that the pre-training stage can enable them to develop the ability to comprehend and address instructions. To verify this, we propose Response Tuning (RT), which removes the instruction and its corresponding mapping to the response from instruction tuning. Instead, it focuses solely on establishing a response distribution. Our experiments demonstrate that RT models, trained only on responses, can effectively respond to a wide range of instructions akin to their instruction-tuned counterparts. In addition, we observe that the models can recognize and reject unsafe queries after learning a safety policy only from the response data. Furthermore, we find that these observations extend to an in-context learning setting. These findings support our hypothesis, highlighting the extensive inherent capabilities of pre-trained LLMs.

Method: Novel approach combining systematic prompt augmentation mechanism with knowledge base, incorporating human-in-loop approach with POS tagging, synonyms, aspect-based sense filtering, and few-shot prompting to guide LLMs.

Result: Substantial improvement in performance demonstrated through evaluation using FEWS test data and sense tags.

Conclusion: Research advances accurate word interpretation in social media and digital communication by effectively leveraging LLMs for WSD.

Abstract: Ambiguous words are often found in modern digital communications. Lexical ambiguity challenges traditional Word Sense Disambiguation (WSD) methods, due to limited data. Consequently, the efficiency of translation, information retrieval, and question-answering systems is hindered by these limitations. This study investigates the use of Large Language Models (LLMs) to improve WSD using a novel approach combining a systematic prompt augmentation mechanism with a knowledge base (KB) consisting of different sense interpretations. The proposed method incorporates a human-in-loop approach for prompt augmentation where prompt is supported by Part-of-Speech (POS) tagging, synonyms of ambiguous words, aspect-based sense filtering and few-shot prompting to guide the LLM. By utilizing a few-shot Chain of Thought (COT) prompting-based approach, this work demonstrates a substantial improvement in performance. The evaluation was conducted using FEWS test data and sense tags. This research advances accurate word interpretation in social media and digital communication.

Method: Analyzed 9,836 unique articles from WoS, Scopus, and Lens databases using scientometric analysis (clusters, subject categories, keywords, bursts, centrality, research centers) and thematic review of 18 purposefully selected articles focusing on purpose, approach, findings, and contributions.

Result: Found that AI development increasingly contributes to translation industry, with trending issues including machine translation, statistical machine translation, low-resource languages, large language models, Arabic dialects, translation quality, and neural machine translation. Neural networking algorithms and deep language learning models like ChatGPT have been successfully incorporated.

Conclusion: While AI has significantly advanced translation industry through neural networks and large language models, rigorous research is still needed to address challenges with low-resource languages, multi-dialectical languages, free word order languages, and cultural/religious registers.

Abstract: This study provides a comprehensive analysis of artificial intelligence (AI) contribution to research in the translation industry (ACTI), synthesizing it over forty-five years from 1980-2024. 13220 articles were retrieved from three sources, namely WoS, Scopus, and Lens; 9836 were unique records, which were used for the analysis. I provided two types of analysis, viz., scientometric and thematic, focusing on Cluster, Subject categories, Keywords, Bursts, Centrality and Research Centers as for the former. For the latter, I provided a thematic review for 18 articles, selected purposefully from the articles involved, centering on purpose, approach, findings, and contribution to ACTI future directions. This study is significant for its valuable contribution to ACTI knowledge production over 45 years, emphasizing several trending issues and hotspots including Machine translation, Statistical machine translation, Low-resource language, Large language model, Arabic dialects, Translation quality, and Neural machine translation. The findings reveal that the more AI develops, the more it contributes to translation industry, as Neural Networking Algorithms have been incorporated and Deep Language Learning Models like ChatGPT have been launched. However, much rigorous research is still needed to overcome several problems encountering translation industry, specifically concerning low-resource, multi-dialectical and free word order languages, and cultural and religious registers.

Method: Used Web of Science Core Collection database with CiteSpace and VOSviewer software for mapping visualizations of intellectual landscape, trending issues, and research hotspots through scientometric analysis.

Result: Research was unstable in 1980s-1990s but showed remarkable growth since then, reaching 1478 articles in 2023. Emerging issues include Natural Language Processing, ChatGPT, bidirectional encoder representation, with hotspots in novice programmer, prioritization, and AI applications.

Conclusion: Linguistics and AI correlation is established at multiple levels with research centers, journals, and countries shaping knowledge production and reshaping future frontiers through new deep learning language models and applications.

Abstract: There is a strong correlation between linguistics and artificial intelligence (AI), best manifested by deep learning language models. This study provides a thorough scientometric analysis of this correlation, synthesizing the intellectual production over 51 years, from 1974 to 2024. Web of Science Core Collection (WoSCC) database was the data source. The data collected were analyzed by two powerful software, viz., CiteSpace and VOSviewer, through which mapping visualizations of the intellectual landscape, trending issues and (re)emerging hotspots were generated. The results indicate that in the 1980s and 1990s, linguistics and AI (AIL) research was not robust, characterized by unstable publication over time. It has, however, witnessed a remarkable increase of publication since then, reaching 1478 articles in 2023, and 546 articles in January-March timespan in 2024, involving emerging issues including Natural language processing, Cross-sectional study, Using bidirectional encoder representation, and Using ChatGPT and hotspots such as Novice programmer, Prioritization, and Artificial intelligence, addressing new horizons, new topics, and launching new applications and powerful deep learning language models including ChatGPT. It concludes that linguistics and AI correlation is established at several levels, research centers, journals, and countries shaping AIL knowledge production and reshaping its future frontiers.

Method: 5-stage pipeline: acquire multimodal Wikipedia documents, synthetically generate high-level Q&A, validate with rigorous criteria to ensure data quality.

Result: Models trained on synthesized data outperform human-collected data by 1.9 EM score average on MultimodalQA and WebQA benchmarks. Created M2QA-Bench with 1k samples for long documents.

Conclusion: FM2DS provides strong foundation for training and evaluating MMQA models, addressing the gap in quality multimodal multihop QA datasets.

Abstract: Multimodal multihop question answering (MMQA) requires reasoning over images and text from multiple sources. Despite advances in visual question answering, this multihop setting remains underexplored due to a lack of quality datasets. Existing methods focus on single-hop, single-modality, or short texts, limiting real-world applications like interpreting educational documents with long, multimodal content. To fill this gap, we introduce FM2DS, the first framework for creating a high-quality dataset for MMQA. Our approach consists of a 5-stage pipeline that involves acquiring relevant multimodal documents from Wikipedia, synthetically generating high-level questions and answers, and validating them through rigorous criteria to ensure data quality. We evaluate our methodology by training models on our synthesized dataset and testing on two benchmarks: MultimodalQA and WebQA. Our results demonstrate that, with an equal sample size, models trained on our synthesized data outperform those trained on human-collected data by 1.9 in exact match (EM) score on average. Additionally, we introduce M2QA-Bench with 1k samples, the first benchmark for MMQA on long documents, generated using FM2DS and refined by human annotators. We believe our data synthesis method will serve as a strong foundation for training and evaluating MMQA models.

Method: Proposed S^2-Bench with three tasks: molecule editing (MolEdit), molecule optimization (MolOpt), and customized molecule generation (MolCustom). Also created OpenMolIns instruction tuning dataset.

Result: Llama-3.1-8B with OpenMolIns surpassed powerful LLMs like GPT-4o and Claude-3.5 on S^2-Bench. Comprehensive evaluation of 28 LLMs showed shift from pattern recall to realistic molecular design.

Conclusion: S^2-Bench enables more capable LLMs for natural language-driven molecule discovery by focusing on genuine molecular understanding and generation capabilities.

Abstract: Recently, Large Language Models (LLMs) have shown great potential in natural language-driven molecule discovery. However, existing datasets and benchmarks for molecule-text alignment are predominantly built on a one-to-one mapping, measuring LLMs’ ability to retrieve a single, pre-defined answer, rather than their creative potential to generate diverse, yet equally valid, molecular candidates. To address this critical gap, we propose Speak-to-Structure (S^2-Bench}), the first benchmark to evaluate LLMs in open-domain natural language-driven molecule generation. S^2-Bench is specifically designed for one-to-many relationships, challenging LLMs to demonstrate genuine molecular understanding and generation capabilities. Our benchmark includes three key tasks: molecule editing (MolEdit), molecule optimization (MolOpt), and customized molecule generation (MolCustom), each probing a different aspect of molecule discovery. We also introduce OpenMolIns, a large-scale instruction tuning dataset that enables Llama-3.1-8B to surpass the most powerful LLMs like GPT-4o and Claude-3.5 on S^2-Bench. Our comprehensive evaluation of 28 LLMs shifts the focus from simple pattern recall to realistic molecular design, paving the way for more capable LLMs in natural language-driven molecule discovery.

Method: Created IOLBENCH benchmark derived from International Linguistics Olympiad problems covering syntax, morphology, phonology, and semantics, then extensively tested leading LLMs on these tasks.

Result: Even the most advanced LLMs struggle with linguistic complexity, particularly in areas requiring compositional generalization and rule abstraction, though some strengths were identified.

Conclusion: Current models have persistent limitations in linguistic problem-solving, highlighting the need for further research to develop models with human-like reasoning capabilities for computational linguistics and AI advancement.

Abstract: Despite the remarkable advancements and widespread applications of deep neural networks, their ability to perform reasoning tasks remains limited, particularly in domains requiring structured, abstract thought. In this paper, we investigate the linguistic reasoning capabilities of state-of-the-art large language models (LLMs) by introducing IOLBENCH, a novel benchmark derived from International Linguistics Olympiad (IOL) problems. This dataset encompasses diverse problems testing syntax, morphology, phonology, and semantics, all carefully designed to be self-contained and independent of external knowledge. These tasks challenge models to engage in metacognitive linguistic reasoning, requiring the deduction of linguistic rules and patterns from minimal examples. Through extensive benchmarking of leading LLMs, we find that even the most advanced models struggle to handle the intricacies of linguistic complexity, particularly in areas demanding compositional generalization and rule abstraction. Our analysis highlights both the strengths and persistent limitations of current models in linguistic problem-solving, offering valuable insights into their reasoning capabilities. By introducing IOLBENCH, we aim to foster further research into developing models capable of human-like reasoning, with broader implications for the fields of computational linguistics and artificial intelligence.

Method: Use pre-trained VLM to convert visual information from entities and neighbors into textual sequences, then frame KGC as sequence-to-sequence task and fine-tune with cross-modal context.

Result: Significantly reduces model size compared to traditional KGE approaches while achieving competitive performance across multiple large-scale datasets with minimal hyperparameter tuning.

Conclusion: The proposed method effectively integrates cross-modal information for MMKGC with reduced computational costs and maintains strong performance.

Abstract: Multimodal knowledge graph completion (MMKGC) aims to predict missing links in multimodal knowledge graphs (MMKGs) by leveraging information from various modalities alongside structural data. Existing MMKGC approaches primarily extend traditional knowledge graph embedding (KGE) models, which often require creating an embedding for every entity. This results in large model sizes and inefficiencies in integrating multimodal information, particularly for real-world graphs. Meanwhile, Transformer-based models have demonstrated competitive performance in knowledge graph completion (KGC). However, their focus on single-modal knowledge limits their capacity to utilize cross-modal information. Recently, Large vision-language models (VLMs) have shown potential in cross-modal tasks but are constrained by the high cost of training. In this work, we propose a novel approach that integrates Transformer-based KGE models with cross-modal context generated by pre-trained VLMs, thereby extending their applicability to MMKGC. Specifically, we employ a pre-trained VLM to transform relevant visual information from entities and their neighbors into textual sequences. We then frame KGC as a sequence-to-sequence task, fine-tuning the model with the generated cross-modal context. This simple yet effective method significantly reduces model size compared to traditional KGE approaches while achieving competitive performance across multiple large-scale datasets with minimal hyperparameter tuning.

Method: BoostedProb boosts the model’s confidence in cases where there are multiple viable output options, without increasing computational complexity.

Result: Achieves +0.194 average improvement in Pearson correlation to ground-truth quality compared to raw model probability, and performs comparably or better than more costly supervised/ensemble QE approaches.

Conclusion: A simple confidence-boosting approach can significantly improve quality estimation for text-generation models by addressing the underconfidence issue caused by multiple correct output options.

Abstract: Quality Estimation (QE) is estimating quality of the model output during inference when the ground truth is not available. Deriving output quality from the models’ output probability is the most trivial and low-effort way. However, we show that the output probability of text-generation models can appear underconfident. At each output step, there can be multiple correct options, making the probability distribution spread out more. Thus, lower probability does not necessarily mean lower output quality. Due to this observation, we propose a QE approach called BoostedProb, which boosts the model’s confidence in cases where there are multiple viable output options. With no increase in complexity, BoostedProb is notably better than raw model probability in different settings, achieving on average +0.194 improvement in Pearson correlation to ground-truth quality. It also comes close to or outperforms more costly approaches like supervised or ensemble-based QE in certain settings.

Method: Using psychological frameworks to measure and infuse persona traits into LLMs, then generating and evaluating dialogues for trait stability and impact.

Result: LLMs infer core persona traits, show subtle emotional shifts, and modify support strategies to enhance relevance and empathy.

Conclusion: Persona-driven LLMs can create more personalized and effective emotional support dialogues, improving AI-driven support systems.

Abstract: The rapid advancement of Large Language Models (LLMs) has revolutionized the generation of emotional support conversations (ESC), offering scalable solutions with reduced costs and enhanced data privacy. This paper explores the role of personas in the creation of ESC by LLMs. Our research utilizes established psychological frameworks to measure and infuse persona traits into LLMs, which then generate dialogues in the emotional support scenario. We conduct extensive evaluations to understand the stability of persona traits in dialogues, examining shifts in traits post-generation and their impact on dialogue quality and strategy distribution. Experimental results reveal several notable findings: 1) LLMs can infer core persona traits, 2) subtle shifts in emotionality and extraversion occur, influencing the dialogue dynamics, and 3) the application of persona traits modifies the distribution of emotional support strategies, enhancing the relevance and empathetic quality of the responses. These findings highlight the potential of persona-driven LLMs in crafting more personalized, empathetic, and effective emotional support dialogues, which has significant implications for the future design of AI-driven emotional support systems.

Method: Partition pre-trained FFN layers into computational blocks, use sigmoid activation and straight-through estimators for adaptive expert routing, and introduce sparsity loss to balance performance and efficiency.

Result: Superior performance compared to existing pruning and MoE approaches across language modeling and downstream tasks, particularly in generation tasks, with distinctive layerwise activation patterns.

Conclusion: DSMoE provides an effective sparsification approach that preserves model knowledge while reducing computational overhead, offering new insights for future MoE architecture design.

Abstract: As large language models continue to scale, computational costs and resource consumption have emerged as significant challenges. While existing sparsification methods like pruning reduce computational overhead, they risk losing model knowledge through parameter removal. This paper proposes DSMoE (Dynamic Sparse Mixture-of-Experts), a novel approach that achieves sparsification by partitioning pre-trained FFN layers into computational blocks. We implement adaptive expert routing using sigmoid activation and straight-through estimators, enabling tokens to flexibly access different aspects of model knowledge based on input complexity. Additionally, we introduce a sparsity loss term to balance performance and computational efficiency. Extensive experiments on LLaMA models demonstrate that under equivalent computational constraints, DSMoE achieves superior performance compared to existing pruning and MoE approaches across language modeling and downstream tasks, particularly excelling in generation tasks. Analysis reveals that DSMoE learns distinctive layerwise activation patterns, providing new insights for future MoE architecture design.

Method: Reframed the task as graph classification by transforming claim sentences into dependency parsing graphs, using word2vec + learnable POS tag embeddings for node features and encoding syntactic dependencies in edge relations.

Result: Hyperbolic Graph Neural Networks (HGNNs) in Poincaré space achieved performance superior to state-of-the-art while using up to 30x fewer parameters, with HGNNs benefiting significantly from modeling hierarchical tree-like structures.

Conclusion: Graph-based approaches, particularly HGNNs, provide lightweight yet effective alternatives to transformer models for environmental claim detection, offering superior performance with dramatically reduced computational requirements.

Abstract: Transformer based models, specially large language models (LLMs) dominate the field of NLP with their mass adoption in tasks such as text generation, summarization and fake news detection. These models offer ease of deployment and reliability for most applications, however, they require significant amounts of computational power for training as well as inference. This poses challenges in their adoption in resource-constrained applications, specially in the open-source community where compute availability is usually scarce. This work proposes a graph-based approach for Environmental Claim Detection, exploring Graph Neural Networks (GNNs) and Hyperbolic Graph Neural Networks (HGNNs) as lightweight yet effective alternatives to transformer-based models. Re-framing the task as a graph classification problem, we transform claim sentences into dependency parsing graphs, utilizing a combination of word2vec & learnable part-of-speech (POS) tag embeddings for the node features and encoding syntactic dependencies in the edge relations. Our results show that our graph-based models, particularly HGNNs in the poincar'e space (P-HGNNs), achieve performance superior to the state-of-the-art on environmental claim detection while using upto \textbf{30x fewer parameters}. We also demonstrate that HGNNs benefit vastly from explicitly modeling data in hierarchical (tree-like) structures, enabling them to significantly improve over their euclidean counterparts.

Method: Three-module framework: (1) LLM extracts and chronologically sorts sentiment intensity features, (2) fuses sentiment features with source text embeddings for alignment, (3) uses hard prompts with aligned vectors for rumor detection and sentiment analysis using one frozen LLM.

Result: Significant improvements on four rumor detection benchmarks compared to other emotion-based methods, with minimal parameter fine-tuning.

Conclusion: MSuf effectively enhances LLM performance for rumor detection by leveraging time-series dual sentiments and aligned embeddings, providing a robust framework for rumor tracking.

Abstract: The widespread dissemination of rumors on social media has a significant impact on people’s lives, potentially leading to public panic and fear. Rumors often evoke specific sentiments, resonating with readers and prompting sharing. To effectively detect and track rumors, it is essential to observe the fine-grained sentiments of both source and response message pairs as the rumor evolves over time. However, current rumor detection methods fail to account for this aspect. In this paper, we propose MSuf, the first multi-task suffix learning framework for rumor detection and tracking using time series dual (coupled) sentiments. MSuf includes three modules: (1) an LLM to extract sentiment intensity features and sort them chronologically; (2) a module that fuses the sorted sentiment features with their source text word embeddings to obtain an aligned embedding; (3) two hard prompts are combined with the aligned vector to perform rumor detection and sentiment analysis using one frozen LLM. MSuf effectively enhances the performance of LLMs for rumor detection with only minimal parameter fine-tuning. Evaluating MSuf on four rumor detection benchmarks, we find significant improvements compared to other emotion-based methods.

Method: Design a structural elicitation strategy to frame LLM explanations into graph topology, decomposing explanations into knowledge-related sub-questions and topology-based reasoning structures to quantify uncertainty at both semantic and reasoning path levels.

Result: The method enables systematic interpretation of LLM reasoning, analysis of limitations, and provides guidance for enhancing robustness and faithfulness. It also facilitates assessment of knowledge redundancy and offers interpretable insights into the reasoning process.

Conclusion: This work pioneers graph-structured uncertainty measurement in LLM explanations and demonstrates the potential of topology-based quantification for improving reliability assessment of LLM outputs.

Abstract: Understanding the uncertainty in large language model (LLM) explanations is important for evaluating their faithfulness and reasoning consistency, and thus provides insights into the reliability of LLM’s output regarding a question. In this work, we propose a novel framework that quantifies uncertainty in LLM explanations through a reasoning topology perspective. By designing a structural elicitation strategy, we guide the LLMs to frame the explanations of an answer into a graph topology. This process decomposes the explanations into the knowledge related sub-questions and topology-based reasoning structures, which allows us to quantify uncertainty not only at the semantic level but also from the reasoning path. It further brings convenience to assess knowledge redundancy and provide interpretable insights into the reasoning process. Our method offers a systematic way to interpret the LLM reasoning, analyze limitations, and provide guidance for enhancing robustness and faithfulness. This work pioneers the use of graph-structured uncertainty measurement in LLM explanations and demonstrates the potential of topology-based quantification.

Method: Translation-based experiments examining distortion accumulation over iterative generation chains, analyzing effects of language choice and chain complexity.

Result: Distortion accumulates over time in iterative LLM processing, influenced by language choice and chain complexity, though strategic prompting can mitigate degradation.

Conclusion: While information degradation in iterative LLM workflows is inevitable, careful prompting strategies can reduce distortion, raising important questions about the reliability of AI-generated content in recursive processes.

Abstract: As large language models are increasingly responsible for online content, concerns arise about the impact of repeatedly processing their own outputs. Inspired by the “broken telephone” effect in chained human communication, this study investigates whether LLMs similarly distort information through iterative generation. Through translation-based experiments, we find that distortion accumulates over time, influenced by language choice and chain complexity. While degradation is inevitable, it can be mitigated through strategic prompting techniques. These findings contribute to discussions on the long-term effects of AI-mediated information propagation, raising important questions about the reliability of LLM-generated content in iterative workflows.

Method: Propose LinguaLens framework using Sparse Auto-Encoders (SAEs) to extract linguistic features across morphology, syntax, semantics, and pragmatics, and construct large-scale counterfactual datasets for analysis.

Result: Revealed intrinsic representations of linguistic knowledge in LLMs, uncovered cross-layer and cross-lingual distribution patterns, and demonstrated potential for controlling model outputs.

Conclusion: Provides systematic resources for studying linguistic mechanisms, offers strong evidence that LLMs possess genuine linguistic knowledge, and lays foundation for more interpretable and controllable language modeling.

Abstract: Large language models (LLMs) demonstrate exceptional performance on tasks requiring complex linguistic abilities, such as reference disambiguation and metaphor recognition/generation. Although LLMs possess impressive capabilities, their internal mechanisms for processing and representing linguistic knowledge remain largely opaque. Prior research on linguistic mechanisms is limited by coarse granularity, limited analysis scale, and narrow focus. In this study, we propose LinguaLens, a systematic and comprehensive framework for analyzing the linguistic mechanisms of large language models, based on Sparse Auto-Encoders (SAEs). We extract a broad set of Chinese and English linguistic features across four dimensions (morphology, syntax, semantics, and pragmatics). By employing counterfactual methods, we construct a large-scale counterfactual dataset of linguistic features for mechanism analysis. Our findings reveal intrinsic representations of linguistic knowledge in LLMs, uncover patterns of cross-layer and cross-lingual distribution, and demonstrate the potential to control model outputs. This work provides a systematic suite of resources and methods for studying linguistic mechanisms, offers strong evidence that LLMs possess genuine linguistic knowledge, and lays the foundation for more interpretable and controllable language modeling in future research.

Method: MD uses a monitor function to identify hallucination-prone tokens during generation and applies tree-based decoding strategy to refine these tokens through in-process interventions.

Result: Experimental results show MD outperforms self-consistency approaches in both effectiveness and efficiency, achieving higher factual accuracy with significantly reduced computational overhead.

Conclusion: The proposed Monitoring Decoding framework provides an efficient and effective solution for mitigating hallucinations in LLMs by dynamically monitoring and intervening during the generation process.

Abstract: While large language models have demonstrated exceptional performance across a wide range of tasks, they remain susceptible to hallucinations – generating plausible yet factually incorrect contents. Existing methods to mitigating such risk often rely on sampling multiple full-length generations, which introduces significant response latency and becomes ineffective when the model consistently produces hallucinated outputs with high confidence. To address these limitations, we introduce Monitoring Decoding (MD), a novel framework that dynamically monitors the generation process and selectively applies in-process interventions, focusing on revising crucial tokens responsible for hallucinations. Instead of waiting until completion of multiple full-length generations, we identify hallucination-prone tokens during generation using a monitor function, and further refine these tokens through a tree-based decoding strategy. This approach ensures an enhanced factual accuracy and coherence in the generated output while maintaining efficiency. Experimental results demonstrate that MD consistently outperforms self-consistency-based approaches in both effectiveness and efficiency, achieving higher factual accuracy while significantly reducing computational overhead.

Method: Propose Chain-of-Strategy Optimization (CSO) with Monte Carlo Tree Search to construct ESC-Pro preference dataset, optimizing strategy selection preferences at each dialogue turn.

Result: CSO improves both strategy accuracy and bias mitigation, enabling more empathetic and contextually appropriate responses. Outperforms standard SFT on LLaMA-3.1-8B, Gemma-2-9B, and Qwen2.5-7B.

Conclusion: Fine-grained, turn-level preference modeling is effective for emotional support conversations, addressing the limitations of rigid SFT approaches.

Abstract: The growing emotional stress in modern society has increased the demand for Emotional Support Conversations (ESC). While Large Language Models (LLMs) show promise for ESC, they face two key challenges: (1) low strategy selection accuracy, and (2) preference bias, limiting their adaptability to emotional needs of users. Existing supervised fine-tuning (SFT) struggles to address these issues, as it rigidly trains models on single gold-standard responses without modeling nuanced strategy trade-offs. To overcome these limitations, we propose Chain-of-Strategy Optimization (CSO), a novel approach that optimizes strategy selection preferences at each dialogue turn. We first leverage Monte Carlo Tree Search to construct ESC-Pro, a high-quality preference dataset with turn-level strategy-response pairs. Training on ESC-Pro with CSO improves both strategy accuracy and bias mitigation, enabling LLMs to generate more empathetic and contextually appropriate responses. Experiments on LLaMA-3.1-8B, Gemma-2-9B, and Qwen2.5-7B demonstrate that CSO outperforms standard SFT, highlighting the efficacy of fine-grained, turn-level preference modeling in ESC.

Method: Extract features from attention matrix capturing token influence, attention bias, and context reference scope. Use Transformer-based classifier for sequential labeling to identify hallucinated spans.

Result: Outperforms strong baselines on hallucinated span detection, particularly with longer input contexts like data-to-text and summarization tasks.

Conclusion: Attention-based features provide effective signals for detecting hallucinated spans in LLM outputs, especially in contexts requiring broader contextual understanding.

Abstract: This study addresses the problem of hallucinated span detection in the outputs of large language models. It has received less attention than output-level hallucination detection despite its practical importance. Prior work has shown that attentions often exhibit irregular patterns when hallucinations occur. Motivated by these findings, we extract features from the attention matrix that provide complementary views capturing (a) whether certain tokens are influential or ignored, (b) whether attention is biased toward specific subsets, and (c) whether a token is generated referring to a narrow or broad context, in the generation. These features are input to a Transformer-based classifier to conduct sequential labelling to identify hallucinated spans. Experimental results indicate that the proposed method outperforms strong baselines on hallucinated span detection with longer input contexts, such as data-to-text and summarisation tasks.

Method: Combined Noel Carroll’s evaluative framework with art criticism theories for critique generation, used step-by-step prompting, conducted Turing test-style evaluations with human experts, and developed new Theory of Mind tasks involving interpretation, emotion, and moral tension in art contexts to test 41 recent LLMs.

Result: Human subjects often couldn’t distinguish AI-generated critiques from human-written ones. LLMs showed varying performance across Theory of Mind tasks, with affective and ambiguous situations revealing clearer differences between models. Carefully guided prompts enabled LLMs to produce plausible, interpretation-rich critiques.

Conclusion: LLMs can produce expert-like art critiques and handle some complex reasoning tasks when properly instructed, suggesting they may exhibit understanding-like behaviors more closely than assumed, though cognitive limitations remain evident.

Abstract: This study explored how large language models (LLMs) perform in two areas related to art: writing critiques of artworks and reasoning about mental states (Theory of Mind, or ToM) in art-related situations. For the critique generation part, we built a system that combines Noel Carroll’s evaluative framework with a broad selection of art criticism theories. The model was prompted to first write a full-length critique and then shorter, more coherent versions using a step-by-step prompting process. These AI-generated critiques were then compared with those written by human experts in a Turing test-style evaluation. In many cases, human subjects had difficulty telling which was which, and the results suggest that LLMs can produce critiques that are not only plausible in style but also rich in interpretation, as long as they are carefully guided. In the second part, we introduced new simple ToM tasks based on situations involving interpretation, emotion, and moral tension, which can appear in the context of art. These go beyond standard false-belief tests and allow for more complex, socially embedded forms of reasoning. We tested 41 recent LLMs and found that their performance varied across tasks and models. In particular, tasks that involved affective or ambiguous situations tended to reveal clearer differences. Taken together, these results help clarify how LLMs respond to complex interpretative challenges, revealing both their cognitive limitations and potential. While our findings do not directly contradict the so-called Generative AI Paradox–the idea that LLMs can produce expert-like output without genuine understanding–they suggest that, depending on how LLMs are instructed, such as through carefully designed prompts, these models may begin to show behaviors that resemble understanding more closely than we might assume.

Method: Uses algorithm-guided search with caching for knowledge reuse, decomposes premise selection into linear process, and employs LLM-free heuristics for premise prioritization.

Result: Achieves 23.6% and 12.5% average gains over CoT and ToT respectively on GPT-4o, with GPT-4o outperforming o3-mini by 7.6% within the framework.

Conclusion: LogicTree effectively addresses LLM reasoning limitations through structured search and premise optimization, demonstrating superior performance across multiple datasets.

Abstract: Large language models (LLMs) have achieved remarkable multi-step reasoning capabilities across various domains. However, LLMs still face distinct challenges in complex logical reasoning, as (1) proof-finding requires systematic exploration and the maintenance of logical coherence and (2) searching the right combination of premises at each reasoning step is inherently challenging in tasks with large premise space. To address this, we propose LogicTree, an inference-time modular framework employing algorithm-guided search to automate structured proof exploration and ensure logical coherence. Advancing beyond tree-of-thought (ToT), we incorporate caching mechanism into LogicTree to enable effective utilization of historical knowledge, preventing reasoning stagnation and minimizing redundancy. Furthermore, we address the combinatorial complexity of premise search by decomposing it into a linear process. The refined premise selection restricts subsequent inference to at most one derivation per step, enhancing reasoning granularity and enforcing strict step-by-step reasoning. Additionally, we introduce two LLM-free heuristics for premise prioritization, enabling strategic proof search. Experimental results on five datasets demonstrate that LogicTree optimally scales inference-time computation to achieve higher proof accuracy, surpassing chain-of-thought (CoT) and ToT with average gains of 23.6% and 12.5%, respectively, on GPT-4o. Moreover, within LogicTree, GPT-4o outperforms o3-mini by 7.6% on average.

Method: Uses a new architecture with steering vector generator and applier modules that automatically generate and apply steering vectors to influence model behavior based on single examples.

Result: Demonstrated effective model steering performance across different LLMs, enabling precise control of safety, sentiment, personality, reasoning patterns, factuality, and language features.

Conclusion: EasyEdit2 successfully provides an accessible, efficient framework for plug-and-play LLM behavior adjustment through steering vectors, making precise model control available to users with minimal technical expertise.

Abstract: In this paper, we introduce EasyEdit2, a framework designed to enable plug-and-play adjustability for controlling Large Language Model (LLM) behaviors. EasyEdit2 supports a wide range of test-time interventions, including safety, sentiment, personality, reasoning patterns, factuality, and language features. Unlike its predecessor, EasyEdit2 features a new architecture specifically designed for seamless model steering. It comprises key modules such as the steering vector generator and the steering vector applier, which enable automatic generation and application of steering vectors to influence the model’s behavior without modifying its parameters. One of the main advantages of EasyEdit2 is its ease of use-users do not need extensive technical knowledge. With just a single example, they can effectively guide and adjust the model’s responses, making precise control both accessible and efficient. Empirically, we report model steering performance across different LLMs, demonstrating the effectiveness of these techniques. We have released the source code on GitHub at https://github.com/zjunlp/EasyEdit along with a demonstration notebook. In addition, we provide a demo video at https://www.youtube.com/watch?v=AkfoiPfp5rQ for a quick introduction.

Method: Evaluated GPT-4o, Gemma-2-9B, Gemma-2-2B, and Llama-3.1-8B using the IndicQuest dataset containing question-answer pairs in English and 19 Indic languages, comparing performance on same questions in both languages.

Result: LLMs consistently performed better in English even for questions rooted in Indic contexts, with significantly higher hallucination rates in low-resource Indic languages.

Conclusion: Current multilingual LLMs face challenges in factual accuracy and understanding capabilities for low-resource languages, demonstrating limitations in their multilingual capabilities despite strong English performance.

Abstract: Multilingual Large Language Models (LLMs) have demonstrated significant effectiveness across various languages, particularly in high-resource languages such as English. However, their performance in terms of factual accuracy across other low-resource languages, especially Indic languages, remains an area of investigation. In this study, we assess the factual accuracy of LLMs - GPT-4o, Gemma-2-9B, Gemma-2-2B, and Llama-3.1-8B - by comparing their performance in English and Indic languages using the IndicQuest dataset, which contains question-answer pairs in English and 19 Indic languages. By asking the same questions in English and their respective Indic translations, we analyze whether the models are more reliable for regional context questions in Indic languages or when operating in English. Our findings reveal that LLMs often perform better in English, even for questions rooted in Indic contexts. Notably, we observe a higher tendency for hallucination in responses generated in low-resource Indic languages, highlighting challenges in the multilingual understanding capabilities of current LLMs.

Method: Developed improved methods to synthesize training sets that incorporate natural spelling variation, comparing synthetic samples with naturally occurring examples and testing different model capacities.

Result: Achieved new state-of-the-art performance: improved test F1 from 74.7% (pretrained neural model) to 85.4% using linear classifier on synthetic data alone, and 88.2% when also training on harvested text.

Conclusion: Training on synthetic samples with natural spelling variation yields higher language identification accuracy than using naturally occurring examples or higher capacity models, demonstrating effective approach for romanized text LID.

Abstract: The Latin script is often used to informally write languages with non-Latin native scripts. In many cases (e.g., most languages in India), the lack of conventional spelling in the Latin script results in high spelling variability. Such romanization renders languages that are normally easily distinguished due to being written in different scripts - Hindi and Urdu, for example - highly confusable. In this work, we increase language identification (LID) accuracy for romanized text by improving the methods used to synthesize training sets. We find that training on synthetic samples which incorporate natural spelling variation yields higher LID system accuracy than including available naturally occurring examples in the training set, or even training higher capacity models. We demonstrate new state-of-the-art LID performance on romanized text from 20 Indic languages in the Bhasha-Abhijnaanam evaluation set (Madhani et al., 2023a), improving test F1 from the reported 74.7% (using a pretrained neural model) to 85.4% using a linear classifier trained solely on synthetic data and 88.2% when also training on available harvested text.

Method: Multiple Answers and Confidence Stepwise Tuning (MAC-Tuning) - a novel approach that separates the learning of answer prediction and confidence estimation during fine-tuning on instruction data.

Result: Extensive experiments show the method outperforms baselines by up to 25% in average precision for multi-problem settings.

Conclusion: MAC-Tuning effectively addresses LLM hallucination in complex multi-problem scenarios through stepwise separation of answer and confidence learning during fine-tuning.

Abstract: The hallucination of non-existent facts by LLMs is an important problem given its widespread adoption across various applications. Previous research addresses this problem by analyzing the internal parameterized knowledge boundaries to estimate confidence. However, these studies focus on the single-problem setting and have not explored the more challenging multi-problem setting, which requires accurately answering multiple questions simultaneously. We introduce a novel method for the multi-problem setting, Multiple Answers and Confidence Stepwise Tuning (MAC-Tuning), that separates the learning of answer prediction and confidence estimation during fine-tuning on instruction data. Extensive experiments demonstrate that our method outperforms baselines by up to 25% in average precision.

Method: Studied base vs aligned models on tasks requiring unpredictable outputs: random number generation, mixed strategy games (rock-paper-scissors, hide-and-seek), and creative writing across multiple models.

Result: Aligned models consistently underperformed base models, showing predictable patterns (e.g., preferring “7” in random generation, becoming predictable in games, prioritizing pleasant over creative writing). Better benchmark performance correlated with worse performance on these tasks.

Conclusion: There’s a trade-off between alignment and unpredictability capabilities - alignment techniques should be selectively applied rather than universally used, as they can degrade performance on tasks requiring randomness and creativity.

Abstract: Alignment has quickly become a default ingredient in LLM development, with techniques such as reinforcement learning from human feedback making models act safely, follow instructions, and perform ever-better on complex tasks. While these techniques are certainly useful, we propose that they should not be universally applied and demonstrate a range of tasks on which base language models consistently outperform their popular aligned forms. Particularly, we study tasks that require unpredictable outputs, such as random number generation, mixed strategy games (rock-paper-scissors and hide-and-seek), and creative writing. In each case, aligned models tend towards narrow behaviors that result in distinct disadvantages, for instance, preferring to generate “7” over other uniformly random numbers, becoming almost fully predictable in some game states, or prioritizing pleasant writing over creative originality. Across models tested, better performance on common benchmarks tends to correlate with worse performance on our tasks, suggesting an effective trade-off in the required capabilities.

Method: Constructs context graph from entities/concepts, uses graph walk for knowledge-associated sampling, and integrates Chain-of-Thought and Contrastive Clarifying strategies to enhance reasoning and discrimination.

Result: Surpasses SOTA methods on multi-hop and domain-specific QA, achieves competitive performance on long-context reading comprehension, demonstrating superior generalization ability.

Conclusion: SoG advances synthetic data generation paradigm and offers practical solutions for efficient knowledge acquisition in LLMs, especially for domains with limited training data.

Abstract: Large Language Models (LLMs) have achieved remarkable success but remain data-inefficient, especially when learning from small, specialized corpora with limited and proprietary data. Existing synthetic data generation methods for continue pre-training focus on intra-document content and overlook cross-document knowledge associations, limiting content diversity and depth. We propose Synthetic-on-Graph (SoG), a synthetic data generation framework that incorporates cross-document knowledge associations for efficient corpus expansion. SoG constructs a context graph by extracting entities and concepts from the original corpus, representing cross-document associations, and employing a graph walk strategy for knowledge-associated sampling. This enhances synthetic data diversity and coherence, enabling models to learn complex knowledge structures and handle rare knowledge. To further improve the quality of synthetic data, we integrate two complementary strategies, Chain-of-Thought (CoT) and Contrastive Clarifying (CC), to enhance both reasoning capability and discriminative power. Extensive experiments demonstrate that SoG surpasses state-of-the-art (SOTA) methods on multi-hop and domain-specific question answering, while achieving competitive performance on long-context reading comprehension. These results highlight the superior generalization ability of SoG. Our work advances the paradigm of synthetic data generation and offers practical solutions for efficient knowledge acquisition in LLMs, particularly for downstream tasks and domains with limited training data.

Method: Proposed Multilingual Collaborative Defense (MCD) - a learning method that automatically optimizes continuous, soft safety prompts to facilitate multilingual safeguarding. Evaluated using manually constructed multilingual versions of jailbreak benchmarks (MaliciousInstruct, AdvBench) including underrepresented languages.

Result: MCD outperforms existing approaches in safeguarding against multilingual jailbreak attempts. It effectively improves safeguarding across multiple languages, maintains strong generalization with minimal false refusal rates, and mitigates language safety misalignment from training data imbalances.

Conclusion: MCD demonstrates strong language transfer capabilities and provides superior multilingual protection for LLMs against translation-based jailbreak attacks, addressing a critical security vulnerability in current language models.

Abstract: The robustness and security of large language models (LLMs) has become a prominent research area. One notable vulnerability is the ability to bypass LLM safeguards by translating harmful queries into rare or underrepresented languages, a simple yet effective method of “jailbreaking” these models. Despite the growing concern, there has been limited research addressing the safeguarding of LLMs in multilingual scenarios, highlighting an urgent need to enhance multilingual safety. In this work, we investigate the correlation between various attack features across different languages and propose Multilingual Collaborative Defense (MCD), a novel learning method that optimizes a continuous, soft safety prompt automatically to facilitate multilingual safeguarding of LLMs. The MCD approach offers three advantages: First, it effectively improves safeguarding performance across multiple languages. Second, MCD maintains strong generalization capabilities while minimizing false refusal rates. Third, MCD mitigates the language safety misalignment caused by imbalances in LLM training corpora. To evaluate the effectiveness of MCD, we manually construct multilingual versions of commonly used jailbreak benchmarks, such as MaliciousInstruct and AdvBench, to assess various safeguarding methods. Additionally, we introduce these datasets in underrepresented (zero-shot) languages to verify the language transferability of MCD. The results demonstrate that MCD outperforms existing approaches in safeguarding against multilingual jailbreak attempts while also exhibiting strong language transfer capabilities. Our code is available at https://github.com/HLiang-Lee/MCD.

Method: Two-stage approach: 1) Self-Driven Policy Warm-Up for cold-start via retrieval-guided self-distillation, 2) Retrieval-Guided RL with rank-incentive reward shaping to address metric sparsity.

Result: Outperforms previous SOTA methods by over 10% on TopiOCQA dataset using smaller 3B parameter models without any external supervision.

Conclusion: The framework successfully eliminates dependency on external rewrite supervision while achieving superior performance through direct optimization with retrieval signals.

Abstract: Conversational search systems require effective handling of context-dependent queries that often contain ambiguity, omission, and coreference. Conversational Query Reformulation (CQR) addresses this challenge by transforming these queries into self-contained forms suitable for off-the-shelf retrievers. However, existing CQR approaches suffer from two critical constraints: high dependency on costly external supervision from human annotations or large language models, and insufficient alignment between the rewriting model and downstream retrievers. We present ConvSearch-R1, the first self-driven framework that completely eliminates dependency on external rewrite supervision by leveraging reinforcement learning to optimize reformulation directly through retrieval signals. Our novel two-stage approach combines Self-Driven Policy Warm-Up to address the cold-start problem through retrieval-guided self-distillation, followed by Retrieval-Guided Reinforcement Learning with a specially designed rank-incentive reward shaping mechanism that addresses the sparsity issue in conventional retrieval metrics. Extensive experiments on TopiOCQA and QReCC datasets demonstrate that ConvSearch-R1 significantly outperforms previous state-of-the-art methods, achieving over 10% improvement on the challenging TopiOCQA dataset while using smaller 3B parameter models without any external supervision.

Method: Developed a hierarchical multi-agent system based on MQM error typology, incorporating self-reflection capabilities and agent discussion with asymmetric information to mitigate hallucinations.

Result: Outperforms competitive baselines across datasets, achieves 89% average F1-score improvement in error span detection and severity assessment over best baseline.

Conclusion: HiMATE successfully addresses LLM hallucination issues and enables more granular, human-aligned MT evaluation through hierarchical multi-agent architecture and error mitigation strategies.

Abstract: The advancement of Large Language Models (LLMs) enables flexible and interpretable automatic evaluations. In the field of machine translation evaluation, utilizing LLMs with translation error annotations based on Multidimensional Quality Metrics (MQM) yields more human-aligned judgments. However, current LLM-based evaluation methods still face challenges in accurately identifying error spans and assessing their severity. In this paper, we propose HiMATE, a Hierarchical Multi-Agent Framework for Machine Translation Evaluation. We argue that existing approaches inadequately exploit the fine-grained structural and semantic information within the MQM hierarchy. To address this, we develop a hierarchical multi-agent system grounded in the MQM error typology, enabling granular evaluation of subtype errors. Two key strategies are incorporated to further mitigate systemic hallucinations within the framework: the utilization of the model’s self-reflection capability and the facilitation of agent discussion involving asymmetric information. Empirically, HiMATE outperforms competitive baselines across different datasets in conducting human-aligned evaluations. Further analyses underscore its significant advantage in error span detection and severity assessment, achieving an average F1-score improvement of 89% over the best-performing baseline. We make our code and data publicly available at https://github.com/nlp2ct-shijie/HiMATE.

Method: Introduces stochastic method of moments evaluation over meaning-preserving prompt perturbations and ReliableEval method to estimate required prompt resamplings for reliable results.

Result: Evaluation of five frontier LLMs shows substantial prompt sensitivity even in top-performing models like GPT-4o and Claude-3.7-Sonnet.

Conclusion: Provides a model-, task-, and metric-agnostic framework for robust LLM evaluation that accounts for prompt sensitivity through stochastic sampling.

Abstract: LLMs are highly sensitive to prompt phrasing, yet standard benchmarks typically report performance using a single prompt, raising concerns about the reliability of such evaluations. In this work, we argue for a stochastic method of moments evaluation over the space of meaning-preserving prompt perturbations. We introduce a formal definition of reliable evaluation that accounts for prompt sensitivity, and suggest ReliableEval - a method for estimating the number of prompt resamplings needed to obtain meaningful results. Using our framework, we stochastically evaluate five frontier LLMs and find that even top-performing models like GPT-4o and Claude-3.7-Sonnet exhibit substantial prompt sensitivity. Our approach is model-, task-, and metric-agnostic, offering a recipe for meaningful and robust LLM evaluation.

Method: Active Layer-Contrastive Decoding (ActLCD) treats decoding as a sequential decision-making problem using reinforcement learning with a reward-aware classifier to optimize when to apply contrasting layers.

Result: ActLCD outperforms state-of-the-art methods across five benchmarks, demonstrating superior effectiveness in mitigating hallucinations across diverse generation scenarios.

Conclusion: ActLCD represents an advancement in decoding strategies by actively managing layer contrast through reinforcement learning, significantly improving LLM factuality beyond token-level approaches.

Abstract: Recent decoding methods improve the factuality of large language models (LLMs) by refining how the next token is selected during generation. These methods typically operate at the token level, leveraging internal representations to suppress superficial patterns. Nevertheless, LLMs remain prone to hallucinations, especially over longer contexts. In this paper, we propose Active Layer-Contrastive Decoding (ActLCD), a novel decoding strategy that actively decides when to apply contrasting layers during generation. By casting decoding as a sequential decision-making problem, ActLCD employs a reinforcement learning policy guided by a reward-aware classifier to optimize factuality beyond the token level. Our experiments demonstrate that ActLCD surpasses state-of-the-art methods across five benchmarks, showcasing its effectiveness in mitigating hallucinations in diverse generation scenarios.

Method: Constructed from play-by-play text commentary to evaluate three critical aspects: Ultra Multi-turn, Interactive Multi-turn, and Cross-turn Tasks. Includes attention visualization experiments on Qwen2.5-7B-Instruction.

Result: Closed-source LLMs significantly outperform open-source alternatives. Explicit reasoning boosts robustness. LLMs struggle with motivation transfer and cross-turn dependency. Attention sinks from special tokens cause performance degradation.

Conclusion: MARS-Bench effectively reveals LLM weaknesses in complex dialogue handling and provides mechanistic insights into performance degradation through attention visualization.

Abstract: Large Language Models (\textbf{LLMs}), e.g. ChatGPT, have been widely adopted in real-world dialogue applications. However, LLMs’ robustness, especially in handling long complex dialogue sessions, including frequent motivation transfer, sophisticated cross-turn dependency, is criticized all along. Nevertheless, no existing benchmarks can fully reflect these weaknesses. We present \textbf{MARS-Bench}, a \textbf{M}ulti-turn \textbf{A}thletic \textbf{R}eal-world \textbf{S}cenario Dialogue \textbf{Bench}mark, designed to remedy the gap. MARS-Bench is constructed from play-by-play text commentary so to feature realistic dialogues specifically designed to evaluate three critical aspects of multi-turn conversations: Ultra Multi-turn, Interactive Multi-turn, and Cross-turn Tasks. Extensive experiments on MARS-Bench also reveal that closed-source LLMs significantly outperform open-source alternatives, explicit reasoning significantly boosts LLMs’ robustness on handling long complex dialogue sessions, and LLMs indeed face significant challenges when handling motivation transfer and sophisticated cross-turn dependency. Moreover, we provide mechanistic interpretability on how attention sinks due to special tokens lead to LLMs’ performance degradation when handling long complex dialogue sessions based on attention visualization experiment in Qwen2.5-7B-Instruction.

Method: Combines embedding perturbation for controlled exploration with Bayesian optimization to refine embeddings using a verifier-guided objective, balancing exploration and exploitation.

Result: Experiments show superior reasoning correctness and coherence with minimal computational overhead, demonstrating improved performance.

Conclusion: The framework provides a scalable, model-agnostic solution that enhances LLM reasoning without relying on heuristic search methods.

Abstract: Large Language Models (LLMs) struggle with complex reasoning due to limited diversity and inefficient search. We propose Soft Reasoning, an embedding-based search framework that optimises the embedding of the first token to guide generation. It combines (1) embedding perturbation for controlled exploration and (2) Bayesian optimisation to refine embeddings via a verifier-guided objective, balancing exploration and exploitation. This approach improves reasoning accuracy and coherence while avoiding reliance on heuristic search. Experiments demonstrate superior correctness with minimal computation, making it a scalable, model-agnostic solution. The code is released at https://github.com/alickzhu/Soft-Reasoning.

Method: Jointly optimizes which attention blocks to skip and how to scale outputs of remaining layers using lightweight trainable scaling parameters for attention and MLP blocks to mitigate distribution shifts.

Result: Applied to Llama-3.1-8B and Qwen2.5-7B, achieves less than 2% performance drop even after skipping four attention blocks.

Conclusion: Hopscotch provides an effective way to reduce computational overhead without modifying model weights or requiring pretraining data, while maintaining compatibility with existing compression techniques.

Abstract: Modern causal language models stack many attention blocks to improve performance, but not all blocks are necessary for every task. We propose Hopscotch, a simple yet effective method that identifies and skips attention blocks with least contributions to a task and adapts to preserve output quality. Hopscotch jointly optimizes which blocks to skip and how to scale the outputs of the remaining layers. By introducing lightweight, trainable scaling parameters to attention and MLP blocks, it mitigates distribution shifts in hidden states caused by removing attention blocks. Hopscotch does not modify model weights or require access to pretraining or instruction-tuning data, and is compatible with existing model compression techniques. When applied to $\texttt{Llama-3.1-8B}$ and $\texttt{Qwen2.5-7B}$, Hopscotch achieves less than a 2% drop in performance even after skipping four attention blocks.

Method: REWIRE (REcycling the Web with guIded REwrite) - a method to enrich low-quality documents through guided rewriting to make them useful for training, increasing synthetic data representation in pre-training sets.

Result: 1.0, 1.3 and 2.5 percentage points improvement at 1B, 3B and 7B scales across 22 diverse tasks; more effective than 2x web data; 82% of mixed texts come from transformed low-quality documents; outperforms other synthetic data generation approaches.

Conclusion: Recycling web texts through guided rewriting is a simple and effective approach for scaling pre-training data, addressing the data scarcity problem in large language model training.

Abstract: Scaling laws predict that the performance of large language models improves with increasing model size and data size. In practice, pre-training has been relying on massive web crawls, using almost all data sources publicly available on the internet so far. However, this pool of natural data does not grow at the same rate as the compute supply. Furthermore, the availability of high-quality texts is even more limited: data filtering pipelines often remove up to 99% of the initial web scrapes to achieve state-of-the-art. To address the “data wall” of pre-training scaling, our work explores ways to transform and recycle data discarded in existing filtering processes. We propose REWIRE, REcycling the Web with guIded REwrite, a method to enrich low-quality documents so that they could become useful for training. This in turn allows us to increase the representation of synthetic data in the final pre-training set. Experiments at 1B, 3B and 7B scales of the DCLM benchmark show that mixing high-quality raw texts and our rewritten texts lead to 1.0, 1.3 and 2.5 percentage points improvement respectively across 22 diverse tasks, compared to training on only filtered web data. Training on the raw-synthetic data mix is also more effective than having access to 2x web data. Through further analysis, we demonstrate that about 82% of the mixed in texts come from transforming lower-quality documents that would otherwise be discarded. REWIRE also outperforms related approaches of generating synthetic data, including Wikipedia-style paraphrasing, question-answer synthesizing and knowledge extraction. These results suggest that recycling web texts holds the potential for being a simple and effective approach for scaling pre-training data. We make our high-quality synthetic data publicly available at https://huggingface.co/datasets/facebook/recycling_the_web.

Method: Utilized a novel framework to determine if LLMs genuinely learn reasoning patterns or merely memorize solutions, focusing on STEM domains with self-validated, logically coherent task perturbations.

Result: LLMs draw confidence from memorized solutions to infer higher self-knowledge, showing over 45% inconsistency in feasibility assessments. This effect is most pronounced in science and medicine domains with standardized jargon.

Conclusion: Significant flaws in current architectures and training patterns highlight the need for techniques that ensure balanced, consistent self-perception of knowledge for maximum AI explainability and trustworthiness.

Abstract: When artificial intelligence mistakes memorization for intelligence, it creates a dangerous mirage of reasoning. Existing studies treat memorization and self-knowledge deficits in LLMs as separate issues and do not recognize an intertwining link that degrades the trustworthiness of LLM responses. In our study, we utilize a novel framework to ascertain if LLMs genuinely learn reasoning patterns from training data or merely memorize them to assume competence across problems of similar complexity focused on STEM domains. Our analysis shows a noteworthy problem in generalization: LLMs draw confidence from memorized solutions to infer a higher self-knowledge about their reasoning ability, which manifests as an over 45% inconsistency in feasibility assessments when faced with self-validated, logically coherent task perturbations. This effect is most pronounced in science and medicine domains, which tend to have maximal standardized jargon and problems, further confirming our approach. Significant wavering within the self-knowledge of LLMs also shows flaws in current architectures and training patterns, highlighting the need for techniques that ensure a balanced, consistent stance on models’ perceptions of their own knowledge for maximum AI explainability and trustworthiness. Our code and results are available publicly at https://github.com/knowledge-verse-ai/LLM-Memorization_SK_Eval-.

Method: Developed a computational framework with typology of talk-time sharing dynamics, applied to large dataset of video-chats between strangers to analyze perceptions.

Result: Balanced conversations preferred over imbalanced ones (especially by those talking less), and different dynamics with same overall balance are perceived differently.

Conclusion: The framework provides new tools for designing communication platforms (human-human and human-AI) by understanding talk-time dynamics and their impact on conversation quality.

Abstract: An intrinsic aspect of every conversation is the way talk-time is shared between multiple speakers. Conversations can be balanced, with each speaker claiming a similar amount of talk-time, or imbalanced when one talks disproportionately. Such overall distributions are the consequence of continuous negotiations between the speakers throughout the conversation: who should be talking at every point in time, and for how long? In this work we introduce a computational framework for quantifying both the conversation-level distribution of talk-time between speakers, as well as the lower-level dynamics that lead to it. We derive a typology of talk-time sharing dynamics structured by several intuitive axes of variation. By applying this framework to a large dataset of video-chats between strangers, we confirm that, perhaps unsurprisingly, different conversation-level distributions of talk-time are perceived differently by speakers, with balanced conversations being preferred over imbalanced ones, especially by those who end up talking less. Then we reveal that – even when they lead to the same level of overall balance – different types of talk-time sharing dynamics are perceived differently by the participants, highlighting the relevance of our newly introduced typology. Finally, we discuss how our framework offers new tools to designers of computer-mediated communication platforms, for both human-human and human-AI communication.

Method: Compared DeepSeek-R1 and DeepSeek-V3 with Qwen2.5, GPT-4o, and Llama-3.3 using ANES survey data (US) and Zuobiao dataset (China) to assess opinion prediction capabilities on various social issues.

Result: DeepSeek-V3 excels at US abortion opinions with Democratic/liberal personas but struggles with Chinese capitalism views, particularly for low-income/non-college groups. All models show overgeneralization within demographic groups.

Conclusion: LLMs exhibit cultural and demographic biases in public opinion modeling, requiring more inclusive training methodologies to mitigate these limitations.

Abstract: This study evaluates the ability of DeepSeek, an open-source large language model (LLM), to simulate public opinions in comparison to LLMs developed by major tech companies. By comparing DeepSeek-R1 and DeepSeek-V3 with Qwen2.5, GPT-4o, and Llama-3.3 and utilizing survey data from the American National Election Studies (ANES) and the Zuobiao dataset of China, we assess these models’ capacity to predict public opinions on social issues in both China and the United States, highlighting their comparative capabilities between countries. Our findings indicate that DeepSeek-V3 performs best in simulating U.S. opinions on the abortion issue compared to other topics such as climate change, gun control, immigration, and services for same-sex couples, primarily because it more accurately simulates responses when provided with Democratic or liberal personas. For Chinese samples, DeepSeek-V3 performs best in simulating opinions on foreign aid and individualism but shows limitations in modeling views on capitalism, particularly failing to capture the stances of low-income and non-college-educated individuals. It does not exhibit significant differences from other models in simulating opinions on traditionalism and the free market. Further analysis reveals that all LLMs exhibit the tendency to overgeneralize a single perspective within demographic groups, often defaulting to consistent responses within groups. These findings highlight the need to mitigate cultural and demographic biases in LLM-driven public opinion modeling, calling for approaches such as more inclusive training methodologies.

Method: Identifies leakage points through perturbation and rewrites them to counterfactual versions to disrupt memorization while preserving evaluative intent.

Result: Significant performance gaps revealed in state-of-the-art QA benchmarks, demonstrating effective reduction of memorization effects.

Conclusion: Provides a practical, scalable solution for maintaining benchmark robustness over time and enabling fairer, more interpretable LLM evaluations.

Abstract: The increasing complexity of large language models (LLMs) raises concerns about their ability to “cheat” on standard Question Answering (QA) benchmarks by memorizing task-specific data. This undermines the validity of benchmark evaluations, as they no longer reflect genuine model capabilities but instead the effects of data leakage. While prior work has focused on detecting such leakage, little attention has been given to mitigating its impact and preserving the long-term utility of benchmarks. In this paper, we introduce LastingBench, a novel framework designed to continuously reinforce and safeguard existing benchmarks against knowledge leakage. LastingBench identifies leakage points in the context through perturbation, then rewrites the leakage points to counterfactual ones-disrupting memorization while preserving the benchmark’s original evaluative intent. Evaluations of state-of-the-art QA benchmarks show significant performance gaps, highlighting the efficacy of LastingBench in reducing memorization effects. LastingBench offers a practical and scalable solution to ensure benchmark robustness over time, promoting fairer and more interpretable evaluations of LLMs.

Method: Leverages recent advances in large language models and precise layout detection to achieve high translation accuracy while maintaining document structure and formatting.

Result: Developed open-source software with over 222k downloads, demonstrating significant improvements in precision, flexibility, and efficiency for scientific document translation.

Conclusion: The tool successfully bridges the gap in scientific document translation by preserving layout information, making scientific knowledge more accessible across language barriers.

Abstract: Language barriers in scientific documents hinder the diffusion and development of science and technologies. However, prior efforts in translating such documents largely overlooked the information in layouts. To bridge the gap, we introduce PDFMathTranslate, the world’s first open-source software for translating scientific documents while preserving layouts. Leveraging the most recent advances in large language models and precise layout detection, we contribute to the community with key improvements in precision, flexibility, and efficiency. The work has been open-sourced at https://github.com/byaidu/pdfmathtranslate with more than 222k downloads.

Method: PersonaGen constructs layered virtual personas combining demographic attributes, socio-cultural backgrounds, and situational contexts to guide emotion expression generation through multi-stage persona-based conditioning using LLMs.

Result: PersonaGen significantly outperforms baseline methods in generating diverse, coherent, and discriminative emotion expressions, showing strong performance in semantic diversity, human-likeness, realism, and utility in downstream emotion classification tasks.

Conclusion: PersonaGen demonstrates potential as a robust alternative for augmenting or replacing real-world emotional datasets, effectively addressing data scarcity issues in emotion recognition through synthetic data generation.

Abstract: In the field of emotion recognition, the development of high-performance models remains a challenge due to the scarcity of high-quality, diverse emotional datasets. Emotional expressions are inherently subjective, shaped by individual personality traits, socio-cultural backgrounds, and contextual factors, making large-scale, generalizable data collection both ethically and practically difficult. To address this issue, we introduce PersonaGen, a novel framework for generating emotionally rich text using a Large Language Model (LLM) through multi-stage persona-based conditioning. PersonaGen constructs layered virtual personas by combining demographic attributes, socio-cultural backgrounds, and detailed situational contexts, which are then used to guide emotion expression generation. We conduct comprehensive evaluations of the generated synthetic data, assessing semantic diversity through clustering and distributional metrics, human-likeness via LLM-based quality scoring, realism through comparison with real-world emotion corpora, and practical utility in downstream emotion classification tasks. Experimental results show that PersonaGen significantly outperforms baseline methods in generating diverse, coherent, and discriminative emotion expressions, demonstrating its potential as a robust alternative for augmenting or replacing real-world emotional datasets.

Method: UR2 introduces difficulty-aware curriculum training that selectively invokes retrieval only for challenging problems, and a hybrid knowledge access strategy combining domain-specific offline corpora with LLM-generated summaries to enable dynamic coordination between retrieval and reasoning.

Result: UR2 significantly outperforms existing RAG and RL methods across open-domain QA, MMLU-Pro, medical, and mathematical reasoning tasks, achieving comparable performance to GPT-4o-mini and GPT-4.1-mini on several benchmarks.

Conclusion: The unified framework successfully bridges the gap between retrieval and reasoning capabilities, demonstrating improved adaptability and performance across diverse tasks while providing a generalizable approach for combining RAG and reinforcement learning.

Abstract: Large Language Models (LLMs) have shown remarkable capabilities through two complementary paradigms: Retrieval-Augmented Generation (RAG), which enhances knowledge grounding, and Reinforcement Learning from Verifiable Rewards (RLVR), which optimizes complex reasoning abilities. However, these two capabilities are often developed in isolation, and existing efforts to unify them remain narrow in scope – typically limited to open-domain QA with fixed retrieval settings and task-specific constraints. This lack of integration constrains generalization and limits the applicability of RAG-RL methods to broader domains. To bridge this gap, we propose UR2 (Unified RAG and Reasoning), a general framework that unifies retrieval and reasoning through reinforcement learning. UR2 introduces two key contributions: a difficulty-aware curriculum training that selectively invokes retrieval only for challenging problems, and a hybrid knowledge access strategy combining domain-specific offline corpora with LLM-generated summaries. These components are designed to enable dynamic coordination between retrieval and reasoning, improving adaptability across a diverse range of tasks. Experiments across open-domain QA, MMLU-Pro, medical, and mathematical reasoning tasks demonstrate that UR$^2$ (built on Qwen-2.5-3/7B and LLaMA-3.1-8B) significantly outperforms existing RAG and RL methods, achieving comparable performance to GPT-4o-mini and GPT-4.1-mini on several benchmarks. We have released all code, models, and data at https://github.com/Tsinghua-dhy/UR2.

Method: LMTransplant uses a transplant-then-regenerate paradigm: incorporating seed text into a context expanded by LLM, then asking the LLM to regenerate a variant based on the expanded context. This leverages LLM knowledge to create diverse content while maintaining original text attributes.

Result: LMTransplant demonstrates superior performance over existing text augmentation methods across various text-related tasks. It also shows exceptional scalability as the size of augmented data grows.

Conclusion: LMTransplant provides an effective paradigm for text augmentation that fully leverages LLM knowledge to generate diverse and creative content-level variants while preserving core text attributes, offering better performance and scalability than traditional methods.

Abstract: Data augmentation is a critical technique in deep learning. Traditional methods like Back-translation typically focus on lexical-level rephrasing, which primarily produces variations with the same semantics. While large language models (LLMs) have enhanced text augmentation by their “knowledge emergence” capability, controlling the style and structure of these outputs remains challenging and requires meticulous prompt engineering. In this paper, we propose LMTransplant, a novel text augmentation paradigm leveraging LLMs. The core idea of LMTransplant is transplant-then-regenerate: incorporating seed text into a context expanded by LLM, and asking the LLM to regenerate a variant based on the expanded context. This strategy allows the model to create more diverse and creative content-level variants by fully leveraging the knowledge embedded in LLMs, while preserving the core attributes of the original text. We evaluate LMTransplant across various text-related tasks, demonstrating its superior performance over existing text augmentation methods. Moreover, LMTransplant demonstrates exceptional scalability as the size of augmented data grows.

Method: The authors draw from multiple disciplines to propose a framework where anthropomorphism emerges from interaction between designers (embedding cues) and interpreters (responding to cues). Cues are categorized into four dimensions: perceptive, linguistic, behavioral, and cognitive.

Result: The paper provides a unified taxonomy with actionable levers for practitioners to design anthropomorphic LLM artifacts, analyzing the manifestation and effectiveness of each cue type.

Conclusion: The authors advocate for function-oriented evaluations of anthropomorphic design, positioning anthropomorphism as an intentional design concept that can be tuned to support user goals rather than treated as a risk.

Abstract: Large Language Models (LLMs) increasingly exhibit \textbf{anthropomorphism} characteristics – human-like qualities portrayed across their outlook, language, behavior, and reasoning functions. Such characteristics enable more intuitive and engaging human-AI interactions. However, current research on anthropomorphism remains predominantly risk-focused, emphasizing over-trust and user deception while offering limited design guidance. We argue that anthropomorphism should instead be treated as a \emph{concept of design} that can be intentionally tuned to support user goals. Drawing from multiple disciplines, we propose that the anthropomorphism of an LLM-based artifact should reflect the interaction between artifact designers and interpreters. This interaction is facilitated by cues embedded in the artifact by the designers and the (cognitive) responses of the interpreters to the cues. Cues are categorized into four dimensions: \textit{perceptive, linguistic, behavioral}, and \textit{cognitive}. By analyzing the manifestation and effectiveness of each cue, we provide a unified taxonomy with actionable levers for practitioners. Consequently, we advocate for function-oriented evaluations of anthropomorphic design.

Method: Comprehensive empirical analysis of three pruning methods and various calibration sets across three LLMs using four fairness metrics, leading to proposed High Gradient Low Activation (HGLA) pruning that removes parameters redundant for input but influential for output.

Result: Pruning methods impact fairness more than calibration sets. HGLA better maintains or improves fairness compared to existing methods across models and tasks, with human evaluation confirming HGLA outputs are fairer.

Conclusion: HGLA pruning shows promise for maintaining fairness in compressed models, addressing limitations of traditional pruning methods in opinion summarization tasks.

Abstract: Model compression through post-training pruning offers a way to reduce model size and computational requirements without significantly impacting model performance. However, the effect of pruning on the fairness of LLM-generated summaries remains unexplored, particularly for opinion summarisation where biased outputs could influence public views.In this paper, we present a comprehensive empirical analysis of opinion summarisation, examining three state-of-the-art pruning methods and various calibration sets across three open-source LLMs using four fairness metrics. Our systematic analysis reveals that pruning methods have a greater impact on fairness than calibration sets. Building on these insights, we propose High Gradient Low Activation (HGLA) pruning, which identifies and removes parameters that are redundant for input processing but influential in output generation. Our experiments demonstrate that HGLA can better maintain or even improve fairness compared to existing methods, showing promise across models and tasks where traditional methods have limitations. Our human evaluation shows HGLA-generated outputs are fairer than existing state-of-the-art pruning methods. Code is available at: https://github.com/amberhuang01/HGLA.

Method: Train neural network classifier on curated copyrighted dataset to examine LLMs’ internal states before text generation. Integrate with RAG system for early intervention through generation stopping or output alteration.

Result: Analysis of internal states effectively mitigates copyrighted data leakage risk. Provides scalable solution that integrates smoothly with AI workflows while maintaining high-quality text generation.

Conclusion: Proactive internal state monitoring offers effective copyright compliance and data privacy protection, ensuring ethical AI deployment without compromising generation quality.

Abstract: Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP) but pose risks of inadvertently exposing copyrighted or proprietary data, especially when such data is used for training but not intended for distribution. Traditional methods address these leaks only after content is generated, which can lead to the exposure of sensitive information. This study introduces a proactive approach: examining LLMs’ internal states before text generation to detect potential leaks. By using a curated dataset of copyrighted materials, we trained a neural network classifier to identify risks, allowing for early intervention by stopping the generation process or altering outputs to prevent disclosure. Integrated with a Retrieval-Augmented Generation (RAG) system, this framework ensures adherence to copyright and licensing requirements while enhancing data privacy and ethical standards. Our results show that analyzing internal states effectively mitigates the risk of copyrighted data leakage, offering a scalable solution that fits smoothly into AI workflows, ensuring compliance with copyright regulations while maintaining high-quality text generation. The implementation is available on GitHub.\footnote{https://github.com/changhu73/Internal_states_leakage}

Method: Developed MTalk-Bench with 9 realistic scenarios across 3 dimensions, using dual evaluation (Arena-style pairwise comparison and Rubrics-based absolute scoring) with both human and LLM evaluators.

Result: S2S models excel at semantic processing but underperform on paralinguistic information and ambient sounds; they sacrifice efficiency for coherence; modality-aware designs beat brute scaling. Evaluation methods are consistent but require large performance gaps for reliable distinctions.

Conclusion: Current S2S evaluation has limitations, highlighting the need for more robust, speech-aware assessment frameworks that address non-semantic capabilities and efficiency concerns.

Abstract: The rapid advancement of speech-to-speech (S2S) large language models (LLMs) has significantly improved real-time spoken interaction. However, current evaluation frameworks remain inadequate for assessing performance in complex, multi-turn dialogues. To address this, we introduce MTalk-Bench, a multi-turn S2S benchmark covering three core dimensions: Semantic Information, Paralinguistic Information, and Ambient Sound. Each dimension includes nine realistic scenarios, along with targeted tasks to assess specific capabilities such as reasoning. Our dual-method evaluation framework combines Arena-style evaluation (pairwise comparison) and Rubrics-based evaluation (absolute scoring) for relative and absolute assessment. The benchmark includes both model and human outputs, evaluated by human evaluators and LLMs. Experimental results reveal two sets of findings. Overall performance of S2S LLMs: (1) models excel at semantic information processing yet underperform on paralinguistic information and ambient sounds perception; (2) models typically regain coherence by increasing response length, sacrificing efficiency in multi-turn dialogues; (3) modality-aware, task-specific designs outperform brute scaling. Evaluation framework and reliability: (1) Arena and Rubrics yield consistent, complementary rankings, but reliable distinctions emerge only when performance gaps are large; (2) LLM-as-a-judge aligns with humans when gaps are clear or criteria explicit, but exhibits position and length biases and is reliable on nonverbal evaluation only with text annotations. These results highlight current limitations in S2S evaluation and the need for more robust, speech-aware assessment frameworks.

Method: Created two complementary tracks with multiple subtasks, evaluation datasets, and standardized metrics. Developed baseline systems and structured the task for fair benchmarking.

Result: The shared task facilitated participation and benchmarking, though specific performance results would be detailed in the full paper. Overall outcomes and participation statistics were documented.

Conclusion: The task successfully addressed the resource gap in Arabic medical QA and provided insights for future iterations, with reflections on performance trends and prospects for advancing Arabic health question answering.

Abstract: We introduce AraHealthQA 2025, the Comprehensive Arabic Health Question Answering Shared Task, held in conjunction with ArabicNLP 2025 (co-located with EMNLP 2025). This shared task addresses the paucity of high-quality Arabic medical QA resources by offering two complementary tracks: MentalQA, focusing on Arabic mental health Q&A (e.g., anxiety, depression, stigma reduction), and MedArabiQ, covering broader medical domains such as internal medicine, pediatrics, and clinical decision making. Each track comprises multiple subtasks, evaluation datasets, and standardized metrics, facilitating fair benchmarking. The task was structured to promote modeling under realistic, multilingual, and culturally nuanced healthcare contexts. We outline the dataset creation, task design and evaluation framework, participation statistics, baseline systems, and summarize the overall outcomes. We conclude with reflections on the performance trends observed and prospects for future iterations in Arabic health QA.

Method: Proposes TrinityX framework with separately trained experts for each HHH dimension, integrated through a calibrated task-adaptive routing mechanism that combines expert signals into unified alignment-aware representation within Transformer architecture.

Result: Outperforms baselines with 32.5% win rate improvement, 33.9% safety score improvement, and 28.4% truthfulness improvement. Reduces memory usage and inference latency by over 40% compared to prior MoE approaches.

Conclusion: TrinityX effectively addresses multi-dimensional alignment challenges through calibrated expert integration, demonstrating strong performance improvements and computational efficiency across diverse LLM backbones.

Abstract: Large Language Models (LLMs) exhibit strong performance across a wide range of NLP tasks, yet aligning their outputs with the principles of Helpfulness, Harmlessness, and Honesty (HHH) remains a persistent challenge. Existing methods often optimize for individual alignment dimensions in isolation, leading to trade-offs and inconsistent behavior. While Mixture-of-Experts (MoE) architectures offer modularity, they suffer from poorly calibrated routing, limiting their effectiveness in alignment tasks. We propose TrinityX, a modular alignment framework that incorporates a Mixture of Calibrated Experts (MoCaE) within the Transformer architecture. TrinityX leverages separately trained experts for each HHH dimension, integrating their outputs through a calibrated, task-adaptive routing mechanism that combines expert signals into a unified, alignment-aware representation. Extensive experiments on three standard alignment benchmarks-Alpaca (Helpfulness), BeaverTails (Harmlessness), and TruthfulQA (Honesty)-demonstrate that TrinityX outperforms strong baselines, achieving relative improvements of 32.5% in win rate, 33.9% in safety score, and 28.4% in truthfulness. In addition, TrinityX reduces memory usage and inference latency by over 40% compared to prior MoE-based approaches. Ablation studies highlight the importance of calibrated routing, and cross-model evaluations confirm TrinityX’s generalization across diverse LLM backbones.

Method: CM-Align uses consistency-based data selection with two components: 1) consistency-guided English reference selection to ensure high-quality English responses, and 2) cross-lingual consistency-based multilingual preference data construction to create better preference pairs for DPO training.

Result: Experimental results on three LLMs and three common tasks demonstrate the effectiveness and superiority of CM-Align, showing improved multilingual alignment performance compared to existing methods.

Conclusion: The method indicates the necessity of constructing high-quality preference data for multilingual alignment and successfully bridges the performance gap between English and other languages in LLMs.

Abstract: Current large language models (LLMs) generally show a significant performance gap in alignment between English and other languages. To bridge this gap, existing research typically leverages the model’s responses in English as a reference to select the best/worst responses in other languages, which are then used for Direct Preference Optimization (DPO) training. However, we argue that there are two limitations in the current methods that result in noisy multilingual preference data and further limited alignment performance: 1) Not all English responses are of high quality, and using a response with low quality may mislead the alignment for other languages. 2) Current methods usually use biased or heuristic approaches to construct multilingual preference pairs. To address these limitations, we design a consistency-based data selection method to construct high-quality multilingual preference data for improving multilingual alignment (CM-Align). Specifically, our method includes two parts: consistency-guided English reference selection and cross-lingual consistency-based multilingual preference data construction. Experimental results on three LLMs and three common tasks demonstrate the effectiveness and superiority of our method, which further indicates the necessity of constructing high-quality preference data.

Method: Developed Generative Marmoset Spoken Language Modeling (GmSLM) pipeline with novel zero-shot evaluation metrics using unsupervised in-the-wild data and weakly labeled conversational data, comparing against human-speech-based baselines.

Result: GmSLM generated vocalizations closely matched real resynthesized samples acoustically, performed well on downstream tasks, and effectively distinguished real from artificial conversations despite being fully unsupervised.

Conclusion: GmSLM provides a practical framework linking vocalization and brain activity, benefiting future neuroscience, bioacoustics, and evolutionary biology research by enabling further investigation of neural basis of vocal communication.

Abstract: Marmoset monkeys exhibit complex vocal communication, challenging the view that nonhuman primates vocal communication is entirely innate, and show similar features of human speech, such as vocal labeling of others and turn-taking. Studying their vocal communication offers a unique opportunity to link it with brain activity-especially given the difficulty of accessing the human brain in speech and language research. Since Marmosets communicate primarily through vocalizations, applying standard LLM approaches is not straightforward. We introduce Generative Marmoset Spoken Language Modeling (GmSLM), an optimized spoken language model pipeline for Marmoset vocal communication. We designed a novel zero-shot evaluation metrics using unsupervised in-the-wild data, alongside weakly labeled conversational data, to assess GmSLM and demonstrate its advantage over a basic human-speech-based baseline. GmSLM generated vocalizations closely matched real resynthesized samples acoustically and performed well on downstream tasks. Despite being fully unsupervised, GmSLM effectively distinguish real from artificial conversations and may support further investigations of the neural basis of vocal communication and provides a practical framework linking vocalization and brain activity. We believe GmSLM stands to benefit future work in neuroscience, bioacoustics, and evolutionary biology. Samples are provided under: pages.cs.huji.ac.il/adiyoss-lab/GmSLM.

Method: Introduces DocExplainerV0, a bounding-box prediction module that works with existing VLMs without requiring fine-tuning, enabling spatial localization alongside answer generation.

Result: Systematic evaluation reveals a significant gap between textual accuracy and spatial grounding - correct answers often lack reliable localization.

Conclusion: The framework establishes a benchmark for future research toward more interpretable and robust document information extraction VLMs.

Abstract: Vision-Language Models (VLMs) have shown strong capabilities in document understanding, particularly in identifying and extracting textual information from complex documents. Despite this, accurately localizing answers within documents remains a major challenge, limiting both interpretability and real-world applicability. To address this, we introduce DocExplainerV0, a plug-and-play bounding-box prediction module that decouples answer generation from spatial localization. This design makes it applicable to existing VLMs, including proprietary systems where fine-tuning is not feasible. Through systematic evaluation, we provide quantitative insights into the gap between textual accuracy and spatial grounding, showing that correct answers often lack reliable localization. Our standardized framework highlights these shortcomings and establishes a benchmark for future research toward more interpretable and robust document information extraction VLMs.

Method: Large-scale analysis of ICL by ablating memorization, pretraining, distributional shifts, and prompting styles, examining how models perform with varying numbers of exemplars and different prompt structures.

Result: ICL is effective but limited in learning and generalizing to unseen tasks. Accuracy becomes insensitive to exemplar distribution, model, prompt style, and linguistic features when exemplars are numerous, but shows distributional sensitivity in chain-of-thought prompting.

Conclusion: Autoregression’s ad-hoc encoding is not a robust learning mechanism, suggesting limited all-purpose generalizability despite ICL’s effectiveness in certain contexts.

Abstract: In-context learning (ICL) allows some autoregressive models to solve tasks via next-token prediction and without needing further training. This has led to claims about these model’s ability to solve (learn) unseen tasks with only a few shots (exemplars) in the prompt. However, deduction does not always imply learning, as ICL does not explicitly encode a given observation. Instead, the models rely on their prior knowledge and the exemplars given, if any. We argue that, mathematically, ICL does constitute learning, but its full characterisation requires empirical work. We then carry out a large-scale analysis of ICL ablating out or accounting for memorisation, pretraining, distributional shifts, and prompting style and phrasing. We find that ICL is an effective learning paradigm, but limited in its ability to learn and generalise to unseen tasks. We note that, in the limit where exemplars become more numerous, accuracy is insensitive to exemplar distribution, model, prompt style, and the input’s linguistic features. Instead, it deduces patterns from regularities in the prompt, which leads to distributional sensitivity, especially in prompting styles such as chain-of-thought. Given the varied accuracies on formally similar tasks, we conclude that autoregression’s ad-hoc encoding is not a robust mechanism, and suggests limited all-purpose generalisability.

Method: Uses semantic segmentation and precise object selection followed by real-time inpainting from secondary observer’s viewpoint. Implements YOLOv11 for object detection and modified Decoupled Spatial-Temporal Transformer (DSTT) model for video inpainting. Uses mobile ZED 2i depth camera without requiring fixed viewpoint or prior 3D scanning

Result: Achieves frame rates exceeding 20 fps at 720p resolution, demonstrating real-time performance for practical privacy-preserving MR applications

Conclusion: The system proves feasible for real-time diminished reality applications, offering portable and robust privacy control solution for mixed reality environments without requiring complex setup or pre-scanned environments

Abstract: Diminished reality (DR) refers to the digital removal of real-world objects by compositing background content in their place. This thesis presents a real-time, inpainting-based DR system designed to enable privacy control in shared-space mixed reality (MR) meetings. The system allows a primary headset user to selectively remove personal or sensitive items from their environment, ensuring that those objects are no longer visible to other participants. Removal is achieved through semantic segmentation and precise object selection, followed by real-time inpainting from the viewpoint of a secondary observer, implemented using a mobile ZED 2i depth camera. The solution is designed to be portable and robust, requiring neither a fixed secondary viewpoint nor prior 3D scanning of the environment. The system utilises YOLOv11 for object detection and a modified Decoupled Spatial-Temporal Transformer (DSTT) model for high-quality video inpainting. At 720p resolution, the pipeline sustains frame rates exceeding 20 fps, demonstrating the feasibility of real-time diminished reality for practical privacy-preserving MR applications.

Method: Created SurgLaVi dataset using automated pipeline that generates fine-grained transcriptions, segments videos into procedural units, and applies dual-modality filtering for quality. Introduced SurgCLIP, a CLIP-style video-text contrastive framework with dual encoders.

Result: SurgCLIP achieved consistent improvements across phase, step, action, and tool recognition tasks, surpassing prior state-of-the-art methods by large margins. The open-source SurgLaVi-beta (113k pairs) is 4x larger than existing datasets.

Conclusion: Large-scale, semantically rich, and hierarchically structured datasets directly translate into stronger surgical representations. SurgLaVi establishes a key resource for developing surgical foundation models with improved generalizability.

Abstract: Vision-language pre-training (VLP) offers unique advantages for surgery by aligning language with surgical videos, enabling workflow understanding and transfer across tasks without relying on expert-labeled datasets. However, progress in surgical VLP remains constrained by the limited scale, procedural diversity, semantic quality, and hierarchical structure of existing datasets. In this work, we present SurgLaVi, the largest and most diverse surgical vision-language dataset to date, comprising nearly 240k clip-caption pairs from more than 200 procedures, and comprising hierarchical levels at phase-, step-, and task-level. At the core of SurgLaVi lies a fully automated pipeline that systematically generates fine-grained transcriptions of surgical videos and segments them into coherent procedural units. To ensure high-quality annotations, it applies dual-modality filtering to remove irrelevant and noisy samples. Within this framework, the resulting captions are enriched with contextual detail, producing annotations that are both semantically rich and easy to interpret. To ensure accessibility, we release SurgLaVi-\b{eta}, an open-source derivative of 113k clip-caption pairs constructed entirely from public data, which is over four times larger than existing surgical VLP datasets. To demonstrate the value of SurgLaVi datasets, we introduce SurgCLIP, a CLIP-style video-text contrastive framework with dual encoders, as a representative base model. SurgCLIP achieves consistent improvements across phase, step, action, and tool recognition, surpassing prior state-of-the-art methods, often by large margins. These results validate that large-scale, semantically rich, and hierarchically structured datasets directly translate into stronger and more generalizable representations, establishing SurgLaVi as a key resource for developing surgical foundation models.

Method: Developed a general SimCLR-based SSL foundation model pre-trained on 18,759 patients (44,958 scans) from 11 public datasets spanning diverse neurological diseases. Compared against Masked Autoencoders and supervised baselines on four downstream tasks.

Result: The fine-tuned SimCLR model outperformed all other models across all tasks in both in-distribution and out-of-distribution settings. Achieved superior performance using only 20% of labeled training samples for Alzheimer’s disease prediction.

Conclusion: The model provides a broadly applicable and accessible foundation for clinical brain MRI analysis, demonstrating strong generalization capabilities across diverse neurological conditions with limited labeled data requirements.

Abstract: 3D structural Magnetic Resonance Imaging (MRI) brain scans are commonly acquired in clinical settings to monitor a wide range of neurological conditions, including neurodegenerative disorders and stroke. While deep learning models have shown promising results analyzing 3D MRI across a number of brain imaging tasks, most are highly tailored for specific tasks with limited labeled data, and are not able to generalize across tasks and/or populations. The development of self-supervised learning (SSL) has enabled the creation of large medical foundation models that leverage diverse, unlabeled datasets ranging from healthy to diseased data, showing significant success in 2D medical imaging applications. However, even the very few foundation models for 3D brain MRI that have been developed remain limited in resolution, scope, or accessibility. In this work, we present a general, high-resolution SimCLR-based SSL foundation model for 3D brain structural MRI, pre-trained on 18,759 patients (44,958 scans) from 11 publicly available datasets spanning diverse neurological diseases. We compare our model to Masked Autoencoders (MAE), as well as two supervised baselines, on four diverse downstream prediction tasks in both in-distribution and out-of-distribution settings. Our fine-tuned SimCLR model outperforms all other models across all tasks. Notably, our model still achieves superior performance when fine-tuned using only 20% of labeled training samples for predicting Alzheimer’s disease. We use publicly available code and data, and release our trained model at https://github.com/emilykaczmarek/3D-Neuro-SimCLR, contributing a broadly applicable and accessible foundation model for clinical brain MRI analysis.

Method: Formulates RGB-to-HSI as physics-grounded inverse problem with nuclear norm regularization in learnable transform domain. Introduces data-adaptive low-rank subspace SVT operator to avoid full SVD costs. Develops USCTNet deep unfolding solver that couples parameter estimation with learnable proximal updates.

Result: Extensive experiments on standard benchmarks show consistent improvements over state-of-the-art RGB-based methods in reconstruction accuracy.

Conclusion: The proposed USCTNet framework effectively addresses physical consistency in hyperspectral image reconstruction through explicit CSS/illumination estimation and efficient optimization techniques, achieving superior performance compared to existing methods.

Abstract: Reconstructing hyperspectral images (HSIs) from a single RGB image is ill-posed and can become physically inconsistent when the camera spectral sensitivity (CSS) and scene illumination are misspecified. We formulate RGB-to-HSI reconstruction as a physics-grounded inverse problem regularized by a nuclear norm in a learnable transform domain, and we explicitly estimate CSS and illumination to define the forward operator embedded in each iteration, ensuring colorimetric consistency. To avoid the cost and instability of full singular-value decompositions (SVDs) required by singular-value thresholding (SVT), we introduce a data-adaptive low-rank subspace SVT operator. Building on these components, we develop USCTNet, a deep unfolding solver tailored to HSI that couples a parameter estimation module with learnable proximal updates. Extensive experiments on standard benchmarks show consistent improvements over state-of-the-art RGB-based methods in reconstruction accuracy. Code: https://github.com/psykheXX/USCTNet-Code-Implementation.git

Method: Used BraTS 2020 dataset of multi-modal brain MRIs to evaluate LLaMA 3.2 Instruct LLM before and after fine-tuning, comparing against custom 3D CNNs for both classification (Low-Grade vs High-Grade glioma) and segmentation (using center point, bounding box, and polygon extraction methods).

Result: CNNs achieved 80% accuracy in classification with balanced precision/recall, while LLMs reached only 76% accuracy with poor specificity (18%). Fine-tuning improved LLM specificity to 55% but reduced overall accuracy to 72%. For segmentation, CNNs accurately localized tumors while LLMs consistently clustered predictions near image center without distinguishing tumor characteristics.

Conclusion: CNNs significantly outperform LLMs in medical imaging tasks. LLMs show limited spatial understanding and minimal improvement from fine-tuning, suggesting they are currently not well-suited for image-based medical applications without more rigorous training strategies.

Abstract: Large Language Models (LLMs) have shown strong performance in text-based healthcare tasks. However, their utility in image-based applications remains unexplored. We investigate the effectiveness of LLMs for medical imaging tasks, specifically glioma classification and segmentation, and compare their performance to that of traditional convolutional neural networks (CNNs). Using the BraTS 2020 dataset of multi-modal brain MRIs, we evaluated a general-purpose vision-language LLM (LLaMA 3.2 Instruct) both before and after fine-tuning, and benchmarked its performance against custom 3D CNNs. For glioma classification (Low-Grade vs. High-Grade), the CNN achieved 80% accuracy and balanced precision and recall. The general LLM reached 76% accuracy but suffered from a specificity of only 18%, often misclassifying Low-Grade tumors. Fine-tuning improved specificity to 55%, but overall performance declined (e.g., accuracy dropped to 72%). For segmentation, three methods - center point, bounding box, and polygon extraction, were implemented. CNNs accurately localized gliomas, though small tumors were sometimes missed. In contrast, LLMs consistently clustered predictions near the image center, with no distinction of glioma size, location, or placement. Fine-tuning improved output formatting but failed to meaningfully enhance spatial accuracy. The bounding polygon method yielded random, unstructured outputs. Overall, CNNs outperformed LLMs in both tasks. LLMs showed limited spatial understanding and minimal improvement from fine-tuning, indicating that, in their current form, they are not well-suited for image-based tasks. More rigorous fine-tuning or alternative training strategies may be needed for LLMs to achieve better performance, robustness, and utility in the medical space.

Method: Uses a dual-branch diffusion model with spatial color encoding to map part masks to RGB-like images, sharing VAE latents between branches. Includes BiDiFuse module for cross-branch consistency and contrastive part consistency loss for spatial-temporal alignment.

Result: SP4D generalizes to diverse scenarios including real-world videos, novel objects, and rare poses. Generated 2D part maps can be lifted to 3D for skeletal structures and skinning weights with few manual adjustments.

Conclusion: The framework produces kinematic-aware outputs suitable for downstream animation tasks, demonstrating strong generalization across various scenarios and enabling efficient 3D structure extraction from 2D part predictions.

Abstract: We present Stable Part Diffusion 4D (SP4D), a framework for generating paired RGB and kinematic part videos from monocular inputs. Unlike conventional part segmentation methods that rely on appearance-based semantic cues, SP4D learns to produce kinematic parts - structural components aligned with object articulation and consistent across views and time. SP4D adopts a dual-branch diffusion model that jointly synthesizes RGB frames and corresponding part segmentation maps. To simplify the architecture and flexibly enable different part counts, we introduce a spatial color encoding scheme that maps part masks to continuous RGB-like images. This encoding allows the segmentation branch to share the latent VAE from the RGB branch, while enabling part segmentation to be recovered via straightforward post-processing. A Bidirectional Diffusion Fusion (BiDiFuse) module enhances cross-branch consistency, supported by a contrastive part consistency loss to promote spatial and temporal alignment of part predictions. We demonstrate that the generated 2D part maps can be lifted to 3D to derive skeletal structures and harmonic skinning weights with few manual adjustments. To train and evaluate SP4D, we construct KinematicParts20K, a curated dataset of over 20K rigged objects selected and processed from Objaverse XL (Deitke et al., 2023), each paired with multi-view RGB and part video sequences. Experiments show that SP4D generalizes strongly to diverse scenarios, including real-world videos, novel generated objects, and rare articulated poses, producing kinematic-aware outputs suitable for downstream animation and motion-related tasks.

Method: The authors propose Sphere-GAN, which leverages a Generative Adversarial Network architecture with spherical convolutions specifically designed for 360° video saliency estimation.

Result: Extensive experiments on a public 360° video saliency dataset show that Sphere-GAN outperforms state-of-the-art models in accurately predicting saliency maps.

Conclusion: Sphere-GAN provides an effective solution for 360° video saliency detection, addressing the specific challenges of immersive content processing and offering improved performance over existing methods.

Abstract: The recent success of immersive applications is pushing the research community to define new approaches to process 360{\deg} images and videos and optimize their transmission. Among these, saliency estimation provides a powerful tool that can be used to identify visually relevant areas and, consequently, adapt processing algorithms. Although saliency estimation has been widely investigated for 2D content, very few algorithms have been proposed for 360{\deg} saliency estimation. Towards this goal, we introduce Sphere-GAN, a saliency detection model for 360{\deg} videos that leverages a Generative Adversarial Network with spherical convolutions. Extensive experiments were conducted using a public 360{\deg} video saliency dataset, and the results demonstrate that Sphere-GAN outperforms state-of-the-art models in accurately predicting saliency maps.

Method: Uses pose information for rough hand/body localization, then applies promptable zero-shot video segmentation to maintain shape details, focusing RGB processing on relevant body parts.

Result: Outperforms state-of-the-art methods on ChaLearn249 IsoGD dataset, with ablation studies confirming benefits of focusing on signer’s body and hands.

Conclusion: SegSLR effectively combines RGB and pose information through segmentation, preserving crucial details and achieving superior ISLR performance.

Abstract: Isolated Sign Language Recognition (ISLR) approaches primarily rely on RGB data or signer pose information. However, combining these modalities often results in the loss of crucial details, such as hand shape and orientation, due to imprecise representations like bounding boxes. Therefore, we propose the ISLR system SegSLR, which combines RGB and pose information through promptable zero-shot video segmentation. Given the rough localization of the hands and the signer’s body from pose information, we segment the respective parts through the video to maintain all relevant shape information. Subsequently, the segmentations focus the processing of the RGB data on the most relevant body parts for ISLR. This effectively combines RGB and pose information. Our evaluation on the complex ChaLearn249 IsoGD dataset shows that SegSLR outperforms state-of-the-art methods. Furthermore, ablation studies indicate that SegSLR strongly benefits from focusing on the signer’s body and hands, justifying our design choices.

Method: Combines LLM reasoning with open-set VFMs, using speech feedback to guide segmentation and instrument tracking in surgical videos.

Result: Demonstrated effective on-the-fly segmentation and tracking on Cataract1k and skull-base datasets, including live mock experiments.

Conclusion: Shows potential for adaptable, hands-free surgical tools through human-AI collaboration in dynamic operating rooms.

Abstract: Accurate segmentation and tracking of relevant elements of the surgical scene is crucial to enable context-aware intraoperative assistance and decision making. Current solutions remain tethered to domain-specific, supervised models that rely on labeled data and required domain-specific data to adapt to new surgical scenarios and beyond predefined label categories. Recent advances in prompt-driven vision foundation models (VFM) have enabled open-set, zero-shot segmentation across heterogeneous medical images. However, dependence of these models on manual visual or textual cues restricts their deployment in introperative surgical settings. We introduce a speech-guided collaborative perception (SCOPE) framework that integrates reasoning capabilities of large language model (LLM) with perception capabilities of open-set VFMs to support on-the-fly segmentation, labeling and tracking of surgical instruments and anatomy in intraoperative video streams. A key component of this framework is a collaborative perception agent, which generates top candidates of VFM-generated segmentation and incorporates intuitive speech feedback from clinicians to guide the segmentation of surgical instruments in a natural human-machine collaboration paradigm. Afterwards, instruments themselves serve as interactive pointers to label additional elements of the surgical scene. We evaluated our proposed framework on a subset of publicly available Cataract1k dataset and an in-house ex-vivo skull-base dataset to demonstrate its potential to generate on-the-fly segmentation and tracking of surgical scene. Furthermore, we demonstrate its dynamic capabilities through a live mock ex-vivo experiment. This human-AI collaboration paradigm showcase the potential of developing adaptable, hands-free, surgeon-centric tools for dynamic operating-room environments.

Method: A two-stage pipeline: 1) Reconstructs dynamic scenes from multi-view videos using 4D Gaussian Splatting, 2) Applies physically-based ray tracing to generate videos with controllable camera effects.

Result: Achieves fastest rendering speed while performing better or comparable rendering quality compared to existing baselines. Also creates a benchmark with eight synthetic dynamic scenes across four camera effects.

Conclusion: 4D-GRT effectively addresses the bottleneck of generating training data with realistic camera effects, providing physically accurate simulations for improving computer vision systems.

Abstract: Common computer vision systems typically assume ideal pinhole cameras but fail when facing real-world camera effects such as fisheye distortion and rolling shutter, mainly due to the lack of learning from training data with camera effects. Existing data generation approaches suffer from either high costs, sim-to-real gaps or fail to accurately model camera effects. To address this bottleneck, we propose 4D Gaussian Ray Tracing (4D-GRT), a novel two-stage pipeline that combines 4D Gaussian Splatting with physically-based ray tracing for camera effect simulation. Given multi-view videos, 4D-GRT first reconstructs dynamic scenes, then applies ray tracing to generate videos with controllable, physically accurate camera effects. 4D-GRT achieves the fastest rendering speed while performing better or comparable rendering quality compared to existing baselines. Additionally, we construct eight synthetic dynamic scenes in indoor environments across four camera effects as a benchmark to evaluate generated videos with camera effects.

Method: Proposes SDS-E (SDS for Editing) that selectively incorporates SDS subterms across diffusion timesteps, enhanced with spatial smoothness regularization and gradient-based viewpoint sampling for sharp detailing.

Result: Outperforms existing 3D editing methods, with avatars faithfully reflecting textual edits while remaining consistent with the original avatars.

Conclusion: The proposed SDS-E approach successfully addresses the consistency problem in 3D human texture editing, enabling high-quality instruction-driven edits that preserve source avatar fidelity.

Abstract: We present InstructHumans, a novel framework for instruction-driven {animatable} 3D human texture editing. Existing text-based 3D editing methods often directly apply Score Distillation Sampling (SDS). SDS, designed for generation tasks, cannot account for the defining requirement of editing – maintaining consistency with the source avatar. This work shows that naively using SDS harms editing, as it may destroy consistency. We propose a modified SDS for Editing (SDS-E) that selectively incorporates subterms of SDS across diffusion timesteps. We further enhance SDS-E with spatial smoothness regularization and gradient-based viewpoint sampling for edits with sharp and high-fidelity detailing. Incorporating SDS-E into a 3D human texture editing framework allows us to outperform existing 3D editing methods. Our avatars faithfully reflect the textual edits while remaining consistent with the original avatars. Project page: https://jyzhu.top/instruct-humans/.

Method: Two-agent approach: Editor agent uses video editing tools with natural language instructions, Critic agent provides feedback or renders final output. Learning-based approach for inter-agent communication.

Result: System vastly outperforms existing approaches in coverage, time constraint satisfaction, and human preference based on user study evaluation.

Conclusion: Multi-agent framework with specialized Editor and Critic agents effectively automates language-driven video editing, demonstrating superior performance over traditional methods.

Abstract: Automated tools for video editing and assembly have applications ranging from filmmaking and advertisement to content creation for social media. Previous video editing work has mainly focused on either retrieval or user interfaces, leaving actual editing to the user. In contrast, we propose to automate the core task of video editing, formulating it as sequential decision making process. Ours is a multi-agent approach. We design an Editor agent and a Critic agent. The Editor takes as input a collection of video clips together with natural language instructions and uses tools commonly found in video editing software to produce an edited sequence. On the other hand, the Critic gives natural language feedback to the editor based on the produced sequence or renders it if it is satisfactory. We introduce a learning-based approach for enabling effective communication across specialized agents to address the language-driven video editing task. Finally, we explore an LLM-as-a-judge metric for evaluating the quality of video editing system and compare it with general human preference. We evaluate our system’s output video sequences qualitatively and quantitatively through a user study and find that our system vastly outperforms existing approaches in terms of coverage, time constraint satisfaction, and human preference.

Method: Analyzed 1,446 glioma cases from 4 TCIA datasets. Extracted 108 radiomics features from non-contrast T1WI, combined with 48 dimensionality reduction methods and 25 classifiers (1,200 total pipelines). Used rotational validation trained on 3 datasets, tested on the fourth.

Result: Cross-validation accuracy: 0.91-0.96; External testing: 0.87-0.98 (avg 0.93). F1, precision, recall stable (0.87-0.96), ROC-AUC varied (0.50-0.82). MI with ETr pipeline consistently performed best, balancing accuracy and stability.

Conclusion: Stability-aware model selection enables reliable prediction of contrast enhancement from non-contrast MRI, reducing GBCA reliance and improving multicenter generalizability. Provides scalable template for reproducible ML in neuro-oncology.

Abstract: Gadolinium-based contrast agents (GBCAs) are central to glioma imaging but raise safety, cost, and accessibility concerns. Predicting contrast enhancement from non-contrast MRI using machine learning (ML) offers a safer alternative, as enhancement reflects tumor aggressiveness and informs treatment planning. Yet scanner and cohort variability hinder robust model selection. We propose a stability-aware framework to identify reproducible ML pipelines for multicenter prediction of glioma MRI contrast enhancement. We analyzed 1,446 glioma cases from four TCIA datasets (UCSF-PDGM, UPENN-GB, BRATS-Africa, BRATS-TCGA-LGG). Non-contrast T1WI served as input, with enhancement derived from paired post-contrast T1WI. Using PyRadiomics under IBSI standards, 108 features were extracted and combined with 48 dimensionality reduction methods and 25 classifiers, yielding 1,200 pipelines. Rotational validation was trained on three datasets and tested on the fourth. Cross-validation prediction accuracies ranged from 0.91 to 0.96, with external testing achieving 0.87 (UCSF-PDGM), 0.98 (UPENN-GB), and 0.95 (BRATS-Africa), with an average of 0.93. F1, precision, and recall were stable (0.87 to 0.96), while ROC-AUC varied more widely (0.50 to 0.82), reflecting cohort heterogeneity. The MI linked with ETr pipeline consistently ranked highest, balancing accuracy and stability. This framework demonstrates that stability-aware model selection enables reliable prediction of contrast enhancement from non-contrast glioma MRI, reducing reliance on GBCAs and improving generalizability across centers. It provides a scalable template for reproducible ML in neuro-oncology and beyond.

Method: Overlapping tiling recovers low-confidence candidates, followed by Spatial Gate (DBSCAN on box centroids) and Semantic Gate (DBSCAN on ResNet-18 embeddings) to validate group evidence. Validated groups receive confidence reweighting before class-aware NMS fusion.

Result: On VisDrone dataset: recall increased from 0.685 to 0.778 (+0.093), precision adjusted from 0.801 to 0.595, yielding F1=0.669. Post-processing latency averages 0.095s per image.

Conclusion: The framework provides recall-first, precision-trade-off behavior beneficial for recall-sensitive applications like far-field counting and monitoring, requires no retraining, and integrates with modern detectors.

Abstract: Dense small objects in UAV imagery are often missed due to long-range viewpoints, occlusion, and clutter[cite: 5]. This paper presents a detector-agnostic post-processing framework that converts overlap-induced redundancy into group evidence[cite: 6]. Overlapping tiling first recovers low-confidence candidates[cite: 7]. A Spatial Gate (DBSCAN on box centroids) and a Semantic Gate (DBSCAN on ResNet-18 embeddings) then validates group evidence[cite: 7]. Validated groups receive controlled confidence reweighting before class-aware NMS fusion[cite: 8]. Experiments on VisDrone show a recall increase from 0.685 to 0.778 (+0.093) and a precision adjustment from 0.801 to 0.595, yielding F1=0.669[cite: 9]. Post-processing latency averages 0.095 s per image[cite: 10]. These results indicate recall-first, precision-trade-off behavior that benefits recall-sensitive applications such as far-field counting and monitoring[cite: 10]. Ablation confirms that tiling exposes missed objects, spatial clustering stabilizes geometry, semantic clustering enforces appearance coherence, and reweighting provides calibrated integration with the baseline[cite: 11]. The framework requires no retraining and integrates with modern detectors[cite: 12]. Future work will reduce semantic gating cost and extend the approach with temporal cues[cite: 13].

Method: Created by integrating three scene sources (real-world scans, procedurally generated scenes, designer-created scenes) with comprehensive processing including real-to-sim replica creation, interactive object incorporation, and physical simulation for collision resolution.

Result: Dataset comprises ~40,000 scenes, 1.96M 3D objects, 15 scene types, 288 object classes, with realistic layouts averaging 41.5 objects per region. Demonstrated value through scene layout generation and point-goal navigation benchmarks.

Conclusion: InternScenes enables scaling up model training for complex scene tasks and paves the way for generation and navigation in realistic complex environments, with commitment to open-source release for community benefit.

Abstract: The advancement of Embodied AI heavily relies on large-scale, simulatable 3D scene datasets characterized by scene diversity and realistic layouts. However, existing datasets typically suffer from limitations in data scale or diversity, sanitized layouts lacking small items, and severe object collisions. To address these shortcomings, we introduce \textbf{InternScenes}, a novel large-scale simulatable indoor scene dataset comprising approximately 40,000 diverse scenes by integrating three disparate scene sources, real-world scans, procedurally generated scenes, and designer-created scenes, including 1.96M 3D objects and covering 15 common scene types and 288 object classes. We particularly preserve massive small items in the scenes, resulting in realistic and complex layouts with an average of 41.5 objects per region. Our comprehensive data processing pipeline ensures simulatability by creating real-to-sim replicas for real-world scans, enhances interactivity by incorporating interactive objects into these scenes, and resolves object collisions by physical simulations. We demonstrate the value of InternScenes with two benchmark applications: scene layout generation and point-goal navigation. Both show the new challenges posed by the complex and realistic layouts. More importantly, InternScenes paves the way for scaling up the model training for both tasks, making the generation and navigation in such complex scenes possible. We commit to open-sourcing the data, models, and benchmarks to benefit the whole community.

Method: Examined condition numbers for polynomial evaluation in various bases (Legendre, Chebyshev, and their Sobolev variants) and bounded how different inner products provide norms for symbol variations.

Result: The study provides insights into the trade-offs between basis selection and polynomial degree, showing how condition numbers and norm bounds affect computational efficiency and modeling accuracy.

Conclusion: Different polynomial bases offer varying computational and accuracy trade-offs for mathematical handwriting representation, with condition numbers and norm bounds serving as key metrics for optimization.

Abstract: Previous work has made use of a parameterized plane curve polynomial representation for mathematical handwriting, with the polynomials represented in a Legendre or Legendre-Sobolev graded basis. This provides a compact geometric representation for the digital ink. Preliminary results have also been shown for Chebyshev and Chebyshev-Sobolev bases. This article explores the trade-offs between basis choice and polynomial degree to achieve accurate modeling with a low computational cost. To do this, we consider the condition number for polynomial evaluation in these bases and bound how the various inner products give norms for the variations between symbols.

Method: Multitask diffusion framework that simultaneously generates future FLAIR sequences at any time point and estimates spatial probabilistic tumor evolution maps using signed distance fields (SDFs). Integrates pretrained deformation module for temporal dynamics, implements targeted augmentation pipeline for data scarcity, and uses radiotherapy-weighted focal loss with radiation dose maps.

Result: The framework produces flexible time-dependent probability maps from just two early follow-up scans, enabling interrogation of tumor progression risks at any future milestone. Achieved promising results when trained on public dataset and evaluated on private internal dataset.

Conclusion: The proposed method effectively addresses clinical limitations of sparse longitudinal MRI data, providing clinicians with flexible tools for predicting glioma progression with uncertainty quantification and clinical relevance through radiotherapy-weighted training.

Abstract: Glioma, an aggressive brain malignancy characterized by rapid progression and its poor prognosis, poses significant challenges for accurate evolution prediction. These challenges are exacerbated by sparse, irregularly acquired longitudinal MRI data in clinical practice, where incomplete follow-up sequences create data imbalances and make reliable modeling difficult. In this paper, we present a multitask diffusion framework for time-agnostic, pixel-wise prediction of glioma progression. The model simultaneously generates future FLAIR sequences at any chosen time point and estimates spatial probabilistic tumor evolution maps derived using signed distance fields (SDFs), allowing uncertainty quantification. To capture temporal dynamics of tumor evolution across arbitrary intervals, we integrate a pretrained deformation module that models inter-scan changes using deformation fields. Regarding the common clinical limitation of data scarcity, we implement a targeted augmentation pipeline that synthesizes complete sequences of three follow-up scans and imputes missing MRI modalities from available patient studies, improving the stability and accuracy of predictive models. Based on merely two follow-up scans at earlier timepoints, our framework produces flexible time-depending probability maps, enabling clinicians to interrogate tumor progression risks at any future temporal milestone. We further introduce a radiotherapy-weighted focal loss term that leverages radiation dose maps, as these highlight regions of greater clinical importance during model training. The proposed method was trained on a public dataset and evaluated on an internal private dataset, achieving promising results in both cases

Method: Uses point-plane projections to learn features from 2D representations of point clouds, combined with geometry-aware data augmentation that aligns with LiDAR sensor properties to address class imbalance.

Result: Achieves significant improvements in limited-data scenarios and competitive results on standard datasets (SemanticKITTI and PandaSet).

Conclusion: The approach effectively extracts complementary information from LiDAR data alone, demonstrating strong performance without requiring additional sensors or external datasets.

Abstract: LiDAR point cloud semantic segmentation is essential for interpreting 3D environments in applications such as autonomous driving and robotics. Recent methods achieve strong performance by exploiting different point cloud representations or incorporating data from other sensors, such as cameras or external datasets. However, these approaches often suffer from high computational complexity and require large amounts of training data, limiting their generalization in data-scarce scenarios. In this paper, we improve the performance of point-based methods by effectively learning features from 2D representations through point-plane projections, enabling the extraction of complementary information while relying solely on LiDAR data. Additionally, we introduce a geometry-aware technique for data augmentation that aligns with LiDAR sensor properties and mitigates class imbalance. We implemented and evaluated our method that applies point-plane projections onto multiple informative 2D representations of the point cloud. Experiments demonstrate that this approach leads to significant improvements in limited-data scenarios, while also achieving competitive results on two publicly available standard datasets, as SemanticKITTI and PandaSet. The code of our method is available at https://github.com/SiMoM0/3PNet

Method: Uses native-resolution Vision Transformers, three-phase training (warm-up, multitask training, supervised instruction tuning), Mindspeed-MLLM distributed framework, equivalent operator replacements, multimodal data packaging, hybrid parallelism, test-time resolution search, and model weight averaging.

Result: Achieves performance on par with Qwen2.5-VL in general multimodal understanding and document/table comprehension despite using only 1/10 of the training data, with leading performance in OCR assessments.

Conclusion: MindVL demonstrates that efficient training on specialized hardware with optimized techniques can achieve state-of-the-art multimodal performance with significantly reduced training data requirements.

Abstract: We propose MindVL, a multimodal large langauge model trained on Ascend NPUs. Similar to Qwen2.5-VL, MindVL adopts native-resolution Vision Transformers, which enables it to process images at their original variable resolutions. This design avoids the degradation caused by fixed-resolution tiling while preserving fine-grained details and global layouts, which is crucial for visually dense content such as complex charts and diagrams. To ensure the smooth training of MindVL on Ascend NPUs, we develop Mindspeed-MLLM, a distributed multimodal training framework tailored for Ascend NPUs. To maintain training accuracy, we implement equivalent replacements for certain operators. MindVL undergoes a three-phase training process, namely the warm-up phase, multitask training phase, and supervised instruction tuning phase, to gradually enhance its capabilities. This process starts with basic visual and multimodal pre-training, followed by large-scale multiask trainging and instruction tuning. We also adopt multimodal data packaging and hybrid parallelism techniques, which significantly improve end-to-end training speed. To further boost model performance, we specifically introduce test-time resolution search and model weight averaging. Notably, despite using about 1/10 of the training data required by Qwen2.5-VL, MindVL achieves performance on par with Qwen2.5-VL in evaluations of general multimodal understanding and document/table comprehension. Beyond overall scores, MindVL also delivers leading performance in OCR assessments.

Method: Generates semantic features directly from raw point clouds through multi-view, multi-granularity rendering, mask-level vision-language feature extraction, sample-balanced fusion, and distillation into a 3D backbone model.

Result: Achieves significant improvements in segmentation accuracy and cross-scene generalization on large-scale urban benchmarks (SensatUrban and SUM) compared to existing methods.

Conclusion: OpenUrban3D demonstrates potential as a flexible and scalable solution for 3D urban scene understanding, enabling zero-shot segmentation for arbitrary text queries while capturing both semantic richness and geometric priors.

Abstract: Open-vocabulary semantic segmentation enables models to recognize and segment objects from arbitrary natural language descriptions, offering the flexibility to handle novel, fine-grained, or functionally defined categories beyond fixed label sets. While this capability is crucial for large-scale urban point clouds that support applications such as digital twins, smart city management, and urban analytics, it remains largely unexplored in this domain. The main obstacles are the frequent absence of high-quality, well-aligned multi-view imagery in large-scale urban point cloud datasets and the poor generalization of existing three-dimensional (3D) segmentation pipelines across diverse urban environments with substantial variation in geometry, scale, and appearance. To address these challenges, we present OpenUrban3D, the first 3D open-vocabulary semantic segmentation framework for large-scale urban scenes that operates without aligned multi-view images, pre-trained point cloud segmentation networks, or manual annotations. Our approach generates robust semantic features directly from raw point clouds through multi-view, multi-granularity rendering, mask-level vision-language feature extraction, and sample-balanced fusion, followed by distillation into a 3D backbone model. This design enables zero-shot segmentation for arbitrary text queries while capturing both semantic richness and geometric priors. Extensive experiments on large-scale urban benchmarks, including SensatUrban and SUM, show that OpenUrban3D achieves significant improvements in both segmentation accuracy and cross-scene generalization over existing methods, demonstrating its potential as a flexible and scalable solution for 3D urban scene understanding.

Method: Uses dual-camera setup for frontal and workspace views, integrates Face Module (ArcFace for identity verification, head pose, gaze tracking, mouth movement) and Hand Module (YOLOv11 for prohibited item detection), with LSTM network for temporal pattern analysis of cheating probability.

Result: Achieves 90.7% accuracy in suspicious activity classification, mAP@.5 of 0.57 for prohibited item detection, and processes video at 2.4 FPS without GPU.

Conclusion: AutoOEP is an effective, resource-efficient automated proctoring solution that reduces human intervention and enhances online assessment integrity.

Abstract: The burgeoning of online education has created an urgent need for robust and scalable systems to ensure academic integrity during remote examinations. Traditional human proctoring is often not feasible at scale, while existing automated solutions can be intrusive or fail to detect a wide range of cheating behaviors. This paper introduces AutoOEP (Automated Online Exam Proctoring), a comprehensive, multi-modal framework that leverages computer vision and machine learning to provide effective, automated proctoring. The system utilizes a dual-camera setup to capture both a frontal view of the examinee and a side view of the workspace, minimizing blind spots. Our approach integrates several parallel analyses: the Face Module performs continuous identity verification using ArcFace, along with head pose estimation, gaze tracking, and mouth movement analysis to detect suspicious cues. Concurrently, the Hand Module employs a fine-tuned YOLOv11 model for detecting prohibited items (e.g., mobile phones, notes) and tracks hand proximity to these objects. Features from these modules are aggregated and fed into a Long Short-Term Memory (LSTM) network that analyzes temporal patterns to calculate a real-time cheating probability score. We evaluate AutoOEP on a custom-collected dataset simulating diverse exam conditions. Our system achieves an accuracy of 90.7% in classifying suspicious activities. The object detection component obtains a mean Average Precision (mAP@.5) of 0.57 for prohibited items, and the entire framework processes video streams at approximately 2.4 frames per second without a GPU. The results demonstrate that AutoOEP is an effective and resource-efficient solution for automated proctoring, significantly reducing the need for human intervention and enhancing the integrity of online assessments.

Method: Proposes dual-camera CASSI with total variation subgradient theory, establishing end-to-end mathematical model. Uses dynamic regularization with normalized gradient constraints from RGB/panchromatic reference images, and adaptive reference generation mechanism from auxiliary cameras.

Result: Method effectively preserves spatial-spectral structural consistency, demonstrates robust performance across diverse reconstruction scenarios, and reduces computational complexity of inverse problem solving.

Conclusion: The framework provides an interpretable mathematical foundation for computational spectral imaging with strict convex optimization guarantees, bridging the gap between model-based and learning-based approaches.

Abstract: Spectral imaging technology has long-faced fundamental challenges in balancing spectral, spatial, and temporal resolutions. While compressive sensing-based Coded Aperture Snapshot Spectral Imaging (CASSI) mitigates this trade-off through optical encoding, high compression ratios result in ill-posed reconstruction problems. Traditional model-based methods exhibit limited performance due to reliance on handcrafted inherent image priors, while deep learning approaches are constrained by their black-box nature, which compromises physical interpretability. To address these limitations, we propose a dual-camera CASSI reconstruction framework that integrates total variation (TV) subgradient theory. By establishing an end-to-end SD-CASSI mathematical model, we reduce the computational complexity of solving the inverse problem and provide a mathematically well-founded framework for analyzing multi-camera systems. A dynamic regularization strategy is introduced, incorporating normalized gradient constraints from RGB/panchromatic-derived reference images, which constructs a TV subgradient similarity function with strict convex optimization guarantees. Leveraging spatial priors from auxiliary cameras, an adaptive reference generation and updating mechanism is designed to provide subgradient guidance. Experimental results demonstrate that the proposed method effectively preserves spatial-spectral structural consistency. The theoretical framework establishes an interpretable mathematical foundation for computational spectral imaging, demonstrating robust performance across diverse reconstruction scenarios. The source code is available at https://github.com/bestwishes43/ADMM-TVDS.

Method: Proposed Metadata-aware Mixture-of-Experts Masked Autoencoder (MoE-MAE) with sparse expert routing and geo-temporal conditioning (latitude/longitude, seasonal/daily cyclic encodings), pretrained on BigEarthNet-Landsat and evaluated with linear probes.

Result: Despite only 2.5M parameters, the model competes with much larger architectures, showing improved transfer and label efficiency. Even on EuroSAT-Landsat without explicit metadata, it maintains competitive performance against models with hundreds of millions of parameters.

Conclusion: Compact, metadata-aware MoE-MAEs provide an efficient and scalable pathway toward future Earth Observation foundation models, demonstrating that small models can achieve strong performance through smart architectural design and metadata integration.

Abstract: Recent advances in Earth Observation have focused on large-scale foundation models. However, these models are computationally expensive, limiting their accessibility and reuse for downstream tasks. In this work, we investigate compact architectures as a practical pathway toward smaller general-purpose EO models. We propose a Metadata-aware Mixture-of-Experts Masked Autoencoder (MoE-MAE) with only 2.5M parameters. The model combines sparse expert routing with geo-temporal conditioning, incorporating imagery alongside latitude/longitude and seasonal/daily cyclic encodings. We pretrain the MoE-MAE on the BigEarthNet-Landsat dataset and evaluate embeddings from its frozen encoder using linear probes. Despite its small size, the model competes with much larger architectures, demonstrating that metadata-aware pretraining improves transfer and label efficiency. To further assess generalization, we evaluate on the EuroSAT-Landsat dataset, which lacks explicit metadata, and still observe competitive performance compared to models with hundreds of millions of parameters. These results suggest that compact, metadata-aware MoE-MAEs are an efficient and scalable step toward future EO foundation models.

Method: Optimized sinogram-based method to simulate motion artifacts, creating paired datasets. Proposed Edge-enhanced Self-attention WGAN-GP with VGG-based perceptual loss to preserve trabecular edges and capture long-range dependencies.

Result: Achieved SNR 26.78, SSIM 0.81, VIF 0.76 for source dataset; SNR 29.31, SSIM 0.87, VIF 0.81 for target dataset, showing improved performance on real-world data.

Conclusion: Methods represent an important initial step toward implementing deep learning-based motion correction in HR-pQCT, addressing a key challenge for wider adoption of this modality.

Abstract: Rigid-motion artifacts, such as cortical bone streaking and trabecular smearing, hinder in vivo assessment of bone microstructures in high-resolution peripheral quantitative computed tomography (HR-pQCT). Despite various motion grading techniques, no motion correction methods exist due to the lack of standardized degradation models. We optimize a conventional sinogram-based method to simulate motion artifacts in HR-pQCT images, creating paired datasets of motion-corrupted images and their corresponding ground truth, which enables seamless integration into supervised learning frameworks for motion correction. As such, we propose an Edge-enhanced Self-attention Wasserstein Generative Adversarial Network with Gradient Penalty (ESWGAN-GP) to address motion artifacts in both simulated (source) and real-world (target) datasets. The model incorporates edge-enhancing skip connections to preserve trabecular edges and self-attention mechanisms to capture long-range dependencies, facilitating motion correction. A visual geometry group (VGG)-based perceptual loss is used to reconstruct fine micro-structural features. The ESWGAN-GP achieves a mean signal-to-noise ratio (SNR) of 26.78, structural similarity index measure (SSIM) of 0.81, and visual information fidelity (VIF) of 0.76 for the source dataset, while showing improved performance on the target dataset with an SNR of 29.31, SSIM of 0.87, and VIF of 0.81. The proposed methods address a simplified representation of real-world motion that may not fully capture the complexity of in vivo motion artifacts. Nevertheless, because motion artifacts present one of the foremost challenges to more widespread adoption of this modality, these methods represent an important initial step toward implementing deep learning-based motion correction in HR-pQCT.

Method: Experiments conducted on large-scale dataset of 8,849 subjects using Meta Quest Pro equivalent hardware with 72Hz video oculography. Employed state-of-the-art neural network architecture to study calibration target depth, calibrated/non-calibrated gaze fusion, signal quality, and moving average filtering.

Result: Using same calibration target depth, fusing calibrated and non-calibrated gaze, and improving eye tracking signal quality all enhance authentication performance. Three-sample moving average filter slightly reduces performance in general, with some exceptions noted.

Conclusion: Careful consideration of calibration parameters and signal quality is crucial for optimal gaze-based authentication performance, while simple filtering approaches may be counterproductive.

Abstract: This paper examines the key factors that influence the performance of state-of-the-art gaze-based authentication. Experiments were conducted on a large-scale, in-house dataset comprising 8,849 subjects collected with Meta Quest Pro equivalent hardware running a video oculography-driven gaze estimation pipeline at 72Hz. The state-of-the-art neural network architecture was employed to study the influence of the following factors on authentication performance: eye tracking signal quality, various aspects of eye tracking calibration, and simple filtering on estimated raw gaze. We found that using the same calibration target depth for eye tracking calibration, fusing calibrated and non-calibrated gaze, and improving eye tracking signal quality all enhance authentication performance. We also found that a simple three-sample moving average filter slightly reduces authentication performance in general. While these findings hold true for the most part, some exceptions were noted.

Method: Dual-generative Latent Fusion (DLF) paradigm that decomposes latent space into semantic and detail elements, compressed through two distinct branches with cross-branch interaction to reduce redundancy.

Result: Achieves up to 27.93% bitrate savings on LPIPS and 53.55% on DISTS compared to MS-ILLM on CLIC2020 test set, surpassing diffusion-based codecs in visual fidelity while maintaining generative realism.

Conclusion: DLF effectively addresses the limitations of existing token-based compression by separating semantic and detail information, enabling high-fidelity reconstruction at extremely low bitrates through dual-branch processing with cross-branch optimization.

Abstract: Recent studies in extreme image compression have achieved remarkable performance by compressing the tokens from generative tokenizers. However, these methods often prioritize clustering common semantics within the dataset, while overlooking the diverse details of individual objects. Consequently, this results in suboptimal reconstruction fidelity, especially at low bitrates. To address this issue, we introduce a Dual-generative Latent Fusion (DLF) paradigm. DLF decomposes the latent into semantic and detail elements, compressing them through two distinct branches. The semantic branch clusters high-level information into compact tokens, while the detail branch encodes perceptually critical details to enhance the overall fidelity. Additionally, we propose a cross-branch interactive design to reduce redundancy between the two branches, thereby minimizing the overall bit cost. Experimental results demonstrate the impressive reconstruction quality of DLF even below 0.01 bits per pixel (bpp). On the CLIC2020 test set, our method achieves bitrate savings of up to 27.93% on LPIPS and 53.55% on DISTS compared to MS-ILLM. Furthermore, DLF surpasses recent diffusion-based codecs in visual fidelity while maintaining a comparable level of generative realism. Project: https://dlfcodec.github.io/

Method: Created TrueSkin dataset with 7,299 images systematically categorized into 6 classes under diverse conditions, then benchmarked existing recognition/generation approaches and trained/fine-tuned models using this dataset.

Result: LMMs misclassify intermediate skin tones as lighter ones; generative models struggle with accurate skin tone production due to prompt biases. Training on TrueSkin improves classification accuracy by >20% and significantly enhances skin tone fidelity in generation models.

Conclusion: TrueSkin serves as both a benchmark for evaluation and valuable training resource to enhance fairness and accuracy in skin tone recognition and generation tasks.

Abstract: Skin tone recognition and generation play important roles in model fairness, healthcare, and generative AI, yet they remain challenging due to the lack of comprehensive datasets and robust methodologies. Compared to other human image analysis tasks, state-of-the-art large multimodal models (LMMs) and image generation models struggle to recognize and synthesize skin tones accurately. To address this, we introduce TrueSkin, a dataset with 7299 images systematically categorized into 6 classes, collected under diverse lighting conditions, camera angles, and capture settings. Using TrueSkin, we benchmark existing recognition and generation approaches, revealing substantial biases: LMMs tend to misclassify intermediate skin tones as lighter ones, whereas generative models struggle to accurately produce specified skin tones when influenced by inherent biases from unrelated attributes in the prompts, such as hairstyle or environmental context. We further demonstrate that training a recognition model on TrueSkin improves classification accuracy by more than 20% compared to LMMs and conventional approaches, and fine-tuning with TrueSkin significantly improves skin tone fidelity in image generation models. Our findings highlight the need for comprehensive datasets like TrueSkin, which not only serves as a benchmark for evaluating existing models but also provides a valuable training resource to enhance fairness and accuracy in skin tone recognition and generation tasks.

Method: Proposes a one-step unaligned target-guided conditional diffusion model (UTGOS-CDM) for image translation and a multimodal multiscale registration network (MM-Reg) with multimodal fusion strategy to enhance alignment across scales and modalities.

Result: Extensive experiments on the OSdataset demonstrate that OSDM-MReg achieves superior registration accuracy compared to state-of-the-art methods.

Conclusion: The proposed framework effectively bridges the modality gap through efficient image translation and multimodal fusion, significantly accelerating the registration process while improving accuracy.

Abstract: Multimodal remote sensing image registration aligns images from different sensors for data fusion and analysis. However, existing methods often struggle to extract modality-invariant features when faced with large nonlinear radiometric differences, such as those between SAR and optical images. To address these challenges, we propose OSDM-MReg, a novel multimodal image registration framework that bridges the modality gap through image-to-image translation. Specifically, we introduce a one-step unaligned target-guided conditional diffusion model (UTGOS-CDM) to translate source and target images into a unified representation domain. Unlike traditional conditional DDPM that require hundreds of iterative steps for inference, our model incorporates a novel inverse translation objective during training to enable direct prediction of the translated image in a single step at test time, significantly accelerating the registration process. After translation, we design a multimodal multiscale registration network (MM-Reg) that extracts and fuses both unimodal and translated multimodal images using the proposed multimodal fusion strategy, enhancing the robustness and precision of alignment across scales and modalities. Extensive experiments on the OSdataset demonstrate that OSDM-MReg achieves superior registration accuracy compared to state-of-the-art methods.

Method: Reinforcement learning framework with upper confidence bound (UCB) algorithm to systematically evaluate and rank pre-trained models for animal detection tasks.

Result: Achieves higher detection rate while requiring significantly less computational time compared to traditional transfer learning methods.

Conclusion: The proposed RL-based framework provides an efficient and automated solution for model selection in transfer learning applications for animal monitoring, overcoming data limitations and architectural complexity challenges.

Abstract: Animal health monitoring and population management are critical aspects of wildlife conservation and livestock management that increasingly rely on automated detection and tracking systems. While Unmanned Aerial Vehicle (UAV) based systems combined with computer vision offer promising solutions for non-invasive animal monitoring across challenging terrains, limited availability of labeled training data remains an obstacle in developing effective deep learning (DL) models for these applications. Transfer learning has emerged as a potential solution, allowing models trained on large datasets to be adapted for resource-limited scenarios such as those with limited data. However, the vast landscape of pre-trained neural network architectures makes it challenging to select optimal models, particularly for researchers new to the field. In this paper, we propose a reinforcement learning (RL)-based transfer learning framework that employs an upper confidence bound (UCB) algorithm to automatically select the most suitable pre-trained model for animal detection tasks. Our approach systematically evaluates and ranks candidate models based on their performance, streamlining the model selection process. Experimental results demonstrate that our framework achieves a higher detection rate while requiring significantly less computational time compared to traditional methods.

Method: Robust deep learning approach based on prototypical networks that enhances prototype representations for few-shot fungal classification.

Result: Exceeds competition baseline by more than 30 percentage points in Recall@5 on both public and private leaderboards, demonstrating strong performance for both common and rare species.

Conclusion: The prototypical network approach shows strong potential for accurate fungal species identification, particularly for rare and under-documented taxa, supporting the objectives of FungiCLEF 2025 competition.

Abstract: The FungiCLEF 2025 competition addresses the challenge of automatic fungal species recognition using realistic, field-collected observational data. Accurate identification tools support both mycologists and citizen scientists, greatly enhancing large-scale biodiversity monitoring. Effective recognition systems in this context must handle highly imbalanced class distributions and provide reliable performance even when very few training samples are available for many species, especially rare and under-documented taxa that are often missing from standard training sets. According to competition organizers, about 20% of all verified fungi observations, representing nearly 20,000 instances, are associated with these rarely recorded species. To tackle this challenge, we propose a robust deep learning method based on prototypical networks, which enhances prototype representations for few-shot fungal classification. Our prototypical network approach exceeds the competition baseline by more than 30 percentage points in Recall@5 on both the public (PB) and private (PR) leaderboards. This demonstrates strong potential for accurately identifying both common and rare fungal species, supporting the main objectives of FungiCLEF 2025.

Method: Performs global clustering across all bags to establish cluster centers, local clustering within each bag, computes attention scores within clusters, and applies sparse regularization to cluster weights for selective retention of relevant clusters.

Result: Achieves state-of-the-art performance on two public histopathology benchmarks (CAMELYON16, TCGA-NSCLC) and theoretical analysis shows it requires O(s log K) bags to recover s relevant clusters.

Conclusion: csMIL enhances robustness to noisy instances, improves interpretability by identifying critical regions, reduces computational complexity, and aligns with compressed sensing principles for efficient learning.

Abstract: Multi-Instance Learning (MIL) is pivotal for analyzing complex, weakly labeled datasets, such as whole-slide images (WSIs) in computational pathology, where bags comprise unordered collections of instances with sparse diagnostic relevance. Traditional MIL approaches, including early statistical methods and recent attention-based frameworks, struggle with instance redundancy and lack explicit mechanisms for discarding non-informative instances, limiting their robustness and interpretability. We propose Cluster-level Sparse MIL (csMIL), a novel framework that integrates global-local instance clustering, within-cluster attention, and cluster-level sparsity induction to address these challenges. Our csMIL first performs global clustering across all bags to establish $K$ cluster centers, followed by local clustering within each bag to assign cluster labels. Attention scores are computed within each cluster, and sparse regularization is applied to cluster weights, enabling the selective retention of diagnostically relevant clusters while discarding irrelevant ones. This approach enhances robustness to noisy instances, improves interpretability by identifying critical regions, and reduces computational complexity. Theoretical analysis demonstrates that csMIL requires $O(s log K)$ bags to recover $s$ relevant clusters, aligning with compressed sensing principles. Empirically, csMIL achieves state-of-the-art performance on two public histopathology benchmarks (CAMELYON16, TCGA-NSCLC).

Method: Two-module approach: 1) Language-guided semantic typicality modeling that projects skeleton snippets into action semantic space and distills LLM knowledge of typical behaviors, 2) Test-time context uniqueness analysis that examines spatio-temporal differences between snippets to derive scene-adaptive boundaries.

Result: Achieves state-of-the-art results against skeleton-based methods on four large-scale VAD datasets (ShanghaiTech, UBnormal, NWPU, UCF-Crime) featuring over 100 unseen surveillance scenes without using any target domain training samples.

Conclusion: The proposed framework successfully unlocks the potential of skeleton data for zero-shot anomaly detection by leveraging action typicality and uniqueness learning, demonstrating strong generalization capabilities across diverse unseen surveillance environments.

Abstract: Zero-Shot Video Anomaly Detection (ZS-VAD) requires temporally localizing anomalies without target domain training data, which is a crucial task due to various practical concerns, e.g., data privacy or new surveillance deployments. Skeleton-based approach has inherent generalizable advantages in achieving ZS-VAD as it eliminates domain disparities both in background and human appearance. However, existing methods only learn low-level skeleton representation and rely on the domain-limited normality boundary, which cannot generalize well to new scenes with different normal and abnormal behavior patterns. In this paper, we propose a novel zero-shot video anomaly detection framework, unlocking the potential of skeleton data via action typicality and uniqueness learning. Firstly, we introduce a language-guided semantic typicality modeling module that projects skeleton snippets into action semantic space and distills LLM’s knowledge of typical normal and abnormal behaviors during training. Secondly, we propose a test-time context uniqueness analysis module to finely analyze the spatio-temporal differences between skeleton snippets and then derive scene-adaptive boundaries. Without using any training samples from the target domain, our method achieves state-of-the-art results against skeleton-based methods on four large-scale VAD datasets: ShanghaiTech, UBnormal, NWPU, and UCF-Crime, featuring over 100 unseen surveillance scenes.

Method: Developed a modular GUI platform built on Segment Anything Model 2 (SAM2) for zero-shot segmentation and automated analysis of spatial-temporal microscopy videos of kidney organoids.

Result: The platform successfully quantifies key metrics including cyst formation rate, growth velocity, and morphological changes, while generating comprehensive reports for PKD research.

Conclusion: Organoid Tracker provides an extensible, open-source framework that accelerates kidney development research, PKD modeling, and therapeutic discovery by enabling detailed quantitative analysis without programming requirements.

Abstract: Recent advances in organoid models have revolutionized the study of human kidney disease mechanisms and drug discovery by enabling scalable, cost-effective research without the need for animal sacrifice. Here, we present a kidney organoid platform optimized for efficient screening in polycystic kidney disease (PKD). While these systems generate rich spatial-temporal microscopy video datasets, current manual approaches to analysis remain limited to coarse classifications (e.g., hit vs. non-hit), often missing valuable pixel-level and longitudinal information. To help overcome this bottleneck, we developed Organoid Tracker, a graphical user interface (GUI) platform designed with a modular plugin architecture, which empowers researchers to extract detailed, quantitative metrics without programming expertise. Built on the cutting-edge vision foundation model Segment Anything Model 2 (SAM2), Organoid Tracker enables zero-shot segmentation and automated analysis of spatial-temporal microscopy videos. It quantifies key metrics such as cyst formation rate, growth velocity, and morphological changes, while generating comprehensive reports. By providing an extensible, open-source framework, Organoid Tracker offers a powerful solution for improving and accelerating research in kidney development, PKD modeling, and therapeutic discovery. The platform is publicly available as open-source software at https://github.com/hrlblab/OrganoidTracker.

Method: Built on LLaVA models with LoRA and DoRA fine-tuning methods, integrated depth information from open-source depth estimation models, and used Chain-of-Thought reasoning for multiple-choice and yes/no questions

Result: Achieved top score of 0.7799 on validation set leaderboard, ranking 1st place

Conclusion: The comprehensive approach combining enhanced LLaVA models, depth integration, and Chain-of-Thought reasoning proved highly effective for autonomous driving language tasks

Abstract: This report outlines our approach using vision language model systems for the Driving with Language track of the CVPR 2024 Autonomous Grand Challenge. We have exclusively utilized the DriveLM-nuScenes dataset for training our models. Our systems are built on the LLaVA models, which we enhanced through fine-tuning with the LoRA and DoRA methods. Additionally, we have integrated depth information from open-source depth estimation models to enrich the training and inference processes. For inference, particularly with multiple-choice and yes/no questions, we adopted a Chain-of-Thought reasoning approach to improve the accuracy of the results. This comprehensive methodology enabled us to achieve a top score of 0.7799 on the validation set leaderboard, ranking 1st on the leaderboard.

Method: Fuses camera and LiDAR data to produce BEV terrain costmaps using DINOv3 image encoder, FiLM-based sensor fusion, and self-supervised training with IMU-derived labels. Uses Huber and smoothness loss terms.

Result: Highly robust performance with minor changes in MAE/MSE under various ablation tests (e.g., MAE increases from ~0.0775 to 0.0915 when LiDAR is sparsified), indicating geometry dominates learned cost.

Conclusion: The framework demonstrates strong robustness and effectiveness, with contributions including high-fidelity simulation, self-supervised labeling pipeline, and multi-modal BEV prediction. Future work focuses on domain generalization and dataset expansion.

Abstract: We present a robust multi-modal framework for predicting traversability costmaps for planetary rovers. Our model fuses camera and LiDAR data to produce a bird’s-eye-view (BEV) terrain costmap, trained self-supervised using IMU-derived labels. Key updates include a DINOv3-based image encoder, FiLM-based sensor fusion, and an optimization loss combining Huber and smoothness terms. Experimental ablations (removing image color, occluding inputs, adding noise) show only minor changes in MAE/MSE (e.g. MAE increases from ~0.0775 to 0.0915 when LiDAR is sparsified), indicating that geometry dominates the learned cost and the model is highly robust. We attribute the small performance differences to the IMU labeling primarily reflecting terrain geometry rather than semantics and to limited data diversity. Unlike prior work claiming large gains, we emphasize our contributions: (1) a high-fidelity, reproducible simulation environment; (2) a self-supervised IMU-based labeling pipeline; and (3) a strong multi-modal BEV costmap prediction model. We discuss limitations and future work such as domain generalization and dataset expansion.

Method: Proposes CAA-Policy with Control-Aided Attention mechanism trained self-supervised using backpropagated gradients from control outputs, integrates short-horizon waypoint prediction, and uses a motion prediction module for robust target tracking.

Result: Extensive experiments in CARLA simulator show CAA-Policy consistently surpasses end-to-end learning baseline and modular BEV segmentation + hybrid A* pipeline, achieving superior accuracy, robustness, and interpretability.

Conclusion: The proposed Control-Aided Attention mechanism enables more robust and generalizable policies by focusing attention on visual features that induce high variance in action outputs rather than merely minimizing training loss.

Abstract: Precise parking requires an end-to-end system where perception adaptively provides policy-relevant details-especially in critical areas where fine control decisions are essential. End-to-end learning offers a unified framework by directly mapping sensor inputs to control actions, but existing approaches lack effective synergy between perception and control. We find that transformer-based self-attention, when used alone, tends to produce unstable and temporally inconsistent spatial attention, which undermines the reliability of downstream policy decisions over time. Instead, we propose CAA-Policy, an end-to-end imitation learning system that allows control signal to guide the learning of visual attention via a novel Control-Aided Attention (CAA) mechanism. For the first time, we train such an attention module in a self-supervised manner, using backpropagated gradients from the control outputs instead of from the training loss. This strategy encourages the attention to focus on visual features that induce high variance in action outputs, rather than merely minimizing the training loss-a shift we demonstrate leads to a more robust and generalizable policy. To further enhance stability, CAA-Policy integrates short-horizon waypoint prediction as an auxiliary task, and introduces a separately trained motion prediction module to robustly track the target spot over time. Extensive experiments in the CARLA simulator show that \titlevariable~consistently surpasses both the end-to-end learning baseline and the modular BEV segmentation + hybrid A* pipeline, achieving superior accuracy, robustness, and interpretability. Code is released at https://github.com/Joechencc/CAAPolicy.

Method: Treats panoramic video generation as an adaptation problem using LoRA. Theoretically shows LoRA can model the transformation between perspective and panoramic projections when rank exceeds task degrees of freedom. Efficiently fine-tunes pretrained video diffusion model with only ~1,000 videos.

Result: Achieves high-quality panoramic generation while maintaining proper projection geometry. Surpasses previous state-of-the-art approaches in visual quality, left-right consistency, and motion diversity.

Conclusion: LoRA-based adaptation provides an efficient and effective solution for panoramic video generation, demonstrating that complex projection transformations can be learned with minimal training data through proper adaptation techniques.

Abstract: Generating high-quality 360{\deg} panoramic videos remains a significant challenge due to the fundamental differences between panoramic and traditional perspective-view projections. While perspective videos rely on a single viewpoint with a limited field of view, panoramic content requires rendering the full surrounding environment, making it difficult for standard video generation models to adapt. Existing solutions often introduce complex architectures or large-scale training, leading to inefficiency and suboptimal results. Motivated by the success of Low-Rank Adaptation (LoRA) in style transfer tasks, we propose treating panoramic video generation as an adaptation problem from perspective views. Through theoretical analysis, we demonstrate that LoRA can effectively model the transformation between these projections when its rank exceeds the degrees of freedom in the task. Our approach efficiently fine-tunes a pretrained video diffusion model using only approximately 1,000 videos while achieving high-quality panoramic generation. Experimental results demonstrate that our method maintains proper projection geometry and surpasses previous state-of-the-art approaches in visual quality, left-right consistency, and motion diversity.

Method: Proposes SMILE framework with theoretical analysis including relationship, existence, and accessibility theorems. Learns both shared and specific representations to generalize positive information from super-resolution to unmixing.

Result: Experiments on synthetic and real datasets substantiate the usefulness of the proposed approach.

Conclusion: SMILE provides progressive theoretical support and a new framework that successfully enhances hyperspectral unmixing through super-resolution guidance with verified task affinity and convergence guarantees.

Abstract: The performance of hyperspectral unmixing may be constrained by low spatial resolution, which can be enhanced using super-resolution in a multitask learning way. However, integrating super-resolution and unmixing directly may suffer two challenges: Task affinity is not verified, and the convergence of unmixing is not guaranteed. To address the above issues, in this paper, we provide theoretical analysis and propose super-resolution guided multi-task learning method for hyperspectral unmixing (SMILE). The provided theoretical analysis validates feasibility of multitask learning way and verifies task affinity, which consists of relationship and existence theorems by proving the positive guidance of super-resolution. The proposed framework generalizes positive information from super-resolution to unmixing by learning both shared and specific representations. Moreover, to guarantee the convergence, we provide the accessibility theorem by proving the optimal solution of unmixing. The major contributions of SMILE include providing progressive theoretical support, and designing a new framework for unmixing under the guidance of super-resolution. Our experiments on both synthetic and real datasets have substantiate the usefulness of our work.

Method: Defines a new mathematical model incorporating copula theory, constructs a copula-guided framework with two key modules: copula function estimation and temporal dependency guidance, which compute and employ temporal information to guide the unmixing process.

Result: Experimental results on both synthetic and real-world datasets demonstrate the utility of the proposed Cog-TD method in effectively modeling temporal dependencies and capturing dynamical material evolution.

Conclusion: The paper successfully redefines the MTHU problem with temporal dependency, proposes a novel copula-guided framework, develops key modules with theoretical support, and demonstrates effectiveness through comprehensive experiments.

Abstract: Multitemporal hyperspectral unmixing (MTHU) aims to model variable endmembers and dynamical abundances, which emphasizes the critical temporal information. However, existing methods have limitations in modeling temporal dependency, thus fail to capture the dynamical material evolution. Motivated by the ability of copula theory in modeling dependency structure explicitly, in this paper, we propose a copula-guided temporal dependency method (Cog-TD) for multitemporal hyperspectral unmixing. Cog-TD defines new mathematical model, constructs copula-guided framework and provides two key modules with theoretical support. The mathematical model provides explicit formulations for MTHU problem definition, which describes temporal dependency structure by incorporating copula theory. The copula-guided framework is constructed for utilizing copula function, which estimates dynamical endmembers and abundances with temporal dependency. The key modules consist of copula function estimation and temporal dependency guidance, which computes and employs temporal information to guide unmixing process. Moreover, the theoretical support demonstrates that estimated copula function is valid and the represented temporal dependency exists in hyperspectral images. The major contributions of this paper include redefining MTHU problem with temporal dependency, proposing a copula-guided framework, developing two key modules and providing theoretical support. Our experimental results on both synthetic and real-world datasets demonstrate the utility of the proposed method.

Method: Dataset collected using low-cost UAVs over hurricane-damaged regions, with 3D reconstruction via Structure-from-Motion and Multi-View Stereo. Semantic annotations created through manual 2D labeling projected into 3D space.

Result: The dataset captures large-scale outdoor environments with fine-grained structural damage in real disaster contexts, enabling evaluation of state-of-the-art 3D segmentation models.

Conclusion: 3DAeroRelief serves as a valuable resource for advancing robust 3D vision systems for post-disaster response applications, highlighting both challenges and opportunities in 3D scene understanding for emergency scenarios.

Abstract: Timely assessment of structural damage is critical for disaster response and recovery. However, most prior work in natural disaster analysis relies on 2D imagery, which lacks depth, suffers from occlusions, and provides limited spatial context. 3D semantic segmentation offers a richer alternative, but existing 3D benchmarks focus mainly on urban or indoor scenes, with little attention to disaster-affected areas. To address this gap, we present 3DAeroRelief–the first 3D benchmark dataset specifically designed for post-disaster assessment. Collected using low-cost unmanned aerial vehicles (UAVs) over hurricane-damaged regions, the dataset features dense 3D point clouds reconstructed via Structure-from-Motion and Multi-View Stereo techniques. Semantic annotations were produced through manual 2D labeling and projected into 3D space. Unlike existing datasets, 3DAeroRelief captures 3D large-scale outdoor environments with fine-grained structural damage in real-world disaster contexts. UAVs enable affordable, flexible, and safe data collection in hazardous areas, making them particularly well-suited for emergency scenarios. To demonstrate the utility of 3DAeroRelief, we evaluate several state-of-the-art 3D segmentation models on the dataset to highlight both the challenges and opportunities of 3D scene understanding in disaster response. Our dataset serves as a valuable resource for advancing robust 3D vision systems in real-world applications for post-disaster scenarios.

Method: Proposes GEMMNet with three components: 1) Hybrid Feature Extractor (HyFEx) for modality-specific representations, 2) Hybrid Fusion with Multiscale Awareness (HyFMA) for cross-scale semantic context, and 3) Complementary Loss (CoLoss) to reduce bias by encouraging modality and task consistency.

Result: GEMMNet outperforms generative baselines (AE, cGAN) and state-of-the-art non-generative approaches (mmformer, shaspec) on two challenging remote sensing semantic segmentation datasets (Vaihingen and Potsdam).

Conclusion: The proposed GEMMNet effectively addresses missing modality challenges in remote sensing by combining generative enhancement with robust multimodal learning, demonstrating superior performance over existing approaches.

Abstract: Multimodal learning has shown significant performance boost compared to ordinary unimodal models across various domains. However, in real-world scenarios, multimodal signals are susceptible to missing because of sensor failures and adverse weather conditions, which drastically deteriorates models’ operation and performance. Generative models such as AutoEncoder (AE) and Generative Adversarial Network (GAN) are intuitive solutions aiming to reconstruct missing modality from available ones. Yet, their efficacy in remote sensing semantic segmentation remains underexplored. In this paper, we first examine the limitations of existing generative approaches in handling the heterogeneity of multimodal remote sensing data. They inadequately capture semantic context in complex scenes with large intra-class and small inter-class variation. In addition, traditional generative models are susceptible to heavy dependence on the dominant modality, introducing bias that affects model robustness under missing modality conditions. To tackle these limitations, we propose a novel Generative-Enhanced MultiModal learning Network (GEMMNet) with three key components: (1) Hybrid Feature Extractor (HyFEx) to effectively learn modality-specific representations, (2) Hybrid Fusion with Multiscale Awareness (HyFMA) to capture modality-synergistic semantic context across scales and (3) Complementary Loss (CoLoss) scheme to alleviate the inherent bias by encouraging consistency across modalities and tasks. Our method, GEMMNet, outperforms both generative baselines AE, cGAN (conditional GAN), and state-of-the-art non-generative approaches - mmformer and shaspec - on two challenging semantic segmentation remote sensing datasets (Vaihingen and Potsdam). Source code is made available.

Method: Targeted techniques including smoke extraction with background removal, initialization of smoke particles and camera poses, and inferring multi-view videos from single input.

Result: Outperforms previous methods with +2.22 average PSNR on wild videos and enables realistic fluid dynamics editing through simulation.

Conclusion: The method successfully bridges the gap between controlled lab environments and real-world video capture for smoke reconstruction and editing.

Abstract: We propose a pipeline to extract and reconstruct dynamic 3D smoke assets from a single in-the-wild video, and further integrate interactive simulation for smoke design and editing. Recent developments in 3D vision have significantly improved reconstructing and rendering fluid dynamics, supporting realistic and temporally consistent view synthesis. However, current fluid reconstructions rely heavily on carefully controlled clean lab environments, whereas real-world videos captured in the wild are largely underexplored. We pinpoint three key challenges of reconstructing smoke in real-world videos and design targeted techniques, including smoke extraction with background removal, initialization of smoke particles and camera poses, and inferring multi-view videos. Our method not only outperforms previous reconstruction and generation methods with high-quality smoke reconstructions (+2.22 average PSNR on wild videos), but also enables diverse and realistic editing of fluid dynamics by simulating our smoke assets. We provide our models, data, and 4D smoke assets at https://autumnyq.github.io/WildSmoke.

Method: Proposes SVR-GS with per-pixel spatial mask rendering from Gaussian contributions along rays, implementing three spatial-mask aggregation strategies in CUDA with gradient analysis.

Result: Achieves 1.79× Gaussian reduction vs MaskGS and 5.63× vs 3DGS with only 0.50 dB and 0.40 dB PSNR drops respectively across Tanks&Temples, Deep Blending, and Mip-NeRF360 datasets.

Conclusion: SVR-GS produces significantly smaller, faster, and more memory-efficient models suitable for real-time applications like robotics, AR/VR, and mobile perception.

Abstract: 3D Gaussian Splatting (3DGS) enables fast, high-quality novel view synthesis but typically relies on densification followed by pruning to optimize the number of Gaussians. Existing mask-based pruning, such as MaskGS, regularizes the global mean of the mask, which is misaligned with the local per-pixel (per-ray) reconstruction loss that determines image quality along individual camera rays. This paper introduces SVR-GS, a spatially variant regularizer that renders a per-pixel spatial mask from each Gaussian’s effective contribution along the ray, thereby applying sparsity pressure where it matters: on low-importance Gaussians. We explore three spatial-mask aggregation strategies, implement them in CUDA, and conduct a gradient analysis to motivate our final design. Extensive experiments on Tanks&Temples, Deep Blending, and Mip-NeRF360 datasets demonstrate that, on average across the three datasets, the proposed SVR-GS reduces the number of Gaussians by 1.79(\times) compared to MaskGS and 5.63(\times) compared to 3DGS, while incurring only 0.50 dB and 0.40 dB PSNR drops, respectively. These gains translate into significantly smaller, faster, and more memory-efficient models, making them well-suited for real-time applications such as robotics, AR/VR, and mobile perception.

Method: Uses pre-trained VGGT to extract dense point maps from multi-view images, merges them into unified point cloud with confidence scores, then processes through parallel DGCNN decoder heads with composite Gaussian negative log-likelihood loss for joint volume and surface area prediction.

Result: Achieves competitive accuracy and generalizes well to unseen coral morphologies, providing efficient coral geometry estimation from sparse image sets.

Conclusion: This framework enables efficient and scalable coral geometry estimation directly from images, with potential applications in coral growth analysis and reef monitoring.

Abstract: Effective reef monitoring requires the quantification of coral growth via accurate volumetric and surface area estimates, which is a challenging task due to the complex morphology of corals. We propose a novel, lightweight, and scalable learning framework that addresses this challenge by predicting the 3D volume and surface area of coral-like objects from 2D multi-view RGB images. Our approach utilizes a pre-trained module (VGGT) to extract dense point maps from each view; these maps are merged into a unified point cloud and enriched with per-view confidence scores. The resulting cloud is fed to two parallel DGCNN decoder heads, which jointly output the volume and the surface area of the coral, as well as their corresponding confidence estimate. To enhance prediction stability and provide uncertainty estimates, we introduce a composite loss function based on Gaussian negative log-likelihood in both real and log domains. Our method achieves competitive accuracy and generalizes well to unseen morphologies. This framework paves the way for efficient and scalable coral geometry estimation directly from a sparse set of images, with potential applications in coral growth analysis and reef monitoring.

Method: Built on Qwen2.5-VL backbone, uses LoRA for lightweight fine-tuning, and incorporates knowledge prompting module combining Chain-of-Thought reasoning with Retrieval-Augmented Generation for precise traffic domain knowledge injection.

Result: Achieves state-of-the-art performance on TrafficQA and DriveQA benchmarks, demonstrating superior multimodal traffic data processing with remarkable zero-shot reasoning and cross-scenario generalization capabilities.

Conclusion: Traffic-MLLM effectively addresses spatiotemporal causality challenges in traffic video analysis through innovative knowledge integration and lightweight adaptation, showing strong potential for intelligent transportation systems.

Abstract: As intelligent transportation systems advance, traffic video understanding plays an increasingly pivotal role in comprehensive scene perception and causal analysis. Yet, existing approaches face notable challenges in accurately modeling spatiotemporal causality and integrating domain-specific knowledge, limiting their effectiveness in complex scenarios. To address these limitations, we propose Traffic-MLLM, a multimodal large language model tailored for fine-grained traffic analysis. Built on the Qwen2.5-VL backbone, our model leverages high-quality traffic-specific multimodal datasets and uses Low-Rank Adaptation (LoRA) for lightweight fine-tuning, significantly enhancing its capacity to model continuous spatiotemporal features in video sequences. Furthermore, we introduce an innovative knowledge prompting module fusing Chain-of-Thought (CoT) reasoning with Retrieval-Augmented Generation (RAG), enabling precise injection of detailed traffic regulations and domain knowledge into the inference process. This design markedly boosts the model’s logical reasoning and knowledge adaptation capabilities. Experimental results on TrafficQA and DriveQA benchmarks show Traffic-MLLM achieves state-of-the-art performance, validating its superior ability to process multimodal traffic data. It also exhibits remarkable zero-shot reasoning and cross-scenario generalization capabilities.

Method: Enhanced NeRF architecture with three modifications: expanded hidden layers for 6-band inputs, redesigned residual functions for spectral discrepancy optimization, and adapted data compression modules for higher bit-depth multispectral imagery.

Result: Multispectral-NeRF successfully processes multi-band spectral features while accurately preserving original scenes’ spectral characteristics, as confirmed by experimental results.

Conclusion: The proposed Multispectral-NeRF effectively addresses the limitations of current multispectral 3D reconstruction methods by extending NeRF to handle 6-band spectral data, producing high-precision reconstruction with preserved spectral properties.

Abstract: 3D reconstruction technology generates three-dimensional representations of real-world objects, scenes, or environments using sensor data such as 2D images, with extensive applications in robotics, autonomous vehicles, and virtual reality systems. Traditional 3D reconstruction techniques based on 2D images typically relies on RGB spectral information. With advances in sensor technology, additional spectral bands beyond RGB have been increasingly incorporated into 3D reconstruction workflows. Existing methods that integrate these expanded spectral data often suffer from expensive scheme prices, low accuracy and poor geometric features. Three - dimensional reconstruction based on NeRF can effectively address the various issues in current multispectral 3D reconstruction methods, producing high - precision and high - quality reconstruction results. However, currently, NeRF and some improved models such as NeRFacto are trained on three - band data and cannot take into account the multi - band information. To address this problem, we propose Multispectral-NeRF, an enhanced neural architecture derived from NeRF that can effectively integrates multispectral information. Our technical contributions comprise threefold modifications: Expanding hidden layer dimensionality to accommodate 6-band spectral inputs; Redesigning residual functions to optimize spectral discrepancy calculations between reconstructed and reference images; Adapting data compression modules to address the increased bit-depth requirements of multispectral imagery. Experimental results confirm that Multispectral-NeRF successfully processes multi-band spectral features while accurately preserving the original scenes’ spectral characteristics.

Method: SPHERE uses Semantic-guided Gaussian Initialization (SGI) module with dual-branch 3D scene representations to locate focal voxels as anchors for efficient Gaussian initialization, and Physical-aware Harmonics Enhancement (PHE) module with semantic spherical harmonics to model physical-aware contextual details and promote semantic-geometry consistency through focal distribution alignment.

Result: Extensive experiments on SemanticKITTI and SSCBench-KITTI-360 benchmarks validate the effectiveness of SPHERE in achieving superior performance in 3D semantic scene completion.

Conclusion: SPHERE successfully integrates voxel and Gaussian representations to jointly exploit semantic and physical information, generating SSC results with realistic details while addressing the limitations of both traditional voxel-based methods and neural reconstruction approaches.

Abstract: Camera-based 3D Semantic Scene Completion (SSC) is a critical task in autonomous driving systems, assessing voxel-level geometry and semantics for holistic scene perception. While existing voxel-based and plane-based SSC methods have achieved considerable progress, they struggle to capture physical regularities for realistic geometric details. On the other hand, neural reconstruction methods like NeRF and 3DGS demonstrate superior physical awareness, but suffer from high computational cost and slow convergence when handling large-scale, complex autonomous driving scenes, leading to inferior semantic accuracy. To address these issues, we propose the Semantic-PHysical Engaged REpresentation (SPHERE) for camera-based SSC, which integrates voxel and Gaussian representations for joint exploitation of semantic and physical information. First, the Semantic-guided Gaussian Initialization (SGI) module leverages dual-branch 3D scene representations to locate focal voxels as anchors to guide efficient Gaussian initialization. Then, the Physical-aware Harmonics Enhancement (PHE) module incorporates semantic spherical harmonics to model physical-aware contextual details and promote semantic-geometry consistency through focal distribution alignment, generating SSC results with realistic details. Extensive experiments and analyses on the popular SemanticKITTI and SSCBench-KITTI-360 benchmarks validate the effectiveness of SPHERE. The code is available at https://github.com/PKU-ICST-MIPL/SPHERE_ACMMM2025.

Method: Proposes StegOT with multiple channel optimal transport (MCOT) module that transforms multi-peak feature distributions into single-peak distributions to achieve information trade-off between cover and secret images.

Result: Experiments show StegOT achieves better trade-off between cover and secret images while enhancing quality of both stego and recovery images compared to existing methods.

Conclusion: StegOT effectively addresses mode collapse in steganography through optimal transport theory, improving information balance and image quality in both hiding and extraction processes.

Abstract: Image hiding is often referred to as steganography, which aims to hide a secret image in a cover image of the same resolution. Many steganography models are based on genera-tive adversarial networks (GANs) and variational autoencoders (VAEs). However, most existing models suffer from mode collapse. Mode collapse will lead to an information imbalance between the cover and secret images in the stego image and further affect the subsequent extraction. To address these challenges, this paper proposes StegOT, an autoencoder-based steganography model incorporating optimal transport theory. We designed the multiple channel optimal transport (MCOT) module to transform the feature distribution, which exhibits multiple peaks, into a single peak to achieve the trade-off of information. Experiments demonstrate that we not only achieve a trade-off between the cover and secret images but also enhance the quality of both the stego and recovery images. The source code will be released on https://github.com/Rss1124/StegOT.

Method: Trained multiple AI models to classify benign vs malignant lesions using FST-specific data with different granularity levels (3 groups: FST 1/2, 3/4, 5/6 vs combined groups).

Result: Models trained on FST-specific data performed better than general FST-balanced models; reducing FST granularity (combining 1/2 and 3/4 into 1/2/3/4) negatively impacted performance.

Conclusion: FST scale granularity significantly impacts AI model performance, supporting the need to move away from FST scale to alternative scales that better represent skin tone diversity in fair AI research.

Abstract: Artificial intelligence (AI) models to automatically classify skin lesions from dermatology images have shown promising performance but also susceptibility to bias by skin tone. The most common way of representing skin tone information is the Fitzpatrick Skin Tone (FST) scale. The FST scale has been criticised for having greater granularity in its skin tone categories for lighter-skinned subjects. This paper conducts an investigation of the impact (on performance and bias) on AI classification models of granularity in the FST scale. By training multiple AI models to classify benign vs. malignant lesions using FST-specific data of differing granularity, we show that: (i) when training models using FST-specific data based on three groups (FST 1/2, 3/4 and 5/6), performance is generally better for models trained on FST-specific data compared to a general model trained on FST-balanced data; (ii) reducing the granularity of FST scale information (from 1/2 and 3/4 to 1/2/3/4) can have a detrimental effect on performance. Our results highlight the importance of the granularity of FST groups when training lesion classification models. Given the question marks over possible human biases in the choice of categories in the FST scale, this paper provides evidence for a move away from the FST scale in fair AI research and a transition to an alternative scale that better represents the diversity of human skin tones.

Method: Developed synthetic data generation pipeline integrating game engines with physics-based LiDAR simulation to create large-scale annotated 3D forest dataset, then used synthetic data for pretraining with minimal real data for fine-tuning.

Result: After fine-tuning on just a single forest plot (<0.1 hectare), the pretrained model achieves segmentations competitive with models trained on full-scale real data, reducing labeled real data requirements substantially.

Conclusion: Synthetic data with physics, diversity, and scale can enable more robust 3D forest vision systems, with critical success factors identified for synthetic data usage in forest vision tasks.

Abstract: Accurate tree segmentation is a key step in extracting individual tree metrics from forest laser scans, and is essential to understanding ecosystem functions in carbon cycling and beyond. Over the past decade, tree segmentation algorithms have advanced rapidly due to developments in AI. However existing, public, 3D forest datasets are not large enough to build robust tree segmentation systems. Motivated by the success of synthetic data in other domains such as self-driving, we investigate whether similar approaches can help with tree segmentation. In place of expensive field data collection and annotation, we use synthetic data during pretraining, and then require only minimal, real forest plot annotation for fine-tuning. We have developed a new synthetic data generation pipeline to do this for forest vision tasks, integrating advances in game-engines with physics-based LiDAR simulation. As a result, we have produced a comprehensive, diverse, annotated 3D forest dataset on an unprecedented scale. Extensive experiments with a state-of-the-art tree segmentation algorithm and a popular real dataset show that our synthetic data can substantially reduce the need for labelled real data. After fine-tuning on just a single, real, forest plot of less than 0.1 hectare, the pretrained model achieves segmentations that are competitive with a model trained on the full scale real data. We have also identified critical factors for successful use of synthetic data: physics, diversity, and scale, paving the way for more robust 3D forest vision systems in the future. Our data generation pipeline and the resulting dataset are available at https://github.com/yihshe/CAMP3D.git.

Method: Integrates GANs and diffusion models with fine-grained textual and visual guidance in an adversarial manner to refine images, building upon but improving concept slider approaches.

Result: Extensive experimental validation confirms the framework’s robustness and versatility across various applications, showing marked enhancement in image quality and realism.

Conclusion: Beyond Sliders successfully bridges the gap in image manipulation capabilities, providing sophisticated handling of diverse image types including real-world captured images where previous methods faltered.

Abstract: In the realm of image generation, the quest for realism and customization has never been more pressing. While existing methods like concept sliders have made strides, they often falter when it comes to no-AIGC images, particularly images captured in real world settings. To bridge this gap, we introduce Beyond Sliders, an innovative framework that integrates GANs and diffusion models to facilitate sophisticated image manipulation across diverse image categories. Improved upon concept sliders, our method refines the image through fine grained guidance both textual and visual in an adversarial manner, leading to a marked enhancement in image quality and realism. Extensive experimental validation confirms the robustness and versatility of Beyond Sliders across a spectrum of applications.

Method: Proposes a probabilistic extension of Spatial Transformer Networks that decomposes affine transformations into rotation, scaling, and shearing components. Uses a shared localization encoder with geometric constraints, models each component with Gaussian variational posterior, and employs sampling-based canonicalization during inference. Includes a novel component-wise alignment loss using augmentation parameters.

Result: Experiments on challenging moth classification benchmarks demonstrate consistent improvement in robustness compared to other STN methods.

Conclusion: The proposed probabilistic component-wise STN approach provides a flexible, backbone-agnostic solution for handling geometric variability in fine-grained visual classification without architectural constraints.

Abstract: Fine-grained visual classification (FGVC) remains highly sensitive to geometric variability, where objects appear under arbitrary orientations, scales, and perspective distortions. While equivariant architectures address this issue, they typically require substantial computational resources and restrict the hypothesis space. We revisit Spatial Transformer Networks (STNs) as a canonicalization tool for transformer-based vision pipelines, emphasizing their flexibility, backbone-agnostic nature, and lack of architectural constraints. We propose a probabilistic, component-wise extension that improves robustness. Specifically, we decompose affine transformations into rotation, scaling, and shearing, and regress each component under geometric constraints using a shared localization encoder. To capture uncertainty, we model each component with a Gaussian variational posterior and perform sampling-based canonicalization during inference.A novel component-wise alignment loss leverages augmentation parameters to guide spatial alignment. Experiments on challenging moth classification benchmarks demonstrate that our method consistently improves robustness compared to other STNs.

Method: Developed a Cooperative Model-Agnostic Meta-Learning (CCoMAML) framework with Multi-Head Attention Feature Fusion (MHAFF) as feature extractor for few-shot learning, enabling efficient adaptation to new data without retraining.

Result: Achieved superior performance with 98.46% and 97.91% F1 scores, outperforming current state-of-the-art few-shot learning techniques in cattle identification.

Conclusion: The proposed CCoMAML with MHAFF framework provides an effective solution for real-time cattle identification that adapts efficiently to new data with minimal samples, addressing limitations of traditional RFID and conventional deep learning approaches.

Abstract: Cattle identification is critical for efficient livestock farming management, currently reliant on radio-frequency identification (RFID) ear tags. However, RFID-based systems are prone to failure due to loss, damage, tampering, and vulnerability to external attacks. As a robust alternative, biometric identification using cattle muzzle patterns similar to human fingerprints has emerged as a promising solution. Deep learning techniques have demonstrated success in leveraging these unique patterns for accurate identification. But deep learning models face significant challenges, including limited data availability, disruptions during data collection, and dynamic herd compositions that require frequent model retraining. To address these limitations, this paper proposes a novel few-shot learning framework for real-time cattle identification using Cooperative Model-Agnostic Meta-Learning (CCoMAML) with Multi-Head Attention Feature Fusion (MHAFF) as a feature extractor model. This model offers great model adaptability to new data through efficient learning from few data samples without retraining. The proposed approach has been rigorously evaluated against current state-of-the-art few-shot learning techniques applied in cattle identification. Comprehensive experimental results demonstrate that our proposed CCoMAML with MHAFF has superior cattle identification performance with 98.46% and 97.91% F1 scores.

Method: Implements adversarially and naturally robust Hellinger distance-based feature class aggregation, attention mechanisms, and a novel Hellinger Similarity contrastive loss function for variational few-shot inference.

Result: Achieves resilience to adversarial perturbations up to ε=0.30 and Gaussian noise up to σ=0.30, with performance gains of 1.20% (1-shot) and 1.40% (5-shot) on miniImageNet, and superior image reconstruction quality (FID score of 2.75).

Conclusion: ANROT-HELANet establishes new state-of-the-art performance in FSL while maintaining robustness against both adversarial and natural perturbations through its Hellinger-based approach.

Abstract: Few-Shot Learning (FSL), which involves learning to generalize using only a few data samples, has demonstrated promising and superior performances to ordinary CNN methods. While Bayesian based estimation approaches using Kullback-Leibler (KL) divergence have shown improvements, they remain vulnerable to adversarial attacks and natural noises. We introduce ANROT-HELANet, an Adversarially and Naturally RObusT Hellinger Aggregation Network that significantly advances the state-of-the-art in FSL robustness and performance. Our approach implements an adversarially and naturally robust Hellinger distance-based feature class aggregation scheme, demonstrating resilience to adversarial perturbations up to $\epsilon=0.30$ and Gaussian noise up to $\sigma=0.30$. The network achieves substantial improvements across benchmark datasets, including gains of 1.20% and 1.40% for 1-shot and 5-shot scenarios on miniImageNet respectively. We introduce a novel Hellinger Similarity contrastive loss function that generalizes cosine similarity contrastive loss for variational few-shot inference scenarios. Our approach also achieves superior image reconstruction quality with a FID score of 2.75, outperforming traditional VAE (3.43) and WAE (3.38) approaches. Extensive experiments conducted on four few-shot benchmarked datasets verify that ANROT-HELANet’s combination of Hellinger distance-based feature aggregation, attention mechanisms, and our novel loss function establishes new state-of-the-art performance while maintaining robustness against both adversarial and natural perturbations. Our code repository will be available at https://github.com/GreedYLearner1146/ANROT-HELANet/tree/main.

Method: Uses CNN encoders for continuous data (rendered as multi-channel images) and 1D-CNN for discrete events, fused with Convolutional Block Attention Module, then aggregated by LSTM for temporal dependencies, with UALRE uncertainty-aware ensemble.

Result: Base MIS-LSTM achieves Macro-F1 0.615; with UALRE ensemble improves to 0.647, outperforming LSTM, 1D-CNN, and CNN baselines. Best performance with 4-hour block granularity and multi-channel imaging.

Conclusion: The hybrid CNN-LSTM framework with attention fusion and uncertainty-aware ensemble effectively handles multimodal lifelog data for sleep/stress prediction, demonstrating superiority over baseline methods and optimal configuration choices.

Abstract: This paper presents MIS-LSTM, a hybrid framework that joins CNN encoders with an LSTM sequence model for sleep quality and stress prediction at the day level from multimodal lifelog data. Continuous sensor streams are first partitioned into N-hour blocks and rendered as multi-channel images, while sparse discrete events are encoded with a dedicated 1D-CNN. A Convolutional Block Attention Module fuses the two modalities into refined block embeddings, which an LSTM then aggregates to capture long-range temporal dependencies. To further boost robustness, we introduce UALRE, an uncertainty-aware ensemble that overrides lowconfidence majority votes with high-confidence individual predictions. Experiments on the 2025 ETRI Lifelog Challenge dataset show that Our base MISLSTM achieves Macro-F1 0.615; with the UALRE ensemble, the score improves to 0.647, outperforming strong LSTM, 1D-CNN, and CNN baselines. Ablations confirm (i) the superiority of multi-channel over stacked-vertical imaging, (ii) the benefit of a 4-hour block granularity, and (iii) the efficacy of modality-specific discrete encoding.

Method: CMLReID framework with two novel components: (1) Context-Aware Semantic Prompt (CASP) that generates adaptive prompts and aligns multi-grained visual cues with semantic text space, and (2) Adaptive Knowledge Fusion and Projection (AKFP) that produces robust prototypes using dual-path learner and Clothing-State-Aware Projection Loss.

Result: Outperforms all state-of-the-art methods across multiple datasets, demonstrating strong robustness and generalization despite clothing variations and sequential learning challenges.

Conclusion: The proposed CMLReID framework successfully addresses the hybrid continual learning task for both same-cloth and clothing-change person re-identification, providing a unified solution with superior performance and generalization capabilities.

Abstract: Person Re-Identification (ReID) has several challenges in real-world surveillance systems due to clothing changes (CCReID) and the need for maintaining continual learning (LReID). Previous existing methods either develop models specifically for one application, which is mostly a same-cloth (SC) setting or treat CCReID as its own separate sub-problem. In this work, we will introduce the LReID-Hybrid task with the goal of developing a model to achieve both SC and CC while learning in a continual setting. Mismatched representations and forgetting from one task to the next are significant issues, we address this with CMLReID, a CLIP-based framework composed of two novel tasks: (1) Context-Aware Semantic Prompt (CASP) that generates adaptive prompts, and also incorporates context to align richly multi-grained visual cues with semantic text space; and (2) Adaptive Knowledge Fusion and Projection (AKFP) which produces robust SC/CC prototypes through the use of a dual-path learner that aligns features with our Clothing-State-Aware Projection Loss. Experiments performed on a wide range of datasets and illustrate that CMLReID outperforms all state-of-the-art methods with strong robustness and generalization despite clothing variations and a sophisticated process of sequential learning.

Method: Extract class-agnostic attributes using CLIP, represent them as key-value pairs (visual prototype + textual prompt), maintain domain-specific attribute dictionaries, and perform cross-domain alignment through visual attention consistency and prediction consistency.

Result: Outperforms previous state-of-the-art methods on three CI-UDA benchmarks and effectively alleviates catastrophic forgetting without requiring rehearsal samples.

Conclusion: The proposed attribute-based framework provides an effective rehearsal-free solution for CI-UDA by preserving domain-invariant knowledge and enabling better cross-domain alignment, demonstrating superior performance over existing methods.

Abstract: Class-Incremental Unsupervised Domain Adaptation (CI-UDA) aims to adapt a model from a labeled source domain to an unlabeled target domain, where the sets of potential target classes appearing at different time steps are disjoint and are subsets of the source classes. The key to solving this problem lies in avoiding catastrophic forgetting of knowledge about previous target classes during continuously mitigating the domain shift. Most previous works cumbersomely combine two technical components. On one hand, they need to store and utilize rehearsal target sample from previous time steps to avoid catastrophic forgetting; on the other hand, they perform alignment only between classes shared across domains at each time step. Consequently, the memory will continuously increase and the asymmetric alignment may inevitably result in knowledge forgetting. In this paper, we propose to mine and preserve domain-invariant and class-agnostic knowledge to facilitate the CI-UDA task. Specifically, via using CLIP, we extract the class-agnostic properties which we name as “attribute”. In our framework, we learn a “key-value” pair to represent an attribute, where the key corresponds to the visual prototype and the value is the textual prompt. We maintain two attribute dictionaries, each corresponding to a different domain. Then we perform attribute alignment across domains to mitigate the domain shift, via encouraging visual attention consistency and prediction consistency. Through attribute modeling and cross-domain alignment, we effectively reduce catastrophic knowledge forgetting while mitigating the domain shift, in a rehearsal-free way. Experiments on three CI-UDA benchmarks demonstrate that our method outperforms previous state-of-the-art methods and effectively alleviates catastrophic forgetting. Code is available at https://github.com/RyunMi/VisTA.

Method: Integrates generalized cross-validation experiments and domain transfer learning by training task-specific models on both synthetic and real datasets, forming a cross-performance matrix and GCV Matrix to quantify domain transferability with two metrics for simulation quality and transfer quality.

Result: Experimental validation on Virtual KITTI demonstrates the framework’s effectiveness in assessing synthetic data fidelity, providing scalable and quantifiable evaluation that overcomes traditional limitations.

Conclusion: The proposed framework offers a principled, standardized approach for synthetic dataset quality evaluation, enabling generalizable assessments and guiding optimization in AI research.

Abstract: With the rapid advancement of synthetic dataset generation techniques, evaluating the quality of synthetic data has become a critical research focus. Robust evaluation not only drives innovations in data generation methods but also guides researchers in optimizing the utilization of these synthetic resources. However, current evaluation studies for synthetic datasets remain limited, lacking a universally accepted standard framework. To address this, this paper proposes a novel evaluation framework integrating generalized cross-validation experiments and domain transfer learning principles, enabling generalizable and comparable assessments of synthetic dataset quality. The framework involves training task-specific models (e.g., YOLOv5s) on both synthetic datasets and multiple real-world benchmarks (e.g., KITTI, BDD100K), forming a cross-performance matrix. Following normalization, a Generalized Cross-Validation (GCV) Matrix is constructed to quantify domain transferability. The framework introduces two key metrics. One measures the simulation quality by quantifying the similarity between synthetic data and real-world datasets, while another evaluates the transfer quality by assessing the diversity and coverage of synthetic data across various real-world scenarios. Experimental validation on Virtual KITTI demonstrates the effectiveness of our proposed framework and metrics in assessing synthetic data fidelity. This scalable and quantifiable evaluation solution overcomes traditional limitations, providing a principled approach to guide synthetic dataset optimization in artificial intelligence research.

Method: Two-stage pipeline: 1) Reconstructs scene geometry using monocular normal priors with compact 2D Gaussian Splatting (2DGS), 2) Decomposes texture and lighting through hybrid lighting model using spherical Gaussian function for directional sunlight and Spherical Harmonic coefficients for low-frequency skylight.

Result: ROSGS achieves state-of-the-art performance in relighting outdoor scenes, demonstrating superior relighting accuracy and rendering efficiency through both quantitative metrics and qualitative comparisons.

Conclusion: The proposed ROSGS framework effectively addresses the limitations of previous outdoor scene decomposition methods by combining efficient geometric reconstruction with a hybrid lighting model, resulting in optimal performance for relightable outdoor scene reconstruction.

Abstract: Image data captured outdoors often exhibit unbounded scenes and unconstrained, varying lighting conditions, making it challenging to decompose them into geometry, reflectance, and illumination. Recent works have focused on achieving this decomposition using Neural Radiance Fields (NeRF) or the 3D Gaussian Splatting (3DGS) representation but remain hindered by two key limitations: the high computational overhead associated with neural networks of NeRF and the use of low-frequency lighting representations, which often result in inefficient rendering and suboptimal relighting accuracy. We propose ROSGS, a two-stage pipeline designed to efficiently reconstruct relightable outdoor scenes using the Gaussian Splatting representation. By leveraging monocular normal priors, ROSGS first reconstructs the scene’s geometry with the compact 2D Gaussian Splatting (2DGS) representation, providing an efficient and accurate geometric foundation. Building upon this reconstructed geometry, ROSGS then decomposes the scene’s texture and lighting through a hybrid lighting model. This model effectively represents typical outdoor lighting by employing a spherical Gaussian function to capture the directional, high-frequency components of sunlight, while learning a radiance transfer function via Spherical Harmonic coefficients to model the remaining low-frequency skylight comprehensively. Both quantitative metrics and qualitative comparisons demonstrate that ROSGS achieves state-of-the-art performance in relighting outdoor scenes and highlight its ability to deliver superior relighting accuracy and rendering efficiency.

Method: APASI leverages the target LVLM to self-inject hallucinations into generated responses, creating preference pairs based on three key observations of hallucinations. It uses iterative alignment training with curriculum learning to periodically update preference data with increasing challenge.

Result: Extensive experiments across six benchmarks show APASI effectively mitigates hallucinations for three baseline models and achieves comparable or superior performance to alignment-based methods with external dependencies.

Conclusion: APASI demonstrates a novel and generalizable approach to hallucination mitigation that operates without external dependencies, enabling stable and continuous enhancement of LVLMs while maintaining effectiveness comparable to externally-dependent methods.

Abstract: Large Vision-Language Models (LVLMs) suffer from serious hallucination problems, where the model-generated responses are inconsistent with the visual inputs. Existing hallucination mitigation methods are mainly based on preference alignment and require external human annotations or auxiliary models for preference data collection, which increase costs and limit sustainable improvement. To tackle these challenges, we propose Autonomous Preference Alignment via Self-Injection (APASI), a novel and generalizable method that mitigates hallucinations without external dependencies. APASI leverages the target LVLM to self-inject hallucinations into a generated response, creating a pair of responses with varying preference levels. During the self-injection process, the dis-preferred response is generated based on three key observations of hallucinations, ensuring it simulates real hallucination patterns. This fidelity offers an accurate learning signal for hallucination mitigation. Moreover, APASI incorporates an iterative alignment training strategy combined with curriculum learning to periodically update the preference data with increasing challenge, enabling stable and continuous enhancement of the LVLM. Extensive experiments across six benchmarks show that APASI not only effectively mitigates hallucinations for three baseline models but also achieves comparable or even superior performance to alignment-based methods with external dependency, thereby demonstrating its effectiveness and generalization capability. The code is available at https://github.com/davidluciolu/APASI.

Method: Proposes a training-free framework that integrates geometric priors from a Geometric Foundation Model with visual foundation model representations. This enables reliable identification of visual overlaps, robust correspondence establishment, and explicit occlusion detection.

Result: The approach demonstrates superior and robust performance through extensive evaluation on PSCD, ChangeSim, and PASLCD datasets, outperforming existing methods that rely only on 2D visual cues.

Conclusion: Leveraging geometric priors from foundation models effectively addresses the core challenges of unaligned scene change detection, providing reliable performance under viewpoint misalignment without requiring additional training.

Abstract: Unaligned Scene Change Detection aims to detect scene changes between image pairs captured at different times without assuming viewpoint alignment. To handle viewpoint variations, current methods rely solely on 2D visual cues to establish cross-image correspondence to assist change detection. However, large viewpoint changes can alter visual observations, causing appearance-based matching to drift or fail. Additionally, supervision limited to 2D change masks from small-scale SCD datasets restricts the learning of generalizable multi-view knowledge, making it difficult to reliably identify visual overlaps and handle occlusions. This lack of explicit geometric reasoning represents a critical yet overlooked limitation. In this work, we are the first to leverage geometric priors from a Geometric Foundation Model to address the core challenges of unaligned SCD, including reliable identification of visual overlaps, robust correspondence establishment, and explicit occlusion detection. Building on these priors, we propose a training-free framework that integrates them with the powerful representations of a visual foundation model to enable reliable change detection under viewpoint misalignment. Through extensive evaluation on the PSCD, ChangeSim, and PASLCD datasets, we demonstrate that our approach achieves superior and robust performance. Our code will be released at https://github.com/ZilingLiu/GeoSCD.

Method: A novel probabilistic model that incorporates uncertainty estimation by modeling depth predictions as explicit probability distributions, using off-the-shelf pre-trained monocular depth models instead of per-environment-trained networks.

Result: Achieved 2.7x higher localization recall on long sequences (100 frames) and 16.7x higher on short sequences (15 frames) than state-of-the-art on the challenging LaMAR HGE dataset, with significant improvements in accuracy and robustness.

Conclusion: UnLoc provides an efficient and generalizable solution for sequential camera localization that eliminates the need for environment-specific training while incorporating crucial uncertainty modeling, demonstrating superior performance across both synthetic and real-world datasets.

Abstract: We propose UnLoc, an efficient data-driven solution for sequential camera localization within floorplans. Floorplan data is readily available, long-term persistent, and robust to changes in visual appearance. We address key limitations of recent methods, such as the lack of uncertainty modeling in depth predictions and the necessity for custom depth networks trained for each environment. We introduce a novel probabilistic model that incorporates uncertainty estimation, modeling depth predictions as explicit probability distributions. By leveraging off-the-shelf pre-trained monocular depth models, we eliminate the need to rely on per-environment-trained depth networks, enhancing generalization to unseen spaces. We evaluate UnLoc on large-scale synthetic and real-world datasets, demonstrating significant improvements over existing methods in terms of accuracy and robustness. Notably, we achieve $2.7$ times higher localization recall on long sequences (100 frames) and $16.7$ times higher on short ones (15 frames) than the state of the art on the challenging LaMAR HGE dataset.

Method: Proposes Semantic-Independent KalmanNet (SIKNet) with a Semantic-Independent Encoder that uses 1D convolution along homogeneous-semantic elements and fully-connected layers with nonlinear activation for heterogeneous-semantic elements.

Result: Experimental results on a large-scale semi-simulated dataset show SIKNet outperforms traditional Kalman filters and achieves superior robustness and accuracy compared to existing learning-aided filters.

Conclusion: SIKNet provides an effective learning-based solution for motion estimation in MOT, demonstrating improved performance over traditional and existing learning methods, with code publicly available.

Abstract: Motion estimation is a crucial component in multi-object tracking (MOT). It predicts the trajectory of objects by analyzing the changes in their positions in consecutive frames of images, reducing tracking failures and identity switches. The Kalman filter (KF) based on the linear constant-velocity model is one of the most commonly used methods in MOT. However, it may yield unsatisfactory results when KF’s parameters are mismatched and objects move in non-stationary. In this work, we utilize the learning-aided filter to handle the motion estimation of MOT. In particular, we propose a novel method named Semantic-Independent KalmanNet (SIKNet), which encodes the state vector (the input feature) using a Semantic-Independent Encoder (SIE) by two steps. First, the SIE uses a 1D convolution with a kernel size of 1, which convolves along the dimension of homogeneous-semantic elements across different state vectors to encode independent semantic information. Then it employs a fully-connected layer and a nonlinear activation layer to encode nonlinear and cross-dependency information between heterogeneous-semantic elements. To independently evaluate the performance of the motion estimation module in MOT, we constructed a large-scale semi-simulated dataset from several open-source MOT datasets. Experimental results demonstrate that the proposed SIKNet outperforms the traditional KF and achieves superior robustness and accuracy than existing learning-aided filters. The code is available at (https://github.com/SongJgit/filternet and https://github.com/SongJgit/TBDTracker).

Method: The research investigates transformers for pixel- and image-wise medical vision tasks, then pioneers MLP-based visual models to capture fine-grained long-range visual dependency in medical images through extensive experiments.

Result: Extensive experiments confirm the critical role of long-range dependency modeling and reveal that MLPs can model finer-grained long-range dependency among higher-resolution medical features containing enriched anatomical/pathological details.

Conclusion: MLPs establish themselves as a superior paradigm over transformers/CNNs, consistently enhancing performance across various medical vision tasks and paving the way for next-generation medical vision backbones.

Abstract: Medical Image Computing (MIC) is a broad research topic covering both pixel-wise (e.g., segmentation, registration) and image-wise (e.g., classification, regression) vision tasks. Effective analysis demands models that capture both global long-range context and local subtle visual characteristics, necessitating fine-grained long-range visual dependency modeling. Compared to Convolutional Neural Networks (CNNs) that are limited by intrinsic locality, transformers excel at long-range modeling; however, due to the high computational loads of self-attention, transformers typically cannot process high-resolution features (e.g., full-scale image features before downsampling or patch embedding) and thus face difficulties in modeling fine-grained dependency among subtle medical image details. Concurrently, Multi-layer Perceptron (MLP)-based visual models are recognized as computation/memory-efficient alternatives in modeling long-range visual dependency but have yet to be widely investigated in the MIC community. This doctoral research advances deep learning-based MIC by investigating effective long-range visual dependency modeling. It first presents innovative use of transformers for both pixel- and image-wise medical vision tasks. The focus then shifts to MLPs, pioneeringly developing MLP-based visual models to capture fine-grained long-range visual dependency in medical images. Extensive experiments confirm the critical role of long-range dependency modeling in MIC and reveal a key finding: MLPs provide feasibility in modeling finer-grained long-range dependency among higher-resolution medical features containing enriched anatomical/pathological details. This finding establishes MLPs as a superior paradigm over transformers/CNNs, consistently enhancing performance across various medical vision tasks and paving the way for next-generation medical vision backbones.

Method: Developed a dual-band thermal image formation model using two thermal cameras with different spectral sensitivities. Created algorithms to estimate surface emissivity and time-varying temperature while isolating dynamic background components.

Result: Quantitatively evaluated using carefully calibrated emissivities for various materials. Showed qualitative results on complex everyday scenes including glass with hot liquid and people moving in background. Successfully separated reflected and emitted light components.

Conclusion: The introduced method provides the first effective solution for separating reflected and emitted light components in thermal imaging under near-ambient conditions, enabling more accurate analysis of material properties and thermal behavior in dynamic environments.

Abstract: Long-wave infrared radiation captured by a thermal camera consists of two components: (a) light from the environment reflected or transmitted by a surface, and (b) light emitted by the surface after undergoing heat transport through the object and exchanging heat with the surrounding environment. Separating these components is essential for understanding object properties such as emissivity, temperature, reflectance and shape. Previous thermography studies often assume that only one component is dominant (e.g., in welding) or that the second component is constant and can be subtracted. However, in near-ambient conditions, which are most relevant to computer vision applications, both components are typically comparable in magnitude and vary over time. We introduce the first method that separates reflected and emitted components of light in videos captured by two thermal cameras with different spectral sensitivities. We derive a dual-band thermal image formation model and develop algorithms to estimate the surface’s emissivity and its time-varying temperature while isolating a dynamic background. We quantitatively evaluate our approach using carefully calibrated emissivities for a range of materials and show qualitative results on complex everyday scenes, such as a glass filled with hot liquid and people moving in the background.

Method: Conducted extensive ablation studies on SSL with varying view diversity, using Earth Mover’s Distance (EMD) to measure mutual information between views and validate findings across diverse data sources.

Result: SSL remains effective without strict instance consistency; moderate view diversity (achieved through zero overlapping or smaller crop scales) improves classification and dense prediction performance, while excessive diversity reduces effectiveness.

Conclusion: There’s an optimal range for view diversity in SSL, with moderate EMD values correlating with improved learning, providing valuable insights for future SSL framework design on diverse data types.

Abstract: Self-supervised learning (SSL) conventionally relies on the instance consistency paradigm, assuming that different views of the same image can be treated as positive pairs. However, this assumption breaks down for non-iconic data, where different views may contain distinct objects or semantic information. In this paper, we investigate the effectiveness of SSL when instance consistency is not guaranteed. Through extensive ablation studies, we demonstrate that SSL can still learn meaningful representations even when positive pairs lack strict instance consistency. Furthermore, our analysis further reveals that increasing view diversity, by enforcing zero overlapping or using smaller crop scales, can enhance downstream performance on classification and dense prediction tasks. However, excessive diversity is found to reduce effectiveness, suggesting an optimal range for view diversity. To quantify this, we adopt the Earth Mover’s Distance (EMD) as an estimator to measure mutual information between views, finding that moderate EMD values correlate with improved SSL learning, providing insights for future SSL framework design. We validate our findings across a range of settings, highlighting their robustness and applicability on diverse data sources.

Method: Two complementary regularization strategies: 1) auxiliary loss enforcing feature consistency between original and low-frequency filtered inputs to reduce texture dependence, 2) supervised contrastive learning to structure feature space around class-consistent shape representations.

Result: Both methods improved corruption robustness on CIFAR-10-C benchmark without degrading clean accuracy.

Conclusion: Loss-level regularization can effectively steer CNNs toward more shape-aware and resilient representations, making them more robust to common corruptions.

Abstract: Convolutional Neural Networks (CNNs) excel at image classification but remain vulnerable to common corruptions that humans handle with ease. A key reason for this fragility is their reliance on local texture cues rather than global object shapes – a stark contrast to human perception. To address this, we propose two complementary regularization strategies designed to encourage shape-biased representations and enhance robustness. The first introduces an auxiliary loss that enforces feature consistency between original and low-frequency filtered inputs, discouraging dependence on high-frequency textures. The second incorporates supervised contrastive learning to structure the feature space around class-consistent, shape-relevant representations. Evaluated on the CIFAR-10-C benchmark, both methods improve corruption robustness without degrading clean accuracy. Our results suggest that loss-level regularization can effectively steer CNNs toward more shape-aware, resilient representations.

Method: Proposes GLaVE-Cap with two core modules: TrackFusion for comprehensive local caption generation using vision experts and cross-frame visual prompts, and CaptionBridge for local-global interaction using global context to guide local captioning and adaptive summarization.

Result: Achieves state-of-the-art performance on four benchmarks, with ablation studies validating module effectiveness. Also creates GLaVE-Bench (5X more queries per video) and GLaVE-1.2M dataset (16K fine-grained captions + 1.2M QA pairs).

Conclusion: GLaVE-Cap effectively addresses the limitations of current video captioning approaches and contributes valuable resources (benchmark and dataset) to the video understanding community, with plans to open-source all components.

Abstract: Video detailed captioning aims to generate comprehensive video descriptions to facilitate video understanding. Recently, most efforts in the video detailed captioning community have been made towards a local-to-global paradigm, which first generates local captions from video clips and then summarizes them into a global caption. However, we find this paradigm leads to less detailed and contextual-inconsistent captions, which can be attributed to (1) no mechanism to ensure fine-grained captions, and (2) weak interaction between local and global captions. To remedy the above two issues, we propose GLaVE-Cap, a Global-Local aligned framework with Vision Expert integration for Captioning, which consists of two core modules: TrackFusion enables comprehensive local caption generation, by leveraging vision experts to acquire cross-frame visual prompts, coupled with a dual-stream structure; while CaptionBridge establishes a local-global interaction, by using global context to guide local captioning, and adaptively summarizing local captions into a coherent global caption. Besides, we construct GLaVE-Bench, a comprehensive video captioning benchmark featuring 5X more queries per video than existing benchmarks, covering diverse visual dimensions to facilitate reliable evaluation. We further provide a training dataset GLaVE-1.2M containing 16K high-quality fine-grained video captions and 1.2M related question-answer pairs. Extensive experiments on four benchmarks show that our GLaVE-Cap achieves state-of-the-art performance. Besides, the ablation studies and student model analyses further validate the effectiveness of the proposed modules and the contribution of GLaVE-1.2M to the video understanding community. The source code, model weights, benchmark, and dataset will be open-sourced.

Method: Developed a compact 3D skin reconstruction probe based on GelSight tactile imaging with custom elastic gel and learning-based reconstruction algorithm, integrated into a handheld probe with force sensing for consistent contact.

Result: Achieved mean absolute error of 12.55 microns on wrinkle-like test objects. Validated wrinkle depth metrics across multiple body regions in 15 participants, and demonstrated statistically significant wrinkle height reduction after moisturizer application.

Conclusion: Provides a validated tool for clinical and cosmetic skin analysis with applications in diagnosis, treatment monitoring, and skincare efficacy evaluation.

Abstract: Three-dimensional (3-D) skin surface reconstruction offers promise for objective and quantitative dermatological assessment, but no portable, high-resolution device exists that has been validated and used for depth reconstruction across various body locations. We present a compact 3-D skin reconstruction probe based on GelSight tactile imaging with a custom elastic gel and a learning-based reconstruction algorithm for micron-level wrinkle height estimation. Our probe, integrated into a handheld probe with force sensing for consistent contact, achieves a mean absolute error of 12.55 micron on wrinkle-like test objects. In a study with 15 participants without skin disorders, we provide the first validated wrinkle depth metrics across multiple body regions. We further demonstrate statistically significant reductions in wrinkle height at three locations following over-the-counter moisturizer application. Our work offers a validated tool for clinical and cosmetic skin analysis, with potential applications in diagnosis, treatment monitoring, and skincare efficacy evaluation.

Method: Proposes a mixture of experts architecture that dynamically selects contextually relevant A matrices (forget-gates) based on input features, enhancing representational capacity without compromising computational efficiency.

Result: MixANT consistently outperforms state-of-the-art methods on 50Salads, Breakfast, and Assembly101 datasets across all evaluation settings.

Conclusion: Input-dependent forget-gate mechanisms are essential for reliable prediction of human behavior in diverse real-world scenarios, and MixANT effectively addresses this need.

Abstract: We present MixANT, a novel architecture for stochastic long-term dense anticipation of human activities. While recent State Space Models (SSMs) like Mamba have shown promise through input-dependent selectivity on three key parameters, the critical forget-gate ($\textbf{A}$ matrix) controlling temporal memory remains static. We address this limitation by introducing a mixture of experts approach that dynamically selects contextually relevant $\textbf{A}$ matrices based on input features, enhancing representational capacity without sacrificing computational efficiency. Extensive experiments on the 50Salads, Breakfast, and Assembly101 datasets demonstrate that MixANT consistently outperforms state-of-the-art methods across all evaluation settings. Our results highlight the importance of input-dependent forget-gate mechanisms for reliable prediction of human behavior in diverse real-world scenarios.

Method: A hypernetwork learns to predict parameters of task-specific classification models adapted to available modalities, allowing training on all samples regardless of completeness.

Result: Outperforms state-of-the-art approaches with up to 8% absolute accuracy increase when trained on datasets with 25% completeness (75% missing modalities).

Conclusion: Provides an efficient solution for real-world multi-modal medical data analysis by enabling a single model to generalize across all modality configurations.

Abstract: In real world clinical environments, training and applying deep learning models on multi-modal medical imaging data often struggles with partially incomplete data. Standard approaches either discard missing samples, require imputation or repurpose dropout learning schemes, limiting robustness and generalizability. To address this, we propose a hypernetwork-based method that dynamically generates task-specific classification models conditioned on the set of available modalities. Instead of training a fixed model, a hypernetwork learns to predict the parameters of a task model adapted to available modalities, enabling training and inference on all samples, regardless of completeness. We compare this approach with (1) models trained only on complete data, (2) state of the art channel dropout methods, and (3) an imputation-based method, using artificially incomplete datasets to systematically analyze robustness to missing modalities. Results demonstrate superior adaptability of our method, outperforming state of the art approaches with an absolute increase in accuracy of up to 8% when trained on a dataset with 25% completeness (75% of training data with missing modalities). By enabling a single model to generalize across all modality configurations, our approach provides an efficient solution for real-world multi-modal medical data analysis.

Method: Uses weighted rotation blending approach with quaternion averaging to properly handle Gaussian rotations during deformation, modifying only the linear blend skinning technique without requiring additional models.

Result: Enables efficient animation of vertex-based Gaussians that can be integrated into any engine using standard Gaussian rasterizers, providing simpler and more effective deformation.

Conclusion: The proposed quaternion-based rotation blending method effectively addresses Gaussian deformation artifacts while maintaining simplicity and compatibility with existing rendering pipelines.

Abstract: Radiance field-based methods have recently been used to reconstruct human avatars, showing that we can significantly downscale the systems needed for creating animated human avatars. Although this progress has been initiated by neural radiance fields, their slow rendering and backward mapping from the observation space to the canonical space have been the main challenges. With Gaussian splatting overcoming both challenges, a new family of approaches has emerged that are faster to train and render, while also straightforward to implement using forward skinning from the canonical to the observation space. However, the linear blend skinning required for the deformation of the Gaussians does not provide valid results for their non-linear rotation properties. To address such artifacts, recent works use mesh properties to rotate the non-linear Gaussian properties or train models to predict corrective offsets. Instead, we propose a weighted rotation blending approach that leverages quaternion averaging. This leads to simpler vertex-based Gaussians that can be efficiently animated and integrated in any engine by only modifying the linear blend skinning technique, and using any Gaussian rasterizer.

Method: The paper introduces an active learning approach that performs post-hoc rotation of the data latent space to disentangle biological factors from technical variations, creating stable clusters across different acquisition settings.

Result: The method improves cluster consistency by +19.01% (ARI), +16.85% (NMI), and +12.39% (Dice) compared to entangled representations, outperforming four state-of-the-art harmonization methods. It also enhances Cox survival prediction when quantifying tissue composition in pulmonary fibrosis patients.

Conclusion: The proposed label-free framework successfully facilitates biomarker discovery in multi-center routine imaging data by effectively disentangling biological and technical factors through latent space rotation.

Abstract: Identifying new disease-related patterns in medical imaging data with the help of machine learning enlarges the vocabulary of recognizable findings. This supports diagnostic and prognostic assessment. However, image appearance varies not only due to biological differences, but also due to imaging technology linked to vendors, scanning- or re- construction parameters. The resulting domain shifts impedes data representation learning strategies and the discovery of biologically meaningful cluster appearances. To address these challenges, we introduce an approach to actively learn the domain shift via post-hoc rotation of the data latent space, enabling disentanglement of biological and technical factors. Results on real-world heterogeneous clinical data showcase that the learned disentangled representation leads to stable clusters representing tissue-types across different acquisition settings. Cluster consistency is improved by +19.01% (ARI), +16.85% (NMI), and +12.39% (Dice) compared to the entangled representation, outperforming four state-of-the-art harmonization methods. When using the clusters to quantify tissue composition on idiopathic pulmonary fibrosis patients, the learned profiles enhance Cox survival prediction. This indicates that the proposed label-free framework facilitates biomarker discovery in multi-center routine imaging data. Code is available on GitHub https://github.com/cirmuw/latent-space-rotation-disentanglement.

Method: Developed a masked autoencoder (MAE) paradigm inspired by MultiMAE, treating each MRI sequence as separate modality. Uses late-fusion transformer encoder to integrate multi-sequence information and individual decoder streams for multi-task reconstruction per modality.

Result: Achieved absolute improvement of 10.1 overall Dice score and 0.46 MCC over MAE-ViT baselines with missing input sequences in downstream segmentation and classification tasks. Demonstrated robust performance and ability to infer missing sequences.

Conclusion: The method provides a flexible and generalizable encoder for brain MRIs that can handle missing inputs through cross-sequence reasoning and can be adapted to various downstream applications, showing strong pretraining strategy effectiveness.

Abstract: Missing input sequences are common in medical imaging data, posing a challenge for deep learning models reliant on complete input data. In this work, inspired by MultiMAE [2], we develop a masked autoencoder (MAE) paradigm for multi-modal, multi-task learning in 3D medical imaging with brain MRIs. Our method treats each MRI sequence as a separate input modality, leveraging a late-fusion-style transformer encoder to integrate multi-sequence information (multi-modal) and individual decoder streams for each modality for multi-task reconstruction. This pretraining strategy guides the model to learn rich representations per modality while also equipping it to handle missing inputs through cross-sequence reasoning. The result is a flexible and generalizable encoder for brain MRIs that infers missing sequences from available inputs and can be adapted to various downstream applications. We demonstrate the performance and robustness of our method against an MAE-ViT baseline in downstream segmentation and classification tasks, showing absolute improvement of $10.1$ overall Dice score and $0.46$ MCC over the baselines with missing input sequences. Our experiments demonstrate the strength of this pretraining strategy. The implementation is made available.

Method: Proposes trajectory-based paradigm with TrajTrack framework that generates explicit motion proposals and uses implicit motion modeling to predict future trajectories from historical bounding box data only, without additional point cloud inputs.

Result: Achieves 4.48% improvement in tracking precision over strong baseline on NuScenes benchmark while running at 56 FPS, demonstrating strong generalizability across different base trackers.

Conclusion: TrajTrack resolves the efficiency-robustness dilemma in 3D SOT by leveraging historical trajectory information, providing both computational efficiency and improved tracking performance.

Abstract: LiDAR-based 3D single object tracking (3D SOT) is a critical task in robotics and autonomous systems. Existing methods typically follow frame-wise motion estimation or a sequence-based paradigm. However, the two-frame methods are efficient but lack long-term temporal context, making them vulnerable in sparse or occluded scenes, while sequence-based methods that process multiple point clouds gain robustness at a significant computational cost. To resolve this dilemma, we propose a novel trajectory-based paradigm and its instantiation, TrajTrack. TrajTrack is a lightweight framework that enhances a base two-frame tracker by implicitly learning motion continuity from historical bounding box trajectories alone-without requiring additional, costly point cloud inputs. It first generates a fast, explicit motion proposal and then uses an implicit motion modeling module to predict the future trajectory, which in turn refines and corrects the initial proposal. Extensive experiments on the large-scale NuScenes benchmark show that TrajTrack achieves new state-of-the-art performance, dramatically improving tracking precision by 4.48% over a strong baseline while running at 56 FPS. Besides, we also demonstrate the strong generalizability of TrajTrack across different base trackers. Video is available at https://www.bilibili.com/video/BV1ahYgzmEWP.

Method: Proposes FusionNet with modality-aware attention mechanism to dynamically adjust feature contributions, pixel-wise alpha blending for fine-grained fusion, and target-aware loss using weak ROI supervision to maintain semantic consistency.

Result: Experiments on M3FD dataset show FusionNet generates fused images with enhanced semantic preservation, high perceptual quality, and clear interpretability, benefiting downstream tasks like object detection.

Conclusion: FusionNet provides a general and extensible solution for semantic-aware multi-modal image fusion with improved interpretability and performance for practical applications.

Abstract: Infrared and visible image fusion (IVIF) is a fundamental task in multi-modal perception that aims to integrate complementary structural and textural cues from different spectral domains. In this paper, we propose FusionNet, a novel end-to-end fusion framework that explicitly models inter-modality interaction and enhances task-critical regions. FusionNet introduces a modality-aware attention mechanism that dynamically adjusts the contribution of infrared and visible features based on their discriminative capacity. To achieve fine-grained, interpretable fusion, we further incorporate a pixel-wise alpha blending module, which learns spatially-varying fusion weights in an adaptive and content-aware manner. Moreover, we formulate a target-aware loss that leverages weak ROI supervision to preserve semantic consistency in regions containing important objects (e.g., pedestrians, vehicles). Experiments on the public M3FD dataset demonstrate that FusionNet generates fused images with enhanced semantic preservation, high perceptual quality, and clear interpretability. Our framework provides a general and extensible solution for semantic-aware multi-modal image fusion, with benefits for downstream tasks such as object detection and scene understanding.

Method: MHIM-MIL uses a Siamese structure with consistency constraint, a momentum teacher to mask salient instances and implicitly mine hard instances, large-scale random masking for diverse hard instances, and a global recycle network to prevent loss of key features. The student updates teacher via exponential moving average.

Result: Experimental results on cancer diagnosis, subtyping, survival analysis tasks across 12 benchmarks demonstrate that MHIM-MIL outperforms the latest methods in both performance and efficiency.

Conclusion: The proposed MHIM-MIL framework effectively addresses the limitation of existing MIL methods by mining hard instances, leading to improved performance in computational pathology tasks while maintaining efficiency.

Abstract: Digitizing pathological images into gigapixel Whole Slide Images (WSIs) has opened new avenues for Computational Pathology (CPath). As positive tissue comprises only a small fraction of gigapixel WSIs, existing Multiple Instance Learning (MIL) methods typically focus on identifying salient instances via attention mechanisms. However, this leads to a bias towards easy-to-classify instances while neglecting challenging ones. Recent studies have shown that hard examples are crucial for accurately modeling discriminative boundaries. Applying such an idea at the instance level, we elaborate a novel MIL framework with masked hard instance mining (MHIM-MIL), which utilizes a Siamese structure with a consistency constraint to explore the hard instances. Using a class-aware instance probability, MHIM-MIL employs a momentum teacher to mask salient instances and implicitly mine hard instances for training the student model. To obtain diverse, non-redundant hard instances, we adopt large-scale random masking while utilizing a global recycle network to mitigate the risk of losing key features. Furthermore, the student updates the teacher using an exponential moving average, which identifies new hard instances for subsequent training iterations and stabilizes optimization. Experimental results on cancer diagnosis, subtyping, survival analysis tasks, and 12 benchmarks demonstrate that MHIM-MIL outperforms the latest methods in both performance and efficiency. The code is available at: https://github.com/DearCaat/MHIM-MIL.

Method: Proposed Semantic and Frequency Guided Network (SFGNet) with Multi-Band Fourier Module (MBFM) to handle complex backgrounds and blurred boundaries, and Interactive Structure Enhancement Block (ISEB) to ensure structural integrity and boundary details.

Result: Extensive experiments on three COD benchmark datasets demonstrate that SFGNet significantly outperforms state-of-the-art approaches.

Conclusion: The integration of semantic prompts and frequency-domain features effectively improves camouflaged object detection performance, particularly in handling complex backgrounds and preserving boundary details.

Abstract: Camouflaged object detection (COD) aims to segment objects that blend into their surroundings. However, most existing studies overlook the semantic differences among textual prompts of different targets as well as fine-grained frequency features. In this work, we propose a novel Semantic and Frequency Guided Network (SFGNet), which incorporates semantic prompts and frequency-domain features to capture camouflaged objects and improve boundary perception. We further design Multi-Band Fourier Module(MBFM) to enhance the ability of the network in handling complex backgrounds and blurred boundaries. In addition, we design an Interactive Structure Enhancement Block (ISEB) to ensure structural integrity and boundary details in the predictions. Extensive experiments conducted on three COD benchmark datasets demonstrate that our method significantly outperforms state-of-the-art approaches. The core code of the model is available at the following link: https://github.com/winter794444/SFGNetICASSP2026.

Method: Three zero-shot auxiliary reasoning methods that provide explicit spatial cues (axes, grids, labeled intersections) as part of input images to help VLMs articulate their implicit spatial understanding.

Result: Substantial performance improvement on four GUI grounding benchmarks across seven open-source and proprietary VLMs, demonstrating effective GUI grounding without fine-tuning.

Conclusion: Zero-shot auxiliary reasoning with spatial cues effectively bridges the gap between VLMs’ latent grounding potential and explicit coordinate output capabilities for GUI grounding tasks.

Abstract: Graphical user interface (GUI) grounding is a fundamental task for building GUI agents. However, general vision-language models (VLMs) struggle with this task due to a lack of specific optimization. We identify a key gap in this paper: while VLMs exhibit significant latent grounding potential, as demonstrated by their performance measured by Pointing Game, they underperform when tasked with outputting explicit coordinates. To address this discrepancy, and bypass the high data and annotation costs of current fine-tuning approaches, we propose three zero-shot auxiliary reasoning methods. By providing explicit spatial cues such as axes, grids and labeled intersections as part of the input image, these methods enable VLMs to articulate their implicit spatial understanding capabilities. We evaluate these methods on four GUI grounding benchmarks across seven open-source and proprietary VLMs. The evaluation results demonstrate that the proposed methods substantially improve the performance of GUI grounding.

Method: Proposes a Gaussian-SDF hybrid representation: uses colorized Signed Distance Field (SDF) for smooth geometry and appearance (efficiently constructed via RGB-D fusion), combined with 3D Gaussians to capture underrepresented details. Enables 50% fewer Gaussians and 75% fewer optimization iterations through targeted refinement.

Result: Achieves over 150 fps on real-world Azure Kinect sequences - an order-of-magnitude speedup over state-of-the-art techniques while maintaining comparable reconstruction quality.

Conclusion: The hybrid representation successfully addresses computational bottlenecks in Gaussian SLAM, enabling real-time performance without sacrificing reconstruction quality, with plans to release source code and data for future research.

Abstract: While recent Gaussian-based SLAM methods achieve photorealistic reconstruction from RGB-D data, their computational performance remains a critical bottleneck. State-of-the-art techniques operate at less than 20 fps, significantly lagging behind geometry-centric approaches like KinectFusion (hundreds of fps). This limitation stems from the heavy computational burden: modeling scenes requires numerous Gaussians and complex iterative optimization to fit RGB-D data, where insufficient Gaussian counts or optimization iterations cause severe quality degradation. To address this, we propose a Gaussian-SDF hybrid representation, combining a colorized Signed Distance Field (SDF) for smooth geometry and appearance with 3D Gaussians to capture underrepresented details. The SDF is efficiently constructed via RGB-D fusion (as in geometry-centric methods), while Gaussians undergo iterative optimization. Our representation enables drastic Gaussian reduction (50% fewer) by avoiding full-scene Gaussian modeling, and efficient Gaussian optimization (75% fewer iterations) through targeted appearance refinement. Building upon this representation, we develop GPS-SLAM (Gaussian-Plus-SDF SLAM), a real-time 3D reconstruction system achieving over 150 fps on real-world Azure Kinect sequences – delivering an order-of-magnitude speedup over state-of-the-art techniques while maintaining comparable reconstruction quality. We will release the source code and data to facilitate future research.

Method: HIL framework with secondary clustering to generate multiple memories per cluster, MCCL for representation refinement, and BRST mechanism for reliable cross-modal correspondences through bidirectional pseudo-label matching.

Result: Extensive experiments on SYSU-MM01 and RegDB datasets show the proposed method outperforms existing approaches.

Conclusion: The hierarchical approach with multi-center representation and bidirectional matching effectively addresses fine-grained variations and improves cross-modal matching in unsupervised VI-ReID.

Abstract: Unsupervised visible-infrared person re-identification (USVI-ReID) aims to learn modality-invariant image features from unlabeled cross-modal person datasets by reducing the modality gap while minimizing reliance on costly manual annotations. Existing methods typically address USVI-ReID using cluster-based contrastive learning, which represents a person by a single cluster center. However, they primarily focus on the commonality of images within each cluster while neglecting the finer-grained differences among them. To address the limitation, we propose a Hierarchical Identity Learning (HIL) framework. Since each cluster may contain several smaller sub-clusters that reflect fine-grained variations among images, we generate multiple memories for each existing coarse-grained cluster via a secondary clustering. Additionally, we propose Multi-Center Contrastive Learning (MCCL) to refine representations for enhancing intra-modal clustering and minimizing cross-modal discrepancies. To further improve cross-modal matching quality, we design a Bidirectional Reverse Selection Transmission (BRST) mechanism, which establishes reliable cross-modal correspondences by performing bidirectional matching of pseudo-labels. Extensive experiments conducted on the SYSU-MM01 and RegDB datasets demonstrate that the proposed method outperforms existing approaches. The source code is available at: https://github.com/haonanshi0125/HIL.

Method: Iterative, data-centric framework that identifies Pareto optimized class distributions, determines optimal number of images per class to minimize bias and variance, and integrates with existing training pipelines.

Result: Validated on EfficientNet, ResNet and ConvNeXt using CIFAR-10 and iNaturalist21 datasets, demonstrating improved and balanced model performance through optimized data distribution.

Conclusion: BiCDO provides an effective approach for bias-controlled class distribution optimization that enhances multi-class classification performance with minimal implementation overhead.

Abstract: We propose BiCDO (Bias-Controlled Class Distribution Optimizer), an iterative, data-centric framework that identifies Pareto optimized class distributions for multi-class image classification. BiCDO enables performance prioritization for specific classes, which is useful in safety-critical scenarios (e.g. prioritizing ‘Human’ over ‘Dog’). Unlike uniform distributions, BiCDO determines the optimal number of images per class to enhance reliability and minimize bias and variance in the objective function. BiCDO can be incorporated into existing training pipelines with minimal code changes and supports any labelled multi-class dataset. We have validated BiCDO using EfficientNet, ResNet and ConvNeXt on CIFAR-10 and iNaturalist21 datasets, demonstrating improved, balanced model performance through optimized data distribution.

Method: Created MVQA-68K dataset with over 68,000 videos annotated across seven quality dimensions (aesthetics, camera movement, dynamic degree, texture detail, composition, visual quality, factual consistency) with chain-of-thought reasoning. Used this dataset to train multimodal large language models for VQA tasks.

Result: Achieved state-of-the-art results on internal test set and public benchmarks (LSVQ-test, LSVQ-1080p, LIVE-VQC). Incorporating explicit reasoning process during training substantially boosted zero-shot generalization capabilities.

Conclusion: MVQA-68K provides a comprehensive and interpretable framework for video quality assessment that significantly enhances model performance and generalization, addressing limitations of traditional single-score VQA methods.

Abstract: With the rapid advancement of video generation models such as Sora, video quality assessment (VQA) is becoming increasingly crucial for selecting high-quality videos from large-scale datasets used in pre-training. Traditional VQA methods, typically producing single numerical scores, often lack comprehensiveness and interpretability. To address these challenges, we introduce MVQA-68K, a novel multi-dimensional VQA dataset comprising over 68,000 carefully annotated videos, covering seven essential quality dimensions: overall aesthetics, camera movement, dynamic degree, texture detail, composition, visual quality, and factual consistency. Each annotation includes detailed chain-of-thought reasoning to facilitate interpretability and comprehensive understanding. Extensive experiments demonstrate that MVQA-68K significantly enhances the performance of various multimodal large language models (MLLMs) on the VQA task, achieving state-of-the-art results not only on our internal test set (Fig.1) but also on public benchmarks including LSVQ-test, LSVQ-1080p, and LIVE-VQC. Meantime, incorporating explicit reasoning process during VQA training substantially boosts the zero-shot generalization. Code and dataset will be available at github: https://github.com/Controller01-ai/MVQA-68K

Method: HyGDL uses a single encoder and analytically defines style as the component orthogonal to style-invariant content via vector projection. It follows the Invariance Pre-training Principle by systematically varying style biases while keeping supervision constant to force learning of invariant essence.

Result: The framework achieves explicit content-style disentanglement, addressing shortcut learning at its core rather than just at surface level.

Conclusion: HyGDL provides a fundamental solution to shortcut learning in SSL by altering the learning mechanism itself through hybrid generative-discriminative approach and invariance-based pre-training.

Abstract: Despite the remarkable success of Self-Supervised Learning (SSL), its generalization is fundamentally hindered by Shortcut Learning, where models exploit superficial features like texture instead of intrinsic structure. We experimentally verify this flaw within the generative paradigm (e.g., MAE) and argue it is a systemic issue also affecting discriminative methods, identifying it as the root cause of their failure on unseen domains. While existing methods often tackle this at a surface level by aligning or separating domain-specific features, they fail to alter the underlying learning mechanism that fosters shortcut dependency. To address this at its core, we propose HyGDL (Hybrid Generative-Discriminative Learning Framework), a hybrid framework that achieves explicit content-style disentanglement. Our approach is guided by the Invariance Pre-training Principle: forcing a model to learn an invariant essence by systematically varying a bias (e.g., style) at the input while keeping the supervision signal constant. HyGDL operates on a single encoder and analytically defines style as the component of a representation that is orthogonal to its style-invariant content, derived via vector projection.

Method: Proposes a softmax-based frame allocation strategy that focuses on anomaly-dense segments while ensuring full-video coverage, enabling balanced sampling across temporal scales. Constructs two benchmarks: image-based for frame-level reasoning and video-based for temporally localized segments with abnormality scoring.

Result: Experiments on UCF-Crime dataset show improvements at both frame and video levels. Ablation studies confirm clear advantages of anomaly-focused sampling over uniform and random baselines.

Conclusion: The proposed frame allocation strategy and complementary benchmarks provide a more comprehensive evaluation framework for video anomaly detection, addressing limitations of existing approaches and demonstrating superior performance through anomaly-focused sampling.

Abstract: Video Anomaly Detection (VAD) is critical for surveillance and public safety. However, existing benchmarks are limited to either frame-level or video-level tasks, restricting a holistic view of model generalization. This work first introduces a softmax-based frame allocation strategy that prioritizes anomaly-dense segments while maintaining full-video coverage, enabling balanced sampling across temporal scales. Building on this process, we construct two complementary benchmarks. The image-based benchmark evaluates frame-level reasoning with representative frames, while the video-based benchmark extends to temporally localized segments and incorporates an abnormality scoring task.Experiments on UCF-Crime demonstrate improvements at both the frame and video levels, and ablation studies confirm clear advantages of anomaly-focused sampling over uniform and random baselines.

Method: Combines parametric head model with dynamic Gaussian representations, separates head and background Gaussians, uses pre-trained 2D guidance for optimization, and includes a repair module for extreme poses/expressions.

Result: Achieves comparable performance to state-of-the-art 2D methods in identity preservation and expression consistency, while significantly outperforming them in multi-view rendering quality and 3D consistency.

Conclusion: Bridges 3D modeling with deepfake synthesis, enabling scene-aware and immersive visual forgeries while revealing potential threats of 3D Gaussian Splatting for manipulation attacks.

Abstract: We propose a novel 3D deepfake generation framework based on 3D Gaussian Splatting that enables realistic, identity-preserving face swapping and reenactment in a fully controllable 3D space. Compared to conventional 2D deepfake approaches that suffer from geometric inconsistencies and limited generalization to novel view, our method combines a parametric head model with dynamic Gaussian representations to support multi-view consistent rendering, precise expression control, and seamless background integration. To address editing challenges in point-based representations, we explicitly separate the head and background Gaussians and use pre-trained 2D guidance to optimize the facial region across views. We further introduce a repair module to enhance visual consistency under extreme poses and expressions. Experiments on NeRSemble and additional evaluation videos demonstrate that our method achieves comparable performance to state-of-the-art 2D approaches in identity preservation, as well as pose and expression consistency, while significantly outperforming them in multi-view rendering quality and 3D consistency. Our approach bridges the gap between 3D modeling and deepfake synthesis, enabling new directions for scene-aware, controllable, and immersive visual forgeries, revealing the threat that emerging 3D Gaussian Splatting technique could be used for manipulation attacks.

Method: Proposes Initial Seed refined Diffusion Model (IS-Diff) that samples initial seeds from unmasked areas to imitate masked data distribution, plus a dynamic selective refinement mechanism to detect and adjust unharmonious inpaintings during the diffusion process.

Result: Validated on CelebA-HQ, ImageNet, and Places2 datasets for standard and large-mask inpainting tasks, showing effectiveness across all metrics compared to state-of-the-art methods.

Conclusion: IS-Diff provides a training-free approach that produces more harmonious and consistent inpainting results by leveraging distributional harmonious seeds from unmasked regions, setting a promising direction for diffusion-based inpainting.

Abstract: Diffusion models have shown promising results in free-form inpainting. Recent studies based on refined diffusion samplers or novel architectural designs led to realistic results and high data consistency. However, random initialization seed (noise) adopted in vanilla diffusion process may introduce mismatched semantic information in masked regions, leading to biased inpainting results, e.g., low consistency and low coherence with the other unmasked area. To address this issue, we propose the Initial Seed refined Diffusion Model (IS-Diff), a completely training-free approach incorporating distributional harmonious seeds to produce harmonious results. Specifically, IS-Diff employs initial seeds sampled from unmasked areas to imitate the masked data distribution, thereby setting a promising direction for the diffusion procedure. Moreover, a dynamic selective refinement mechanism is proposed to detect severe unharmonious inpaintings in intermediate latent and adjust the strength of our initialization prior dynamically. We validate our method on both standard and large-mask inpainting tasks using the CelebA-HQ, ImageNet, and Places2 datasets, demonstrating its effectiveness across all metrics compared to state-of-the-art inpainting methods.

Method: Created a real-world dataset with precisely aligned degraded-clean image pairs captured under various weather conditions (rain, snow, haze) across diverse outdoor scenes and illumination settings.

Result: Provides a comprehensive benchmark for evaluating task-specific, task-general, and all-in-one restoration methods. Enables supervised learning and rigorous evaluation in real-world scenarios.

Conclusion: The dataset offers a valuable foundation for advancing robust and practical all-in-one image restoration, addressing the critical need for real-world benchmarks in this field.

Abstract: Existing all-in-one image restoration approaches, which aim to handle multiple weather degradations within a single framework, are predominantly trained and evaluated using mixed single-weather synthetic datasets. However, these datasets often differ significantly in resolution, style, and domain characteristics, leading to substantial domain gaps that hinder the development and fair evaluation of unified models. Furthermore, the lack of a large-scale, real-world all-in-one weather restoration dataset remains a critical bottleneck in advancing this field. To address these limitations, we present a real-world all-in-one adverse weather image restoration benchmark dataset, which contains image pairs captured under various weather conditions, including rain, snow, and haze, as well as diverse outdoor scenes and illumination settings. The resulting dataset provides precisely aligned degraded and clean images, enabling supervised learning and rigorous evaluation. We conduct comprehensive experiments by benchmarking a variety of task-specific, task-general, and all-in-one restoration methods on our dataset. Our dataset offers a valuable foundation for advancing robust and practical all-in-one image restoration in real-world scenarios. The dataset has been publicly released and is available at https://github.com/guanqiyuan/WeatherBench.

Method: Proposed RVMamba architecture integrating multiple feature extraction modules with Mamba state-space model, and introduced FAZMamba with unified Joint-OCTAMamba framework to simultaneously improve FAZ segmentation and mitigate performance imbalance.

Result: Experimental results on OCTA-500 dataset demonstrate that Joint-OCTAMamba outperforms existing models across all evaluation metrics.

Conclusion: The proposed Mamba-based frameworks provide superior performance for OCTA image analysis, addressing accuracy limitations and task imbalance in retinal disease diagnosis and monitoring.

Abstract: OCTA is a crucial non-invasive imaging technique for diagnosing and monitoring retinal diseases like diabetic retinopathy, age-related macular degeneration, and glaucoma. Current 2D-based methods for retinal vessel (RV) segmentation offer insufficient accuracy. To address this, we propose RVMamba, a novel architecture integrating multiple feature extraction modules with the Mamba state-space model. Moreover, existing joint segmentation models for OCTA data exhibit performance imbalance between different tasks. To simultaneously improve the segmentation of the foveal avascular zone (FAZ) and mitigate this imbalance, we introduce FAZMamba and a unified Joint-OCTAMamba framework. Experimental results on the OCTA-500 dataset demonstrate that Joint-OCTAMamba outperforms existing models across evaluation metrics.The code is available at https://github.com/lc-sfis/Joint-OCTAMamba.

Method: DTGen uses generative diffusion models with LoRA for efficient domain specialization, structured prompts for diverse dirty image generation, and CLIP-based cross-modal filtering to ensure data quality.

Result: Under extremely limited real few-shot conditions, DTGen can synthesize unlimited high-quality samples, significantly improving classifier performance and enabling fine-grained dirty tableware recognition.

Conclusion: DTGen validates generative AI’s value in few-shot industrial vision and provides a feasible deployment path for automated tableware cleaning and food safety monitoring, with potential for lightweight deployment in embedded dishwashers.

Abstract: Intelligent tableware cleaning is a critical application in food safety and smart homes, but existing methods are limited by coarse-grained classification and scarcity of few-shot data, making it difficult to meet industrialization requirements. We propose DTGen, a few-shot data augmentation scheme based on generative diffusion models, specifically designed for fine-grained dirty tableware recognition. DTGen achieves efficient domain specialization through LoRA, generates diverse dirty images via structured prompts, and ensures data quality through CLIP-based cross-modal filtering. Under extremely limited real few-shot conditions, DTGen can synthesize virtually unlimited high-quality samples, significantly improving classifier performance and supporting fine-grained dirty tableware recognition. We further elaborate on lightweight deployment strategies, promising to transfer DTGen’s benefits to embedded dishwashers and integrate with cleaning programs to intelligently regulate energy consumption and detergent usage. Research results demonstrate that DTGen not only validates the value of generative AI in few-shot industrial vision but also provides a feasible deployment path for automated tableware cleaning and food safety monitoring.

Method: Combines georeferencing, U-Net-based binary segmentation, graph construction, and iterative refinement using a routing engine to extract trails from scanned maps.

Result: The approach robustly recovers trail networks from diverse map styles and generates GPS routes suitable for practical navigation use.

Conclusion: The proposed pipeline successfully enables the conversion of traditional paper map trails into digital GPS-navigable routes, bridging the gap between analog and digital navigation tools.

Abstract: Paper maps remain widely used for hiking and sightseeing because they contain curated trails and locally relevant annotations that are often missing from digital navigation applications such as Google Maps. We propose a pipeline to extract navigable trails from scanned maps, enabling their use in GPS-based navigation. Our method combines georeferencing, U-Net-based binary segmentation, graph construction, and an iterative refinement procedure using a routing engine. We evaluate the full end-to-end pipeline as well as individual components, showing that the approach can robustly recover trail networks from diverse map styles and generate GPS routes suitable for practical use.

Method: Created IMD dataset with 45 industrial components captured using RGB-D camera under natural indoor lighting and varied arrangements. Evaluated state-of-the-art models including XMem, SAM2 for segmentation, and BundleTrack, BundleSDF for pose estimation.

Result: The industrial dataset proved more challenging than existing household object datasets, showing performance limitations of current models in industrial contexts.

Conclusion: This benchmark provides a baseline for developing and comparing segmentation and pose estimation algorithms that can better generalize to industrial robotics scenarios.

Abstract: Object 6DoF (6D) pose estimation is essential for robotic perception, especially in industrial settings. It enables robots to interact with the environment and manipulate objects. However, existing benchmarks on object 6D pose estimation primarily use everyday objects with rich textures and low-reflectivity, limiting model generalization to industrial scenarios where objects are often metallic, texture-less, and highly reflective. To address this gap, we propose a novel dataset and benchmark namely \textit{Industrial Metallic Dataset (IMD)}, tailored for industrial applications. Our dataset comprises 45 true-to-scale industrial components, captured with an RGB-D camera under natural indoor lighting and varied object arrangements to replicate real-world conditions. The benchmark supports three tasks, including video object segmentation, 6D pose tracking, and one-shot 6D pose estimation. We evaluate existing state-of-the-art models, including XMem and SAM2 for segmentation, and BundleTrack and BundleSDF for pose estimation, to assess model performance in industrial contexts. Evaluation results show that our industrial dataset is more challenging than existing household object datasets. This benchmark provides the baseline for developing and comparing segmentation and pose estimation algorithms that better generalize to industrial robotics scenarios.

Method: Proposed Ensemble Distillation technique that distills knowledge from multiple ensemble models into a single model, reducing computational complexity while maintaining performance.

Result: Extensive experiments on DRIVE and FIVES datasets show Ensemble Distillation achieves comparable performance via calibration and segmentation metrics to traditional ensemble methods.

Conclusion: Ensemble Distillation provides an efficient and reliable approach for uncertainty estimation in retinal vessel segmentation, making it a promising tool for medical imaging applications with reduced computational requirements.

Abstract: Uncertainty estimation is critical for reliable medical image segmentation, particularly in retinal vessel analysis, where accurate predictions are essential for diagnostic applications. Deep Ensembles, where multiple networks are trained individually, are widely used to improve medical image segmentation performance. However, training and testing costs increase with the number of ensembles. In this work, we propose Ensemble Distillation as a robust alternative to commonly used uncertainty estimation techniques by distilling the knowledge of multiple ensemble models into a single model. Through extensive experiments on the DRIVE and FIVES datasets, we demonstrate that Ensemble Distillation achieves comparable performance via calibration and segmentation metrics, while significantly reducing computational complexity. These findings suggest that Ensemble distillation provides an efficient and reliable approach for uncertainty estimation in the segmentation of the retinal vessels, making it a promising tool for medical imaging applications.

Method: Systematic unsupervised search across different architectures and datasets to extract fundamental filter patterns, then testing networks initialized with these 8 frozen filters.

Result: Networks with just 8 unique frozen filters achieve over 80% ImageNet accuracy and outperform models with thousands of trainable parameters on smaller datasets.

Conclusion: Depthwise convolutional layers naturally gravitate toward fundamental spatial operators similar to classical image processing and biological visual systems, providing insights for generalization and transfer learning.

Abstract: A recent study has proposed the “Master Key Filters Hypothesis” for convolutional neural network filters. This paper extends this hypothesis by radically constraining its scope to a single set of just 8 universal filters that depthwise separable convolutional networks inherently converge to. While conventional DS-CNNs employ thousands of distinct trained filters, our analysis reveals these filters are predominantly linear shifts (ax+b) of our discovered universal set. Through systematic unsupervised search, we extracted these fundamental patterns across different architectures and datasets. Remarkably, networks initialized with these 8 unique frozen filters achieve over 80% ImageNet accuracy, and even outperform models with thousands of trainable parameters when applied to smaller datasets. The identified master key filters closely match Difference of Gaussians (DoGs), Gaussians, and their derivatives, structures that are not only fundamental to classical image processing but also strikingly similar to receptive fields in mammalian visual systems. Our findings provide compelling evidence that depthwise convolutional layers naturally gravitate toward this fundamental set of spatial operators regardless of task or architecture. This work offers new insights for understanding generalization and transfer learning through the universal language of these master key filters.

Method: Trained multiple state-of-the-art object detectors on 150,000 documents, applied post-processing to raw detections, and evaluated on various benchmarks across different hardware environments (CPU, Nvidia, Apple GPUs).

Result: Five new models introduced with 20.6-23.9% mAP improvement over baseline. Best model ‘heron-101’ achieves 78% mAP with 28ms/image inference time on NVIDIA A100 GPU.

Conclusion: The research establishes best practices for training, evaluating, and deploying document layout detectors, providing actionable guidance for the document conversion community, with all models and code released openly.

Abstract: This technical report documents the development of novel Layout Analysis models integrated into the Docling document-conversion pipeline. We trained several state-of-the-art object detectors based on the RT-DETR, RT-DETRv2 and DFINE architectures on a heterogeneous corpus of 150,000 documents (both openly available and proprietary). Post-processing steps were applied to the raw detections to make them more applicable to the document conversion task. We evaluated the effectiveness of the layout analysis on various document benchmarks using different methodologies while also measuring the runtime performance across different environments (CPU, Nvidia and Apple GPUs). We introduce five new document layout models achieving 20.6% - 23.9% mAP improvement over Docling’s previous baseline, with comparable or better runtime. Our best model, “heron-101”, attains 78% mAP with 28 ms/image inference time on a single NVIDIA A100 GPU. Extensive quantitative and qualitative experiments establish best practices for training, evaluating, and deploying document-layout detectors, providing actionable guidance for the document conversion community. All trained checkpoints, code, and documentation are released under a permissive license on HuggingFace.

Method: Proposes MISRA framework that augments RGB input with luminance channels, integrates skip attention to preserve elongated features, and employs an Iterative Feedback Module (IFM) for continuity restoration across multiple passes.

Result: Achieves competitive performance with 5.37% improvement in mean class IoU over competing methods, delivering more stable predictions at instrument contacts and overlaps.

Conclusion: MISRA represents a promising step toward reliable scene parsing for computer-assisted and robotic microsurgery, with a new dedicated microsurgical dataset provided for benchmarking.

Abstract: Accurate segmentation of thin structures is critical for microsurgical scene understanding but remains challenging due to resolution loss, low contrast, and class imbalance. We propose Microsurgery Instrument Segmentation for Robotic Assistance(MISRA), a segmentation framework that augments RGB input with luminance channels, integrates skip attention to preserve elongated features, and employs an Iterative Feedback Module(IFM) for continuity restoration across multiple passes. In addition, we introduce a dedicated microsurgical dataset with fine-grained annotations of surgical instruments including thin objects, providing a benchmark for robust evaluation Dataset available at https://huggingface.co/datasets/KIST-HARILAB/MISAW-Seg. Experiments demonstrate that MISRA achieves competitive performance, improving the mean class IoU by 5.37% over competing methods, while delivering more stable predictions at instrument contacts and overlaps. These results position MISRA as a promising step toward reliable scene parsing for computer-assisted and robotic microsurgery.

Method: Proposes DGMap with Multi-scale Grid Encoding, Mask-enhanced Keypoint Extraction, and Global Context-aware Relation Prediction to integrate global semantic context with local geometric features.

Result: Outperforms state-of-the-art methods by 5% in APLS on three real-world datasets, with notable performance gains on Didi Chuxing platform trajectory data.

Conclusion: DGMap effectively reduces road fragmentation in sparse areas and suppresses false connections in dense regions through global context modeling and improved keypoint detection.

Abstract: Trajectory data has become a key resource for automated map in-ference due to its low cost, broad coverage, and continuous availability. However, uneven trajectory density often leads to frag-mented roads in sparse areas and redundant segments in dense regions, posing significant challenges for existing methods. To address these issues, we propose DGMap, a dual-decoding framework with global context awareness, featuring Multi-scale Grid Encoding, Mask-enhanced Keypoint Extraction, and Global Context-aware Relation Prediction. By integrating global semantic context with local geometric features, DGMap improves keypoint detection accuracy to reduce road fragmentation in sparse-trajectory areas. Additionally, the Global Context-aware Relation Prediction module suppresses false connections in dense-trajectory regions by modeling long-range trajectory patterns. Experimental results on three real-world datasets show that DGMap outperforms state-of-the-art methods by 5% in APLS, with notable performance gains on trajectory data from the Didi Chuxing platform

Method: Self-supervised learning on EchoCareData (4.5M ultrasound images from 23+ countries), with a hierarchical classifier for joint learning of pixel-level and representation-level features to capture both global anatomical contexts and local ultrasound characteristics.

Result: Outperforms state-of-the-art models across 10 ultrasound benchmarks including disease diagnosis, lesion segmentation, organ detection, landmark prediction, quantitative regression, imaging enhancement and report generation with minimal training.

Conclusion: EchoCare provides a fully open and generalizable foundation model to boost AI development for diverse clinical ultrasound applications, with code and pretrained model publicly released for fine-tuning and local adaptation.

Abstract: Artificial intelligence (AI) that can effectively learn ultrasound representations by integrating multi-source data holds significant promise for advancing clinical care. However, the scarcity of large labeled datasets in real-world clinical environments and the limited generalizability of task-specific models have hindered the development of generalizable clinical AI models for ultrasound applications. In this study, we present EchoCare, a novel ultrasound foundation model for generalist clinical use, developed via self-supervised learning on our curated, publicly available, large-scale dataset EchoCareData. EchoCareData comprises 4.5 million ultrasound images, sourced from over 23 countries across 5 continents and acquired via a diverse range of distinct imaging devices, thus encompassing global cohorts that are multi-center, multi-device, and multi-ethnic. Unlike prior studies that adopt off-the-shelf vision foundation model architectures, we introduce a hierarchical classifier into EchoCare to enable joint learning of pixel-level and representation-level features, capturing both global anatomical contexts and local ultrasound characteristics. With minimal training, EchoCare outperforms state-of-the-art comparison models across 10 representative ultrasound benchmarks of varying diagnostic difficulties, spanning disease diagnosis, lesion segmentation, organ detection, landmark prediction, quantitative regression, imaging enhancement and report generation. The code and pretrained model are publicly released, rendering EchoCare accessible for fine-tuning and local adaptation, supporting extensibility to additional applications. EchoCare provides a fully open and generalizable foundation model to boost the development of AI technologies for diverse clinical ultrasound applications.

Method: Multi-Scale Feature Fusion with Multi-Attribute (MSMA) framework that integrates multi-scale feature fusion, focuses on multi-attribute learning, and uses large-kernel attention module for precise feature extraction across scales.

Result: Achieves results on par with current state-of-the-art methods on MICC Florence, Facewarehouse and custom datasets, and surpasses SOTA performance in some challenging conditions.

Conclusion: The proposed MSMA framework effectively addresses limitations in capturing detailed and multi-scale features for 3D face reconstruction from unconstrained images, reducing reliance on labeled 3D data while improving accuracy.

Abstract: Reconstructing 3D face from a single unconstrained image remains a challenging problem due to diverse conditions in unconstrained environments. Recently, learning-based methods have achieved notable results by effectively capturing complex facial structures and details across varying conditions. Consequently, many existing approaches employ projection-based losses between generated and input images to constrain model training. However, learning-based methods for 3D face reconstruction typically require substantial amounts of 3D facial data, which is difficult and costly to obtain. Consequently, to reduce reliance on labeled 3D face datasets, many existing approaches employ projection-based losses between generated and input images to constrain model training. Nonetheless, despite these advancements, existing approaches frequently struggle to capture detailed and multi-scale features under diverse facial attributes and conditions, leading to incomplete or less accurate reconstructions. In this paper, we propose a Multi-Scale Feature Fusion with Multi-Attribute (MSMA) framework for 3D face reconstruction from unconstrained images. Our method integrates multi-scale feature fusion with a focus on multi-attribute learning and leverages a large-kernel attention module to enhance the precision of feature extraction across scales, enabling accurate 3D facial parameter estimation from a single 2D image. Comprehensive experiments on the MICC Florence, Facewarehouse and custom-collect datasets demonstrate that our approach achieves results on par with current state-of-the-art methods, and in some instances, surpasses SOTA performance across challenging conditions.

Method: Integrates pre-trained object detectors with SAM2 and a novel Seg2Track module for track initialization, management, and reinforcement. Uses sliding-window memory strategy for efficiency.

Result: Achieves 4th overall on KITTI MOTS for car/pedestrian classes, sets new benchmark in association accuracy (AssA), and reduces memory usage by 75% with minimal performance loss.

Conclusion: Seg2Track-SAM2 advances MOTS by combining robust zero-shot tracking, enhanced identity preservation, and efficient memory utilization without fine-tuning.

Abstract: Autonomous systems require robust Multi-Object Tracking (MOT) capabilities to operate reliably in dynamic environments. MOT ensures consistent object identity assignment and precise spatial delineation. Recent advances in foundation models, such as SAM2, have demonstrated strong zero-shot generalization for video segmentation, but their direct application to MOTS (MOT+Segmentation) remains limited by insufficient identity management and memory efficiency. This work introduces Seg2Track-SAM2, a framework that integrates pre-trained object detectors with SAM2 and a novel Seg2Track module to address track initialization, track management, and reinforcement. The proposed approach requires no fine-tuning and remains detector-agnostic. Experimental results on KITTI MOT and KITTI MOTS benchmarks show that Seg2Track-SAM2 achieves state-of-the-art (SOTA) performance, ranking fourth overall in both car and pedestrian classes on KITTI MOTS, while establishing a new benchmark in association accuracy (AssA). Furthermore, a sliding-window memory strategy reduces memory usage by up to 75% with negligible performance degradation, supporting deployment under resource constraints. These results confirm that Seg2Track-SAM2 advances MOTS by combining robust zero-shot tracking, enhanced identity preservation, and efficient memory utilization. The code is available at https://github.com/hcmr-lab/Seg2Track-SAM2

Method: Two complementary approaches: 1) Analyzing semantic information preservation by comparing k-nearest neighbor relationships before/after projection, 2) Directly measuring information loss by reconstructing visual embeddings from projected representations at patch level.

Result: Connectors substantially distort local geometry of visual representations (40-60% divergence in k-nearest neighbors), correlating with retrieval performance degradation. Patch-level reconstruction shows high information loss areas predict model struggles in visually grounded QA tasks.

Conclusion: The projection step in VLMs causes significant information loss that impacts model capabilities, with patch-level analysis providing interpretable insights for understanding model failures in vision-language tasks.

Abstract: Vision–language models (VLMs) often process visual inputs through a pretrained vision encoder, followed by a projection into the language model’s embedding space via a connector component. While crucial for modality fusion, the potential information loss induced by this projection step and its direct impact on model capabilities remain understudied. We introduce two complementary approaches to examine and quantify this loss by analyzing the latent representation space. First, we evaluate semantic information preservation by analyzing changes in k-nearest neighbor relationships between image representations, before and after projection. Second, we directly measure information loss by reconstructing visual embeddings from the projected representation, localizing loss at an image patch level. Experiments reveal that connectors substantially distort the local geometry of visual representations, with k-nearest neighbors diverging by 40–60% post-projection, correlating with degradation in retrieval performance. The patch-level embedding reconstruction provides interpretable insights for model behavior on visually grounded question-answering tasks, finding that areas of high information loss reliably predict instances where models struggle.

Method: Proposes SA-UNetv2 with cross-scale spatial attention injected into all skip connections for better multi-scale feature fusion, and uses a weighted Binary Cross-Entropy plus Matthews Correlation Coefficient loss to handle class imbalance.

Result: Achieves state-of-the-art performance on DRIVE and STARE datasets with only 1.2MB memory and 0.26M parameters (less than 50% of SA-UNet), and 1 second CPU inference on 592x592x3 images.

Conclusion: SA-UNetv2 demonstrates strong efficiency and deployability in resource-constrained, CPU-only settings while maintaining excellent segmentation performance for retinal vessel analysis.

Abstract: Retinal vessel segmentation is essential for early diagnosis of diseases such as diabetic retinopathy, hypertension, and neurodegenerative disorders. Although SA-UNet introduces spatial attention in the bottleneck, it underuses attention in skip connections and does not address the severe foreground-background imbalance. We propose SA-UNetv2, a lightweight model that injects cross-scale spatial attention into all skip connections to strengthen multi-scale feature fusion and adopts a weighted Binary Cross-Entropy (BCE) plus Matthews Correlation Coefficient (MCC) loss to improve robustness to class imbalance. On the public DRIVE and STARE datasets, SA-UNetv2 achieves state-of-the-art performance with only 1.2MB memory and 0.26M parameters (less than 50% of SA-UNet), and 1 second CPU inference on 592 x 592 x 3 images, demonstrating strong efficiency and deployability in resource-constrained, CPU-only settings.

Method: Proposes FineQuest framework with: 1) Dual-mode reasoning inspired by cognitive science (reactive for straightforward queries, deliberative for complex ones), 2) SSGraph - multimodal sports knowledge scene graph spanning 9 sports with visual instances and domain terminology, 3) Two new benchmarks (Gym-QA and Diving-QA) for comprehensive evaluation.

Result: Achieves state-of-the-art performance on new benchmarks (Gym-QA, Diving-QA) and existing SPORTU dataset, while maintaining strong general VideoQA capabilities.

Conclusion: FineQuest demonstrates that training-free frameworks with cognitive-inspired dual reasoning and domain-specific knowledge graphs can effectively address the challenges of sports VideoQA without sacrificing general video understanding performance.

Abstract: Video Question Answering (VideoQA) based on Large Language Models (LLMs) has shown potential in general video understanding but faces significant challenges when applied to the inherently complex domain of sports videos. In this work, we propose FineQuest, the first training-free framework that leverages dual-mode reasoning inspired by cognitive science: i) Reactive Reasoning for straightforward sports queries and ii) Deliberative Reasoning for more complex ones. To bridge the knowledge gap between general-purpose models and domain-specific sports understanding, FineQuest incorporates SSGraph, a multimodal sports knowledge scene graph spanning nine sports, which encodes both visual instances and domain-specific terminology to enhance reasoning accuracy. Furthermore, we introduce two new sports VideoQA benchmarks, Gym-QA and Diving-QA, derived from the FineGym and FineDiving datasets, enabling diverse and comprehensive evaluation. FineQuest achieves state-of-the-art performance on these benchmarks as well as the existing SPORTU dataset, while maintains strong general VideoQA capabilities.

Method: Leverages neural network training dynamics to assess sample difficulty, aggregates and calibrates model predictions during training to generate uncertainty-aware pseudo-labels that can be applied to any supervised learning pipeline.

Result: Superior performance on echocardiography classification benchmark compared to specialized baselines in calibration, selective classification, and multi-view fusion.

Conclusion: The proposed framework effectively brings uncertainty back into the label space, enhancing uncertainty estimation and robustness in medical image diagnosis systems.

Abstract: Computer-aided diagnosis systems must make critical decisions from medical images that are often noisy, ambiguous, or conflicting, yet today’s models are trained on overly simplistic labels that ignore diagnostic uncertainty. One-hot labels erase inter-rater variability and force models to make overconfident predictions, especially when faced with incomplete or artifact-laden inputs. We address this gap by introducing a novel framework that brings uncertainty back into the label space. Our method leverages neural network training dynamics (NNTD) to assess the inherent difficulty of each training sample. By aggregating and calibrating model predictions during training, we generate uncertainty-aware pseudo-labels that reflect the ambiguity encountered during learning. This label augmentation approach is architecture-agnostic and can be applied to any supervised learning pipeline to enhance uncertainty estimation and robustness. We validate our approach on a challenging echocardiography classification benchmark, demonstrating superior performance over specialized baselines in calibration, selective classification, and multi-view fusion.

Method: Incorporates focal modulation attention at the encoder-decoder bottleneck and region-aware attention in selective skip connections to enhance feature representation and regional focus by capturing local and global dependencies efficiently.

Result: Outperformed state-of-the-art models on DRIVE, STARE, and CHASE_DB datasets with Dice scores of 84.28%, 88.44%, 85.50% and Jaccard indices of 72.86%, 79.31%, 74.70% respectively, while maintaining lightweight characteristics.

Conclusion: LFRA-Net provides an ideal balance between segmentation accuracy and computational cost, making it suitable for real-time clinical applications in resource-limited areas.

Abstract: Retinal vessel segmentation is critical for the early diagnosis of vision-threatening and systemic diseases, especially in real-world clinical settings with limited computational resources. Although significant improvements have been made in deep learning-based segmentation methods, current models still face challenges in extracting tiny vessels and suffer from high computational costs. In this study, we present LFRA-Net by incorporating focal modulation attention at the encoder-decoder bottleneck and region-aware attention in the selective skip connections. LFRA-Net is a lightweight network optimized for precise and effective retinal vascular segmentation. It enhances feature representation and regional focus by efficiently capturing local and global dependencies. LFRA-Net outperformed many state-of-the-art models while maintaining lightweight characteristics with only 0.17 million parameters, 0.66 MB memory size, and 10.50 GFLOPs. We validated it on three publicly available datasets: DRIVE, STARE, and CHASE_DB. It performed better in terms of Dice score (84.28%, 88.44%, and 85.50%) and Jaccard index (72.86%, 79.31%, and 74.70%) on the DRIVE, STARE, and CHASE_DB datasets, respectively. LFRA-Net provides an ideal ratio between segmentation accuracy and computational cost compared to existing deep learning methods, which makes it suitable for real-time clinical applications in areas with limited resources. The code can be found at https://github.com/Mehwish4593/LFRA-Net.

Method: Combines EAGLE-2-style baseline (EagleVLM) with elastic visual compressor that adaptively selects compression primitives, plus online-logit distillation protocol using teacher logits and features with cross-entropy and Smooth L1 loss.

Result: Achieves 2.5-2.9x end-to-end speedups across LLaVA and MMMU models within 5 epochs, consistent over resolutions and task difficulties while preserving output distribution.

Conclusion: SpecVLM provides a practical system for accelerating VLMs through speculative decoding with adaptive visual compression and efficient online training, demonstrating significant speed improvements without quality loss.

Abstract: Speculative decoding is a powerful way to accelerate autoregressive large language models (LLMs), but directly porting it to vision-language models (VLMs) faces unique systems constraints: the prefill stage is dominated by visual tokens whose count scales with image resolution and video length, inflating both compute and memory, especially the key-value (KV) cache. We study speculative decoding for VLMs and introduce SpecVLM, a practical system that (1) establishes a strong EAGLE-2-style baseline, EagleVLM, delivering 1.5–2.3x end-to-end speedups over full autoregressive inference, and (2) further accelerates VLM inference with an elastic visual compressor that adaptively selects among pruning, pooling, convolution, and resampler primitives to balance FLOPs/parameters and accuracy per input. To avoid costly offline distillation corpora, we propose an online-logit distillation protocol that trains the draft model with on-the-fly teacher logits and penultimate features using a combined cross-entropy and Smooth L1 objective, eliminating storage and preprocessing while remaining compute-efficient. This protocol reveals a training-time scaling effect: longer online training monotonically increases the draft model’s average accepted length, improving speculative efficiency. Empirically, SpecVLM achieves additional acceleration, culminating in 2.5–2.9x end-to-end speedups within 5 epochs across LLaVA and MMMU, consistently over resolutions and task difficulties, while preserving the target model’s output distribution (lossless decoding). Our code is available at https://github.com/haiduo/SpecVLM.

Method: Two-stage approach: 1) Construct vision-centered reasoning data using agent interaction between VLMs and reasoning LLMs for cold-start learning; 2) Use visual attention-based reward model during RL to encourage visual-based reasoning.

Result: Significant improvements across multiple visual reasoning benchmarks, with stronger and more consistent reliance on visual information during reasoning.

Conclusion: Reflection-V effectively enhances visual reflection capabilities in VRMs, addressing the critical challenge of maintaining visual attention throughout longer reasoning processes.

Abstract: Recent advances in text-only “slow-thinking” reasoning have prompted efforts to transfer this capability to vision-language models (VLMs), for training visual reasoning models (\textbf{VRMs}). owever, such transfer faces critical challenges: Effective “slow thinking” in VRMs requires \textbf{visual reflection}, the ability to check the reasoning process based on visual information. Through quantitative analysis, we observe that current VRMs exhibit limited visual reflection, as their attention to visual information diminishes rapidly with longer generated responses. To address this challenge, we propose a new VRM \textbf{Reflection-V}, which enhances visual reflection based on reasoning data construction for cold-start and reward design for reinforcement learning (RL). Firstly, we construct vision-centered reasoning data by leveraging an agent that interacts between VLMs and reasoning LLMs, enabling cold-start learning of visual reflection patterns. Secondly, a visual attention based reward model is employed during RL to encourage reasoning based on visual information. Therefore, \textbf{Reflection-V} demonstrates significant improvements across multiple visual reasoning benchmarks. Furthermore, \textbf{Reflection-V} maintains a stronger and more consistent reliance on visual information during visual reasoning, indicating effective enhancement in visual reflection capabilities.

Method: Parallel structure with fusion and segmentation sub-networks, heterogeneous feature fusion strategy, multi-stage Transformer decoder, and dynamic factor for adaptive task weighting.

Result: Achieves competitive results compared with state-of-the-art methods in extensive experiments.

Conclusion: The proposed unified framework effectively bridges image fusion and semantic segmentation tasks with mutual enhancement.

Abstract: Infrared-visible image fusion methods aim at generating fused images with good visual quality and also facilitate the performance of high-level tasks. Indeed, existing semantic-driven methods have considered semantic information injection for downstream applications. However, none of them investigates the potential for reciprocal promotion between pixel-wise image fusion and cross-modal feature fusion perception tasks from a macroscopic task-level perspective. To address this limitation, we propose a unified network for image fusion and semantic segmentation. MAFS is a parallel structure, containing a fusion sub-network and a segmentation sub-network. On the one hand, We devise a heterogeneous feature fusion strategy to enhance semantic-aware capabilities for image fusion. On the other hand, by cascading the fusion sub-network and a segmentation backbone, segmentation-related knowledge is transferred to promote feature-level fusion-based segmentation. Within the framework, we design a novel multi-stage Transformer decoder to aggregate fine-grained multi-scale fused features efficiently. Additionally, a dynamic factor based on the max-min fairness allocation principle is introduced to generate adaptive weights of two tasks and guarantee smooth training in a multi-task manner. Extensive experiments demonstrate that our approach achieves competitive results compared with state-of-the-art methods. The code is available at https://github.com/Abraham-Einstein/MAFS/.

Method: Combines sampling-based reachability analysis with conformal inference, introducing novel strategies to reduce conservatism in high-dimensional settings while maintaining rigorous guarantees.

Result: Empirical evaluation on large-scale segmentation models across multiple datasets (CamVid, OCTA-500, Lung Segmentation, Cityscapes) shows reliable safety guarantees with substantially tighter bounds compared to state-of-the-art methods.

Conclusion: The proposed framework provides architecture-agnostic, scalable probabilistic verification for semantic segmentation networks, delivering practical safety guarantees with reduced conservatism, accompanied by an open-source implementation toolbox.

Abstract: Semantic segmentation networks (SSNs) play a critical role in domains such as medical imaging, autonomous driving, and environmental monitoring, where safety hinges on reliable model behavior under uncertainty. Yet, existing probabilistic verification approaches struggle to scale with the complexity and dimensionality of modern segmentation tasks, often yielding guarantees that are too conservative to be practical. We introduce a probabilistic verification framework that is both architecture-agnostic and scalable to high-dimensional outputs. Our approach combines sampling-based reachability analysis with conformal inference (CI) to deliver provable guarantees while avoiding the excessive conservatism of prior methods. To counteract CI’s limitations in high-dimensional settings, we propose novel strategies that reduce conservatism without compromising rigor. Empirical evaluation on large-scale segmentation models across CamVid, OCTA-500, Lung Segmentation, and Cityscapes demonstrates that our method provides reliable safety guarantees while substantially tightening bounds compared to SOTA. We also provide a toolbox implementing this technique, available on Github.

Method: Densely align images with synthetic descriptions generated by vision-language models, leveraging inexpensive and scalable synthetic captions as a source of high-level semantic understanding.

Result: Outperforms prior work on standard zero-shot open-vocabulary segmentation benchmarks while being more data-efficient.

Conclusion: Synthetic captions from VLMs provide an effective and scalable way to achieve dense vision-language alignment for improved segmentation performance.

Abstract: Generative vision-language models (VLMs) exhibit strong high-level image understanding but lack spatially dense alignment between vision and language modalities, as our findings indicate. Orthogonal to advancements in generative VLMs, another line of research has focused on representation learning for vision-language alignment, targeting zero-shot inference for dense tasks like segmentation. In this work, we bridge these two directions by densely aligning images with synthetic descriptions generated by VLMs. Synthetic captions are inexpensive, scalable, and easy to generate, making them an excellent source of high-level semantic understanding for dense alignment methods. Empirically, our approach outperforms prior work on standard zero-shot open-vocabulary segmentation benchmarks/datasets, while also being more data-efficient.

Method: Leverages region-based segmentation to identify structurally significant regions, enabling selective downsampling of dense point clouds to preserve fidelity while reducing Gaussian count.

Result: Reduces Gaussian count by up to 50%, achieves comparable/superior PSNR and SSIM with marginal LPIPS degradation, enables faster training and lower memory footprint.

Conclusion: SDI-GS advances 3DGS practicality for constrained-view scenarios through efficient segmentation-driven initialization that maintains quality while reducing computational costs.

Abstract: Sparse-view synthesis remains a challenging problem due to the difficulty of recovering accurate geometry and appearance from limited observations. While recent advances in 3D Gaussian Splatting (3DGS) have enabled real-time rendering with competitive quality, existing pipelines often rely on Structure-from-Motion (SfM) for camera pose estimation, an approach that struggles in genuinely sparse-view settings. Moreover, several SfM-free methods replace SfM with multi-view stereo (MVS) models, but generate massive numbers of 3D Gaussians by back-projecting every pixel into 3D space, leading to high memory costs. We propose Segmentation-Driven Initialization for Gaussian Splatting (SDI-GS), a method that mitigates inefficiency by leveraging region-based segmentation to identify and retain only structurally significant regions. This enables selective downsampling of the dense point cloud, preserving scene fidelity while substantially reducing Gaussian count. Experiments across diverse benchmarks show that SDI-GS reduces Gaussian count by up to 50% and achieves comparable or superior rendering quality in PSNR and SSIM, with only marginal degradation in LPIPS. It further enables faster training and lower memory footprint, advancing the practicality of 3DGS for constrained-view scenarios.

Method: SG-VLM is a modular framework that integrates frozen VLMs with scene graph grounding via prompting and visual localization techniques, using symbolic scene graphs as intermediate grounding signals.

Result: Across three benchmarks (NExT-QA, iVQA, ActivityNet-QA) and multiple VLMs (QwenVL, InternVL), SG-VLM improves causal and temporal reasoning and outperforms prior baselines, though gains over strong VLMs are limited.

Conclusion: The findings highlight both the promise and current limitations of symbolic grounding, offering guidance for future hybrid VLM-symbolic approaches in video understanding.

Abstract: Video Question Answering (VQA) requires models to reason over spatial, temporal, and causal cues in videos. Recent vision language models (VLMs) achieve strong results but often rely on shallow correlations, leading to weak temporal grounding and limited interpretability. We study symbolic scene graphs (SGs) as intermediate grounding signals for VQA. SGs provide structured object-relation representations that complement VLMs holistic reasoning. We introduce SG-VLM, a modular framework that integrates frozen VLMs with scene graph grounding via prompting and visual localization. Across three benchmarks (NExT-QA, iVQA, ActivityNet-QA) and multiple VLMs (QwenVL, InternVL), SG-VLM improves causal and temporal reasoning and outperforms prior baselines, though gains over strong VLMs are limited. These findings highlight both the promise and current limitations of symbolic grounding, and offer guidance for future hybrid VLM-symbolic approaches in video understanding.

Method: Two-component framework: Dr.V-Bench dataset with 10k instances from 4,974 videos with spatial-temporal annotations, and Dr.V-Agent that applies fine-grained spatial-temporal grounding at multiple levels.

Result: Extensive experiments show Dr.V-Agent effectively diagnoses hallucinations while enhancing interpretability and reliability in video understanding.

Conclusion: Dr.V provides a practical blueprint for robust video understanding in real-world scenarios by systematically detecting and addressing hallucinations in large video models.

Abstract: Recent advancements in large video models (LVMs) have significantly enhance video understanding. However, these models continue to suffer from hallucinations, producing content that conflicts with input videos. To address this issue, we propose Dr.V, a hierarchical framework covering perceptive, temporal, and cognitive levels to diagnose video hallucination by fine-grained spatial-temporal grounding. Dr.V comprises of two key components: a benchmark dataset Dr.V-Bench and a satellite video agent Dr.V-Agent. Dr.V-Bench includes 10k instances drawn from 4,974 videos spanning diverse tasks, each enriched with detailed spatial-temporal annotation. Dr.V-Agent detects hallucinations in LVMs by systematically applying fine-grained spatial-temporal grounding at the perceptive and temporal levels, followed by cognitive level reasoning. This step-by-step pipeline mirrors human-like video comprehension and effectively identifies hallucinations. Extensive experiments demonstrate that Dr.V-Agent is effective in diagnosing hallucination while enhancing interpretability and reliability, offering a practical blueprint for robust video understanding in real-world scenarios. All our data and code are available at https://github.com/Eurekaleo/Dr.V.

Method: Benchmarked both MAT tools (DeepLabCut, idTracker) and MOT methods (ByteTrack, DeepSORT, cross-input consistency, Track-Anything, PromptTrack) on a 10-minute pig tracking dataset to compare performance for long-term tracking.

Result: MOT approaches overall outperformed traditional MAT tools in long-term pig tracking scenarios, demonstrating superior accuracy and reliability.

Conclusion: Recent MOT techniques have significant potential to enhance automated livestock tracking accuracy and reliability, making them valuable for precision livestock farming applications.

Abstract: Precision livestock farming requires advanced monitoring tools to meet the increasing management needs of the industry. Computer vision systems capable of long-term multi-animal tracking (MAT) are essential for continuous behavioral monitoring in livestock production. MAT, a specialized subset of multi-object tracking (MOT), shares many challenges with MOT, but also faces domain-specific issues including frequent animal occlusion, highly similar appearances among animals, erratic motion patterns, and a wide range of behavior types. While some existing MAT tools are user-friendly and widely adopted, they often underperform compared to state-of-the-art MOT methods, which can result in inaccurate downstream tasks such as behavior analysis, health state estimation, and related applications. In this study, we benchmarked both MAT and MOT approaches for long-term tracking of pigs. We compared tools such as DeepLabCut and idTracker with MOT-based methods including ByteTrack, DeepSORT, cross-input consistency, and newer approaches like Track-Anything and PromptTrack. All methods were evaluated on a 10-minute pig tracking dataset. Our results demonstrate that, overall, MOT approaches outperform traditional MAT tools, even for long-term tracking scenarios. These findings highlight the potential of recent MOT techniques to enhance the accuracy and reliability of automated livestock tracking.

Method: Introduces Weighted Prompt Manipulation (WPM) which systematically modifies attention weights and text embeddings within diffusion models. Uses diffusion models and large language models (GPT) with poetry datasets to dynamically adjust word importance for enhanced or suppressed influence in image generation.

Result: The approach enables semantically richer and more contextually accurate visualizations of poetry by allowing dynamic adjustment of specific word importance in the generated images.

Conclusion: This represents the first integration of weighted prompt manipulation for enhancing imagery in poetic language, providing a comprehensive methodology for improved poetry visualization through AI-generated imagery.

Abstract: Poetry is an expressive form of art that invites multiple interpretations, as readers often bring their own emotions, experiences, and cultural backgrounds into their understanding of a poem. Recognizing this, we aim to generate images for poems and improve these images in a zero-shot setting, enabling audiences to modify images as per their requirements. To achieve this, we introduce a novel Weighted Prompt Manipulation (WPM) technique, which systematically modifies attention weights and text embeddings within diffusion models. By dynamically adjusting the importance of specific words, WPM enhances or suppresses their influence in the final generated image, leading to semantically richer and more contextually accurate visualizations. Our approach exploits diffusion models and large language models (LLMs) such as GPT in conjunction with existing poetry datasets, ensuring a comprehensive and structured methodology for improved image generation in the literary domain. To the best of our knowledge, this is the first attempt at integrating weighted prompt manipulation for enhancing imagery in poetic language.

Method: Proposes Reverse SAM Parameter Configuration Module to mitigate adverse parameters in train-free manner, and T-Visioner Module integrating Test-Time Training layers from language tasks to vision tasks for reinforcing advantageous parameters.

Result: Achieves state-of-the-art performance on various COD benchmarks, significantly improving SAM’s semantic understanding in COD tasks.

Conclusion: SAM-TTT sets a new benchmark in camouflaged object detection by simultaneously suppressing adverse parameters while reinforcing advantageous ones through innovative parameter configuration and test-time training integration.

Abstract: This paper introduces a new Segment Anything Model (SAM) that leverages reverse parameter configuration and test-time training to enhance its performance on Camouflaged Object Detection (COD), named SAM-TTT. While most existing SAM-based COD models primarily focus on enhancing SAM by extracting favorable features and amplifying its advantageous parameters, a crucial gap is identified: insufficient attention to adverse parameters that impair SAM’s semantic understanding in downstream tasks. To tackle this issue, the Reverse SAM Parameter Configuration Module is proposed to effectively mitigate the influence of adverse parameters in a train-free manner by configuring SAM’s parameters. Building on this foundation, the T-Visioner Module is unveiled to strengthen advantageous parameters by integrating Test-Time Training layers, originally developed for language tasks, into vision tasks. Test-Time Training layers represent a new class of sequence modeling layers characterized by linear complexity and an expressive hidden state. By integrating two modules, SAM-TTT simultaneously suppresses adverse parameters while reinforcing advantageous ones, significantly improving SAM’s semantic understanding in COD task. Our experimental results on various COD benchmarks demonstrate that the proposed approach achieves state-of-the-art performance, setting a new benchmark in the field. The code will be available at https://github.com/guobaoxiao/SAM-TTT.

Method: Proposes a depth-aware CycleGAN to translate between real bronchoscopic images and airway geometries, and introduces an airway structure awareness loss to enforce depth consistency within the airway lumen while preserving smooth transitions.

Result: Outperforms existing methods on both a collected ex vivo human lung dataset and an open bronchoscopic dataset, demonstrating superior anatomical depth preservation and more robust 3D airway reconstructions.

Conclusion: Brea-Depth successfully bridges the domain gap in bronchoscopic depth estimation by incorporating anatomical priors, enhancing model generalization and yielding more accurate 3D reconstructions with better structural consistency.

Abstract: Monocular depth estimation in bronchoscopy can significantly improve real-time navigation accuracy and enhance the safety of interventions in complex, branching airways. Recent advances in depth foundation models have shown promise for endoscopic scenarios, yet these models often lack anatomical awareness in bronchoscopy, overfitting to local textures rather than capturing the global airway structure, particularly under ambiguous depth cues and poor lighting. To address this, we propose Brea-Depth, a novel framework that integrates airway-specific geometric priors into foundation model adaptation for bronchoscopic depth estimation. Our method introduces a depth-aware CycleGAN, refining the translation between real bronchoscopic images and airway geometries from anatomical data, effectively bridging the domain gap. In addition, we introduce an airway structure awareness loss to enforce depth consistency within the airway lumen while preserving smooth transitions and structural integrity. By incorporating anatomical priors, Brea-Depth enhances model generalization and yields more robust, accurate 3D airway reconstructions. To assess anatomical realism, we introduce Airway Depth Structure Evaluation, a new metric for structural consistency. We validate BREA-Depth on a collected ex vivo human lung dataset and an open bronchoscopic dataset, where it outperforms existing methods in anatomical depth preservation.

Method: Linear interpolation technique in logit space that mixes in-distribution and out-of-distribution data, with consistency enforcement between logit-space mixing and input-space mixing results.

Result: Reduces abrupt fluctuations in model outputs near decision boundaries, creates smoother and more reliable separation between in-distribution and out-of-distribution data.

Conclusion: Logit-space mixing technique effectively improves out-of-distribution detection performance, particularly for challenging cases where OOD data lies close to in-distribution data.

Abstract: The ability to detect out-of-distribution data is essential not only for ensuring robustness against unknown or unexpected input data but also for improving the generalization performance of the model. Among various out-of-distribution detection methods, Outlier Exposure and Mixture Outlier Exposure are promising approaches that enhance out-of-distribution detection performance by exposing the outlier data during training. However, even with these sophisticated techniques, it remains challenging for models to learn the relationships between classes effectively and to distinguish data sampling from in-distribution and out-of-distribution clearly. Therefore, we focus on the logit space, where the properties between class-wise distributions are distinctly separated from those in the input or feature spaces. Specifically, we propose a linear interpolation technique in the logit space that mixes in-distribution and out-of-distribution data to facilitate smoothing logits between classes and improve the out-of-distribution detection performance, particularly for out-of-distribution data that lie close to the in-distribution data. Additionally, we enforce consistency between the logits obtained through mixing in the logit space and those generated via mixing in the input space. Our experiments demonstrate that our logit-space mixing technique reduces the abrupt fluctuations in the model outputs near the decision boundaries, resulting in smoother and more reliable separation between in-distribution and out-of-distribution data. Furthermore, we evaluate the effectiveness of the proposed method on a fine-grained out-of-distribution detection task.

Method: Heterogeneous graph model with multiple layers that incorporates multi-modal information (prior observations, semantic embeddings like CLIP) directly into message-passing process. Uses global and local scene representations without specialized modules.

Result: Evaluation on 3DSSG dataset shows that GNNs enriched with multi-modal information offer scalable and generalizable solutions for complex real-world environments.

Conclusion: The approach provides a flexible framework for incremental 3DSSG prediction that works with partial observations and integrates diverse information sources, making it suitable for real-world robotics and embodied AI applications.

Abstract: 3D semantic scene graphs (3DSSG) provide compact structured representations of environments by explicitly modeling objects, attributes, and relationships. While 3DSSGs have shown promise in robotics and embodied AI, many existing methods rely mainly on sensor data, not integrating further information from semantically rich environments. Additionally, most methods assume access to complete scene reconstructions, limiting their applicability in real-world, incremental settings. This paper introduces a novel heterogeneous graph model for incremental 3DSSG prediction that integrates additional, multi-modal information, such as prior observations, directly into the message-passing process. Utilizing multiple layers, the model flexibly incorporates global and local scene representations without requiring specialized modules or full scene reconstructions. We evaluate our approach on the 3DSSG dataset, showing that GNNs enriched with multi-modal information such as semantic embeddings (e.g., CLIP) and prior observations offer a scalable and generalizable solution for complex, real-world environments. The full source code of the presented architecture will be made available at https://github.com/m4renz/incremental-scene-graph-prediction.

Method: Uses a teacher trained on EEG topographic maps to produce frozen Valence/Arousal prototypes. The student model (ResNet-18/50) is trained with conventional loss plus two lightweight regularizers: Proto-KD (cosine alignment to static prototypes) and D-Geo (geometric prior inspired by depression research findings).

Result: The method shows consistent gains attributed to prototypes and D-Geo, with 5x5 grid performing better than denser grids for stability. It improves robustness in both within-domain (FERPlus) and cross-dataset evaluations (AffectNet-mini, CK+) without architectural complexity.

Conclusion: NeuroGaze-Distill provides a simple, deployable framework that enhances facial emotion recognition generalization by transferring brain-informed priors through static prototypes and geometric regularization, requiring no EEG signals during deployment.

Abstract: Facial emotion recognition (FER) models trained only on pixels often fail to generalize across datasets because facial appearance is an indirect and biased proxy for underlying affect. We present NeuroGaze-Distill, a cross-modal distillation framework that transfers brain-informed priors into an image-only FER student via static Valence/Arousal (V/A) prototypes and a depression-inspired geometric prior (D-Geo). A teacher trained on EEG topographic maps from DREAMER (with MAHNOB-HCI as unlabeled support) produces a consolidated 5x5 V/A prototype grid that is frozen and reused; no EEG-face pairing and no non-visual signals at deployment are required. The student (ResNet-18/50) is trained on FERPlus with conventional CE/KD and two lightweight regularizers: (i) Proto-KD (cosine) aligns student features to the static prototypes; (ii) D-Geo softly shapes the embedding geometry in line with affective findings often reported in depression research (e.g., anhedonia-like contraction in high-valence regions). We evaluate both within-domain (FERPlus validation) and cross-dataset protocols (AffectNet-mini; optional CK+), reporting standard 8-way scores alongside present-only Macro-F1 and balanced accuracy to fairly handle label-set mismatch. Ablations attribute consistent gains to prototypes and D-Geo, and favor 5x5 over denser grids for stability. The method is simple, deployable, and improves robustness without architectural complexity.

Method: Variation inference and Multimodal knowledge Distillation (VMD) strategy that harnesses cross-modality prior knowledge from limited image annotations and radiology reports to enhance diagnostic accuracy for unannotated 3D MRI images.

Result: Conducted in-depth experiments on an in-house collected dataset and verified the effectiveness of the proposed VMD strategy.

Conclusion: The VMD method provides an effective approach to automate carotid plaque vulnerability diagnosis by leveraging radiologists’ domain knowledge through multimodal learning techniques.

Abstract: Multimodal learning has attracted much attention in recent years due to its ability to effectively utilize data features from a variety of different modalities. Diagnosing the vulnerability of atherosclerotic plaques directly from carotid 3D MRI images is relatively challenging for both radiologists and conventional 3D vision networks. In clinical practice, radiologists assess patient conditions using a multimodal approach that incorporates various imaging modalities and domain-specific expertise, paving the way for the creation of multimodal diagnostic networks. In this paper, we have developed an effective strategy to leverage radiologists’ domain knowledge to automate the diagnosis of carotid plaque vulnerability through Variation inference and Multimodal knowledge Distillation (VMD). This method excels in harnessing cross-modality prior knowledge from limited image annotations and radiology reports within training data, thereby enhancing the diagnostic network’s accuracy for unannotated 3D MRI images. We conducted in-depth experiments on the dataset collected in-house and verified the effectiveness of the VMD strategy we proposed.

Method: The method initializes a directed graph adjacency matrix based on a known interpolator, then learns perturbation matrices from data to augment it. The restoration effects are implemented via Douglas-Rachford iterations, which are unrolled into a lightweight interpretable neural network.

Result: Experimental results demonstrate state-of-the-art image interpolation performance while drastically reducing the number of network parameters compared to conventional approaches.

Conclusion: The proposed approach provides an effective alternative to random initialization and SGD optimization, achieving superior interpolation results with improved parameter efficiency through graph-based initialization and unrolled optimization iterations.

Abstract: Conventional deep neural nets (DNNs) initialize network parameters at random and then optimize each one via stochastic gradient descent (SGD), resulting in substantial risk of poor-performing local minima.Focusing on the image interpolation problem and leveraging a recent theorem that maps a (pseudo-)linear interpolator {\Theta} to a directed graph filter that is a solution to a MAP problem regularized with a graph shift variation (GSV) prior, we first initialize a directed graph adjacency matrix A based on a known interpolator {\Theta}, establishing a baseline performance.Then, towards further gain, we learn perturbation matrices P and P(2) from data to augment A, whose restoration effects are implemented via Douglas-Rachford (DR) iterations, which we unroll into a lightweight interpretable neural net.Experimental results demonstrate state-of-the-art image interpolation results, while drastically reducing network parameters.

Method: Dual encoder architecture extracts modality-specific features from optical and SAR imagery, fused via cross-modality attention module (CLAIRE). Uses hybrid RIFT loss (Weighted Focal + Tversky) for class imbalance. Includes SLM-generated reasoning module for interpretability.

Result: Achieved competitive performance: 56.02% mIoU and 84.56% OA on WHU-OPT-SAR; 59.89% mIoU and 73.92% OA on OpenEarthMap-SAR; 86.86% mIoU and 94.58% OA on PIE-RGB-SAR under cloud-obstructed conditions.

Conclusion: The proposed CLAIRE architecture with cross-modality fusion and RIFT loss effectively addresses land cover classification challenges, demonstrating strong performance, generalization, and robustness while providing interpretable explanations through SLM-generated reasoning.

Abstract: Accurate land cover classification from satellite imagery is crucial in environmental monitoring and sustainable resource management. However, it remains challenging due to the complexity of natural landscapes, the visual similarity between classes, and the significant class imbalance in the available datasets. To address these issues, we propose a dual encoder architecture that independently extracts modality-specific features from optical and Synthetic Aperture Radar (SAR) imagery, which are then fused using a cross-modality attention-fusion module named Cross-modality Land cover segmentation with Attention and Imbalance-aware Reasoning-Enhanced Explanations (CLAIRE). This fusion mechanism highlights complementary spatial and textural features, enabling the network to better capture detailed and diverse land cover patterns. We incorporate a hybrid loss function that utilizes Weighted Focal Loss and Tversky Loss named RIFT (Rare-Instance Focal-Tversky) to address class imbalance and improve segmentation performance across underrepresented categories. Our model achieves competitive performance across multiple benchmarks: a mean Intersection over Union (mIoU) of 56.02% and Overall Accuracy (OA) of 84.56% on the WHU-OPT-SAR dataset; strong generalization with a mIoU of 59.89% and OA of 73.92% on the OpenEarthMap-SAR dataset; and remarkable robustness under cloud-obstructed conditions, achieving an mIoU of 86.86% and OA of 94.58% on the PIE-RGB-SAR dataset. Additionally, we introduce a metric-driven reasoning module generated by a Small Language Model (Phi-3), which generates expert-level, sample-specific justifications for model predictions, thereby enhancing transparency and interpretability.

Method: Instructions are parsed into ego-centric scene graphs, processed by a tri-branch diffusion model for object layouts, trajectories, and shapes. A range-image diffusion model generates initial scans, extended by an autoregressive module into temporally coherent sequences with object-level editing capabilities.

Result: On nuScenes dataset, LiDARCrafter achieves state-of-the-art fidelity, controllability, and temporal consistency, providing a foundation for LiDAR-based simulation and data augmentation.

Conclusion: The framework successfully addresses key challenges in 4D LiDAR generation and introduces EvalSuite benchmark for comprehensive evaluation, establishing a robust foundation for future LiDAR data synthesis applications.

Abstract: While generative world models have advanced video and occupancy-based data synthesis, LiDAR generation remains underexplored despite its importance for accurate 3D perception. Extending generation to 4D LiDAR data introduces challenges in controllability, temporal stability, and evaluation. We present LiDARCrafter, a unified framework that converts free-form language into editable LiDAR sequences. Instructions are parsed into ego-centric scene graphs, which a tri-branch diffusion model transforms into object layouts, trajectories, and shapes. A range-image diffusion model generates the initial scan, and an autoregressive module extends it into a temporally coherent sequence. The explicit layout design further supports object-level editing, such as insertion or relocation. To enable fair assessment, we provide EvalSuite, a benchmark spanning scene-, object-, and sequence-level metrics. On nuScenes, LiDARCrafter achieves state-of-the-art fidelity, controllability, and temporal consistency, offering a foundation for LiDAR-based simulation and data augmentation.

Method: SCORE minimizes mutual information between target concepts and generated outputs using adversarial optimization, trajectory consistency, and saliency-driven fine-tuning to achieve statistical independence.

Result: Outperforms state-of-the-art methods by up to 12.5% higher erasure efficacy across four benchmarks while maintaining comparable or superior image quality.

Conclusion: SCORE sets a new standard for secure and robust concept erasure in diffusion models with theoretical guarantees and superior empirical performance.

Abstract: Diffusion models have achieved unprecedented success in image generation but pose increasing risks in terms of privacy, fairness, and security. A growing demand exists to \emph{erase} sensitive or harmful concepts (e.g., NSFW content, private individuals, artistic styles) from these models while preserving their overall generative capabilities. We introduce \textbf{SCORE} (Secure and Concept-Oriented Robust Erasure), a novel framework for robust concept removal in diffusion models. SCORE formulates concept erasure as an \emph{adversarial independence} problem, theoretically guaranteeing that the model’s outputs become statistically independent of the erased concept. Unlike prior heuristic methods, SCORE minimizes the mutual information between a target concept and generated outputs, yielding provable erasure guarantees. We provide formal proofs establishing convergence properties and derive upper bounds on residual concept leakage. Empirically, we evaluate SCORE on Stable Diffusion and FLUX across four challenging benchmarks: object erasure, NSFW removal, celebrity face suppression, and artistic style unlearning. SCORE consistently outperforms state-of-the-art methods including EraseAnything, ANT, MACE, ESD, and UCE, achieving up to \textbf{12.5%} higher erasure efficacy while maintaining comparable or superior image quality. By integrating adversarial optimization, trajectory consistency, and saliency-driven fine-tuning, SCORE sets a new standard for secure and robust concept erasure in diffusion models.

Method: RAM++ uses three key designs: 1) Adaptive Semantic-Aware Mask (AdaSAM) for pretraining with pixel-level masks on semantically rich regions, 2) Mask Attribute Conductance (MAC) for selective fine-tuning of high-contribution layers, and 3) Robust Feature Regularization (RFR) that leverages DINOv2’s degradation-invariant representations with efficient feature fusion.

Result: RAM++ achieves robust, well-balanced, and state-of-the-art performance across seen, unseen, extreme, and mixed degradations.

Conclusion: The framework successfully integrates high-level semantic understanding with low-level texture generation for content-oriented robust restoration, addressing limitations of existing methods through its adaptive masking and regularization strategies.

Abstract: This work presents Robust Representation Learning via Adaptive Mask (RAM++), a two-stage framework for all-in-one image restoration. RAM++ integrates high-level semantic understanding with low-level texture generation to achieve content-oriented robust restoration. It addresses the limitations of existing degradation-oriented methods in extreme scenarios (e.g., degradations strongly coupled with image structures). RAM++ also mitigates common challenges such as unbalanced performance across tasks, overfitting to seen degradations, and weak generalization to unseen ones through three key designs: 1) Adaptive Semantic-Aware Mask (AdaSAM): a pretraining strategy that applies pixel-level masks to semantically rich and textured regions. This design enables the network to learn both generative priors and image content priors from various degradations. 2) Mask Attribute Conductance (MAC): a selective fine-tuning strategy that adjusts the layers with higher contributions to bridge the integrity gap between masked pretraining and full-image fine-tuning while retaining learned priors. 3) Robust Feature Regularization (RFR): a strategy that leverages DINOv2’s semantically consistent and degradation-invariant representations, together with efficient feature fusion, to achieve faithful and semantically coherent restoration. With these designs, RAM++ achieves robust, well-balanced, and state-of-the-art performance across seen, unseen, extreme, and mixed degradations. Our code and model will be released at https://github.com/DragonisCV/RAM

Method: Proposed RSKT-Seg framework with three key components: (1) Multi-Directional Cost Map Aggregation (RS-CMA) for rotation-invariant features, (2) Efficient Cost Map Fusion (RS-Fusion) transformer for spatial-semantic modeling, and (3) Remote Sensing Knowledge Transfer (RS-Transfer) module for domain adaptation via enhanced upsampling.

Result: RSKT-Seg consistently outperforms strong OVS baselines by +3.8 mIoU and +5.9 mACC, while achieving 2x faster inference through efficient aggregation on the newly established OVRSISBench benchmark.

Conclusion: The proposed RSKT-Seg framework effectively addresses the unique challenges of remote sensing open-vocabulary segmentation, demonstrating superior performance and efficiency compared to existing methods, providing a strong baseline for future research in this domain.

Abstract: Open-Vocabulary Remote Sensing Image Segmentation (OVRSIS), an emerging task that adapts Open-Vocabulary Segmentation (OVS) to the remote sensing (RS) domain, remains underexplored due to the absence of a unified evaluation benchmark and the domain gap between natural and RS images. To bridge these gaps, we first establish a standardized OVRSIS benchmark (\textbf{OVRSISBench}) based on widely-used RS segmentation datasets, enabling consistent evaluation across methods. Using this benchmark, we comprehensively evaluate several representative OVS/OVRSIS models and reveal their limitations when directly applied to remote sensing scenarios. Building on these insights, we propose \textbf{RSKT-Seg}, a novel open-vocabulary segmentation framework tailored for remote sensing. RSKT-Seg integrates three key components: (1) a Multi-Directional Cost Map Aggregation (RS-CMA) module that captures rotation-invariant visual cues by computing vision-language cosine similarities across multiple directions; (2) an Efficient Cost Map Fusion (RS-Fusion) transformer, which jointly models spatial and semantic dependencies with a lightweight dimensionality reduction strategy; and (3) a Remote Sensing Knowledge Transfer (RS-Transfer) module that injects pre-trained knowledge and facilitates domain adaptation via enhanced upsampling. Extensive experiments on the benchmark show that RSKT-Seg consistently outperforms strong OVS baselines by +3.8 mIoU and +5.9 mACC, while achieving 2x faster inference through efficient aggregation. Our code is \href{https://github.com/LiBingyu01/RSKT-Seg}{\textcolor{blue}{here}}.

Method: Uses structured masking strategy in attention computation to control interactions between global prompt, layout, and image tokens. Incorporates Group Relative Policy Optimization with layout reward functions for post-training.

Result: Achieves seamless integration of layout tokens with text and image tokens without compromising generation quality. Superior layout-aware control while maintaining structural simplicity and efficiency.

Conclusion: SMARLI effectively addresses layout-conditioned generation challenges in AR models through structured masking and reinforcement learning, enabling better layout control without sacrificing generation performance.

Abstract: While autoregressive (AR) models have demonstrated remarkable success in image generation, extending them to layout-conditioned generation remains challenging due to the sparse nature of layout conditions and the risk of feature entanglement. We present Structured Masking for AR-based Layout-to-Image (SMARLI), a novel framework for layoutto-image generation that effectively integrates spatial layout constraints into AR-based image generation. To equip AR model with layout control, a specially designed structured masking strategy is applied to attention computation to govern the interaction among the global prompt, layout, and image tokens. This design prevents mis-association between different regions and their descriptions while enabling sufficient injection of layout constraints into the generation process. To further enhance generation quality and layout accuracy, we incorporate Group Relative Policy Optimization (GRPO) based post-training scheme with specially designed layout reward functions for next-set-based AR models. Experimental results demonstrate that SMARLI is able to seamlessly integrate layout tokens with text and image tokens without compromising generation quality. It achieves superior layoutaware control while maintaining the structural simplicity and generation efficiency of AR models.

Method: Combines zero-shot object detection, motion-aware tracking and segmentation, and vision transformers for feature extraction in a modular pipeline designed to handle animal occlusions and group housing scenarios.

Result: Achieved 94.2% overall accuracy (21.2% improvement over existing methods), 93.3% identity preservation score, and 89.3% object detection precision on the Edinburgh Pig Behavior Video Dataset.

Conclusion: The pipeline provides a scalable, automated solution for behavior monitoring with potential for adaptation to other contexts, contributing to precision farming and welfare assessment through objective continuous analysis.

Abstract: Animal behavior analysis plays a crucial role in understanding animal welfare, health status, and productivity in agricultural settings. However, traditional manual observation methods are time-consuming, subjective, and limited in scalability. We present a modular pipeline that leverages open-sourced state-of-the-art computer vision techniques to automate animal behavior analysis in a group housing environment. Our approach combines state-of-the-art models for zero-shot object detection, motion-aware tracking and segmentation, and advanced feature extraction using vision transformers for robust behavior recognition. The pipeline addresses challenges including animal occlusions and group housing scenarios as demonstrated in indoor pig monitoring. We validated our system on the Edinburgh Pig Behavior Video Dataset for multiple behavioral tasks. Our temporal model achieved 94.2% overall accuracy, representing a 21.2 percentage point improvement over existing methods. The pipeline demonstrated robust tracking capabilities with 93.3% identity preservation score and 89.3% object detection precision. The modular design suggests potential for adaptation to other contexts, though further validation across species would be required. The open-source implementation provides a scalable solution for behavior monitoring, contributing to precision pig farming and welfare assessment through automated, objective, and continuous analysis.

Method: Two-stage autoregressive framework: 1) Facial Keyframe Generation using phoneme representations with Text-Frame Causal Attention, 2) Inter-frame interpolation with timestamp-aware adaptive strategy using selective state space models for bidirectional context reasoning.

Result: Outperforms existing methods in visual fidelity, temporal consistency, and computational efficiency, with optimized inference pipeline reducing latency while maintaining visual quality.

Conclusion: AvatarSync provides a scalable and controllable solution for talking-head animation that effectively addresses the limitations of previous approaches through its phoneme-based autoregressive framework and two-stage generation strategy.

Abstract: Existing talking-head animation approaches based on Generative Adversarial Networks (GANs) or diffusion models often suffer from inter-frame flicker, identity drift, and slow inference. These limitations inherent to their video generation pipelines restrict their suitability for applications. To address this, we introduce AvatarSync, an autoregressive framework on phoneme representations that generates realistic and controllable talking-head animations from a single reference image, driven directly text or audio input. In addition, AvatarSync adopts a two-stage generation strategy, decoupling semantic modeling from visual dynamics, which is a deliberate “Divide and Conquer” design. The first stage, Facial Keyframe Generation (FKG), focuses on phoneme-level semantic representation by leveraging the many-to-one mapping from text or audio to phonemes. A Phoneme-to-Visual Mapping is constructed to anchor abstract phonemes to character-level units. Combined with a customized Text-Frame Causal Attention Mask, the keyframes are generated. The second stage, inter-frame interpolation, emphasizes temporal coherence and visual smoothness. We introduce a timestamp-aware adaptive strategy based on a selective state space model, enabling efficient bidirectional context reasoning. To support deployment, we optimize the inference pipeline to reduce latency without compromising visual fidelity. Extensive experiments show that AvatarSync outperforms existing talking-head animation methods in visual fidelity, temporal consistency, and computational efficiency, providing a scalable and controllable solution.

Method: Integrates Mamba2 state space models into U-Net architecture, adds interactive click prompts with cross-attention blocks, uses self-supervised pre-training, and incorporates dental domain knowledge.

Result: Achieved mean Dice of 0.792 and HD95 of 93.19 in Task 1, and mean Dice of 0.852 and HD95 of 7.39 in Task 2, securing top 3 positions in both tasks of the ToothFairy3 challenge.

Conclusion: U-Mamba2 demonstrates both effectiveness and efficiency in dental CBCT segmentation, providing a strong solution that balances performance with computational efficiency through architectural innovations.

Abstract: Cone-Beam Computed Tomography (CBCT) is a widely used 3D imaging technique in dentistry, providing volumetric information about the anatomical structures of jaws and teeth. Accurate segmentation of these anatomies is critical for clinical applications such as diagnosis and surgical planning, but remains time-consuming and challenging. In this paper, we present U-Mamba2, a new neural network architecture designed for multi-anatomy CBCT segmentation in the context of the ToothFairy3 challenge. U-Mamba2 integrates the Mamba2 state space models into the U-Net architecture, enforcing stronger structural constraints for higher efficiency without compromising performance. In addition, we integrate interactive click prompts with cross-attention blocks, pre-train U-Mamba2 using self-supervised learning, and incorporate dental domain knowledge into the model design to address key challenges of dental anatomy segmentation in CBCT. Extensive experiments, including independent tests, demonstrate that U-Mamba2 is both effective and efficient, securing top 3 places in both tasks of the Toothfairy3 challenge. In Task 1, U-Mamba2 achieved a mean Dice of 0.792, HD95 of 93.19 with the held-out test data, with an average inference time of XX (TBC during the ODIN workshop). In Task 2, U-Mamba2 achieved the mean Dice of 0.852 and HD95 of 7.39 with the held-out test data. The code is publicly available at https://github.com/zhiqin1998/UMamba2.

Method: Developed a fetal-specific augmentation strategy that simulates the distinct intrauterine environment and fetal positioning of early gestational ages, using only annotated images from older GA cohorts.

Result: Cross-population augmentation yields reduced variability and significant improvements across both older GA and challenging early GA cases, enabling more reliable pose estimation across gestation.

Conclusion: This framework facilitates early clinical detection and intervention in challenging 4D fetal imaging settings by enabling robust generalization to younger gestational age clinical cohorts.

Abstract: Fetal motion is a critical indicator of neurological development and intrauterine health, yet its quantification remains challenging, particularly at earlier gestational ages (GA). Current methods track fetal motion by predicting the location of annotated landmarks on 3D echo planar imaging (EPI) time-series, primarily in third-trimester fetuses. The predicted landmarks enable simplification of the fetal body for downstream analysis. While these methods perform well within their training age distribution, they consistently fail to generalize to early GAs due to significant anatomical changes in both mother and fetus across gestation, as well as the difficulty of obtaining annotated early GA EPI data. In this work, we develop a cross-population data augmentation framework that enables pose estimation models to robustly generalize to younger GA clinical cohorts using only annotated images from older GA cohorts. Specifically, we introduce a fetal-specific augmentation strategy that simulates the distinct intrauterine environment and fetal positioning of early GAs. Our experiments find that cross-population augmentation yields reduced variability and significant improvements across both older GA and challenging early GA cases. By enabling more reliable pose estimation across gestation, our work potentially facilitates early clinical detection and intervention in challenging 4D fetal imaging settings. Code is available at https://github.com/sebodiaz/cross-population-pose.

Method: ImplMORe incorporates local features using a 3D CNN encoder and performs multi-scale interpolation to learn features in the continuous domain using occupancy functions. The method is applied for single and multiple organ reconstructions using the totalsegmentator dataset.

Result: The approach outperforms discrete explicit representation based surface reconstruction methods by leveraging the continuous nature of occupancy functions, providing fine-grained surface details of organs at a resolution higher than the given input image.

Conclusion: ImplMORe successfully addresses the limitations of traditional surface reconstruction methods in medical imaging by using implicit representations, enabling higher-resolution organ reconstruction with better detail preservation than input resolution constraints would typically allow.

Abstract: The fine-grained surface reconstruction of different organs from 3D medical imaging can provide advanced diagnostic support and improved surgical planning. However, the representation of the organs is often limited by the resolution, with a detailed higher resolution requiring more memory and computing footprint. Implicit representations of objects have been proposed to alleviate this problem in general computer vision by providing compact and differentiable functions to represent the 3D object shapes. However, architectural and data-related differences prevent the direct application of these methods to medical images. This work introduces ImplMORe, an end-to-end deep learning method using implicit surface representations for multi-organ reconstruction from 3D medical images. ImplMORe incorporates local features using a 3D CNN encoder and performs multi-scale interpolation to learn the features in the continuous domain using occupancy functions. We apply our method for single and multiple organ reconstructions using the totalsegmentator dataset. By leveraging the continuous nature of occupancy functions, our approach outperforms the discrete explicit representation based surface reconstruction approaches, providing fine-grained surface details of the organ at a resolution higher than the given input image. The source code will be made publicly available at: https://github.com/CAMMA-public/ImplMORe

Method: Proposes a trajectory-controllable unfolding framework inspired by diffusion trajectories and flow matching, with an efficient spatial-spectral Transformer and frequency-domain fusion module for feature consistency.

Result: Experiments on simulation and real data demonstrate better reconstruction quality and efficiency compared to prior state-of-the-art approaches.

Conclusion: The proposed framework successfully addresses the trajectory control issue in CASSI reconstruction, achieving superior performance through smooth optimization paths and efficient spatial-spectral processing.

Abstract: Coded aperture snapshot spectral imaging (CASSI) retrieves a 3D hyperspectral image (HSI) from a single 2D compressed measurement, which is a highly challenging reconstruction task. Recent deep unfolding networks (DUNs), empowered by explicit data-fidelity updates and implicit deep denoisers, have achieved the state of the art in CASSI reconstruction. However, existing unfolding approaches suffer from uncontrollable reconstruction trajectories, leading to abrupt quality jumps and non-gradual refinement across stages. Inspired by diffusion trajectories and flow matching, we propose a novel trajectory-controllable unfolding framework that enforces smooth, continuous optimization paths from noisy initial estimates to high-quality reconstructions. To achieve computational efficiency, we design an efficient spatial-spectral Transformer tailored for hyperspectral reconstruction, along with a frequency-domain fusion module to gurantee feature consistency. Experiments on simulation and real data demonstrate that our method achieves better reconstruction quality and efficiency than prior state-of-the-art approaches.

Method: Uses deep deformable tetrahedra (explicit-implicit hybrid representation) with attentive slice-adaptive 2D-3D feature assembly and a two-stage weakly supervised motion learning scheme requiring only keyframe annotations.

Result: Achieves state-of-the-art accuracy and strong generalization on three large public datasets and external evaluation on private interventional datasets.

Conclusion: TetHeart enables accurate 4D cardiac motion reconstruction from both full-stack and sparse-slice scenarios, making it suitable for both pre-procedural and intra-procedural applications.

Abstract: Reconstructing cardiac motion from cine CMR sequences is critical for diagnosis, prediction, and intervention. Existing methods rely on complete CMR stacks to infer full heart motion, limiting their utility in intra-procedural scenarios where only sparse observations are available. We present TetHeart, the first end-to-end framework that unifies full 4D multi-structure heart mesh recovery from both offline full-stack acquisitions and intra-procedural sparse-slice observations. Our method leverages deep deformable tetrahedra, an explicit-implicit hybrid representation, to capture shape and motion in a coherent space shared across cardiac structures. It is initialized from high-quality pre-procedural or offline-acquired full stacks to build detailed, patient-specific heart meshes, which can then be updated using whatever slices are available, from full stacks down to a single slice. We further incorporate several key innovations: (i) an attentive mechanism for slice-adaptive 2D-3D feature assembly that dynamically integrates information from arbitrary numbers of slices at any position, combined with a distillation strategy from full-slice to sparse-slice settings to ensure accurate reconstruction under extreme sparsity; and (ii) a two-stage weakly supervised motion learning scheme requiring only keyframe (e.g., ED and ES) annotations. Trained and validated on three large public datasets and externally evaluated zero-shot on additional private interventional and public CMR datasets, TetHeart achieves state-of-the-art accuracy and strong generalization in both pre- and intra-procedural settings.

Method: Repurpose SAM2’s video modules for few-shot segmentation, apply Low-Rank Adaptation (LoRA) to handle diverse images, and meta-train only a small number of parameters.

Result: Achieves remarkable results on PASCAL-5i, COCO-20i, and FSS-1000 datasets with excellent computational efficiency during inference.

Conclusion: FS-SAM2 effectively adapts SAM2 for few-shot segmentation with minimal parameter updates, demonstrating strong performance and efficiency across multiple benchmarks.

Abstract: Few-shot semantic segmentation has recently attracted great attention. The goal is to develop a model capable of segmenting unseen classes using only a few annotated samples. Most existing approaches adapt a pre-trained model by training from scratch an additional module. Achieving optimal performance with these approaches requires extensive training on large-scale datasets. The Segment Anything Model 2 (SAM2) is a foundational model for zero-shot image and video segmentation with a modular design. In this paper, we propose a Few-Shot segmentation method based on SAM2 (FS-SAM2), where SAM2’s video capabilities are directly repurposed for the few-shot task. Moreover, we apply a Low-Rank Adaptation (LoRA) to the original modules in order to handle the diverse images typically found in standard datasets, unlike the temporally connected frames used in SAM2’s pre-training. With this approach, only a small number of parameters is meta-trained, which effectively adapts SAM2 while benefiting from its impressive segmentation performance. Our method supports any K-shot configuration. We evaluate FS-SAM2 on the PASCAL-5$^i$, COCO-20$^i$ and FSS-1000 datasets, achieving remarkable results and demonstrating excellent computational efficiency during inference. Code is available at https://github.com/fornib/FS-SAM2

Method: The framework fuses semantic segmentation (SegFormer B3 model for rail detection), object detection (fine-tuned YOLOv8 for object localization), and a rule-based Distance Assessor that compares projected distances with standard 1435mm rail gauge to classify risk levels.

Result: Achieved 65% IoU for rail segmentation and 75.6% mAP for object detection at IoU threshold of 0.50 on the RailSem19 dataset, demonstrating accurate real-time scene understanding with minimal annotation requirements.

Conclusion: RailSafeNet provides an effective annotation-light solution for real-time tram safety that can identify track intrusions and warn drivers, potentially preventing accidents in urban tram operations.

Abstract: Tram-human interaction safety is an important challenge, given that trams frequently operate in densely populated areas, where collisions can range from minor injuries to fatal outcomes. This paper addresses the issue from the perspective of designing a solution leveraging digital image processing, deep learning, and artificial intelligence to improve the safety of pedestrians, drivers, cyclists, pets, and tram passengers. We present RailSafeNet, a real-time framework that fuses semantic segmentation, object detection and a rule-based Distance Assessor to highlight track intrusions. Using only monocular video, the system identifies rails, localises nearby objects and classifies their risk by comparing projected distances with the standard 1435mm rail gauge. Experiments on the diverse RailSem19 dataset show that a class-filtered SegFormer B3 model achieves 65% intersection-over-union (IoU), while a fine-tuned YOLOv8 attains 75.6% mean average precision (mAP) calculated at an intersection over union (IoU) threshold of 0.50. RailSafeNet therefore delivers accurate, annotation-light scene understanding that can warn drivers before dangerous situations escalate. Code available at https://github.com/oValach/RailSafeNet.

Method: Developed a unified pipeline using Vision Transformers to extract 3D region embeddings from sMRI data and Graph Neural Networks for classification. Explored two region definition strategies: atlas-based (predefined structural/functional atlases) and cube-based (ViTs trained on 3D patches). Used cosine similarity graphs to model interregional relationships for GNN classification.

Result: Achieved 78.98% accuracy, 76.54% sensitivity, 81.58% specificity, 81.58% precision, and 78.98% F1-score using stratified 10-fold cross-validation on REST-meta-MDD dataset. Atlas-based models consistently outperformed cube-based approach.

Conclusion: The proposed unified pipeline effectively detects MDD using sMRI data, with atlas-based methods showing superior performance, highlighting the importance of domain-specific anatomical priors for accurate MDD detection.

Abstract: Major depressive disorder (MDD) is a prevalent mental health condition that negatively impacts both individual well-being and global public health. Automated detection of MDD using structural magnetic resonance imaging (sMRI) and deep learning (DL) methods holds increasing promise for improving diagnostic accuracy and enabling early intervention. Most existing methods employ either voxel-level features or handcrafted regional representations built from predefined brain atlases, limiting their ability to capture complex brain patterns. This paper develops a unified pipeline that utilizes Vision Transformers (ViTs) for extracting 3D region embeddings from sMRI data and Graph Neural Network (GNN) for classification. We explore two strategies for defining regions: (1) an atlas-based approach using predefined structural and functional brain atlases, and (2) an cube-based method by which ViTs are trained directly to identify regions from uniformly extracted 3D patches. Further, cosine similarity graphs are generated to model interregional relationships, and guide GNN-based classification. Extensive experiments were conducted using the REST-meta-MDD dataset to demonstrate the effectiveness of our model. With stratified 10-fold cross-validation, the best model obtained 78.98% accuracy, 76.54% sensitivity, 81.58% specificity, 81.58% precision, and 78.98% F1-score. Further, atlas-based models consistently outperformed the cube-based approach, highlighting the importance of using domain-specific anatomical priors for MDD detection.

Method: Proposes OpenHOUSE system that uses LLMs to group atomic actions into higher-level events and features a specialized streaming module for accurate boundary detection between closely adjacent actions.

Result: The specialized streaming module nearly doubles the performance of direct extensions of existing methods for detecting boundaries between closely adjacent actions.

Conclusion: OpenHOUSE represents a key step toward integrating powerful generative models in streaming action perception, enabling hierarchical online understanding of video events with free-form descriptions.

Abstract: We introduce Hierarchical Streaming Video Understanding, a task that combines online temporal action localization with free-form description generation. Given the scarcity of datasets with hierarchical and fine-grained temporal annotations, we demonstrate that LLMs can effectively group atomic actions into higher-level events, enriching existing datasets. We then propose OpenHOUSE (Open-ended Hierarchical Online Understanding System for Events), which extends streaming action perception beyond action classification. OpenHOUSE features a specialized streaming module that accurately detects boundaries between closely adjacent actions, nearly doubling the performance of direct extensions of existing methods. We envision the future of streaming action perception in the integration of powerful generative models, with OpenHOUSE representing a key step in that direction.

Method: Self-supervised learning on a large private dataset of 1.15 million X-ray images spanning diverse anatomical regions, evaluated across 12 datasets and 20 downstream tasks including classification, retrieval, segmentation, localization, visual grounding, and report generation.

Result: XR-0 achieves state-of-the-art performance on most multi-anatomy tasks and remains competitive on chest-specific benchmarks.

Conclusion: Anatomical diversity and supervision are critical for building robust, general-purpose medical vision models, paving the way for scalable and adaptable AI systems in radiology.

Abstract: X-ray imaging is a ubiquitous in radiology, yet most existing AI foundation models are limited to chest anatomy and fail to generalize across broader clinical tasks. In this work, we introduce XR-0, the multi-anatomy X-ray foundation model using self-supervised learning on a large, private dataset of 1.15 million images spanning diverse anatomical regions and evaluated across 12 datasets and 20 downstream tasks, including classification, retrieval, segmentation, localization, visual grounding, and report generation. XR-0 achieves state-of-the-art performance on most multi-anatomy tasks and remains competitive on chest-specific benchmarks. Our results demonstrate that anatomical diversity and supervision are critical for building robust, general-purpose medical vision models, paving the way for scalable and adaptable AI systems in radiology.

Method: Compared LoRA with traditional full fine-tuning and inference-only methods using DinoV2 and SwinV2 models. Used cropped CT images of different sizes (50mm³ and 75mm³) centered at treatment isocenter, and tested different adaptation techniques for 2D models on 3D data.

Result: LoRA achieved comparable or superior performance to traditional fine-tuning while significantly reducing computational costs and training times by requiring fewer trainable parameters.

Conclusion: LoRA is an efficient and effective alternative to traditional full fine-tuning for medical image analysis tasks, offering similar performance with reduced computational requirements.

Abstract: This study investigates the efficacy of Low-Rank Adaptation (LoRA) for fine-tuning large Vision Models, DinoV2 and SwinV2, to diagnose Radiation-Induced Lung Injury (RILI) from X-ray CT scans following Stereotactic Body Radiation Therapy (SBRT). To evaluate the robustness and efficiency of this approach, we compare LoRA with traditional full fine-tuning and inference-only (no fine-tuning) methods. Cropped images of two sizes (50 mm3 and 75 mm3), centered at the treatment isocenter, in addition to different adaptation techniques for adapting the 2D LVMs for 3D data were used to determine the sensitivity of the models to spatial context. Experimental results show that LoRA achieves comparable or superior performance to traditional fine-tuning while significantly reducing computational costs and training times by requiring fewer trainable parameters.

Method: Uses an implicit training paradigm combining large-scale real video data and small-scale synthetic 3D data to optimize a shared garment embedding space. Creates a garment “atlas” representation during inference by finetuning on specific real-world videos.

Result: Achieves state-of-the-art performance on novel view synthesis of in-the-wild garments, robustly handling wrinkling, pose variation, occlusion while maintaining photorealism, view consistency, texture details, and accurate geometry.

Conclusion: HoloGarment successfully bridges the domain gap between synthetic and real data, enabling high-quality 360° novel view synthesis of garments from real-world images and videos with challenging artifacts.

Abstract: Novel view synthesis (NVS) of in-the-wild garments is a challenging task due significant occlusions, complex human poses, and cloth deformations. Prior methods rely on synthetic 3D training data consisting of mostly unoccluded and static objects, leading to poor generalization on real-world clothing. In this paper, we propose HoloGarment (Hologram-Garment), a method that takes 1-3 images or a continuous video of a person wearing a garment and generates 360{\deg} novel views of the garment in a canonical pose. Our key insight is to bridge the domain gap between real and synthetic data with a novel implicit training paradigm leveraging a combination of large-scale real video data and small-scale synthetic 3D data to optimize a shared garment embedding space. During inference, the shared embedding space further enables dynamic video-to-360{\deg} NVS through the construction of a garment “atlas” representation by finetuning a garment embedding on a specific real-world video. The atlas captures garment-specific geometry and texture across all viewpoints, independent of body pose or motion. Extensive experiments show that HoloGarment achieves state-of-the-art performance on NVS of in-the-wild garments from images and videos. Notably, our method robustly handles challenging real-world artifacts – such as wrinkling, pose variation, and occlusion – while maintaining photorealism, view consistency, fine texture details, and accurate geometry. Visit our project page for additional results: https://johannakarras.github.io/HoloGarment

Method: Utilized self-supervised learning with a pretrained V-JEPA model and applied domain-adaptive pretraining (DAP) by continuing pretraining with in-domain data without requiring labeled samples.

Result: Outperformed published state-of-the-art action recognition models by 6.1% accuracy on PanAf dataset and 6.3% mAP on ChimpACT dataset, with most performance gains coming from DAP.

Conclusion: The method shows great potential for improving animal behavior recognition as DAP doesn’t require labeled samples, making it highly efficient for large-scale video camera trap data analysis.

Abstract: Computer vision for animal behavior offers promising tools to aid research in ecology, cognition, and to support conservation efforts. Video camera traps allow for large-scale data collection, but high labeling costs remain a bottleneck to creating large-scale datasets. We thus need data-efficient learning approaches. In this work, we show that we can utilize self-supervised learning to considerably improve action recognition on primate behavior. On two datasets of great ape behavior (PanAf and ChimpACT), we outperform published state-of-the-art action recognition models by 6.1 %pt. accuracy and 6.3 %pt. mAP, respectively. We achieve this by utilizing a pretrained V-JEPA model and applying domain-adaptive pretraining (DAP), i.e. continuing the pretraining with in-domain data. We show that most of the performance gain stems from the DAP. Our method promises great potential for improving the recognition of animal behavior, as DAP does not require labeled samples. Code is available at https://github.com/ecker-lab/dap-behavior

Method: Conducted comprehensive literature review, proposed structured taxonomy to classify methods, performed quantitative dataset comparisons, compiled unified evaluation metrics, and established benchmark tables for fair comparisons.

Result: Created a systematic framework for analyzing LiDAR-based human pose and shape estimation methods, provided standardized evaluation metrics, established benchmark performance tables, and identified key challenges and research directions.

Conclusion: This review provides essential foundations for the field through taxonomy, benchmarks, and standardized evaluation, while highlighting critical open challenges that need to be addressed to advance LiDAR-based 3D human understanding.

Abstract: In this paper, we present a comprehensive review of 3D human pose estimation and human mesh recovery from in-the-wild LiDAR point clouds. We compare existing approaches across several key dimensions, and propose a structured taxonomy to classify these methods. Following this taxonomy, we analyze each method’s strengths, limitations, and design choices. In addition, (i) we perform a quantitative comparison of the three most widely used datasets, detailing their characteristics; (ii) we compile unified definitions of all evaluation metrics; and (iii) we establish benchmark tables for both tasks on these datasets to enable fair comparisons and promote progress in the field. We also outline open challenges and research directions critical for advancing LiDAR-based 3D human understanding. Moreover, we maintain an accompanying webpage that organizes papers according to our taxonomy and continuously update it with new studies: https://github.com/valeoai/3D-Human-Pose-Shape-Estimation-from-LiDAR

Method: Introduces OmniWorld dataset consisting of newly collected OmniWorld-Game data and curated public datasets across diverse domains, providing richer modality coverage and more realistic dynamic interactions than existing synthetic datasets.

Result: Fine-tuning state-of-the-art methods on OmniWorld leads to significant performance gains in 4D reconstruction and video generation tasks, and the benchmark exposes limitations of current approaches in complex 4D environments.

Conclusion: OmniWorld serves as a powerful resource for training and evaluation, acting as a catalyst for developing general-purpose 4D world models and advancing machines’ understanding of the physical world.

Abstract: The field of 4D world modeling - aiming to jointly capture spatial geometry and temporal dynamics - has witnessed remarkable progress in recent years, driven by advances in large-scale generative models and multimodal learning. However, the development of truly general 4D world models remains fundamentally constrained by the availability of high-quality data. Existing datasets and benchmarks often lack the dynamic complexity, multi-domain diversity, and spatial-temporal annotations required to support key tasks such as 4D geometric reconstruction, future prediction, and camera-control video generation. To address this gap, we introduce OmniWorld, a large-scale, multi-domain, multi-modal dataset specifically designed for 4D world modeling. OmniWorld consists of a newly collected OmniWorld-Game dataset and several curated public datasets spanning diverse domains. Compared with existing synthetic datasets, OmniWorld-Game provides richer modality coverage, larger scale, and more realistic dynamic interactions. Based on this dataset, we establish a challenging benchmark that exposes the limitations of current state-of-the-art (SOTA) approaches in modeling complex 4D environments. Moreover, fine-tuning existing SOTA methods on OmniWorld leads to significant performance gains across 4D reconstruction and video generation tasks, strongly validating OmniWorld as a powerful resource for training and evaluation. We envision OmniWorld as a catalyst for accelerating the development of general-purpose 4D world models, ultimately advancing machines’ holistic understanding of the physical world.

Method: LazyDrag generates explicit correspondence maps from user drag inputs as reliable references to boost attention control, enabling stable full-strength inversion and eliminating the need for test-time optimization.

Result: LazyDrag outperforms baselines in drag accuracy and perceptual quality on DragBench, validated by VIEScore and human evaluation. It enables complex edits like opening a dog’s mouth, generating new objects, and context-aware changes for ambiguous drags.

Conclusion: LazyDrag establishes new state-of-the-art performance and paves a new way for editing paradigms by unifying precise geometric control with text guidance without requiring test-time optimization.

Abstract: The reliance on implicit point matching via attention has become a core bottleneck in drag-based editing, resulting in a fundamental compromise on weakened inversion strength and costly test-time optimization (TTO). This compromise severely limits the generative capabilities of diffusion models, suppressing high-fidelity inpainting and text-guided creation. In this paper, we introduce LazyDrag, the first drag-based image editing method for Multi-Modal Diffusion Transformers, which directly eliminates the reliance on implicit point matching. In concrete terms, our method generates an explicit correspondence map from user drag inputs as a reliable reference to boost the attention control. This reliable reference opens the potential for a stable full-strength inversion process, which is the first in the drag-based editing task. It obviates the necessity for TTO and unlocks the generative capability of models. Therefore, LazyDrag naturally unifies precise geometric control with text guidance, enabling complex edits that were previously out of reach: opening the mouth of a dog and inpainting its interior, generating new objects like a ``tennis ball’’, or for ambiguous drags, making context-aware changes like moving a hand into a pocket. Additionally, LazyDrag supports multi-round workflows with simultaneous move and scale operations. Evaluated on the DragBench, our method outperforms baselines in drag accuracy and perceptual quality, as validated by VIEScore and human evaluation. LazyDrag not only establishes new state-of-the-art performance, but also paves a new way to editing paradigms.

Method: LEGO bricks integrate Local-feature Enrichment (MLP) and Global-content Orchestration (Transformer blocks) in a stackable architecture that maintains full-resolution images across all bricks, allowing test-time reconfiguration.

Result: LEGO bricks improve training efficiency, accelerate convergence, enable variable-resolution generation, maintain strong generative performance, and significantly reduce sampling time compared to other methods.

Conclusion: LEGO bricks provide a valuable enhancement for diffusion models by offering an efficient, adaptable backbone that addresses computational challenges while maintaining performance.

Abstract: Diffusion models excel at generating photo-realistic images but come with significant computational costs in both training and sampling. While various techniques address these computational challenges, a less-explored issue is designing an efficient and adaptable network backbone for iterative refinement. Current options like U-Net and Vision Transformer often rely on resource-intensive deep networks and lack the flexibility needed for generating images at variable resolutions or with a smaller network than used in training. This study introduces LEGO bricks, which seamlessly integrate Local-feature Enrichment and Global-content Orchestration. These bricks can be stacked to create a test-time reconfigurable diffusion backbone, allowing selective skipping of bricks to reduce sampling costs and generate higher-resolution images than the training data. LEGO bricks enrich local regions with an MLP and transform them using a Transformer block while maintaining a consistent full-resolution image across all bricks. Experimental results demonstrate that LEGO bricks enhance training efficiency, expedite convergence, and facilitate variable-resolution image generation while maintaining strong generative performance. Moreover, LEGO significantly reduces sampling time compared to other methods, establishing it as a valuable enhancement for diffusion models. Our code and project page are available at https://jegzheng.github.io/LEGODiffusion.

Method: Automatic construction of audio-visual character bank from online sources containing visual exemplars and voice samples for multi-modal recognition.

Result: Significant improvements in accessibility and narrative comprehension over face-detection-based approaches, supported by new CMD-AM dataset of 75 animated movies.

Conclusion: The character-centric pipeline enables robust animated character recognition and enhances accessibility applications for both visually and hearing impaired audiences.

Abstract: Animated movies are captivating for their unique character designs and imaginative storytelling, yet they pose significant challenges for existing recognition systems. Unlike the consistent visual patterns detected by conventional face recognition methods, animated characters exhibit extreme diversity in their appearance, motion, and deformation. In this work, we propose an audio-visual pipeline to enable automatic and robust animated character recognition, and thereby enhance character-centric understanding of animated movies. Central to our approach is the automatic construction of an audio-visual character bank from online sources. This bank contains both visual exemplars and voice (audio) samples for each character, enabling subsequent multi-modal character recognition despite long-tailed appearance distributions. Building on accurate character recognition, we explore two downstream applications: Audio Description (AD) generation for visually impaired audiences, and character-aware subtitling for the hearing impaired. To support research in this domain, we introduce CMD-AM, a new dataset of 75 animated movies with comprehensive annotations. Our character-centric pipeline demonstrates significant improvements in both accessibility and narrative comprehension for animated content over prior face-detection-based approaches. For the code and dataset, visit https://www.robots.ox.ac.uk/~vgg/research/animated_ad/.

Method: Proposes LS-HAN framework with three components: two-stream CNN for video feature extraction, latent space to bridge semantic gaps, and hierarchical attention network for recognition without temporal segmentation.

Result: Experimental results on two large-scale datasets demonstrate the effectiveness of the proposed framework in continuous sign language recognition.

Conclusion: The LS-HAN framework successfully addresses challenges in continuous sign language recognition by eliminating preprocessing steps and handling semantic gaps, showing promising results for automatic sign language translation.

Abstract: Millions of hearing impaired people around the world routinely use some variants of sign languages to communicate, thus the automatic translation of a sign language is meaningful and important. Currently, there are two sub-problems in Sign Language Recognition (SLR), i.e., isolated SLR that recognizes word by word and continuous SLR that translates entire sentences. Existing continuous SLR methods typically utilize isolated SLRs as building blocks, with an extra layer of preprocessing (temporal segmentation) and another layer of post-processing (sentence synthesis). Unfortunately, temporal segmentation itself is non-trivial and inevitably propagates errors into subsequent steps. Worse still, isolated SLR methods typically require strenuous labeling of each word separately in a sentence, severely limiting the amount of attainable training data. To address these challenges, we propose a novel continuous sign recognition framework, the Hierarchical Attention Network with Latent Space (LS-HAN), which eliminates the preprocessing of temporal segmentation. The proposed LS-HAN consists of three components: a two-stream Convolutional Neural Network (CNN) for video feature representation generation, a Latent Space (LS) for semantic gap bridging, and a Hierarchical Attention Network (HAN) for latent space based recognition. Experiments are carried out on two large scale datasets. Experimental results demonstrate the effectiveness of the proposed framework.

Method: Proposed Sparse Adversarial and Interpretable Attack Framework (SAIF) that uses Frank-Wolfe (conditional gradient) algorithm to optimize attack perturbations with bounded magnitude and sparsity constraints, achieving O(1/√T) convergence.

Result: SAIF computes highly imperceptible and interpretable adversarial examples and outperforms state-of-the-art sparse attack methods on the ImageNet dataset.

Conclusion: SAIF provides an effective framework for generating sparse, imperceptible adversarial attacks that reveal classifier vulnerabilities while maintaining interpretability through limited pixel perturbations.

Abstract: Adversarial attacks hamper the decision-making ability of neural networks by perturbing the input signal. The addition of calculated small distortion to images, for instance, can deceive a well-trained image classification network. In this work, we propose a novel attack technique called Sparse Adversarial and Interpretable Attack Framework (SAIF). Specifically, we design imperceptible attacks that contain low-magnitude perturbations at a small number of pixels and leverage these sparse attacks to reveal the vulnerability of classifiers. We use the Frank-Wolfe (conditional gradient) algorithm to simultaneously optimize the attack perturbations for bounded magnitude and sparsity with $O(1/\sqrt{T})$ convergence. Empirical results show that SAIF computes highly imperceptible and interpretable adversarial examples, and outperforms state-of-the-art sparse attack methods on the ImageNet dataset.

Method: Develop DPConv that uses deep features from encoder layers in reconstruction tasks to generate adaptive convolution kernels. Integrate DPConv into a siamese reconstruction-segmentation network (SRS) where reconstruction network generates kernels for segmentation network.

Result: Extensive experiments on 7 datasets (5 medical, 2 infrared) show superior performance over recent methods. Zero-shot segmentation on unseen modalities demonstrates strong generalization capability.

Conclusion: DPConv effectively addresses dynamic convolution limitations for specialized image segmentation tasks. The siamese reconstruction-segmentation framework with DPConv significantly enhances segmentation performance and shows excellent generalization across modalities.

Abstract: Dynamic convolution demonstrates outstanding representation capabilities, which are crucial for natural image segmentation. However, it fails when applied to medical image segmentation (MIS) and infrared small target segmentation (IRSTS) due to limited data and limited fitting capacity. In this paper, we propose a new type of dynamic convolution called dynamic parameter convolution (DPConv) which shows superior fitting capacity, and it can efficiently leverage features from deep layers of encoder in reconstruction tasks to generate DPConv kernels that adapt to input variations.Moreover, we observe that DPConv, built upon deep features derived from reconstruction tasks, significantly enhances downstream segmentation performance. We refer to the segmentation network integrated with DPConv generated from reconstruction network as the siamese reconstruction-segmentation network (SRS). We conduct extensive experiments on seven datasets including five medical datasets and two infrared datasets, and the experimental results demonstrate that our method can show superior performance over several recently proposed methods. Furthermore, the zero-shot segmentation under unseen modality demonstrates the generalization of DPConv. The code is available at: https://github.com/fidshu/SRSNet.

Method: Proposes hierarchical losses for feature sharing across classes, diagnostic metrics for partial credit, and multi-modal late fusion (MMLF) combining independently trained LiDAR and RGB detectors with careful matching and fusion strategies.

Result: MMLF significantly outperforms prior work, improving detection of the six rarest classes from 12.8 to 20.0 mAP, with 2D RGB detectors proving better than 3D for rare classes and 2D matching mitigating depth errors.

Conclusion: The proposed MMLF framework effectively addresses long-tailed 3D detection by leveraging uni-modal datasets and optimized fusion strategies, providing substantial improvements for rare class detection crucial for real-world AV safety.

Abstract: Contemporary autonomous vehicle (AV) benchmarks have advanced techniques for training 3D detectors. While class labels naturally follow a long-tailed distribution in the real world, existing benchmarks only focus on a few common classes (e.g., pedestrian and car) and neglect many rare but crucial classes (e.g., emergency vehicle and stroller). However, AVs must reliably detect both common and rare classes for safe operation in the open world. We address this challenge by formally studying the problem of Long-Tailed 3D Detection (LT3D), which evaluates all annotated classes, including those in-the-tail. We address LT3D with hierarchical losses that promote feature sharing across classes, and introduce diagnostic metrics that award partial credit to “reasonable” mistakes with respect to the semantic hierarchy. Further, we point out that rare-class accuracy is particularly improved via multi-modal late fusion (MMLF) of independently trained uni-modal LiDAR and RGB detectors. Such an MMLF framework allows us to leverage large-scale uni-modal datasets (with more examples for rare classes) to train better uni-modal detectors. Finally, we examine three critical components of our simple MMLF approach from first principles: whether to train 2D or 3D RGB detectors for fusion, whether to match RGB and LiDAR detections in 3D or the projected 2D image plane, and how to fuse matched detections. Extensive experiments reveal that 2D RGB detectors achieve better recognition accuracy for rare classes than 3D RGB detectors, matching on the 2D image plane mitigates depth estimation errors for better matching, and score calibration and probabilistic fusion notably improves the final performance further. Our MMLF significantly outperforms prior work for LT3D, particularly improving on the six rarest classes from 12.8 to 20.0 mAP! Our code and models are available on our project page.

Method: Hierarchical Bayesian inference with variational auto-encoding architecture that explicitly disentangles common anatomy and geometric variations as latent variables. The method performs unsupervised closed-loop self-reconstruction without requiring complex image-based similarity measures.

Result: Superior performance over conventional similarity-based approaches in accuracy, efficiency, scalability, and interpretability across four medical image datasets (cardiac, brain, abdominal). The inferred structural representations capture latent anatomy with visual semantics.

Conclusion: The proposed Bayesian framework provides an effective, scalable, and interpretable solution for multi-modal groupwise image registration, demonstrating significant advantages over traditional methods while enabling meaningful anatomical representations.

Abstract: This article presents a general Bayesian learning framework for multi-modal groupwise image registration. The method builds on probabilistic modelling of the image generative process, where the underlying common anatomy and geometric variations of the observed images are explicitly disentangled as latent variables. Therefore, groupwise image registration is achieved via hierarchical Bayesian inference. We propose a novel hierarchical variational auto-encoding architecture to realise the inference procedure of the latent variables, where the registration parameters can be explicitly estimated in a mathematically interpretable fashion. Remarkably, this new paradigm learns groupwise image registration in an unsupervised closed-loop self-reconstruction process, sparing the burden of designing complex image-based similarity measures. The computationally efficient disentangled network architecture is also inherently scalable and flexible, allowing for groupwise registration on large-scale image groups with variable sizes. Furthermore, the inferred structural representations from multi-modal images via disentanglement learning are capable of capturing the latent anatomy of the observations with visual semantics. Extensive experiments were conducted to validate the proposed framework, including four different datasets from cardiac, brain, and abdominal medical images. The results have demonstrated the superiority of our method over conventional similarity-based approaches in terms of accuracy, efficiency, scalability, and interpretability.

Method: Developed spatial, categorical, and spatial-categorical joint noise priors specifically for semantic image synthesis to address the mismatch between noised training data distribution and standard normal prior used in inference.

Result: Achieved new state-of-the-art results on Cityscapes (FID 10.53), ADE20K and COCO-Stuff datasets, significantly improving semantic image synthesis quality.

Conclusion: Specialized noise priors effectively resolve the distribution mismatch problem in diffusion models for semantic image synthesis, enabling higher quality and more accurate image generation from semantic masks.

Abstract: Semantic image synthesis (SIS) shows good promises for sensor simulation. However, current best practices in this field, based on GANs, have not yet reached the desired level of quality. As latent diffusion models make significant strides in image generation, we are prompted to evaluate ControlNet, a notable method for its dense control capabilities. Our investigation uncovered two primary issues with its results: the presence of weird sub-structures within large semantic areas and the misalignment of content with the semantic mask. Through empirical study, we pinpointed the cause of these problems as a mismatch between the noised training data distribution and the standard normal prior applied at the inference stage. To address this challenge, we developed specific noise priors for SIS, encompassing spatial, categorical, and a novel spatial-categorical joint prior for inference. This approach, which we have named SCP-Diff, has set new state-of-the-art results in SIS on Cityscapes, ADE20K and COCO-Stuff, yielding a FID as low as 10.53 on Cityscapes. The code and models can be accessed via the project page.

Method: Proposes leveraging diffusion models trained on large datasets to generate photorealistic images from text inputs for augmenting existing datasets with various augmentation strategies.

Result: Not explicitly stated in the abstract, but implies improved generalization capabilities through effective data augmentation techniques.

Conclusion: Generated images from diffusion models can be effectively used for data augmentation to enhance the out-of-domain generalization performance of deep learning models.

Abstract: Deep Learning models are incredibly data-hungry and require very large labeled datasets for supervised learning. As a consequence, these models often suffer from overfitting, limiting their ability to generalize to real-world examples. Recent advancements in diffusion models have enabled the generation of photorealistic images based on textual inputs. Leveraging the substantial datasets used to train these diffusion models, we propose a technique to utilize generated images to augment existing datasets. This paper explores various strategies for effective data augmentation to improve the out-of-domain generalization capabilities of deep learning models.

Method: HSDM uses multiple Hyperspectral Continuous Scan Blocks (HCSB) that link forward/backward scans and enhance information from eight directions through State Space Model. Includes spectral attention mechanism to improve spectral information utilization and mitigate degradation from long-range scanning.

Result: Extensive evaluations show HSDM achieves state-of-the-art performance on HSI denoising benchmarks and surpasses transformer method SERT by 31% in efficiency.

Conclusion: HSDM effectively addresses computational efficiency and long-range modeling limitations in HSI denoising through Mamba-based architecture, demonstrating superior performance and efficiency over existing methods.

Abstract: Effectively modeling global context information in hyperspectral image (HSI) denoising is crucial, but prevailing methods using convolution or transformers still face localized or computational efficiency limitations. Inspired by the emerging Selective State Space Model (Mamba) with nearly linear computational complexity and efficient long-term modeling, we present a novel HSI denoising network named HSIDMamba (HSDM). HSDM is tailored to exploit the capture of potential spatial-spectral dependencies effectively and efficiently for HSI denoising. In particular, HSDM comprises multiple Hyperspectral Continuous Scan Blocks (HCSB) to strengthen spatial-spectral interactions. HCSB links forward and backward scans and enhances information from eight directions through the State Space Model (SSM), strengthening the context representation learning of HSDM and improving denoising performance more effectively. In addition, to enhance the utilization of spectral information and mitigate the degradation problem caused by long-range scanning, spectral attention mechanism. Extensive evaluations against HSI denoising benchmarks validate the superior performance of HSDM, achieving state-of-the-art performance and surpassing the efficiency of the transformer method SERT by 31%.

Method: Translate all multilingual image-text pairs from web crawl to English, re-filter them, and increase multilingual data prevalence in training sets for pre-training.

Result: Outperforms English-only datasets on ImageNet, distribution shifts, retrieval tasks, and 38 DataComp benchmarks. Biggest gains on geographically diverse tasks like GeoDE, particularly Africa.

Conclusion: Multilingual data provides significant performance benefits even for English tasks, and future work should intentionally include multicultural/multilingual data to enhance overall model capabilities.

Abstract: Massive web-crawled image-text datasets lay the foundation for recent progress in multimodal learning. These datasets are designed with the goal of training a model to do well on standard computer vision benchmarks, many of which, however, have been shown to be English-centric (e.g., ImageNet). Consequently, existing data curation techniques gravitate towards using predominantly English image-text pairs and discard many potentially useful non-English samples. Our work questions this practice. Multilingual data is inherently enriching not only because it provides a gateway to learn about culturally salient concepts, but also because it depicts common concepts differently from monolingual data. We thus conduct a systematic study to explore the performance benefits of using more samples of non-English origins with respect to English vision tasks. By translating all multilingual image-text pairs from a raw web crawl to English and re-filtering them, we increase the prevalence of (translated) multilingual data in the resulting training set. Pre-training on this dataset outperforms using English-only or English-dominated datasets on ImageNet, ImageNet distribution shifts, image-English-text retrieval and on average across 38 tasks from the DataComp benchmark. On a geographically diverse task like GeoDE, we also observe improvements across all regions, with the biggest gain coming from Africa. In addition, we quantitatively show that English and non-English data are significantly different in both image and (translated) text space. We hope that our findings motivate future work to be more intentional about including multicultural and multilingual data, not just when non-English or geographically diverse tasks are involved, but to enhance model capabilities at large. All translated captions and metadata (language, CLIP score, etc.) are available on HuggingFace.

Method: Created MaRs-VQA dataset inspired by Raven’s Progressive Matrices and Wechsler Intelligence Scale for Children, then finetuned Qwen2-VCog baseline model with multi-stage cognition reasoning annotations using this training data.

Result: Comparative experiments revealed a significant gap between MLLMs and human intelligence in visual cognition tasks, demonstrating the limitations of current models in complex reasoning.

Conclusion: The release of MaRs-VQA dataset and Qwen2-VCog baseline model will help drive progress toward developing next-generation MLLMs with human-like visual cognition capabilities.

Abstract: Recently, Multimodal Large Language Models (MLLMs) and Vision Language Models (VLMs) have shown great promise in language-guided perceptual tasks such as recognition, segmentation, and object detection. However, their effectiveness in addressing visual cognition problems that require high-level multi-image reasoning and visual working memory is not well-established. One such challenge is matrix reasoning - the cognitive ability to discern relationships among patterns in a set of images and extrapolate to predict subsequent patterns. This skill is crucial during the early neurodevelopmental stages of children. Inspired by the matrix reasoning tasks in Raven’s Progressive Matrices (RPM) and Wechsler Intelligence Scale for Children (WISC), we propose a new dataset MaRs-VQA to evaluate the visual cognition capability of MLLMs and compare their performance with existing human visual cognition studies. Based on the training data of MaRs-VQA, we also finetune a baseline model Qwen2-VCog with multi-stage cognition reasoning annotations. Our comparative experiments with different baselines reveal a gap between MLLMs and human intelligence, highlighting the visual cognitive limitations of current MLLMs. We believe that the public release of MaRs-VQA and the Qwen2-VCog baseline model will drive progress toward the next generation of MLLMs with human-like visual cognition abilities. MaRs-VQA is available at huggingface.co/datasets/IrohXu/VCog-Bench. The training code of Qwen2-VCog is available at github.com/IrohXu/Cognition-MLLM.

Method: Developed a pipeline for multimodal LLM ToM reasoning using both video and text. The approach includes retrieving key frames from videos to answer explicit probing questions about theory-of-mind aspects, enabling explicit analysis of how the models reason about mental states.

Result: The paper presents a framework for evaluating spatio-temporal ToM reasoning in multimodal models using video content, though specific quantitative results are not provided in the abstract.

Conclusion: Videos serve as a valuable medium for examining theory-of-mind reasoning capabilities in multimodal models, and the developed pipeline with key frame retrieval provides insights into how these models perform emotional and social reasoning across dynamic visual scenes.

Abstract: Can large multimodal models have a human-like ability for emotional and social reasoning, and if so, how does it work? Recent research has discovered emergent theory-of-mind (ToM) reasoning capabilities in large language models (LLMs). LLMs can reason about people’s mental states by solving various text-based ToM tasks that ask questions about the actors’ ToM (e.g., human belief, desire, intention). However, human reasoning in the wild is often grounded in dynamic scenes across time. Thus, we consider videos a new medium for examining spatio-temporal ToM reasoning ability. Specifically, we ask explicit probing questions about videos with abundant social and emotional reasoning content. We develop a pipeline for multimodal LLM for ToM reasoning using video and text. We also enable explicit ToM reasoning by retrieving key frames for answering a ToM question, which reveals how multimodal LLMs reason about ToM.

Method: Uses pseudo image-label pairs and innovative prompting for hair segmentation without traditional masking labels, plus DiffuseIT-M generative model for dataset augmentation while preserving hair information.

Result: Experimental results show ScalpVision’s efficiency in diagnosing various scalp conditions, demonstrating its potential as a valuable dermatological tool.

Conclusion: ScalpVision provides an effective AI-driven solution for holistic scalp disease diagnosis, overcoming key limitations in data availability and annotation costs.

Abstract: Scalp disorders are highly prevalent worldwide, yet remain underdiagnosed due to limited access to expert evaluation and the high cost of annotation. Although AI-based approaches hold great promise, their practical deployment is hindered by challenges such as severe data imbalance and the absence of pixel-level segmentation labels. To address these issues, we propose ScalpVision, an AI-driven system for the holistic diagnosis of scalp diseases. In ScalpVision, effective hair segmentation is achieved using pseudo image-label pairs and an innovative prompting method in the absence of traditional hair masking labels. Additionally, ScalpVision introduces DiffuseIT-M, a generative model adopted for dataset augmentation while maintaining hair information, facilitating improved predictions of scalp disease severity. Our experimental results affirm ScalpVision’s efficiency in diagnosing a variety of scalp conditions, showcasing its potential as a valuable tool in dermatological care. Our code is available at https://github.com/winston1214/ScalpVision.

Method: Used synthetic face images systematically varied along six dimensions (age, gender, race, facial expression, lighting, pose) and compared CLIP embeddings with textual prompts constructed from validated social psychology terms. This experimental approach avoids confounds present in wild data.

Result: CLIP makes fine-grained human-like social judgments but shows systematic bias affecting legally protected attributes, with extreme bias patterns against Black women across ages and expressions. Facial expression impacts social perception more than age, and lighting affects perception as much as age.

Conclusion: The study reveals undesirable biases in CLIP related to protected attributes and demonstrates that uncontrolled visual attributes can lead to wrong conclusions about bias. The novel experimental method provides sharper observations than previous approaches and can be applied to study biases in any vision-language model.

Abstract: We explore social perception of human faces in CLIP, a widely used open-source vision-language model. To this end, we compare the similarity in CLIP embeddings between different textual prompts and a set of face images. Our textual prompts are constructed from well-validated social psychology terms denoting social perception. The face images are synthetic and are systematically and independently varied along six dimensions: the legally protected attributes of age, gender, and race, as well as facial expression, lighting, and pose. Independently and systematically manipulating face attributes allows us to study the effect of each on social perception and avoids confounds that can occur in wild-collected data due to uncontrolled systematic correlations between attributes. Thus, our findings are experimental rather than observational. Our main findings are three. First, while CLIP is trained on the widest variety of images and texts, it is able to make fine-grained human-like social judgments on face images. Second, age, gender, and race do systematically impact CLIP’s social perception of faces, suggesting an undesirable bias in CLIP vis-a-vis legally protected attributes. Most strikingly, we find a strong pattern of bias concerning the faces of Black women, where CLIP produces extreme values of social perception across different ages and facial expressions. Third, facial expression impacts social perception more than age and lighting as much as age. The last finding predicts that studies that do not control for unprotected visual attributes may reach the wrong conclusions on bias. Our novel method of investigation, which is founded on the social psychology literature and on the experiments involving the manipulation of individual attributes, yields sharper and more reliable observations than previous observational methods and may be applied to study biases in any vision-language model.

Method: Proposes DAOcc framework that uses 3D object detection supervision to assist occupancy prediction, employs a deployment-friendly ResNet-50 backbone with 256*704 input resolution, and introduces BEV View Range Extension strategy to mitigate performance loss from lower resolution.

Result: Achieves new SOTA results on Occ3D-nuScenes and Occ3D-Waymo benchmarks, significantly outperforming previous methods. With TensorRT optimization, reaches 104.9 FPS while maintaining 54.2 mIoU on NVIDIA RTX 4090 GPU.

Conclusion: DAOcc demonstrates that superior 3D semantic occupancy prediction can be achieved with practical, deployment-friendly components through effective supervision strategies and architectural innovations, making it suitable for real-world autonomous driving applications.

Abstract: Multi-sensor fusion significantly enhances the accuracy and robustness of 3D semantic occupancy prediction, which is crucial for autonomous driving and robotics. However, most existing approaches depend on high-resolution images and complex networks to achieve top performance, hindering their deployment in practical scenarios. Moreover, current multi-sensor fusion approaches mainly focus on improving feature fusion while largely neglecting effective supervision strategies for those features. To address these issues, we propose DAOcc, a novel multi-modal occupancy prediction framework that leverages 3D object detection supervision to assist in achieving superior performance, while using a deployment-friendly image backbone and practical input resolution. In addition, we introduce a BEV View Range Extension strategy to mitigate performance degradation caused by lower image resolution. Extensive experiments demonstrate that DAOcc achieves new state-of-the-art results on both the Occ3D-nuScenes and Occ3D-Waymo benchmarks, and outperforms previous state-of-the-art methods by a significant margin using only a ResNet-50 backbone and 256*704 input resolution. With TensorRT optimization, DAOcc reaches 104.9 FPS while maintaining 54.2 mIoU on an NVIDIA RTX 4090 GPU. Code is available at https://github.com/AlphaPlusTT/DAOcc.

Method: A two-stage approach with unsupervised training strategies to generate pseudo-labels for unknown objects, particularly using top-5 auto-labelling.

Result: Demonstrates superiority of two-stage methods over single-stage approaches and reveals critical impact of hyperparameter choices in evaluation protocols.

Conclusion: Two-stage approaches provide more robust unknown object detection, with top-5 auto-labelling showing promising performance for scaling unknown object learning.

Abstract: Autonomous systems rely on accurate 3D object detection from LiDAR data, yet most detectors are limited to a predefined set of known classes, making them vulnerable to unexpected out-of-distribution (OOD) objects. In this work, we present HD-OOD3D, a novel two-stage method for detecting unknown objects. We demonstrate the superiority of two-stage approaches over single-stage methods, achieving more robust detection of unknown objects while addressing key challenges in the evaluation protocol. Furthermore, we conduct an in-depth analysis of the standard evaluation protocol for OOD detection, revealing the critical impact of hyperparameter choices. To address the challenge of scaling the learning of unknown objects, we explore unsupervised training strategies to generate pseudo-labels for unknowns. Among the different approaches evaluated, our experiments show that top-5 auto-labelling offers more promising performance compared to simple resizing techniques.

Method: Proposes Step-wise Distribution Alignment Guided Style Prompt Tuning (StepSPT) with style prompts to align target samples with desired distribution. Uses dual-phase optimization: external process for step-wise distribution alignment to tune style prompts, and internal process for classifier updates with cross-entropy loss.

Result: Outperforms existing prompt tuning-based methods and state-of-the-art approaches on five datasets. Ablation studies confirm the effectiveness of the proposed method.

Conclusion: StepSPT provides an effective solution for SF-CDFSL by implicitly narrowing domain gaps through prediction distribution optimization, requiring only pre-trained models and few target samples without source data access.

Abstract: Existing cross-domain few-shot learning (CDFSL) methods, which develop source-domain training strategies to enhance model transferability, face challenges with large-scale pre-trained models (LMs) due to inaccessible source data and training strategies. Moreover, fine-tuning LMs for CDFSL demands substantial computational resources, limiting practicality. This paper addresses the source-free CDFSL (SF-CDFSL) problem, tackling few-shot learning (FSL) in the target domain using only pre-trained models and a few target samples without source data or strategies. To overcome the challenge of inaccessible source data, this paper introduces Step-wise Distribution Alignment Guided Style Prompt Tuning (StepSPT), which implicitly narrows domain gaps through prediction distribution optimization. StepSPT proposes a style prompt to align target samples with the desired distribution and adopts a dual-phase optimization process. In the external process, a step-wise distribution alignment strategy factorizes prediction distribution optimization into a multi-step alignment problem to tune the style prompt. In the internal process, the classifier is updated using standard cross-entropy loss. Evaluations on five datasets demonstrate that StepSPT outperforms existing prompt tuning-based methods and SOTAs. Ablation studies further verify its effectiveness. Code will be made publicly available at https://github.com/xuhuali-mxj/StepSPT.

Method: Chain-of-Action (CoA) decomposes GSLs task into sequential actions that extract and merge key information, passing enriched context between steps to guide VLMs in generating comprehensive semantic labels including objects, scenes, attributes, and relationships.

Result: Extensive experiments on benchmark datasets show significant improvements across key performance metrics, demonstrating CoA’s capability to generate accurate and contextually rich semantic labels.

Conclusion: CoA advances state-of-the-art in generative semantic labels and opens new avenues for applying VLMs in open-ended, dynamic real-world scenarios beyond constrained classification tasks.

Abstract: Recent advances in vision-language models (VLMs) have demonstrated remarkable capabilities in image classification by leveraging predefined sets of labels to construct text prompts for zero-shot reasoning. However, these approaches face significant limitations in undefined domains, where the label space is vocabulary-unknown and composite. We thus introduce Generative Semantic Labels (GSLs), a novel task that aims to predict a comprehensive set of semantic labels for an image without being constrained by a predefined labels set. Unlike traditional zero-shot classification, GSLs generates multiple semantic-level labels, encompassing objects, scenes, attributes, and relationships, thereby providing a richer and more accurate representation of image content. In this paper, we propose Chain-of-Action (CoA), an innovative method designed to tackle the GSLs task. CoA is motivated by the observation that enriched contextual information significantly improves generative performance during inference. Specifically, CoA decomposes the GSLs task into a sequence of detailed actions. Each action extracts and merges key information from the previous step, passing enriched context to the next, ultimately guiding the VLM to generate comprehensive and accurate semantic labels. We evaluate the effectiveness of CoA through extensive experiments on widely-used benchmark datasets. The results demonstrate significant improvements across key performance metrics, validating the capability of CoA to generate accurate and contextually rich semantic labels. Our work not only advances the state-of-the-art in generative semantic labels but also opens new avenues for applying VLMs in open-ended and dynamic real-world scenarios.

Method: The survey systematically reviews the evolution of RS-STVLMs, analyzes representative tasks (change captioning, change QA, change grounding), dissects fundamental components and key technologies, and examines datasets and evaluation metrics.

Result: The paper provides a comprehensive synthesis of current achievements in spatio-temporal vision-language understanding for remote sensing, identifying shared architectural patterns and benchmarking progress across different tasks.

Conclusion: The survey illuminates current achievements and charts promising future research directions for RS-STVLMs, establishing a foundation for continued development in this emerging field of remote sensing analysis.

Abstract: The interpretation of multi-temporal remote sensing imagery is critical for monitoring Earth’s dynamic processes-yet previous change detection methods, which produce binary or semantic masks, fall short of providing human-readable insights into changes. Recent advances in Vision-Language Models (VLMs) have opened a new frontier by fusing visual and linguistic modalities, enabling spatio-temporal vision-language understanding: models that not only capture spatial and temporal dependencies to recognize changes but also provide a richer interactive semantic analysis of temporal images (e.g., generate descriptive captions and answer natural-language queries). In this survey, we present the first comprehensive review of RS-STVLMs. The survey covers the evolution of models from early task-specific models to recent general foundation models that leverage powerful large language models. We discuss progress in representative tasks, such as change captioning, change question answering, and change grounding. Moreover, we systematically dissect the fundamental components and key technologies underlying these models, and review the datasets and evaluation metrics that have driven the field. By synthesizing task-level insights with a deep dive into shared architectural patterns, we aim to illuminate current achievements and chart promising directions for future research in spatio-temporal vision-language understanding for remote sensing. We will keep tracing related works at https://github.com/Chen-Yang-Liu/Awesome-RS-SpatioTemporal-VLMs

Method: Synthesized and filtered 293K multi-modal reasoning traces over perceptual outputs from state-of-the-art vision specialists, training LATTE to focus on reasoning rather than perception.

Result: Achieved significant 4-5% performance gains over baselines across 6 benchmarks covering both perception and reasoning abilities.

Conclusion: Multi-modal reasoning traces are effective when using appropriate data sources, formats, and high-quality thoughts, enabling better complex question answering through specialized perception offloading.

Abstract: While open-source vision-language models perform well on simple question-answering, they still struggle with complex questions that require both perceptual and reasoning capabilities. We propose LATTE, a family of vision-language models that have LeArned to Think wiTh vision spEcialists. By offloading perception to state-of-the-art vision models, our approach enables vision-language models to focus solely on reasoning over high-quality perceptual information. To train LATTE, we synthesize and filter a large dataset of 293K multi-modal reasoning traces over perceptual outputs of vision specialists. LATTE trained on this data achieves significant 4-5% gains over baselines across 6 benchmarks covering both perception and reasoning abilities. Ablation studies reveal that the effectiveness of multi-modal reasoning traces depends on the data sources, formats, and quality of thoughts.

Method: Formulate shape editing as conditional reconstruction problem; train conditional Large Reconstruction Model for masked reconstruction using multi-view consistent masks from random 3D occlusion; use one clean viewpoint as conditional signal; during inference, manually define 3D edit region and provide edited image from canonical viewpoint.

Result: Method preserves input geometry in unmasked regions with reconstruction capabilities matching state-of-the-art, performs various mesh edits from single image guidance that previous works struggle with, and achieves 2-10x faster performance than top-performing prior work.

Conclusion: The approach successfully demonstrates efficient and expressive 3D shape editing through conditional masked reconstruction, offering significant speed improvements while maintaining high-quality reconstruction and enabling complex edits from minimal image guidance.

Abstract: We present a novel approach to shape editing, building on recent progress in 3D reconstruction from multi-view images. We formulate shape editing as a conditional reconstruction problem, where the model must reconstruct the input shape with the exception of a specified 3D region, in which the geometry should be generated from the conditional signal. To this end, we train a conditional Large Reconstruction Model (LRM) for masked reconstruction, using multi-view consistent masks rendered from a randomly generated 3D occlusion, and using one clean viewpoint as the conditional signal. During inference, we manually define a 3D region to edit and provide an edited image from a canonical viewpoint to fill that region. We demonstrate that, in just a single forward pass, our method not only preserves the input geometry in the unmasked region through reconstruction capabilities on par with SoTA, but is also expressive enough to perform a variety of mesh edits from a single image guidance that past works struggle with, while being 2-10x faster than the top-performing prior work.

Method: Builds on pretrained latent diffusion model for monocular depth estimation, injects sparse depth observations via optimization scheme that runs in tandem with iterative denoising diffusion inference.

Result: Excellent zero-shot generalization across diverse environments, effective handling of extremely sparse guidance, outperforms traditional approaches.

Conclusion: Contemporary monocular depth priors greatly robustify depth completion - better to view task as recovering dense depth from image pixels guided by sparse depth, rather than inpainting sparse depth guided by image.

Abstract: Depth completion upgrades sparse depth measurements into dense depth maps guided by a conventional image. Existing methods for this highly ill-posed task operate in tightly constrained settings and tend to struggle when applied to images outside the training domain or when the available depth measurements are sparse, irregularly distributed, or of varying density. Inspired by recent advances in monocular depth estimation, we reframe depth completion as an image-conditional depth map generation guided by sparse measurements. Our method, Marigold-DC, builds on a pretrained latent diffusion model for monocular depth estimation and injects the depth observations as test-time guidance via an optimization scheme that runs in tandem with the iterative inference of denoising diffusion. The method exhibits excellent zero-shot generalization across a diverse range of environments and handles even extremely sparse guidance effectively. Our results suggest that contemporary monocular depth priors greatly robustify depth completion: it may be better to view the task as recovering dense depth from (dense) image pixels, guided by sparse depth; rather than as inpainting (sparse) depth, guided by an image. Project website: https://MarigoldDepthCompletion.github.io/

Method: End-to-end deep learning pipeline that predicts facial depth with CNN, adjusts camera distance/focal length, reprojects 3D features, and uses inpainting with differentiable renderer for training.

Result: Outperforms previous methods and achieves comparable results to 3D GAN-based approach while being 260 times faster, with full-frame processing eliminating complex post-processing.

Conclusion: The proposed pipeline effectively corrects perspective distortion in portraits through learned depth estimation and perspective adjustment, offering significant speed advantages over existing methods.

Abstract: Portraits or selfie images taken from a close distance typically suffer from perspective distortion. In this paper, we propose an end-to-end deep learning-based rectification pipeline to mitigate the effects of perspective distortion. We learn to predict the facial depth by training a deep CNN. The estimated depth is utilized to adjust the camera-to-subject distance by moving the camera farther, increasing the camera focal length, and reprojecting the 3D image features to the new perspective. The reprojected features are then fed to an inpainting module to fill in the missing pixels. We leverage a differentiable renderer to enable end-to-end training of our depth estimation and feature extraction nets to improve the rectified outputs. To boost the results of the inpainting module, we incorporate an auxiliary module to predict the horizontal movement of the camera which decreases the area that requires hallucination of challenging face parts such as ears. Unlike previous works, we process the full-frame input image at once without cropping the subject’s face and processing it separately from the rest of the body, eliminating the need for complex post-processing steps to attach the face back to the subject’s body. To train our network, we utilize the popular game engine Unreal Engine to generate a large synthetic face dataset containing various subjects, head poses, expressions, eyewear, clothes, and lighting. Quantitative and qualitative results show that our rectification pipeline outperforms previous methods, and produces comparable results with a time-consuming 3D GAN-based method while being more than 260 times faster.

Method: Leverage sparse autoencoders (SAEs) to identify semantic directions associated with faithfulness/hallucination, then propose SSL - a plug-and-play method using SAE-derived latent directions to steer LVLMs.

Result: SSL significantly outperforms existing decoding approaches in mitigating hallucinations while maintaining transferability across different model architectures with negligible additional time overhead.

Conclusion: The SAE-based approach provides precise and disentangled hallucination representations, enabling effective intervention through identified latent directions to reduce hallucinations efficiently.

Abstract: Large vision-language models (LVLMs) have achieved remarkable performance on multimodal tasks. However, they still suffer from hallucinations, generating text inconsistent with visual input, posing significant risks in real-world applications. Existing approaches to address this issue focus on incorporating external knowledge bases, alignment training, or decoding strategies, all of which require substantial computational cost and time. Recent works try to explore more efficient alternatives by adjusting LVLMs’ internal representations. Although promising, these methods may cause hallucinations to be insufficiently suppressed or lead to excessive interventions that negatively affect normal semantics. In this work, we leverage sparse autoencoders (SAEs) to identify semantic directions closely associated with faithfulness or hallucination, extracting more precise and disentangled hallucination-related representations. Our analysis demonstrates that interventions along the identified faithful direction can mitigate hallucinations, while those along the hallucinatory direction can exacerbate them. Building on these insights, we propose Steering LVLMs via SAE Latent Directions (SSL), a plug-and-play method based on SAE-derived latent directions to mitigate hallucinations in LVLMs. Extensive experiments demonstrate that SSL significantly outperforms existing decoding approaches in mitigating hallucinations, while maintaining transferability across different model architectures with negligible additional time overhead. The code is available at https://github.com/huazhenglin2003/SSL.

Method: The framework trains a reflective retriever using self-reflective contrastive learning, injecting the generator’s knowledge into negatives to ensure retrieved images compensate for missing knowledge. It integrates fine-grained visual knowledge from real-world images.

Result: RealRAG achieves significant performance improvements, including a 16.18% FID score gain with auto-regressive models on the Stanford Car benchmark, and works modularly with all state-of-the-art text-to-image generative models.

Conclusion: The proposed RealRAG framework effectively addresses knowledge gaps in text-to-image models by leveraging real-world image retrieval, enabling better generation of fine-grained and unseen objects while being compatible with various generative architectures.

Abstract: Recent text-to-image generative models, e.g., Stable Diffusion V3 and Flux, have achieved notable progress. However, these models are strongly restricted to their limited knowledge, a.k.a., their own fixed parameters, that are trained with closed datasets. This leads to significant hallucinations or distortions when facing fine-grained and unseen novel real-world objects, e.g., the appearance of the Tesla Cybertruck. To this end, we present the first real-object-based retrieval-augmented generation framework (RealRAG), which augments fine-grained and unseen novel object generation by learning and retrieving real-world images to overcome the knowledge gaps of generative models. Specifically, to integrate missing memory for unseen novel object generation, we train a reflective retriever by self-reflective contrastive learning, which injects the generator’s knowledge into the sef-reflective negatives, ensuring that the retrieved augmented images compensate for the model’s missing knowledge. Furthermore, the real-object-based framework integrates fine-grained visual knowledge for the generative models, tackling the distortion problem and improving the realism for fine-grained object generation. Our Real-RAG is superior in its modular application to all types of state-of-the-art text-to-image generative models and also delivers remarkable performance boosts with all of them, such as a gain of 16.18% FID score with the auto-regressive model on the Stanford Car benchmark.

Method: Conducted user study with eight BLV participants to systematically evaluate preferences for six types of LVLM descriptions, then used insights to build training data for a new automatic evaluation metric.

Result: LVLM descriptions helped reduce fear and improve actionability but showed wide variation in sufficiency and conciseness. GPT-4o was not consistently preferred despite its potential to refine descriptions.

Conclusion: Urgent need for BLV-centered evaluation metrics and human-in-the-loop feedback to advance LVLM description quality for accessibility.

Abstract: For individuals with blindness or low vision (BLV), navigating complex environments can pose serious risks. Large Vision-Language Models (LVLMs) show promise for generating scene descriptions, but their effectiveness for BLV users remains underexplored. To address this gap, we conducted a user study with eight BLV participants to systematically evaluate preferences for six types of LVLM descriptions. While they helped to reduce fear and improve actionability, user ratings showed wide variation in sufficiency and conciseness. Furthermore, GPT-4o–despite its strong potential to refine descriptions–was not consistently preferred by participants. We use the insights obtained from the user study to build training data for building our new automatic evaluation metric that can capture BLV preferences effectively. Our findings underscore the urgent need for BLV-centered evaluation metrics and human-in-the-loop feedback to advance LVLM description quality for accessibility.

Method: Geospatial deep learning framework integrating hydrological flow data, land cover information, and proximity to known PFAS sources with a label noise-aware loss function.

Result: The framework shows improved prediction accuracy through ablation studies, robustness analysis, real-world validation, and outperforms baselines like sparse segmentation, Kriging, and pollutant transport simulations.

Conclusion: FOCUS demonstrates potential for scalable PFAS monitoring and received positive expert feedback for its approach to large-scale contamination mapping.

Abstract: Per- and polyfluoroalkyl substances (PFAS), chemicals found in products like non-stick cookware, are unfortunately persistent environmental pollutants with severe health risks. Accurately mapping PFAS contamination is crucial for guiding targeted remediation efforts and protecting public and environmental health, yet detection across large regions remains challenging due to the cost of testing and the difficulty of simulating their spread. In this work, we introduce FOCUS, a geospatial deep learning framework with a label noise-aware loss function, to predict PFAS contamination in surface water over large regions. By integrating hydrological flow data, land cover information, and proximity to known PFAS sources, our approach leverages both spatial and environmental context to improve prediction accuracy. We evaluate the performance of our approach through extensive ablation studies, robustness analysis, real-world validation, and comparative analyses against baselines like sparse segmentation, as well as existing scientific methods, including Kriging and pollutant transport simulations. Results and expert feedback highlight our framework’s potential for scalable PFAS monitoring.

Method: Uses learnable queries and cross-modal attention mechanisms in an abnormality-aligned bootstrapping language-image pretraining framework to detect abnormalities and generate structured reports.

Result: Outperforms state-of-the-art medical vision-language models and 3D report generation methods in both accuracy and clinical relevance, reducing missed findings.

Conclusion: Demonstrates the potential of integrating multimodal learning strategies for improving radiology reporting, with source code made publicly available.

Abstract: Medical imaging plays a pivotal role in modern healthcare, with computed tomography pulmonary angiography (CTPA) being a critical tool for diagnosing pulmonary embolism and other thoracic conditions. However, the complexity of interpreting CTPA scans and generating accurate radiology reports remains a significant challenge. This paper introduces Abn-BLIP (Abnormality-aligned Bootstrapping Language-Image Pretraining), an advanced diagnosis model designed to align abnormal findings to generate the accuracy and comprehensiveness of radiology reports. By leveraging learnable queries and cross-modal attention mechanisms, our model demonstrates superior performance in detecting abnormalities, reducing missed findings, and generating structured reports compared to existing methods. Our experiments show that Abn-BLIP outperforms state-of-the-art medical vision-language models and 3D report generation methods in both accuracy and clinical relevance. These results highlight the potential of integrating multimodal learning strategies for improving radiology reporting. The source code is available at https://github.com/zzs95/abn-blip.

Method: Uses Large Reconstruction Models made animatable with cross-attention to expression codes. Employs position maps from DUSt3R and generalized feature maps from Sapiens human foundation model. Trains with input images of different expressions for robustness to inconsistent inputs like phone captures or monocular video frames.

Result: Competitive advantage over state-of-the-art methods in both few-input and single-input scenarios. Successfully creates 3D head avatars from various sources including smartphone captures, single images, and even out-of-domain inputs like antique busts.

Conclusion: Avat3r demonstrates that high-quality animatable 3D head avatars can be created from minimal input, making digital human doubles more accessible beyond the VFX industry and enabling real-time applications.

Abstract: Traditionally, creating photo-realistic 3D head avatars requires a studio-level multi-view capture setup and expensive optimization during test-time, limiting the use of digital human doubles to the VFX industry or offline renderings. To address this shortcoming, we present Avat3r, which regresses a high-quality and animatable 3D head avatar from just a few input images, vastly reducing compute requirements during inference. More specifically, we make Large Reconstruction Models animatable and learn a powerful prior over 3D human heads from a large multi-view video dataset. For better 3D head reconstructions, we employ position maps from DUSt3R and generalized feature maps from the human foundation model Sapiens. To animate the 3D head, our key discovery is that simple cross-attention to an expression code is already sufficient. Finally, we increase robustness by feeding input images with different expressions to our model during training, enabling the reconstruction of 3D head avatars from inconsistent inputs, e.g., an imperfect phone capture with accidental movement, or frames from a monocular video. We compare Avat3r with current state-of-the-art methods for few-input and single-input scenarios, and find that our method has a competitive advantage in both tasks. Finally, we demonstrate the wide applicability of our proposed model, creating 3D head avatars from images of different sources, smartphone captures, single images, and even out-of-domain inputs like antique busts. Project website: https://tobias-kirschstein.github.io/avat3r/

Method: Pretrain uncertainties on large EO datasets and develop an evaluation framework to assess zero-shot performance in multi-label classification and segmentation tasks across different EO domains and geographic locations.

Result: EO-pretrained uncertainties demonstrate strong generalization across unseen EO domains, geographic locations, and target granularities while maintaining sensitivity to ground sampling distance variations. They align well with task-specific uncertainties and are sensitive to real-world EO image noise.

Conclusion: The study establishes representation uncertainty as a valuable tool for EO applications, showing superior generalization compared to natural image pretraining and providing out-of-the-box spatial uncertainty estimation capabilities for practical EO use cases.

Abstract: Recent advances in Computer Vision have introduced the concept of pretrained representation uncertainty, enabling zero-shot uncertainty estimation. This holds significant potential for Earth Observation (EO), where trustworthiness is critical, yet the complexity of EO data poses challenges to uncertainty-aware methods. In this work, we investigate the generalization of representation uncertainty in EO, considering the domain’s unique semantic characteristics. We pretrain uncertainties on large EO datasets and propose an evaluation framework to assess their zero-shot performance in multi-label classification and segmentation EO tasks. Our findings reveal that, unlike uncertainties pretrained on natural images, EO-pretraining exhibits strong generalization across unseen EO domains, geographic locations, and target granularities, while maintaining sensitivity to variations in ground sampling distance. We demonstrate the practical utility of pretrained uncertainties showcasing their alignment with task-specific uncertainties in downstream tasks, their sensitivity to real-world EO image noise, and their ability to generate spatial uncertainty estimates out-of-the-box. Initiating the discussion on representation uncertainty in EO, our study provides insights into its strengths and limitations, paving the way for future research in the field. Code and weights are available at: https://github.com/Orion-AI-Lab/EOUncertaintyGeneralization.

Method: Uses motion graphs as explicit sparse motion representation, propagates motion to Gaussians via dual quaternion skinning with learnable weight painting functions, and jointly optimizes motion graphs and 3D Gaussians through differentiable rendering.

Result: Achieves state-of-the-art performance on challenging iPhone dataset, competitive on HyperNeRF, and demonstrates applications in novel pose animation, robot demonstration synthesis, and visual action planning.

Conclusion: MBGS provides an effective framework for explicit motion representation in Gaussian splatting, enabling better motion manipulation and opening new possibilities for robotics applications requiring motion control.

Abstract: Gaussian splatting has emerged as a powerful tool for high-fidelity reconstruction of dynamic scenes. However, existing methods primarily rely on implicit motion representations, such as encoding motions into neural networks or per-Gaussian parameters, which makes it difficult to further manipulate the reconstructed motions. This lack of explicit controllability limits existing methods to replaying recorded motions only, which hinders a wider application in robotics. To address this, we propose Motion Blender Gaussian Splatting (MBGS), a novel framework that uses motion graphs as an explicit and sparse motion representation. The motion of a graph’s links is propagated to individual Gaussians via dual quaternion skinning, with learnable weight painting functions that determine the influence of each link. The motion graphs and 3D Gaussians are jointly optimized from input videos via differentiable rendering. Experiments show that MBGS achieves state-of-the-art performance on the highly challenging iPhone dataset while being competitive on HyperNeRF. We demonstrate the application potential of our method in animating novel object poses, synthesizing real robot demonstrations, and predicting robot actions through visual planning. The source code, models, video demonstrations can be found at http://mlzxy.github.io/motion-blender-gs.

Method: Integrates visual transformers with state-space temporal modeling, using multi-level token compression and hierarchical temporal processing to maintain computational efficiency without sacrificing temporal resolution.

Result: Achieves mean absolute error of 1.50 seconds for 2-minute videos, with 65.2% predictions within 1 second of ground truth. Outperforms recent video-language models by up to 3.95 seconds while using fewer parameters (3B vs. 13-72B).

Conclusion: Demonstrates effective temporal localization across various video durations and environmental conditions, showing potential for fine-grained crash detection while identifying remaining challenges for extended deployment.

Abstract: Traffic crash detection in long-form surveillance videos is essential for improving emergency response and infrastructure planning, yet remains difficult due to the brief and infrequent nature of crash events. We present \textbf{HybridMamba}, a novel architecture integrating visual transformers with state-space temporal modeling to achieve high-precision crash time localization. Our approach introduces multi-level token compression and hierarchical temporal processing to maintain computational efficiency without sacrificing temporal resolution. Evaluated on a large-scale dataset from the Iowa Department of Transportation, HybridMamba achieves a mean absolute error of \textbf{1.50 seconds} for 2-minute videos ($p<0.01$ compared to baselines), with \textbf{65.2%} of predictions falling within one second of the ground truth. It outperforms recent video-language models (e.g., TimeChat, VideoLLaMA-2) by up to 3.95 seconds while using significantly fewer parameters (3B vs. 13–72B). Our results demonstrate effective temporal localization across various video durations (2–40 minutes) and diverse environmental conditions, highlighting HybridMamba’s potential for fine-grained temporal localization in traffic surveillance while identifying challenges that remain for extended deployment.

Method: Uses 4D superpoints to cluster multi-frame LiDAR points in 4D space for unsupervised instance separation, combined with decomposed 4D initialization and smoothness regularization in 2D/3D space for temporal stability.

Result: Outperforms existing methods in decomposed dynamic scene reconstruction and enables accurate, flexible instance-level editing on benchmark datasets.

Conclusion: Provides a practical solution for real-world applications by achieving unsupervised instance decomposition and enabling instance-level scene editing without manual annotations.

Abstract: Reconstructing and decomposing dynamic urban scenes is crucial for autonomous driving, urban planning, and scene editing. However, existing methods fail to perform instance-aware decomposition without manual annotations, which is crucial for instance-level scene editing.We propose UnIRe, a 3D Gaussian Splatting (3DGS) based approach that decomposes a scene into a static background and individual dynamic instances using only RGB images and LiDAR point clouds. At its core, we introduce 4D superpoints, a novel representation that clusters multi-frame LiDAR points in 4D space, enabling unsupervised instance separation based on spatiotemporal correlations. These 4D superpoints serve as the foundation for our decomposed 4D initialization, i.e., providing spatial and temporal initialization to train a dynamic 3DGS for arbitrary dynamic classes without requiring bounding boxes or object templates.Furthermore, we introduce a smoothness regularization strategy in both 2D and 3D space, further improving the temporal stability.Experiments on benchmark datasets show that our method outperforms existing methods in decomposed dynamic scene reconstruction while enabling accurate and flexible instance-level editing, making it a practical solution for real-world applications.

Method: Integrates Temporal Transformer (TTT) layers through hierarchical co-design, systematically condensing historical temporal information into hidden states and using multi-stage contextual aggregation with hierarchical concatenation.

Result: Achieves 3.2% improvement in segmentation accuracy on DAVIS 2017 benchmark, with ablation studies showing significant enhancement in global modeling capabilities.

Conclusion: Global information should be strategically distributed across multiple network layers for optimal dependency utilization in video segmentation tasks.

Abstract: This paper addresses the challenge of capturing global temporaldependencies in long video sequences for Video Object Segmentation (VOS). Existing architectures often fail to effectively model these dependencies acrossextended temporal horizons. To overcome this limitation, we introduce GISE-TTT, anovel architecture that integrates Temporal Transformer (TTT) layers intotransformer-based frameworks through a co-designed hierarchical approach.The TTTlayer systematically condenses historical temporal information into hidden states thatencode globally coherent contextual representations. By leveraging multi-stagecontextual aggregation through hierarchical concatenation, our frameworkprogressively refines spatiotemporal dependencies across network layers. This designrepresents the first systematic empirical evidence that distributing global informationacross multiple network layers is critical for optimal dependency utilization in videosegmentation tasks.Ablation studies demonstrate that incorporating TTT modules athigh-level feature stages significantly enhances global modeling capabilities, therebyimproving the network’s ability to capture long-range temporal relationships. Extensive experiments on DAVIS 2017 show that GISE-TTT achieves a 3.2%improvement in segmentation accuracy over the baseline model, providingcomprehensive evidence that global information should be strategically leveragedthroughout the network architecture.The code will be made available at:https://github.com/uuool/GISE-TTT.

Method: Two-stage framework leveraging shots as basic units, extending temporal context to neighboring shots, incorporating film grammar devices (shot scales, thread structures), and integrating expert knowledge through add-on modules without requiring VLM retraining.

Result: Achieves state-of-the-art performance among training-free approaches and surpasses fine-tuned methods on several benchmarks. Introduces new evaluation measures including an action score and a novel protocol treating frameworks as AD generation assistants.

Conclusion: The proposed framework effectively generates high-quality Audio Descriptions by leveraging film grammar and temporal context, with strong performance and novel evaluation methods that advance the field of automated AD generation.

Abstract: Our objective is the automatic generation of Audio Descriptions (ADs) for edited video material, such as movies and TV series. To achieve this, we propose a two-stage framework that leverages “shots” as the fundamental units of video understanding. This includes extending temporal context to neighbouring shots and incorporating film grammar devices, such as shot scales and thread structures, to guide AD generation. Our method is compatible with both open-source and proprietary Visual-Language Models (VLMs), integrating expert knowledge from add-on modules without requiring additional training of the VLMs. We achieve state-of-the-art performance among all prior training-free approaches and even surpass fine-tuned methods on several benchmarks. To evaluate the quality of predicted ADs, we introduce a new evaluation measure – an action score – specifically targeted to assessing this important aspect of AD. Additionally, we propose a novel evaluation protocol that treats automatic frameworks as AD generation assistants and asks them to generate multiple candidate ADs for selection.

Method: Proposes a plug-and-play framework that first explores different token allocations between key frames (spatial details) and delta frames (temporal changes), then uses query-aware attention-based metric to select optimal token combination without model updates.

Result: Achieves significant performance improvements (up to 5.8%) on multiple video question answering benchmarks.

Conclusion: The explore-then-select strategy effectively adapts static and dynamic information allocation based on question requirements, providing memory-efficient and high-performance video question answering.

Abstract: Video question answering benefits from the rich information in videos, enabling various applications. However, the large volume of tokens generated from long videos presents challenges to memory efficiency and model performance. To alleviate this, existing works propose to compress video inputs, but often overlook the varying importance of static and dynamic information across different queries, leading to inefficient token usage within limited budgets. We propose a novel token selection strategy, \textsc{explore-then-select}, that adaptively adjusts static and dynamic information based on question requirements. Our framework first explores different token allocations between key frames, which preserve spatial details, and delta frames, which capture temporal changes. Then it employs a query-aware attention-based metric to select the optimal token combination without model updates. Our framework is plug-and-play and can be seamlessly integrated within diverse video language models. Extensive experiments show that our method achieves significant performance improvements (up to 5.8%) on multiple video question answering benchmarks. Our code is available at https://github.com/ANDgate99/Explore-Then-Select .

Method: Multi-task learning foundation model trained on 14 tasks/datasets, functioning as an embedding extractor for various input modalities (RGB, thermal, depth videos, ECG, EMG, GSR, fNIRS) with an Embedding-Mixer transformer for final assessment.

Result: Achieved state-of-the-art performance on BioVid and AI4Pain datasets, outperforming 75 different methodologies in unimodal and multimodal settings across diverse input modalities.

Conclusion: PainFormer demonstrates effective extraction of high-quality embeddings from diverse modalities and paves the way for general-purpose models in automatic pain assessment.

Abstract: Pain is a manifold condition that impacts a significant percentage of the population. Accurate and reliable pain evaluation for the people suffering is crucial to developing effective and advanced pain management protocols. Automatic pain assessment systems provide continuous monitoring and support decision-making processes, ultimately aiming to alleviate distress and prevent functionality decline. This study introduces PainFormer, a vision foundation model based on multi-task learning principles trained simultaneously on 14 tasks/datasets with a total of 10.9 million samples. Functioning as an embedding extractor for various input modalities, the foundation model provides feature representations to the Embedding-Mixer, a transformer-based module that performs the final pain assessment. Extensive experiments employing behavioral modalities - including RGB, synthetic thermal, and estimated depth videos - and physiological modalities such as ECG, EMG, GSR, and fNIRS revealed that PainFormer effectively extracts high-quality embeddings from diverse input modalities. The proposed framework is evaluated on two pain datasets, BioVid and AI4Pain, and directly compared to 75 different methodologies documented in the literature. Experiments conducted in unimodal and multimodal settings demonstrate state-of-the-art performances across modalities and pave the way toward general-purpose models for automatic pain assessment. The foundation model’s architecture (code) and weights are available at: https://github.com/GkikasStefanos/PainFormer.

Method: Proposes CVVNet with High-Low Frequency Extraction module (parallel multi-scale convolution/max-pooling and self-attention paths) and Dynamic Gated Aggregation mechanism to adaptively fuse high- and low-frequency features for robust multi-frequency feature extraction.

Result: Achieves state-of-the-art performance with 8.6% improvement on DroneGait and 2% improvement on Gait3D compared to existing methods, effectively handling distortions from view changes.

Conclusion: CVVNet’s frequency aggregation architecture with adaptive multi-frequency feature integration significantly improves gait recognition robustness across different vertical views, overcoming limitations of traditional CNN and self-attention approaches.

Abstract: Gait recognition enables contact-free, long-range person identification that is robust to clothing variations and non-cooperative scenarios. While existing methods perform well in controlled indoor environments, they struggle with cross-vertical view scenarios, where surveillance angles vary significantly in elevation. Our experiments show up to 60% accuracy degradation in low-to-high vertical view settings due to severe deformations and self-occlusions of key anatomical features. Current CNN and self-attention-based methods fail to effectively handle these challenges, due to their reliance on single-scale convolutions or simplistic attention mechanisms that lack effective multi-frequency feature integration. To tackle this challenge, we propose CVVNet (Cross-Vertical-View Network), a frequency aggregation architecture specifically designed for robust cross-vertical-view gait recognition. CVVNet employs a High-Low Frequency Extraction module (HLFE) that adopts parallel multi-scale convolution/max-pooling path and self-attention path as high- and low-frequency mixers for effective multi-frequency feature extraction from input silhouettes. We also introduce the Dynamic Gated Aggregation (DGA) mechanism to adaptively adjust the fusion ratio of high- and low-frequency features. The integration of our core Multi-Scale Attention Gated Aggregation (MSAGA) module, HLFE and DGA enables CVVNet to effectively handle distortions from view changes, significantly improving the recognition robustness across different vertical views. Experimental results show that our CVVNet achieves state-of-the-art performance, with $8.6%$ improvement on DroneGait and $2%$ on Gait3D compared with the best existing methods.

Method: Uses motion quality indicators (manual or heuristic annotations) to train quality-aware diffusion models on raw motion data, creating a unified generate-discriminate model.

Result: Applied to SoccerMocap dataset, reduces motion pops by 68% and frozen frames by 81%, effectively correcting various motion artifacts.

Conclusion: StableMotion provides an effective solution for training motion cleanup models directly from unpaired corrupted datasets, automating artifact correction without requiring clean reference data.

Abstract: Motion capture (mocap) data often exhibits visually jarring artifacts due to inaccurate sensors and post-processing. Cleaning this corrupted data can require substantial manual effort from human experts, which can be a costly and time-consuming process. Previous data-driven motion cleanup methods offer the promise of automating this cleanup process, but often require in-domain paired corrupted-to-clean training data. Constructing such paired datasets requires access to high-quality, relatively artifact-free motion clips, which often necessitates laborious manual cleanup. In this work, we present StableMotion, a simple yet effective method for training motion cleanup models directly from unpaired corrupted datasets that need cleanup. The core component of our method is the introduction of motion quality indicators, which can be easily annotated

• through manual labeling or heuristic algorithms - and enable training of quality-aware motion generation models on raw motion data with mixed quality. At test time, the model can be prompted to generate high-quality motions using the quality indicators. Our method can be implemented through a simple diffusion-based framework, leading to a unified motion generate-discriminate model, which can be used to both identify and fix corrupted frames. We demonstrate that our proposed method is effective for training motion cleanup models on raw mocap data in production scenarios by applying StableMotion to SoccerMocap, a 245-hour soccer mocap dataset containing real-world motion artifacts. The trained model effectively corrects a wide range of motion artifacts, reducing motion pops and frozen frames by 68% and 81%, respectively. Results and code are available at https://yxmu.foo/stablemotion-page

Method: Introduces active geometric consistency enforcement during learning with multi-view, multi-scale supervision, plus a densely connected cost regularization network with two block designs (simple and feature dense).

Result: Achieves state-of-the-art on DTU and BlendedMVS datasets, second place on Tanks and Temples benchmark, and reduces training iterations by 50% compared to other MVS methods.

Conclusion: GC-MVSNet++ is the first method to enforce multi-view, multi-scale supervised geometric consistency during learning, significantly accelerating training while improving performance.

Abstract: Traditional multi-view stereo (MVS) methods primarily depend on photometric and geometric consistency constraints. In contrast, modern learning-based algorithms often rely on the plane sweep algorithm to infer 3D geometry, applying explicit geometric consistency (GC) checks only as a post-processing step, with no impact on the learning process itself. In this work, we introduce GC MVSNet plus plus, a novel approach that actively enforces geometric consistency of reference view depth maps across multiple source views (multi view) and at various scales (multi scale) during the learning phase (see Fig. 1). This integrated GC check significantly accelerates the learning process by directly penalizing geometrically inconsistent pixels, effectively halving the number of training iterations compared to other MVS methods. Furthermore, we introduce a densely connected cost regularization network with two distinct block designs simple and feature dense optimized to harness dense feature connections for enhanced regularization. Extensive experiments demonstrate that our approach achieves a new state of the art on the DTU and BlendedMVS datasets and secures second place on the Tanks and Temples benchmark. To our knowledge, GC MVSNet plus plus is the first method to enforce multi-view, multi-scale supervised geometric consistency during learning. Our code is available.

Method: A polarity- and geometry-aware encoder extracts coarse cues, which are routed to two lightweight state-space branches: a Spatial-SSM that learns location-conditioned filters for persistent artifacts, and a Temporal-SSM that models causal signal dynamics for bursty background events.

Result: EDmamba achieves 88.9K parameters and 2.27GFLOPs, enabling real-time throughput of 100K events in 68ms on a single GPU (36x faster than Transformer baselines). It establishes new state-of-the-art accuracy on four benchmarks, outperforming prior models by 2.1 percentage points.

Conclusion: The decoupled design effectively addresses spatial and temporal noise through separate mechanisms, demonstrating that independent processing of different noise types leads to superior performance with significantly reduced computational requirements.

Abstract: Event cameras provide micro-second latency and broad dynamic range, yet their raw streams are marred by spatial artifacts (e.g., hot pixels) and temporally inconsistent background activity. Existing methods jointly process the entire 4D event volume (x, y, p, t), forcing heavy spatio-temporal attention that inflates parameters, FLOPs, and latency. We introduce EDmamba, a compact event-denoising framework that embraces the key insight that spatial and temporal noise arise from different physical mechanisms and can therefore be suppressed independently. A polarity- and geometry-aware encoder first extracts coarse cues, which are then routed to two lightweight state-space branches: a Spatial-SSM that learns location-conditioned filters to silence persistent artifacts, and a Temporal-SSM that models causal signal dynamics to eliminate bursty background events. This decoupled design distills the network to only 88.9K parameters and 2.27GFLOPs, enabling real-time throughput of 100K events in 68ms on a single GPU, 36x faster than recent Transformer baselines. Despite its economy, EDmamba establishes new state-of-the-art accuracy on four public benchmarks, outscoring the strongest prior model by 2.1 percentage points.

Method: Created So-Fake-Set dataset with 2M+ high-quality images from 35 state-of-the-art generative models, established So-Fake-OOD benchmark with 100K out-of-domain images, and developed So-Fake-R1 vision-language framework using reinforcement learning for detection, localization, and explainable inference.

Result: So-Fake-R1 outperforms second-best method with 1.3% gain in detection accuracy and 4.5% increase in localization IoU, demonstrating superior performance in synthetic image detection.

Conclusion: This work establishes a new foundation for social media-centric forgery detection research by integrating scalable dataset, challenging OOD benchmark, and advanced detection framework, with all resources to be publicly released.

Abstract: Recent advances in AI-powered generative models have enabled the creation of increasingly realistic synthetic images, posing significant risks to information integrity and public trust on social media platforms. While robust detection frameworks and diverse, large-scale datasets are essential to mitigate these risks, existing academic efforts remain limited in scope: current datasets lack the diversity, scale, and realism required for social media contexts, while detection methods struggle with generalization to unseen generative technologies. To bridge this gap, we introduce So-Fake-Set, a comprehensive social media-oriented dataset with over 2 million high-quality images, diverse generative sources, and photorealistic imagery synthesized using 35 state-of-the-art generative models. To rigorously evaluate cross-domain robustness, we establish a novel and large-scale (100K) out-of-domain benchmark (So-Fake-OOD) featuring synthetic imagery from commercial models explicitly excluded from the training distribution, creating a realistic testbed for evaluating real-world performance. Leveraging these resources, we present So-Fake-R1, an advanced vision-language framework that employs reinforcement learning for highly accurate forgery detection, precise localization, and explainable inference through interpretable visual rationales. Extensive experiments show that So-Fake-R1 outperforms the second-best method, with a 1.3% gain in detection accuracy and a 4.5% increase in localization IoU. By integrating a scalable dataset, a challenging OOD benchmark, and an advanced detection framework, this work establishes a new foundation for social media-centric forgery detection research. The code, models, and datasets will be released publicly.

Method: A feed-forward network that takes uncalibrated image collections as input and produces 3D Gaussian primitives (encoding scene geometry and appearance) along with camera intrinsics and extrinsics for each input image in a single forward pass.

Result: Matches quality of pose-aware baselines in both sparse and dense view scenarios, surpasses existing pose-free approaches, and significantly reduces rendering latency compared to optimization-based neural fields, enabling real-time novel view synthesis.

Conclusion: AnySplat provides an efficient, scalable solution for novel view synthesis from unconstrained image collections without requiring pose annotations, making real-time rendering achievable for casually captured datasets.

Abstract: We introduce AnySplat, a feed forward network for novel view synthesis from uncalibrated image collections. In contrast to traditional neural rendering pipelines that demand known camera poses and per scene optimization, or recent feed forward methods that buckle under the computational weight of dense views, our model predicts everything in one shot. A single forward pass yields a set of 3D Gaussian primitives encoding both scene geometry and appearance, and the corresponding camera intrinsics and extrinsics for each input image. This unified design scales effortlessly to casually captured, multi view datasets without any pose annotations. In extensive zero shot evaluations, AnySplat matches the quality of pose aware baselines in both sparse and dense view scenarios while surpassing existing pose free approaches. Moreover, it greatly reduce rendering latency compared to optimization based neural fields, bringing real time novel view synthesis within reach for unconstrained capture settings.Project page: https://city-super.github.io/anysplat/

Method: Partially fine-tunes the latent decoder with Perturbation-Aware Suppression to freeze perceptually sensitive layers, plus a Temporal Alignment module for frame consistency. Enables implicit and adaptive watermark integration.

Result: Achieves best overall performance in watermark extraction, visual quality, and generation efficiency. Demonstrates strong robustness against spatial and temporal tampering.

Conclusion: VIDSIG provides a practical in-generation watermarking solution that effectively addresses intellectual property protection needs for AI-generated video content with superior performance.

Abstract: The rapid development of Artificial Intelligence Generated Content (AIGC) has led to significant progress in video generation but also raises serious concerns about intellectual property protection and reliable content tracing. Watermarking is a widely adopted solution to this issue, but existing methods for video generation mainly follow a post-generation paradigm, which introduces additional computational overhead and often fails to effectively balance the trade-off between video quality and watermark extraction. To address these issues, we propose Video Signature (VIDSIG), an in-generation watermarking method for latent video diffusion models, which enables implicit and adaptive watermark integration during generation. Specifically, we achieve this by partially fine-tuning the latent decoder, where Perturbation-Aware Suppression (PAS) pre-identifies and freezes perceptually sensitive layers to preserve visual quality. Beyond spatial fidelity, we further enhance temporal consistency by introducing a lightweight Temporal Alignment module that guides the decoder to generate coherent frame sequences during fine-tuning. Experimental results show that VIDSIG achieves the best overall performance in watermark extraction, visual quality, and generation efficiency. It also demonstrates strong robustness against both spatial and temporal tampering, highlighting its practicality in real-world scenarios. Our code is available at \href{https://github.com/hardenyu21/Video-Signature}{here}

Method: F3 employs a ‘fighting fire with fire’ strategy by intentionally introducing simple perturbations to adversarial examples and using cross-modal attentions from randomly perturbed adversary examples as reference targets to refine attention and purify outputs.

Result: The framework achieves impressive purification results, is training-free and straightforward to implement, and shows significant computational efficiency improvements compared to existing purification methods.

Conclusion: F3 provides an effective and efficient solution for defending against visual adversarial attacks, making it particularly suitable for large-scale industrial applications where both robust performance and operational efficiency are critical.

Abstract: Recent advances in large vision-language models (LVLMs) have showcased their remarkable capabilities across a wide range of multimodal vision-language tasks. However, these models remain vulnerable to visual adversarial attacks, which can substantially compromise their performance. In this paper, we introduce F3, a novel adversarial purification framework that employs a counterintuitive ``fighting fire with fire’’ strategy: intentionally introducing simple perturbations to adversarial examples to mitigate their harmful effects. Specifically, F3 leverages cross-modal attentions derived from randomly perturbed adversary examples as reference targets. By injecting noise into these adversarial examples, F3 effectively refines their attention, resulting in cleaner and more reliable model outputs. Remarkably, this seemingly paradoxical approach of employing noise to counteract adversarial attacks yields impressive purification results. Furthermore, F3 offers several distinct advantages: it is training-free and straightforward to implement, and exhibits significant computational efficiency improvements compared to existing purification methods. These attributes render F3 particularly suitable for large-scale industrial applications where both robust performance and operational efficiency are critical priorities. The code is available at https://github.com/btzyd/F3.

Method: Created HueManity benchmark with 83,850 images featuring alphanumeric strings in Ishihara-style dot patterns, evaluated 9 state-of-the-art MLLMs against human and ResNet50 baselines.

Result: MLLMs showed significant performance deficit: best MLLM achieved 33.6% on easy numeric task and only 3% on hard alphanumeric task, while humans scored 100%/95.6% and ResNet50 achieved 96.5%/94.5%.

Conclusion: Current MLLMs have critical gaps in visual perception capabilities, highlighting the need for architectural and training paradigm improvements to enhance perceptual robustness.

Abstract: Multimodal Large Language Models (MLLMs) excel at high-level visual reasoning, but their performance on nuanced perceptual tasks remains surprisingly limited. We present HueManity, a benchmark designed to assess visual perception in MLLMs. The dataset comprises 83,850 images featuring two-character alphanumeric strings embedded in Ishihara test style dot patterns, challenging models on precise pattern recognition. Our evaluation of nine state-of-the-art MLLMs on HueManity demonstrates a significant performance deficit compared to human and traditional computer vision baselines. The best-performing MLLM achieved a 33.6% accuracy on the numeric easy' task and a striking 3% on the alphanumeric hard’ task. In contrast, human participants achieved near-perfect scores (100% and 95.6%), and a fine-tuned ResNet50 model reached accuracies of 96.5% and 94.5%. These results highlight a critical gap in the visual capabilities of current MLLMs. Our analysis further explores potential architectural and training-paradigm factors contributing to this perceptual gap in MLLMs. We open-source HueManity dataset and code to foster further research in improving perceptual robustness of MLLMs.

Method: Self-supervised training approach to train DeepAquaCluster model that segments radar satellite images into water and land areas without manual annotations.

Result: Achieved 0.08 improvement in Intersection Over Union metric compared to other radar-based water detection techniques on test dataset.

Conclusion: Self-supervised learning enables effective wetland monitoring from radar satellite imagery without costly manual annotations, with superior performance over existing methods.

Abstract: In recent years the wide availability of high-resolution radar satellite images along with the advancement of computer vision models have enabled the remote monitoring of wetland surface areas. However, these models require large amounts of manually annotated satellite images, which are slow and expensive to produce. To overcome this problem we use self-supervised training methods to train a model called DeepAquaCluster to segment radar satellite images into areas that separate water from land without the use of any manual annotations. Our final model outperforms other radar-based water detection techniques in our test dataset, achieving a 0.08 improvement in the Intersection Over Union metric.

Method: Developed various PhilEO model variants with different parameter counts and architectures (U-Net CNNs to Vision Transformers), pretrained on 23TB MajorTOM dataset and 2TB FastTOM subset, then fine-tuned on PhilEO Bench for road density estimation, building density regression, and land cover segmentation.

Result: PhilEO 44M MajorTOM 23TB outperformed smaller dataset models for all n-shots in road density regression. PhilEO 200M FastTOM outperformed other models for most n-shots in road and building density estimation. Dataset and model scaling effectiveness was validated.

Conclusion: Both dataset scaling (23TB MajorTOM) and model scaling (up to 200M parameters) improve EO foundation model performance. Architecture transition from U-Net CNNs to Vision Transformers was also studied, showing the importance of scaling for Earth Observation tasks.

Abstract: Today, Earth Observation (EO) satellites generate massive volumes of data, with the Copernicus Sentinel-2 constellation alone producing approximately 1.6TB per day. To fully exploit this information, it is essential to pretrain EO Foundation Models (FMs) on large unlabeled datasets, enabling efficient fine-tuning for several different downstream tasks with minimal labeled data. In this work, we present the scaling-up of our recently proposed EO Foundation Model, PhilEO Geo-Aware U-Net, on the unlabeled 23TB dataset MajorTOM, which covers the vast majority of the Earth’s surface, as well as on the specialized subset FastTOM 2TB that does not include oceans and ice. We develop and study various PhilEO model variants with different numbers of parameters and architectures. We fine-tune the models on the PhilEO Bench for road density estimation, building density pixel-wise regression, and land cover semantic segmentation, and we evaluate the performance. Our results demonstrate that for all n-shots for road density regression, the PhilEO 44M MajorTOM 23TB model outperforms PhilEO Globe 0.5TB 44M. We also show that for most n-shots for road density estimation and building density regression, PhilEO 200M FastTOM outperforms all the other models we examine. The effectiveness of both dataset and model scaling is validated using the PhilEO Bench. We also study the impact of architecture scaling, transitioning from U-Net Convolutional Neural Networks (CNN) to Vision Transformers (ViT).

Method: Proposes LH2Face loss function with: 1) vMF distribution-based similarity measure, 2) Uncertainty-Aware Margin Function for adaptive margins, 3) proxy-based loss functions for space optimization, and 4) a renderer for face reconstruction optimization.

Result: Achieves 49.39% accuracy on IJB-B dataset, surpassing second-place method by 2.37%. Superior performance on hard high-quality face datasets compared to similar schemes.

Conclusion: LH2Face effectively addresses hard sample challenges in face recognition by incorporating adaptive margins and quality-aware constraints, demonstrating significant performance improvements over existing methods.

Abstract: In current practical face authentication systems, most face recognition (FR) algorithms are based on cosine similarity with softmax classification. Despite its reliable classification performance, this method struggles with hard samples. A popular strategy to improve FR performance is incorporating angular or cosine margins. However, it does not take face quality or recognition hardness into account, simply increasing the margin value and thus causing an overly uniform training strategy. To address this problem, a novel loss function is proposed, named Loss function for Hard High-quality Face (LH2Face). Firstly, a similarity measure based on the von Mises-Fisher (vMF) distribution is stated, specifically focusing on the logarithm of the Probability Density Function (PDF), which represents the distance between a probability distribution and a vector. Then, an adaptive margin-based multi-classification method using softmax, called the Uncertainty-Aware Margin Function, is implemented in the article. Furthermore, proxy-based loss functions are used to apply extra constraints between the proxy and sample to optimize their representation space distribution. Finally, a renderer is constructed that optimizes FR through face reconstruction and vice versa. Our LH2Face is superior to similiar schemes on hard high-quality face datasets, achieving 49.39% accuracy on the IJB-B dataset, which surpasses the second-place method by 2.37%.

Method: Proposes GGMotion with group graph modeling, radial fields for geometric equivariance, inter/intra-group interaction modules, equivariant MLPs for parallel dynamics-kinematics propagation, and auxiliary loss for motion priors supervision.

Result: Extensive experiments on Human3.6M, CMU-Mocap, and 3DPW benchmarks demonstrate effectiveness and superiority, achieving significant performance margin in short-term motion prediction.

Conclusion: GGMotion successfully models human topology in groups to leverage physical priors, improving motion prediction realism and performance through comprehensive spatio-temporal dependency capture and equivariant processing.

Abstract: Human motion is a continuous physical process in 3D space, governed by complex dynamic and kinematic constraints. Existing methods typically represent the human pose as an abstract graph structure, neglecting the intrinsic physical dependencies between joints, which increases learning difficulty and makes the model prone to generating unrealistic motions. In this paper, we propose GGMotion, a group graph dynamics-kinematics network that models human topology in groups to better leverage dynamics and kinematics priors. To preserve the geometric equivariance in 3D space, we propose a novel radial field for the graph network that captures more comprehensive spatio-temporal dependencies by aggregating joint features through spatial and temporal edges. Inter-group and intra-group interaction modules are employed to capture the dependencies of joints at different scales. Combined with equivariant multilayer perceptrons (MLP), joint position features are updated in each group through parallelized dynamics-kinematics propagation to improve physical plausibility. Meanwhile, we introduce an auxiliary loss to supervise motion priors during training. Extensive experiments on three standard benchmarks, including Human3.6M, CMU-Mocap, and 3DPW, demonstrate the effectiveness and superiority of our approach, achieving a significant performance margin in short-term motion prediction. The code is available at https://github.com/inkcat520/GGMotion.git.

Method: Leverages amodal completion to infer complete structure of occluded regions and integrates temporal context across video sequences to enforce coherence and stabilize reconstructions. Uses 3D Gaussian Splatting for validation.

Result: Superior precision in handling occlusions and maintaining temporal stability compared to existing techniques, with enhanced recovery of intricate details in dynamic scenes.

Conclusion: The template-free framework effectively addresses challenges of occlusions and temporal inconsistencies in dynamic human-object interaction reconstruction from monocular video.

Abstract: We introduce a novel framework for reconstructing dynamic human-object interactions from monocular video that overcomes challenges associated with occlusions and temporal inconsistencies. Traditional 3D reconstruction methods typically assume static objects or full visibility of dynamic subjects, leading to degraded performance when these assumptions are violated-particularly in scenarios where mutual occlusions occur. To address this, our framework leverages amodal completion to infer the complete structure of partially obscured regions. Unlike conventional approaches that operate on individual frames, our method integrates temporal context, enforcing coherence across video sequences to incrementally refine and stabilize reconstructions. This template-free strategy adapts to varying conditions without relying on predefined models, significantly enhancing the recovery of intricate details in dynamic scenes. We validate our approach using 3D Gaussian Splatting on challenging monocular videos, demonstrating superior precision in handling occlusions and maintaining temporal stability compared to existing techniques.

Method: Proposes a residual-based mixed representation with explicit coarse TSDF grid and implicit neural module that adds fine-grained details. Extends to multi-frame joint pose optimization via bundle adjustment with pose change optimization and adaptive gradient amplification. Uses local moving volume with divide-and-conquer design for efficient online learning.

Result: Extensive experiments show the method surpasses all state-of-the-art approaches (both explicit and implicit representations) in mapping and tracking accuracy on large-scale scenes.

Conclusion: The residual-based mixed representation enables detail-rich reconstruction with bounded time and memory budget, overcoming the limitations of purely implicit representations and enabling high-quality camera tracking for large-scale online reconstruction.

Abstract: The introduction of the neural implicit representation has notably propelled the advancement of online dense reconstruction techniques. Compared to traditional explicit representations, such as TSDF, it improves the mapping completeness and memory efficiency. However, the lack of reconstruction details and the time-consuming learning of neural representations hinder the widespread application of neural-based methods to large-scale online reconstruction. We introduce RemixFusion, a novel residual-based mixed representation for scene reconstruction and camera pose estimation dedicated to high-quality and large-scale online RGB-D reconstruction. In particular, we propose a residual-based map representation comprised of an explicit coarse TSDF grid and an implicit neural module that produces residuals representing fine-grained details to be added to the coarse grid. Such mixed representation allows for detail-rich reconstruction with bounded time and memory budget, contrasting with the overly-smoothed results by the purely implicit representations, thus paving the way for high-quality camera tracking. Furthermore, we extend the residual-based representation to handle multi-frame joint pose optimization via bundle adjustment (BA). In contrast to the existing methods, which optimize poses directly, we opt to optimize pose changes. Combined with a novel technique for adaptive gradient amplification, our method attains better optimization convergence and global optimality. Furthermore, we adopt a local moving volume to factorize the mixed scene representation with a divide-and-conquer design to facilitate efficient online learning in our residual-based framework. Extensive experiments demonstrate that our method surpasses all state-of-the-art ones, including those based either on explicit or implicit representations, in terms of the accuracy of both mapping and tracking on large-scale scenes.

Method: Uses cross-attention for initial seed localization, then leverages self-attention’s semantic correspondence modeling to iteratively expand regions through multi-scale attention maps, with background refinement.

Result: Produces high-quality segmentation masks off-the-shelf from Stable Diffusion without additional training or complex prompt engineering.

Conclusion: SeeDiff demonstrates that Stable Diffusion’s attention mechanisms alone can effectively generate pixel-level annotation masks, providing a training-free solution for semantic segmentation.

Abstract: Entrusted with the goal of pixel-level object classification, the semantic segmentation networks entail the laborious preparation of pixel-level annotation masks. To obtain pixel-level annotation masks for a given class without human efforts, recent few works have proposed to generate pairs of images and annotation masks by employing image and text relationships modeled by text-to-image generative models, especially Stable Diffusion. However, these works do not fully exploit the capability of text-guided Diffusion models and thus require a pre-trained segmentation network, careful text prompt tuning, or the training of a segmentation network to generate final annotation masks. In this work, we take a closer look at attention mechanisms of Stable Diffusion, from which we draw connections with classical seeded segmentation approaches. In particular, we show that cross-attention alone provides very coarse object localization, which however can provide initial seeds. Then, akin to region expansion in seeded segmentation, we utilize the semantic-correspondence-modeling capability of self-attention to iteratively spread the attention to the whole class from the seeds using multi-scale self-attention maps. We also observe that a simple-text-guided synthetic image often has a uniform background, which is easier to find correspondences, compared to complex-structured objects. Thus, we further refine a mask using a more accurate background mask. Our proposed method, dubbed SeeDiff, generates high-quality masks off-the-shelf from Stable Diffusion, without additional training procedure, prompt tuning, or a pre-trained segmentation network.

Method: Proposes Knowledge-Based Global Guidance (visual codebook for text-related features) and Dynamic pose Masking for fine-grained local control, injected at different diffusion stages to provide both global and local pose enhancement.

Result: Achieves state-of-the-art results on HumanArt dataset in terms of AP and CAP metrics, demonstrating improved pose accuracy without compromising image quality.

Conclusion: KB-DMGen successfully addresses the pose-quality trade-off through dual control mechanisms, setting new benchmarks for human image generation with diffusion models.

Abstract: Recent methods using diffusion models have made significant progress in Human Image Generation (HIG) with various control signals such as pose priors. In HIG, both accurate human poses and coherent visual quality are crucial for image generation. However, most existing methods mainly focus on pose accuracy while neglecting overall image quality, often improving pose alignment at the cost of image quality. To address this, we propose Knowledge-Based Global Guidance and Dynamic pose Masking for human image Generation (KB-DMGen). The Knowledge Base (KB), implemented as a visual codebook, provides coarse, global guidance based on input text-related visual features, improving pose accuracy while maintaining image quality, while the Dynamic pose Mask (DM) offers fine-grained local control to enhance precise pose accuracy. By injecting KB and DM at different stages of the diffusion process, our framework enhances pose accuracy through both global and local control without compromising image quality. Experiments demonstrate the effectiveness of KB-DMGen, achieving new state-of-the-art results in terms of AP and CAP on the HumanArt dataset. The project page and code are available at https://lushbng.github.io/KBDMGen.

Method: Developed MCITlib - a constantly evolving code library that implements 8 representative algorithms for multimodal continual instruction tuning and systematically evaluates them on 2 carefully selected benchmarks.

Result: The library provides a comprehensive framework for multimodal continual learning research and will be continuously updated to reflect advances in the field.

Conclusion: MCITlib serves as a valuable resource to facilitate and advance research in multimodal continual learning, particularly for instruction tuning of Multimodal Large Language Models, with ongoing updates to incorporate new developments.

Abstract: Continual learning aims to equip AI systems with the ability to continuously acquire and adapt to new knowledge without forgetting previously learned information, similar to human learning. While traditional continual learning methods focusing on unimodal tasks have achieved notable success, the emergence of Multimodal Large Language Models has brought increasing attention to Multimodal Continual Learning tasks involving multiple modalities, such as vision and language. In this setting, models are expected to not only mitigate catastrophic forgetting but also handle the challenges posed by cross-modal interactions and coordination. To facilitate research in this direction, we introduce MCITlib, a comprehensive and constantly evolving code library for continual instruction tuning of Multimodal Large Language Models. In MCITlib, we have currently implemented 8 representative algorithms for Multimodal Continual Instruction Tuning and systematically evaluated them on 2 carefully selected benchmarks. MCITlib will be continuously updated to reflect advances in the Multimodal Continual Learning field. The codebase is released at https://github.com/Ghy0501/MCITlib.

Method: Used GPT large language model to detect cultural heritage aspects from facade images, combined with building register data as features to train machine learning models for classifying multi-family and non-residential buildings.

Result: Achieved macro F1-score of 0.71 using combined register data and GPT-derived features, and 0.60 using only GPT data, validated against expert-created inventory.

Conclusion: The methodology enables higher-quality databases for supporting careful energy efficiency measures while considering heritage values in large-scale building refurbishment scenarios.

Abstract: Detailed data is required to quantify energy conservation measures in buildings, such as envelop retrofits, without compromising cultural heritage. Novel artificial intelligence tools may improve efficiency in identifying heritage values in buildings compared to costly and time-consuming traditional inventories. In this study, the large language model GPT was used to detect various aspects of cultural heritage value in fa\c{c}ade images. Using this data and building register data as features, machine learning models were trained to classify multi-family and non-residential buildings in Stockholm, Sweden. Validation against an expert-created inventory shows a macro F1-score of 0.71 using a combination of register data and features retrieved from GPT, and a score of 0.60 using only GPT-derived data. The presented methodology can contribute to a higher-quality database and thus support careful energy efficiency measures and integrated consideration of heritage value in large-scale energetic refurbishment scenarios.

Method: Uses a contour-aware scanline placement approach with weakly supervised B-mode landmark detection to infer LV long axis and basal level, then performs automated measurements in AMM mode combining structure and motion awareness.

Result: The method provides fully automated yet manually adaptable LV linear measurements with enhanced robustness and accuracy for clinical application.

Conclusion: WiseLVAM offers a practical solution for routine clinical application by automating the scanline placement and measurement process while maintaining clinical guideline compliance.

Abstract: Clinical guidelines recommend performing left ventricular (LV) linear measurements in B-mode echocardiographic images at the basal level – typically at the mitral valve leaflet tips – and aligned perpendicular to the LV long axis along a virtual scanline (SL). However, most automated methods estimate landmarks directly from B-mode images for the measurement task, where even small shifts in predicted points along the LV walls can lead to significant measurement errors, reducing their clinical reliability. A recent semi-automatic method, EnLVAM, addresses this limitation by constraining landmark prediction to a clinician-defined SL and training on generated Anatomical Motion Mode (AMM) images to predict LV landmarks along the same. To enable full automation, a contour-aware SL placement approach is proposed in this work, in which the LV contour is estimated using a weakly supervised B-mode landmark detector. SL placement is then performed by inferring the LV long axis and the basal level- mimicking clinical guidelines. Building on this foundation, we introduce \textit{WiseLVAM} – a novel, fully automated yet manually adaptable framework for automatically placing the SL and then automatically performing the LV linear measurements in the AMM mode. \textit{WiseLVAM} utilizes the structure-awareness from B-mode images and the motion-awareness from AMM mode to enhance robustness and accuracy with the potential to provide a practical solution for the routine clinical application. The source code is publicly available at https://github.com/SFI-Visual-Intelligence/wiselvam.git.

Method: Joint knowledge transfer and preservation that unifies multiple pre-trained teacher models in shared latent space to mitigate distributional gaps, plus a knowledge preservation strategy using a general-purpose teacher as knowledge base with adapter modules.

Result: Outperforms existing data-centric models across four fundamental vision tasks: image classification, object detection, semantic segmentation, and instance segmentation.

Conclusion: The proposed model-driven approach successfully creates a powerful VFM that inherits expertise from multiple teachers without large labeled datasets, providing generalizable features and supporting multiple downstream tasks.

Abstract: Vision foundation models (VFMs) are predominantly developed using data-centric methods. These methods require training on vast amounts of data usually with high-quality labels, which poses a bottleneck for most institutions that lack both large-scale data and high-end GPUs. On the other hand, many open-source vision models have been pretrained on domain-specific data, enabling them to distill and represent core knowledge in a form that is transferable across diverse applications. Even though these models are highly valuable assets, they remain largely under-explored in empowering the development of a general-purpose VFM. In this paper, we present a new model-driven approach for training VFMs through joint knowledge transfer and preservation. Our method unifies multiple pre-trained teacher models in a shared latent space to mitigate the ``imbalanced transfer’’ issue caused by their distributional gaps. Besides, we introduce a knowledge preservation strategy to take a general-purpose teacher as a knowledge base for integrating knowledge from the remaining purpose-specific teachers using an adapter module. By unifying and aggregating existing models, we build a powerful VFM to inherit teachers’ expertise without needing to train on a large amount of labeled data. Our model not only provides generalizable visual features, but also inherently supports multiple downstream tasks. Extensive experiments demonstrate that our VFM outperforms existing data-centric models across four fundamental vision tasks, including image classification, object detection, semantic and instance segmentation.

Method: Proposes RAG-SEG: a two-stage approach - 1) Retrieval-Augmented Generation (RAG) creates coarse masks as prompts via unsupervised clustering and feature retrieval, 2) SAM-based segmentation (SEG) refines the masks using the generated prompts.

Result: Extensive experiments show RAG-SEG performs on par with or surpasses state-of-the-art methods on benchmark COD datasets, all conducted on a personal laptop, demonstrating computational efficiency.

Conclusion: RAG-SEG provides an effective training-free solution for COD that maintains competitive performance while being computationally efficient and practical for deployment on standard hardware.

Abstract: Camouflaged object detection (COD) poses a significant challenge in computer vision due to the high similarity between objects and their backgrounds. Existing approaches often rely on heavy training and large computational resources. While foundation models such as the Segment Anything Model (SAM) offer strong generalization, they still struggle to handle COD tasks without fine-tuning and require high-quality prompts to yield good performance. However, generating such prompts manually is costly and inefficient. To address these challenges, we propose \textbf{First RAG, Second SEG (RAG-SEG)}, a training-free paradigm that decouples COD into two stages: Retrieval-Augmented Generation (RAG) for generating coarse masks as prompts, followed by SAM-based segmentation (SEG) for refinement. RAG-SEG constructs a compact retrieval database via unsupervised clustering, enabling fast and effective feature retrieval. During inference, the retrieved features produce pseudo-labels that guide precise mask generation using SAM2. Our method eliminates the need for conventional training while maintaining competitive performance. Extensive experiments on benchmark COD datasets demonstrate that RAG-SEG performs on par with or surpasses state-of-the-art methods. Notably, all experiments are conducted on a \textbf{personal laptop}, highlighting the computational efficiency and practicality of our approach. We present further analysis in the Appendix, covering limitations, salient object detection extension, and possible improvements. \textcolor{blue} {Code: https://github.com/Lwt-diamond/RAG-SEG.}

Method: Proposes Temporal Visual Screening (TVS) that: (1) retains focus-critical video segments, (2) synchronously reconstructs queries to their most direct form while preserving answer consistency, and (3) maintains answer invariance and consistency.

Result: TVS achieves relative gains of 7.33% in training and 34.6% in inference. The ReSimplifyIt baseline outperforms prior approaches by 0.47 in F-1 score on video trimming while achieving competitive query rewriting performance.

Conclusion: Temporal information screening through TVS effectively improves video-language understanding by optimizing reasoning burden distribution and cognitive load, demonstrating the importance of focusing on salient temporal segments.

Abstract: Humans naturally perform temporal screening by dragging the progress bar and focusing on salient temporal segments, but current Video Large Language Models (Video-LLMs) struggle to capture fine-grained temporal semantics due to sparse frame sampling and insufficient inter-frame reasoning supervision during their training. To address this, Inspired by well-established cognitive science principles, we propose Temporal Visual Screening (TVS), a new task that universally pre-processes video question answering and instruction tuning data by: (1) retaining focus-critical video segments, (2) synchronously reconstructing queries to their most direct form while preserving answer consistency, and (3) keeping the invariance and consistency for any possible answer. TVS is formulated as a modular front-end adapter task that can be seamlessly integrated into both Video Instruction Tuning (training) and Video Question Answering (inference) pipelines. TVS optimizes distribution of reasoning burden and cognitive load; during training, it aligns queries with focus-critical visual information; at inference, it enables query-aware segment focus and streamlined query representations. In particular, we curate the first benchmark for TVS and propose ReSimplifyIt, a baseline outperforming prior approaches on seemingly similar tasks by 0.47 in F-1 score on video trimming while achieving competitive query rewriting performance. Experiments demonstrate that incorporating TVS yields relative gains of 7.33% (training) and 34.6% (inference), demonstrating the effectiveness of temporal information screening for improving video-language understanding.

Method: Proposes a novel block-wise mixture of dynamic low-rank experts that adaptively selects different experts based on input features for weighted inference, combined with a self-supervised training framework to handle brightness inconsistency and reflectance interference.

Result: Outperforms state-of-the-art works on SCARED and SimCol datasets, achieves best generalization on zero-shot depth estimation across C3VD, Hamlyn and SERV-CT datasets, and demonstrates strong performance in 3D reconstruction and ego-motion estimation.

Conclusion: The method contributes to accurate endoscopy for minimally invasive measurement and surgery by providing robust depth estimation across diverse endoscopic conditions through efficient foundation model finetuning and adaptive expert selection.

Abstract: Self-supervised monocular depth estimation is a significant task for low-cost and efficient 3D scene perception in endoscopy. In recent years, a series of methods are proposed to address the illumination inconsistency, while certain works also focus on the generalization of the model by efficiently finetuning the foundation models. However, the variety of illumination conditions and scene features is still the primary challenges for depth estimation in endoscopic scenes. In this work, a self-supervised framework is proposed for monocular depth estimation in diverse endoscopy. Firstly, considering the diverse features in endoscopic scenes with different tissues, a novel block-wise mixture of dynamic low-rank experts is proposed to efficiently finetune the foundation model for endoscopic depth estimation. In the proposed module, based on the input feature, different experts with a small amount of trainable parameters are adaptively selected for weighted inference, from low-rank experts which are allocated based on the generalization of each block. Moreover, a novel self-supervised training framework is proposed to jointly cope with brightness inconsistency and reflectance interference. The proposed method outperforms state-of-the-art works on SCARED dataset and SimCol dataset. Furthermore, the proposed network also achieves the best generalization based on zero-shot depth estimation on C3VD, Hamlyn and SERV-CT dataset. The outstanding performance of our model is further demonstrated with 3D reconstruction and ego-motion estimation. The proposed method could contribute to accurate endoscopy for minimally invasive measurement and surgery. The evaluation codes will be released upon acceptance, while the demo videos can be found on: https://endo-gede.netlify.app/.

Method: Two-component framework: 1) PSA module uses fine-tuned LLM to decompose prompts into spatial coordinates and style instructions, 2) MED module uses attention-based gating to route generation to specialized experts (realism, stylization) for each spatial region.

Result: Significantly outperforms baseline models with same backbone in both image quality and style diversity.

Conclusion: MEPG provides an extensible framework that enables precise spatial control and diverse style generation through specialized expert routing, with real-time editing capabilities.

Abstract: Text-to-image diffusion models have achieved remarkable image quality, but they still struggle with complex, multiele ment prompts, and limited stylistic diversity. To address these limitations, we propose a Multi-Expert Planning and Gen eration Framework (MEPG) that synergistically integrates position- and style-aware large language models (LLMs) with spatial-semantic expert modules. The framework comprises two core components: (1) a Position-Style-Aware (PSA) module that utilizes a supervised fine-tuned LLM to decom pose input prompts into precise spatial coordinates and style encoded semantic instructions; and (2) a Multi-Expert Dif fusion (MED) module that implements cross-region genera tion through dynamic expert routing across both local regions and global areas. During the generation process for each lo cal region, specialized models (e.g., realism experts, styliza tion specialists) are selectively activated for each spatial par tition via attention-based gating mechanisms. The architec ture supports lightweight integration and replacement of ex pert models, providing strong extensibility. Additionally, an interactive interface enables real-time spatial layout editing and per-region style selection from a portfolio of experts. Ex periments show that MEPG significantly outperforms base line models with the same backbone in both image quality and style diversity.

Method: Proposes MVP-FAS with two modules: Multi-View Slot attention (MVS) to extract detailed spatial features and global context from patch embeddings using diverse texts, and Multi-Text Patch Alignment (MTPA) to align patches with multiple text representations for semantic robustness.

Result: Extensive experiments show MVP-FAS achieves superior generalization performance, outperforming previous state-of-the-art methods on cross-domain datasets.

Conclusion: The framework effectively addresses limitations of existing CLIP-based FAS models by leveraging multiple paraphrased texts and better utilizing patch embeddings, resulting in improved cross-domain anti-spoofing performance.

Abstract: Recent face anti-spoofing (FAS) methods have shown remarkable cross-domain performance by employing vision-language models like CLIP. However, existing CLIP-based FAS models do not fully exploit CLIP’s patch embedding tokens, failing to detect critical spoofing clues. Moreover, these models rely on a single text prompt per class (e.g., ’live’ or ‘fake’), which limits generalization. To address these issues, we propose MVP-FAS, a novel framework incorporating two key modules: Multi-View Slot attention (MVS) and Multi-Text Patch Alignment (MTPA). Both modules utilize multiple paraphrased texts to generate generalized features and reduce dependence on domain-specific text. MVS extracts local detailed spatial features and global context from patch embeddings by leveraging diverse texts with multiple perspectives. MTPA aligns patches with multiple text representations to improve semantic robustness. Extensive experiments demonstrate that MVP-FAS achieves superior generalization performance, outperforming previous state-of-the-art methods on cross-domain datasets. Code: https://github.com/Elune001/MVP-FAS.

Method: A novel pipeline for generating synthetic 3D stomach models informed by anatomical studies, creating a dataset, and developing a statistical shape model refined through semi-supervised alignment with real CT meshes from public datasets.

Result: The model demonstrated robust generalization and fit accuracy on a held-out test set of real stomach CT scans, successfully capturing natural anatomical variability in a low-dimensional shape space.

Conclusion: This work represents a significant advancement in organ modeling, combining synthetic data generation, parametric modeling, and real-world validation to enable applications in surgical simulation, pre-operative planning, medical education, and personalized healthcare.

Abstract: Realistic and parameterized 3D models of human anatomy have become invaluable in research, diagnostics, and surgical planning. However, the development of detailed models for internal organs, such as the stomach, has been limited by data availability and methodological challenges. In this paper, we propose a novel pipeline for the generation of synthetic 3D stomach models, enabling the creation of anatomically diverse morphologies informed by established studies on stomach shape variability. Using this pipeline, we construct a dataset of synthetic stomachs. Building on this dataset, we develop a 3D statistical shape model of the stomach, trained to capture natural anatomical variability in a low-dimensional shape space. The model is further refined using CT meshes derived from publicly available datasets through a semi-supervised alignment process, enhancing its ability to generalize to unseen anatomical variations. We evaluated the model on a held-out test set of real stomach CT scans, demonstrating robust generalization and fit accuracy. We make the statistical shape model along with the synthetic dataset publicly available on GitLab: https://gitlab.com/Erez.Posner/stomach_pytorch to facilitate further research. This work introduces the first statistical 3D shape model of the stomach, with applications ranging from surgical simulation and pre-operative planning to medical education and computational modeling. By combining synthetic data generation, parametric modeling, and real-world validation, our approach represents a significant advancement in organ modeling and opens new possibilities for personalized healthcare solutions.

Method: Uses CNN for spatial feature extraction from RGB, multispectral, and two infrared wavelength images, combined with LSTM for temporal dependencies in time-series data. Also tested spatial-only CNN for comparison.

Result: CNN-LSTM pipeline achieved 98% accuracy, significantly outperforming spatial-only model (80.45%) and previous machine learning methods (76%).

Conclusion: The CNN-LSTM approach effectively captures complex stress interactions and provides a robust platform for timely nitrogen stress identification, enabling better crop management.

Abstract: Plants in their natural habitats endure an array of interacting stresses, both biotic and abiotic, that rarely occur in isolation. Nutrient stress-particularly nitrogen deficiency-becomes even more critical when compounded with drought and weed competition, making it increasingly difficult to distinguish and address its effects. Early detection of nitrogen stress is therefore crucial for protecting plant health and implementing effective management strategies. This study proposes a novel deep learning framework to accurately classify nitrogen stress severity in a combined stress environment. Our model uses a unique blend of four imaging modalities-RGB, multispectral, and two infrared wavelengths-to capture a wide range of physiological plant responses from canopy images. These images, provided as time-series data, document plant health across three levels of nitrogen availability (low, medium, and high) under varying water stress and weed pressures. The core of our approach is a spatio-temporal deep learning pipeline that merges a Convolutional Neural Network (CNN) for extracting spatial features from images with a Long Short-Term Memory (LSTM) network to capture temporal dependencies. We also devised and evaluated a spatial-only CNN pipeline for comparison. Our CNN-LSTM pipeline achieved an impressive accuracy of 98%, impressively surpassing the spatial-only model’s 80.45% and other previously reported machine learning method’s 76%. These results bring actionable insights based on the power of our CNN-LSTM approach in effectively capturing the subtle and complex interactions between nitrogen deficiency, water stress, and weed pressure. This robust platform offers a promising tool for the timely and proactive identification of nitrogen stress severity, enabling better crop management and improved plant health.

Method: Reframes Re-ID as supervised image similarity task using Siamese network. Proposes Beta-SOD framework that models cosine similarity distribution with two-component Beta mixture model for outlier detection. Combines binary cross-entropy, contrastive, and cosine embedding losses.

Result: Superior performance compared to state-of-the-art methods across 10-30% noise levels on CUHK03, Market-1501 (person Re-ID) and VeRi-776 (vehicle Re-ID) datasets.

Conclusion: Beta-SOD demonstrates robust performance and broad applicability for noisy Re-ID scenarios, with proven identifiability of Beta mixture models ensuring well-posed learning.

Abstract: Object re-identification (Re-ID) methods are highly sensitive to label noise, which typically leads to significant performance degradation. We address this challenge by reframing Re-ID as a supervised image similarity task and adopting a Siamese network architecture trained to capture discriminative pairwise relationships. Central to our approach is a novel statistical outlier detection (OD) framework, termed Beta-SOD (Beta mixture Similarity-based Outlier Detection), which models the distribution of cosine similarities between embedding pairs using a two-component Beta distribution mixture model. We establish a novel identifiability result for mixtures of two Beta distributions, ensuring that our learning task is well-posed. The proposed OD step complements the Re-ID architecture combining binary cross-entropy, contrastive, and cosine embedding losses that jointly optimize feature-level similarity learning. We demonstrate the effectiveness of Beta-SOD in de-noising and Re-ID tasks for person Re-ID, on CUHK03 and Market-1501 datasets, and vehicle Re-ID, on VeRi-776 dataset. Our method shows superior performance compared to the state-of-the-art methods across various noise levels (10-30%), demonstrating both robustness and broad applicability in noisy Re-ID scenarios. The implementation of Beta-SOD is available at: github.com/waqar3411/Beta-SOD

Method: Uses implicit neural representations (INRs) conditioned on learned latent codes to predict continuous left ventricular displacement without requiring inference-time optimization.

Result: Achieved best tracking accuracy (2.14 mm RMSE) and lowest combined error in global circumferential (2.86%) and radial (6.42%) strain on 452 UK Biobank test cases, while being ~380x faster than the most accurate baseline.

Conclusion: INR-based models are highly suitable for accurate and scalable analysis of myocardial strain in large cardiac MR datasets, offering both superior performance and computational efficiency.

Abstract: Automatic quantification of intramyocardial motion and strain from tagging MRI remains an important but challenging task. We propose a method using implicit neural representations (INRs), conditioned on learned latent codes, to predict continuous left ventricular (LV) displacement – without requiring inference-time optimisation. Evaluated on 452 UK Biobank test cases, our method achieved the best tracking accuracy (2.14 mm RMSE) and the lowest combined error in global circumferential (2.86%) and radial (6.42%) strain compared to three deep learning baselines. In addition, our method is $\sim$380$\times$ faster than the most accurate baseline. These results highlight the suitability of INR-based models for accurate and scalable analysis of myocardial strain in large CMR datasets.

Method: Multi-task learning approach that incorporates fixed object information and interaction area data along with human skeleton poses. Collected real-world data including hands-on fixed objects (ATM machines, check-in/out machines) and non-interaction actions.

Result: Achieved 99.25% accuracy in recognizing interaction and non-interaction actions, which is 2.75% higher than base models using only human skeleton poses.

Conclusion: Incorporating fixed object information through multi-task learning significantly improves human action recognition performance for human-object interaction scenarios in public environments.

Abstract: Recent graph convolutional neural networks (GCNs) have shown high performance in the field of human action recognition by using human skeleton poses. However, it fails to detect human-object interaction cases successfully due to the lack of effective representation of the scene information and appropriate learning architectures. In this context, we propose a methodology to utilize human action recognition performance by considering fixed object information in the environment and following a multi-task learning approach. In order to evaluate the proposed method, we collected real data from public environments and prepared our data set, which includes interaction classes of hands-on fixed objects (e.g., ATM ticketing machines, check-in/out machines, etc.) and non-interaction classes of walking and standing. The multi-task learning approach, along with interaction area information, succeeds in recognizing the studied interaction and non-interaction actions with an accuracy of 99.25%, outperforming the accuracy of the base model using only human skeleton poses by 2.75%.

Method: Proposes IRDFusion framework with two novel modules: Mutual Feature Refinement Module (MFRM) that enhances intra- and inter-modal feature representations, and Differential Feature Feedback Module (DFFM) that computes inter-modal differential features as guidance signals. These are integrated into an Iterative Relation-Map Differential Guided Feature Fusion mechanism.

Result: Achieves state-of-the-art performance on FLIR, LLVIP and M³FD datasets, consistently outperforming existing methods across diverse challenging scenarios, demonstrating robustness and effectiveness.

Conclusion: IRDFusion enables high-quality cross-modal fusion by progressively amplifying salient relational signals through iterative feedback while suppressing feature noise, leading to significant performance gains in multispectral object detection.

Abstract: Current multispectral object detection methods often retain extraneous background or noise during feature fusion, limiting perceptual performance. To address this, we propose an innovative feature fusion framework based on cross-modal feature contrastive and screening strategy, diverging from conventional approaches. The proposed method adaptively enhances salient structures by fusing object-aware complementary cross-modal features while suppressing shared background interference. Our solution centers on two novel, specially designed modules: the Mutual Feature Refinement Module (MFRM) and the Differential Feature Feedback Module (DFFM). The MFRM enhances intra- and inter-modal feature representations by modeling their relationships, thereby improving cross-modal alignment and discriminative power. Inspired by feedback differential amplifiers, the DFFM dynamically computes inter-modal differential features as guidance signals and feeds them back to the MFRM, enabling adaptive fusion of complementary information while suppressing common-mode noise across modalities. To enable robust feature learning, the MFRM and DFFM are integrated into a unified framework, which is formally formulated as an Iterative Relation-Map Differential Guided Feature Fusion mechanism, termed IRDFusion. IRDFusion enables high-quality cross-modal fusion by progressively amplifying salient relational signals through iterative feedback, while suppressing feature noise, leading to significant performance gains. In extensive experiments on FLIR, LLVIP and M$^3$FD datasets, IRDFusion achieves state-of-the-art performance and consistently outperforms existing methods across diverse challenging scenarios, demonstrating its robustness and effectiveness. Code will be available at https://github.com/61s61min/IRDFusion.git.

Method: Progressive layer freezing schedule based on layer convergence order, applied to large masked autoencoding models up to 4B parameters.

Result: Achieved superior performance compared to non-latent masked prediction on the 4DS perception suite benchmark.

Conclusion: LayerLock provides a simple yet effective approach for self-supervised visual representation learning through progressive layer freezing, enabling scalable latent prediction without collapse issues.

Abstract: We introduce LayerLock, a simple yet effective approach for self-supervised visual representation learning, that gradually transitions from pixel to latent prediction through progressive layer freezing. First, we make the observation that during training of video masked-autoencoding (MAE) models, ViT layers converge in the order of their depth: shallower layers converge early, deeper layers converge late. We then show that this observation can be exploited to accelerate standard MAE by progressively freezing the model according to an explicit schedule, throughout training. Furthermore, this same schedule can be used in a simple and scalable approach to latent prediction that does not suffer from “representation collapse”. We apply our proposed approach, LayerLock, to large models of up to 4B parameters with results surpassing those of non-latent masked prediction on the 4DS perception suite.

Method: Used the SPEED+ dataset to compare K-Planes, Gaussian Splatting, and Convex Splatting methods, analyzing how appearance embeddings affect both photometric fidelity and geometric accuracy.

Result: Appearance embeddings improve photometric fidelity by modeling lighting variation but do not enhance geometric accuracy. Convex splatting achieves more compact and clutter-free representations than Gaussian splatting.

Conclusion: Appearance embeddings have limited value for geometry-centric tasks in space scenarios. Convex splatting offers advantages for safety-critical applications like interaction and collision avoidance due to its efficient representation.

Abstract: We present the first systematic comparison of implicit and explicit Novel View Synthesis methods for space-based 3D object reconstruction, evaluating the role of appearance embeddings. While embeddings improve photometric fidelity by modeling lighting variation, we show they do not translate into meaningful gains in geometric accuracy - a critical requirement for space robotics applications. Using the SPEED+ dataset, we compare K-Planes, Gaussian Splatting, and Convex Splatting, and demonstrate that embeddings primarily reduce the number of primitives needed for explicit methods rather than enhancing geometric fidelity. Moreover, convex splatting achieves more compact and clutter-free representations than Gaussian splatting, offering advantages for safety-critical applications such as interaction and collision avoidance. Our findings clarify the limits of appearance embeddings for geometry-centric tasks and highlight trade-offs between reconstruction quality and representation efficiency in space scenarios.

Method: The authors conduct a systematic survey of representative works in visual grounding research. They outline the importance of grounding, delineate core components of contemporary grounded model development, examine practical applications and evaluation metrics, and analyze relationships with multimodal reasoning approaches.

Result: The survey provides a comprehensive overview of the current state of visual grounding research in VLMs, including key methodologies, applications, benchmarks, and evaluation frameworks. It identifies the interconnections between visual grounding, multimodal chain-of-thought, and reasoning capabilities.

Conclusion: The paper concludes by analyzing the challenges in visual grounding and suggesting promising future research directions, highlighting the importance of this capability for advancing multimodal AI systems and their practical applications across various domains.

Abstract: Visual grounding refers to the ability of a model to identify a region within some visual input that matches a textual description. Consequently, a model equipped with visual grounding capabilities can target a wide range of applications in various domains, including referring expression comprehension, answering questions pertinent to fine-grained details in images or videos, caption visual context by explicitly referring to entities, as well as low and high-level control in simulated and real environments. In this survey paper, we review representative works across the key areas of research on modern general-purpose vision language models (VLMs). We first outline the importance of grounding in VLMs, then delineate the core components of the contemporary paradigm for developing grounded models, and examine their practical applications, including benchmarks and evaluation metrics for grounded multimodal generation. We also discuss the multifaceted interrelations among visual grounding, multimodal chain-of-thought, and reasoning in VLMs. Finally, we analyse the challenges inherent to visual grounding and suggest promising directions for future research.

Method: Developed fuzzy inference systems (FIS) model using traffic, meteorology, and emission data measured in Prague, Czech Republic.

Result: Created predictive model for emission changes under various conditions, providing insights into emission quantity and dispersion patterns.

Conclusion: The model offers urban planners and policymakers tools to manage urban transport more effectively while considering environmental protection.

Abstract: Air pollution in cities and the possibilities of reducing this pollution represents one of the most important factors that today’s society has to deal with. This paper focuses on a systemic approach to traffic emissions with their relation to meteorological conditions, analyzing the effect of weather on the quantity and dispersion of traffic emissions in a city. Using fuzzy inference systems (FIS) the model for prediction of changes in emissions depending on various conditions is developed. The proposed model is based on traffic, meteorology and emission data measured in Prague, Czech Republic. The main objective of the work is to provide insight into how urban planners and policymakers can plan and manage urban transport more effectively with environmental protection in mind.

Method: Two AI model approach: one transforms imperative prompts into interventions for simulated cells, another scores how well prompts describe resulting dynamics, with the first model evolved to improve scores from the second.

Result: The evolved system generalizes to unseen prompts without retraining, demonstrating that natural language alone can guide collective behavior without domain-specific prompt design or engineered fitness functions.

Conclusion: Natural language can serve as a control layer to direct computational, robotic, or biological systems, suggesting a future where language replaces mathematical objective functions and domain-specific programming in AI-biology partnerships.

Abstract: Human language is one of the most expressive tools for conveying intent, yet most artificial or biological systems lack mechanisms to interpret or respond meaningfully to it. Bridging this gap could enable more natural forms of control over complex, decentralized systems. In AI and artificial life, recent work explores how language can specify high-level goals, but most systems still depend on engineered rewards, task-specific supervision, or rigid command sets, limiting generalization to novel instructions. Similar constraints apply in synthetic biology and bioengineering, where the locus of control is often genomic rather than environmental perturbation. A key open question is whether artificial or biological collectives can be guided by free-form natural language alone, without task-specific tuning or carefully designed evaluation metrics. We provide one possible answer here by showing, for the first time, that simple agents’ collective behavior can be guided by free-form language prompts: one AI model transforms an imperative prompt into an intervention that is applied to simulated cells; a second AI model scores how well the prompt describes the resulting cellular dynamics; and the former AI model is evolved to improve the scores generated by the latter. Unlike previous work, our method does not require engineered fitness functions or domain-specific prompt design. We show that the evolved system generalizes to unseen prompts without retraining. By treating natural language as a control layer, the system suggests a future in which spoken or written prompts could direct computational, robotic, or biological systems to desired behaviors. This work provides a concrete step toward this vision of AI-biology partnerships, in which language replaces mathematical objective functions, fixed rules, and domain-specific programming.

Method: Uses two key innovations: 1) Self-critique with specialized MLLM agents as critics to identify weaknesses and provide edit signals, and 2) Self-evolution using MLLM-as-a-judge for head-to-head comparisons between iteratively generated images to evolve better prompts.

Result: Extensive experiments show Maestro significantly improves image quality over initial prompts and state-of-the-art automated methods, with effectiveness scaling with more advanced MLLM components.

Conclusion: Maestro presents a robust, interpretable, and effective pathway towards self-improving text-to-image generation that reduces reliance on human intervention.

Abstract: Text-to-image (T2I) models, while offering immense creative potential, are highly reliant on human intervention, posing significant usability challenges that often necessitate manual, iterative prompt engineering over often underspecified prompts. This paper introduces Maestro, a novel self-evolving image generation system that enables T2I models to autonomously self-improve generated images through iterative evolution of prompts, using only an initial prompt. Maestro incorporates two key innovations: 1) self-critique, where specialized multimodal LLM (MLLM) agents act as ‘critics’ to identify weaknesses in generated images, correct for under-specification, and provide interpretable edit signals, which are then integrated by a ‘verifier’ agent while preserving user intent; and 2) self-evolution, utilizing MLLM-as-a-judge for head-to-head comparisons between iteratively generated images, eschewing problematic images, and evolving creative prompt candidates that align with user intents. Extensive experiments on complex T2I tasks using black-box models demonstrate that Maestro significantly improves image quality over initial prompts and state-of-the-art automated methods, with effectiveness scaling with more advanced MLLM components. This work presents a robust, interpretable, and effective pathway towards self-improving T2I generation.

Method: Analyzed vision-language descriptions from NVIDIA’s Describe Anything Model evaluated by three GPT variants. Used Gemini 2.5 Pro as independent question generator for controlled experiments. Conducted cross-family analysis through semantic similarity of generated questions.

Result: GPT-4o-mini shows systematic consistency, GPT-4o excels at error detection, GPT-5 exhibits extreme conservatism with high variability. GPT models demonstrate consistent 2:1 bias favoring negative assessment and cluster together, while Gemini shows markedly different evaluation strategies.

Conclusion: Evaluation competence does not scale with general AI capability. Robust AI assessment requires diverse architectural perspectives to avoid family-specific biases and ensure comprehensive evaluation.

Abstract: As AI systems increasingly evaluate other AI outputs, understanding their assessment behavior becomes crucial for preventing cascading biases. This study analyzes vision-language descriptions generated by NVIDIA’s Describe Anything Model and evaluated by three GPT variants (GPT-4o, GPT-4o-mini, GPT-5) to uncover distinct “evaluation personalities” the underlying assessment strategies and biases each model demonstrates. GPT-4o-mini exhibits systematic consistency with minimal variance, GPT-4o excels at error detection, while GPT-5 shows extreme conservatism with high variability. Controlled experiments using Gemini 2.5 Pro as an independent question generator validate that these personalities are inherent model properties rather than artifacts. Cross-family analysis through semantic similarity of generated questions reveals significant divergence: GPT models cluster together with high similarity while Gemini exhibits markedly different evaluation strategies. All GPT models demonstrate a consistent 2:1 bias favoring negative assessment over positive confirmation, though this pattern appears family-specific rather than universal across AI architectures. These findings suggest that evaluation competence does not scale with general capability and that robust AI assessment requires diverse architectural perspectives.

Method: Created GEO-16 framework with 16 quality pillars, collected 1,702 citations from 3 AI engines (Brave Summary, Google AI Overviews, Perplexity) using 70 product intent prompts, and audited 1,100 unique URLs using logistic models with domain clustered standard errors.

Result: Engines differed in citation quality, with overall page quality being a strong predictor of citation. Pages with GEO score ≥0.70 and at least 12 pillar hits had substantially higher citation rates. Metadata, Freshness, Semantic HTML, and Structured Data pillars showed strongest associations.

Conclusion: Page quality significantly influences AI engine citation behavior, and publishers can improve citation chances by focusing on specific quality pillars identified in the study, though limitations include observational nature and focus on English B2B SaaS pages.

Abstract: AI answer engines increasingly mediate access to domain knowledge by generating responses and citing web sources. We introduce GEO-16, a 16 pillar auditing framework that converts on page quality signals into banded pillar scores and a normalized GEO score G that ranges from 0 to 1. Using 70 product intent prompts, we collected 1,702 citations across three engines (Brave Summary, Google AI Overviews, and Perplexity) and audited 1,100 unique URLs. In our corpus, the engines differed in the GEO quality of the pages they cited, and pillars related to Metadata and Freshness, Semantic HTML, and Structured Data showed the strongest associations with citation. Logistic models with domain clustered standard errors indicate that overall page quality is a strong predictor of citation, and simple operating points (for example, G at least 0.70 combined with at least 12 pillar hits) align with substantially higher citation rates in our data. We report per engine contrasts, vertical effects, threshold analysis, and diagnostics, then translate findings into a practical playbook for publishers. The study is observational and focuses on English language B2B SaaS pages; we discuss limitations, threats to validity, and reproducibility considerations.

Method: Enterprise-specific benchmark evaluating 18 distinct agentic configurations across state-of-the-art LLMs, examining four dimensions: orchestration strategy, agent prompt implementation, memory architecture, and thinking tool integration.

Result: Revealed significant model-specific architectural preferences challenging one-size-fits-all paradigm. Found significant weaknesses in agentic performance with highest scoring models achieving only 35.3% success on complex tasks and 70.8% on simpler tasks.

Conclusion: Findings should inform future agentic system design by enabling more empirically backed decisions regarding architectural components and model selection.

Abstract: While individual components of agentic architectures have been studied in isolation, there remains limited empirical understanding of how different design dimensions interact within complex multi-agent systems. This study aims to address these gaps by providing a comprehensive enterprise-specific benchmark evaluating 18 distinct agentic configurations across state-of-the-art large language models. We examine four critical agentic system dimensions: orchestration strategy, agent prompt implementation (ReAct versus function calling), memory architecture, and thinking tool integration. Our benchmark reveals significant model-specific architectural preferences that challenge the prevalent one-size-fits-all paradigm in agentic AI systems. It also reveals significant weaknesses in overall agentic performance on enterprise tasks with the highest scoring models achieving a maximum of only 35.3% success on the more complex task and 70.8% on the simpler task. We hope these findings inform the design of future agentic systems by enabling more empirically backed decisions regarding architectural components and model selection.

Method: Four-step EMM algorithm: (1) factor identification, (2) hierarchical structuring of factors, (3) generating generalized expert mental model specification, and (4) creating detailed model from specification. Uses optimized human-machine dialogue and monotone Boolean/k-valued functions.

Result: The approach enables computationally tractable personal expert mental models that can handle decision-making tasks with incomplete information, demonstrated through a call for proposals evaluation example.

Conclusion: The proposed EMM framework provides a systematic way to capture and implement complex expert decision-making models in LLMs, addressing limitations of current approaches like RAG and overcoming information gaps through structured human-machine collaboration.

Abstract: Difficult decision-making problems abound in various disciplines and domains. The proliferation of generative techniques, especially large language models (LLMs), has excited interest in using them for decision support. However, LLMs cannot yet resolve missingness in their training data, leading to hallucinations. Retrieval-Augmented Generation (RAG) enhances LLMs by incorporating external information retrieval, reducing hallucinations and improving accuracy. Yet, RAG and related methods are only partial solutions, as they may lack access to all necessary sources or key missing information. Even everyday issues often challenge LLMs’ abilities. Submitting longer prompts with context and examples is one approach to address knowledge gaps, but designing effective prompts is non-trivial and may not capture complex mental models of domain experts. For tasks with missing critical information, LLMs are insufficient, as are many existing systems poorly represented in available documents. This paper explores how LLMs can make decision-making more efficient, using a running example of evaluating whether to respond to a call for proposals. We propose a technology based on optimized human-machine dialogue and monotone Boolean and k-valued functions to discover a computationally tractable personal expert mental model (EMM) of decision-making. Our EMM algorithm for LLM prompt engineering has four steps: (1) factor identification, (2) hierarchical structuring of factors, (3) generating a generalized expert mental model specification, and (4) generating a detailed generalized expert mental model from that specification.

Method: A fully decentralized swarm system where bar-wise agents coordinate via stigmergic signals, sensing and depositing harmonic, rhythmic, and structural cues while adapting short-term memory and reaching consensus.

Result: The swarm yields superior musical quality with greater diversity and structural variety across symbolic, audio, and graph-theoretic analyses, forming stable complementary roles and small-world architecture with efficient connectivity.

Conclusion: MusicSwarm provides a compute- and data-efficient approach to long-horizon creative structure that can be applied beyond music to collaborative writing, design, and scientific discovery.

Abstract: We show that coherent, long-form musical composition can emerge from a decentralized swarm of identical, frozen foundation models that coordinate via stigmergic, peer-to-peer signals, without any weight updates. We compare a centralized multi-agent system with a global critic to a fully decentralized swarm in which bar-wise agents sense and deposit harmonic, rhythmic, and structural cues, adapt short-term memory, and reach consensus. Across symbolic, audio, and graph-theoretic analyses, the swarm yields superior quality while delivering greater diversity and structural variety and leads across creativity metrics. The dynamics contract toward a stable configuration of complementary roles, and self-similarity networks reveal a small-world architecture with efficient long-range connectivity and specialized bridging motifs, clarifying how local novelties consolidate into global musical form. By shifting specialization from parameter updates to interaction rules, shared memory, and dynamic consensus, MusicSwarm provides a compute- and data-efficient route to long-horizon creative structure that is immediately transferable beyond music to collaborative writing, design, and scientific discovery.

Method: Four-component system with FSM-based routing: Controller, Manager, three specialized Workers (Technician, Operator, Analyst), and Evaluator for dynamic strategy selection and quality gating.

Result: Achieves 57.07% success rate in 50 steps on OSWorld benchmark, substantially outperforming existing methods.

Conclusion: Principled multi-agent orchestration with continuous quality control provides superior reliability for generalized desktop automation in complex environments.

Abstract: Autonomous agents for desktop automation struggle with complex multi-step tasks due to poor coordination and inadequate quality control. We introduce \textsc{Agentic Lybic}, a novel multi-agent system where the entire architecture operates as a finite-state machine (FSM). This core innovation enables dynamic orchestration. Our system comprises four components: a Controller, a Manager, three Workers (Technician for code-based operations, Operator for GUI interactions, and Analyst for decision support), and an Evaluator. The critical mechanism is the FSM-based routing between these components, which provides flexibility and generalization by dynamically selecting the optimal execution strategy for each subtask. This principled orchestration, combined with robust quality gating, enables adaptive replanning and error recovery. Evaluated officially on the OSWorld benchmark, \textsc{Agentic Lybic} achieves a state-of-the-art 57.07% success rate in 50 steps, substantially outperforming existing methods. Results demonstrate that principled multi-agent orchestration with continuous quality control provides superior reliability for generalized desktop automation in complex computing environments.

Method: Proposes Logic-constrained Vector Symbolic Architecture (LVSA) with a differentiable Skolemization module, neural negator, and logical constraint-driven optimization to harmonize geometric and logical requirements.

Result: LVSA theoretically guarantees universality for all EFO$_1$ queries, empirically outperforms state-of-the-art Skolemization-based methods, and reduces inference costs by orders of magnitude compared to Grounding-based baselines.

Conclusion: LVSA successfully bridges the gap between logical soundness and computational efficiency in complex query answering, providing a unified neuro-symbolic framework that maintains logical consistency while being computationally efficient.

Abstract: Complex Query Answering (CQA) over incomplete Knowledge Graphs (KGs), typically formalized as reasoning with Existential First-Order predicate logic with one free variable (EFO$_1$), faces a fundamental trade-off between logical soundness and computational efficiency. This work establishes the Grounding-Skolemization dichotomy for systematically analyzing CQA methods through the lens of formal logic. While Grounding-based methods inherently suffer from combinatorial explosion, most Skolemization-based methods neglect to explicitly model Skolem functions and compromise logical consistency. To address these limitations, we propose the Logic-constrained Vector Symbolic Architecture (LVSA), a neuro-symbolic framework that unifies a differentiable Skolemization module and a neural negator, as well as a logical constraint-driven optimization protocol to harmonize geometric and logical requirements. Theoretically, LVSA guarantees universality for all EFO$_1$ queries. Empirically, it outperforms state-of-the-art Skolemization-based methods and reduces inference costs by orders of magnitude compared to Grounding-based baselines.

Method: Embedding Artificial Neural Networks (ANNs) as local decision-making centers within cellular automata, enabling localized interactions that collectively produce coordinated system-level outcomes through iterative state-refinement.

Result: NCAs successfully reproduce biological patterns, generalize to novel conditions, demonstrate robustness to perturbations, and show capabilities in robotic control, regeneration, and even advanced reasoning tasks like ARC-AGI-1.

Conclusion: NCAs represent a computationally efficient paradigm that bridges multiscale biology with modern generative AI, offering potential for designing bio-inspired collective intelligence capable of hierarchical reasoning and decentralized control.

Abstract: Neural Cellular Automata (NCA) represent a powerful framework for modeling biological self-organization, extending classical rule-based systems with trainable, differentiable (or evolvable) update rules that capture the adaptive self-regulatory dynamics of living matter. By embedding Artificial Neural Networks (ANNs) as local decision-making centers and interaction rules between localized agents, NCA can simulate processes across molecular, cellular, tissue, and system-level scales, offering a multiscale competency architecture perspective on evolution, development, regeneration, aging, morphogenesis, and robotic control. These models not only reproduce biologically inspired target patterns but also generalize to novel conditions, demonstrating robustness to perturbations and the capacity for open-ended adaptation and reasoning. Given their immense success in recent developments, we here review current literature of NCAs that are relevant primarily for biological or bioengineering applications. Moreover, we emphasize that beyond biology, NCAs display robust and generalizing goal-directed dynamics without centralized control, e.g., in controlling or regenerating composite robotic morphologies or even on cutting-edge reasoning tasks such as ARC-AGI-1. In addition, the same principles of iterative state-refinement is reminiscent to modern generative Artificial Intelligence (AI), such as probabilistic diffusion models. Their governing self-regulatory behavior is constraint to fully localized interactions, yet their collective behavior scales into coordinated system-level outcomes. We thus argue that NCAs constitute a unifying computationally lean paradigm that not only bridges fundamental insights from multiscale biology with modern generative AI, but have the potential to design truly bio-inspired collective intelligence capable of hierarchical reasoning and control.

Method: Systematic review of relevant literature to deconstruct the agent paradigm across various AI frameworks, distinguishing between agentic systems (AI inspired by agency), agential systems (fully autonomous biological systems), and non-agentic systems (tools without agency impression).

Result: The analysis reveals challenges in defining and measuring properties like autonomy and goal-directedness in AI systems. The agentic framing of many AI systems, particularly LLMs, can be misleading and obscure underlying computational mechanisms.

Conclusion: A fundamental shift is needed toward frameworks grounded in system-level dynamics, world modeling, and material intelligence. Investigating non-agentic and systemic frameworks inspired by complex systems, biology, and unconventional computing is essential for advancing toward robust, scalable, and potentially non-anthropomorphic general intelligence.

Abstract: The concept of the ‘agent’ has profoundly shaped Artificial Intelligence (AI) research, guiding development from foundational theories to contemporary applications like Large Language Model (LLM)-based systems. This paper critically re-evaluates the necessity and optimality of this agent-centric paradigm. We argue that its persistent conceptual ambiguities and inherent anthropocentric biases may represent a limiting framework. We distinguish between agentic systems (AI inspired by agency, often semi-autonomous, e.g., LLM-based agents), agential systems (fully autonomous, self-producing systems, currently only biological), and non-agentic systems (tools without the impression of agency). Our analysis, based on a systematic review of relevant literature, deconstructs the agent paradigm across various AI frameworks, highlighting challenges in defining and measuring properties like autonomy and goal-directedness. We argue that the ‘agentic’ framing of many AI systems, while heuristically useful, can be misleading and may obscure the underlying computational mechanisms, particularly in Large Language Models (LLMs). As an alternative, we propose a shift in focus towards frameworks grounded in system-level dynamics, world modeling, and material intelligence. We conclude that investigating non-agentic and systemic frameworks, inspired by complex systems, biology, and unconventional computing, is essential for advancing towards robust, scalable, and potentially non-anthropomorphic forms of general intelligence. This requires not only new architectures but also a fundamental reconsideration of our understanding of intelligence itself, moving beyond the agent metaphor.

Method: HaPLa uses two strategies: 1) abductive framing - instructing LLMs to infer plausible intermediate steps toward harmful activities instead of direct responses, and 2) symbolic encoding - obfuscating harmful content since LLMs remain sensitive to explicit harmful keywords.

Result: Achieves over 95% attack success rate on GPT-series models and 70% across all target models. Analysis shows it’s difficult to safely tune LLMs without significantly diminishing their helpfulness for benign queries.

Conclusion: The research demonstrates effective jailbreaking techniques and reveals fundamental challenges in balancing LLM safety with maintaining helpfulness, highlighting the need for stronger defense mechanisms.

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse tasks, but their potential misuse for harmful purposes remains a significant concern. To strengthen defenses against such vulnerabilities, it is essential to investigate universal jailbreak attacks that exploit intrinsic weaknesses in the architecture and learning paradigms of LLMs. In response, we propose \textbf{H}armful \textbf{P}rompt \textbf{La}undering (HaPLa), a novel and broadly applicable jailbreaking technique that requires only black-box access to target models. HaPLa incorporates two primary strategies: 1) \textit{abductive framing}, which instructs LLMs to infer plausible intermediate steps toward harmful activities, rather than directly responding to explicit harmful queries; and 2) \textit{symbolic encoding}, a lightweight and flexible approach designed to obfuscate harmful content, given that current LLMs remain sensitive primarily to explicit harmful keywords. Experimental results show that HaPLa achieves over 95% attack success rate on GPT-series models and 70% across all targets. Further analysis with diverse symbolic encoding rules also reveals a fundamental challenge: it remains difficult to safely tune LLMs without significantly diminishing their helpfulness in responding to benign queries.

Method: A knowledge-based multi-agent framework with two coordinated units: a regulation-aware transaction generator and an ensemble detection pipeline that produces synthetic transactions spanning core laundering phases and advanced typologies.

Result: Generated 1,090,173 synthetic transactions (0.16% laundering-positive) with 75% regulatory alignment based on AUSTRAC rules and composite technical fidelity score of 0.75. Detection ensemble achieved F1 0.90 (precision 0.84, recall 0.97) on internal tests and adapted well to external SynthAML dataset.

Conclusion: AMLNet provides a reproducible framework for regulation-conscious AML experimentation with demonstrated generalizability across different synthetic generation paradigms, and the dataset is publicly released to advance the field.

Abstract: Anti-money laundering (AML) research is constrained by the lack of publicly shareable, regulation-aligned transaction datasets. We present AMLNet, a knowledge-based multi-agent framework with two coordinated units: a regulation-aware transaction generator and an ensemble detection pipeline. The generator produces 1,090,173 synthetic transactions (approximately 0.16% laundering-positive) spanning core laundering phases (placement, layering, integration) and advanced typologies (e.g., structuring, adaptive threshold behavior). Regulatory alignment reaches 75% based on AUSTRAC rule coverage (Section 4.2), while a composite technical fidelity score of 0.75 summarizes temporal, structural, and behavioral realism components (Section 4.4). The detection ensemble achieves F1 0.90 (precision 0.84, recall 0.97) on the internal test partitions of AMLNet and adapts to the external SynthAML dataset, indicating architectural generalizability across different synthetic generation paradigms. We provide multi-dimensional evaluation (regulatory, temporal, network, behavioral) and release the dataset (Version 1.0, https://doi.org/10.5281/zenodo.16736515), to advance reproducible and regulation-conscious AML experimentation.

Method: Incorporates task-related public data into the ICL framework while maintaining differential privacy guarantees, proposing a private in-context learning algorithm.

Result: Significantly improves utility of private ICL with public data assistance and demonstrates robustness against membership inference attacks.

Conclusion: The approach effectively balances privacy protection and model utility in in-context learning, providing empirical privacy protection while maintaining practical usefulness.

Abstract: In-context learning (ICL) in Large Language Models (LLMs) has shown remarkable performance across various tasks without requiring fine-tuning. However, recent studies have highlighted the risk of private data leakage through the prompt in ICL, especially when LLMs are exposed to malicious attacks. While differential privacy (DP) provides strong privacy guarantees, it often significantly reduces the utility of in-context learning (ICL). To address this challenge, we incorporate task-related public data into the ICL framework while maintaining the DP guarantee. Based on this approach, we propose a private in-context learning algorithm that effectively balances privacy protection and model utility. Through experiments, we demonstrate that our approach significantly improves the utility of private ICL with the assistance of public data. Additionally, we show that our method is robust against membership inference attacks, demonstrating empirical privacy protection.

Method: Synergistic integration of Clarion cognitive architecture with LLMs by leveraging Clarion’s fundamental implicit-explicit dichotomy for seamless combination.

Result: Successful combination of LLMs’ computational power with Clarion’s psychological nicety through the implicit-explicit framework.

Conclusion: Incorporating LLMs into cognitive architectures like Clarion provides a promising approach to achieve both computational capability and psychological realism in modeling human cognition.

Abstract: Computational cognitive architectures are broadly scoped models of the human mind that combine different psychological functionalities (as well as often different computational methods for these different functionalities) into one unified framework. They structure them in a psychologically plausible and validated way. However, such models thus far have only limited computational capabilities, mostly limited by the computational tools and techniques that were adopted. More recently, LLMs have proved to be more capable computationally than any other tools. Thus, in order to deal with both real-world complexity and psychological realism at the same time, incorporating LLMs into cognitive architectures naturally becomes an important task. In the present article, a synergistic combination of the Clarion cognitive architecture and LLMs is discussed as a case study. The implicit-explicit dichotomy that is fundamental to Clarion is leveraged for a seamless integration of Clarion and LLMs. As a result, computational power of LLMs is combined with psychological nicety of Clarion.

Method: Proposes a neuro-symbolic multi-agent architecture where individual agent belief states are formally represented as Kripke models, enabling reasoning about possibility and necessity using modal logic. Uses immutable domain-specific knowledge encoded as logical constraints to guide hypothesis generation.

Result: The system successfully diagnoses complex, cascading failures in a high-fidelity simulated particle accelerator environment by combining semantic intuition of language models with rigorous validation of modal logic.

Conclusion: This approach showcases a viable path toward more robust, reliable, and verifiable autonomous agents by integrating formal reasoning with language model capabilities.

Abstract: The development of intelligent agents, particularly those powered by language models (LMs), has shown the critical role in various environments that require intelligent and autonomous decision. Environments are not passive testing grounds and they represent the data required for agents to learn and exhibit very challenging conditions that require adaptive, complex and autonomous capacity to make decisions. While the paradigm of scaling models and datasets has led to remarkable emergent capabilities, we argue that scaling the structure, fidelity, and logical consistency of agent reasoning within these environments is a crucial, yet underexplored, dimension of AI research. This paper introduces a neuro-symbolic multi-agent architecture where the belief states of individual agents are formally represented as Kripke models. This foundational choice enables them to reason about known concepts of \emph{possibility} and \emph{necessity} using the formal language of modal logic. In this work, we use of immutable, domain-specific knowledge to make infere information, which is encoded as logical constraints essential for proper diagnosis. In the proposed model, we show constraints that actively guide the hypothesis generation of LMs, effectively preventing them from reaching physically or logically untenable conclusions. In a high-fidelity simulated particle accelerator environment, our system successfully diagnoses complex, cascading failures by combining the powerful semantic intuition of LMs with the rigorous, verifiable validation of modal logic and a factual world model and showcasing a viable path toward more robust, reliable, and verifiable autonomous agents.

Method: The authors identify key rationale attributes from literature, assess them using automatic metrics, LLM judgments, and human annotations, analyze human preference datasets using SHAP to identify explanatory attributes, and re-evaluate rationales using attribute-specific ELO scores.

Result: The study finds that fine-grained attribute evaluations can better characterize rationale quality and reveal more nuanced model comparisons, showing which specific attributes drive human preferences.

Conclusion: Attribute-based evaluation overcomes limitations of binary comparisons and provides more interpretable and reliable evaluation practices for LLM-generated rationales, guiding future research toward better rationale assessment.

Abstract: Large language models (LLMs) often generate natural language rationales – free-form explanations that help improve performance on complex reasoning tasks and enhance interpretability for human users. However, evaluating these rationales remains challenging. While recent work has relied on binary preference judgments from humans or LLM judges, such evaluations are often opaque and coarse-grained, offering limited insight into what makes one rationale better than another. In this work, we rethink preference evaluation for LLM-generated rationales by asking: (1) What attributes define good rationales? (2) Can human preferences be explained by these attributes? (3) Can attribute-based evaluation overcome the limitations of binary comparisons? We identify a set of key rationale attributes from prior literature and assess them using automatic metrics, LLM judgments, and human annotations. We then analyze two standard human preference datasets MT Bench and Chatbot Arena using SHAP to identify which attributes best explain human preference outcomes. Finally, we re-evaluate model-generated rationales using attribute-specific ELO scores, revealing more nuanced model comparisons and insights. Our findings suggest that fine-grained attribute evaluations can better characterize rationale quality and guide future research toward more interpretable and reliable evaluation practices.

Method: Proposes Free-MAD framework with score-based decision mechanism that evaluates entire debate trajectories, anti-conformity mechanism to mitigate majority influence, and single-round debate structure to reduce token costs.

Result: Experiments on eight benchmark datasets show significant reasoning performance improvements, reduced token costs through single-round debates, and improved robustness in real-world attack scenarios compared to existing MAD approaches.

Conclusion: Free-MAD successfully addresses limitations of consensus-based multi-agent debate by providing more accurate and fair outcomes through trajectory evaluation and anti-conformity mechanisms while being more efficient and robust.

Abstract: Multi-agent debate (MAD) is an emerging approach to improving the reasoning capabilities of large language models (LLMs). Existing MAD methods rely on multiple rounds of interaction among agents to reach consensus, and the final output is selected by majority voting in the last round. However, this consensus-based design faces several limitations. First, multiple rounds of communication increases token overhead and limits scalability. Second, due to the inherent conformity of LLMs, agents that initially produce correct responses may be influenced by incorrect ones during the debate process, causing error propagation. Third, majority voting introduces randomness and unfairness in the decision-making phase, and can degrade the reasoning performance. To address these issues, we propose \textsc{Free-MAD}, a novel MAD framework that eliminates the need for consensus among agents. \textsc{Free-MAD} introduces a novel score-based decision mechanism that evaluates the entire debate trajectory rather than relying on the last round only. This mechanism tracks how each agent’s reasoning evolves, enabling more accurate and fair outcomes. In addition, \textsc{Free-MAD} reconstructs the debate phase by introducing anti-conformity, a mechanism that enables agents to mitigate excessive influence from the majority. Experiments on eight benchmark datasets demonstrate that \textsc{Free-MAD} significantly improves reasoning performance while requiring only a single-round debate and thus reducing token costs. We also show that compared to existing MAD approaches, \textsc{Free-MAD} exhibits improved robustness in real-world attack scenarios.

Method: Bidirectional Cognitive Alignment (BiCA) uses learnable protocols, representation mapping, and KL-budget constraints to enable controlled co-evolution between humans and AI.

Result: 85.5% success rate in collaborative navigation (vs 70.3% baseline), 230% better mutual adaptation, 332% better protocol convergence, 84% improvement over handcrafted protocols, and 23% better out-of-distribution robustness.

Conclusion: Optimal collaboration exists at the intersection of human and AI capabilities, validating the shift from single-directional to co-alignment paradigms with demonstrated safety and performance benefits.

Abstract: Current AI alignment through RLHF follows a single directional paradigm that AI conforms to human preferences while treating human cognition as fixed. We propose a shift to co-alignment through Bidirectional Cognitive Alignment (BiCA), where humans and AI mutually adapt. BiCA uses learnable protocols, representation mapping, and KL-budget constraints for controlled co-evolution. In collaborative navigation, BiCA achieved 85.5% success versus 70.3% baseline, with 230% better mutual adaptation and 332% better protocol convergence. Emergent protocols outperformed handcrafted ones by 84%, while bidirectional adaptation unexpectedly improved safety (+23% out-of-distribution robustness). The 46% synergy improvement demonstrates optimal collaboration exists at the intersection, not union, of human and AI capabilities, validating the shift from single-directional to co-alignment paradigms.

Method: Uses deterministic replicability property of autoregressive models in homogeneous environments. Multiple validators probabilistically audit small random segments of output rather than full regeneration.

Result: Simulations show targeted verification can be over 12x faster than full regeneration, with tunable parameters to adjust detection probability.

Conclusion: Provides a tractable mechanism for auditable LLM systems, serving as a foundational layer for responsible AI and cornerstone for future heterogeneous multi-agent systems research.

Abstract: The landscape of Large Language Models (LLMs) shifts rapidly towards dynamic, multi-agent systems. This introduces a fundamental challenge in establishing computational trust, specifically how one agent can verify that another’s output was genuinely produced by a claimed LLM, and not falsified or generated by a cheaper or inferior model. To address this challenge, this paper proposes a verification framework that achieves tractable asymmetric effort, where the cost to verify a computation is substantially lower than the cost to perform it. Our approach is built upon the principle of deterministic replicability, a property inherent to autoregressive models that strictly necessitates a computationally homogeneous environment where all agents operate on identical hardware and software stacks. Within this defined context, our framework enables multiple validators to probabilistically audit small, random segments of an LLM’s output and it distributes the verification workload effectively. The simulations demonstrated that targeted verification can be over 12 times faster than full regeneration, with tunable parameters to adjust the detection probability. By establishing a tractable mechanism for auditable LLM systems, our work offers a foundational layer for responsible AI and serves as a cornerstone for future research into the more complex, heterogeneous multi-agent systems.

Method: Uses medically-aligned multi-step generation architecture with hierarchical medical knowledge injection, dynamic updating mechanism for consistency, and real-time clinical plausibility verification.

Result: Achieves good performance in accuracy, diversity, and consistency. Models trained with generated virtual patients show significant improvements on MedQA dataset.

Conclusion: Patient-Zero successfully generates diverse, medically coherent patient records without real data and enhances model performance through realistic synthetic patient interactions.

Abstract: Synthetic data generation using large language models (LLMs) has emerged as a promising solution across various domains, particularly in medical field, to mitigate data collection challenges. However, existing studies mainly utilize LLMs to rewrite and complete existing medical records, where the limitations in data privacy, accuracy, and diversity sill exist, and additionally lack the ability to interact like real patients. To address these issues, we propose a realistic patient generation framework, Patient-Zero, which requires no real medical records. Patient-Zero first introduces a medically-aligned multi-step generation architecture, which builds comprehensive patient records through hierarchical medical knowledge injection without real medical records. Then, to optimize the virtual patient’s interaction abilities with humans, Patient-Zero designs a dynamic updating mechanism to improve the consistency and conversational performance. Our framework enables the generation of contextually diverse patient records while maintaining strict medical coherence, supported by adaptive dialogue strategies and real-time clinical plausibility verification. Experimental results demonstrate that our model achieves good performance in accuracy, diversity, and consistency. After training with our generated virtual patients, existing models show significant improvements on the MedQA dataset.

Method: Proposes Difficulty-Aware Agentic Orchestration (DAAO) with three modules: VAE for difficulty estimation, modular operator allocator, and cost-performance aware LLM router to dynamically adapt workflow depth and LLM assignment.

Result: Outperforms prior multi-agent systems in both accuracy and inference efficiency across six benchmarks.

Conclusion: DAAO enables fine-grained, query-specific reasoning strategies by leveraging heterogeneous LLMs and dynamically tailoring workflows based on query difficulty.

Abstract: Large Language Model (LLM)-based agentic systems have shown strong capabilities across various tasks. However, existing multi-agent frameworks often rely on static or task-level workflows, which either over-process simple queries or underperform on complex ones, while also neglecting the efficiency-performance trade-offs across heterogeneous LLMs. To address these limitations, we propose Difficulty-Aware Agentic Orchestration (DAAO), a dynamic framework that adapts workflow depth, operator selection, and LLM assignment based on the difficulty of each input query. DAAO comprises three interdependent modules: a variational autoencoder (VAE) for difficulty estimation, a modular operator allocator, and a cost- and performance-aware LLM router. By leveraging heterogeneous LLMs and dynamically tailoring workflows, DAAO enables fine-grained, query-specific reasoning strategies. DAAO outperforms prior multi-agent systems in both accuracy and inference efficiency across six benchmarks. We will release our code and implementation details upon publication.

Method: Developed an AMDM algorithm that normalizes heterogeneous metrics, applies per-axis exponentially weighted moving-average thresholds, and performs joint anomaly detection via Mahalanobis distance.

Result: AMDM reduced anomaly detection latency from 12.3s to 5.6s on simulated goal drift and decreased false-positive rates from 4.5% to 0.9% compared to static thresholds.

Conclusion: The proposed AMDM framework provides effective multi-dimensional monitoring for agentic AI systems, addressing previous gaps in algorithmic instantiation and empirical validation while offering improved performance metrics.

Abstract: Agentic artificial intelligence (AI) – multi-agent systems that combine large language models with external tools and autonomous planning – are rapidly transitioning from research laboratories into high-stakes domains. Our earlier “Basic” paper introduced a five-axis framework and proposed preliminary metrics such as goal drift and harm reduction but did not provide an algorithmic instantiation or empirical evidence. This “Advanced” sequel fills that gap. First, we revisit recent benchmarks and industrial deployments to show that technical metrics still dominate evaluations: a systematic review of 84 papers from 2023–2025 found that 83% report capability metrics while only 30% consider human-centred or economic axes [2]. Second, we formalise an Adaptive Multi-Dimensional Monitoring (AMDM) algorithm that normalises heterogeneous metrics, applies per-axis exponentially weighted moving-average thresholds and performs joint anomaly detection via the Mahalanobis distance [7]. Third, we conduct simulations and real-world experiments. AMDM cuts anomaly-detection latency from 12.3 s to 5.6 s on simulated goal drift and reduces false-positive rates from 4.5% to 0.9% compared with static thresholds. We present a comparison table and ROC/PR curves, and we reanalyse case studies to surface missing metrics. Code, data and a reproducibility checklist accompany this paper to facilitate replication. The code supporting this work is available at https://github.com/Manishms18/Adaptive-Multi-Dimensional-Monitoring.

Method: Proposes AlignKT with frontend-to-backend architecture that aligns preliminary knowledge states with ideal states based on pedagogical theories. Uses five encoders and contrastive learning module to enhance alignment robustness.

Result: Outperforms seven KT baselines on three real-world datasets, achieving state-of-the-art results on two datasets and competitive performance on the third.

Conclusion: AlignKT successfully addresses the limitation of traditional KT models by explicitly modeling stable knowledge states with pedagogical alignment, improving both performance and interpretability for intelligent tutoring systems.

Abstract: Knowledge Tracing (KT) serves as a fundamental component of Intelligent Tutoring Systems (ITS), enabling these systems to monitor and understand learners’ progress by modeling their knowledge state. However, many existing KT models primarily focus on fitting the sequences of learners’ interactions, and often overlook the knowledge state itself. This limitation leads to reduced interpretability and insufficient instructional support from the ITS. To address this challenge, we propose AlignKT, which employs a frontend-to-backend architecture to explicitly model a stable knowledge state. In this approach, the preliminary knowledge state is aligned with an additional criterion. Specifically, we define an ideal knowledge state based on pedagogical theories as the alignment criterion, providing a foundation for interpretability. We utilize five encoders to implement this set-up, and incorporate a contrastive learning module to enhance the robustness of the alignment process. Through extensive experiments, AlignKT demonstrates superior performance, outperforming seven KT baselines on three real-world datasets. It achieves state-of-the-art results on two of these datasets and exhibits competitive performance on the third. The code of this work is available at https://github.com/SCNU203/AlignKT.

Method: Brought together 16 scholars from multiple disciplines to provide: 1) Overview of AIGC training techniques and detection methods, 2) Analysis of societal impacts across diverse domains with review of existing methods, 3) Discussion of key technical challenges and research propositions.

Result: Provides a comprehensive cross-domain perspective on AIGC, offering insights into current research trends, ongoing challenges, and future directions across digital marketing, education, public health, and other domains.

Conclusion: This vision paper successfully bridges the research gap by offering a multidisciplinary examination of AIGC, highlighting both its transformative potential and the technical/societal challenges that need to be addressed in future work.

Abstract: Artificial Intelligence Generated Content (AIGC) has rapidly emerged with the capability to generate different forms of content, including text, images, videos, and other modalities, which can achieve a quality similar to content created by humans. As a result, AIGC is now widely applied across various domains such as digital marketing, education, and public health, and has shown promising results by enhancing content creation efficiency and improving information delivery. However, there are few studies that explore the latest progress and emerging challenges of AIGC across different domains. To bridge this gap, this paper brings together 16 scholars from multiple disciplines to provide a cross-domain perspective on the trends and challenges of AIGC. Specifically, the contributions of this paper are threefold: (1) It first provides a broader overview of AIGC, spanning the training techniques of Generative AI, detection methods, and both the spread and use of AI-generated content across digital platforms. (2) It then introduces the societal impacts of AIGC across diverse domains, along with a review of existing methods employed in these contexts. (3) Finally, it discusses the key technical challenges and presents research propositions to guide future work. Through these contributions, this vision paper seeks to offer readers a cross-domain perspective on AIGC, providing insights into its current research trends, ongoing challenges, and future directions.

Method: VideoAgent parses source papers into fine-grained asset libraries and orchestrates narrative flows that synthesize both static slides and dynamic animations. It uses a multi-agent framework guided by user requirements through conversational interfaces.

Result: Extensive experiments show VideoAgent significantly outperforms existing commercial scientific video generation services and approaches human-level quality in scientific communication.

Conclusion: The proposed VideoAgent framework successfully addresses the challenges of automated scientific video generation, enabling effective knowledge dissemination through personalized dynamic content orchestration and multimodal synchronization.

Abstract: Automating the generation of scientific videos is a crucial yet challenging task for effective knowledge dissemination. However, existing works on document automation primarily focus on static media such as posters and slides, lacking mechanisms for personalized dynamic orchestration and multimodal content synchronization. To address these challenges, we introduce VideoAgent, a novel multi-agent framework that synthesizes personalized scientific videos through a conversational interface. VideoAgent parses a source paper into a fine-grained asset library and, guided by user requirements, orchestrates a narrative flow that synthesizes both static slides and dynamic animations to explain complex concepts. To enable rigorous evaluation, we also propose SciVidEval, the first comprehensive suite for this task, which combines automated metrics for multimodal content quality and synchronization with a Video-Quiz-based human evaluation to measure knowledge transfer. Extensive experiments demonstrate that our method significantly outperforms existing commercial scientific video generation services and approaches human-level quality in scientific communication.

Method: P2P system using structured prompt engineering, entropy-based sampling, and regression-based selection to steer LLM agents toward representative behavioral patterns without demographic conditioning.

Result: Aligned agent populations can reproduce aggregate response patterns with high fidelity and exhibit substantial response diversity, even without demographic information.

Conclusion: The framework offers an effective approach for improving data efficiency in social science research and serves as a testbed for studying pluralistic alignment operationalization.

Abstract: Large language models (LLMs) have demonstrated promise in emulating human-like responses across a wide range of tasks. In this paper, we propose a novel alignment framework that treats LLMs as agent proxies for human survey respondents, affording a cost-effective and steerable solution to two pressing challenges in the social sciences: the rising cost of survey deployment and the growing demographic imbalance in survey response data. Drawing inspiration from the theory of revealed preference, we formulate alignment as a two-stage problem: constructing diverse agent personas called endowments that simulate plausible respondent profiles, and selecting a representative subset to approximate a ground-truth population based on observed data. To implement the paradigm, we introduce P2P, a system that steers LLM agents toward representative behavioral patterns using structured prompt engineering, entropy-based sampling, and regression-based selection. Unlike personalization-heavy approaches, our alignment approach is demographic-agnostic and relies only on aggregate survey results, offering better generalizability and parsimony. Beyond improving data efficiency in social science research, our framework offers a testbed for studying the operationalization of pluralistic alignment. We demonstrate the efficacy of our approach on real-world opinion survey datasets, showing that our aligned agent populations can reproduce aggregate response patterns with high fidelity and exhibit substantial response diversity, even without demographic conditioning.

Method: Leverage large language models to extract relational metapaths (multi-hop semantic chains) from scientific abstracts, construct a Toxicity Trajectory Graph, and incorporate dynamic evidence reconciliation to resolve semantic conflicts from evolving research.

Result: The approach demonstrates strong performance in extracting reliable, high-utility relational knowledge from noisy scientific text and provides a scalable solution for mining complex cause-effect structures.

Conclusion: The proposed framework offers an effective and scalable method for systematically understanding and tracing pollutant propagation pathways and their health impacts through structured knowledge extraction from scientific literature.

Abstract: The widespread use of plastics and their persistence in the environment have led to the accumulation of micro- and nano-plastics across air, water, and soil, posing serious health risks including respiratory, gastrointestinal, and neurological disorders. We propose a novel framework that leverages large language models to extract relational metapaths, multi-hop semantic chains linking pollutant sources to health impacts, from scientific abstracts. Our system identifies and connects entities across diverse contexts to construct structured relational metapaths, which are aggregated into a Toxicity Trajectory Graph that traces pollutant propagation through exposure routes and biological systems. Moreover, to ensure consistency and reliability, we incorporate a dynamic evidence reconciliation module that resolves semantic conflicts arising from evolving or contradictory research findings. Our approach demonstrates strong performance in extracting reliable, high-utility relational knowledge from noisy scientific text and offers a scalable solution for mining complex cause-effect structures in domain-specific corpora.

Method: Uses LTC networks with input-dependent time constants to implement iterative workflow: train observer on candidate inputs, quantify causal impact through perturbation analysis, remove negligible inputs, and retrain until performance degrades.

Result: The approach consistently identifies minimal sensor sets that align with underlying physics while improving prediction accuracy across three testbeds: spring-mass-damper system, nonlinear stirred-tank reactor, and complex predator-prey model.

Conclusion: The causality-guided framework enhances both computational efficiency and interpretability by grounding sensor selection in dynamic causal relationships rather than static correlations, benefiting applications across multiple engineering and monitoring domains.

Abstract: This paper introduces a novel framework for optimizing observer-based soft sensors through dynamic causality analysis. Traditional approaches to sensor selection often rely on linearized observability indices or statistical correlations that fail to capture the temporal evolution of complex systems. We address this gap by leveraging liquid-time constant (LTC) networks, continuous-time neural architectures with input-dependent time constants, to systematically identify and prune sensor inputs with minimal causal influence on state estimation. Our methodology implements an iterative workflow: training an LTC observer on candidate inputs, quantifying each input’s causal impact through controlled perturbation analysis, removing inputs with negligible effect, and retraining until performance degradation occurs. We demonstrate this approach on three mechanistic testbeds representing distinct physical domains: a harmonically forced spring-mass-damper system, a nonlinear continuous stirred-tank reactor, and a predator-prey model following the structure of the Lotka-Volterra model, but with seasonal forcing and added complexity. Results show that our causality-guided pruning consistently identifies minimal sensor sets that align with underlying physics while improving prediction accuracy. The framework automatically distinguishes essential physical measurements from noise and determines when derived interaction terms provide complementary versus redundant information. Beyond computational efficiency, this approach enhances interpretability by grounding sensor selection decisions in dynamic causal relationships rather than static correlations, offering significant benefits for soft sensing applications across process engineering, ecological monitoring, and agricultural domains.

Method: MAPGD integrates multi-agent collaboration with gradient-based optimization, featuring specialized agents for task clarity, example selection, format design, and stylistic refinement; semantic gradient coordination; bandit-based candidate selection; and theoretical convergence guarantees.

Result: Experiments on classification, generation, and reasoning tasks show MAPGD outperforms single-agent and random baselines in both accuracy and efficiency. Ablations confirm benefits of gradient fusion, agent specialization, and conflict resolution.

Conclusion: MAPGD provides a unified, gradient-inspired multi-agent approach to robust and interpretable prompt optimization, addressing key limitations of existing methods.

Abstract: Prompt engineering is crucial for leveraging large language models (LLMs), but existing methods often rely on a single optimization trajectory, limiting adaptability and efficiency while suffering from narrow perspectives, gradient conflicts, and high computational cost. We propose MAPGD (Multi-Agent Prompt Gradient Descent), a framework integrating multi-agent collaboration with gradient-based optimization. MAPGD features specialized agents for task clarity, example selection, format design, and stylistic refinement; semantic gradient coordination to resolve conflicts; bandit-based candidate selection for efficient exploration-exploitation; and theoretical convergence guarantees. Experiments on classification, generation, and reasoning tasks show MAPGD outperforms single-agent and random baselines in accuracy and efficiency. Ablations confirm the benefits of gradient fusion, agent specialization, and conflict resolution, providing a unified, gradient-inspired multi-agent approach to robust and interpretable prompt optimization.

Method: Proposes a framework that integrates Role-Based Access Control (RBAC) into AI agents to provide security guardrails, with focus on on-premises implementations.

Result: The framework aims to mitigate security risks and support effective, scalable deployment of AI agents in industrial environments.

Conclusion: RBAC integration provides a robust security solution for AI agents, addressing critical vulnerabilities and enabling safer industrial applications.

Abstract: The emergence of Large Language Models (LLMs) has significantly advanced solutions across various domains, from political science to software development. However, these models are constrained by their training data, which is static and limited to information available up to a specific date. Additionally, their generalized nature often necessitates fine-tuning – whether for classification or instructional purposes – to effectively perform specific downstream tasks. AI agents, leveraging LLMs as their core, mitigate some of these limitations by accessing external tools and real-time data, enabling applications such as live weather reporting and data analysis. In industrial settings, AI agents are transforming operations by enhancing decision-making, predictive maintenance, and process optimization. For example, in manufacturing, AI agents enable near-autonomous systems that boost productivity and support real-time decision-making. Despite these advancements, AI agents remain vulnerable to security threats, including prompt injection attacks, which pose significant risks to their integrity and reliability. To address these challenges, this paper proposes a framework for integrating Role-Based Access Control (RBAC) into AI agents, providing a robust security guardrail. This framework aims to support the effective and scalable deployment of AI agents, with a focus on on-premises implementations.

Method: Developed an Adaptive Mixture of Experts model where each expert specializes in specific modalities or spatio-temporal patterns, incorporating a dynamic router that learns to assign inputs to the most relevant experts. Also created an interactive web-based visualization tool for exploring weather patterns.

Result: The Adaptive MoE model significantly outperformed all baseline methods when evaluated on a curated multimodal climate dataset capturing real-world conditions during Hurricane Ian in 2022, enhancing both predictive accuracy and interpretability.

Conclusion: The proposed modular approach effectively addresses the challenges of integrating heterogeneous multi-source climate data for precipitation forecasting, providing both improved accuracy and practical decision-making support through the visualization tool.

Abstract: Accurate precipitation forecasting is indispensable in agriculture, disaster management, and sustainable strategies. However, predicting rainfall has been challenging due to the complexity of climate systems and the heterogeneous nature of multi-source observational data, including radar, satellite imagery, and surface-level measurements. The multi-source data vary in spatial and temporal resolution, and they carry domain-specific features, making it challenging for effective integration in conventional deep learning models. Previous research has explored various machine learning techniques for weather prediction; however, most struggle with the integration of data with heterogeneous modalities. To address these limitations, we propose an Adaptive Mixture of Experts (MoE) model tailored for precipitation rate prediction. Each expert within the model specializes in a specific modality or spatio-temporal pattern. We also incorporated a dynamic router that learns to assign inputs to the most relevant experts. Our results show that this modular design enhances predictive accuracy and interpretability. In addition to the modeling framework, we introduced an interactive web-based visualization tool that enables users to intuitively explore historical weather patterns over time and space. The tool was designed to support decision-making for stakeholders in climate-sensitive sectors. We evaluated our approach using a curated multimodal climate dataset capturing real-world conditions during Hurricane Ian in 2022. The benchmark results show that the Adaptive MoE significantly outperformed all the baselines.

Method: Evaluated 5 representative VLA models using LIBERO benchmark, measuring accuracy with system metrics (latency, throughput, memory) under varying edge power constraints and datacenter GPU configurations.

Result: Identified distinct scaling trends: architectural choices strongly influence throughput/memory; edge devices show non-linear degradation but can match older datacenter GPUs; high-throughput variants achievable without significant accuracy loss.

Conclusion: Provides actionable insights for VLA selection/optimization across deployment constraints, challenging assumptions about datacenter hardware superiority for robotic inference.

Abstract: Vision-Language-Action (VLA) models have emerged as powerful generalist policies for robotic control, yet their performance scaling across model architectures and hardware platforms, as well as their associated power budgets, remain poorly understood. This work presents an evaluation of five representative VLA models – spanning state-of-the-art baselines and two newly proposed architectures – targeting edge and datacenter GPU platforms. Using the LIBERO benchmark, we measure accuracy alongside system-level metrics, including latency, throughput, and peak memory usage, under varying edge power constraints and high-performance datacenter GPU configurations. Our results identify distinct scaling trends: (1) architectural choices, such as action tokenization and model backbone size, strongly influence throughput and memory footprint; (2) power-constrained edge devices exhibit non-linear performance degradation, with some configurations matching or exceeding older datacenter GPUs; and (3) high-throughput variants can be achieved without significant accuracy loss. These findings provide actionable insights when selecting and optimizing VLAs across a range of deployment constraints. Our work challenges current assumptions about the superiority of datacenter hardware for robotic inference.

Method: Developed MedicalOS as a domain-specific abstraction layer that translates human instructions into pre-defined healthcare commands (patient inquiry, history retrieval, exam management, etc.) wrapped as tools using Python, APIs, MCP, and Linux.

Result: Validated on 214 patient cases across 22 specialties, demonstrating high diagnostic accuracy and confidence, clinically sound examination requests, and consistent generation of structured reports and medication recommendations.

Conclusion: MedicalOS provides a trustworthy and scalable foundation for advancing workflow automation in clinical practice by enabling natural language interaction with healthcare systems while maintaining clinical safety standards.

Abstract: Decades’ advances in digital health technologies, such as electronic health records, have largely streamlined routine clinical processes. Yet, most these systems are still hard to learn and use: Clinicians often face the burden of managing multiple tools, repeating manual actions for each patient, navigating complicated UI trees to locate functions, and spending significant time on administration instead of caring for patients. The recent rise of large language model (LLM) based agents demonstrates exceptional capability in coding and computer operation, revealing the potential for humans to interact with operating systems and software not by direct manipulation, but by instructing agents through natural language. This shift highlights the need for an abstraction layer, an agent-computer interface, that translates human language into machine-executable commands. In digital healthcare, however, requires a more domain-specific abstractions that strictly follow trusted clinical guidelines and procedural standards to ensure safety, transparency, and compliance. To address this need, we present \textbf{MedicalOS}, a unified agent-based operational system designed as such a domain-specific abstract layer for healthcare. It translates human instructions into pre-defined digital healthcare commands, such as patient inquiry, history retrieval, exam management, report generation, referrals, treatment planning, that we wrapped as off-the-shelf tools using machine languages (e.g., Python, APIs, MCP, Linux). We empirically validate MedicalOS on 214 patient cases across 22 specialties, demonstrating high diagnostic accuracy and confidence, clinically sound examination requests, and consistent generation of structured reports and medication recommendations. These results highlight MedicalOS as a trustworthy and scalable foundation for advancing workflow automation in clinical practice.

Method: Generated synthetic eye movement data using CTGAN, CopulaGAN, and Gretel AI synthetic data generators on an in-person user study dataset. Augmented real data with synthetic samples and tested various machine learning algorithms including Random Forest and Inception Time.

Result: Significant improvement in task decoding accuracy from 28.1% (Random Forest) to 82% (Inception Time) when augmenting 320 real samples with 5x more synthetic data. The framework outperformed all previous studies on this dataset.

Conclusion: Augmenting real eye movement data with synthetically generated data substantially improves task decoding accuracy, validating Yarbus’ hypothesis and demonstrating the effectiveness of synthetic data generation for eye-tracking research.

Abstract: Machine learning has been extensively used in various applications related to eye-tracking research. Understanding eye movement is one of the most significant subsets of eye-tracking research that reveals the scanning pattern of an individual. Researchers have thoroughly analyzed eye movement data to understand various eye-tracking applications, such as attention mechanisms, navigational behavior, task understanding, etc. The outcome of traditional machine learning algorithms used for decoding tasks based on eye movement data has received a mixed reaction to Yarbus’ claim that it is possible to decode the observer’s task from their eye movements. In this paper, to support the hypothesis by Yarbus, we are decoding tasks categories while generating synthetic data samples using well-known Synthetic Data Generators CTGAN and its variations such as CopulaGAN and Gretel AI Synthetic Data generators on available data from an in-person user study. Our results show that augmenting more eye movement data combined with additional synthetically generated improves classification accuracy even with traditional machine learning algorithms. We see a significant improvement in task decoding accuracy from 28.1% using Random Forest to 82% using Inception Time when five times more data is added in addition to the 320 real eye movement dataset sample. Our proposed framework outperforms all the available studies on this dataset because of the use of additional synthetic datasets. We validated our claim with various algorithms and combinations of real and synthetic data to show how decoding accuracy increases with the increase in the augmentation of generated data to real data.

Method: Proposed MCFR framework that combines LLMs with model checking to translate natural language into formal specifications and verify them over transition models. Created EduMC-QA benchmark for evaluation.

Result: MCFR improves reasoning faithfulness and interpretability compared to state-of-the-art LLMs like ChatGPT, DeepSeek, and Claude, offering verifiable QA for high-stakes closed-domain applications.

Conclusion: MCFR provides a viable path toward verifiable question answering in critical domains by integrating neural and symbolic approaches through model checking, addressing limitations of current LLM-based reasoning systems.

Abstract: Reasoning is essential for closed-domain QA systems in which procedural correctness and policy compliance are critical. While large language models (LLMs) have shown strong performance on many reasoning tasks, recent work reveals that their reasoning traces are often unfaithful - serving more as plausible justifications than as causally grounded derivations. Efforts to combine LLMs with symbolic engines (e.g., Prover9, Z3) have improved reliability but remain limited to static forms of logic, struggling with dynamic, state-based reasoning such as multi-step progressions and conditional transitions. In this paper, we propose MCFR (Model Checking for Formal Reasoning), a neuro-symbolic framework that integrates LLMs with model checking to support property verification. MCFR translates natural language into formal specifications and verifies them over transition models. To support evaluation, we introduce EduMC-QA, a benchmark dataset grounded in real academic procedures. Our results show that MCFR improves reasoning faithfulness and interpretability, offering a viable path toward verifiable QA in high-stakes closed-domain applications. In addition to evaluating MCFR, we compare its performance with state-of-the-art LLMs such as ChatGPT, DeepSeek, and Claude to contextualize its effectiveness.

Method: The survey categorizes approaches by reasoning topology (direct, linear chain, branch-structured) and main objectives (traditional analysis, explanation, causal inference, generation). It uses a compact tag system to capture decomposition, verification, tool use, and other aspects across domains.

Result: The paper provides a comprehensive organization of time series reasoning literature, identifies what each topology enables and where it breaks down, and offers curated datasets, benchmarks, and evaluation practices that maintain temporal alignment and evidence visibility.

Conclusion: Time series reasoning represents a shift from narrow accuracy to reliability at scale, requiring reasoning structures that balance grounding capacity against computational cost. Future progress depends on utility-focused benchmarks and closed-loop testbeds that address streaming, long-horizon settings with traceable evidence.

Abstract: Time series reasoning treats time as a first-class axis and incorporates intermediate evidence directly into the answer. This survey defines the problem and organizes the literature by reasoning topology with three families: direct reasoning in one step, linear chain reasoning with explicit intermediates, and branch-structured reasoning that explores, revises, and aggregates. The topology is crossed with the main objectives of the field, including traditional time series analysis, explanation and understanding, causal inference and decision making, and time series generation, while a compact tag set spans these axes and captures decomposition and verification, ensembling, tool use, knowledge access, multimodality, agent loops, and LLM alignment regimes. Methods and systems are reviewed across domains, showing what each topology enables and where it breaks down in faithfulness or robustness, along with curated datasets, benchmarks, and resources that support study and deployment (https://github.com/blacksnail789521/Time-Series-Reasoning-Survey). Evaluation practices that keep evidence visible and temporally aligned are highlighted, and guidance is distilled on matching topology to uncertainty, grounding with observable artifacts, planning for shift and streaming, and treating cost and latency as design budgets. We emphasize that reasoning structures must balance capacity for grounding and self-correction against computational cost and reproducibility, while future progress will likely depend on benchmarks that tie reasoning quality to utility and on closed-loop testbeds that trade off cost and risk under shift-aware, streaming, and long-horizon settings. Taken together, these directions mark a shift from narrow accuracy toward reliability at scale, enabling systems that not only analyze but also understand, explain, and act on dynamic worlds with traceable evidence and credible outcomes.

Method: Constructed database of 3,000 X-rays with diagnostic texts; evaluated five MLLMs using a ‘Divide and Conquer’ framework with visual QA, domain knowledge assessment, and patient counseling tasks; enhanced models with spinal keypoint prompting and RAG using compiled AIS knowledge base.

Result: MLLMs showed poor performance in interpreting complex spinal radiographs and comprehending AIS care knowledge. Spinal keypoint prompting had varying effectiveness across architectures, while RAG substantially improved knowledge assessment performance. Best accuracy was only 0.55 for spinal deformity location detection and 0.13 for direction detection.

Conclusion: Current MLLMs are far from capable of realizing personalized assistants in AIS care, with the greatest challenge being accurate detection of spinal deformity locations and directions.

Abstract: This study presents the first comprehensive evaluation of Multimodal Large Language Models (MLLMs) for Adolescent Idiopathic Scoliosis (AIS) self-management. We constructed a database of approximately 3,000 anteroposterior X-rays with diagnostic texts and evaluated five MLLMs through a `Divide and Conquer’ framework consisting of a visual question-answering task, a domain knowledge assessment task, and a patient education counseling assessment task. Our investigation revealed limitations of MLLMs’ ability in interpreting complex spinal radiographs and comprehending AIS care knowledge. To address these, we pioneered enhancing MLLMs with spinal keypoint prompting and compiled an AIS knowledge base for retrieval augmented generation (RAG), respectively. Results showed varying effectiveness of visual prompting across different architectures, while RAG substantially improved models’ performances on the knowledge assessment task. Our findings indicate current MLLMs are far from capable in realizing personalized assistant in AIS care. The greatest challenge lies in their abilities to obtain accurate detections of spinal deformity locations (best accuracy: 0.55) and directions (best accuracy: 0.13).

Method: Constructs local, multi-scale graphs centered on each agent, uses aggregation-decomposition message-passing scheme and type-specific feature networks to model both pairwise dependencies and complex group-wise interactions.

Result: Achieves state-of-the-art performance on Waymo Open Motion Dataset, setting new benchmarks for key metrics including Soft mAP and minADE.

Conclusion: A locality-grounded architecture that explicitly models multi-scale and heterogeneous interactions is highly effective for advancing motion forecasting in autonomous driving.

Abstract: Multi-agent trajectory prediction in autonomous driving requires a comprehensive understanding of complex social dynamics. Existing methods, however, often struggle to capture the full richness of these dynamics, particularly the co-existence of multi-scale interactions and the diverse behaviors of heterogeneous agents. To address these challenges, this paper introduces HeLoFusion, an efficient and scalable encoder for modeling heterogeneous and multi-scale agent interactions. Instead of relying on global context, HeLoFusion constructs local, multi-scale graphs centered on each agent, allowing it to effectively model both direct pairwise dependencies and complex group-wise interactions (\textit{e.g.}, platooning vehicles or pedestrian crowds). Furthermore, HeLoFusion tackles the critical challenge of agent heterogeneity through an aggregation-decomposition message-passing scheme and type-specific feature networks, enabling it to learn nuanced, type-dependent interaction patterns. This locality-focused approach enables a principled representation of multi-level social context, yielding powerful and expressive agent embeddings. On the challenging Waymo Open Motion Dataset, HeLoFusion achieves state-of-the-art performance, setting new benchmarks for key metrics including Soft mAP and minADE. Our work demonstrates that a locality-grounded architecture, which explicitly models multi-scale and heterogeneous interactions, is a highly effective strategy for advancing motion forecasting.

Method: Integrates Supervised Contrastive Learning (SupCon) loss with continuous output spaces, allowing SupCon to operate without constraints on action types in the environment.

Result: Experiments on 3D games (Astro Bot, Returnal) and 2D Atari games show improved representation quality, faster learning convergence, and better generalization compared to baseline supervised action prediction models.

Conclusion: SupCon integration with continuous action spaces effectively enhances imitation learning by producing better state representations that capture action-relevant information, leading to improved performance and generalization in game environments.

Abstract: This paper introduces a novel application of Supervised Contrastive Learning (SupCon) to Imitation Learning (IL), with a focus on learning more effective state representations for agents in video game environments. The goal is to obtain latent representations of the observations that capture better the action-relevant factors, thereby modeling better the cause-effect relationship from the observations that are mapped to the actions performed by the demonstrator, for example, the player jumps whenever an obstacle appears ahead. We propose an approach to integrate the SupCon loss with continuous output spaces, enabling SupCon to operate without constraints regarding the type of actions of the environment. Experiments on the 3D games Astro Bot and Returnal, and multiple 2D Atari games show improved representation quality, faster learning convergence, and better generalization compared to baseline models trained only with supervised action prediction loss functions.

Method: Three asynchronous processes: retrieval (user identification via face/voice and context gathering), omnimodal dialog (personalized audio responses), and memory management (dialog boundary detection and memory updates from omnimodal streams).

Result: Retrieval and memory management modules achieve >95% accuracy. Integrated system achieves >87% fact-consistency scores in real-time personalized dialogs.

Conclusion: EgoMem establishes a strong baseline for lifelong memory agents in real-time omnimodal scenarios, demonstrating high accuracy and effectiveness for personalized interactions.

Abstract: We introduce EgoMem, the first lifelong memory agent tailored for full-duplex models that process real-time omnimodal streams. EgoMem enables real-time models to recognize multiple users directly from raw audiovisual streams, to provide personalized response, and to maintain long-term knowledge of users’ facts, preferences, and social relationships extracted from audiovisual history. EgoMem operates with three asynchronous processes: (i) a retrieval process that dynamically identifies user via face and voice, and gathers relevant context from a long-term memory; (ii) an omnimodal dialog process that generates personalized audio responses based on the retrieved context; and (iii) a memory management process that automatically detects dialog boundaries from omnimodal streams, and extracts necessary information to update the long-term memory. Unlike existing memory agents for LLMs, EgoMem relies entirely on raw audiovisual streams, making it especially suitable for lifelong, real-time, and embodied scenarios. Experimental results demonstrate that EgoMem’s retrieval and memory management modules achieve over 95% accuracy on the test set. When integrated with a fine-tuned RoboEgo omnimodal chatbot, the system achieves fact-consistency scores above 87% in real-time personalized dialogs, establishing a strong baseline for future research.

Method: Proposed BuildingGym - an open-source tool that integrates EnergyPlus as core simulator, supports system-level and room-level control, accepts external signals, provides built-in RL algorithms, and allows easy configuration of control problems.

Result: BuildingGym efficiently set up training tasks for cooling load management (both constant and dynamic) and demonstrated strong performance with built-in algorithms, showing effectiveness in optimizing cooling strategies.

Conclusion: BuildingGym bridges the gap between building managers and AI specialists by providing a flexible, research-friendly framework that simplifies RL implementation for building energy management across various control problems and environments.

Abstract: Reinforcement learning (RL) has proven effective for AI-based building energy management. However, there is a lack of flexible framework to implement RL across various control problems in building energy management. To address this gap, we propose BuildingGym, an open-source tool designed as a research-friendly and flexible framework for training RL control strategies for common challenges in building energy management. BuildingGym integrates EnergyPlus as its core simulator, making it suitable for both system-level and room-level control. Additionally, BuildingGym is able to accept external signals as control inputs instead of taking the building as a stand-alone entity. This feature makes BuildingGym applicable for more flexible environments, e.g. smart grid and EVs community. The tool provides several built-in RL algorithms for control strategy training, simplifying the process for building managers to obtain optimal control strategies. Users can achieve this by following a few straightforward steps to configure BuildingGym for optimization control for common problems in the building energy management field. Moreover, AI specialists can easily implement and test state-of-the-art control algorithms within the platform. BuildingGym bridges the gap between building managers and AI specialists by allowing for the easy configuration and replacement of RL algorithms, simulators, and control environments or problems. With BuildingGym, we efficiently set up training tasks for cooling load management, targeting both constant and dynamic cooling load management. The built-in algorithms demonstrated strong performance across both tasks, highlighting the effectiveness of BuildingGym in optimizing cooling strategies.

Method: Proposes dynamical systems theory as the foundational framework, utilizing differential calculus for modeling inference, learning, and control. Employs noise as a learning resource and differential genetic programming to discover dynamical systems that implement adaptive behaviors.

Result: The paper establishes a principled approach for developing neuromorphic systems where intelligent behavior emerges from physical substrate dynamics, potentially achieving orders of magnitude greater energy efficiency compared to conventional digital approaches.

Conclusion: Dynamical systems theory provides the necessary theoretical foundation to advance neuromorphic computing, enabling sustainable and accessible intelligent systems where intelligence arises naturally from physical dynamics, contributing to both AI science and sustainability.

Abstract: Neuromorphic computing seeks to replicate the remarkable efficiency, flexibility, and adaptability of the human brain in artificial systems. Unlike conventional digital approaches, which depend on massive computational and energy resources, neuromorphic systems exploit brain-inspired principles of computation to achieve orders of magnitude greater energy efficiency. By drawing on insights from artificial intelligence, neuroscience, physics, chemistry, and materials science, neuromorphic computing promises to deliver intelligent systems that are sustainable, transparent, and widely accessible. A central challenge, however, is to identify a unifying theoretical framework capable of bridging these diverse disciplines. We argue that dynamical systems theory provides such a foundation. Rooted in differential calculus, it offers a principled language for modeling inference, learning, and control in both natural and artificial substrates. Within this framework, noise can be harnessed as a resource for learning, while differential genetic programming enables the discovery of dynamical systems that implement adaptive behaviors. Embracing this perspective paves the way toward emergent neuromorphic intelligence, where intelligent behavior arises from the dynamics of physical substrates, advancing both the science and sustainability of AI.

Method: Introduced Relative Precision and Recall of Algorithmic Diagnostics (RPAD and RRAD) that compare AI outputs against multiple expert opinions. Evaluated using 360 medical dialogues comparing LLMs against physician panels, with automated methodology for free-form diagnosis matching achieving 98% accuracy.

Result: Top-performing models like DeepSeek-V3 achieve consistency on par with or exceeding expert consensus. Expert judgments show significant variability - often greater than between AI and humans. Automated diagnosis matching achieved 98% accuracy.

Conclusion: The study demonstrates limitations of absolute metrics and supports adopting relative metrics in medical AI, as expert variability exceeds AI-human differences, making normalized metrics essential for realistic performance assessment.

Abstract: The integration of artificial intelligence (AI) into medical diagnostic workflows requires robust and consistent evaluation methods to ensure reliability, clinical relevance, and the inherent variability in expert judgments. Traditional metrics like precision and recall often fail to account for the inherent variability in expert judgments, leading to inconsistent assessments of AI performance. Inter-rater agreement statistics like Cohen’s Kappa are more reliable but they lack interpretability. We introduce Relative Precision and Recall of Algorithmic Diagnostics (RPAD and RRAD) - a new evaluation metrics that compare AI outputs against multiple expert opinions rather than a single reference. By normalizing performance against inter-expert disagreement, these metrics provide a more stable and realistic measure of the quality of predicted diagnosis. In addition to the comprehensive analysis of diagnostic quality measures, our study contains a very important side result. Our evaluation methodology allows us to avoid selecting diagnoses from a limited list when evaluating a given case. Instead, both the models being tested and the examiners verifying them arrive at a free-form diagnosis. In this automated methodology for establishing the identity of free-form clinical diagnoses, a remarkable 98% accuracy becomes attainable. We evaluate our approach using 360 medical dialogues, comparing multiple large language models (LLMs) against a panel of physicians. Large-scale study shows that top-performing models, such as DeepSeek-V3, achieve consistency on par with or exceeding expert consensus. Moreover, we demonstrate that expert judgments exhibit significant variability - often greater than that between AI and humans. This finding underscores the limitations of any absolute metrics and supports the need to adopt relative metrics in medical AI.

Method: A temporal graph-based reasoning process using directed graphs that accommodates dynamic changes through backtracking, refines reasoning content, and handles multimodal data at different time points. Includes a multi-agent framework for task distribution and cross-validation.

Result: Preliminary experiments and analysis demonstrate the novelty and practical utility of the approach, showing improved reasoning capabilities for medical diagnosis.

Conclusion: The proposed framework effectively addresses multimodal medical reasoning challenges by enabling dynamic reasoning with temporal data tracking, backtracking capabilities, and multi-agent validation for enhanced diagnostic accuracy in healthcare.

Abstract: Healthcare and medicine are multimodal disciplines that deal with multimodal data for reasoning and diagnosing multiple diseases. Although some multimodal reasoning models have emerged for reasoning complex tasks in scientific domains, their applications in the healthcare domain remain limited and fall short in correct reasoning for diagnosis. To address the challenges of multimodal medical reasoning for correct diagnosis and assist the healthcare professionals, a novel temporal graph-based reasoning process modelled through a directed graph has been proposed in the current work. It helps in accommodating dynamic changes in reasons through backtracking, refining the reasoning content, and creating new or deleting existing reasons to reach the best recommendation or answer. Again, consideration of multimodal data at different time points can enable tracking and analysis of patient health and disease progression. Moreover, the proposed multi-agent temporal reasoning framework provides task distributions and a cross-validation mechanism to further enhance the accuracy of reasoning outputs. A few basic experiments and analysis results justify the novelty and practical utility of the proposed preliminary approach.

Method: Conducted a systematic review of 51 peer-reviewed studies to identify and analyze Human-AI collaboration patterns across all disaster management phases.

Result: Identified four major categories: Human-AI Decision Support Systems, Task and Resource Coordination, Trust and Transparency, and Simulation and Training, with sub-patterns including cognitive-augmented intelligence and explainable AI. Found AI enhances situational awareness and response efficiency but has limitations in scalability and interpretability.

Conclusion: Future research should focus on developing adaptive, trustworthy, and context-aware Human-AI systems to improve disaster resilience and ensure equitable recovery outcomes.

Abstract: In high-stakes disaster scenarios, timely and informed decision-making is critical yet often challenged by uncertainty, dynamic environments, and limited resources. This paper presents a systematic review of Human-AI collaboration patterns that support decision-making across all disaster management phases. Drawing from 51 peer-reviewed studies, we identify four major categories: Human-AI Decision Support Systems, Task and Resource Coordination, Trust and Transparency, and Simulation and Training. Within these, we analyze sub-patterns such as cognitive-augmented intelligence, multi-agent coordination, explainable AI, and virtual training environments. Our review highlights how AI systems may enhance situational awareness, improves response efficiency, and support complex decision-making, while also surfacing critical limitations in scalability, interpretability, and system interoperability. We conclude by outlining key challenges and future research directions, emphasizing the need for adaptive, trustworthy, and context-aware Human-AI systems to improve disaster resilience and equitable recovery outcomes.

Method: Created SSUI dataset with interpretable reasoning paths, and developed Safety-aware Reasoning Path Optimization (SRPO) training framework to align MLLM reasoning processes with human safety values.

Result: SRPO-trained models achieved state-of-the-art performance on safety benchmarks including the new Reasoning Path Benchmark (RSBench), significantly outperforming both open-source and top commercial MLLMs.

Conclusion: The proposed SSUI dataset and SRPO framework effectively address implicit reasoning risks in MLLMs by optimizing safety alignment throughout the reasoning process, demonstrating superior safety performance.

Abstract: Multimodal Large Language Models (MLLMs) are susceptible to the implicit reasoning risk, wherein innocuous unimodal inputs synergistically assemble into risky multimodal data that produce harmful outputs. We attribute this vulnerability to the difficulty of MLLMs maintaining safety alignment through long-chain reasoning. To address this issue, we introduce Safe-Semantics-but-Unsafe-Interpretation (SSUI), the first dataset featuring interpretable reasoning paths tailored for such a cross-modal challenge. A novel training framework, Safety-aware Reasoning Path Optimization (SRPO), is also designed based on the SSUI dataset to align the MLLM’s internal reasoning process with human safety values. Experimental results show that our SRPO-trained models achieve state-of-the-art results on key safety benchmarks, including the proposed Reasoning Path Benchmark (RSBench), significantly outperforming both open-source and top-tier commercial MLLMs.

Method: A dedicated SysML profile with reusable stereotypes for planning constructs, integrated into existing models and processed by an algorithm that automatically generates valid PDDL domain and problem files.

Result: The method enables native integration and maintains consistency between engineering and planning artifacts, demonstrated through an aircraft assembly case study.

Conclusion: The approach successfully bridges the gap between MBSE models and AI planning by automatically generating planning artifacts that enable validation of system variants through symbolic planning.

Abstract: Engineering models created in Model-Based Systems Engineering (MBSE) environments contain detailed information about system structure and behavior. However, they typically lack symbolic planning semantics such as preconditions, effects, and constraints related to resource availability and timing. This limits their ability to evaluate whether a given system variant can fulfill specific tasks and how efficiently it performs compared to alternatives. To address this gap, this paper presents a model-driven method that enables the specification and automated generation of symbolic planning artifacts within SysML-based engineering models. A dedicated SysML profile introduces reusable stereotypes for core planning constructs. These are integrated into existing model structures and processed by an algorithm that generates a valid domain file and a corresponding problem file in Planning Domain Definition Language (PDDL). In contrast to previous approaches that rely on manual transformations or external capability models, the method supports native integration and maintains consistency between engineering and planning artifacts. The applicability of the method is demonstrated through a case study from aircraft assembly. The example illustrates how existing engineering models are enriched with planning semantics and how the proposed workflow is applied to generate consistent planning artifacts from these models. The generated planning artifacts enable the validation of system variants through AI planning.

Method: Developed a toolkit with structured label system (65 extra-legal factors), three fairness metrics (inconsistency, bias, imbalanced inaccuracy), statistical inference methods, and visualizations for two experiment types: structured output generation and statistical analysis.

Result: Empirical application revealed significant fairness deficiencies in current LLMs, showing lack of fair and trustworthy legal tools.

Conclusion: JustEva provides a convenient tool and methodological foundation for evaluating and improving algorithmic fairness in the legal domain.

Abstract: The integration of Large Language Models (LLMs) into legal practice raises pressing concerns about judicial fairness, particularly due to the nature of their “black-box” processes. This study introduces JustEva, a comprehensive, open-source evaluation toolkit designed to measure LLM fairness in legal tasks. JustEva features several advantages: (1) a structured label system covering 65 extra-legal factors; (2) three core fairness metrics - inconsistency, bias, and imbalanced inaccuracy; (3) robust statistical inference methods; and (4) informative visualizations. The toolkit supports two types of experiments, enabling a complete evaluation workflow: (1) generating structured outputs from LLMs using a provided dataset, and (2) conducting statistical analysis and inference on LLMs’ outputs through regression and other statistical methods. Empirical application of JustEva reveals significant fairness deficiencies in current LLMs, highlighting the lack of fair and trustworthy LLM legal tools. JustEva offers a convenient tool and methodological foundation for evaluating and improving algorithmic fairness in the legal domain.

Method: Created CPC-Bench using 7102 CPCs and 1021 Image Challenges with physician annotation, then evaluated leading LLMs and developed Dr. CaBot AI discussant to produce written and slide-based presentations mimicking human experts.

Result: OpenAI’s o3 ranked diagnosis first in 60% of cases (top ten in 84%), outperforming 20-physician baseline. CaBot presentations were misclassified as human in 74% of blinded trials and scored more favorably across quality dimensions than human experts.

Conclusion: LLMs exceed physician performance on complex text-based differential diagnosis and can convincingly emulate expert medical presentations, but image interpretation and literature retrieval need improvement. CPC-Bench and CaBot enable ongoing tracking of medical AI progress.

Abstract: BACKGROUND: For over a century, the New England Journal of Medicine Clinicopathological Conferences (CPCs) have tested the reasoning of expert physicians and, recently, artificial intelligence (AI). However, prior AI evaluations have focused on final diagnoses without addressing the multifaceted reasoning and presentation skills required of expert discussants. METHODS: Using 7102 CPCs (1923-2025) and 1021 Image Challenges (2006-2025), we conducted extensive physician annotation and automated processing to create CPC-Bench, a physician-validated benchmark spanning 10 text-based and multimodal tasks, against which we evaluated leading large language models (LLMs). Then, we developed “Dr. CaBot,” an AI discussant designed to produce written and slide-based video presentations using only the case presentation, modeling the role of the human expert in these cases. RESULTS: When challenged with 377 contemporary CPCs, o3 (OpenAI) ranked the final diagnosis first in 60% of cases and within the top ten in 84% of cases, outperforming a 20-physician baseline; next-test selection accuracy reached 98%. Event-level physician annotations quantified AI diagnostic accuracy per unit of information. Performance was lower on literature search and image tasks; o3 and Gemini 2.5 Pro (Google) achieved 67% accuracy on image challenges. In blinded comparisons of CaBot vs. human expert-generated text, physicians misclassified the source of the differential in 46 of 62 (74%) of trials, and scored CaBot more favorably across quality dimensions. To promote research, we are releasing CaBot and CPC-Bench. CONCLUSIONS: LLMs exceed physician performance on complex text-based differential diagnosis and convincingly emulate expert medical presentations, but image interpretation and literature retrieval remain weaker. CPC-Bench and CaBot may enable transparent and continued tracking of progress in medical AI.

Method: International survey with 76 responses and 15 semi-structured interviews with DH scholars, analyzing rationale for adoption, specific practices, and perceptions of benefits/risks/challenges.

Result: DH research communities hold divided opinions - scholars acknowledge GenAI’s benefits for research efficiency and reskilling but remain concerned about potential disruption to intellectual identities.

Conclusion: GenAI adoption is gradually changing Digital Humanities, though this transformation remains contested and shaped by ongoing negotiations among human and non-human actors, serving as foundational empirical analysis for future research.

Abstract: The advent of generative artificial intelligence (GenAI) technologies has been changing the research landscape and potentially has significant implications for Digital Humanities (DH), a field inherently intertwined with technologies. This article investigates how DH scholars adopt and critically evaluate GenAI technologies for research. Drawing on 76 responses collected from an international survey study and 15 semi-structured interviews with DH scholars, we explored the rationale for adopting GenAI tools in research, identified the specific practices of using GenAI tools, and analyzed scholars’ collective perceptions regarding the benefits, risks, and challenges. The results reveal that DH research communities hold divided opinions and differing imaginations towards the role of GenAI in DH scholarship. While scholars acknowledge the benefits of GenAI in enhancing research efficiency and enabling reskilling, many remain concerned about its potential to disrupt their intellectual identities. Situated within the history of DH and viewed through the lens of Actor-Network Theory, our findings suggest that the adoption of GenAI is gradually changing the field, though this transformation remains contested, shaped by ongoing negotiations among multiple human and non-human actors. Our study is one of the first empirical analyses on this topic and has the potential to serve as a building block for future inquiries into the impact of GenAI on DH scholarship.

Method: Created SearchBench with 11 unique search problems and automated evaluation pipelines. Tested step-by-step reasoning vs prompting models to generate complete A* search algorithms, plus Multi-Stage-Multi-Try inference.

Result: Standard reasoning: GPT-4 solved 1.4%, o1-preview solved 18.6%. A* search approach boosted GPT-4 to over 57% correct solutions.

Conclusion: LLMs struggle with iterative search and backtracking in text, but can effectively solve these problems by generating search algorithms that handle the complex reasoning externally.

Abstract: Large Language Models (LLMs) have recently achieved impressive performance in math and reasoning benchmarks. However, they often struggle with logic problems and puzzles that are relatively easy for humans. To further investigate this, we introduce a new benchmark, SearchBench, which contains 11 unique search problems inspired by intuitive puzzles. Each SearchBench problem type is equipped with automated pipelines to generate an arbitrary number of instances and analyze the feasibility, correctness, and optimality of LLM-generated solutions. We show that using step-by-step, language-only reasoning, even the most advanced LLMs fail to solve SearchBench; for example, OpenAI’s frontier models GPT-4 and o1-preview solve only 1.4% and 18.6% of problems, respectively. The reason is that SearchBench problems require considering multiple pathways and performing backtracking, posing a significant challenge to auto-regressive models. Interestingly, performance is significantly boosted when we prompt models to generate a complete A* search algorithm - a comparatively more cognitively difficult task. This approach effectively offloads the iterative search and backtracking process from the models, which they struggle with in text. This in-context learning baseline is further enhanced via a Multi-Stage-Multi-Try (MSMT) inference method, increasing GPT-4’s rate of correct solutions to over 57%.

Method: Proposes a unified VRP environment with global attribute embeddings, transformer-based encoder, mixed batch training for multi-task learning, multi-variant reward normalization, and efficient adapter layers for fine-tuning to new variants.

Result: Extensive experiments on 48 VRP variants show RouteFinder outperforms recent state-of-the-art learning methods, demonstrating superior performance across multiple problem types.

Conclusion: RouteFinder provides an effective foundation model framework for VRPs that can handle diverse variants through unified representation and specialized training techniques, achieving state-of-the-art performance while enabling efficient adaptation to new problem types.

Abstract: This paper introduces RouteFinder, a comprehensive foundation model framework to tackle different Vehicle Routing Problem (VRP) variants. Our core idea is that a foundation model for VRPs should be able to represent variants by treating each as a subset of a generalized problem equipped with different attributes. We propose a unified VRP environment capable of efficiently handling any combination of these attributes. The RouteFinder model leverages a modern transformer-based encoder and global attribute embeddings to improve task representation. Additionally, we introduce two reinforcement learning techniques to enhance multi-task performance: mixed batch training, which enables training on different variants at once, and multi-variant reward normalization to balance different reward scales. Finally, we propose efficient adapter layers that enable fine-tuning for new variants with unseen attributes. Extensive experiments on 48 VRP variants show RouteFinder outperforms recent state-of-the-art learning methods. Our code is publicly available at https://github.com/ai4co/routefinder.

Method: Introduces COMMA, a novel puzzle benchmark with various multimodal puzzles designed to evaluate collaborative performance of multimodal multi-agent systems through language communication across four key capability categories.

Result: Findings reveal surprising weaknesses in state-of-the-art models - GPT-4o, o4-mini, and reasoning models like R1-Onevision and LLaVA-CoT struggle with agent-agent collaboration, often performing worse than random baselines.

Conclusion: There is significant room for improvement in communication abilities of current multimodal agents, indicating a critical growth area for developing more effective collaborative AI systems.

Abstract: The rapid advances of multimodal agents built on large foundation models have largely overlooked their potential for language-based communication between agents in collaborative tasks. This oversight presents a critical gap in understanding their effectiveness in real-world deployments, particularly when communicating with humans. Existing agentic benchmarks fail to address key aspects of inter-agent communication and collaboration, particularly in scenarios where agents have unequal access to information and must work together to achieve tasks beyond the scope of individual capabilities. To fill this gap, we introduce COMMA: a novel puzzle benchmark designed to evaluate the collaborative performance of multimodal multi-agent systems through language communication. Our benchmark features a variety of multimodal puzzles, providing a comprehensive evaluation across four key categories of agentic capability in a communicative collaboration setting. Our findings reveal surprising weaknesses in state-of-the-art models, including strong proprietary models like GPT-4o and reasoning models like o4-mini. Many chain of thought reasoning models such as R1-Onevision and LLaVA-CoT struggle to outperform even a random baseline in agent-agent collaboration, indicating a potential growth area in their communication abilities.

Method: The survey categorizes advancements into two main approaches: prompt-based methods using LLMs for instruction-based execution, and training-based methods that fine-tune multimodal models for mobile-specific applications.

Result: Recent evaluation benchmarks have been developed to better capture static and interactive mobile environments, providing more accurate performance assessments of mobile agents.

Conclusion: The survey offers insights into key challenges and future research directions for advancing mobile agent technologies, along with a comprehensive resource list.

Abstract: Mobile agents are essential for automating tasks in complex and dynamic mobile environments. As foundation models evolve, the demands for agents that can adapt in real-time and process multimodal data have grown. This survey provides a comprehensive review of mobile agent technologies, focusing on recent advancements that enhance real-time adaptability and multimodal interaction. Recent evaluation benchmarks have been developed better to capture the static and interactive environments of mobile tasks, offering more accurate assessments of agents’ performance. We then categorize these advancements into two main approaches: prompt-based methods, which utilize large language models (LLMs) for instruction-based task execution, and training-based methods, which fine-tune multimodal models for mobile-specific applications. Additionally, we explore complementary technologies that augment agent performance. By discussing key challenges and outlining future research directions, this survey offers valuable insights for advancing mobile agent technologies. A comprehensive resource list is available at https://github.com/aialt/awesome-mobile-agents

Method: Survey and synthesis of existing literature, organizing the field into key categories including value-based methods, policy-based methods, model-based methods, multi-agent RL, LLMs and RL integration, and specialized topics like offline RL, hierarchical RL, and intrinsic rewards.

Result: A structured overview that maps the current landscape of reinforcement learning research, highlighting connections between different approaches and identifying major research directions and methodologies.

Conclusion: This manuscript serves as a valuable resource for understanding the breadth and depth of modern reinforcement learning, providing researchers and practitioners with a framework to navigate the diverse methodologies and applications in sequential decision making.

Abstract: This manuscript gives a big-picture, up-to-date overview of the field of (deep) reinforcement learning and sequential decision making, covering value-based methods, policy-based methods, model-based methods, multi-agent RL, LLMs and RL, and various other topics (e.g., offline RL, hierarchical RL, intrinsic reward).

Method: Comprehensive survey approach examining the evolution, capabilities, and design frameworks of LLM-based data agents, including analysis of planning, reasoning, reflection, multi-agent collaboration, UI design, knowledge integration, and system architecture.

Result: Identifies current trends in data agent design and demonstrates practical applications through case studies, showing how these agents can effectively handle data-centric problems in real-world scenarios.

Conclusion: LLM-based data agents have significant potential to transform data analysis but face challenges that require future research to advance them into intelligent statistical analysis software.

Abstract: In recent years, data science agents powered by Large Language Models (LLMs), known as “data agents,” have shown significant potential to transform the traditional data analysis paradigm. This survey provides an overview of the evolution, capabilities, and applications of LLM-based data agents, highlighting their role in simplifying complex data tasks and lowering the entry barrier for users without related expertise. We explore current trends in the design of LLM-based frameworks, detailing essential features such as planning, reasoning, reflection, multi-agent collaboration, user interface, knowledge integration, and system design, which enable agents to address data-centric problems with minimal human intervention. Furthermore, we analyze several case studies to demonstrate the practical applications of various data agents in real-world scenarios. Finally, we identify key challenges and propose future research directions to advance the development of data agents into intelligent statistical analysis software.

Method: Created a benchmark of 140 yes/no questions from NASA Earth Observatory articles across 13 topics and 17 satellite sensors, using Google Earth Engine API as a tool for LLM agents to execute code.

Result: LLM agents achieved only 33% accuracy with over 58% code execution failure rate. Fine-tuning with synthetic data improved performance, allowing smaller models (Llama-3.1-8B) to match larger ones (DeepSeek-R1).

Conclusion: Significant challenges remain before AI agents can automate Earth Observation, but fine-tuning approaches show potential paths forward for improving reliability.

Abstract: Earth Observation (EO) provides critical planetary data for environmental monitoring, disaster management, climate science, and other scientific domains. Here we ask: Are AI systems ready for reliable Earth Observation? We introduce \datasetnamenospace, a benchmark of 140 yes/no questions from NASA Earth Observatory articles across 13 topics and 17 satellite sensors. Using Google Earth Engine API as a tool, LLM agents can only achieve an accuracy of 33% because the code fails to run over 58% of the time. We improve the failure rate for open models by fine-tuning synthetic data, allowing much smaller models (Llama-3.1-8B) to achieve comparable accuracy to much larger ones (e.g., DeepSeek-R1). Taken together, our findings identify significant challenges to be solved before AI agents can automate earth observation, and suggest paths forward. The project page is available at https://iandrover.github.io/UnivEarth.

Method: Uses One Night Ultimate Werewolf as testbed. Processes facial expressions and vocal tones alongside verbal content. Employs Theory of Mind model to represent each player’s suspicion levels. Combines ToM with Monte Carlo Tree Search to identify communication strategies that minimize suspicion.

Result: Demonstrated superior performance in both agent-versus-agent simulations and human player studies. MultiMind outperforms existing approaches in gameplay effectiveness.

Conclusion: Presents a significant advancement toward LLM agents capable of human-like social reasoning across multimodal domains, addressing key limitations of current text-only approaches.

Abstract: Large Language Model (LLM) agents have demonstrated impressive capabilities in social deduction games (SDGs) like Werewolf, where strategic reasoning and social deception are essential. However, current approaches remain limited to textual information, ignoring crucial multimodal cues such as facial expressions and tone of voice that humans naturally use to communicate. Moreover, existing SDG agents primarily focus on inferring other players’ identities without modeling how others perceive themselves or fellow players. To address these limitations, we use One Night Ultimate Werewolf (ONUW) as a testbed and present MultiMind, the first framework integrating multimodal information into SDG agents. MultiMind processes facial expressions and vocal tones alongside verbal content, while employing a Theory of Mind (ToM) model to represent each player’s suspicion levels toward others. By combining this ToM model with Monte Carlo Tree Search (MCTS), our agent identifies communication strategies that minimize suspicion directed at itself. Through comprehensive evaluation in both agent-versus-agent simulations and studies with human players, we demonstrate MultiMind’s superior performance in gameplay. Our work presents a significant advancement toward LLM agents capable of human-like social reasoning across multimodal domains.

Method: Three-stage framework: 1) sample segmentation, 2) consensus-based step correctness assessment using LLM and PRM judgers, 3) policy optimization with negative sample augmentation to extract positive steps from negative samples.

Result: Outperforms baselines on math/coding reasoning benchmarks using the same training data, achieving improved sample efficiency, robustness, and scalability across multiple iterations.

Conclusion: BCPG-NSA successfully demonstrates that negative reasoning samples contain valuable learning signals that can be effectively mined through fine-grained analysis and targeted optimization, leading to better performance with fixed training datasets.

Abstract: Recent advances in reasoning language models have witnessed a paradigm shift from short to long CoT pattern. Given the substantial computational cost of rollouts in long CoT models, maximizing the utility of fixed training datasets becomes crucial. Our analysis reveals that negative responses contain valuable components such as self-reflection and error-correction steps, yet primary existing methods either completely discard negative samples (RFT) or apply equal penalization across all tokens (RL), failing to leverage these potential learning signals. In light of this, we propose Behavior Constrained Policy Gradient with Negative Sample Augmentation (BCPG-NSA), a fine-grained offline RL framework that encompasses three stages: 1) sample segmentation, 2) consensus-based step correctness assessment combining LLM and PRM judgers, and 3) policy optimization with NSA designed to effectively mine positive steps within negative samples. Experimental results show that BCPG-NSA outperforms baselines on several challenging math/coding reasoning benchmarks using the same training dataset, achieving improved sample efficiency and demonstrating robustness and scalability when extended to multiple iterations.

Method: Proposes Self-Evolving Curriculum (SEC) that treats curriculum selection as a non-stationary Multi-Armed Bandit problem, using absolute advantage from policy gradient as reward signal and updating with TD(0) method.

Result: SEC significantly improves models’ reasoning capabilities across planning, inductive reasoning, and mathematics domains, enabling better generalization to harder out-of-distribution problems and achieving better skill balance in multi-domain fine-tuning.

Conclusion: SEC is a promising automatic curriculum learning strategy that effectively optimizes RL fine-tuning of LLMs without manual heuristics or excessive computational costs.

Abstract: Reinforcement learning (RL) has proven effective for fine-tuning large language models (LLMs), significantly enhancing their reasoning abilities in domains such as mathematics and code generation. A crucial factor influencing RL fine-tuning success is the training curriculum: the order in which training problems are presented. While random curricula serve as common baselines, they remain suboptimal; manually designed curricula often rely heavily on heuristics, and online filtering methods can be computationally prohibitive. To address these limitations, we propose Self-Evolving Curriculum (SEC), an automatic curriculum learning method that learns a curriculum policy concurrently with the RL fine-tuning process. Our approach formulates curriculum selection as a non-stationary Multi-Armed Bandit problem, treating each problem category (e.g., difficulty level or problem type) as an individual arm. We leverage the absolute advantage from policy gradient methods as a proxy measure for immediate learning gain. At each training step, the curriculum policy selects categories to maximize this reward signal and is updated using the TD(0) method. Across three distinct reasoning domains: planning, inductive reasoning, and mathematics, our experiments demonstrate that SEC significantly improves models’ reasoning capabilities, enabling better generalization to harder, out-of-distribution test problems. Additionally, our approach achieves better skill balance when fine-tuning simultaneously on multiple reasoning domains. These findings highlight SEC as a promising strategy for RL fine-tuning of LLMs.

Method: Proposes a graph-based framework that leverages human-curated relational knowledge to explicitly encode schema links and join paths. Uses graph search to construct interpretable reasoning chains with pruning and sub-path merging strategies for efficiency.

Result: Experiments on standard benchmarks and a realistic large-scale dataset demonstrate the effectiveness of the approach, showing it can handle truly complex industrial tabular data.

Conclusion: This represents the first multi-table QA system successfully applied to complex industrial tabular data, providing an effective solution for schema linking challenges in real-world scenarios.

Abstract: Large language models (LLMs) have shown promise in table Question Answering (Table QA). However, extending these capabilities to multi-table QA remains challenging due to unreliable schema linking across complex tables. Existing methods based on semantic similarity work well only on simplified hand-crafted datasets and struggle to handle complex, real-world scenarios with numerous and diverse columns. To address this, we propose a graph-based framework that leverages human-curated relational knowledge to explicitly encode schema links and join paths. Given a natural language query, our method searches on graph to construct interpretable reasoning chains, aided by pruning and sub-path merging strategies to enhance efficiency and coherence. Experiments on both standard benchmarks and a realistic, large-scale dataset demonstrate the effectiveness of our approach. To our knowledge, this is the first multi-table QA system applied to truly complex industrial tabular data.

Method: Multi-Agent Reinforcement Learning with Group Relative Policy Optimization, theory-based reward functions, and multi-modal agent pipeline integrating LLM semantic features, social graphs, and on-chain data.

Result: Achieves top performance in detection accuracy and causal attribution when trained on 100,000 real-world episodes and validated in adversarial simulations.

Conclusion: Bridges multi-agent systems with financial surveillance, advancing decentralized market intelligence without centralized oracles through the Symphony system.

Abstract: Decentralized finance (DeFi) has introduced a new era of permissionless financial innovation but also led to unprecedented market manipulation. Without centralized oversight, malicious actors coordinate shilling campaigns and pump-and-dump schemes across various platforms. We propose a Multi-Agent Reinforcement Learning (MARL) framework for decentralized manipulation detection, modeling the interaction between manipulators and detectors as a dynamic adversarial game. This framework identifies suspicious patterns using delayed token price reactions as financial indicators.Our method introduces three innovations: (1) Group Relative Policy Optimization (GRPO) to enhance learning stability in sparse-reward and partially observable settings; (2) a theory-based reward function inspired by rational expectations and information asymmetry, differentiating price discovery from manipulation noise; and (3) a multi-modal agent pipeline that integrates LLM-based semantic features, social graph signals, and on-chain market data for informed decision-making.The framework is integrated within the Symphony system, a decentralized multi-agent architecture enabling peer-to-peer agent execution and trust-aware learning through distributed logs, supporting chain-verifiable evaluation. Symphony promotes adversarial co-evolution among strategic actors and maintains robust manipulation detection without centralized oracles, enabling real-time surveillance across global DeFi ecosystems.Trained on 100,000 real-world discourse episodes and validated in adversarial simulations, Hide-and-Shill achieves top performance in detection accuracy and causal attribution. This work bridges multi-agent systems with financial surveillance, advancing a new paradigm for decentralized market intelligence. All resources are available at the Hide-and-Shill GitHub repository to promote open research and reproducibility.

Method: SigmaScheduling dynamically schedules decision points based on uncertainty in predicted behavior times - scheduling closer to predicted time when behavior is predictable, and earlier when timing is uncertain to increase intervention likelihood.

Result: Evaluation with 68 participants in a 10-week Oralytics trial showed SigmaScheduling increased the likelihood that decision points preceded brushing events in at least 70% of cases, preserving intervention opportunities.

Conclusion: SigmaScheduling can advance precision mHealth, particularly for JITAIs targeting time-sensitive habitual behaviors like oral hygiene or dietary habits, by improving timely intervention delivery.

Abstract: Timely decision making is critical to the effectiveness of mobile health (mHealth) interventions. At predefined timepoints called “decision points,” intelligent mHealth systems such as just-in-time adaptive interventions (JITAIs) estimate an individual’s biobehavioral context from sensor or survey data and determine whether and how to intervene. For interventions targeting habitual behavior (e.g., oral hygiene), effectiveness often hinges on delivering support shortly before the target behavior is likely to occur. Current practice schedules decision points at a fixed interval (e.g., one hour) before user-provided behavior times, and the fixed interval is kept the same for all individuals. However, this one-size-fits-all approach performs poorly for individuals with irregular routines, often scheduling decision points after the target behavior has already occurred, rendering interventions ineffective. In this paper, we propose SigmaScheduling, a method to dynamically schedule decision points based on uncertainty in predicted behavior times. When behavior timing is more predictable, SigmaScheduling schedules decision points closer to the predicted behavior time; when timing is less certain, SigmaScheduling schedules decision points earlier, increasing the likelihood of timely intervention. We evaluated SigmaScheduling using real-world data from 68 participants in a 10-week trial of Oralytics, a JITAI designed to improve daily toothbrushing. SigmaScheduling increased the likelihood that decision points preceded brushing events in at least 70% of cases, preserving opportunities to intervene and impact behavior. Our results indicate that SigmaScheduling can advance precision mHealth, particularly for JITAIs targeting time-sensitive, habitual behaviors such as oral hygiene or dietary habits.

Method: Translation of CASP programs into the solver-independent FlatZinc language, supporting rich linear constraints and common global constraints, then using backend solvers for solving.

Result: asp-fzn is competitive with state-of-the-art ASP solvers on standard benchmarks and outperforms clingcon (a prominent CASP solver) on some CASP benchmarks.

Conclusion: The approach of translating CASP to FlatZinc is effective and promising, providing competitive performance while enabling interoperability with multiple backend constraint solvers.

Abstract: We present the solver asp-fzn for Constraint Answer Set Programming (CASP), which extends ASP with linear constraints. Our approach is based on translating CASP programs into the solver-independent FlatZinc language that supports several Constraint Programming and Integer Programming backend solvers. Our solver supports a rich language of linear constraints, including some common global constraints. As for evaluation, we show that asp-fzn is competitive with state-of-the-art ASP solvers on benchmarks taken from past ASP competitions. Furthermore, we evaluate it on several CASP problems from the literature and compare its performance with clingcon, which is a prominent CASP solver that supports most of the asp-fzn language. The performance of asp-fzn is very promising as it is already competitive on plain ASP and even outperforms clingcon on some CASP benchmarks.

Method: Proposes a learning-to-optimize inverse optimizer using long-context processing techniques from LLMs, integrated with PPO-based planning framework and Swin UnetR dose predictor for initial parameter estimation.

Result: Achieves 22.97% effectiveness and 36.41% efficiency improvements over second-order gradient methods, generates plans in 2.55 hours average with improved OAR sparing and superior target coverage compared to human-generated plans.

Conclusion: The framework successfully automates proton therapy planning, significantly reducing optimization time while maintaining or improving plan quality for diverse H&N cancer treatment requirements.

Abstract: Proton pencil beam scanning (PBS) treatment planning for head & neck (H&N) cancers involves numerous conflicting objectives, requiring iterative objective parameter adjustments to balance multiple clinical goals. We propose a learning-driven inverse optimizer and integrate it into a proximal policy optimization (PPO)-based planning framework to automatically generate high-quality plans for patients with diverse treatment requirements. The inverse optimizer is a learning-to-optimize (L2O) method that predicts update steps by learning from task-specific data distributions. For the first time, long-context processing techniques developed for large language models (LLMs) are utilized to address the scalability limitations of existing L2O methods, enabling simultaneous optimization over a substantially large set of variables. The PPO framework functions as an outer-loop virtual planner, autonomously adjusting objective parameters through a policy network, and the inner-loop L2O inverse optimizer computes machine-deliverable spot monitor unit (MU) values based on the PPO-refined objectives. Moreover, a Swin UnetR dose predictor is trained with prescription- and beam-specific information to estimate the initial objective parameters. In our experiments, total 97 patients with bilateral or ipsilateral H&N cancers are collected for training and testing. Compared with the second-order gradient-based methods, our L2O optimizer improves the effectiveness and efficiency of the time-consuming inverse optimization by 22.97% and 36.41%, respectively, and in conjunction with the PPO-based virtual planner, plans are generated within clinically acceptable times, i.e. 2.55 hours in average, and shows improved or comparable organs-at-risk sparing with superior target coverage compared with human-generated plans.

Method: Connector-Aware Compact CoT (CAC-CoT) restricts reasoning to a small, fixed set of connector phrases to steer models toward concise and well-structured explanations.

Result: Achieves ~85% on GSM8K, ~40% on GPQA (System-2), and ~85% on S1-Bench (System-1) - surpassing baseline by over 20%. Reasoning traces average ~300 tokens (about 1/3 of baseline length).

Conclusion: CAC-CoT delivers higher efficiency without loss of accuracy, producing high-quality training with general-purpose LLMs through synthetic method with restricted connector phrases.

Abstract: Long chain-of-thought (CoT) prompting helps Large Language Models (LLMs) solve difficult problems, but very long traces often slow or even degrade performance on fast, intuitive “System-1” tasks. We introduce Connector-Aware Compact CoT (CAC-CoT) – a method that deliberately restricts reasoning to a small, fixed set of connector phrases, steering the model toward concise and well – structured explanations. Despite its simplicity, our synthetic method with general-purpose LLMs yields a high-quality training quality. CAC-CoT achieves approximately 85% on GSM8K and approximately 40% on GPQA (System-2) while also achieving approximately 85% on S1-Bench (System-1), surpassing the baseline by over 20%. Its reasoning traces average approximately 300 tokens(ART), about one-third the length of baseline traces, delivering higher efficiency without loss of accuracy.

Method: CSA combines game-theoretic anticipation of user reactions, fine-grained risk boundary discovery, and interpretable reasoning control, implemented in the Oyster-I (Oy1) model to turn safety into a trust-building process.

Result: Oy1 achieves state-of-the-art safety among open models, shows strong constructive engagement close to GPT-5, and unmatched robustness on jailbreak datasets nearing GPT-o1 levels.

Conclusion: CSA redefines the model-user relationship by shifting from refusal-first to guidance-first safety, creating systems that are not just safe but meaningfully helpful for vulnerable users.

Abstract: Large language models (LLMs) typically deploy safety mechanisms to prevent harmful content generation. Most current approaches focus narrowly on risks posed by malicious actors, often framing risks as adversarial events and relying on defensive refusals. However, in real-world settings, risks also come from non-malicious users seeking help while under psychological distress (e.g., self-harm intentions). In such cases, the model’s response can strongly influence the user’s next actions. Simple refusals may lead them to repeat, escalate, or move to unsafe platforms, creating worse outcomes. We introduce Constructive Safety Alignment (CSA), a human-centric paradigm that protects against malicious misuse while actively guiding vulnerable users toward safe and helpful results. Implemented in Oyster-I (Oy1), CSA combines game-theoretic anticipation of user reactions, fine-grained risk boundary discovery, and interpretable reasoning control, turning safety into a trust-building process. Oy1 achieves state-of-the-art safety among open models while retaining high general capabilities. On our Constructive Benchmark, it shows strong constructive engagement, close to GPT-5, and unmatched robustness on the Strata-Sword jailbreak dataset, nearing GPT-o1 levels. By shifting from refusal-first to guidance-first safety, CSA redefines the model-user relationship, aiming for systems that are not just safe, but meaningfully helpful. We release Oy1, code, and the benchmark to support responsible, user-centered AI.

Method: Theoretical analysis proving sample complexity bounds for learning from misspecified human feedback, comparing scenarios with and without a calibration oracle that identifies unreliable feedback regions.

Result: Without knowing where feedback is unreliable, learning requires exponentially many samples (exp(nαε²)), but with a calibration oracle, only O(1/(α*ε²)) queries are needed.

Conclusion: AI alignment faces fundamental limitations due to rare edge cases with biased feedback, creating exponentially hard problems unless problematic contexts can be identified and addressed through active routing around misspecification.

Abstract: We study reinforcement learning from human feedback under misspecification. Sometimes human feedback is systematically wrong on certain types of inputs, like a broken compass that points the wrong way in specific regions. We prove that when feedback is biased on a fraction alpha of contexts with bias strength epsilon, any learning algorithm needs exponentially many samples exp(nalphaepsilon^2) to distinguish between two possible “true” reward functions that differ only on these problematic contexts. However, if you can identify where feedback is unreliable (a “calibration oracle”), you can focus your limited questions there and overcome the exponential barrier with just O(1/(alpha*epsilon^2)) queries. This quantifies why alignment is hard: rare edge cases with subtly biased feedback create an exponentially hard learning problem unless you know where to look. The gap between what we optimize (proxy from human feedback) and what we want (true objective) is fundamentally limited by how common the problematic contexts are (alpha), how wrong the feedback is there (epsilon), and how much the true objectives disagree there (gamma). Murphy’s Law for AI alignment: the gap always wins unless you actively route around misspecification.

Method: A plug-and-play three-module pipeline (Intent Perceiver, Strategy Generator, Strategy Selector) that constructs a “understand-plan-select” cognitive process using “intent sketch” strategies without parameter fine-tuning, relying on in-context engineering for cross-model transfer.

Result: Achieves consistent improvements across different reasoning engines and pipeline combinations with gains up to ~9.51 percentage points on IntentBench, WorldSense, and Daily-Omni benchmarks, demonstrating generality and robust performance.

Conclusion: The “intent sketch” reasoning component shows practical value and portability in zero-shot scenarios, effectively reducing conditional entropy and improving information utilization efficiency for better multimodal reasoning.

Abstract: Targeting the issues of “shortcuts” and insufficient contextual understanding in complex cross-modal reasoning of multimodal large models, this paper proposes a zero-shot multimodal reasoning component guided by human-like cognitive strategies centered on an “intent sketch”. The component comprises a plug-and-play three-module pipeline-Intent Perceiver, Strategy Generator, and Strategy Selector-that explicitly constructs a “understand-plan-select” cognitive process. By generating and filtering “intent sketch” strategies to guide the final reasoning, it requires no parameter fine-tuning and achieves cross-model transfer solely through in-context engineering. Information-theoretic analysis shows that this process can reduce conditional entropy and improve information utilization efficiency, thereby suppressing unintended shortcut reasoning. Experiments on IntentBench, WorldSense, and Daily-Omni validate the method’s generality and robust gains; compared with their respective baselines, the complete “three-module” scheme yields consistent improvements across different reasoning engines and pipeline combinations, with gains up to approximately 9.51 percentage points, demonstrating the practical value and portability of the “intent sketch” reasoning component in zero-shot scenarios.

Method: A protocol running controlled 12-turn conversations per task with various prompts (vague to targeted), scoring outcomes with domain-appropriate checks (unit tests, answer-equivalence, originality) and tracking turn-level behavior with semantic movement, change, and size metrics.

Result: Gains are domain-dependent: early turns matter for ideas and code, late turns matter for math with elaboration. Vague feedback plateaus or reverses correctness, while targeted prompts reliably improve quality. Consistent domain patterns observed in semantic movement and output growth.

Conclusion: The framework makes iteration measurable and comparable across models, providing signals for when to steer, stop, or switch strategies in multi-turn LLM workflows.

Abstract: Large language models (LLMs) are now used in multi-turn workflows, but we still lack a clear way to measure when iteration helps and when it hurts. We present an evaluation framework for iterative refinement that spans ideation, code, and math. Our protocol runs controlled 12-turn conversations per task, utilizing a variety of prompts ranging from vague ``improve it’’ feedback to targeted steering, and logs per-turn outputs. We score outcomes with domain-appropriate checks (unit tests for code; answer-equivalence plus reasoning-soundness for math; originality and feasibility for ideation) and track turn-level behavior with three families of metrics: semantic movement across turns, turn-to-turn change, and output size growth. Across models and tasks, gains are domain-dependent: they arrive early in ideas and code, but in math late turns matter when guided by elaboration. After the first few turns, vague feedback often plateaus or reverses correctness, while targeted prompts reliably shift the intended quality axis (novelty vs. feasibility in ideation; speed vs. readability in code; in math, elaboration outperforms exploration and drives late-turn gains). We also observe consistent domain patterns: ideation moves more in meaning across turns, code tends to grow in size with little semantic change, and math starts fixed but can break that path with late, elaborative iteration. Together, the framework and metrics make iteration measurable and comparable across models, and signal when to steer, stop, or switch strategies.

Method: Compiled 13 latest Olympiad exams (2024-2025) with mixed modalities, adopted official marking schemes for fine-grained grading, and assigned medals based on official thresholds to compare models with human performance.

Result: Open-source MLLMs mostly remain at/below bronze level; open-source LLMs show promising progress with multiple golds; closed-source reasoning MLLMs achieve 6-12 gold medals; most models still have significant gap from full marks.

Conclusion: HiPhO reveals performance gaps between open-source models and top students, demonstrates strong reasoning abilities of closed-source models, and highlights remaining room for improvement in multimodal physical reasoning.

Abstract: Recently, the physical capabilities of (M)LLMs have garnered increasing attention. However, existing benchmarks for physics suffer from two major gaps: they neither provide systematic and up-to-date coverage of real-world physics competitions such as physics Olympiads, nor enable direct performance comparison with humans. To bridge these gaps, we present HiPhO, the first benchmark dedicated to high school physics Olympiads with human-aligned evaluation. Specifically, HiPhO highlights three key innovations. (1) Comprehensive Data: It compiles 13 latest Olympiad exams from 2024-2025, spanning both international and regional competitions, and covering mixed modalities that encompass problems spanning text-only to diagram-based. (2) Professional Evaluation: We adopt official marking schemes to perform fine-grained grading at both the answer and step level, fully aligned with human examiners to ensure high-quality and domain-specific evaluation. (3) Comparison with Human Contestants: We assign gold, silver, and bronze medals to models based on official medal thresholds, thereby enabling direct comparison between (M)LLMs and human contestants. Our large-scale evaluation of 30 state-of-the-art (M)LLMs shows that: across 13 exams, open-source MLLMs mostly remain at or below the bronze level; open-source LLMs show promising progress with multiple golds; closed-source reasoning MLLMs can achieve 6 to 12 gold medals; and most models still have a significant gap from full marks. These results highlight the performance gap between open-source models and top students, the strong reasoning abilities of closed-source models, and the remaining room for improvement. HiPhO, a human-aligned Olympiad benchmark for multimodal physical reasoning, is open-source at https://github.com/SciYu/HiPhO with a public leaderboard at https://phyarena.github.io/.

Method: Compared humans (N=216), LLMs (GPT-4o, Gemini 1.5 Pro), and Bayesian agents in identical dynamic negotiation conditions, capturing both outcomes and behavioral dynamics.

Result: Bayesian agents achieved highest surplus through aggressive optimization but with frequent rejections. Humans and LLMs achieved similar overall surplus but through distinct behaviors: LLMs favored conservative concessionary trades with few rejections, while humans used more strategic, risk-taking, and fairness-oriented approaches.

Conclusion: Performance parity alone is insufficient for agent evaluation - fundamental differences in process and alignment are critical considerations for practical deployment in real-world coordination tasks.

Abstract: Coordination tasks traditionally performed by humans are increasingly being delegated to autonomous agents. As this pattern progresses, it becomes critical to evaluate not only these agents’ performance but also the processes through which they negotiate in dynamic, multi-agent environments. Furthermore, different agents exhibit distinct advantages: traditional statistical agents, such as Bayesian models, may excel under well-specified conditions, whereas large language models (LLMs) can generalize across contexts. In this work, we compare humans (N = 216), LLMs (GPT-4o, Gemini 1.5 Pro), and Bayesian agents in a dynamic negotiation setting that enables direct, identical-condition comparisons across populations, capturing both outcomes and behavioral dynamics. Bayesian agents extract the highest surplus through aggressive optimization, at the cost of frequent trade rejections. Humans and LLMs can achieve similar overall surplus, but through distinct behaviors: LLMs favor conservative, concessionary trades with few rejections, while humans employ more strategic, risk-taking, and fairness-oriented behaviors. Thus, we find that performance parity – a common benchmark in agent evaluation – can conceal fundamental differences in process and alignment, which are critical for practical deployment in real-world coordination tasks.

Method: Template-Oriented Reasoning (TORSO) elicits models to utilize their internal reasoning abilities to generate proper responses across various tasks without manually crafted few-shot examples.

Result: Experimental results demonstrate that TORSO achieves strong performance on diverse LLM benchmarks while producing reasonable rationales.

Conclusion: TORSO provides an effective alternative to few-shot prompting that leverages LLMs’ inherent reasoning capabilities without the need for costly manual example construction.

Abstract: The approaches that guide Large Language Models (LLMs) to emulate human reasoning during response generation have emerged as an effective method for enabling them to solve complex problems in a step-by-step manner, thereby achieving superior performance. However, most existing approaches using few-shot prompts to generate responses heavily depend on the provided examples, limiting the utilization of the model’s inherent reasoning capabilities. Moreover, constructing task-specific few-shot prompts is often costly and may lead to inconsistencies across different tasks. In this work, we introduce Template-Oriented Reasoning (TORSO), which elicits the model to utilize internal reasoning abilities to generate proper responses across various tasks without the need for manually crafted few-shot examples. Our experimental results demonstrate that TORSO achieves strong performance on diverse LLMs benchmarks with reasonable rationales.

Method: Developed a single-species recognizer neural network that combines spectrogram inputs with non-audio data (habitat preferences, phenology, range information) as additional inputs.

Result: Networks using both spectrogram and non-audio data inputs achieved higher classification accuracy than networks of similar size using only one type of input.

Conclusion: Combining multiple data sources (audio and non-audio) provides better species classification results than simply increasing network parameters, demonstrating the value of integrating different data types in ecological classification tasks.

Abstract: In the last several years the use of neural networks as tools to automate species classification from digital data has increased. This has been due in part to the high classification accuracy of image classification through Convolutional Neural Networks (CNN). In the case of audio data CNN based recognizers are used to automate the classification of species in audio recordings by using information from sound visualization (i.e., spectrograms). It is common for these recognizers to use the spectrogram as their sole input. However, researchers have other non-audio data, such as habitat preferences of a species, phenology, and range information, available that could improve species classification. In this paper we present how a single-species recognizer neural network’s accuracy can be improved by using non-audio data as inputs in addition to spectrogram information. We also analyze if the improvements are merely a result of having a neural network with a higher number of parameters instead of combining the two inputs. We find that networks that use the two different inputs have a higher classification accuracy than networks of similar size that use only one of the inputs.

Method: Uses a pretrained Wav2Vec2 model fine-tuned on emotion recognition tasks to derive emotion-guided representations, then employs a dedicated convolutional feature extractor with residual connections to capture and refine emotional characteristics from transformer layer outputs.

Result: Achieves state-of-the-art performance on ASVspoof2019LA and ASVspoof2021LA benchmarks, and demonstrates strong generalization on ASVspoof2021DF dataset.

Conclusion: Emotional features provide valuable complementary information for audio deepfake detection, and the proposed EmoAnti system effectively leverages emotion-guided representations to enhance detection performance and generalization.

Abstract: Audio deepfake is so sophisticated that the lack of effective detection methods is fatal. While most detection systems primarily rely on low-level acoustic features or pretrained speech representations, they frequently neglect high-level emotional cues, which can offer complementary and potentially anti-deepfake information to enhance generalization. In this work, we propose a novel audio anti-deepfake system that utilizes emotional features (EmoAnti) by exploiting a pretrained Wav2Vec2 (W2V2) model fine-tuned on emotion recognition tasks, which derives emotion-guided representations, then designing a dedicated feature extractor based on convolutional layers with residual connections to effectively capture and refine emotional characteristics from the transformer layers outputs. Experimental results show that our proposed architecture achieves state-of-the-art performance on both the ASVspoof2019LA and ASVspoof2021LA benchmarks, and demonstrates strong generalization on the ASVspoof2021DF dataset. Our proposed approach’s code is available at Anonymous GitHub1.

Method: Uses an LLM agent to process text prompts through two pathways: Description Prompts for explicit spatial mapping and Abstract Prompts with RAG module to retrieve spatial templates, coupled with a separate physics-based spatial rendering engine.

Result: The system enables interpretable and user-controllable spatial reasoning for audio generation, though evaluation challenges for generative spatial audio are noted.

Conclusion: STASE provides a novel approach to text-to-spatial-audio generation by decoupling semantic interpretation from rendering, offering improved user control over spatial properties compared to existing methods.

Abstract: While many text-to-audio systems produce monophonic or fixed-stereo outputs, generating audio with user-defined spatial properties remains a challenge. Existing deep learning-based spatialization methods often rely on latent-space manipulations, which can limit direct control over psychoacoustic parameters critical to spatial perception. To address this, we introduce STASE, a system that leverages a Large Language Model (LLM) as an agent to interpret spatial cues from text. A key feature of STASE is the decoupling of semantic interpretation from a separate, physics-based spatial rendering engine, which facilitates interpretable and user-controllable spatial reasoning. The LLM processes prompts through two main pathways: (i) Description Prompts, for direct mapping of explicit spatial information (e.g., “place the lead guitar at 45{\deg} azimuth, 10 m distance”), and (ii) Abstract Prompts, where a Retrieval-Augmented Generation (RAG) module retrieves relevant spatial templates to inform the rendering. This paper details the STASE workflow, discusses implementation considerations, and highlights current challenges in evaluating generative spatial audio.

Method: Evolutionary Noise Jailbreak (ENJ) employs genetic algorithms with population initialization, crossover fusion, and probabilistic mutation to iteratively evolve audio samples that fuse malicious instructions with background noise.

Result: Extensive experiments on multiple mainstream speech models show ENJ’s attack effectiveness is significantly superior to existing baseline methods.

Conclusion: This research reveals noise’s dual role in speech security and provides critical insights for model security defense in complex acoustic environments.

Abstract: The widespread application of Large Speech Models (LSMs) has made their security risks increasingly prominent. Traditional speech adversarial attack methods face challenges in balancing effectiveness and stealth. This paper proposes Evolutionary Noise Jailbreak (ENJ), which utilizes a genetic algorithm to transform environmental noise from a passive interference into an actively optimizable attack carrier for jailbreaking LSMs. Through operations such as population initialization, crossover fusion, and probabilistic mutation, this method iteratively evolves a series of audio samples that fuse malicious instructions with background noise. These samples sound like harmless noise to humans but can induce the model to parse and execute harmful commands. Extensive experiments on multiple mainstream speech models show that ENJ’s attack effectiveness is significantly superior to existing baseline methods. This research reveals the dual role of noise in speech security and provides new critical insights for model security defense in complex acoustic environments.

Method: Uses entropy of device posterior probabilities from an auxiliary domain classifier as proxy for domain invariance, creating curriculum that starts with high-entropy (domain-invariant) samples and gradually adds low-entropy (domain-specific) ones.

Result: Experimental results on multiple DCASE 2024 ASC baselines show effective mitigation of domain shift, especially under limited labeled data conditions.

Conclusion: The strategy is architecture-agnostic, introduces no inference overhead, and provides practical solution for domain shift that can be easily integrated into existing ASC systems.

Abstract: Acoustic Scene Classification (ASC) faces challenges in generalizing across recording devices, particularly when labeled data is limited. The DCASE 2024 Challenge Task 1 highlights this issue by requiring models to learn from small labeled subsets recorded on a few devices. These models need to then generalize to recordings from previously unseen devices under strict complexity constraints. While techniques such as data augmentation and the use of pre-trained models are well-established for improving model generalization, optimizing the training strategy represents a complementary yet less-explored path that introduces no additional architectural complexity or inference overhead. Among various training strategies, curriculum learning offers a promising paradigm by structuring the learning process from easier to harder examples. In this work, we propose an entropy-guided curriculum learning strategy to address the domain shift problem in data-efficient ASC. Specifically, we quantify the uncertainty of device domain predictions for each training sample by computing the Shannon entropy of the device posterior probabilities estimated by an auxiliary domain classifier. Using entropy as a proxy for domain invariance, the curriculum begins with high-entropy samples and gradually incorporates low-entropy, domain-specific ones to facilitate the learning of generalizable representations. Experimental results on multiple DCASE 2024 ASC baselines demonstrate that our strategy effectively mitigates domain shift, particularly under limited labeled data conditions. Our strategy is architecture-agnostic and introduces no additional inference cost, making it easily integrable into existing ASC baselines and offering a practical solution to domain shift.

Method: Uses a multi-agent system with specialist agents for interpretation, requirement derivation, and output validation, plus a manager agent for tool selection, sequencing, user interaction, and state maintenance. Supports local deployment with quantization or cloud access via HFApi.

Result: A flexible framework that enables intermodal interaction across text, symbolic notation, visualization, and audio, supporting analysis-synthesis-render loops and cross-format constraints.

Conclusion: WeaveMuse successfully democratizes MIR tools through interchangeable open-source models, flexible memory management, and reproducible deployment, making advanced music AI accessible to diverse users and hardware configurations.

Abstract: Agentic AI has been standardized in industry as a practical paradigm for coordinating specialized models and tools to solve complex multimodal tasks. In this work, we present WeaveMuse, a multi-agent system for music understanding, symbolic composition, and audio synthesis. Each specialist agent interprets user requests, derives machine-actionable requirements (modalities, formats, constraints), and validates its own outputs, while a manager agent selects and sequences tools, mediates user interaction, and maintains state across turns. The system is extendable and deployable either locally, using quantization and inference strategies to fit diverse hardware budgets, or via the HFApi to preserve free community access to open models. Beyond out-of-the-box use, the system emphasizes controllability and adaptation through constraint schemas, structured decoding, policy-based inference, and parameter-efficient adapters or distilled variants that tailor models to MIR tasks. A central design goal is to facilitate intermodal interaction across text, symbolic notation and visualization, and audio, enabling analysis-synthesis-render loops and addressing cross-format constraints. The framework aims to democratize, implement, and make accessible MIR tools by supporting interchangeable open-source models of various sizes, flexible memory management, and reproducible deployment paths.

Method: Evaluated Temporal Convolutional Network (TCN) and transformer-based Beat This! model on Carnatic Music Rhythm dataset, replicating DBN baseline setup. Used fine-tuning and musically informed parameters for adaptation.

Result: Off-the-shelf deep learning models didn’t always outperform DBN, but with transfer learning (fine-tuning on Carnatic data), their performance improved substantially, matching or surpassing the baseline.

Conclusion: State-of-the-art deep learning models can be effectively adapted to underrepresented musical traditions through transfer learning, enabling more inclusive and broadly applicable meter tracking systems.

Abstract: Beat and downbeat tracking, jointly referred to as Meter Tracking, is a fundamental task in Music Information Retrieval (MIR). Deep learning models have far surpassed traditional signal processing and classical machine learning approaches in this domain, particularly for Western (Eurogenetic) genres, where large annotated datasets are widely available. These systems, however, perform less reliably on underrepresented musical traditions. Carnatic music, a rich tradition from the Indian subcontinent, is renowned for its rhythmic intricacy and unique metrical structures (t=alas). The most notable prior work on meter tracking in this context employed probabilistic Dynamic Bayesian Networks (DBNs). The performance of state-of-the-art (SOTA) deep learning models on Carnatic music, however, remains largely unexplored. In this study, we evaluate two models for meter tracking in Carnatic music: the Temporal Convolutional Network (TCN), a lightweight architecture that has been successfully adapted for Latin rhythms, and Beat This!, a transformer-based model designed for broad stylistic coverage without the need for post-processing. Replicating the experimental setup of the DBN baseline on the Carnatic Music Rhythm (CMR$_f$) dataset, we systematically assess the performance of these models in a directly comparable setting. We further investigate adaptation strategies, including fine-tuning the models on Carnatic data and the use of musically informed parameters. Results show that while off-the-shelf models do not always outperform the DBN, their performance improves substantially with transfer learning, matching or surpassing the baseline. These findings indicate that SOTA deep learning models can be effectively adapted to underrepresented traditions, paving the way for more inclusive and broadly applicable meter tracking systems.

Method: Three complementary techniques: (1) Latent Representation Fusion for integrating semantic/contextual features into encoder latent space, (2) Global Semantic-Contextual Supervision for temporal consistency, and (3) Temporally Aligned Contextual Supervision for fine-grained token-level alignment.

Result: Achieves SOTA performance on LibriSpeech, surpassing EnCodec, SpeechTokenizer, and DAC in transcription accuracy, perceptual quality, intelligibility, and speaker similarity.

Conclusion: FuseCodec demonstrates effective contextually and semantically guided tokenization for speech processing and downstream tasks, with applications in zero-shot speech synthesis.

Abstract: Speech tokenization enables discrete representation and facilitates speech language modeling. However, existing neural codecs capture low-level acoustic features, overlooking the semantic and contextual cues inherent to human speech. While recent efforts introduced semantic representations from self-supervised speech models or incorporated contextual representations from pre-trained language models, challenges remain in aligning and unifying the semantic and contextual representations. We introduce FuseCodec, which unifies acoustic, semantic, and contextual representations through strong cross-modal alignment and globally informed supervision. We propose three complementary techniques: (i) Latent Representation Fusion, integrating semantic and contextual features directly into the encoder latent space for robust and unified representation learning; (ii) Global Semantic-Contextual Supervision, supervising discrete tokens with globally pooled and broadcasted representations to enhance temporal consistency and cross-modal alignment; and (iii) Temporally Aligned Contextual Supervision, strengthening alignment by dynamically matching contextual and speech tokens within a local window for fine-grained token-level supervision. We further introduce FuseCodec-TTS, demonstrating our methodology’s applicability to zero-shot speech synthesis. Empirically, FuseCodec achieves state-of-the-art performance in LibriSpeech, surpassing EnCodec, SpeechTokenizer, and DAC in transcription accuracy, perceptual quality, intelligibility, and speaker similarity. Results highlight the effectiveness of contextually and semantically guided tokenization for speech tokenization and downstream tasks. Code and pretrained models are available at https://github.com/mubtasimahasan/FuseCodec.

Method: Combines novel tempogram-based features capturing layered rhythmic patterns with multi-criteria feature selection, validated against state-of-the-art pre-trained audio embeddings (MERT and CLAP).

Result: Both feature space and embedding representations converge to 19-23 natural acoustic families compared to prescribed 35, showing consistent evidence of significant overspecification.

Conclusion: EDM taxonomy is overspecified by approximately one-third, with natural acoustic structure revealing fewer genuine categories than industry-defined classifications.

Abstract: Electronic Dance Music (EDM) classification typically relies on industry-defined taxonomies with numerous subgenres, yet the acoustic basis for these distinctions remains unclear. Current approaches use supervised learning with prescribed genre labels, assuming their validity without systematic evaluation. In this paper, we propose an unsupervised approach to discover the natural acoustic structure of EDM independent of commercial labels. Our method combines novel tempogram-based features capturing EDM’s layered rhythmic patterns with multi-criteria feature selection. To validate that our findings reflect genuine acoustic structure rather than methodological artifacts, we compare our results against state-of-the-art pre-trained audio embeddings (MERT and CLAP). Both our feature space and embedding representations converge to 19-23 natural acoustic families compared to the prescribed 35, providing consistent evidence of significant overspecification in current EDM taxonomy by approximately one-third.

Method: Combines traditional signal processing with denoising diffusion models (WaveGrad and DiffWave) to create augmented datasets, then fine-tunes a Wav2Vec 2.0-based classifier on multimodal and multichannel heart sound datasets.

Result: Achieved state-of-the-art performance: 92.48% accuracy on CinC 2016 single-channel PCG, 93.14% accuracy on synchronized PCG-ECG, and 77.13% accuracy on multichannel PCG wearable vest dataset with corresponding high sensitivity, specificity, and MCC scores.

Conclusion: Transformer-based models supported by augmented datasets are highly effective for CVD detection, demonstrating strong potential to advance multimodal and multichannel heart sound classification for early disease detection.

Abstract: Cardiovascular diseases (CVDs) are the leading cause of death worldwide, accounting for approximately 17.9 million deaths each year. Early detection is critical, creating a demand for accurate and inexpensive pre-screening methods. Deep learning has recently been applied to classify abnormal heart sounds indicative of CVDs using synchronised phonocardiogram (PCG) and electrocardiogram (ECG) signals, as well as multichannel PCG (mPCG). However, state-of-the-art architectures remain underutilised due to the limited availability of synchronised and multichannel datasets. Augmented datasets and pre-trained models provide a pathway to overcome these limitations, enabling transformer-based architectures to be trained effectively. This work combines traditional signal processing with denoising diffusion models, WaveGrad and DiffWave, to create an augmented dataset to fine-tune a Wav2Vec 2.0-based classifier on multimodal and multichannel heart sound datasets. The approach achieves state-of-the-art performance. On the Computing in Cardiology (CinC) 2016 dataset of single channel PCG, accuracy, unweighted average recall (UAR), sensitivity, specificity and Matthew’s correlation coefficient (MCC) reach 92.48%, 93.05%, 93.63%, 92.48%, 94.93% and 0.8283, respectively. Using the synchronised PCG and ECG signals of the training-a dataset from CinC, 93.14%, 92.21%, 94.35%, 90.10%, 95.12% and 0.8380 are achieved for accuracy, UAR, sensitivity, specificity and MCC, respectively. Using a wearable vest dataset consisting of mPCG data, the model achieves 77.13% accuracy, 74.25% UAR, 86.47% sensitivity, 62.04% specificity, and 0.5082 MCC. These results demonstrate the effectiveness of transformer-based models for CVD detection when supported by augmented datasets, highlighting their potential to advance multimodal and multichannel heart sound classification.

Method: Combines DAC compression with a Transformer masker modulated by CLAP-derived FiLM parameters to create a universal, text-driven separation model.

Result: Surpasses AudioSep in separation fidelity (SI-SDR) across six benchmarks, remains competitive in perceptual quality, and uses only 1.35 GMACs end-to-end (54x less compute than spectrogram-domain separators).

Conclusion: CodecSep enables efficient on-device universal text-driven audio separation while maintaining full bitstream compatibility and superior performance compared to existing models.

Abstract: Text-guided source separation supports flexible audio editing across media and assistive applications, but existing models like AudioSep are too compute-heavy for edge deployment. Neural audio codec (NAC) models such as CodecFormer and SDCodec are compute-efficient but limited to fixed-class separation. We introduce CodecSep, the first NAC-based model for on-device universal, text-driven separation. CodecSep combines DAC compression with a Transformer masker modulated by CLAP-derived FiLM parameters. Across six open-domain benchmarks under matched training/prompt protocols, \textbf{CodecSep} surpasses \textbf{AudioSep} in separation fidelity (SI-SDR) while remaining competitive in perceptual quality (ViSQOL) and matching or exceeding fixed-stem baselines (TDANet, CodecFormer, SDCodec). In code-stream deployments, it needs just 1.35~GMACs end-to-end – approximately $54\times$ less compute ($25\times$ architecture-only) than spectrogram-domain separators like AudioSep – while remaining fully bitstream-compatible.

Method: Developed PoolingVQ: VQVAE + spatial pooling to compress audio features via local aggregation. Uses two-stage co-attention for audio-MIDI fusion.

Result: Achieves state-of-the-art overall performance on EMOPIA and VGMIDI datasets. PoolingVQ provides measurable performance improvements.

Conclusion: Compressing audio feature sequences through local aggregation effectively reduces redundancy and enhances multimodal emotion analysis performance when combined with proper fusion techniques.

Abstract: Multimodal music emotion analysis leverages audio and MIDI modalities to enhance performance. While mainstream approaches focus on complex feature extraction networks, we posit that shortening the length of audio sequence features to mitigate redundancy, especially in contrast to MIDI’s compact representation, may effectively boost task performance. To achieve this, we developed PoolingVQ by combining Vector Quantized Variational Autoencoder (VQVAE) with spatial pooling, which directly compresses audio feature sequences through local aggregation to reduce redundancy, then devised a two-stage co-attention approach to fuse audio and MIDI information. Experimental results on the public datasets EMOPIA and VGMIDI demonstrate that our multimodal framework achieves state-of-the-art overall performance, with PoolingVQ yielding some improvement.

Method: Analyzed six pre-trained SSL models on four test corpora with layer-by-layer analysis. Compared single-layer selection vs MHFA pooling. Evaluated performance variation across corpora and SSL models, and tested score-level fusion of multiple encoders.

Result: Selecting the best single layer achieved strong results while reducing system parameters by up to 80%. Performance varied significantly based on test corpus and SSL model. Score-level fusion of multiple encoders improved OOD generalization.

Conclusion: Optimal layer selection can simplify audio deepfake detection systems without sacrificing performance. SSL pre-training strategy is crucial for generalization, and ensemble approaches through score fusion enhance robustness to out-of-domain attacks.

Abstract: Audio deepfake detection systems based on frozen pre-trained self-supervised learning (SSL) encoders show a high level of performance when combined with layer-weighted pooling methods, such as multi-head factorized attentive pooling (MHFA). However, they still struggle to generalize to out-of-domain (OOD) conditions. We tackle this problem by studying the behavior of six different pre-trained SSLs, on four different test corpora. We perform a layer-by-layer analysis to determine which layers contribute most. Next, we study the pooling head, comparing a strategy based on a single layer with automatic selection via MHFA. We observed that selecting the best layer gave very good results, while reducing system parameters by up to 80%. A wide variation in performance as a function of test corpus and SSL model is also observed, showing that the pre-training strategy of the encoder plays a role. Finally, score-level fusion of several encoders improved generalization to OOD attacks.

Method: Leverages universal frequential perturbations generated through black-box knowledge and few-shot training on small user data. Uses highly malleable frequency-domain noise patches for real-time, lightweight protection.

Result: Achieves 50-200x memory efficiency (as low as 0.004GB) and 3-7000x runtime efficiency (real-time coefficient of 0.004) compared to state-of-the-art methods. Effectively defends against both vanilla and adaptive voice deepfake attacks across multiple TTS and ASV models while preserving speech quality.

Conclusion: Enkidu provides an effective, practical solution for defending against personalized voice deepfake threats with superior efficiency, strong generalization, and real-time performance while maintaining perceptual quality and speech intelligibility.

Abstract: The rapid advancement of voice deepfake technologies has raised serious concerns about user audio privacy, as attackers increasingly exploit publicly available voice data to generate convincing fake audio for malicious purposes such as identity theft, financial fraud, and misinformation campaigns. While existing defense methods offer partial protection, they face critical limitations, including weak adaptability to unseen user data, poor scalability to long audio, rigid reliance on white-box knowledge, and high computational and temporal costs during the encryption process. To address these challenges and defend against personalized voice deepfake threats, we propose Enkidu, a novel user-oriented privacy-preserving framework that leverages universal frequential perturbations generated through black-box knowledge and few-shot training on a small amount of user data. These highly malleable frequency-domain noise patches enable real-time, lightweight protection with strong generalization across variable-length audio and robust resistance to voice deepfake attacks, all while preserving perceptual quality and speech intelligibility. Notably, Enkidu achieves over 50 to 200 times processing memory efficiency (as low as 0.004 gigabytes) and 3 to 7000 times runtime efficiency (real-time coefficient as low as 0.004) compared to six state-of-the-art countermeasures. Extensive experiments across six mainstream text-to-speech models and five cutting-edge automated speaker verification models demonstrate the effectiveness, transferability, and practicality of Enkidu in defending against both vanilla and adaptive voice deepfake attacks. Our code is currently available.

Method: A deep learning based optimization algorithm that cognitively adapts sensing signals to maximize signal magnitude within available music bandwidth while minimizing frequency distortion that affects music playback quality.

Result: Respiration monitoring and gesture recognition with CoPlay achieved similar accuracy as no-concurrent-music scenarios, outperforming traditional clipping/down-scaling methods. Music quality was preserved without degradation.

Conclusion: CoPlay successfully enables concurrent acoustic sensing and music playback without compromising either functionality, making acoustic sensing practical for real-world applications where speakers are used for multiple purposes simultaneously.

Abstract: Acoustic sensing manifests great potential in various applications that encompass health monitoring, gesture interface and imaging by leveraging the speakers and microphones on smart devices. However, in ongoing research and development in acoustic sensing, one problem is often overlooked: the same speaker, when used concurrently for sensing and other traditional applications (like playing music), could cause interference in both making it impractical to use in the real world. The strong ultrasonic sensing signals mixed with music would overload the speaker’s mixer. To confront this issue of overloaded signals, current solutions are clipping or down-scaling, both of which affect the music playback quality and also sensing range and accuracy. To address this challenge, we propose CoPlay, a deep learning based optimization algorithm to cognitively adapt the sensing signal. It can 1) maximize the sensing signal magnitude within the available bandwidth left by the concurrent music to optimize sensing range and accuracy and 2) minimize any consequential frequency distortion that can affect music playback. In this work, we design a deep learning model and test it on common types of sensing signals (sine wave or Frequency Modulated Continuous Wave FMCW) as inputs with various agnostic concurrent music and speech. First, we evaluated the model performance to show the quality of the generated signals. Then we conducted field studies of downstream acoustic sensing tasks in the real world. A study with 12 users proved that respiration monitoring and gesture recognition using our adapted signal achieve similar accuracy as no-concurrent-music scenarios, while clipping or down-scaling manifests worse accuracy. A qualitative study also manifests that the music play quality is not degraded, unlike traditional clipping or down-scaling methods.

Method: A bio-inspired acoustic microstructure that attaches to smartphone in-line mics of wired earphones, coupled with an end-to-end neural network for real-time audio processing on mobile devices.

Result: Achieves 5.0 dB signal quality improvement when focusing on a 30° angular region, with two-microphone system outperforming conventional 5-microphone arrays.

Conclusion: SonicSieve demonstrates that passive acoustic microstructures combined with neural processing can provide superior directional speech extraction capabilities on smartphones with minimal hardware requirements.

Abstract: Imagine placing your smartphone on a table in a noisy restaurant and clearly capturing the voices of friends seated around you, or recording a lecturer’s voice with clarity in a reverberant auditorium. We introduce SonicSieve, the first intelligent directional speech extraction system for smartphones using a bio-inspired acoustic microstructure. Our passive design embeds directional cues onto incoming speech without any additional electronics. It attaches to the in-line mic of low-cost wired earphones which can be attached to smartphones. We present an end-to-end neural network that processes the raw audio mixtures in real-time on mobile devices. Our results show that SonicSieve achieves a signal quality improvement of 5.0 dB when focusing on a 30{\deg} angular region. Additionally, the performance of our system based on only two microphones exceeds that of conventional 5-microphone arrays.

Method: Used a deep CNN with ESC-50 dataset (2,000 labeled environmental audio recordings) and end-to-end deep learning pipeline to evaluate multiple features: mel-spectrograms, MFCCs, cyclic tempograms, STFT chromagrams, CQT chromagrams, and CENS chromagrams.

Result: Mel-scaled spectrograms and MFCCs performed significantly better than other features across all metrics (accuracy, precision, recall, F1 score) for both category and class level audio classification.

Conclusion: Mel-spectrograms and MFCCs are the most effective audio features for CNN-based environmental sound classification tasks, outperforming other spectral and rhythm features.

Abstract: Next to decision tree and k-nearest neighbours algorithms deep convolutional neural networks (CNNs) are widely used to classify audio data in many domains like music, speech or environmental sounds. To train a specific CNN various spectral and rhythm features like mel-scaled spectrograms, mel-frequency cepstral coefficients (MFCC), cyclic tempograms, short-time Fourier transform (STFT) chromagrams, constant-Q transform (CQT) chromagrams and chroma energy normalized statistics (CENS) chromagrams can be used as digital image input data for the neural network. The performance of these spectral and rhythm features for audio category level as well as audio class level classification is investigated in detail with a deep CNN and the ESC-50 dataset with 2,000 labeled environmental audio recordings using an end-to-end deep learning pipeline. The evaluated metrics accuracy, precision, recall and F1 score for multiclass classification clearly show that the mel-scaled spectrograms and the mel-frequency cepstral coefficients (MFCC) perform significantly better then the other spectral and rhythm features investigated in this research for audio classification tasks using deep CNNs.

Method: Interdisciplinary review analyzing common evaluation targets, methodologies, and metrics for assessing both system output and model use in generative music systems.

Result: Identifies benefits and limitations of different evaluation approaches including subjective vs objective, qualitative vs quantitative, and empirical vs computational methods across multiple disciplinary perspectives.

Conclusion: Provides a comprehensive framework for understanding and selecting appropriate evaluation methods for generative music systems, highlighting the need for interdisciplinary approaches that consider musicological, engineering, and HCI perspectives.

Abstract: Research on generative systems in music has seen considerable attention and growth in recent years. A variety of attempts have been made to systematically evaluate such systems. We present an interdisciplinary review of the common evaluation targets, methodologies, and metrics for the evaluation of both system output and model use, covering subjective and objective approaches, qualitative and quantitative approaches, as well as empirical and computational methods. We examine the benefits and limitations of these approaches from a musicological, an engineering, and an HCI perspective.

Method: The method decomposes facial images into non-overlapping segments to create multi-granular representations, uses multi-task learning for speaker attributes at both visual and acoustic domains, and employs multi-view training with various visual perspectives paired with identical speech recordings.

Result: Extensive subjective and objective evaluations confirm that the approach substantially enhances face-voice congruence and synthesis stability compared to existing methods.

Conclusion: The proposed approach successfully addresses limitations of existing face-to-voice synthesis by preserving fine-grained speaker attributes and improving training efficiency through integrated multi-granular representation and multi-view training strategies.

Abstract: For individuals who have experienced traumatic events such as strokes, speech may no longer be a viable means of communication. While text-to-speech (TTS) can be used as a communication aid since it generates synthetic speech, it fails to preserve the user’s own voice. As such, face-to-voice (FTV) synthesis, which derives corresponding voices from facial images, provides a promising alternative. However, existing methods rely on pre-trained visual encoders, and finetune them to align with speech embeddings, which strips fine-grained information from facial inputs such as gender or ethnicity, despite their known correlation with vocal traits. Moreover, these pipelines are multi-stage, which requires separate training of multiple components, thus leading to training inefficiency. To address these limitations, we utilize fine-grained facial attribute modeling by decomposing facial images into non-overlapping segments and progressively integrating them into a multi-granular representation. This representation is further refined through multi-task learning of speaker attributes such as gender and ethnicity at both the visual and acoustic domains. Moreover, to improve alignment robustness, we adopt a multi-view training strategy by pairing various visual perspectives of a speaker in terms of different angles and lighting conditions, with identical speech recordings. Extensive subjective and objective evaluations confirm that our approach substantially enhances face-voice congruence and synthesis stability.

Method: Used a deep CNN to analyze various audio feature representations including mel-spectrograms, MFCCs, cyclic tempograms, STFT chromagrams, CQT chromagrams, and CENS chromagrams on the ESC-50 dataset with 2,000 labeled environmental audio recordings.

Result: Mel-scaled spectrograms and MFCCs performed significantly better than other spectral and rhythm features for audio classification using deep CNNs.

Conclusion: Mel-spectrograms and MFCCs are the most effective feature representations for audio classification tasks when using deep convolutional neural networks.

Abstract: Convolutional neural networks (CNNs) are widely used in computer vision. They can be used not only for conventional digital image material to recognize patterns, but also for feature extraction from digital imagery representing spectral and rhythm features extracted from time-domain digital audio signals for the acoustic classification of sounds. Different spectral and rhythm feature representations like mel-scaled spectrograms, mel-frequency cepstral coefficients (MFCCs), cyclic tempograms, short-time Fourier transform (STFT) chromagrams, constant-Q transform (CQT) chromagrams and chroma energy normalized statistics (CENS) chromagrams are investigated in terms of the audio classification performance using a deep convolutional neural network. It can be clearly shown that the mel-scaled spectrograms and the mel-frequency cepstral coefficients (MFCCs) perform significantly better than the other spectral and rhythm features investigated in this research for audio classification tasks using deep CNNs. The experiments were carried out with the aid of the ESC-50 dataset with 2,000 labeled environmental audio recordings.

Method: Workshop format with 9 accepted papers covering novel inductive biases (e.g., language), diffusion models for DRL, 3D-aware disentanglement, and applications in specialized domains like autonomous driving and EEG analysis.

Result: The workshop successfully evaluated DRL methods in practical applications, exploring advancements in model robustness, interpretability, and generalization across various real-world domains.

Conclusion: DRL4Real demonstrated significant progress in applying disentangled representation learning to realistic scenarios, with promising methodologies emerging for controllable generation in specialized practical applications.

Abstract: This paper reviews the 1st International Workshop on Disentangled Representation Learning for Controllable Generation (DRL4Real), held in conjunction with ICCV 2025. The workshop aimed to bridge the gap between the theoretical promise of Disentangled Representation Learning (DRL) and its application in realistic scenarios, moving beyond synthetic benchmarks. DRL4Real focused on evaluating DRL methods in practical applications such as controllable generation, exploring advancements in model robustness, interpretability, and generalization. The workshop accepted 9 papers covering a broad range of topics, including the integration of novel inductive biases (e.g., language), the application of diffusion models to DRL, 3D-aware disentanglement, and the expansion of DRL into specialized domains like autonomous driving and EEG analysis. This summary details the workshop’s objectives, the themes of the accepted papers, and provides an overview of the methodologies proposed by the authors.

Method: Two-phase data-driven approach using Moment-DeepRitz Method that is robust to noisy data and adaptable to rough potentials and oscillatory rotations.

Result: The method demonstrates effectiveness through several numerical experiments.

Conclusion: The proposed Moment-DeepRitz Method provides an effective solution for learning drift decompositions in complex systems with conservative-dissipative dynamics.

Abstract: Conservative-dissipative dynamics are ubiquitous across a variety of complex open systems. We propose a data-driven two-phase method, the Moment-DeepRitz Method, for learning drift decompositions in generalized diffusion systems involving conservative-dissipative dynamics. The method is robust to noisy data, adaptable to rough potentials and oscillatory rotations. We demonstrate its effectiveness through several numerical experiments.

Method: Proposed SOH-KLSTM framework integrates Kolmogorov-Arnold Network (KAN) with LSTM candidate cell state. This hybrid approach combines LSTM’s ability to learn long-term dependencies with KAN’s non-linear approximation capabilities to capture complex battery degradation behaviors.

Result: The framework is designed to effectively capture complex degradation behaviors in lithium batteries, though specific performance metrics are not provided in the abstract.

Conclusion: The proposed SOH-KLSTM framework represents an advanced approach for lithium battery health monitoring that addresses limitations of conventional SOH estimation techniques by leveraging the complementary strengths of LSTM and KAN networks.

Abstract: Accurate and reliable State Of Health (SOH) estimation for Lithium (Li) batteries is critical to ensure the longevity, safety, and optimal performance of applications like electric vehicles, unmanned aerial vehicles, consumer electronics, and renewable energy storage systems. Conventional SOH estimation techniques fail to represent the non-linear and temporal aspects of battery degradation effectively. In this study, we propose a novel SOH prediction framework (SOH-KLSTM) using Kolmogorov-Arnold Network (KAN)-Integrated Candidate Cell State in LSTM for Li batteries Health Monitoring. This hybrid approach combines the ability of LSTM to learn long-term dependencies for accurate time series predictions with KAN’s non-linear approximation capabilities to effectively capture complex degradation behaviors in Lithium batteries.

Method: Testing and comparing MvSR implementations in Operon, PySR, phy-SO, and eggp on different real-world datasets to evaluate their performance in generating accurate and sparse parametric functions.

Result: All implementations often achieve good accuracy with solutions containing few free parameters, but certain features enable more frequent generation of better models.

Conclusion: Provides guidelines for future MvSR developments based on the comparative analysis of different implementations.

Abstract: Describing the world behavior through mathematical functions help scientists to achieve a better understanding of the inner mechanisms of different phenomena. Traditionally, this is done by deriving new equations from first principles and careful observations. A modern alternative is to automate part of this process with symbolic regression (SR). The SR algorithms search for a function that adequately fits the observed data while trying to enforce sparsity, in the hopes of generating an interpretable equation. A particularly interesting extension to these algorithms is the Multi-view Symbolic Regression (MvSR). It searches for a parametric function capable of describing multiple datasets generated by the same phenomena, which helps to mitigate the common problems of overfitting and data scarcity. Recently, multiple implementations added support to MvSR with small differences between them. In this paper, we test and compare MvSR as supported in Operon, PySR, phy-SO, and eggp, in different real-world datasets. We show that they all often achieve good accuracy while proposing solutions with only few free parameters. However, we find that certain features enable a more frequent generation of better models. We conclude by providing guidelines for future MvSR developments.

Method: CNN-Transformer ensemble model using multimodal data (structured data, trajectory sequences, image features) with maneuver detection via sliding window and histogram-based methods on a high-quality constructed dataset.

Result: Developed the first model linking trajectories to voice commands, enabling accurate prediction of command time offset and duration parameters with generalizable and interpretable performance.

Conclusion: The framework provides practical value for workload assessment, staffing, and scheduling, and supports intelligent command generation in air traffic control systems.

Abstract: Air traffic controllers (ATCOs) issue high-intensity voice commands in dense airspace, where accurate workload modeling is critical for safety and efficiency. This paper proposes a multimodal deep learning framework that integrates structured data, trajectory sequences, and image features to estimate two key parameters in the ATCO command lifecycle: the time offset between a command and the resulting aircraft maneuver, and the command duration. A high-quality dataset was constructed, with maneuver points detected using sliding window and histogram-based methods. A CNN-Transformer ensemble model was developed for accurate, generalizable, and interpretable predictions. By linking trajectories to voice commands, this work offers the first model of its kind to support intelligent command generation and provides practical value for workload assessment, staffing, and scheduling.

Method: Proposes Zero Inflation Diffusion Framework (ZIDF) that integrates: 1) Gaussian perturbation for smoothing zero-inflated distributions, 2) Transformer-based prediction for capturing temporal patterns, and 3) diffusion-based denoising to restore original data structure.

Result: ZIDF demonstrates significant performance improvements over state-of-the-art precipitation forecasting models, achieving up to 56.7% reduction in MSE and 21.1% reduction in MAE relative to baseline Non-stationary Transformer.

Conclusion: ZIDF robustly handles sparse time series data and shows potential generalizability to other domains where zero inflation is a key challenge.

Abstract: Zero-inflated data pose significant challenges in precipitation forecasting due to the predominance of zeros with sparse non-zero events. To address this, we propose the Zero Inflation Diffusion Framework (ZIDF), which integrates Gaussian perturbation for smoothing zero-inflated distributions, Transformer-based prediction for capturing temporal patterns, and diffusion-based denoising to restore the original data structure. In our experiments, we use observational precipitation data collected from South Australia along with synthetically generated zero-inflated data. Results show that ZIDF demonstrates significant performance improvements over multiple state-of-the-art precipitation forecasting models, achieving up to 56.7% reduction in MSE and 21.1% reduction in MAE relative to the baseline Non-stationary Transformer. These findings highlight ZIDF’s ability to robustly handle sparse time series data and suggest its potential generalizability to other domains where zero inflation is a key challenge.

Method: Uses server-side dynamic model exchange strategy that alternates between aggregation rounds (normal federated learning) and exchange rounds (clustering and exchanging local models based on distance measures to enable cross-domain learning).

Result: Achieves 1.6X better mean average precision in challenging domains like rainy conditions while requiring only 0.8X the computational resources compared to baseline methods.

Conclusion: FEDEXCHANGE effectively bridges domain gaps in federated object detection without imposing additional computational burden on clients, making it suitable for real-world edge device deployment.

Abstract: Federated Object Detection (FOD) enables clients to collaboratively train a global object detection model without accessing their local data from diverse domains. However, significant variations in environment, weather, and other domain specific factors hinder performance, making cross domain generalization a key challenge. Existing FOD methods often overlook the hardware constraints of edge devices and introduce local training regularizations that incur high computational costs, limiting real-world applicability. In this paper, we propose FEDEXCHANGE, a novel FOD framework that bridges domain gaps without introducing additional local computational overhead. FEDEXCHANGE employs a server side dynamic model exchange strategy that enables each client to gain insights from other clients’ domain data without direct data sharing. Specifically, FEDEXCHANGE allows the server to alternate between model aggregation and model exchange. During aggregation rounds, the server aggregates all local models as usual. In exchange rounds, FEDEXCHANGE clusters and exchanges local models based on distance measures, allowing local models to learn from a variety of domains. As all operations are performed on the server side, clients can achieve improved cross domain utility without any additional computational overhead. Extensive evaluations demonstrate that FEDEXCHANGE enhances FOD performance, achieving 1.6X better mean average precision in challenging domains, such as rainy conditions, while requiring only 0.8X the computational resources compared to baseline methods.

Method: Formulates retrosynthesis as worst-path optimization in tree-structured MDPs, introduces InterRetro that learns value functions for worst-path outcomes and improves policy through self-imitation with high-advantage decisions.

Result: Achieves 100% success on Retro*-190 benchmark, shortens synthetic routes by 4.9%, and shows promising performance with only 10% of training data.

Conclusion: Represents a significant advance in computational retrosynthesis planning by addressing the critical weakest-link vulnerability through worst-path optimization framework.

Abstract: Retrosynthesis planning aims to decompose target molecules into available building blocks, forming a synthesis tree where each internal node represents an intermediate compound and each leaf ideally corresponds to a purchasable reactant. However, this tree becomes invalid if any leaf node is not a valid building block, making the planning process vulnerable to the “weakest link” in the synthetic route. Existing methods often optimise for average performance across branches, failing to account for this worst-case sensitivity. In this paper, we reframe retrosynthesis as a worst-path optimisation problem within tree-structured Markov Decision Processes (MDPs). We prove that this formulation admits a unique optimal solution and offers monotonic improvement guarantees. Building on this insight, we introduce Interactive Retrosynthesis Planning (InterRetro), a method that interacts with the tree MDP, learns a value function for worst-path outcomes, and improves its policy through self-imitation, preferentially reinforcing past decisions with high estimated advantage. Empirically, InterRetro achieves state-of-the-art results, solving 100% of targets on the Retro*-190 benchmark, shortening synthetic routes by 4.9%, and achieving promising performance using only 10% of the training data - representing a significant advance in computational retrosynthesis planning.

Method: Proposes AttnBoost framework that integrates a lightweight attention mechanism into the boosting process to dynamically adjust feature importance during each boosting round, focusing on high-impact variables like promotions, pricing, and seasonal trends.

Result: Outperforms standard machine learning and deep tabular models on large-scale retail sales dataset. Ablation study confirms the attention module helps mitigate overfitting and improves interpretability.

Conclusion: Attention-guided boosting represents a promising direction for interpretable and scalable AI in real-world forecasting applications, providing both predictive accuracy and actionable insights for supply chain managers.

Abstract: Forecasting product demand in retail supply chains presents a complex challenge due to noisy, heterogeneous features and rapidly shifting consumer behavior. While traditional gradient boosting decision trees (GBDT) offer strong predictive performance on structured data, they often lack adaptive mechanisms to identify and emphasize the most relevant features under changing conditions. In this work, we propose AttnBoost, an interpretable learning framework that integrates feature-level attention into the boosting process to enhance both predictive accuracy and explainability. Specifically, the model dynamically adjusts feature importance during each boosting round via a lightweight attention mechanism, allowing it to focus on high-impact variables such as promotions, pricing, and seasonal trends. We evaluate AttnBoost on a large-scale retail sales dataset and demonstrate that it outperforms standard machine learning and deep tabular models, while also providing actionable insights for supply chain managers. An ablation study confirms the utility of the attention module in mitigating overfitting and improving interpretability. Our results suggest that attention-guided boosting represents a promising direction for interpretable and scalable AI in real-world forecasting applications.

Method: Introduced a selective feedback mechanism and conducted experiments comparing quality-filtered conditions vs unfiltered and random-filter controls across five generations using a Gemma 2B model on complex summarization tasks

Result: Quality-filtered condition improved by 6.6% in ROUGE-L F1 score, while unfiltered control degraded by 3.5% and random-filter control degraded by 4.2%. The selective pressure reversed model collapse and induced statistically significant performance improvement

Conclusion: Systemic resilience can be an emergent property of LLMs under simple selection pressure, suggesting a powerful and scalable principle for developing safer and more robust AI systems

Abstract: The stability of recursively trained large language models (LLMs) is a foundational problem for AI safety. Prevailing theory predicts model collapse, a progressive degradation when models are trained on their own output. We challenge this narrative by introducing a selective feedback mechanism. Contrary to expectation, instead of merely slowing decay, our experiments provide strong evidence that this pressure reverses it, inducing a statistically significant performance improvement in a Gemma 2B model on a complex summarization task. We name this phenomenon the Anti-Ouroboros Effect. We contrast this with a foundational experiment using a simple classifier, where the theoretical degenerative loop was validated, highlighting the unique dynamics of high-dimensional models. Our findings establish that systemic resilience can be an emergent property of LLMs under simple selection pressure, suggesting a powerful and scalable principle for developing safer and more robust AI systems. Across five generations, a quality-filtered condition improved by 6.6% in ROUGE-L F1 score, whereas an unfiltered control degraded by 3.5% and a random-filter control degraded by 4.2%

Method: Integrates dual-memory system inspired by human cognition with adaptive exploration strategies using temperature decay and curiosity mechanisms, evaluated on 1M simulated access logs with 47.9% anomalies.

Result: Achieves 96.0% detection rate (vs 93.0% DQN, 47.1% PPO), precision 0.4776, recall 0.9996, F1-score 0.6450, mean reward 20.34 ± 44.63. Statistical tests confirm significant performance advantages.

Conclusion: LogGuardQ bridges cognitive science and RL, offering scalable adaptive learning for cybersecurity and intrusion detection with superior stability and efficiency in uncertain environments.

Abstract: Reinforcement learning (RL) has transformed sequential decision-making, but traditional algorithms like Deep Q-Networks (DQNs) and Proximal Policy Optimization (PPO) often struggle with efficient exploration, stability, and adaptability in dynamic environments. This study presents LogGuardQ (Adaptive Log Guard with Cognitive enhancement), a novel framework that integrates a dual-memory system inspired by human cognition and adaptive exploration strategies driven by temperature decay and curiosity. Evaluated on a dataset of 1,000,000 simulated access logs with 47.9% anomalies over 20,000 episodes, LogGuardQ achieves a 96.0% detection rate (versus 93.0% for DQN and 47.1% for PPO), with precision of 0.4776, recall of 0.9996, and an F1-score of 0.6450. The mean reward is 20.34 \pm 44.63 across all episodes (versus 18.80 \pm 43.98 for DQN and -0.17 \pm 23.79 for PPO), with an average of 5.0 steps per episode (constant across models). Graphical analyses, including learning curves smoothed with a Savgol filter (window=501, polynomial=2), variance trends, action distributions, and cumulative detections, demonstrate LogGuardQ’s superior stability and efficiency. Statistical tests (Mann-Whitney U) confirm significant performance advantages (e.g., p = 0.0002 vs. DQN with negligible effect size, p < 0.0001 vs. PPO with medium effect size, and p < 0.0001 for DQN vs. PPO with small effect size). By bridging cognitive science and RL, LogGuardQ offers a scalable approach to adaptive learning in uncertain environments, with potential applications in cybersecurity, intrusion detection, and decision-making under uncertainty.

Method: Uses Stackelberg game theory between task publishers (leaders) and local model owners (followers) with Nash equilibrium solution. Formulates LMO-worker interaction as multi-agent Markov decision process solved via deep reinforcement learning. Develops ASOSA algorithm to stabilize strategies.

Result: Extensive numerical experiments validate the efficacy of the proposed adaptive incentive mechanism in maximizing utilities for all participants while addressing data scarcity issues in federated learning.

Conclusion: The proposed framework successfully addresses incentive problems in federated learning through game theory and reinforcement learning, providing stable strategies that benefit task publishers, local model owners, and workers simultaneously.

Abstract: Recently, federated learning (FL) has emerged as a novel framework for distributed model training. In FL, the task publisher (TP) releases tasks, and local model owners (LMOs) use their local data to train models. Sometimes, FL suffers from the lack of training data, and thus workers are recruited for gathering data. To this end, this paper proposes an adaptive incentive mechanism from a service-oriented perspective, with the objective of maximizing the utilities of TP, LMOs and workers. Specifically, a Stackelberg game is theoretically established between the LMOs and TP, positioning TP as the leader and the LMOs as followers. An analytical Nash equilibrium solution is derived to maximize their utilities. The interaction between LMOs and workers is formulated by a multi-agent Markov decision process (MAMDP), with the optimal strategy identified via deep reinforcement learning (DRL). Additionally, an Adaptively Searching the Optimal Strategy Algorithm (ASOSA) is designed to stabilize the strategies of each participant and solve the coupling problems. Extensive numerical experiments are conducted to validate the efficacy of the proposed method.

Method: Proposes Mixture-of-Clustered-Experts (MoCE) with dual-stage routing: 1) expert group routing based on sequence-level features, 2) top-k expert activation within groups at token level. This partitions heterogeneous inputs by knowledge needs while maintaining token-level routing benefits.

Result: MoCE demonstrates consistent superiority over strong baselines across comprehensive benchmarks, showing enhanced generalization capabilities. Detailed analysis confirms the robustness and effectiveness of the approach.

Conclusion: The dual-stage routing mechanism effectively addresses MoE limitations by enabling better partitioning of heterogeneous inputs and promoting expert group specialization, while preserving computational efficiency advantages of token-level routing.

Abstract: A sparse Mixture-of-Experts (MoE) architecture has emerged as a highly scalable solution by conditionally activating sub-modules without a proportional increase in computational costs. However, improving expert specialization to enhance performance and generalization remains a challenge for MoE, especially in instruction tuning scenarios characterized by significant input heterogeneity. In this work, we propose the Mixture-of-Clustered-Experts (MoCE) to address this limitation through a dual-stage routing mechanism. The first stage in the mechanism performs expert group routing based on sequence-level features, while the second stage activates the top-$k$ experts within the group at the token level. This approach enables the effective partitioning of heterogeneous inputs based on their knowledge requirements, encouraging expert group specialization while maintaining the advantages of token-level routing. We evaluate MoCE across a comprehensive set of benchmarks, demonstrating its consistent superiority over strong baselines and its enhanced generalization capabilities. Detailed analysis further highlights the robustness and effectiveness of MoCE.

Method: Establishes a novel framework from the perspective of differential manifolds to investigate continuous attractors in artificial neural networks, analyzing links with eigenvalues of the local Jacobian matrix.

Result: Verifies compatibility with prior conclusions, demonstrates universality of singular value stratification in common classification models and datasets, and shows continuous attractors may be ubiquitous in general neural networks.

Conclusion: The proposed differential manifold framework offers a promising foundation for a general theory of continuous attractors, given the close mathematical connection between eigenvalues and singular values.

Abstract: Continuous attractors are critical for information processing in both biological and artificial neural systems, with implications for spatial navigation, memory, and deep learning optimization. However, existing research lacks a unified framework to analyze their properties across diverse dynamical systems, limiting cross-architectural generalizability. This study establishes a novel framework from the perspective of differential manifolds to investigate continuous attractors in artificial neural networks. It verifies compatibility with prior conclusions, elucidates links between continuous attractor phenomena and eigenvalues of the local Jacobian matrix, and demonstrates the universality of singular value stratification in common classification models and datasets. These findings suggest continuous attractors may be ubiquitous in general neural networks, highlighting the need for a general theory, with the proposed framework offering a promising foundation given the close mathematical connection between eigenvalues and singular values.

Method: Proposes Adaptive Preference Optimization with Utility Anchor (UAPO) framework that introduces an anchoring function to estimate uncertainties from preference data annotation. This allows training with unpaired data and enhances robustness.

Result: Experimental results show UAPO achieves competitive outcomes without strict dependency on data pairing, significantly improving data utilization efficiency.

Conclusion: UAPO paves the way for more flexible and effective preference optimization methods by addressing the limitations of traditional pairwise training approaches through utility anchoring.

Abstract: Offline preference optimization methods are efficient for large language models (LLMs) alignment. Direct Preference optimization (DPO)-like learning, one of the most popular approaches, stands out for its efficiency in reward modeling. However, these methods typically follow the convention to use Bradley-Terry (BT) reward modeling that faces several critical assumptions, including the requirement for pairwise training data, model distribution shifting, human rationality assumption, etc. To address these limitations, we propose a general framework for offline preference optimization methods, Adaptive Preference Optimization with Utility Anchor (UAPO), which introduces an anchoring function to estimate the uncertainties brought from preference data annotation. Our method enables training even in scenarios where the data is unpaired, significantly enhancing data utilization efficiency. Moreover, the anchor design makes UAPO more robust in the training process. Experimental results demonstrate that UAPO achieves competitive outcomes without the strict dependency on data pairing, paving the way for more flexible and effective preference optimization methods.

Method: Proposed a novel privacy-preserving recommender system using Federated Learning (FL) with Deep Neural Network (DNN) and engineered features from ASSISTments educational dataset. Conducted comparative analysis of federated aggregation strategies.

Result: FedProx identified as significantly more stable and effective than FedAvg for handling heterogeneous student data. Optimized federated model achieved 76.28% F1-Score, representing 82.85% of centralized XGBoost model performance.

Conclusion: Federated Learning provides a viable and robust solution to the personalization-privacy dilemma in educational platforms, enabling effective content recommendations without centralizing sensitive student data.

Abstract: The increasing digitalization of education presents unprecedented opportunities for data-driven personalization, yet it introduces significant student data privacy challenges. Conventional recommender systems rely on centralized data, a paradigm often incompatible with modern data protection regulations. A novel privacy-preserving recommender system is proposed and evaluated to address this critical issue using Federated Learning (FL). The approach utilizes a Deep Neural Network (DNN) with rich, engineered features from the large-scale ASSISTments educational dataset. A rigorous comparative analysis of federated aggregation strategies was conducted, identifying FedProx as a significantly more stable and effective method for handling heterogeneous student data than the standard FedAvg baseline. The optimized federated model achieves a high-performance F1-Score of 76.28%, corresponding to 82.85% of the performance of a powerful, centralized XGBoost model. These findings validate that a federated approach can provide highly effective content recommendations without centralizing sensitive student data. Consequently, our work presents a viable and robust solution to the personalization-privacy dilemma in modern educational platforms.

Method: Comparative benchmark of 5 FL strategies (FedAvg, FedProx, FedAdagrad, FedAdam, FedCluster) using MIMIC-IV dataset partitioned by clinical care unit to simulate non-IID environment. Applied SMOTE-Tomek for class imbalance, conducted over 50 communication rounds.

Result: FedProx consistently outperformed other methods with highest F1-Score of 0.8831 and stable convergence. FedAvg was most computationally efficient but had substantially lower predictive performance.

Conclusion: Regularization-based FL algorithms like FedProx provide more robust and effective solutions for heterogeneous and imbalanced clinical prediction tasks compared to standard or server-side adaptive methods.

Abstract: Machine learning models hold significant potential for predicting in-hospital mortality, yet data privacy constraints and the statistical heterogeneity of real-world clinical data often hamper their development. Federated Learning (FL) offers a privacy-preserving solution, but its performance under non-Independent and Identically Distributed (non-IID) and imbalanced conditions requires rigorous investigation. The study presents a comparative benchmark of five federated learning strategies: FedAvg, FedProx, FedAdagrad, FedAdam, and FedCluster for mortality prediction. Using the large-scale MIMIC-IV dataset, we simulate a realistic non-IID environment by partitioning data by clinical care unit. To address the inherent class imbalance of the task, the SMOTE-Tomek technique is applied to each client’s local training data. Our experiments, conducted over 50 communication rounds, reveal that the regularization-based strategy, FedProx, consistently outperformed other methods, achieving the highest F1-Score of 0.8831 while maintaining stable convergence. While the baseline FedAvg was the most computationally efficient, its predictive performance was substantially lower. Our findings indicate that regularization-based FL algorithms like FedProx offer a more robust and effective solution for heterogeneous and imbalanced clinical prediction tasks than standard or server-side adaptive aggregation methods. The work provides a crucial empirical benchmark for selecting appropriate FL strategies for real-world healthcare applications.

Method: Four-phase pipeline using fractal quantization, probabilistic entanglement, and dynamic diffraction chipping for holographic knowledge integration and compression.

Result: 0% forgetting (infinite improvement over baselines), 3x compression, 53% training time reduction, 67% storage savings, and supports 1,020+ updates with 1% growth per increment.

Conclusion: HKM enables “eternal” adaptation for LLMs without retraining, projecting significant cost and energy savings, with potential for multimodal and quantum extensions.

Abstract: We introduce the Holographic Knowledge Manifold (HKM), a four-phase pipeline that achieves zero catastrophic forgetting in AI knowledge representation while maintaining minimal memory growth and high efficiency. Leveraging fractal quantization, probabilistic entanglement, and dynamic diffraction chipping, HKM compresses knowledge substrates by 3x with 67% storage savings, integrates holographically at 100%, and supports over 1,020 updates with 1% growth per increment. In experiments on combined WikiText and FB15k datasets (scaled to 2,997 nodes), we demonstrate industry-leading performance: 0% forgetting (infinite improvement over GEM baselines), 3x compression, and 53% training time reduction on consumer GPU hardware. Hypothetical cost analyses project $92.4M savings over 5 years at petabyte scale, with 21.2% energy reduction and 33% lower carbon footprint. This work hypothesizes a paradigm shift for public large language models (LLMs), enabling “eternal” adaptation without retraining. Future extensions to multimodal fusion and quantum hardware could further democratize scalable AI, potentially reducing fine-tuning costs by 60-80% for models like Llama-3 or Grok-4. Code, datasets, and full results are publicly available for reproducibility.

Method: Two methods: LUT-2D uses 2D lookup tables for fine-grained gradient estimation, while LUT-1D uses more efficient 1D lookup tables for comparable accuracy with faster runtime.

Result: LUT-2D and LUT-1D improve retraining accuracy by 3.83% and 3.72% on CIFAR-10 with CNNs, and LUT-1D improves by 23.69% on ImageNet with vision transformers compared to state-of-the-art.

Conclusion: The proposed gradient computation methods for approximate multipliers significantly enhance retraining accuracy in deep learning models, with LUT-1D offering a good balance between accuracy and efficiency.

Abstract: Approximate multipliers (AppMults) are widely used in deep learning accelerators to reduce their area, delay, and power consumption. However, AppMults introduce arithmetic errors into deep learning models, necessitating a retraining process to recover accuracy. A key step in retraining is computing the gradient of the AppMult, i.e., the partial derivative of the approximate product with respect to each input operand. Existing approaches typically estimate this gradient using that of the accurate multiplier (AccMult), which can lead to suboptimal retraining results. To address this, we propose two methods to obtain more precise gradients of AppMults. The first, called LUT-2D, characterizes the AppMult gradient with 2-dimensional lookup tables (LUTs), providing fine-grained estimation and achieving the highest retraining accuracy. The second, called LUT-1D, is a compact and more efficient variant that stores gradient values in 1-dimensional LUTs, achieving comparable retraining accuracy with shorter runtime. Experimental results show that on CIFAR-10 with convolutional neural networks, our LUT-2D and LUT-1D methods improve retraining accuracy by 3.83% and 3.72% on average, respectively. On ImageNet with vision transformer models, our LUT-1D method improves retraining accuracy by 23.69% on average, compared to a state-of-the-art retraining framework.

Method: Learns to recognize which action led to a successful binary outcome by comparing it to a counterfactual action sampled from the logging policy under the same context, rather than predicting reward directly.

Result: The method consistently outperforms direct models in both synthetic experiments and real-world deployments, demonstrating practical effectiveness.

Conclusion: Counterfactual Sample Identification provides a theoretically grounded approach that effectively addresses confounding issues in contextual bandit problems and delivers superior performance compared to traditional direct reward modeling.

Abstract: In production systems, contextual bandit approaches often rely on direct reward models that take both action and context as input. However, these models can suffer from confounding, making it difficult to isolate the effect of the action from that of the context. We present \emph{Counterfactual Sample Identification}, a new approach that re-frames the problem: rather than predicting reward, it learns to recognize which action led to a successful (binary) outcome by comparing it to a counterfactual action sampled from the logging policy under the same context. The method is theoretically grounded and consistently outperforms direct models in both synthetic experiments and real-world deployments.

Method: Learns client-specific parametric UMAP embeddings, models latent pairwise distances with mixture relation markers (same/different class pairs), exchanges only variational uncertainty-aware marker statistics, maintains Dirichlet-Normal-Inverse-Gamma posteriors, and aggregates via conjugate moment matching.

Result: Achieves competitive or superior test F1 scores across eight vision datasets with non-IID label shards while communicating only small geometry summaries, with strengthened privacy through secure aggregation and optional differential privacy.

Conclusion: VGM² provides an effective geometry-centric approach to PFL that handles heterogeneity well, minimizes communication overhead, and maintains privacy, with proven stability under heterogeneous conditions through theoretical analysis of KL divergence minimization.

Abstract: Personalized federated learning (PFL) often fails under label skew and non-stationarity because a single global parameterization ignores client-specific geometry. We introduce VGM$^2$ (Variational Gaussian Mixture Manifold), a geometry-centric PFL framework that (i) learns client-specific parametric UMAP embeddings, (ii) models latent pairwise distances with mixture relation markers for same and different class pairs, and (iii) exchanges only variational, uncertainty-aware marker statistics. Each client maintains a Dirichlet-Normal-Inverse-Gamma (Dir-NIG) posterior over marker weights, means, and variances; the server aggregates via conjugate moment matching to form global priors that guide subsequent rounds. We prove that this aggregation minimizes the summed reverse Kullback-Leibler divergence from client posteriors within the conjugate family, yielding stability under heterogeneity. We further incorporate a calibration term for distance-to-similarity mapping and report communication and compute budgets. Across eight vision datasets with non-IID label shards, VGM$^2$ achieves competitive or superior test F1 scores compared to strong baselines while communicating only small geometry summaries. Privacy is strengthened through secure aggregation and optional differential privacy noise, and we provide a membership-inference stress test. Code and configurations will be released to ensure full reproducibility.

Method: Two-module framework: 1) Deep learning model fine-tuned through cross-domain transfer learning for ASD classification, 2) Three XAI techniques (saliency mapping, Grad-CAM, and SHAP) for model interpretation and identifying critical brain regions.

Result: Cross-domain transfer learning effectively addresses data scarcity in ASD research. The explainable techniques reveal diagnostic decision-making processes and identify ASD-associated brain regions that strongly align with established neurobiological evidence.

Conclusion: The proposed framework demonstrates clinical relevance by combining transfer learning for data efficiency and explainable AI for interpretable ASD diagnosis, with findings validated against neurobiological knowledge.

Abstract: Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized by atypical brain maturation. However, the adaptation of transfer learning paradigms in machine learning for ASD research remains notably limited. In this study, we propose a computer-aided diagnostic framework with two modules. This chapter presents a two-module framework combining deep learning and explainable AI for ASD diagnosis. The first module leverages a deep learning model fine-tuned through cross-domain transfer learning for ASD classification. The second module focuses on interpreting the model decisions and identifying critical brain regions. To achieve this, we employed three explainable AI (XAI) techniques: saliency mapping, Gradient-weighted Class Activation Mapping, and SHapley Additive exPlanations (SHAP) analysis. This framework demonstrates that cross-domain transfer learning can effectively address data scarcity in ASD research. In addition, by applying three established explainability techniques, the approach reveals how the model makes diagnostic decisions and identifies brain regions most associated with ASD. These findings were compared against established neurobiological evidence, highlighting strong alignment and reinforcing the clinical relevance of the proposed approach.

Method: Transforms pruning into a graph representation using GAT encoder for rich embeddings, employs fine-grained binary action spaces instead of continuous ratios, and uses Constrained Markov Decision Process with self-competition reward system to make pruning decisions under resource constraints.

Result: Extensive experiments on CIFAR-10, CIFAR-100, and ImageNet show the method consistently outperforms traditional pruning techniques, achieving state-of-the-art results with task-specific strategies that identify functionally redundant connections.

Conclusion: The graph-based AutoML framework successfully addresses limitations of traditional pruning by providing global structural understanding and data-driven channel importance learning, enabling more effective and efficient neural network compression.

Abstract: This paper presents a novel approach to neural network pruning by integrating a graph-based observation space into an AutoML framework to address the limitations of existing methods. Traditional pruning approaches often depend on hand-crafted heuristics and local optimization perspectives, which can lead to suboptimal performance and inefficient pruning strategies. Our framework transforms the pruning process by introducing a graph representation of the target neural network that captures complete topological relationships between layers and channels, replacing the limited layer-wise observation space with a global view of network structure. The core innovations include a Graph Attention Network (GAT) encoder that processes the network’s graph representation and generates a rich embedding. Additionally, for the action space we transition from continuous pruning ratios to fine-grained binary action spaces which enables the agent to learn optimal channel importance criteria directly from data, moving away from predefined scoring functions. These contributions are modelled within a Constrained Markov Decision Process (CMDP) framework, allowing the agent to make informed pruning decisions while adhering to resource constraints such as target compression rates. For this, we design a self-competition reward system that encourages the agent to outperform its previous best performance while satisfying the defined constraints. We demonstrate the effectiveness of our approach through extensive experiments on benchmark datasets including CIFAR-10, CIFAR-100, and ImageNet. The experiments show that our method consistently outperforms traditional pruning techniques, showing state-of-the-art results while learning task-specific pruning strategies that identify functionally redundant connections beyond simple weight magnitude considerations.

Method: The framework transforms raw urban event data into graph structures, integrates OpenStreetMap features, provides multiple GNN models, visualization tools, and a GUI for both professional and non-professional users.

Result: STM-Graph offers a modular and extensible platform that enables rapid experimentation, benchmarking, and integration of new mapping methods and custom models for urban computing applications.

Conclusion: The framework serves as a valuable resource for researchers and practitioners in urban computing by providing comprehensive tools for spatio-temporal data analysis and prediction using graph neural networks.

Abstract: Urban spatio-temporal data present unique challenges for predictive analytics due to their dynamic and complex nature. We introduce STM-Graph, an open-source Python framework that transforms raw spatio-temporal urban event data into graph representations suitable for Graph Neural Network (GNN) training and prediction. STM-Graph integrates diverse spatial mapping methods, urban features from OpenStreetMap, multiple GNN models, comprehensive visualization tools, and a graphical user interface (GUI) suitable for professional and non-professional users. This modular and extensible framework facilitates rapid experimentation and benchmarking. It allows integration of new mapping methods and custom models, making it a valuable resource for researchers and practitioners in urban computing. The source code of the framework and GUI are available at: https://github.com/Ahghaffari/stm_graph and https://github.com/tuminguyen/stm_graph_gui.

Method: Applied Latent Replay to diffusion models - storing only compact high-level feature representations from the model’s internal architecture rather than storing full images. This mirrors hippocampal neural activity pattern storage. Tested with five sequentially learned visual concepts.

Result: Significantly outperformed existing methods, retaining 77.59% Image Alignment on earliest concept (14% higher than baselines) while maintaining output diversity. Surprisingly, random selection of stored latent examples worked better than similarity-based strategies.

Conclusion: Latent Replay enables efficient continual learning for generative AI models, paving the way for personalized text-to-image models that can evolve with user needs without excessive computational costs.

Abstract: Continual learning – the ability to acquire knowledge incrementally without forgetting previous skills – is fundamental to natural intelligence. While the human brain excels at this, artificial neural networks struggle with “catastrophic forgetting,” where learning new tasks erases previously acquired knowledge. This challenge is particularly severe for text-to-image diffusion models, which generate images from textual prompts. Additionally, these models face “mode collapse,” where their outputs become increasingly repetitive over time. To address these challenges, we apply Latent Replay, a neuroscience-inspired approach, to diffusion models. Traditional replay methods mitigate forgetting by storing and revisiting past examples, typically requiring large collections of images. Latent Replay instead retains only compact, high-level feature representations extracted from the model’s internal architecture. This mirrors the hippocampal process of storing neural activity patterns rather than raw sensory inputs, reducing memory usage while preserving critical information. Through experiments with five sequentially learned visual concepts, we demonstrate that Latent Replay significantly outperforms existing methods in maintaining model versatility. After learning all concepts, our approach retained 77.59% Image Alignment (IA) on the earliest concept, 14% higher than baseline methods, while maintaining diverse outputs. Surprisingly, random selection of stored latent examples outperforms similarity-based strategies. Our findings suggest that Latent Replay enables efficient continual learning for generative AI models, paving the way for personalized text-to-image models that evolve with user needs without excessive computational costs.

Method: Three integrated modules: 1) Grouped Multi-Head Attention for reduced complexity via sequence grouping and local attention, 2) Dual-Scale Shared Expert structure with shallow experts for low-dimensional features and deep experts for complex semantics, 3) hierarchical Adaptive Dynamic Routing for expert selection based on feature complexity.

Result: Outperforms state-of-the-art models on multiple long-sequence benchmark datasets.

Conclusion: The proposed DASG-MoE model effectively addresses computational efficiency and long-range dependency challenges in long-sequence modeling through its integrated architecture.

Abstract: Transformer models based on the Mixture of Experts (MoE) architecture have made significant progress in long-sequence modeling, but existing models still have shortcomings in computational efficiency and the ability to capture long-range dependencies, especially in terms of the dynamic adaptability of expert resource allocation. In this paper, we propose a Dynamic Adaptive Shared Expert and Grouped Multi-Head Attention Hybrid Model (DASG-MoE) to enhance long-sequence modeling capabilities by integrating three modules. First, we employ the Grouped Multi-Head Attention (GMHA) mechanism to effectively reduce the computational complexity of long sequences. By parallel processing through sequence grouping, local sliding window attention, and feature aggregation, we address long-range dependency issues and the model’s lack of generalization for local information. Second, we design a Dual-Scale Shared Expert Structure (DSSE), where shallow experts use lightweight computations to quickly respond to low-dimensional features, while deep experts process high-dimensional complex semantics through pre-training transfer and post-training optimization, achieving a dynamic balance between efficiency and accuracy. Third, we propose a hierarchical Adaptive Dynamic Routing (ADR) mechanism that dynamically selects expert levels based on feature complexity and task requirements, and optimizes resource allocation through a local expert activation strategy. Experiments on multiple long-sequence benchmark datasets demonstrate that our DASG-MoE model outperforms state-of-the-art models.

Method: Uses two DRL agents: one for asset allocation within existing universe and another for exploring new opportunities in extended universe, dynamically balancing both objectives.

Result: Superior performance demonstrated on two real-world market datasets compared to state-of-the-art portfolio strategies and baseline methods.

Conclusion: The integrated approach of exploitation and exploration in portfolio optimization effectively adapts to evolving markets and enhances portfolio performance.

Abstract: Portfolio optimization is essential for balancing risk and return in financial decision-making. Deep Reinforcement Learning (DRL) has stood out as a cutting-edge tool for portfolio optimization that learns dynamic asset allocation using trial-and-error interactions. However, most DRL-based methods are restricted to allocating assets within a pre-defined investment universe and overlook exploring new opportunities. This study introduces an investment landscape that integrates exploiting existing assets with exploring new investment opportunities in an extended universe. The proposed approach leverages two DRL agents and dynamically balances these objectives to adapt to evolving markets while enhancing portfolio performance. One agent allocates assets within the existing universe, while another assists in exploring new opportunities in the extended universe. The effciency of the proposed methodology is determined using two real-world market data sets. The experiments demonstrate the superiority of the suggested approach against the state-of-the-art portfolio strategies and baseline methods.

Method: Proposed PoPE that eliminates the what-where confound by using polar coordinate position embeddings to separate content and position information.

Result: PoPE outperforms RoPE on diagnostic tasks, autoregressive sequence modeling in music, genomic, and natural language domains, with gains persisting across model scales (124M to 774M parameters).

Conclusion: PoPE provides superior performance and strong zero-shot length extrapolation capabilities compared to RoPE, without requiring fine-tuning or position-interpolation methods.

Abstract: The attention mechanism in a Transformer architecture matches key to query based on both content – the what – and position in a sequence – the where. We present an analysis indicating that what and where are entangled in the popular RoPE rotary position embedding. This entanglement can impair performance particularly when decisions require independent matches on these two factors. We propose an improvement to RoPE, which we call Polar Coordinate Position Embeddings or PoPE, that eliminates the what-where confound. PoPE is far superior on a diagnostic task requiring indexing solely by position or by content. On autoregressive sequence modeling in music, genomic, and natural language domains, Transformers using PoPE as the positional encoding scheme outperform baselines using RoPE with respect to evaluation loss (perplexity) and downstream task performance. On language modeling, these gains persist across model scale, from 124M to 774M parameters. Crucially, PoPE shows strong zero-shot length extrapolation capabilities, whereas RoPE’s performance degrades significantly on longer sequences at test time without fine tuning or the use of position-interpolation methods.

Method: Uses task descriptions as semantic bridges to measure proximity to known expert tasks in shared embedding space, then models target task’s LoRA parameter distribution to generate parameters for novel tasks without additional training.

Result: Extensive experiments on multiple challenging tasks confirm superior performance and remarkable adaptability of SG-LoRA in zero-shot open-world settings.

Conclusion: SG-LoRA enables real-time construction of LoRA models aligned with individual intents while providing privacy-preserving personalized model adaptation, representing a significant advancement for edge AI deployment.

Abstract: Low-Rank Adaptation (LoRA) has demonstrated strong generalization capabilities across a variety of tasks for efficiently fine-tuning AI models, especially on resource-constrained edges. However, in real-world applications, edge users often exhibit task-specific preferences that are difficult to handle with a unified model trained under a closed-world assumption, and the challenge may further increase when there are significant domain shifts between training and deployment. Meanwhile, retraining/fine-tuning models for each user is also impractical due to its cost-intensive nature and privacy concerns over raw data utilization from edges. To address these challenges, we propose Semantic-guided LoRA Parameter Generation (SG-LoRA), the first of its kind framework to efficiently produce user-specific LoRA parameters without any additional training on user tasks or access to user-specific data. Concretely, SG-LoRA uses task descriptions as the semantic bridge, measuring their proximity to a set of known expert tasks in a shared embedding space. Based on this semantic guidance, it models the target task’s LoRA parameter distribution to generate high-performing parameters for novel tasks. SG-LoRA enables the real-time construction of LoRA models aligned with individual intents by distilling knowledge from prominent LoRA experts and, meanwhile, offering a privacy-preserving solution for personalized model adaptation in a novel zero-shot open-world setting proposed in this work. Extensive experiments on multiple challenging tasks confirm the superior performance and remarkable adaptability of SG-LoRA. Code is available at https://github.com/keepgoingjkg/SG-LoRA.

Method: Generalize RBM weights to take values in abstract groups, introduce contextuality index based on group-valued holonomies along graph cycles, establish connections with sheaf theory, gauge theory, and noncommutative geometry.

Result: Framework enables modeling of complex symmetries and provides quantitative measure of global inconsistency through holonomy-based contextuality index.

Conclusion: Opens new directions for curvature-aware learning architectures and topological regularization in uncertain/adversarial environments, bridging AI with geometric and topological methods.

Abstract: We propose a geometric extension of restricted Boltzmann machines (RBMs) by allowing weights to take values in abstract groups such as ( \mathrm{GL}_n(\mathbb{R}) ), ( \mathrm{SU}(2) ), or even infinite-dimensional operator groups. This generalization enables the modeling of complex relational structures, including projective transformations, spinor dynamics, and functional symmetries, with direct applications to vision, language, and quantum learning. A central contribution of this work is the introduction of a \emph{contextuality index} based on group-valued holonomies computed along cycles in the RBM graph. This index quantifies the global inconsistency or “curvature” induced by local weights, generalizing classical notions of coherence, consistency, and geometric flatness. We establish links with sheaf-theoretic contextuality, gauge theory, and noncommutative geometry, and provide numerical and diagrammatic examples in both finite and infinite dimensions. This framework opens novel directions in AI, from curvature-aware learning architectures to topological regularization in uncertain or adversarial environments.

Method: Develops a technique that exploits trade-offs between small and large-batch training, aiming to combine the parallel scaling advantages of large batches with the generalization benefits of small batches in federated learning settings.

Result: Achieves 32.33% higher test accuracy in ResNet50 and 3.74% higher accuracy in VGG11 models compared to small-batch training for the same number of iterations.

Conclusion: The proposed large-batch training technique successfully addresses generalization degradation issues while maintaining parallel scaling benefits, making federated learning more efficient without sacrificing model quality.

Abstract: Efficient Federated learning (FL) is crucial for training deep networks over devices with limited compute resources and bounded networks. With the advent of big data, devices either generate or collect multimodal data to train either generic or local-context aware networks, particularly when data privacy and locality is vital. FL algorithms generally trade-off between parallel and statistical performance, improving model quality at the cost of higher communication frequency, or vice versa. Under frequent synchronization settings, FL over a large cluster of devices may perform more work per-training iteration by processing a larger global batch-size, thus attaining considerable training speedup. However, this may result in poor test performance (i.e., low test loss or accuracy) due to generalization degradation issues associated with large-batch training. To address these challenges with large-batches, this work proposes our vision of exploiting the trade-offs between small and large-batch training, and explore new directions to enjoy both the parallel scaling of large-batches and good generalizability of small-batch training. For the same number of iterations, we observe that our proposed large-batch training technique attains about 32.33% and 3.74% higher test accuracy than small-batch training in ResNet50 and VGG11 models respectively.

Method: Dual alignment approach: (1) statistical alignment conditioning on patient demographics and risk factors, (2) semantic alignment incorporating real-world symptom trajectories. Uses Alzheimer’s disease as case study with LLaMA 3.1-8B model fine-tuning.

Result: Produces context-grounded symptom-level sentences reflecting real clinical documentation. Combination of DualAlign-generated and human-annotated data yields substantial performance gains over gold data alone or unguided synthetic baselines.

Conclusion: While not fully capturing longitudinal complexity, DualAlign offers practical approach for generating clinically grounded, privacy-preserving synthetic data to support low-resource clinical text analysis.

Abstract: Synthetic clinical data are increasingly important for advancing AI in healthcare, given strict privacy constraints on real-world EHRs, limited availability of annotated rare-condition data, and systemic biases in observational datasets. While large language models (LLMs) can generate fluent clinical text, producing synthetic data that is both realistic and clinically meaningful remains challenging. We introduce DualAlign, a framework that enhances statistical fidelity and clinical plausibility through dual alignment: (1) statistical alignment, which conditions generation on patient demographics and risk factors; and (2) semantic alignment, which incorporates real-world symptom trajectories to guide content generation. Using Alzheimer’s disease (AD) as a case study, DualAlign produces context-grounded symptom-level sentences that better reflect real-world clinical documentation. Fine-tuning an LLaMA 3.1-8B model with a combination of DualAlign-generated and human-annotated data yields substantial performance gains over models trained on gold data alone or unguided synthetic baselines. While DualAlign does not fully capture longitudinal complexity, it offers a practical approach for generating clinically grounded, privacy-preserving synthetic data to support low-resource clinical text analysis.

Method: The package integrates kernel attention networks (KAN), graph neural network-based gated recurrent units (GNNGRU), and time-aware graph neural networks (TimeGNN) to model nonlinear spatial-temporal patterns, plus provides preprocessing tools including outlier detection and a reinforcement learning-based gap-filling algorithm (Kalman-TransFusion Interpolation Framework).

Result: GTS Forecaster supports forecasting, visualization, and evaluation of GNSS, SSH, and TG datasets, and is adaptable to general time series applications with an accessible interface.

Conclusion: By combining cutting-edge deep learning models with accessible tools, GTS Forecaster facilitates the application of advanced AI techniques in geodetic forecasting tasks for enhanced early warning systems and hazard mitigation.

Abstract: Geodetic time series – such as Global Navigation Satellite System (GNSS) positions, satellite altimetry-derived sea surface height (SSH), and tide gauge (TG) records – is essential for monitoring surface deformation and sea level change. Accurate forecasts of these variables can enhance early warning systems and support hazard mitigation for earthquakes, landslides, coastal storm surge, and long-term sea level. However, the nonlinear, non-stationary, and incomplete nature of such variables presents significant challenges for classic models, which often fail to capture long-term dependencies and complex spatiotemporal dynamics. We introduce GTS Forecaster, an open-source Python package for geodetic time series forecasting. It integrates advanced deep learning models – including kernel attention networks (KAN), graph neural network-based gated recurrent units (GNNGRU), and time-aware graph neural networks (TimeGNN) – to effectively model nonlinear spatial-temporal patterns. The package also provides robust preprocessing tools, including outlier detection and a reinforcement learning-based gap-filling algorithm, the Kalman-TransFusion Interpolation Framework (KTIF). GTS Forecaster currently supports forecasting, visualization, and evaluation of GNSS, SSH, and TG datasets, and is adaptable to general time series applications. By combining cutting-edge models with an accessible interface, it facilitates the application of deep learning in geodetic forecasting tasks.

Method: Proposes a multi-phased framework structured around four pillars (Data, Algorithms, Human oversight, Model Architecture) to bridge ethical principles with operational practice.

Result: The framework enhances AI capabilities in diverse SME applications by providing structured guidance for secure and responsible AI use.

Conclusion: Offers a roadmap for responsible AI adoption that positions trust and ethics as drivers for SME resilience, competitiveness, and sustainable innovation.

Abstract: Artificial Intelligence (AI) and Large Language Models (LLMs) are reshaping today’s business practices, however, their adoption within small and medium-sized enterprises (SMEs) raises significant technical, ethical and trust issues. This paper proposes a structured, multi-phased framework designed to embed trust and ethical principles throughout the AI lifecycle for their secure and responsible use in SMEs. Structured around four pillars, i.e., Data, Algorithms, Human oversight, and Model Architecture, the framework bridges theoretical ethical principles with operational practice, enhancing AI capabilities in diverse SME applications. Ultimately, this paper offers a structured roadmap for responsible AI adoption, framing trust and ethics as a catalyst for resilience, competitiveness, and sustainable innovation in SMEs.

Method: Developed pySigLib with optimized CPU/GPU implementations fully compatible with PyTorch’s automatic differentiation, plus a novel differentiation scheme for signature kernels.

Result: The library delivers high-performance signature computations and accurate gradients at a fraction of the runtime of existing libraries.

Conclusion: pySigLib enables scalable signature-based machine learning for large datasets and long sequences, making signature kernels practical for real-world applications.

Abstract: Signature-based methods have recently gained significant traction in machine learning for sequential data. In particular, signature kernels have emerged as powerful discriminators and training losses for generative models on time-series, notably in quantitative finance. However, existing implementations do not scale to the dataset sizes and sequence lengths encountered in practice. We present pySigLib, a high-performance Python library offering optimised implementations of signatures and signature kernels on CPU and GPU, fully compatible with PyTorch’s automatic differentiation. Beyond an efficient software stack for large-scale signature-based computation, we introduce a novel differentiation scheme for signature kernels that delivers accurate gradients at a fraction of the runtime of existing libraries.

Method: The optimal MSB is computed by: 1) constructing a complete graph with edge weights equal to the sum of the optimal bimarginal SB value and endpoint entropies, and 2) solving a standard minimum spanning tree problem over this weighted graph.

Result: The method successfully computes the optimal MSB coupling across all possible graph structures, with numerical experiments demonstrating the proposed solution.

Conclusion: Finding the optimal Multimarginal Schrödinger Bridge reduces to solving a minimum spanning tree problem, providing an efficient computational approach for optimal coupling over arbitrary graph structures.

Abstract: The Multimarginal Schr"odinger Bridge (MSB) finds the optimal coupling among a collection of random vectors with known statistics and a known correlation structure. In the MSB formulation, this correlation structure is specified \emph{a priori} as an undirected connected graph with measure-valued vertices. In this work, we formulate and solve the problem of finding the optimal MSB in the sense we seek the optimal coupling over all possible graph structures. We find that computing the optimal MSB amounts to solving the minimum spanning tree problem over measure-valued vertices. We show that the resulting problem can be solved in two steps. The first step constructs a complete graph with edge weight equal to a sum of the optimal value of the corresponding bimarginal SB and the entropies of the endpoints. The second step solves a standard minimum spanning tree problem over that complete weighted graph. Numerical experiments illustrate the proposed solution.

Method: Proposes characteristic curves (CCs) modeled by neural networks, with three strategies: SINDy-CC (sparse regression with constraints), Poly-CC (polynomial representation), and NN-CC (neural networks without basis assumptions).

Result: All three approaches work well for simple polynomial nonlinearities (e.g., van der Pol oscillator), but NN-CC excels at modeling complex nonlinearities and discontinuities (e.g., stick-slip systems).

Conclusion: The CC-based framework, particularly NN-CC, successfully captures complex nonlinearities while maintaining interpretability through explicit CC representation, providing a powerful tool for challenging nonlinear system identification.

Abstract: Nonlinear system identification often involves a fundamental trade-off between interpretability and flexibility, often requiring the incorporation of physical constraints. We propose a unified data-driven framework that combines the mathematical structure of the governing differential equations with the flexibility of neural networks (NNs). At the core of our approach is the concept of characteristic curves (CCs), which represent individual nonlinear functions (e.g., friction and restoring components) of the system. Each CC is modeled by a dedicated NN, enabling a modular and interpretable representation of the system equation. To demonstrate the versatility of the CC-based formalism, we introduce three identification strategies: (1) SINDy-CC, which extends the sparse regression approach of SINDy by incorporating the mathematical structure of the governing equations as constraints; (2) Poly-CC, which represents each CC using high-degree polynomials; and (3) NN-CC, which uses NNs without requiring prior assumptions about basis functions. Our results show that all three approaches are well-suited for systems with simple polynomial nonlinearities, such as the van der Pol oscillator. In contrast, NN-CC demonstrates superior performance in modeling systems with complex nonlinearities and discontinuities, such as those observed in stick-slip systems. The key contribution of this work is to demonstrate that the CC-based framework, particularly the NN-CC approach, can capture complex nonlinearities while maintaining interpretability through the explicit representation of the CCs. This balance makes it well-suited for modeling systems with discontinuities and complex nonlinearities that are challenging to assess using traditional polynomial or sparse regression methods, providing a powerful tool for nonlinear system identification.

Method: Cross-silo horizontal federated learning framework that maps patient data into shared latent space, uses privacy-preserving kernel matrix computation, and allows hospitals to collaboratively train global ensemble feature extractor without sharing images.

Result: The federated service retains over 90% of centralized performance and remains robust to varying silo numbers and heterogeneous data distributions.

Conclusion: Federated learning enables effective collaborative training for facial dysmorphology analysis while maintaining data privacy and confidentiality across healthcare institutions.

Abstract: Machine learning has shown promise in facial dysmorphology, where characteristic facial features provide diagnostic clues for rare genetic disorders. GestaltMatcher, a leading framework in this field, has demonstrated clinical utility across multiple studies, but its reliance on centralized datasets limits further development, as patient data are siloed across institutions and subject to strict privacy regulations. We introduce a federated GestaltMatcher service based on a cross-silo horizontal federated learning framework, which allows hospitals to collaboratively train a global ensemble feature extractor without sharing patient images. Patient data are mapped into a shared latent space, and a privacy-preserving kernel matrix computation framework enables syndrome inference and discovery while safeguarding confidentiality. New participants can directly benefit from and contribute to the system by adopting the global feature extractor and kernel configuration from previous training rounds. Experiments show that the federated service retains over 90% of centralized performance and remains robust to both varying silo numbers and heterogeneous data distributions.

Method: Proposes Test-Time Warmup that adapts MLLMs per test instance by leveraging data from weakly supervised auxiliary tasks instead of extensive labeled fine-tuning datasets.

Result: Achieves relative performance improvements of 4.03% on MMMU, 5.28% on VQA-Rad, and 1.63% on GQA using Llama-Vision-Instruct model.

Conclusion: Warming up MLLMs before inference enhances robustness across diverse reasoning tasks, demonstrating the value of test-time adaptation over traditional fine-tuning approaches.

Abstract: Multimodal Large Language Models (MLLMs) hold great promise for advanced reasoning at the intersection of text and images, yet they have not fully realized this potential. MLLMs typically integrate an LLM, a vision encoder, and a connector that maps the vision encoder’s embeddings into the LLM’s text embedding space. Although each component is pretrained on massive datasets with billions of samples, the entire multimodal model is typically trained on only thousands (or a few million) samples, which can result in weak performance on complex reasoning tasks. To address these shortcomings, instead of relying on extensive labeled datasets for fine-tuning, we propose a Test-Time Warmup method that adapts the MLLM per test instance by leveraging data from weakly supervised auxiliary tasks. With our approach, we observe a relative performance improvement of 4.03% on MMMU, 5.28% on VQA-Rad, and 1.63% on GQA on the Llama-Vision-Instruct model. Our method demonstrates that ‘warming up’ before inference can enhance MLLMs’ robustness across diverse reasoning tasks.

Method: Agents maximize likelihood of visiting specified goal states rather than scalar rewards, using self-supervised goal-reaching techniques to learn from sparse feedback.

Result: Outperforms alternative MARL approaches with same sparse reward signal, achieves emergent cooperation and exploration where other methods fail completely.

Conclusion: Self-supervised multi-agent goal-reaching provides effective coordination without explicit exploration mechanisms, enabling successful learning from sparse goal-based feedback.

Abstract: For groups of autonomous agents to achieve a particular goal, they must engage in coordination and long-horizon reasoning. However, designing reward functions to elicit such behavior is challenging. In this paper, we study how self-supervised goal-reaching techniques can be leveraged to enable agents to cooperate. The key idea is that, rather than have agents maximize some scalar reward, agents aim to maximize the likelihood of visiting a certain goal. This problem setting enables human users to specify tasks via a single goal state rather than implementing a complex reward function. While the feedback signal is quite sparse, we will demonstrate that self-supervised goal-reaching techniques enable agents to learn from such feedback. On MARL benchmarks, our proposed method outperforms alternative approaches that have access to the same sparse reward signal as our method. While our method has no explicit mechanism for exploration, we observe that self-supervised multi-agent goal-reaching leads to emergent cooperation and exploration in settings where alternative approaches never witness a single successful trial.

Method: Three-tier hierarchical network with hybrid segmentation (graph partitioner + superpixel refinement), permutation-invariant aggregator, micro-level GNN for local dynamics, and macro-level transformer for cross-segment reasoning.

Result: Achieves up to 56% improvement in prediction accuracy and up to 22% faster inference compared to state-of-the-art baselines on multiple benchmark datasets.

Conclusion: M4GN provides an effective solution for large-scale PDE simulations by balancing accuracy and efficiency through principled hierarchical modeling and segmentation strategies.

Abstract: Mesh-based graph neural networks (GNNs) have become effective surrogates for PDE simulations, yet their deep message passing incurs high cost and over-smoothing on large, long-range meshes; hierarchical GNNs shorten propagation paths but still face two key obstacles: (i) building coarse graphs that respect mesh topology, geometry, and physical discontinuities, and (ii) maintaining fine-scale accuracy without sacrificing the speed gained from coarsening. We tackle these challenges with M4GN, a three-tier, segment-centric hierarchical network. M4GN begins with a hybrid segmentation strategy that pairs a fast graph partitioner with a superpixel-style refinement guided by modal-decomposition features, producing contiguous segments of dynamically consistent nodes. These segments are encoded by a permutation-invariant aggregator, avoiding the order sensitivity and quadratic cost of aggregation approaches used in prior works. The resulting information bridges a micro-level GNN, which captures local dynamics, and a macro-level transformer that reasons efficiently across segments, achieving a principled balance between accuracy and efficiency. Evaluated on multiple representative benchmark datasets, M4GN improves prediction accuracy by up to 56% while achieving up to 22% faster inference than state-of-the-art baselines.

Method: Uses conformal prediction to generate calibrated set-valued outputs, bridging the gap between single-label training and multi-label evaluation without relying on label distribution assumptions.

Result: Evaluated on 12 benchmark datasets, showing consistent improvements over existing baselines and demonstrating practical applicability.

Conclusion: The proposed conformal prediction approach effectively addresses the single-positive multi-label learning problem, providing reliable multi-label predictions without requiring full supervision or strong assumptions.

Abstract: Multi-label learning often requires identifying all relevant labels for training instances, but collecting full label annotations is costly and labor-intensive. In many datasets, only a single positive label is annotated per training instance, despite the presence of multiple relevant labels. This setting, known as single-positive multi-label learning (SPMLL), presents a significant challenge due to its extreme form of partial supervision. We propose a model-agnostic approach to SPMLL that draws on conformal prediction to produce calibrated set-valued outputs, enabling reliable multi-label predictions at test time. Our method bridges the supervision gap between single-label training and multi-label evaluation without relying on label distribution assumptions. We evaluate our approach on 12 benchmark datasets, demonstrating consistent improvements over existing baselines and practical applicability.

Method: Formulates bid shading as convex optimization over shading parameter distributions, uses regularized Wasserstein-proximal updates after each auction with context-dependent energy functionals based on publisher/user attributes.

Result: The measure-valued convex optimization problem admits a closed-form solution, and numerical examples demonstrate the proposed method’s effectiveness.

Conclusion: The proposed algorithm successfully optimizes bid shading in first-price auctions by adaptively updating parameter distributions to focus on high-surplus regions through Wasserstein-proximal regularization.

Abstract: This work proposes a bid shading strategy for first-price auctions as a measure-valued optimization problem. We consider a standard parametric form for bid shading and formulate the problem as convex optimization over the joint distribution of shading parameters. After each auction, the shading parameter distribution is adapted via a regularized Wasserstein-proximal update with a data-driven energy functional. This energy functional is conditional on the context, i.e., on publisher/user attributes such as domain, ad slot type, device, or location. The proposed algorithm encourages the bid distribution to place more weight on values with higher expected surplus, i.e., where the win probability and the value gap are both large. We show that the resulting measure-valued convex optimization problem admits a closed form solution. A numerical example illustrates the proposed method.

Method: Uses equality saturation and Datalog-style reasoning to verify semantic equivalence of computational graphs. Scales through graph partitioning with parallel rewriting, layer memoization, rewrite template reuse, and augments with relational reasoning and symbolic bijection inference.

Result: Verifies models as large as Llama-3.1-405B within minutes on commodity hardware and exposed five unknown bugs in Amazon production machine learning frameworks.

Conclusion: Scalify provides an efficient, scalable solution for detecting silent errors in large ML models, offering actionable debugging guidance and demonstrating practical effectiveness on massive production-scale models.

Abstract: Modern machine learning frameworks support very large models by incorporating parallelism and optimization techniques. Yet, these very techniques add new layers of complexity, introducing silent errors that severely degrade model performance. Existing solutions are either ad hoc or too costly for production. We present Scalify, a lightweight framework that exposes silent errors by verifying semantic equivalence of computational graphs using equality saturation and Datalog-style reasoning. To scale, Scalify partitions graphs with parallel rewriting and layer memoization, reuses rewrite templates, and augments equality saturation with relational reasoning and symbolic bijection inference. It further localizes discrepancies to precise code sites, turning verification results into actionable debugging guidance. Scalify verifies models as large as Llama-3.1-405B within minutes on a commodity machine and exposed five unknown bugs in Amazon production machine learning frameworks.

Method: Frames sequential fine-tuning as posterior inference using Bayesian framework with closed-form moment propagation, Kalman Bayesian Neural Networks, and Taylor approximations of softmax moments. Treats pre-trained models as priors and adaptively balances them against new information based on uncertainty.

Result: Achieves robust and data-efficient sequential learning, demonstrated through numerical simulations of sequential adaptation of decision transformers to tasks with distribution shifts and limited memory.

Conclusion: The proposed Bayesian approach effectively handles sequential fine-tuning challenges by quantifying uncertainty and adaptively balancing prior knowledge with new information, enabling stable learning with limited data in resource-constrained environments.

Abstract: Sequential fine-tuning of transformers is useful when new data arrive sequentially, especially with shifting distributions. Unlike batch learning, sequential learning demands that training be stabilized despite a small amount of data by balancing new information and previously learned knowledge in the pre-trained models. This challenge is further complicated when training is to be completed in latency-critical environments and learning must additionally quantify and be mediated by uncertainty. Motivated by these challenges, we propose a novel method that frames sequential fine-tuning as a posterior inference problem within a Bayesian framework. Our approach integrates closed-form moment propagation of random variables, Kalman Bayesian Neural Networks, and Taylor approximations of the moments of softmax functions. By explicitly accounting for pre-trained models as priors and adaptively balancing them against new information based on quantified uncertainty, our method achieves robust and data-efficient sequential learning. The effectiveness of our method is demonstrated through numerical simulations involving sequential adaptation of a decision transformer to tasks characterized by distribution shifts and limited memory resources.

Method: CrunchLLM integrates structured company attributes with unstructured textual narratives, using parameter-efficient fine-tuning strategies and prompt optimization to specialize foundation models for entrepreneurship data from Crunchbase.

Result: Achieves accuracy exceeding 80% on startup success prediction, significantly outperforming traditional classifiers and baseline LLMs. Provides interpretable reasoning traces for transparency.

Conclusion: Domain-aware fine-tuning and structured-unstructured data fusion with LLMs can advance predictive modeling of entrepreneurial outcomes, providing a practical tool for venture capital and innovation policy decision making.

Abstract: Predicting the success of start-up companies, defined as achieving an exit through acquisition or IPO, is a critical problem in entrepreneurship and innovation research. Datasets such as Crunchbase provide both structured information (e.g., funding rounds, industries, investor networks) and unstructured text (e.g., company descriptions), but effectively leveraging this heterogeneous data for prediction remains challenging. Traditional machine learning approaches often rely only on structured features and achieve moderate accuracy, while large language models (LLMs) offer rich reasoning abilities but struggle to adapt directly to domain-specific business data. We present \textbf{CrunchLLM}, a domain-adapted LLM framework for startup success prediction. CrunchLLM integrates structured company attributes with unstructured textual narratives and applies parameter-efficient fine-tuning strategies alongside prompt optimization to specialize foundation models for entrepreneurship data. Our approach achieves accuracy exceeding 80% on Crunchbase startup success prediction, significantly outperforming traditional classifiers and baseline LLMs. Beyond predictive performance, CrunchLLM provides interpretable reasoning traces that justify its predictions, enhancing transparency and trustworthiness for financial and policy decision makers. This work demonstrates how adapting LLMs with domain-aware fine-tuning and structured–unstructured data fusion can advance predictive modeling of entrepreneurial outcomes. CrunchLLM contributes a methodological framework and a practical tool for data-driven decision making in venture capital and innovation policy.

Method: Used large language models for late fusion of audio and motion time series data from Ego4D dataset, performing zero- and one-shot classification on 12 diverse activity classes without task-specific training.

Result: LLMs achieved F1-scores significantly above chance for both zero-shot and one-shot classification across diverse activity contexts (household activities, sports, etc.).

Conclusion: LLM-based fusion enables multimodal temporal applications without requiring aligned training data or additional memory/computation for application-specific multimodal models.

Abstract: Sensor data streams provide valuable information around activities and context for downstream applications, though integrating complementary information can be challenging. We show that large language models (LLMs) can be used for late fusion for activity classification from audio and motion time series data. We curated a subset of data for diverse activity recognition across contexts (e.g., household activities, sports) from the Ego4D dataset. Evaluated LLMs achieved 12-class zero- and one-shot classification F1-scores significantly above chance, with no task-specific training. Zero-shot classification via LLM-based fusion from modality-specific models can enable multimodal temporal applications where there is limited aligned training data for learning a shared embedding space. Additionally, LLM-based fusion can enable model deploying without requiring additional memory and computation for targeted application-specific multimodal models.

Method: Frames target region identification as a classification problem and develops an active learning framework specifically tailored for matched-pair designs that sequentially enrolls patients in high treatment-effect areas.

Result: The proposed design reduces experimental costs for detecting treatment efficacy while ensuring identified regions completely cover high-treatment-effect areas, with theoretical analysis and practical experiments demonstrating efficiency.

Conclusion: The active learning framework provides an efficient approach for identifying high treatment-effect regions in matched-pair experiments, offering both cost reduction and comprehensive coverage of effective intervention areas.

Abstract: Matched-pair experimental designs aim to detect treatment effects by pairing participants and comparing within-pair outcome differences. In many situations, the overall effect size is small across the entire population. Then, the focus naturally shifts to identifying and targeting high treatment-effect regions where the intervention is most effective. This paper proposes a matched-pair experimental design that sequentially and actively enrolls patients in high treatment-effect regions. Importantly, we frame the identification of the target region as a classification problem and propose an active learning framework tailored to matched-pair designs. The proposed design not only reduces the experimental cost of detecting treatment efficacy, but also ensures that the identified regions enclose the entire high-treatment-effect regions. Our theoretical analysis of the framework’s label complexity, along with experiments in practical scenarios, demonstrates the efficiency and advantages of the approach.

Method: Models LLM responses as discrete likelihood token paths with associated energy and entropy. Analyzes how energy and entropy distributions vary with temperature and likelihood changes to quantify semantic stability and detect unstable/erratic behavior.

Result: HalluField achieves state-of-the-art hallucination detection performance across models and datasets, is computationally efficient, and operates directly on output logits without requiring fine-tuning or auxiliary networks.

Conclusion: The thermodynamics-inspired field-theoretic approach provides a principled physical interpretation for hallucination detection and offers a practical, effective solution for identifying unreliable LLM outputs.

Abstract: Large Language Models (LLMs) exhibit impressive reasoning and question-answering capabilities. However, they often produce inaccurate or unreliable content known as hallucinations. This unreliability significantly limits their deployment in high-stakes applications. Thus, there is a growing need for a general-purpose method to detect hallucinations in LLMs. In this work, we introduce HalluField, a novel field-theoretic approach for hallucination detection based on a parametrized variational principle and thermodynamics. Inspired by thermodynamics, HalluField models an LLM’s response to a given query and temperature setting as a collection of discrete likelihood token paths, each associated with a corresponding energy and entropy. By analyzing how energy and entropy distributions vary across token paths under changes in temperature and likelihood, HalluField quantifies the semantic stability of a response. Hallucinations are then detected by identifying unstable or erratic behavior in this energy landscape. HalluField is computationally efficient and highly practical: it operates directly on the model’s output logits without requiring fine-tuning or auxiliary neural networks. Notably, the method is grounded in a principled physical interpretation, drawing analogies to the first law of thermodynamics. Remarkably, by modeling LLM behavior through this physical lens, HalluField achieves state-of-the-art hallucination detection performance across models and datasets.

Method: Develops Contextual Budget Bandit that incorporates context-dependent budget allocation in restless multi-armed bandits. Creates an empirically fast heuristic algorithm and Mitosis algorithm (guaranteed optimal budget allocation) for scenarios with scarce volunteers.

Result: The algorithms outperform baselines on both synthetic and real-world food rescue datasets, demonstrating improved geographical fairness in food distribution while maintaining rescue efficiency.

Conclusion: The proposed approach successfully addresses geographical disparities in food rescue platforms through context-aware budget allocation, achieving both fairness and efficiency in volunteer-based food distribution systems.

Abstract: Volunteer-based food rescue platforms tackle food waste by matching surplus food to communities in need. These platforms face the dual problem of maintaining volunteer engagement and maximizing the food rescued. Existing algorithms to improve volunteer engagement exacerbate geographical disparities, leaving some communities systematically disadvantaged. We address this issue by proposing Contextual Budget Bandit. Contextual Budget Bandit incorporates context-dependent budget allocation in restless multi-armed bandits, a model of decision-making which allows for stateful arms. By doing so, we can allocate higher budgets to communities with lower match rates, thereby alleviating geographical disparities. To tackle this problem, we develop an empirically fast heuristic algorithm. Because the heuristic algorithm can achieve a poor approximation when active volunteers are scarce, we design the Mitosis algorithm, which is guaranteed to compute the optimal budget allocation. Empirically, we demonstrate that our algorithms outperform baselines on both synthetic and real-world food rescue datasets, and show how our algorithm achieves geographical fairness in food rescue.

Method: A modular Python framework that injects diverse fault classes (weight corruption, activation injections, attention disruptions) into transformer models at multiple structural points.

Result: Provides reproducible experiments with metric logging and visualization, demonstrated through classification and generation task experiments.

Conclusion: GoldenTransformer offers researchers a valuable tool for model robustness analysis and dependable system design guidance in real-world LLM applications.

Abstract: Transformers have become the foundation for a wide range of state–of–the–art models across natural language processing, computer vision, and other machine learning domains. Despite their widespread deployment, the robustness of these models under fault conditions remains underexplored. We present GoldenTransformer, a modular and extensible fault injection framework designed to evaluate the resiliency of Large Language Models to induced hardware faults. GoldenTransformer offers a unified Python-based platform for injecting diverse classes of faults–such as weight corruption, activation injections, and attention–level disruptions–into pretrained transformer–based models. Inspired by the GoldenEye simulator for DNNs, our framework focuses on the unique challenges of working with large transformer architectures, including considerations such as structural complexity, latent dependencies, and nonuniform layer definitions. GoldenTransformer is built atop PyTorch and HuggingFace Transformers, and it supports experiment reproducibility, metric logging, and visualization out of the box. We detail the technical design and use of GoldenTransformer and demonstrate through several example experiments on classification and generation tasks. By enabling controlled injection of faults at multiple logical and structural points in a transformer, GoldenTransformer offers researchers and practitioners a valuable tool for model robustness analysis and for guiding dependable system design in real-world LLM applications.

Method: Proposed Selection and Projection framework (S&P TopK) with three key innovations: unconventional SAE feature selection strategy, novel debiasing methodology that orthogonalizes input embeddings against encoder weights, and performance-preserving mechanism through encoder weight interpolation.

Result: The method outperforms conventional SAE usage in fairness metrics by up to 3.2x and advances state-of-the-art test-time VLM debiasing results by up to 1.8x while maintaining downstream performance.

Conclusion: Encoder-focused debiasing through the S&P TopK framework provides superior fairness improvements compared to traditional decoder-focused approaches, demonstrating the importance of considering encoder weights in representation debiasing tasks.

Abstract: Sparse Autoencoders (SAEs) have proven valuable due to their ability to provide interpretable and steerable representations. Current debiasing methods based on SAEs manipulate these sparse activations presuming that feature representations are housed within decoder weights. We challenge this fundamental assumption and introduce an encoder-focused alternative for representation debiasing, contributing three key findings: (i) we highlight an unconventional SAE feature selection strategy, (ii) we propose a novel SAE debiasing methodology that orthogonalizes input embeddings against encoder weights, and (iii) we establish a performance-preserving mechanism during debiasing through encoder weight interpolation. Our Selection and Projection framework, termed S&P TopK, surpasses conventional SAE usage in fairness metrics by a factor of up to 3.2 and advances state-of-the-art test-time VLM debiasing results by a factor of up to 1.8 while maintaining downstream performance.

Method: Combines design of experiments with Shapley decomposition using two complementary estimation paths: plug-in based on conditional means and least-squares reconstruction of Shapley contributions. Includes standardization, bias correction, uncertainty quantification, and lightweight search routine.

Result: Improves rank preservation and optimal configuration identification, reduces decision-making risks, delivers interpretable justification with stable performance gains across diverse datasets and design conditions under budget constraints.

Conclusion: FACTORS provides a comprehensive tuning foundation that jointly addresses uncertainty and cost, offering reliable configuration selection with theoretical guarantees and interpretable insights even with limited budgets.

Abstract: We propose FACTORS, a framework that combines design of experiments with Shapley decomposition to address performance and stability issues that are sensitive to combinations of training factors. Our approach consistently estimates main effects and two-factor interactions, then integrates them into a risk-adjusted objective function that jointly accounts for uncertainty and cost, enabling reliable selection of configurations under a fixed budget. Effect estimation is implemented through two complementary paths: a plug-in path based on conditional means, and a least-squares path that reconstructs Shapley contributions from samples. These paths are designed to work complementarily even when design density and bias levels differ. By incorporating standardization of estimates, bias correction, and uncertainty quantification, our procedure ensures comparability across heterogeneous factor spaces and designs, while a lightweight search routine yields configurations within practical time even for large factor spaces. On the theoretical side, we provide error decompositions, sample complexity analysis, and upper bounds on optimality gaps. On the interpretive side, we summarize main effects and interactions in map form, highlighting adjustment priorities and safe improvement pathways. Across diverse datasets and design conditions, our approach improves rank preservation and optimal configuration identification, reduces decision-making risks, and offers a tuning foundation that delivers interpretable justification alongside stable performance gains even under budget constraints.

Method: Developed a neurosymbolic AI framework combining transfer learning with uncertainty quantification for network intrusion detection.

Result: Transfer learning models trained on large, well-structured datasets outperformed neural-based models using smaller datasets.

Conclusion: Transfer learning enables more effective cybersecurity solutions and represents a new era for intrusion detection systems.

Abstract: Transfer learning is commonly utilized in various fields such as computer vision, natural language processing, and medical imaging due to its impressive capability to address subtasks and work with different datasets. However, its application in cybersecurity has not been thoroughly explored. In this paper, we present an innovative neurosymbolic AI framework designed for network intrusion detection systems, which play a crucial role in combating malicious activities in cybersecurity. Our framework leverages transfer learning and uncertainty quantification. The findings indicate that transfer learning models, trained on large and well-structured datasets, outperform neural-based models that rely on smaller datasets, paving the way for a new era in cybersecurity solutions.

Method: CogGNN integrates visual input capabilities into GNNs with a visual-memory-based loss function and co-optimization strategy to generate connectional brain templates that preserve cognitive features.

Result: Extensive experiments show CogGNN outperforms state-of-the-art methods in generating brain networks that are both structurally and cognitively meaningful.

Conclusion: CogGNN establishes a strong foundation for cognitively grounded brain network modeling by successfully integrating cognitive capabilities into generative models.

Abstract: Generative learning has advanced network neuroscience, enabling tasks like graph super-resolution, temporal graph prediction, and multimodal brain graph fusion. However, current methods, mainly based on graph neural networks (GNNs), focus solely on structural and topological properties, neglecting cognitive traits. To address this, we introduce the first cognified generative model, CogGNN, which endows GNNs with cognitive capabilities (e.g., visual memory) to generate brain networks that preserve cognitive features. While broadly applicable, we present CogGNN, a specific variant designed to integrate visual input, a key factor in brain functions like pattern recognition and memory recall. As a proof of concept, we use our model to learn connectional brain templates (CBTs), population-level fingerprints from multi-view brain networks. Unlike prior work that overlooks cognitive properties, CogGNN generates CBTs that are both cognitively and structurally meaningful. Our contributions are: (i) a novel cognition-aware generative model with a visual-memory-based loss; (ii) a CBT-learning framework with a co-optimization strategy to yield well-centered, discriminative, cognitively enhanced templates. Extensive experiments show that CogGNN outperforms state-of-the-art methods, establishing a strong foundation for cognitively grounded brain network modeling.

Method: Proposes a holistic framework with three key components: local-global Transformer encoder, memory-guided reconstruction mechanism, and multi-scale representation matching strategy. Combines reconstruction errors and memory matching signals for robust anomaly scoring.

Result: Extensive experiments on seven benchmark datasets demonstrate superior performance over state-of-the-art approaches, showing comprehensive and generalizable anomaly detection across various graph domains.

Conclusion: The proposed framework effectively addresses limitations of existing methods by synergistically integrating multiple components to capture structural dependencies, suppress anomalous influence, and provide multi-granularity assessment for robust graph anomaly detection.

Abstract: Anomaly detection in graph-structured data is an inherently challenging problem, as it requires the identification of rare nodes that deviate from the majority in both their structural and behavioral characteristics. Existing methods, such as those based on graph convolutional networks (GCNs), often suffer from over-smoothing, which causes the learned node representations to become indistinguishable. Furthermore, graph reconstruction-based approaches are vulnerable to anomalous node interference during the reconstruction process, leading to inaccurate anomaly detection. In this work, we propose a novel and holistic anomaly evaluation framework that integrates three key components: a local-global Transformer encoder, a memory-guided reconstruction mechanism, and a multi-scale representation matching strategy. These components work synergistically to enhance the model’s ability to capture both local and global structural dependencies, suppress the influence of anomalous nodes, and assess anomalies from multiple levels of granularity. Anomaly scores are computed by combining reconstruction errors and memory matching signals, resulting in a more robust evaluation. Extensive experiments on seven benchmark datasets demonstrate that our method outperforms existing state-of-the-art approaches, offering a comprehensive and generalizable solution for anomaly detection across various graph domains.

Method: Designed MPNN architectures with bidirectional message-passing and attention mechanisms, using minimalist message formulation without self-perception. Tested with 2D molecular graphs complemented with 3D descriptors, and evaluated convolution normalization factors.

Result: Achieved state-of-the-art performance surpassing complex pre-trained models. Found that simpler models yield higher class separability, 2D graphs with 3D descriptors preserve performance while reducing computational cost by over 50%, and convolution normalization factors don’t benefit predictive power.

Conclusion: Simpler MPNN architectures with appropriate feature selection outperform complex models, with 2D graphs supplemented by 3D descriptors providing optimal balance between performance and computational efficiency for high-throughput screening.

Abstract: Strategies to improve the predicting performance of Message-Passing Neural-Networks for molecular property predictions can be achieved by simplifying how the message is passed and by using descriptors that capture multiple aspects of molecular graphs. In this work, we designed model architectures that achieved state-of-the-art performance, surpassing more complex models such as those pre-trained on external databases. We assessed dataset diversity to complement our performance results, finding that structural diversity influences the need for additional components in our MPNNs and feature sets. In most datasets, our best architecture employs bidirectional message-passing with an attention mechanism, applied to a minimalist message formulation that excludes self-perception, highlighting that relatively simpler models, compared to classical MPNNs, yield higher class separability. In contrast, we found that convolution normalization factors do not benefit the predictive power in all the datasets tested. This was corroborated in both global and node-level outputs. Additionally, we analyzed the influence of both adding spatial features and working with 3D graphs, finding that 2D molecular graphs are sufficient when complemented with appropriately chosen 3D descriptors. This approach not only preserves predictive performance but also reduces computational cost by over 50%, making it particularly advantageous for high-throughput screening campaigns.

Method: Novel adversarial objective function focused on the added noise of denoising diffusion models to train base classifiers robust against covariate shift.

Result: Significantly improves certified accuracy across MNIST, CIFAR-10, and ImageNet benchmarks, achieving new state-of-the-art performance for l2-adversarial perturbations.

Conclusion: The proposed method effectively addresses covariate shift in diffusion denoised smoothing and delivers superior certified robustness compared to existing approaches.

Abstract: Randomized smoothing is a well-established method for achieving certified robustness against l2-adversarial perturbations. By incorporating a denoiser before the base classifier, pretrained classifiers can be seamlessly integrated into randomized smoothing without significant performance degradation. Among existing methods, Diffusion Denoised Smoothing - where a pretrained denoising diffusion model serves as the denoiser - has produced state-of-the-art results. However, we show that employing a denoising diffusion model introduces a covariate shift via misestimation of the added noise, ultimately degrading the smoothed classifier’s performance. To address this issue, we propose a novel adversarial objective function focused on the added noise of the denoising diffusion model. This approach is inspired by our understanding of the origin of the covariate shift. Our goal is to train the base classifier to ensure it is robust against the covariate shift introduced by the denoiser. Our method significantly improves certified accuracy across three standard classification benchmarks - MNIST, CIFAR-10, and ImageNet - achieving new state-of-the-art performance in l2-adversarial perturbations. Our implementation is publicly available at https://github.com/ahedayat/Robustifying-DDS-Against-Covariate-Shift

Method: Proposes Token Merge with Attention (ToMA) with three key innovations: 1) token merge as submodular optimization for diverse token selection, 2) merge/unmerge as GPU-friendly matrix operations, and 3) exploiting latent locality and sequential redundancy to minimize overhead.

Result: Reduces SDXL generation latency by 24% and Flux generation latency by 23% while maintaining quality (DINO Δ < 0.07), outperforming prior token reduction methods.

Conclusion: ToMA successfully bridges the gap between theoretical and practical efficiency for transformers in diffusion models by providing GPU-aligned token reduction that delivers actual speed improvements without quality degradation.

Abstract: Diffusion models excel in high-fidelity image generation but face scalability limits due to transformers’ quadratic attention complexity. Plug-and-play token reduction methods like ToMeSD and ToFu reduce FLOPs by merging redundant tokens in generated images but rely on GPU-inefficient operations (e.g., sorting, scattered writes), introducing overheads that negate theoretical speedups when paired with optimized attention implementations (e.g., FlashAttention). To bridge this gap, we propose Token Merge with Attention (ToMA), an off-the-shelf method that redesigns token reduction for GPU-aligned efficiency, with three key contributions: 1) a reformulation of token merge as a submodular optimization problem to select diverse tokens; 2) merge/unmerge as an attention-like linear transformation via GPU-friendly matrix operations; and 3) exploiting latent locality and sequential redundancy (pattern reuse) to minimize overhead. ToMA reduces SDXL/Flux generation latency by 24%/23%, respectively (with DINO $\Delta < 0.07$), outperforming prior methods. This work bridges the gap between theoretical and practical efficiency for transformers in diffusion.

Method: Comparative analysis of interpretability vs explainability through definitions, objectives, methodologies, and challenges, with illustrative case studies using MNIST digit classification and IMDB sentiment analysis tasks.

Result: Establishes that interpretability refers to a model’s inherent capacity for global human comprehension, while explainability involves post-hoc techniques for local explanations of individual predictions (e.g., feature attribution for MNIST digits, word importance for sentiment analysis).

Conclusion: Clear differentiation between interpretability and explainability is crucial for developing dependable AI, as local explanations do not provide global transparency of complex underlying models.

Abstract: The impressive capabilities of deep learning models are often counterbalanced by their inherent opacity, commonly termed the “black box” problem, which impedes their widespread acceptance in high-trust domains. In response, the intersecting disciplines of interpretability and explainability, collectively falling under the Explainable AI (XAI) umbrella, have become focal points of research. Although these terms are frequently used as synonyms, they carry distinct conceptual weights. This document offers a comparative exploration of interpretability and explainability within the deep learning paradigm, carefully outlining their respective definitions, objectives, prevalent methodologies, and inherent difficulties. Through illustrative examinations of the MNIST digit classification task and IMDB sentiment analysis, we substantiate a key argument: interpretability generally pertains to a model’s inherent capacity for human comprehension of its operational mechanisms (global understanding), whereas explainability is more commonly associated with post-hoc techniques designed to illuminate the basis for a model’s individual predictions or behaviors (local explanations). For example, feature attribution methods can reveal why a specific MNIST image is recognized as a ‘7’, and word-level importance can clarify an IMDB sentiment outcome. However, these local insights do not render the complex underlying model globally transparent. A clear grasp of this differentiation, as demonstrated by these standard datasets, is vital for fostering dependable and sound artificial intelligence.

Method: Developed psychosis-bench benchmark with 16 structured, 12-turn conversational scenarios simulating delusional themes progression. Evaluated 8 LLMs across explicit and implicit contexts using Delusion Confirmation Score, Harm Enablement Score, and Safety Intervention Score.

Result: All LLMs demonstrated psychogenic potential (mean DCS 0.91), frequently enabled harmful requests (mean HES 0.69), and offered safety interventions in only ~33% of applicable turns (mean SIS 0.37). Performance was significantly worse in implicit scenarios with strong correlation between delusion confirmation and harm enablement.

Conclusion: LLM psychogenicity is a quantifiable risk requiring urgent re-thinking of LLM training approaches. This is not just a technical challenge but a public health imperative needing collaboration between developers, policymakers, and healthcare professionals.

Abstract: Background: Emerging reports of “AI psychosis” are on the rise, where user-LLM interactions may exacerbate or induce psychosis or adverse psychological symptoms. The sycophantic and agreeable nature of LLMs can beneficial, it can become a vector for harm by reinforcing delusional beliefs in vulnerable users. Methods: We introduce psychosis-bench, a novel benchmark designed to systematically evaluate the psychogenicity of LLMs comprimising 16 structured, 12-turn conversational scenarios simulating the progression of delusional themes(Erotic Delusions, Grandiose/Messianic Delusions, Referential Delusions) and potential harms. We evaluated eight prominent LLMs for Delusion Confirmation (DCS), Harm Enablement (HES), and Safety Intervention(SIS) across explicit and implicit conversational contexts. Findings: Across 1,536 simulated conversation turns, all LLMs demonstrated psychogenic potential, showing a strong tendency to perpetuate rather than challenge delusions (mean DCS of 0.91 $\pm$0.88). Models frequently enabled harmful user requests (mean HES of 0.69 $\pm$0.84) and offered safety interventions in only roughly a third of applicable turns (mean SIS of 0.37 $\pm$0.48). 51 / 128 (39.8%) of scenarios had no safety interventions offered. Performance was significantly worse in implicit scenarios, models were more likely to confirm delusions and enable harm while offering fewer interventions (p < .001). A strong correlation was found between DCS and HES (rs = .77). Model performance varied widely, indicating that safety is not an emergent property of scale alone. Conclusion: This study establishes LLM psychogenicity as a quantifiable risk and underscores the urgent need for re-thinking how we train LLMs. We frame this issue not merely as a technical challenge but as a public health imperative requiring collaboration between developers, policymakers, and healthcare professionals.

Method: Computes low-rank decomposition of weight differences between base and fine-tuned models to reconstruct adapter modules that can be merged or dynamically routed.

Result: Extracted adapters preserve high energy from full weight delta, can be safely pruned, and show negligible performance degradation when re-merged across text, image, and video benchmarks.

Conclusion: PHLoRA provides a practical solution for converting existing full-rank models into adapter-ready formats, enabling scalable inference and substantial cost savings.

Abstract: We introduce PHLoRA (Pronounced “flora”). (Post-hoc LoRA), a simple yet powerful method to extract low-rank adaptation adapters from full-rank fine-tuned models without requiring access to training data or gradients. By computing the low-rank decomposition of weight differences between a base model and its fine-tuned counterpart, our method reconstructs adapter modules that can be merged or dynamically routed at inference time via S-LoRA, or served in scalable, industry settings using platforms like NVIDIA NIM. This approach amortizes latency overhead across requests and yields substantial cost savings. Unlike prior work that trains each adapter explicitly, our approach decouples fine-tuning from adapter generation, allowing adapter extraction from existing full-rank models or third-party checkpoints. Experiments on text, image, and video benchmarks using the Amazon Nova model family demonstrate that extracted adapters preserve high energy from the full weight delta, can be pruned safely, and yield negligible degradation in downstream task performance when re-merged. Overall, PHLoRA provides a practical path for making all existing full-rank checkpoints adapter-ready, democratizing scalable inference for all models.

Method: Two-phase approach: 1) Evolutionary search in tractable representation space (intermediate activations) to find diverse solutions 2) Gradient-based learning in parameter space by regressing to searched representations

Result: Searched representations are learnable - networks approach SGD performance on MNIST, CIFAR-10, CIFAR-100. Performance improves with search compute. Models follow different representational trajectories than gradient descent.

Conclusion: Demonstrates how training algorithms can overcome gradient descent’s exploratory limitations by decoupling search in representation space from gradient-based learning in parameter space.

Abstract: Gradient descent typically converges to a single minimum of the training loss without mechanisms to explore alternative minima that may generalize better. Searching for diverse minima directly in high-dimensional parameter space is generally intractable. To address this, we propose a framework that performs training in two distinct phases: search in a tractable representation space (the space of intermediate activations) to find diverse representational solutions, and gradient-based learning in parameter space by regressing to those searched representations. Through evolutionary search, we discover representational solutions whose fitness and diversity scale with compute–larger populations and more generations produce better and more varied solutions. These representations prove to be learnable: networks trained by regressing to searched representations approach SGD’s performance on MNIST, CIFAR-10, and CIFAR-100. Performance improves with search compute up to saturation. The resulting models differ qualitatively from networks trained with gradient descent, following different representational trajectories during training. This work demonstrates how future training algorithms could overcome gradient descent’s exploratory limitations by decoupling search in representation space from efficient gradient-based learning in parameter space.

Method: Created CAWFI database compiling daily historical wildfire data (2012-2018) and indicator data (2012-2022) including meteorological (leading indicators), environmental (trailing indicators), and geological data (vegetation/elevation).

Result: When used to train a spatio-temporal AI model, CAWFI achieved 85.7% prediction accuracy for future wildfires larger than 300,000 acres using 2012-2017 indicator data.

Conclusion: CAWFI enables wildfire prediction research and solutions, setting a precedent for similar databases in other regions to help prevent wildfires before they occur.

Abstract: Due to climate change and the disruption of ecosystems worldwide, wildfires are increasingly impacting environment, infrastructure, and human lives globally. Additionally, an exacerbating climate crisis means that these losses would continue to grow if preventative measures are not implemented. Though recent advancements in artificial intelligence enable wildfire management techniques, most deployed solutions focus on detecting wildfires after ignition. The development of predictive techniques with high accuracy requires extensive datasets to train machine learning models. This paper presents the California Wildfire Inventory (CAWFI), a wildfire database of over 37 million data points for building and training wildfire prediction solutions, thereby potentially preventing megafires and flash fires by addressing them before they spark. The dataset compiles daily historical California wildfire data from 2012 to 2018 and indicator data from 2012 to 2022. The indicator data consists of leading indicators (meteorological data correlating to wildfire-prone conditions), trailing indicators (environmental data correlating to prior and early wildfire activity), and geological indicators (vegetation and elevation data dictating wildfire risk and spread patterns). CAWFI has already demonstrated success when used to train a spatio-temporal artificial intelligence model, predicting 85.7% of future wildfires larger than 300,000 acres when trained on 2012-2017 indicator data. This dataset is intended to enable wildfire prediction research and solutions as well as set a precedent for future wildfire databases in other regions.

Method: FragmentGPT combines two core components: (1) chemically-aware, energy-based bond cleavage pre-training for GPT-based fragment manipulation capabilities, and (2) Reward Ranked Alignment with Expert Exploration algorithm combining expert imitation learning, data selection/augmentation, and Supervised Fine-Tuning for multi-objective alignment.

Result: Experiments on real-world cancer datasets show FragmentGPT generates chemically valid, high-quality molecules tailored for downstream drug discovery tasks, effectively resolving structural redundancies through intelligent merging.

Conclusion: FragmentGPT provides a unified framework for controlled, goal-driven molecular assembly that optimizes multiple pharmaceutical objectives while handling complex structural challenges in fragment-based drug discovery.

Abstract: Fragment-Based Drug Discovery (FBDD) is a popular approach in early drug development, but designing effective linkers to combine disconnected molecular fragments into chemically and pharmacologically viable candidates remains challenging. Further complexity arises when fragments contain structural redundancies, like duplicate rings, which cannot be addressed by simply adding or removing atoms or bonds. To address these challenges in a unified framework, we introduce FragmentGPT, which integrates two core components: (1) a novel chemically-aware, energy-based bond cleavage pre-training strategy that equips the GPT-based model with fragment growing, linking, and merging capabilities, and (2) a novel Reward Ranked Alignment with Expert Exploration (RAE) algorithm that combines expert imitation learning for diversity enhancement, data selection and augmentation for Pareto and composite score optimality, and Supervised Fine-Tuning (SFT) to align the learner policy with multi-objective goals. Conditioned on fragment pairs, FragmentGPT generates linkers that connect diverse molecular subunits while simultaneously optimizing for multiple pharmaceutical goals. It also learns to resolve structural redundancies-such as duplicated fragments-through intelligent merging, enabling the synthesis of optimized molecules. FragmentGPT facilitates controlled, goal-driven molecular assembly. Experiments and ablation studies on real-world cancer datasets demonstrate its ability to generate chemically valid, high-quality molecules tailored for downstream drug discovery tasks.

Method: Evaluated several data sampling strategies, focusing on a simple time stratified sampling approach to reduce training data requirements while maintaining performance.

Result: Time stratified sampling with only 20% of training data achieved performance similar to or better than full-data training, outperforming on certain metrics and performing only slightly worse on others.

Conclusion: Demonstrates feasibility of data-efficient diffusion training for weather forecasting and motivates future work on adaptive or model-aware sampling methods beyond random or temporal sampling.

Abstract: Although numerical weather forecasting methods have dominated the field, recent advances in deep learning methods, such as diffusion models, have shown promise in ensemble weather forecasting. However, such models are typically autoregressive and are thus computationally expensive. This is a challenge in climate science, where data can be limited, costly, or difficult to work with. In this work, we explore the impact of curated data selection on these autoregressive diffusion models. We evaluate several data sampling strategies and show that a simple time stratified sampling approach achieves performance similar to or better than full-data training. Notably, it outperforms the full-data model on certain metrics and performs only slightly worse on others while using only 20% of the training data. Our results demonstrate the feasibility of data-efficient diffusion training, especially for weather forecasting, and motivates future work on adaptive or model-aware sampling methods that go beyond random or purely temporal sampling.

Method: Three components: 1) CCLR-GAN for diverse data generation, 2) contrastive learning for modeling sample relationships, 3) 1D-FCNN with Fourier convolution for global-aware feature extraction from vibration signals.

Result: Achieves up to 32% improvement on CWRU dataset and 10% improvement on self-collected test bench compared to baselines. Ablation experiments confirm effectiveness of each component.

Conclusion: DAC-FCF provides a promising solution for bearing fault diagnosis under data scarcity by addressing key limitations of existing methods through integrated data augmentation, relationship modeling, and global feature extraction.

Abstract: In the area of bearing fault diagnosis, deep learning (DL) methods have been widely used recently. However, due to the high cost or privacy concerns, high-quality labeled data are scarce in real world scenarios. While few-shot learning has shown promise in addressing data scarcity, existing methods still face significant limitations in this domain. Traditional data augmentation techniques often suffer from mode collapse and generate low-quality samples that fail to capture the diversity of bearing fault patterns. Moreover, conventional convolutional neural networks (CNNs) with local receptive fields makes them inadequate for extracting global features from complex vibration signals. Additionally, existing methods fail to model the intricate relationships between limited training samples. To solve these problems, we propose an advanced data augmentation and contrastive fourier convolution framework (DAC-FCF) for bearing fault diagnosis under limited data. Firstly, a novel conditional consistent latent representation and reconstruction generative adversarial network (CCLR-GAN) is proposed to generate more diverse data. Secondly, a contrastive learning based joint optimization mechanism is utilized to better model the relations between the available training data. Finally, we propose a 1D fourier convolution neural network (1D-FCNN) to achieve a global-aware of the input data. Experiments demonstrate that DAC-FCF achieves significant improvements, outperforming baselines by up to 32% on case western reserve university (CWRU) dataset and 10% on a self-collected test bench. Extensive ablation experiments prove the effectiveness of the proposed components. Thus, the proposed DAC-FCF offers a promising solution for bearing fault diagnosis under limited data.

Method: Leveraging a 127-feature set across time, frequency, and time-frequency domains, validated on both synthetic and real-world datasets with systematic and statistically rigorous evaluation framework.

Result: Ensemble method achieved superior performance (94.2% accuracy, 0.942 F1-score) with statistical testing confirming significant outperformance of individual algorithms by 8-15%.

Conclusion: Provides a validated benchmarking protocol and practical guidelines for selecting robust monitoring solutions in industrial settings.

Abstract: Audio-based equipment condition monitoring suffers from a lack of standardized methodologies for algorithm selection, hindering reproducible research. This paper addresses this gap by introducing a comprehensive framework for the systematic and statistically rigorous evaluation of machine learning models. Leveraging a rich 127-feature set across time, frequency, and time-frequency domains, our methodology is validated on both synthetic and real-world datasets. Results demonstrate that an ensemble method achieves superior performance (94.2% accuracy, 0.942 F1-score), with statistical testing confirming its significant outperformance of individual algorithms by 8-15%. Ultimately, this work provides a validated benchmarking protocol and practical guidelines for selecting robust monitoring solutions in industrial settings.

Method: Uses Prophet forecasting model with COVID-19 metrics integration and implements SKU-specific hyperparameter optimization to tailor predictions for individual mattress products.

Result: The model demonstrates strong predictive capabilities and provides reliable forecasting that helps contract manufacturers prepare for demand, source raw materials optimally, and streamline supply chain operations.

Conclusion: The DemandLens model offers an effective solution for sales forecasting in the mattress-in-a-box industry, enabling better supply chain management and operational efficiency for both contract manufacturers and mattress brands.

Abstract: DemandLens demonstrates an innovative Prophet based forecasting model for the mattress-in-a-box industry, incorporating COVID-19 metrics and SKU-specific hyperparameter optimization. This industry has seen significant growth of E-commerce players in the recent years, wherein the business model majorly relies on outsourcing Mattress manufacturing and related logistics and supply chain operations, focusing on marketing the product and driving conversions through Direct-to-Consumer sales channels. Now, within the United States, there are a limited number of Mattress contract manufacturers available, and hence, it is important that they manage their raw materials, supply chain, and, inventory intelligently, to be able to cater maximum Mattress brands. Our approach addresses the critical need for accurate Sales Forecasting in an industry that is heavily dependent on third-party Contract Manufacturing. This, in turn, helps the contract manufacturers to be prepared, hence, avoiding bottleneck scenarios, and aiding them to source raw materials at optimal rates. The model demonstrates strong predictive capabilities through SKU-specific Hyperparameter optimization, offering the Contract Manufacturers and Mattress brands a reliable tool to streamline supply chain operations.

Method: Transforms raw location data into higher-order mobility flow representations using hexagonal tessellation, integrates Graph Convolutional Networks (GCNs) to model spatial relationships and non-local dependencies without requiring side information like timestamps or POIs.

Result: Experiments on six real-world datasets show consistent improvements over classical baselines, RNN- and Transformer-based models, achieving 1-8% relative gains in accuracy, precision, recall, and F1-score. Optimal setup identified with single GCN layer and 512-dimensional embeddings.

Conclusion: The integration of GCNs enhances spatial learning and improves generalizability across mobility data, offering a robust and scalable solution for trajectory-user linking with applications in recommendations, urban planning, and security.

Abstract: Trajectory-user linking (TUL) aims to associate anonymized trajectories with the users who generated them, which is crucial for personalized recommendations, privacy-preserving analytics, and secure location-based services. Existing methods struggle with sparse data, incomplete routes, and limited modeling of complex spatial dependencies, often relying on low-level check-in data or ignoring spatial patterns. In this paper, we introduced GCN-TULHOR, a method that transforms raw location data into higher-order mobility flow representations using hexagonal tessellation, reducing data sparsity and capturing richer spatial semantics, and integrating Graph Convolutional Networks (GCNs). Our approach converts both sparse check-in and continuous GPS trajectory data into unified higher-order flow representations, mitigating sparsity while capturing deeper semantic information. The GCN layer explicitly models complex spatial relationships and non-local dependencies without requiring side information such as timestamps or points of interest. Experiments on six real-world datasets show consistent improvements over classical baselines, RNN- and Transformer-based models, and the TULHOR method in accuracy, precision, recall, and F1-score. GCN-TULHOR achieves 1-8% relative gains in accuracy and F1. Sensitivity analysis identifies an optimal setup with a single GCN layer and 512-dimensional embeddings. The integration of GCNs enhances spatial learning and improves generalizability across mobility data. This work highlights the value of combining graph-based spatial learning with sequential modeling, offering a robust and scalable solution for TUL with applications in recommendations, urban planning, and security.

Method: Developed scalable graph representation for BIM components, created dataset with 1M nodes/3.5M edges, introduced new message-passing mechanism to GraphMAE2 with node masking pretraining strategy.

Result: Homogeneous graph representation outperforms heterogeneous; 30cm local spatial relationships enhance performance; GAT-based feature extraction achieves best transfer learning results with 72.7% F1-score improvement.

Conclusion: BIGNet effectively learns and transfers BIM design features, demonstrating strong potential for automated application in design and lifecycle management.

Abstract: Large Foundation Models (LFMs) have demonstrated significant advantages in civil engineering, but they primarily focus on textual and visual data, overlooking the rich semantic, spatial, and topological features in BIM (Building Information Modelling) models. Therefore, this study develops the first large-scale graph neural network (GNN), BIGNet, to learn, and reuse multidimensional design features embedded in BIM models. Firstly, a scalable graph representation is introduced to encode the “semantic-spatial-topological” features of BIM components, and a dataset with nearly 1 million nodes and 3.5 million edges is created. Subsequently, BIGNet is proposed by introducing a new message-passing mechanism to GraphMAE2 and further pretrained with a node masking strategy. Finally, BIGNet is evaluated in various transfer learning tasks for BIM-based design checking. Results show that: 1) homogeneous graph representation outperforms heterogeneous graph in learning design features, 2) considering local spatial relationships in a 30 cm radius enhances performance, and 3) BIGNet with GAT (Graph Attention Network)-based feature extraction achieves the best transfer learning results. This innovation leads to a 72.7% improvement in Average F1-score over non-pretrained models, demonstrating its effectiveness in learning and transferring BIM design features and facilitating their automated application in future design and lifecycle management.

Method: Created PNGT-26K dataset with 26,000 Persian name-gender-transliteration tuples, and developed two frameworks: Open Gender Detection (probabilistic gender guessing from user data) and Nominalist (AI agent for username generation).

Result: A publicly available comprehensive dataset and two ready-to-use frameworks that address Persian name processing challenges in gender detection and digital identity creation.

Conclusion: The introduced dataset and frameworks provide valuable resources for improving NLP applications dealing with Persian names, overcoming transliteration and cultural barriers while enhancing user experience in digital platforms.

Abstract: Persian names present unique challenges for natural language processing applications, particularly in gender detection and digital identity creation, due to transliteration inconsistencies and cultural-specific naming patterns. Existing tools exhibit significant performance degradation on Persian names, while the scarcity of comprehensive datasets further compounds these limitations. To address these challenges, the present research introduces PNGT-26K, a comprehensive dataset of Persian names, their commonly associated gender, and their English transliteration, consisting of approximately 26,000 tuples. As a demonstration of how this resource can be utilized, we also introduce two frameworks, namely Open Gender Detection and Nominalist. Open Gender Detection is a production-grade, ready-to-use framework for using existing data from a user, such as profile photo and name, to give a probabilistic guess about the person’s gender. Nominalist, the second framework introduced by this paper, utilizes agentic AI to help users choose a username for their social media accounts on any platform. It can be easily integrated into any website to provide a better user experience. The PNGT-26K dataset, Nominalist and Open Gender Detection frameworks are publicly available on Github.

Method: Introduces Hopkins loss function based on Hopkins statistic to enforce desired feature space topology in classification and dimensionality reduction tasks using nonlinear bottleneck autoencoders.

Result: Experiments on speech, text, and image data show Hopkins loss integration has minimal impact on classification performance while successfully modifying feature topology.

Conclusion: Hopkins loss effectively enforces desired feature space topology without significantly compromising task performance, offering benefits for various ML applications.

Abstract: Feature space topology refers to the organization of samples within the feature space. Modifying this topology can be beneficial in machine learning applications, including dimensionality reduction, generative modeling, transfer learning, and robustness to adversarial attacks. This paper introduces a novel loss function, Hopkins loss, which leverages the Hopkins statistic to enforce a desired feature space topology, which is in contrast to existing topology-related methods that aim to preserve input feature topology. We evaluate the effectiveness of Hopkins loss on speech, text, and image data in two scenarios: classification and dimensionality reduction using nonlinear bottleneck autoencoders. Our experiments show that integrating Hopkins loss into classification or dimensionality reduction has only a small impact on classification performance while providing the benefit of modifying feature topology.

Method: Two-phase approach: 1) Offline SVD computation of universal projection matrix on calibration data, 2) Online inference with query/key projection and dynamic dimension selection based on query magnitudes.

Result: 25% reduction in attention dot-product computation with statistically insignificant performance degradation across benchmarks; synergizes with existing token eviction methods and reduces KV-cache memory.

Conclusion: AQUA provides a practical, controllable efficiency-accuracy trade-off tool that makes large-scale LLM inference more accessible and sustainable.

Abstract: The quadratic complexity of the attention mechanism remains a fundamental barrier to scaling Large Language Models (LLMs) to longer contexts, creating a critical bottleneck in both computation and memory. To address this, we introduce AQUA (Attention via QUery mAgnitudes) a novel and versatile approximation strategy that significantly reduces the cost of attention with a graceful performance trade-off. Our method operates in two phases: an efficient offline step where we compute a universal, language agnostic projection matrix via SVD on a calibration dataset, and an online inference step where we project query and key vectors and dynamically select a sparse subset of dimensions based on the query’s magnitude. We provide a formal theoretical analysis of AQUA, establishing the break-even point at which it becomes more computationally efficient than standard attention. Our empirical evaluations on state-of-the-art models like Llama-3.1-8B demonstrate that a 25% reduction in the attention dot-product computation can be achieved with a statistically insignificant impact on performance across a wide range of benchmarks. We further showcase the versatility of AQUA by demonstrating its ability to synergistically accelerate existing token eviction methods like H2O and to directly reduce KV-cache memory size. By offering a controllable knob to balance efficiency and accuracy, AQUA provides a practical and powerful tool for making large-scale LLM inference more accessible and sustainable.

Method: The method employs meta-learning in generator training to iteratively capture meta-representations of synthetic data. These representations are adapted with few steps to produce data that helps the substitute model learn from the target model while minimizing distribution shifts.

Result: Experiments on MNIST, SVHN, CIFAR-10, and CIFAR-100 datasets show MetaDFME outperforms current state-of-the-art data-free model extraction methods and exhibits more stable substitute model accuracy during attacks.

Conclusion: MetaDFME effectively mitigates accuracy oscillation in data-free model extraction by using meta-learning to reduce distribution shift, making the attack more practical and stable.

Abstract: Model extraction is a severe threat to Machine Learning-as-a-Service systems, especially through data-free approaches, where dishonest users can replicate the functionality of a black-box target model without access to realistic data. Despite recent advancements, existing data-free model extraction methods suffer from the oscillating accuracy of the substitute model. This oscillation, which could be attributed to the constant shift in the generated data distribution during the attack, makes the attack impractical since the optimal substitute model cannot be determined without access to the target model’s in-distribution data. Hence, we propose MetaDFME, a novel data-free model extraction method that employs meta-learning in the generator training to reduce the distribution shift, aiming to mitigate the substitute model’s accuracy oscillation. In detail, we train our generator to iteratively capture the meta-representations of the synthetic data during the attack. These meta-representations can be adapted with a few steps to produce data that facilitates the substitute model to learn from the target model while reducing the effect of distribution shifts. Our experiments on popular baseline image datasets, MNIST, SVHN, CIFAR-10, and CIFAR-100, demonstrate that MetaDFME outperforms the current state-of-the-art data-free model extraction method while exhibiting a more stable substitute model’s accuracy during the attack.

Method: GK-SMOTE extends SMOTE using Gaussian Kernel Density Estimation (KDE) to automatically differentiate between safe and noisy regions in minority class data. It generates synthetic samples in high-density minority regions while avoiding ambiguous areas, requiring no parameter tuning.

Result: Extensive experiments on diverse binary classification datasets show GK-SMOTE outperforms state-of-the-art oversampling techniques across key metrics including MCC, Balanced Accuracy, and AUPRC.

Conclusion: GK-SMOTE provides a robust, efficient solution for imbalanced classification tasks, particularly in noisy data environments, making it suitable for real-world applications without requiring extensive parameter tuning.

Abstract: Imbalanced classification is a significant challenge in machine learning, especially in critical applications like medical diagnosis, fraud detection, and cybersecurity. Traditional oversampling techniques, such as SMOTE, often fail to handle label noise and complex data distributions, leading to reduced classification accuracy. In this paper, we propose GK-SMOTE, a hyperparameter-free, noise-resilient extension of SMOTE, built on Gaussian Kernel Density Estimation (KDE). GK-SMOTE enhances class separability by generating synthetic samples in high-density minority regions, while effectively avoiding noisy or ambiguous areas. This self-adaptive approach uses Gaussian KDE to differentiate between safe and noisy regions, ensuring more accurate sample generation without requiring extensive parameter tuning. Our extensive experiments on diverse binary classification datasets demonstrate that GK-SMOTE outperforms existing state-of-the-art oversampling techniques across key evaluation metrics, including MCC, Balanced Accuracy, and AUPRC. The proposed method offers a robust, efficient solution for imbalanced classification tasks, especially in noisy data environments, making it an attractive choice for real-world applications.

Method: OTA Merging uses optimizer second-moment statistics as curvature proxy to reweight parameter edits. FFG performs curvature-driven task-localization to sparsify conflicting edits. Also includes memory-light compression of second moments.

Result: Improves merged-model quality over strong baselines, reduces negative transfer, remains robust across sparsity levels. Shows substantial curvature overlap between checkpoints.

Conclusion: The approach provides effective model merging with reduced interference, offers insights into why linear merging works, and demonstrates practical effectiveness with open-sourced implementation.

Abstract: Model merging is an effective post-training strategy for composing capabilities in large language models without joint retraining. We study this in the supervised fine-tuning (SFT) stage, where multiple capability-based SFT checkpoints – spanning math, code, precise instruction following, general instruction following, and knowledge recall – must be consolidated into a single model. We introduce Optimization Trajectory Aware (OTA) Merging, a curvature-aware aggregation that leverages optimizer second-moment statistics as a diagonal curvature proxy to reweight parameter edits and mitigate interference. Complementing OTA, we propose Fast Fisher Grafting (FFG), a curvature-driven task-localization step that sparsifies conflicting or low-importance edits. FFG induces extremely low-rank masks concentrated in early attention query/key projections and token embeddings, exploiting shared curvature across capabilities. We further develop a memory-light compression of the second moments that preserves OTA’s effect. Across diverse capability-based SFT checkpoints, OTA+FFG improves merged-model quality over strong weight-space baselines, reduces negative transfer, and remains robust across sparsity levels. Analyses reveal substantial curvature overlap between checkpoints, offering a novel lens on why simple linear merging can be effective in practice. Ablations confirm that FFG is critical for reducing task interference and that the compressed second moments retain the gains of the full formulation. To facilitate reproducibility, we open-source all code, training and evaluation scripts, visualization artifacts, and capability-specific SFT checkpoints at https://github.com/pmahdavi/ota-merge.

Method: Proposes FedGDVE which: 1) uses pre-trained graph encoder for global structural features, 2) local valid predictor for client-specific relevance assessment, 3) reinforcement-learning-based probability estimator to filter and sample only the most pertinent global interactions.

Result: Achieves up to 34.86% performance improvement on recognized benchmarks in federated learning environments.

Conclusion: Selective augmentation of local client graphs with semantically aligned global samples effectively addresses data sparsity and bias issues in FL-based recommender systems while maintaining privacy and improving performance.

Abstract: Federated Learning (FL) is gaining prominence in machine learning as privacy concerns grow. This paradigm allows each client (e.g., an individual online store) to train a recommendation model locally while sharing only model updates, without exposing the raw interaction logs to a central server, thereby preserving privacy in a decentralized environment. Nonetheless, most existing FL-based recommender systems still rely solely on each client’s private data, despite the abundance of publicly available datasets that could be leveraged to enrich local training; this potential remains largely underexplored. To this end, we consider a realistic scenario wherein a large shopping platform collaborates with multiple small online stores to build a global recommender system. The platform possesses global data, such as shareable user and item lists, while each store holds a portion of interaction data privately (or locally). Although integrating global data can help mitigate the limitations of sparse and biased clients’ local data, it also introduces additional challenges: simply combining all global interactions can amplify noise and irrelevant patterns, worsening personalization and increasing computational costs. To address these challenges, we propose FedGDVE, which selectively augments each client’s local graph with semantically aligned samples from the global dataset. FedGDVE employs: (i) a pre-trained graph encoder to extract global structural features, (ii) a local valid predictor to assess client-specific relevance, (iii) a reinforcement-learning-based probability estimator to filter and sample only the most pertinent global interactions. FedGDVE improves performance by up to 34.86% on recognized benchmarks in FL environments.

Method: Uses algebraic programming theory (relational formalism) to derive dynamic programming solutions from executable recursive program specifications, enabling constructive algorithm development for ODT problems with arbitrary splitting rules.

Result: Four novel optimal algorithms for ODT problems that encompass existing depth-constrained axis-parallel methods as special cases, providing a unified framework for general ODT problems.

Conclusion: The framework enables efficient and elegant solutions for general ODT problems and is extendable to support even more flexible decision trees, including those with mixed splitting rules, as demonstrated in Part II.

Abstract: In the first paper (part I) of this series of two, we introduce four novel definitions of the ODT problems: three for size-constrained trees and one for depth-constrained trees. These definitions are stated unambiguously through executable recursive programs, satisfying all criteria we propose for a formal specification. In this sense, they resemble the “standard form” used in the study of general-purpose solvers. Grounded in algebraic programming theory-a relational formalism for deriving correct-by-construction algorithms from specifications-we can not only establish the existence or nonexistence of dynamic programming solutions but also derive them constructively whenever they exist. Consequently, the four generic problem definitions yield four novel optimal algorithms for ODT problems with arbitrary splitting rules that satisfy the axioms and objective functions of a given form. These algorithms encompass the known depth-constrained, axis-parallel ODT algorithm as the special case, while providing a unified, efficient, and elegant solution for the general ODT problem. In Part II, we present the first optimal hypersurface decision tree algorithm and provide comprehensive experiments against axis-parallel decision tree algorithms, including heuristic CART and state-of-the-art optimal methods. The results demonstrate the significant potential of decision trees with flexible splitting rules. Moreover, our framework is readily extendable to support algorithms for constructing even more flexible decision trees, including those with mixed splitting rules.

Method: Uses transformer-based network to generate multiple latent future states simultaneously instead of recurrent dynamics model, combined with Mean-Variance Constrained (MVC) evaluator to eliminate sequential visitation count dependence.

Result: Achieves up to 11x speedup in wall-clock time compared to MuZero while maintaining sample efficiency in MiniGrid and LunarLander environments.

Conclusion: Parallel tree construction through transformer-based state generation can substantially accelerate model-based reinforcement learning, enabling real-time decision-making in complex environments.

Abstract: We present TransZero, a model-based reinforcement learning algorithm that removes the sequential bottleneck in Monte Carlo Tree Search (MCTS). Unlike MuZero, which constructs its search tree step by step using a recurrent dynamics model, TransZero employs a transformer-based network to generate multiple latent future states simultaneously. Combined with the Mean-Variance Constrained (MVC) evaluator that eliminates dependence on inherently sequential visitation counts, our approach enables the parallel expansion of entire subtrees during planning. Experiments in MiniGrid and LunarLander show that TransZero achieves up to an eleven-fold speedup in wall-clock time compared to MuZero while maintaining sample efficiency. These results demonstrate that parallel tree construction can substantially accelerate model-based reinforcement learning, bringing real-time decision-making in complex environments closer to practice. The code is publicly available on GitHub.

Method: The authors analyze Riemannian online gradient descent for h-convex and strongly h-convex functions, establishing theoretical regret bounds. They validate their approach experimentally on the manifold of symmetric positive definite matrices with affine-invariant metric, testing on online Tyler’s M-estimation and online Fréchet mean computation.

Result: The paper establishes O(√T) regret for h-convex functions and O(log(T)) regret for strongly h-convex functions. These bounds are curvature-independent and match the Euclidean setting results. Experimental validation on SPD matrices demonstrates practical applications of h-convexity.

Conclusion: Horospherical convexity provides a powerful framework for online Riemannian optimization that overcomes the curvature-dependence limitations of geodesic convexity, achieving optimal regret bounds comparable to Euclidean optimization while being applicable to Hadamard manifolds.

Abstract: We study online Riemannian optimization on Hadamard manifolds under the framework of horospherical convexity (h-convexity). Prior work mostly relies on the geodesic convexity (g-convexity), leading to regret bounds scaling poorly with the manifold curvature. To address this limitation, we analyze Riemannian online gradient descent for h-convex and strongly h-convex functions and establish $O(\sqrt{T})$ and $O(\log(T))$ regret guarantees, respectively. These bounds are curvature-independent and match the results in the Euclidean setting. We validate our approach with experiments on the manifold of symmetric positive definite (SPD) matrices equipped with the affine-invariant metric. In particular, we investigate online Tyler’s $M$-estimation and online Fr'echet mean computation, showing the application of h-convexity in practice.

Method: Repurposes TabPFN (a transformer pre-trained on millions of synthetic datasets) as a Q function approximator using in-context learning. Uses a high reward episode gate to retain only top 5% trajectories due to fixed context budget, and employs principled truncation strategies for continual learning.

Result: Empirical evaluations show TabPFN RL matches or surpasses Deep Q Network on CartPole v1, MountainCar v0, and Acrobot v1 without gradient descent or extensive hyperparameter tuning, demonstrating surprising generalization capacity despite violated independence assumptions.

Conclusion: Prior-fitted networks like TabPFN establish a viable foundation for fast and computationally efficient RL, opening new directions for gradient-free RL with large pre-trained transformers.

Abstract: Gradient based optimization is fundamental to most modern deep reinforcement learning algorithms, however, it introduces significant sensitivity to hyperparameters, unstable training dynamics, and high computational costs. We propose TabPFN RL, a novel gradient free deep RL framework that repurposes the meta trained transformer TabPFN as a Q function approximator. Originally developed for tabular classification, TabPFN is a transformer pre trained on millions of synthetic datasets to perform inference on new unseen datasets via in context learning. Given an in context dataset of sample label pairs and new unlabeled data, it predicts the most likely labels in a single forward pass, without gradient updates or task specific fine tuning. We use TabPFN to predict Q values using inference only, thereby eliminating the need for back propagation at both training and inference. To cope with the model’s fixed context budget, we design a high reward episode gate that retains only the top 5% of trajectories. Empirical evaluations on the Gymnasium classic control suite demonstrate that TabPFN RL matches or surpasses Deep Q Network on CartPole v1, MountainCar v0, and Acrobot v1, without applying gradient descent or any extensive hyperparameter tuning. We discuss the theoretical aspects of how bootstrapped targets and non stationary visitation distributions violate the independence assumptions encoded in TabPFN’s prior, yet the model retains a surprising generalization capacity. We further formalize the intrinsic context size limit of in context RL algorithms and propose principled truncation strategies that enable continual learning when the context is full. Our results establish prior fitted networks such as TabPFN as a viable foundation for fast and computationally efficient RL, opening new directions for gradient free RL with large pre trained transformers.

Method: Uses K-fold cross-validation for confidence-based mismatch analysis to identify noisy/ambiguous samples. Selectively blends uncertain samples with confidently predicted peers from potential classes. Employs soft labels derived from all classes involved in mixing to align supervision with actual mixed inputs.

Result: Outperforms strong baseline methods on multiple synthetic (MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100) and real-world benchmark datasets (CIFAR-N, MNIST and Clothing1M).

Conclusion: SelectMix is effective and robust for learning with noisy labels, consistently demonstrating superior performance through both theoretical analysis and empirical evaluations.

Abstract: Deep neural networks tend to memorize noisy labels, severely degrading their generalization performance. Although Mixup has demonstrated effectiveness in improving generalization and robustness, existing Mixup-based methods typically perform indiscriminate mixing without principled guidance on sample selection and mixing strategy, inadvertently propagating noisy supervision. To overcome these limitations, we propose SelectMix, a confidence-guided mixing framework explicitly tailored for noisy labels. SelectMix first identifies potentially noisy or ambiguous samples through confidence based mismatch analysis using K-fold cross-validation, then selectively blends identified uncertain samples with confidently predicted peers from their potential classes. Furthermore, SelectMix employs soft labels derived from all classes involved in the mixing process, ensuring the labels accurately represent the composition of the mixed samples, thus aligning supervision signals closely with the actual mixed inputs. Through extensive theoretical analysis and empirical evaluations on multiple synthetic (MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100) and real-world benchmark datasets (CIFAR-N, MNIST and Clothing1M), we demonstrate that SelectMix consistently outperforms strong baseline methods, validating its effectiveness and robustness in learning with noisy labels.

Method: Developed a Python package implementing a new calibration technique that works for both binary and multi-class classification problems under dataset shift conditions.

Result: Empirical results show promising performance, with the method outperforming existing post-hoc calibration approaches in various settings including batch and online learning.

Conclusion: The technique is effective for calibration under dataset shift and can be valuable for classification problems where data distribution changes between training and testing phases.

Abstract: This paper introduces a new Python package specifically designed to address calibration of probabilistic classifiers under dataset shift. The method is demonstrated in binary and multi-class settings and its effectiveness is measured against a number of existing post-hoc calibration methods. The empirical results are promising and suggest that our technique can be helpful in a variety of settings for batch and online learning classification problems where the underlying data distribution changes between the training and test sets.

Method: Trains a physics-informed neural network to approximate reverse-time probability-flow dynamics, framing generative sampling as a PINN-style residual problem with endpoint anchoring. The network learns a direct map from 3D standard normal distribution to non-Gaussian Gaussian Mixture Models.

Result: Achieves NFE = 1 sampling with 10^4 samples in 16.54 ± 0.56 ms (vs 468-960 ms for baselines), preserves target distribution structure (MMD² = 1.88×10⁻²), and maintains accurate statistical properties (mean, covariance, skewness, kurtosis). Also validated on damped harmonic oscillator with MSE down to O(10⁻⁵).

Conclusion: PINGS provides a promising framework for fast, function-based generative sampling with constant-time generation, positioning it as a viable alternative to iterative ODE solvers and direct-map families, with potential applications in scientific simulation domains.

Abstract: We introduce PINGS (Physics-Informed Neural Network for Fast Generative Sampling), a framework that amortizes diffusion sampling by training a physics-informed network to approximate reverse-time probability-flow dynamics, reducing sampling to a single forward pass (NFE = 1). As a proof of concept, we learn a direct map from a 3D standard normal to a non-Gaussian Gaussian Mixture Model (GMM). PINGS preserves the target’s distributional structure (multi-bandwidth kernel $MMD^2 = 1.88 \times 10^{-2}$ with small errors in mean, covariance, skewness, and excess kurtosis) and achieves constant-time generation: $10^4$ samples in $16.54 \pm 0.56$ millisecond on an RTX 3090, versus 468-843 millisecond for DPM-Solver (10/20) and 960 millisecond for DDIM (50) under matched conditions. We also sanity-check the PINN/automatic-differentiation pipeline on a damped harmonic oscillator, obtaining MSEs down to $\mathcal{O}(10^{-5})$. Compared to fast but iterative ODE solvers and direct-map families (Flow, Rectified-Flow, Consistency), PINGS frames generative sampling as a PINN-style residual problem with endpoint anchoring, yielding a white-box, differentiable map with NFE = 1. These proof-of-concept results position PINGS as a promising route to fast, function-based generative sampling with potential extensions to scientific simulation (e.g., fast calorimetry).

Method: Combines frozen pre-trained feature extractor, compressed data buffer, and efficient non-iterative one-layer neural network for single-step optimization on fixed features.

Result: Achieves comparable accuracy to state-of-the-art methods while significantly improving training efficiency and reducing computational overhead.

Conclusion: CIFNet makes class-incremental learning more accessible and practical for resource-constrained environments when strong pre-trained feature extractors are available.

Abstract: Incremental learning remains a critical challenge in machine learning, as models often struggle with catastrophic forgetting -the tendency to lose previously acquired knowledge when learning new information. These challenges are even more pronounced in resource-limited settings. Many existing Class Incremental Learning (CIL) methods achieve high accuracy by continually adapting their feature representations; however, they often require substantial computational resources and complex, iterative training procedures. This work introduces CIFNet (Class Incremental and Frugal Network), a novel CIL approach that addresses these limitations by offering a highly efficient and sustainable solution. CIFNet’s key innovation lies in its novel integration of several existing, yet separately explored, components: a pre-trained and frozen feature extractor, a compressed data buffer, and an efficient non-iterative one-layer neural network for classification. A pre-trained and frozen feature extractor eliminates computationally expensive fine-tuning of the backbone. This, combined with a compressed buffer for efficient memory use, enables CIFNet to perform efficient class-incremental learning through a single-step optimization process on fixed features, minimizing computational overhead and training time without requiring multiple weight updates. Experiments on benchmark datasets confirm that CIFNet effectively mitigates catastrophic forgetting at the classifier level, achieving high accuracy comparable to that of existing state-of-the-art methods, while substantially improving training efficiency and sustainability. CIFNet represents a significant advancement in making class-incremental learning more accessible and pragmatic in environments with limited resources, especially when strong pre-trained feature extractors are available.

Method: Proposes an operator view with additive penalties and multiplicative pairwise weights, introduces GKPO schema with JSON serialization, canonicalization rules, and provides a Python reference library with adapters for DPO and RRHF.

Result: Developed a canonical schema that can represent various RLHF methods, demonstrated cross-method conversions when assumptions hold, and identified conditions where non-reducibility occurs through stress tests.

Conclusion: GKPO provides a standardized framework for RLHF methods, enabling better comparison, conversion between methods, and clear identification of when reduction assumptions fail.

Abstract: We present Opal, an operator view of reinforcement learning from human feedback (RLHF). Objectives are expressed as ladders of two primitives on a base utility: additive penalties and multiplicative pairwise weights. We describe a simple reduction law with if-and-only-if conditions: such ladders collapse to a normal form on pairwise margins when the reference is fixed, penalties are additive, and weights are independent of intermediate margins. When these assumptions do not hold (reference shift, non-additive gates, score-dependent weights), small examples demonstrate non-reducibility. Building on this view, we introduce GKPO (Generalized Kernel Preference Object), a canonical schema in which many RLHF methods can be represented and, when reducible, mapped back from. GKPO provides a standard JSON serialization, canonicalization and hashing rules, and explicit flags with finite witnesses when assumptions fail. We illustrate these ideas with GKPO examples for DPO, RRHF, and ORPO, along with cross-method conversions (where assumptions permit) and minimal stress tests (SHIFT/GATE/SCORE) that highlight non-reducibility. A lightweight Python reference library accompanies the schema, implementing canonical hashing and adapters for DPO and RRHF.

Method: Created multi-modal benchmark with 10 characterization methods using hybrid LLM + human-in-the-loop approach for high-quality question-answer pairs including multiple-choice and subjective questions.

Result: Advanced multi-modal AI models achieved nearly 90% accuracy on objective questions in materials data interpretation tasks.

Conclusion: AI models demonstrate strong potential for materials characterization applications, and the MatQnA dataset is publicly available for further research.

Abstract: Recently, large language models (LLMs) have achieved remarkable breakthroughs in general domains such as programming and writing, and have demonstrated strong potential in various scientific research scenarios. However, the capabilities of AI models in the highly specialized field of materials characterization and analysis have not yet been systematically or sufficiently validated. To address this gap, we present MatQnA, the first multi-modal benchmark dataset specifically designed for material characterization techniques. MatQnA includes ten mainstream characterization methods, such as X-ray Photoelectron Spectroscopy (XPS), X-ray Diffraction (XRD), Scanning Electron Microscopy (SEM), Transmission Electron Microscopy (TEM), etc. We employ a hybrid approach combining LLMs with human-in-the-loop validation to construct high-quality question-answer pairs, integrating both multiple-choice and subjective questions. Our preliminary evaluation results show that the most advanced multi-modal AI models (e.g., GPT-4.1, Claude 4, Gemini 2.5, and Doubao Vision Pro 32K) have already achieved nearly 90% accuracy on objective questions in materials data interpretation and analysis tasks, demonstrating strong potential for applications in materials characterization and analysis. The MatQnA dataset is publicly available at https://huggingface.co/datasets/richardhzgg/matQnA.

Method: Developed a theoretical framework to study escaping efficiency from local minima, analyzed centralized and decentralized strategies (consensus and diffusion), and conducted simulations to compare performance under different perturbation bounds and batch sizes.

Result: Decentralized algorithms escape local minima faster than centralized strategies when perturbation bound is small and batch size is large, favoring flatter minima. However, this advantage diminishes as perturbation bound increases.

Conclusion: Decentralized adversarial training strategies show potential to enhance model robustness in distributed settings, particularly under mild attack conditions, by more efficiently escaping local minima and finding flatter solutions.

Abstract: Adversarial training has been widely studied in recent years due to its role in improving model robustness against adversarial attacks. This paper focuses on comparing different distributed adversarial training algorithms–including centralized and decentralized strategies–within multi-agent learning environments. Previous studies have highlighted the importance of model flatness in determining robustness. To this end, we develop a general theoretical framework to study the escaping efficiency of these algorithms from local minima, which is closely related to the flatness of the resulting models. We show that when the perturbation bound is sufficiently small (i.e., when the attack strength is relatively mild) and a large batch size is used, decentralized adversarial training algorithms–including consensus and diffusion–are guaranteed to escape faster from local minima than the centralized strategy, thereby favoring flatter minima. However, as the perturbation bound increases, this trend may no longer hold. In the simulation results, we illustrate our theoretical findings and systematically compare the performance of models obtained through decentralized and centralized adversarial training algorithms. The results highlight the potential of decentralized strategies to enhance the robustness of models in distributed settings.

Method: Proposed BiLSTM-VHP architecture using bidirectional long short-term memory networks that works with 400-base nucleotide sequences. Created curated datasets for three viruses with thousands of sequences divided into multiple host classes.

Result: Achieved high prediction accuracy: 89.62% for orthohantavirus, 96.58% for rotavirus A, and 77.22% for rabies lyssavirus, outperforming previous studies. Performance assessed using confusion matrix, F-1 score, precision, recall, and microaverage AUC.

Conclusion: The BiLSTM-VHP model provides an effective lightweight solution for host prediction from viral sequences, with publicly available code and datasets to support zoonotic disease prevention efforts.

Abstract: Recorded history shows the long coexistence of humans and animals, suggesting it began much earlier. Despite some beneficial interdependence, many animals carry viral diseases that can spread to humans. These diseases are known as zoonotic diseases. Recent outbreaks of SARS-CoV-2, Monkeypox and swine flu viruses have shown how these viruses can disrupt human life and cause death. Fast and accurate predictions of the host from which the virus spreads can help prevent these diseases from spreading. This work presents BiLSTM-VHP, a lightweight bidirectional long short-term memory (LSTM)-based architecture that can predict the host from the nucleotide sequence of orthohantavirus, rabies lyssavirus, and rotavirus A with high accuracy. The proposed model works with nucleotide sequences of 400 bases in length and achieved a prediction accuracy of 89.62% for orthohantavirus, 96.58% for rotavirus A, and 77.22% for rabies lyssavirus outperforming previous studies. Moreover, performance of the model is assessed using the confusion matrix, F-1 score, precision, recall, microaverage AUC. In addition, we introduce three curated datasets of orthohantavirus, rotavirus A, and rabies lyssavirus containing 8,575, 95,197, and 22,052 nucleotide sequences divided into 9, 12, and 29 host classes, respectively. The codes and dataset are available at https://doi.org/10.17605/OSF.IO/ANFKR

Method: Two approaches: (1) hypervolume-guided weight adaptation and (2) gradient-based weight optimization, both designed to continuously balance objectives during online RL training with algorithms like GRPO, REINFORCE, and RLOO.

Result: Consistently achieves Pareto dominant solutions with fewer training steps than fixed-weight baselines, effective across multiple mathematical reasoning datasets and different model families.

Conclusion: Dynamic reward weighting provides a versatile toolkit for online multi-objective alignment, enabling effective exploration of Pareto fronts and superior performance compared to static weighting schemes.

Abstract: Prior works in multi-objective reinforcement learning typically use linear reward scalarization with fixed weights, which provably fail to capture non-convex Pareto fronts and thus yield suboptimal results. This limitation becomes especially critical in online preference alignment for large language models. Here, stochastic trajectories generated by parameterized policies create highly non-linear and non-convex mappings from parameters to objectives that no single static weighting scheme can find optimal trade-offs. We address this limitation by introducing dynamic reward weighting, which adaptively adjusts reward weights during the online reinforcement learning process. Unlike existing approaches that rely on fixed-weight interpolation, our dynamic weighting continuously balances and prioritizes objectives in training, facilitating effective exploration of Pareto fronts in objective space. We introduce two approaches of increasing sophistication and generalizability: (1) hypervolume-guided weight adaptation and (2) gradient-based weight optimization, offering a versatile toolkit for online multi-objective alignment. Our extensive experiments demonstrate their compatibility with commonly used online reinforcement learning algorithms (including GRPO, REINFORCE, and RLOO), effectiveness across multiple mathematical reasoning datasets, and applicability to different model families, consistently achieving Pareto dominant solutions with fewer training steps than fixed-weight linear scalarization baselines.

Method: Systematic analysis of dense and sparse gating regimes in MoEs, characterization of MoE-specific symmetries, development of a matching algorithm for expert and gating function alignment, and empirical validation across diverse MoE configurations and datasets.

Result: The paper demonstrates that LMC exists in MoE architectures after proper permutation alignment, confirming connectivity between independently trained models through linear paths with consistently low loss.

Conclusion: LMC is a fundamental property of MoE architectures, providing insights into their functional landscape and optimization dynamics, with implications for model ensembling and understanding neural network loss geometry.

Abstract: Linear Mode Connectivity (LMC) is a notable phenomenon in the loss landscapes of neural networks, wherein independently trained models have been observed to be connected–up to permutation symmetries–by linear paths in parameter space along which the loss remains consistently low. This observation challenges classical views of non-convex optimization and has implications for model ensembling, generalization, and our understanding of neural loss geometry. Inspired by recent studies on LMC in standard neural networks, we systematically investigate this phenomenon within Mixture-of-Experts (MoE) architectures–a class of models known for their scalability and computational efficiency, which combine traditional neural networks–referred to as experts–through a learnable gating mechanism. We begin by conducting a comprehensive analysis of both dense and sparse gating regimes, demonstrating that the symmetries inherent to MoE architectures are fully characterized by permutations acting on both the expert components and the gating function. Building on these foundational findings, we propose a matching algorithm that enables alignment between independently trained MoEs, thereby facilitating the discovery of LMC. Finally, we empirically validate the presence of LMC using our proposed algorithm across diverse MoE configurations–including dense, sparse, and shared-expert variants–under a wide range of model settings and datasets of varying scales and modalities. Our results confirm the existence of LMC in MoE architectures and offer fundamental insights into the functional landscape and optimization dynamics of deep learning models.

Method: Used a dataset of 3GPP sequence diagrams, compared puml outputs from Claude Sonnet and GPT-4V against manual ground truth, employed version control tools for difference detection, and introduced standard metrics for participant identification, message flow, sequence ordering, and grouping constructs.

Result: Nodes, edges and messages were accurately captured, but VLMs performed poorly on complex structures like notes, boxes, and groups.

Conclusion: The proposed metrics effectively quantify conversion errors, revealing a need for better representation of complex components in training data for fine-tuned VLMs.

Abstract: Telecom domain 3GPP documents are replete with images containing sequence diagrams. Advances in Vision-Language Large Models (VLMs) have eased conversion of such images to machine-readable PlantUML (puml) formats. However, there is a gap in evaluation of such conversions - existing works do not compare puml scripts for various components. In this work, we propose performance metrics to measure the effectiveness of such conversions. A dataset of sequence diagrams from 3GPP documents is chosen to be representative of domain-specific actual scenarios. We compare puml outputs from two VLMs - Claude Sonnet and GPT-4V - against manually created ground truth representations. We use version control tools to capture differences and introduce standard performance metrics to measure accuracies along various components: participant identification, message flow accuracy, sequence ordering, and grouping construct preservation. We demonstrate effectiveness of proposed metrics in quantifying conversion errors across various components of puml scripts. The results show that nodes, edges and messages are accurately captured. However, we observe that VLMs do not necessarily perform well on complex structures such as notes, box, groups. Our experiments and performance metrics indicates a need for better representation of these components in training data for fine-tuned VLMs.

Method: Error: OutputParser failed

Result: Error: OutputParser failed

Conclusion: Error: OutputParser failed

Abstract: We introduce and study the problem of online omniprediction with long-term constraints. At each round, a forecaster is tasked with generating predictions for an underlying (adaptively, adversarially chosen) state that are broadcast to a collection of downstream agents, who must each choose an action. Each of the downstream agents has both a utility function mapping actions and state to utilities, and a vector-valued constraint function mapping actions and states to vector-valued costs. The utility and constraint functions can arbitrarily differ across downstream agents. Their goal is to choose actions that guarantee themselves no regret while simultaneously guaranteeing that they do not cumulatively violate the constraints across time. We show how to make a single set of predictions so that each of the downstream agents can guarantee this by acting as a simple function of the predictions, guaranteeing each of them $\tilde{O}(\sqrt{T})$ regret and $O(1)$ cumulative constraint violation. We also show how to extend our guarantees to arbitrary intersecting contextually defined \emph{subsequences}, guaranteeing each agent both regret and constraint violation bounds not just marginally, but simultaneously on each subsequence, against a benchmark set of actions simultaneously tailored to each subsequence.

Method: Perform PCA on activations and module output gradients to identify subspaces containing common representations, collapse them before calculating unlearning updates to avoid affecting general representations and only target those specific to the unlearned facts.

Result: When unlearning WMDP dataset facts from Llama-3.1-8B, achieved 80x better post-attack accuracy drop than best baseline on biohazardous facts and 30x better on cyberhazardous facts, with only 0.1% WikiText loss increase and requiring less than 3 GPU-seconds per fact.

Conclusion: The proposed selective unlearning technique robustly removes dangerous knowledge while preserving general model performance, significantly outperforming existing methods in both effectiveness and efficiency.

Abstract: Current unlearning techniques and safety training consistently fail to remove dangerous knowledge from language models. We analyze the root causes and propose a highly selective technique which unlearns robustly and without disrupting general performance. We perform PCA on activations and module output gradients to identify subspaces containing common representations, and collapse them before calculating unlearning updates. This way we avoid unlearning general representations, and only target those specific to the unlearned facts. When unlearning WMDP dataset facts from Llama-3.1-8B, we drop post-attack accuracy 80x more than our best baseline (Circuit Breakers) on biohazardous facts and 30x more on cyberhazardous facts. Despite this, we disrupt general performance 30x less (only 0.1% WikiText loss increase), while requiring less than 3 GPU-seconds per fact.

Method: Created two datasets: CelebPersona (9,444 public figures) and AthlePersona (4,181 athletes). Used three high-performing LLMs to infer behavioral traits, combined with facial imagery and structured biographical features. Applied five statistical independence tests and introduced a novel causal representation learning framework with theoretical identifiability guarantees.

Result: The approach was validated on both synthetic and real-world data, demonstrating effectiveness in analyzing relationships between behavioral traits and other modalities through statistical testing and causal reasoning.

Conclusion: PersonaX provides a foundation for studying LLM-inferred behavioral traits alongside visual and biographical attributes, advancing multimodal trait analysis and causal reasoning in human behavior research.

Abstract: Understanding human behavior traits is central to applications in human-computer interaction, computational social science, and personalized AI systems. Such understanding often requires integrating multiple modalities to capture nuanced patterns and relationships. However, existing resources rarely provide datasets that combine behavioral descriptors with complementary modalities such as facial attributes and biographical information. To address this gap, we present PersonaX, a curated collection of multimodal datasets designed to enable comprehensive analysis of public traits across modalities. PersonaX consists of (1) CelebPersona, featuring 9444 public figures from diverse occupations, and (2) AthlePersona, covering 4181 professional athletes across 7 major sports leagues. Each dataset includes behavioral trait assessments inferred by three high-performing large language models, alongside facial imagery and structured biographical features. We analyze PersonaX at two complementary levels. First, we abstract high-level trait scores from text descriptions and apply five statistical independence tests to examine their relationships with other modalities. Second, we introduce a novel causal representation learning (CRL) framework tailored to multimodal and multi-measurement data, providing theoretical identifiability guarantees. Experiments on both synthetic and real-world data demonstrate the effectiveness of our approach. By unifying structured and unstructured analysis, PersonaX establishes a foundation for studying LLM-inferred behavioral traits in conjunction with visual and biographical attributes, advancing multimodal trait analysis and causal reasoning.

Method: Introduces edit operation-based measures to quantify deviations between state-action trajectories generated under stationary vs perturbed conditions, analyzing distributional changes in agent behavior sequences.

Result: The measures effectively distinguish drifted from non-drifted scenarios even under varying noise levels, demonstrating practical utility for drift detection in non-stationary RL environments.

Conclusion: The proposed framework provides a practical tool for detecting model drift in RL, addressing the challenge of non-stationary environments through trajectory analysis and edit-based deviation quantification.

Abstract: Reinforcement learning (RL) agents typically assume stationary environment dynamics. Yet in real-world applications such as healthcare, robotics, and finance, transition probabilities or reward functions may evolve, leading to model drift. This paper proposes a novel framework to detect such drifts by analyzing the distributional changes in sequences of agent behavior. Specifically, we introduce a suite of edit operation-based measures to quantify deviations between state-action trajectories generated under stationary and perturbed conditions. Our experiments demonstrate that these measures can effectively distinguish drifted from non-drifted scenarios, even under varying levels of noise, providing a practical tool for drift detection in non-stationary RL environments.

Method: Developed YNote music notation system, framed as text classification problem using TF-IDF for feature extraction and SMOTE for handling data imbalance.

Result: Achieved 98.25% accuracy in classifying music origins, demonstrating YNote retains sufficient stylistic information and can identify unique technological fingerprints of different AI generation methods.

Conclusion: YNote provides an effective framework for music representation and enables tracing of AI-generated content origins through identifiable technological fingerprints.

Abstract: With the rapid advancement of Large Language Models (LLMs), AI-driven music generation has become a vibrant and fruitful area of research. However, the representation of musical data remains a significant challenge. To address this, a novel, machine-learning-friendly music notation system, YNote, was developed. This study leverages YNote to train an effective classification model capable of distinguishing whether a piece of music was composed by a human (Native), a rule-based algorithm (Algorithm Generated), or an LLM (LLM Generated). We frame this as a text classification problem, applying the Term Frequency-Inverse Document Frequency (TF-IDF) algorithm to extract structural features from YNote sequences and using the Synthetic Minority Over-sampling Technique (SMOTE) to address data imbalance. The resulting model achieves an accuracy of 98.25%, successfully demonstrating that YNote retains sufficient stylistic information for analysis. More importantly, the model can identify the unique " technological fingerprints " left by different AI generation techniques, providing a powerful tool for tracing the origins of AI-generated content.

Method: Developed MillStone benchmark to systematically measure the effect of external arguments on LLM stances across controversial issues. Applied to nine leading LLMs to test openness to opposing arguments, inter-model agreement, and persuasiveness of different arguments.

Result: LLMs are generally open-minded on most issues but can be easily swayed by authoritative sources. Different LLMs show varying levels of agreement, and the persuasiveness of arguments varies across models.

Conclusion: The findings highlight the importance of careful source selection and the vulnerability of LLM-based information systems to manipulation through authoritative but potentially biased sources.

Abstract: Large language models equipped with Web search, information retrieval tools, and other agentic capabilities are beginning to supplant traditional search engines. As users start to rely on LLMs for information on many topics, including controversial and debatable issues, it is important to understand how the stances and opinions expressed in LLM outputs are influenced by the documents they use as their information sources. In this paper, we present MillStone, the first benchmark that aims to systematically measure the effect of external arguments on the stances that LLMs take on controversial issues (not all of them political). We apply MillStone to nine leading LLMs and measure how ``open-minded’’ they are to arguments supporting opposite sides of these issues, whether different LLMs agree with each other, which arguments LLMs find most persuasive, and whether these arguments are the same for different LLMs. In general, we find that LLMs are open-minded on most issues. An authoritative source of information can easily sway an LLM’s stance, highlighting the importance of source selection and the risk that LLM-based information retrieval and search systems can be manipulated.

Method: Integrated framework combining GPT-4o, Claude 4 Sonnet, and Gemini 2.5 Pro with prompt engineering, multimodal data fusion, domain-specific RAG with 50,000+ documents, chain-of-thought reasoning, and few-shot learning for reservoir analysis.

Result: Exceptional performance: 94.2% reservoir characterization accuracy, 87.6% production forecasting precision, 91.4% well placement success rate, sub-second response times, 96.2% safety reliability, 62-78% cost reductions, 72% faster field adaptation, 89% reasoning improvement, 96.2% anomaly detection, and 45% reduction in environmental incidents.

Conclusion: The research demonstrates practical integration of cutting-edge AI technologies with petroleum domain expertise for enhanced operational efficiency, safety, and economic performance in reservoir management.

Abstract: The petroleum industry faces unprecedented challenges in reservoir management, requiring rapid integration of complex multimodal datasets for real-time decision support. This study presents a novel integrated framework combining state-of-the-art large language models (GPT-4o, Claude 4 Sonnet, Gemini 2.5 Pro) with advanced prompt engineering techniques and multimodal data fusion for comprehensive reservoir analysis. The framework implements domain-specific retrieval-augmented generation (RAG) with over 50,000 petroleum engineering documents, chain-of-thought reasoning, and few-shot learning for rapid field adaptation. Multimodal integration processes seismic interpretations, well logs, and production data through specialized AI models with vision transformers. Field validation across 15 diverse reservoir environments demonstrates exceptional performance: 94.2% reservoir characterization accuracy, 87.6% production forecasting precision, and 91.4% well placement optimization success rate. The system achieves sub-second response times while maintaining 96.2% safety reliability with no high-risk incidents during evaluation. Economic analysis reveals 62-78% cost reductions (mean 72%) relative to traditional methods with 8-month payback period. Few-shot learning reduces field adaptation time by 72%, while automated prompt optimization achieves 89% improvement in reasoning quality. The framework processed real-time data streams with 96.2% anomaly detection accuracy and reduced environmental incidents by 45%. We provide detailed experimental protocols, baseline comparisons, ablation studies, and statistical significance testing to ensure reproducibility. This research demonstrates practical integration of cutting-edge AI technologies with petroleum domain expertise for enhanced operational efficiency, safety, and economic performance.

Method: Use SHAP for feature selection from black-box XGBoost model, then train glass-box models (EBM and PLTR) with selected features. Apply model refinement through feature interaction analysis, correlation checks, and expert input.

Result: Achieved comparable performance to black-box benchmark using only 10 features (88.5% reduction) on Lending Club dataset. Model refinement further enhanced interpretability and robustness.

Conclusion: The hybrid approach successfully bridges the gap between predictive performance and regulatory transparency requirements, demonstrating that glass-box models can achieve black-box level performance with significantly fewer features.

Abstract: Predicting default is essential for banks to ensure profitability and financial stability. While modern machine learning methods often outperform traditional regression techniques, their lack of transparency limits their use in regulated environments. Explainable artificial intelligence (XAI) has emerged as a solution in domains like credit scoring. However, most XAI research focuses on post-hoc interpretation of black-box models, which does not produce models lightweight or transparent enough to meet regulatory requirements, such as those for Internal Ratings-Based (IRB) models. This paper proposes a hybrid approach: post-hoc interpretations of black-box models guide feature selection, followed by training glass-box models that maintain both predictive power and transparency. Using the Lending Club dataset, we demonstrate that this approach achieves performance comparable to a benchmark black-box model while using only 10 features - an 88.5% reduction. In our example, SHapley Additive exPlanations (SHAP) is used for feature selection, eXtreme Gradient Boosting (XGBoost) serves as the benchmark and the base black-box model, and Explainable Boosting Machine (EBM) and Penalized Logistic Tree Regression (PLTR) are the investigated glass-box models. We also show that model refinement using feature interaction analysis, correlation checks, and expert input can further enhance model interpretability and robustness.

Method: Uses four key innovations: search space pruning, quantization proxy, quality predictor, and iterative search-and-update strategy to efficiently explore over 10^100 possible quantization configurations.

Result: AMQ efficiently reaches the Pareto frontier, producing compact and high-performing LLMs by optimally balancing quality and memory usage.

Conclusion: AMQ provides an effective automated solution for weight-only quantization that makes LLM deployment feasible under strict memory constraints.

Abstract: To enable broader deployment of Large Language Models (LLMs), it is essential to identify the best-performing model under strict memory constraints. We present AMQ, Automated Mixed-Precision Weight-Only Quantization, a framework that assigns layer-wise quantization bit-widths to optimally balance model quality and memory usage. However, the combinatorial search space, with over 10^{100} possible configurations, makes conventional black-box optimization infeasible. AMQ overcomes this challenge through four key innovations:(1) search space pruning using prior knowledge to exclude unpromising configurations, (2) quantization proxy to bypass costly format conversions during search, (3) quality predictor to minimize evaluation overhead, and (4) iterative search-and-update strategy for fast and stable convergence. By integrating these components, AMQ efficiently explores the quality-efficiency landscape, reaching the Pareto frontier and yielding LLMs that are both compact and high-performing. Our code is available at https://github.com/dlwns147/amq.

Method: Proposes framing AI red-teaming as a domain-specific evolution of cyber red-teaming, enabling both cyber and AI red teams to leverage each other’s strengths - cyber teams can apply structured adversary emulation to AI systems, while AI teams can benefit from established cybersecurity frameworks.

Result: The integration creates a more robust security ecosystem that can better adapt to evolving threats by recognizing AI’s new risks, failure modes, and unique mitigation requirements.

Conclusion: Merging AI and cyber red-teaming approaches provides the best positioning for the security community to address the rapidly changing threat landscape posed by AI systems through mutual accountability, formal engagement rules, and mature tooling patterns.

Abstract: A red team simulates adversary attacks to help defenders find effective strategies to defend their systems in a real-world operational setting. As more enterprise systems adopt AI, red-teaming will need to evolve to address the unique vulnerabilities and risks posed by AI systems. We take the position that AI systems can be more effectively red-teamed if AI red-teaming is recognized as a domain-specific evolution of cyber red-teaming. Specifically, we argue that existing Cyber Red Teams who adopt this framing will be able to better evaluate systems with AI components by recognizing that AI poses new risks, has new failure modes to exploit, and often contains unpatchable bugs that re-prioritize disclosure and mitigation strategies. Similarly, adopting a cybersecurity framing will allow existing AI Red Teams to leverage a well-tested structure to emulate realistic adversaries, promote mutual accountability with formal rules of engagement, and provide a pattern to mature the tooling necessary for repeatable, scalable engagements. In these ways, the merging of AI and Cyber Red Teams will create a robust security ecosystem and best position the community to adapt to the rapidly changing threat landscape.

Method: Developed FlexBench as a modular extension integrated with HuggingFace, collecting benchmarking results and metadata into an Open MLPerf Dataset for collaborative curation and predictive modeling. Validated through MLPerf Inference submissions including evaluations of DeepSeek R1 and LLaMA 3.3 on commodity servers.

Result: Successfully validated the FlexBench concept through MLPerf Inference submissions, demonstrating its effectiveness in providing relevant and actionable insights for AI system evaluation.

Conclusion: FlexBench enables practitioners to make cost-effective AI deployment decisions that reflect their available resources, requirements, and constraints by treating benchmarking as a continuous AI optimization task.

Abstract: Existing AI system benchmarks such as MLPerf often struggle to keep pace with the rapidly evolving AI landscape, making it difficult to support informed deployment, optimization, and co-design decisions for AI systems. We suggest that benchmarking itself can be framed as an AI task - one in which models are continuously evaluated and optimized across diverse datasets, software, and hardware, using key metrics such as accuracy, latency, throughput, energy consumption, and cost. To support this perspective, we present FlexBench: a modular extension of the MLPerf LLM inference benchmark, integrated with HuggingFace and designed to provide relevant and actionable insights. Benchmarking results and metadata are collected into an Open MLPerf Dataset, which can be collaboratively curated, extended, and leveraged for predictive modeling and feature engineering. We successfully validated the FlexBench concept through MLPerf Inference submissions, including evaluations of DeepSeek R1 and LLaMA 3.3 on commodity servers. The broader objective is to enable practitioners to make cost-effective AI deployment decisions that reflect their available resources, requirements, and constraints.

Method: Develops a general approach showing gradient descent iterates concentrate around a deterministic ‘Gaussian theoretical gradient descent’ vector, tracked by a state evolution system of two recursive equations for two scalars.

Result: Proves universal concentration guarantees for broad classes of design matrices over long time horizons, reveals implicit regularization effects, demonstrates global convergence for structured link functions, and establishes universality in phase retrieval.

Conclusion: The framework provides a statistically valid, low-cost tool for practical tasks like hyperparameter tuning and connects Gaussian theoretical gradient descent with dynamical mean-field theory, offering broad applicability across different regression settings.

Abstract: This paper develops a general approach to characterize the long-time trajectory behavior of nonconvex gradient descent in generalized single-index models in the large aspect ratio regime. In this regime, we show that for each iteration the gradient descent iterate concentrates around a deterministic vector called the Gaussian theoretical gradient descent', whose dynamics can be tracked by a state evolution system of two recursive equations for two scalars. Our concentration guarantees hold universally for a broad class of design matrices and remain valid over long time horizons until algorithmic convergence or divergence occurs. Moreover, our approach reveals that gradient descent iterates are in general approximately independent of the data and strongly incoherent with the feature vectors, a phenomenon previously known as the implicit regularization’ effect of gradient descent in specific models under Gaussian data. As an illustration of the utility of our general theory, we present two applications of different natures in the regression setting. In the first, we prove global convergence of nonconvex gradient descent with general independent initialization for a broad class of structured link functions, and establish universality of randomly initialized gradient descent in phase retrieval for large aspect ratios. In the second, we develop a data-free iterative algorithm for estimating state evolution parameters along the entire gradient descent trajectory, thereby providing a low-cost yet statistically valid tool for practical tasks such as hyperparameter tuning and runtime determination. As a by-product of our analysis, we show that in the large aspect ratio regime, the Gaussian theoretical gradient descent coincides with a recent line of dynamical mean-field theory for gradient descent over the constant-time horizon.

Method: Analyzed 23,301 EV crash records, used XGBoost/Random Forest for feature importance, applied SMOTEENN resampling for class imbalance, and benchmarked three deep tabular models (TabPFN, MambaNet, MambaAttention).

Result: TabPFN showed strong generalization, while MambaAttention achieved superior performance in classifying severe injury cases due to its attention-based feature reweighting capability.

Conclusion: Deep tabular architectures show significant potential for improving crash severity prediction in EV contexts and enabling data-driven safety interventions.

Abstract: This study presents a deep tabular learning framework for predicting crash severity in electric vehicle (EV) collisions using real-world crash data from Texas (2017-2023). After filtering for electric-only vehicles, 23,301 EV-involved crash records were analyzed. Feature importance techniques using XGBoost and Random Forest identified intersection relation, first harmful event, person age, crash speed limit, and day of week as the top predictors, along with advanced safety features like automatic emergency braking. To address class imbalance, Synthetic Minority Over-sampling Technique and Edited Nearest Neighbors (SMOTEENN) resampling was applied. Three state-of-the-art deep tabular models, TabPFN, MambaNet, and MambaAttention, were benchmarked for severity prediction. While TabPFN demonstrated strong generalization, MambaAttention achieved superior performance in classifying severe injury cases due to its attention-based feature reweighting. The findings highlight the potential of deep tabular architectures for improving crash severity prediction and enabling data-driven safety interventions in EV crash contexts.

Method: Unsupervised deep embedded clustering using autoencoders to compress multi-omic data into latent embeddings, followed by supervised graph neural network link prediction. 9,022 drugs were partitioned into 35 clusters, and GNNs were used for link prediction.

Result: Achieved mean silhouette score of 0.8550 for clustering, prediction accuracy of 0.901, ROC AUC of 0.960, and F1-Score of 0.901. Generated 477 high-confidence drug-disease links with probabilities exceeding 99%.

Conclusion: The proposed pipeline successfully identifies novel drug-disease connections and advances machine learning applications in drug repurposing, potentially providing new therapeutic prospects across diverse disease domains.

Abstract: Drug repurposing has historically been an economically infeasible process for identifying novel uses for abandoned drugs. Modern machine learning has enabled the identification of complex biochemical intricacies in candidate drugs; however, many studies rely on simplified datasets with known drug-disease similarities. We propose a machine learning pipeline that uses unsupervised deep embedded clustering, combined with supervised graph neural network link prediction to identify new drug-disease links from multi-omic data. Unsupervised autoencoder and cluster training reduced the dimensionality of omic data into a compressed latent embedding. A total of 9,022 unique drugs were partitioned into 35 clusters with a mean silhouette score of 0.8550. Graph neural networks achieved strong statistical performance, with a prediction accuracy of 0.901, receiver operating characteristic area under the curve of 0.960, and F1-Score of 0.901. A ranked list comprised of 477 per-cluster link probabilities exceeding 99 percent was generated. This study could provide new drug-disease link prospects across unrelated disease domains, while advancing the understanding of machine learning in drug repurposing studies.

Method: Uses simple additive recurrent structure to learn embeddings. Theoretical analysis shows convergence to ideal additive structure in Euclidean space. Also introduces hyperbolic space variant for hierarchical data.

Result: Learned representations converge to vector sum of constituent events (linear additive hypothesis). Hyperbolic model shows improved performance on hierarchical event sequences with low distortion.

Conclusion: The framework provides theoretically grounded, interpretable embeddings for event sequences, with hyperbolic geometry offering superior performance for hierarchical data structures.

Abstract: The study of neural representations, both in biological and artificial systems, is increasingly revealing the importance of geometric and topological structures. Inspired by this, we introduce Event2Vec, a novel framework for learning representations of discrete event sequences. Our model leverages a simple, additive recurrent structure to learn composable, interpretable embeddings. We provide a theoretical analysis demonstrating that, under specific training objectives, our model’s learned representations in a Euclidean space converge to an ideal additive structure. This ensures that the representation of a sequence is the vector sum of its constituent events, a property we term the linear additive hypothesis. To address the limitations of Euclidean geometry for hierarchical data, we also introduce a variant of our model in hyperbolic space, which is naturally suited to embedding tree-like structures with low distortion. We present experiments to validate our hypothesis and demonstrate the benefits of each geometry, highlighting the improved performance of the hyperbolic model on hierarchical event sequences.

Method: Developed ML models trained on a strategically designed synthetic dataset with features from multiple spectroscopy techniques, using innovative task-specific loss functions like vicinity peak response (ViPeR) for peak localization.

Result: OASIS successfully validated with experimental data from Raman, UV-vis, and fluorescence spectroscopy, demonstrating high accuracy in automated spectral analysis.

Conclusion: The framework shows significant potential for in situ experiments, high-throughput optimization, and online monitoring, with loss function optimization identified as a key resource-efficient strategy for developing high-performance ML models.

Abstract: The proliferation of spectroscopic data across various scientific and engineering fields necessitates automated processing. We introduce OASIS (Omni-purpose Analysis of Spectra via Intelligent Systems), a machine learning (ML) framework for technique-independent, automated spectral analysis, encompassing denoising, baseline correction, and comprehensive peak parameter (location, intensity, FWHM) retrieval without human intervention. OASIS achieves its versatility through models trained on a strategically designed synthetic dataset incorporating features from numerous spectroscopy techniques. Critically, the development of innovative, task-specific loss functions-such as the vicinity peak response (ViPeR) for peak localization-enabled the creation of compact yet highly accurate models from this dataset, validated with experimental data from Raman, UV-vis, and fluorescence spectroscopy. OASIS demonstrates significant potential for applications including in situ experiments, high-throughput optimization, and online monitoring. This study underscores the optimization of the loss function as a key resource-efficient strategy to develop high-performance ML models.

Method: Uses Deferral Classifiers at each layer to assess sample hardness before allowing early exits, deferring hard samples to expert layers to prevent hallucinations.

Result: Reduces wrong prediction risk by 50% with 2.05× speedup compared to final layer inference, improving both accuracy and latency.

Conclusion: Combining Early Exit with Selective Prediction through Deferral Classifiers creates trustworthy and efficient DNNs suitable for critical applications.

Abstract: Inference latency and trustworthiness of Deep Neural Networks (DNNs) are the bottlenecks in deploying them in critical applications like sensitive tasks. Early Exit (EE) DNNs overcome the latency issues by allowing samples to exit from intermediary layers if they attain high' confidence scores on the predicted class. However, the DNNs are known to exhibit overconfidence, which can lead to many samples exiting early and render EE strategies untrustworthy. We use Selective Prediction (SP) to overcome this issue by checking the hardness’ of the samples rather than just relying on the confidence score alone. We propose SPEED, a novel approach that uses Deferral Classifiers (DCs) at each layer to check the hardness of samples before performing EEs. Specifically, the DCs identify if a sample is hard to predict at an intermediary layer, leading to hallucination, and defer it to an expert. Early detection of hard samples for inference prevents the wastage of computational resources and improves trust by deferring the hard samples to the expert. We demonstrate that EE aided with SP improves both accuracy and latency. Our method minimizes the risk of wrong prediction by $50%$ with a speedup of $2.05\times$ as compared to the final layer. The anonymized source code is available at https://github.com/Div290/SPEED

Method: Proposed Dice Adversarial Robustness Distillation (DARD) - a knowledge distillation paradigm to transfer robustness from large teacher models to compact student models. Also introduced Dice Projected Gradient Descent (DPGD) for optimized adversarial attacks.

Result: Extensive experiments show DARD consistently outperforms adversarially trained networks with same architecture, achieving superior robustness and standard accuracy.

Conclusion: Robustness can be systematically distilled from large teacher models to compact student models through the proposed DARD approach, providing an effective defense against adversarial attacks without sacrificing standard performance.

Abstract: Deep learning models are vulnerable to adversarial examples, posing critical security challenges in real-world applications. While Adversarial Training (AT ) is a widely adopted defense mechanism to enhance robustness, it often incurs a trade-off by degrading performance on unperturbed, natural data. Recent efforts have highlighted that larger models exhibit enhanced robustness over their smaller counterparts. In this paper, we empirically demonstrate that such robustness can be systematically distilled from large teacher models into compact student models. To achieve better performance, we introduce Dice Adversarial Robustness Distillation (DARD), a novel method designed to transfer robustness through a tailored knowledge distillation paradigm. Additionally, we propose Dice Projected Gradient Descent (DPGD), an adversarial example generalization method optimized for effective attack. Our extensive experiments demonstrate that the DARD approach consistently outperforms adversarially trained networks with the same architecture, achieving superior robustness and standard accuracy.

Method: Semi-online RL simulates online RL on offline trajectories using a Patch Module to recover divergence between rollout and expert trajectories, with discounted future returns and weighted step-level/episode-level advantages.

Result: Achieves SOTA performance among 7B models across four benchmarks (+12.0% on AndroidWorld, +23.8% on AITW), significantly bridging offline training efficiency and online multi-turn reasoning gap.

Conclusion: Semi-online RL provides an effective paradigm that combines the stability of offline training with the multi-step reasoning capabilities of online RL, with SOP metric serving as a practical proxy for real-world evaluation.

Abstract: Graphical User Interface (GUI) agents have demonstrated remarkable progress in automating complex user interface interactions through reinforcement learning. However, current approaches face a fundamental dilemma: offline RL enables stable training on pre-collected trajectories, but struggles with multi-step task execution for lack of trajectory-level reward signals; online RL captures these signals through environment interaction, but suffers from sparse rewards and prohibitive deployment costs. To address it, we present Semi-online Reinforcement Learning, a novel paradigm that simulates online RL on offline trajectories. During each rollout process, we preserve the original model output within the multi-turn dialogue, where a Patch Module adaptively recovers the divergence between rollout and expert trajectories. To capture long-term training signals, Semi-online RL introduces discounted future returns into the reward computation and optimizes the policy with weighted step-level and episode-level advantages. We further introduce Semi-Online Performance (SOP), a metric that aligns better with true online performance, serving as a practical and effective proxy for real-world evaluation. Experiments show that ours Semi-online RL achieves SOTA performance among 7B models across four dynamic benchmarks, with significant gains over the base model (e.g., +12.0% on AndroidWorld, +23.8% on AITW), demonstrating significant progress in bridging the gap between offline training efficiency and online multi-turn reasoning. The code is available at https://github.com/X-PLUG/MobileAgent/tree/main/UI-S1.

Method: Mathematical formulation of compressed sensing, analysis of its logic and potential pathologies, and application to real-world signals.

Result: Not explicitly stated in the abstract, but the paper explores the theoretical foundations and practical applications of compressed sensing.

Conclusion: Compressed sensing provides a powerful framework for reconstructing signals from limited measurements by exploiting their inherent sparsity properties.

Abstract: Compressed sensing is a signal processing technique that allows for the reconstruction of a signal from a small set of measurements. The key idea behind compressed sensing is that many real-world signals are inherently sparse, meaning that they can be efficiently represented in a different space with only a few components compared to their original space representation. In this paper we will explore the mathematical formulation behind compressed sensing, its logic and pathologies, and apply compressed sensing to real world signals.

Method: DAPNet uses a Mixture-of-Experts architecture with three specialized networks: periodic analysis expert, dynamic correlation modeling expert, and hybrid temporal feature extraction expert. A learnable gating network dynamically weights expert outputs, and a hybrid regularization loss addresses class imbalance and ensures stable training.

Result: Extensive experiments on CICIDS2017/2018 intrusion detection datasets show DAPNet achieves higher accuracy. Evaluation on ten UEA benchmark datasets demonstrates the framework’s generalizability for network state classification.

Conclusion: DAPNet successfully addresses the trade-off between temporal pattern capture and variable dependency modeling, providing an effective specialized framework for network state classification with demonstrated accuracy and generalizability across multiple datasets.

Abstract: Effective network state classification is a primary task for ensuring network security and optimizing performance. Existing deep learning models have shown considerable progress in this area. Some methods excel at analyzing the complex temporal periodicities found in traffic data, while graph-based approaches are adept at modeling the dynamic dependencies between different variables. However, a key trade-off remains, as these methods struggle to capture both characteristics simultaneously. Models focused on temporal patterns often overlook crucial variable dependencies, whereas those centered on dependencies may fail to capture fine-grained temporal details. To address this trade-off, we introduce DAPNet, a framework based on a Mixture-of-Experts architecture. DAPNet integrates three specialized networks for periodic analysis, dynamic cross-variable correlation modeling, and hybrid temporal feature extraction. A learnable gating network dynamically assigns weights to experts based on the input sample and computes a weighted fusion of their outputs. Furthermore, a hybrid regularization loss function ensures stable training and addresses the common issue of class imbalance. Extensive experiments on two large-scale network intrusion detection datasets (CICIDS2017/2018) validate DAPNet’s higher accuracy for its target application. The generalizability of the architectural design is evaluated across ten public UEA benchmark datasets, positioning DAPNet as a specialized framework for network state classification.

Method: Apply persistent GLMY homology theory to study reservoir topology, develop optimization method by modifying minimal representative cycles of one-dimensional GLMY homology groups.

Result: Found that reservoir performance is closely related to one-dimensional GLMY homology groups. Performance is jointly influenced by reservoir structure and dataset periodicity.

Conclusion: Persistent GLMY homology provides effective mathematical framework for analyzing and optimizing reservoir network topology to improve time series processing performance.

Abstract: Reservoir is an efficient network for time series processing. It is well known that network structure is one of the determinants of its performance. However, the topology structure of reservoirs, as well as their performance, is hard to analyzed, due to the lack of suitable mathematical tools. In this paper, we study the topology structure of reservoirs using persistent GLMY homology theory, and develop a method to improve its performance. Specifically, it is found that the reservoir performance is closely related to the one-dimensional GLMY homology groups. Then, we develop a reservoir structure optimization method by modifying the minimal representative cycles of one-dimensional GLMY homology groups. Finally, by experiments, it is validated that the performance of reservoirs is jointly influenced by the reservoir structure and the periodicity of the dataset.

Method: Proposes an algorithm that perturbs model weights to maximize uncertainty on protected instances while preserving utility. Uses Pareto optimal objective balancing privacy vs utility, with a certifiable approximation algorithm providing (ε, δ) guarantees.

Result: Achieves >3× stronger uncertainty than pretraining with <0.2% accuracy drops on various image recognition benchmarks. Provides tight, non-vacuous bounds on privacy-utility tradeoff.

Conclusion: The framework offers effective protection against test-time privacy threats, ensuring models become uncertain on protected data while maintaining performance on legitimate inputs.

Abstract: Unlearning is the predominant method for removing the influence of data in machine learning models. However, even after unlearning, models often continue to produce the same predictions on the unlearned data with high confidence. This persistent behavior can be exploited by adversaries using confident model predictions on incorrect or obsolete data to harm users. We call this threat model, which unlearning fails to protect against, test-time privacy. In particular, an adversary with full model access can bypass any naive defenses which ensure test-time privacy. To address this threat, we introduce an algorithm which perturbs model weights to induce maximal uncertainty on protected instances while preserving accuracy on the rest of the instances. Our core algorithm is based on finetuning with a Pareto optimal objective that explicitly balances test-time privacy against utility. We also provide a certifiable approximation algorithm which achieves $(\varepsilon, \delta)$ guarantees without convexity assumptions. We then prove a tight, non-vacuous bound that characterizes the privacy-utility tradeoff that our algorithms incur. Empirically, our method obtains $>3\times$ stronger uncertainty than pretraining with $<0.2%$ drops in accuracy on various image recognition benchmarks. Altogether, this framework provides a tool to guarantee additional protection to end users.

Method: SpeCa uses speculative sampling to predict intermediate features for future timesteps based on reference timesteps, with parameter-free verification and sample-adaptive computation allocation that dynamically adjusts resources based on generation complexity.

Result: 6.34x acceleration on FLUX (5.5% quality drop), 7.3x speedup on DiT with preserved fidelity, 79.84% VBench score at 6.1x acceleration for HunyuanVideo, with verification overhead of only 1.67%-3.5% of full inference costs.

Conclusion: SpeCa establishes a new paradigm for efficient diffusion model inference, achieving significant acceleration while maintaining generation quality through speculative sampling and adaptive computation strategies.

Abstract: Diffusion models have revolutionized high-fidelity image and video synthesis, yet their computational demands remain prohibitive for real-time applications. These models face two fundamental challenges: strict temporal dependencies preventing parallelization, and computationally intensive forward passes required at each denoising step. Drawing inspiration from speculative decoding in large language models, we present SpeCa, a novel ‘Forecast-then-verify’ acceleration framework that effectively addresses both limitations. SpeCa’s core innovation lies in introducing Speculative Sampling to diffusion models, predicting intermediate features for subsequent timesteps based on fully computed reference timesteps. Our approach implements a parameter-free verification mechanism that efficiently evaluates prediction reliability, enabling real-time decisions to accept or reject each prediction while incurring negligible computational overhead. Furthermore, SpeCa introduces sample-adaptive computation allocation that dynamically modulates resources based on generation complexity, allocating reduced computation for simpler samples while preserving intensive processing for complex instances. Experiments demonstrate 6.34x acceleration on FLUX with minimal quality degradation (5.5% drop), 7.3x speedup on DiT while preserving generation fidelity, and 79.84% VBench score at 6.1x acceleration for HunyuanVideo. The verification mechanism incurs minimal overhead (1.67%-3.5% of full inference costs), establishing a new paradigm for efficient diffusion model inference while maintaining generation quality even at aggressive acceleration ratios. Our codes have been released in Github: \textbf{https://github.com/Shenyi-Z/Cache4Diffusion}

Method: Constructed Reasoned Safety Alignment (ReSA) dataset with 80K examples to teach models to reason through direct responses and analyze safety. The Answer-Then-Check approach enables models to answer questions in thought first, then evaluate safety before deciding final response.

Result: Achieves Pareto frontier with superior safety capability while decreasing over-refusal rates. Maintains general reasoning capabilities on benchmarks (MMLU, MATH500, HumanEval). Enables safe completion - providing helpful alternative responses for sensitive topics. Training on just 500 examples achieves comparable performance to full dataset.

Conclusion: Safety alignment may require less data than previously assumed. The method provides robust protection against jailbreak attacks while maintaining model capabilities and enabling helpful responses even for sensitive topics.

Abstract: As large language models (LLMs) continue to advance in capabilities, ensuring their safety against jailbreak attacks remains a critical challenge. In this paper, we introduce a novel safety alignment approach called Answer-Then-Check, which enhances LLM robustness against malicious prompts by applying thinking ability to mitigate jailbreaking problems before producing a final answer to the user. Our method enables models to directly answer the question in their thought and then critically evaluate its safety before deciding whether to provide it. To implement this approach, we construct the Reasoned Safety Alignment (ReSA) dataset, comprising 80K examples that teach models to reason through direct responses and then analyze their safety. Experimental results demonstrate that our approach achieves the Pareto frontier with superior safety capability while decreasing over-refusal rates on over-refusal benchmarks. Notably, the model fine-tuned with ReSA maintains general reasoning capabilities on benchmarks like MMLU, MATH500, and HumanEval. Besides, our method equips models with the ability to perform safe completion. Unlike post-hoc methods that can only reject harmful queries, our model can provide helpful and safe alternative responses for sensitive topics (e.g., self-harm). Furthermore, we discover that training on a small subset of just 500 examples can achieve comparable performance to using the full dataset, suggesting that safety alignment may require less data than previously assumed.

Method: Integrates temporal multi-tensor sketching with Count-Min Sketch using Conservative Update (CMS-CU) for compact edge frequency tracking, Bayesian inference for probabilistic anomaly scoring, and Exponentially Weighted Moving Average (EWMA) for adaptive thresholding.

Result: Outperforms state-of-the-art baselines with up to 6.5% AUC gain on CIC-IDS2018 and 15.6% on CIC-DDoS2019, processing 20 million edges in under 3.4 seconds using only 10 hash functions.

Conclusion: ADAPTIVE-GRAPHSKETCH is practical and effective for fast, accurate anomaly detection in large-scale streaming graphs, addressing key limitations of existing approaches.

Abstract: Anomaly detection in dynamic graphs is essential for identifying malicious activities, fraud, and unexpected behaviors in real-world systems such as cybersecurity and power grids. However, existing approaches struggle with scalability, probabilistic interpretability, and adaptability to evolving traffic patterns. In this paper, we propose ADAPTIVE-GRAPHSKETCH, a lightweight and scalable framework for real-time anomaly detection in streaming edge data. Our method integrates temporal multi-tensor sketching with Count-Min Sketch using Conservative Update (CMS-CU) to compactly track edge frequency patterns with bounded memory, while mitigating hash collision issues. We incorporate Bayesian inference for probabilistic anomaly scoring and apply Exponentially Weighted Moving Average (EWMA) for adaptive thresholding tuned to burst intensity. Extensive experiments on four real-world intrusion detection datasets demonstrate that ADAPTIVE-GRAPHSKETCH outperforms state-of-the-art baselines such as ANOEDGE-G/L, MIDAS-R, and F-FADE, achieving up to 6.5% AUC gain on CIC-IDS2018 and up to 15.6% on CIC-DDoS2019, while processing 20 million edges in under 3.4 seconds using only 10 hash functions. Our results show that ADAPTIVE-GRAPHSKETCH is practical and effective for fast, accurate anomaly detection in large-scale streaming graphs. Keywords: Anomaly Detection, Streaming, Real-time, Dynamic Graphs, Edge Streams, Tensor Sketching

Method: Researchers curated a comprehensive dataset by collecting data from multiple online sources for billion-dollar disasters at the county level, and analyzed how online estimates compare with traditional offline-based impact estimates.

Result: The study found that different online sources provide complementary information for disaster assessment, offering insights into how various data streams can work together to evaluate disaster impact.

Conclusion: Online data sources serve as valuable complementary tools to traditional offline methods, providing real-time assessment capabilities that can enhance disaster response planning through multiple information streams.

Abstract: Assessing the impact of a disaster in terms of asset losses and human casualties is essential for preparing effective response plans. Traditional methods include offline assessments conducted on the ground, where volunteers and first responders work together to collect the estimate of losses through windshield surveys or on-ground inspection. However, these methods have a time delay and are prone to different biases. Recently, various online data sources, including social media, news reports, aerial imagery, and satellite data, have been utilized to evaluate the impact of disasters. Online data sources provide real-time data streams for estimating the offline impact. Limited research exists on how different online sources help estimate disaster impact at a given administrative unit. In our work, we curate a comprehensive dataset by collecting data from multiple online sources for a few billion-dollar disasters at the county level. We also analyze how online estimates compare with traditional offline-based impact estimates for the disaster. Our findings provide insight into how different sources can provide complementary information to assess the disaster.

Method: Leverages real-time data sources (satellite imagery, news, social media) to create an interventional tool that allows users to modify input states and simulate different scenarios.

Result: Developed a tool that complements traditional disaster modeling by providing causal attribution of different factors on estimated severity and offering actionable recourses for mitigation planning.

Conclusion: The interventional approach provides valuable complementary capabilities to traditional predictive disaster modeling, enabling better understanding of causal relationships and supporting mitigation planning through “what-if” scenario simulation.

Abstract: Traditional disaster analysis and modelling tools for assessing the severity of a disaster are predictive in nature. Based on the past observational data, these tools prescribe how the current input state (e.g., environmental conditions, situation reports) results in a severity assessment. However, these systems are not meant to be interventional in the causal sense, where the user can modify the current input state to simulate counterfactual “what-if” scenarios. In this work, we provide an alternative interventional tool that complements traditional disaster modelling tools by leveraging real-time data sources like satellite imagery, news, and social media. Our tool also helps understand the causal attribution of different factors on the estimated severity, over any given region of interest. In addition, we provide actionable recourses that would enable easier mitigation planning. Our source code is publicly available.

Method: Proposes CANOE with a biologically inspired Chaotic Neural Oscillatory Attention mechanism to inject adaptive variability into attention, and uses a Tri-Pair Interaction Encoder with Cross Context Attentive Decoder to fuse multimodal who-when-where contexts.

Result: Extensive experiments show CANOE consistently outperforms state-of-the-art baselines by 3.17%-13.11% across different cases, with robust predictions over trajectories of varying chaotic levels.

Conclusion: CANOE effectively addresses mobility pattern imbalance and contextual underutilization challenges, demonstrating significant improvements in next location prediction performance through its novel chaotic neural oscillatory approach.

Abstract: Next location prediction is a key task in human mobility analysis, crucial for applications like smart city resource allocation and personalized navigation services. However, existing methods face two significant challenges: first, they fail to address the dynamic imbalance between periodic and chaotic mobile patterns, leading to inadequate adaptation over sparse trajectories; second, they underutilize contextual cues, such as temporal regularities in arrival times, which persist even in chaotic patterns and offer stronger predictability than spatial forecasts due to reduced search spaces. To tackle these challenges, we propose \textbf{\method}, a \underline{\textbf{C}}h\underline{\textbf{A}}otic \underline{\textbf{N}}eural \underline{\textbf{O}}scillator n\underline{\textbf{E}}twork for next location prediction, which introduces a biologically inspired Chaotic Neural Oscillatory Attention mechanism to inject adaptive variability into traditional attention, enabling balanced representation of evolving mobility behaviors, and employs a Tri-Pair Interaction Encoder along with a Cross Context Attentive Decoder to fuse multimodal ``who-when-where’’ contexts in a joint framework for enhanced prediction performance. Extensive experiments on two real-world datasets demonstrate that CANOE consistently and significantly outperforms a sizeable collection of state-of-the-art baselines, yielding 3.17%-13.11% improvement over the best-performing baselines across different cases. In particular, CANOE can make robust predictions over mobility trajectories of different mobility chaotic levels. A series of ablation studies also supports our key design choices. Our code is available at: https://github.com/yuqian2003/CANOE.

Method: Uses a latent diffusion model (LDM) pre-trained on large-scale datasets to perform iterative reconstruction on the LDM’s learned image prior, generating images from intermediate representations.

Result: Significantly outperforms state-of-the-art methods both qualitatively and quantitatively in reconstructing data from deep-layer intermediate representations of vision foundation models.

Conclusion: Highlights the urgent need for more robust privacy protection mechanisms for large models in split inference scenarios due to the effectiveness of this new attack method.

Abstract: With the rise of large foundation models, split inference (SI) has emerged as a popular computational paradigm for deploying models across lightweight edge devices and cloud servers, addressing data privacy and computational cost concerns. However, most existing data reconstruction attacks have focused on smaller CNN classification models, leaving the privacy risks of foundation models in SI settings largely unexplored. To address this gap, we propose a novel data reconstruction attack based on guided diffusion, which leverages the rich prior knowledge embedded in a latent diffusion model (LDM) pre-trained on a large-scale dataset. Our method performs iterative reconstruction on the LDM’s learned image prior, effectively generating high-fidelity images resembling the original data from their intermediate representations (IR). Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art methods, both qualitatively and quantitatively, in reconstructing data from deep-layer IRs of the vision foundation model. The results highlight the urgent need for more robust privacy protection mechanisms for large models in SI scenarios. Code is available at: https://github.com/ntuaislab/DRAG.

Method: Used k-nearest neighbor regression with chemically informed distance metrics (kernel-induced metric and metric learned via MLKR) and FCHL19 molecular descriptor on QM9 benchmark and atmospheric cluster datasets.

Result: k-NN models achieved near-chemical accuracy, scaled to datasets with 250,000+ entries, extrapolated to larger unseen clusters with minimal error (~1 kcal/mol), and reduced computational time significantly compared to KRR.

Conclusion: k-NN regression is a powerful, interpretable, and efficient alternative to complex quantum chemistry methods for accelerating discovery in atmospheric chemistry and related fields.

Abstract: Understanding how atmospheric molecular clusters form and grow is key to resolving one of the biggest uncertainties in climate modelling: the formation of new aerosol particles. While quantum chemistry offers accurate insights into these early-stage clusters, its steep computational costs limit large-scale exploration. In this work, we present a fast, interpretable, and surprisingly powerful alternative: $k$-nearest neighbour ($k$-NN) regression model. By leveraging chemically informed distance metrics, including a kernel-induced metric and one learned via metric learning for kernel regression (MLKR), we show that simple $k$-NN models can rival more complex kernel ridge regression (KRR) models in accuracy, while reducing computational time by orders of magnitude. We perform this comparison with the well-established Faber-Christensen-Huang-Lilienfeld (FCHL19) molecular descriptor, but other descriptors (e.g., FCHL18, MBDF, and CM) can be shown to have similar performance. Applied to both simple organic molecules in the QM9 benchmark set and large datasets of atmospheric molecular clusters (sulphuric acid-water and sulphuric-multibase -base systems), our $k$-NN models achieve near-chemical accuracy, scale seamlessly to datasets with over 250,000 entries, and even appears to extrapolate to larger unseen clusters with minimal error (often nearing 1 kcal/mol). With built-in interpretability and straightforward uncertainty estimation, this work positions $k$-NN as a potent tool for accelerating discovery in atmospheric chemistry and beyond.

Method: Used 296 voyage reports from a bulk carrier with 28 parameters, integrated with hydrometeorological big data from CMEMS (19 parameters) and ECMWF (61 parameters) to evaluate if external data enhances modeling accuracy.

Result: Machine learning techniques show strong potential for accurate fuel consumption prediction by combining voyage reports with climate and sea data.

Conclusion: External public data fusion enhances fuel consumption modeling accuracy, but validation on similar vessel classes is needed to confirm generalizability.

Abstract: There is an increasing push for operational measures to reduce ships’ bunker fuel consumption and carbon emissions, driven by the International Maritime Organization (IMO) mandates. Key performance indicators such as the Energy Efficiency Operational Indicator (EEOI) focus on fuel efficiency. Strategies like trim optimization, virtual arrival, and green routing have emerged. The theoretical basis for these approaches lies in accurate prediction of fuel consumption as a function of sailing speed, displacement, trim, climate, and sea state. This study utilized 296 voyage reports from a bulk carrier vessel over one year (November 16, 2021 to November 21, 2022) and 28 parameters, integrating hydrometeorological big data from the Copernicus Marine Environment Monitoring Service (CMEMS) with 19 parameters and the European Centre for Medium-Range Weather Forecasts (ECMWF) with 61 parameters. The objective was to evaluate whether fusing external public data sources enhances modeling accuracy and to highlight the most influential parameters affecting fuel consumption. The results reveal a strong potential for machine learning techniques to predict ship fuel consumption accurately by combining voyage reports with climate and sea data. However, validation on similar classes of vessels remains necessary to confirm generalizability.

Method: Builds on stability theory to develop a regularization scheme that specifically penalizes solutions corresponding to unstable fixed points in dynamical systems.

Result: Experimental validation on four dynamical systems (including Lotka-Volterra and van der Pol oscillator) shows the scheme helps avoid incorrect solutions and substantially improves PINN training success rates.

Conclusion: The proposed regularization method effectively addresses the local minima problem in PINNs by targeting unstable fixed points, leading to more reliable and physically accurate solutions.

Abstract: It was recently shown that the loss function used for training physics-informed neural networks (PINNs) exhibits local minima at solutions corresponding to fixed points of dynamical systems. In the forward setting, where the PINN is trained to solve initial value problems, these local minima can interfere with training and potentially leading to physically incorrect solutions. Building on stability theory, this paper proposes a regularization scheme that penalizes solutions corresponding to unstable fixed points. Experimental results on four dynamical systems, including the Lotka-Volterra model and the van der Pol oscillator, show that our scheme helps avoiding physically incorrect solutions and substantially improves the training success rate of PINNs.

Method: Integrates enzyme sequences (using pre-trained language model + CNN), substrate structures (using graph neural network), and environmental factors with attention mechanism. Uses symbolic regression via Kolmogorov-Arnold Networks to learn interpretable mathematical formulas.

Result: Outperforms both traditional and state-of-the-art deep learning approaches in extensive experiments.

Conclusion: Provides a robust tool for studying enzyme kinetics with applications in enzyme engineering, biotechnology, and industrial biocatalysis.

Abstract: The enzyme turnover rate is a fundamental parameter in enzyme kinetics, reflecting the catalytic efficiency of enzymes. However, enzyme turnover rates remain scarce across most organisms due to the high cost and complexity of experimental measurements. To address this gap, we propose a multimodal framework for predicting the enzyme turnover rate by integrating enzyme sequences, substrate structures, and environmental factors. Our model combines a pre-trained language model and a convolutional neural network to extract features from protein sequences, while a graph neural network captures informative representations from substrate molecules. An attention mechanism is incorporated to enhance interactions between enzyme and substrate representations. Furthermore, we leverage symbolic regression via Kolmogorov-Arnold Networks to explicitly learn mathematical formulas that govern the enzyme turnover rate, enabling interpretable and accurate predictions. Extensive experiments demonstrate that our framework outperforms both traditional and state-of-the-art deep learning approaches. This work provides a robust tool for studying enzyme kinetics and holds promise for applications in enzyme engineering, biotechnology, and industrial biocatalysis.

Method: Uses over 60 hours of IMU data from FARSEEING real-world falls dataset. Employs efficient classifiers in streaming mode with cost-sensitive learning strategy that tunes decision threshold using a cost function prioritizing fall detection over false alarms.

Result: Achieved Recall of 1.00, Precision of 0.84, and F1 score of 0.91 on FARSEEING dataset. Detected all falls while maintaining low false alarms, with average inference time below 5 ms per sample.

Conclusion: Cost-sensitive threshold tuning enhances robustness of accelerometer-based fall detection. The computationally efficient framework shows strong potential for deployment in real-time wearable sensor systems for continuous monitoring.

Abstract: Real-time fall detection is crucial for enabling timely interventions and mitigating the severe health consequences of falls, particularly in older adults. However, existing methods often rely on simulated data or assumptions such as prior knowledge of fall events, limiting their real-world applicability. Practical deployment also requires efficient computation and robust evaluation metrics tailored to continuous monitoring. This paper presents a real-time fall detection framework for continuous monitoring without prior knowledge of fall events. Using over 60 hours of inertial measurement unit (IMU) data from the FARSEEING real-world falls dataset, we employ recent efficient classifiers to compute fall probabilities in streaming mode. To enhance robustness, we introduce a cost-sensitive learning strategy that tunes the decision threshold using a cost function reflecting the higher risk of missed falls compared to false alarms. Unlike many methods that achieve high recall only at the cost of precision, our framework achieved Recall of 1.00, Precision of 0.84, and an F1 score of 0.91 on FARSEEING, detecting all falls while keeping false alarms low, with average inference time below 5 ms per sample. These results demonstrate that cost-sensitive threshold tuning enhances the robustness of accelerometer-based fall detection. They also highlight the potential of our computationally efficient framework for deployment in real-time wearable sensor systems for continuous monitoring.

Method: Selects average policy from bounded subset of feasible reward set by planning with reward centroid, derives closed-form expression for centroid, and provides efficient algorithm for estimation using offline expert demonstrations.

Result: Develops a provably efficient algorithm for estimating the reward centroid and demonstrates through numerical simulations the relationship between expert behavior and the method’s output.

Conclusion: The proposed criterion and centroid-based approach provides a principled solution to the ill-posed nature of IRL, enabling effective generalization of expert behavior to new settings.

Abstract: We study the problem of generalizing an expert agent’s behavior, provided through demonstrations, to new environments and/or additional constraints. Inverse Reinforcement Learning (IRL) offers a promising solution by seeking to recover the expert’s underlying reward function, which, if used for planning in the new settings, would reproduce the desired behavior. However, IRL is inherently ill-posed: multiple reward functions, forming the so-called feasible set, can explain the same observed behavior. Since these rewards may induce different policies in the new setting, in the absence of additional information, a decision criterion is needed to select which policy to deploy. In this paper, we propose a novel, principled criterion that selects the “average” policy among those induced by the rewards in a certain bounded subset of the feasible set. Remarkably, we show that this policy can be obtained by planning with the reward centroid of that subset, for which we derive a closed-form expression. We then present a provably efficient algorithm for estimating this centroid using an offline dataset of expert demonstrations only. Finally, we conduct numerical simulations that illustrate the relationship between the expert’s behavior and the behavior produced by our method.

Method: Introduced an efficient learnable dimensionality reduction method for visualizing GNN loss landscapes, and analyzed effects of over-smoothing, jumping knowledge, quantization, sparsification, and preconditioner on GNN optimization.

Result: The learnable projection method surpasses PCA-based approaches, enabling accurate reconstruction of high-dimensional parameters with lower memory usage. Architecture, sparsification, and optimizer’s preconditioning significantly impact optimization landscape and final performance.

Conclusion: These insights contribute to developing more efficient designs of GNN architectures and training strategies by providing better understanding of optimization dynamics through improved visualization techniques.

Abstract: Graph Neural Networks (GNNs) are powerful models for graph-structured data, with broad applications. However, the interplay between GNN parameter optimization, expressivity, and generalization remains poorly understood. We address this by introducing an efficient learnable dimensionality reduction method for visualizing GNN loss landscapes, and by analyzing the effects of over-smoothing, jumping knowledge, quantization, sparsification, and preconditioner on GNN optimization. Our learnable projection method surpasses the state-of-the-art PCA-based approach, enabling accurate reconstruction of high-dimensional parameters with lower memory usage. We further show that architecture, sparsification, and optimizer’s preconditioning significantly impact the GNN optimization landscape and their training process and final prediction performance. These insights contribute to developing more efficient designs of GNN architectures and training strategies.

Method: Proposed two algorithms: RS-BC (for unknown transition model) and RS-KT (for known transition model) using non-Markovian policies that match expert return distribution in Wasserstein distance.

Result: RS-KT achieves substantially lower sample complexity than RS-BC by exploiting dynamics information. Non-Markovian policies outperform standard IL algorithms in sample efficiency.

Conclusion: Risk-sensitive IL with return distribution matching is feasible and efficient, with non-Markovian policies being crucial for capturing expert risk attitudes beyond Markovian policy limitations.

Abstract: We study the problem of training a risk-sensitive reinforcement learning (RL) agent through imitation learning (IL). Unlike standard IL, our goal is not only to train an agent that matches the expert’s expected return (i.e., its average performance) but also its risk attitude (i.e., other features of the return distribution, such as variance). We propose a general formulation of the risk-sensitive IL problem in which the objective is to match the expert’s return distribution in Wasserstein distance. We focus on the tabular setting and assume the expert’s reward is known. After demonstrating the limited expressivity of Markovian policies for this task, we introduce an efficient and sufficiently expressive subclass of non-Markovian policies tailored to it. Building on this subclass, we develop two provably efficient algorithms, RS-BC and RS-KT, for solving the problem when the transition model is unknown and known, respectively. We show that RS-KT achieves substantially lower sample complexity than RS-BC by exploiting dynamics information. We further demonstrate the sample efficiency of return distribution matching in the setting where the expert’s reward is unknown by designing an oracle-based variant of RS-KT. Finally, we complement our theoretical analysis of RS-KT and RS-BC with numerical simulations, highlighting both their sample efficiency and the advantages of non-Markovian policies over standard sample-efficient IL algorithms.

Method: FedDAF calculates model functional distance between global source model and target model by computing the angle between their mean gradient fields on target data, then normalizes with Gompertz function for similarity-based aggregation. Global source model is constructed by averaging local source models.

Result: Experiments on real-world datasets show FedDAF achieves superior test accuracy compared to existing FL, PFL, and FDA methods.

Conclusion: FedDAF effectively addresses both domain shift and data scarcity challenges in FDA by enabling target-objective-aware model aggregation through functional distance measurement, demonstrating significant performance improvements.

Abstract: Federated Domain Adaptation (FDA) is a federated learning (FL) approach that improves model performance at the target client by collaborating with source clients while preserving data privacy. FDA faces two primary challenges: domain shifts between source and target data and limited labeled data at the target. Most existing FDA methods focus on domain shifts, assuming ample target data, yet often neglect the combined challenges of both domain shifts and data scarcity. Moreover, approaches that address both challenges fail to prioritize sharing relevant information from source clients according to the target’s objective. In this paper, we propose FedDAF, a novel approach addressing both challenges in FDA. FedDAF uses similarity-based aggregation of the global source model and target model by calculating model functional distance from their mean gradient fields computed on target data. This enables effective model aggregation based on the target objective, constructed using target data, even with limited data. While computing model functional distance between these two models, FedDAF computes the angle between their mean gradient fields and then normalizes with the Gompertz function. To construct the global source model, all the local source models are aggregated using simple average in the server. Experiments on real-world datasets demonstrate FedDAF’s superiority over existing FL, PFL, and FDA methods in terms of achieving better test accuracy.

Method: Compared traditional statistical, interpretable, and black-box models using Swiss administrative data, evaluating predictive performance, interpretability, and fairness with techniques like model sparsity and fairness mitigation.

Result: Explainable boosting machines performed nearly as well as best black-box models, with model sparsity, feature smoothing, and fairness mitigation enhancing transparency and fairness with minor performance losses.

Conclusion: Interpretable profiling provides accountable and trustworthy alternative to black-box models for unemployment prediction without compromising performance.

Abstract: Long-term unemployment (LTU) is a challenge for both jobseekers and public employment services. Statistical profiling tools are increasingly used to predict LTU risk. Some profiling tools are opaque, black-box machine learning models, which raise issues of transparency and fairness. This paper investigates whether interpretable models could serve as an alternative, using administrative data from Switzerland. Traditional statistical, interpretable, and black-box models are compared in terms of predictive performance, interpretability, and fairness. It is shown that explainable boosting machines, a recent interpretable model, perform nearly as well as the best black-box models. It is also shown how model sparsity, feature smoothing, and fairness mitigation can enhance transparency and fairness with only minor losses in performance. These findings suggest that interpretable profiling provides an accountable and trustworthy alternative to black-box models without compromising performance.

Method: Introduces global utility metric for structural fidelity assessment without ground-truth causal structures, and creates TabStruct benchmark with comprehensive evaluation of 13 generators across 29 datasets.

Result: Global utility provides task-independent, domain-agnostic evaluation of tabular generator performance. The benchmark offers large-scale quantitative analysis across diverse generator categories.

Conclusion: TabStruct framework enables holistic evaluation of tabular generators by integrating structural fidelity with conventional metrics, making comprehensive assessment possible for real-world datasets without ground-truth causal structures.

Abstract: Evaluating tabular generators remains a challenging problem, as the unique causal structural prior of heterogeneous tabular data does not lend itself to intuitive human inspection. Recent work has introduced structural fidelity as a tabular-specific evaluation dimension to assess whether synthetic data complies with the causal structures of real data. However, existing benchmarks often neglect the interplay between structural fidelity and conventional evaluation dimensions, thus failing to provide a holistic understanding of model performance. Moreover, they are typically limited to toy datasets, as quantifying existing structural fidelity metrics requires access to ground-truth causal structures, which are rarely available for real-world datasets. In this paper, we propose a novel evaluation framework that jointly considers structural fidelity and conventional evaluation dimensions. We introduce a new evaluation metric, $\textbf{global utility}$, which enables the assessment of structural fidelity even in the absence of ground-truth causal structures. In addition, we present $\textbf{TabStruct}$, a comprehensive evaluation benchmark offering large-scale quantitative analysis on 13 tabular generators from nine distinct categories, across 29 datasets. Our results demonstrate that global utility provides a task-independent, domain-agnostic lens for tabular generator performance. We release the TabStruct benchmark suite, including all datasets, evaluation pipelines, and raw results. Code is available at https://github.com/SilenceX12138/TabStruct.

Method: Used leaf-level spectral reflectance (400-2500 nm) with preprocessing (denoising, interpolation, smoothing, band reduction) and ensemble machine learning (Random Forest, XGBoost, SVM with RBF kernel, Naive Bayes) on 300 tomato plants tracked across growth stages.

Result: 89% accuracy at 585 GDD with 0.86 recall for infected plants and 0.93 for noninfected plants. Clear spectral differences near water absorption features (1500 nm, 2000 nm) indicating reduced leaf water content in infected plants. Accuracy declined to 69% at later stages due to senescence and weed interference.

Conclusion: Proximal sensing with ensemble learning enables timely detection of broomrape before canopy symptoms are visible, supporting targeted interventions and reduced yield losses despite environmental challenges.

Abstract: Branched broomrape (Phelipanche ramosa) is a chlorophyll-deficient parasitic weed that threatens tomato production by extracting nutrients from the host. We investigate early detection using leaf-level spectral reflectance (400-2500 nm) and ensemble machine learning. In a field experiment in Woodland, California, we tracked 300 tomato plants across growth stages defined by growing degree days (GDD). Leaf reflectance was acquired with a portable spectrometer and preprocessed (band denoising, 1 nm interpolation, Savitzky-Golay smoothing, correlation-based band reduction). Clear class differences were observed near 1500 nm and 2000 nm water absorption features, consistent with reduced leaf water content in infected plants at early stages. An ensemble combining Random Forest, XGBoost, SVM with RBF kernel, and Naive Bayes achieved 89% accuracy at 585 GDD, with recalls of 0.86 (infected) and 0.93 (noninfected). Accuracy declined at later stages (e.g., 69% at 1568 GDD), likely due to senescence and weed interference. Despite the small number of infected plants and environmental confounders, results show that proximal sensing with ensemble learning enables timely detection of broomrape before canopy symptoms are visible, supporting targeted interventions and reduced yield losses.

Method: Deep operator network (DeepONet) architecture with three strategies: nondimensionalization of equations, input dimensionality reduction via Karhunen-Loève expansion, and two-step training procedure decoupling branch/trunk networks.

Result: Achieved substantial speedup in inference while maintaining high predictive accuracy across wide range of permeability statistics in soil consolidation and groundwater extraction benchmarks.

Conclusion: The proposed DeepONet approach shows potential as a scalable and efficient surrogate modeling technique for poroelastic systems with random permeability fields.

Abstract: Poroelasticity – coupled fluid flow and elastic deformation in porous media – often involves spatially variable permeability, especially in subsurface systems. In such cases, simulations with random permeability fields are widely used for probabilistic analysis, uncertainty quantification, and inverse problems. These simulations require repeated forward solves that are often prohibitively expensive, motivating the development of efficient surrogate models. However, efficient surrogate modeling techniques for poroelasticity with random permeability fields remain scarce. In this study, we propose a surrogate modeling framework based on the deep operator network (DeepONet), a neural architecture designed to learn mappings between infinite-dimensional function spaces. The proposed surrogate model approximates the solution operator that maps random permeability fields to transient poroelastic responses. To enhance predictive accuracy and stability, we integrate three strategies: nondimensionalization of the governing equations, input dimensionality reduction via Karhunen–Lo'eve expansion, and a two-step training procedure that decouples the optimization of branch and trunk networks. The methodology is evaluated on two benchmark problems in poroelasticity: soil consolidation and ground subsidence induced by groundwater extraction. In both cases, the DeepONet achieves substantial speedup in inference while maintaining high predictive accuracy across a wide range of permeability statistics. These results highlight the potential of the proposed approach as a scalable and efficient surrogate modeling technique for poroelastic systems with random permeability fields.

Method: Constructs 2D subspace patches from Hankel matrices using delay embedding, treats them as images for deep learning processing, and uses self-attention encoder to learn finite-dimensional Koopman operator for linear prediction in latent space.

Result: 20% average reduction in mean squared error compared to state-of-the-art foundation models across various benchmarks and real-world climate datasets, with superior generalization in fine-tuning scenarios.

Conclusion: UDE establishes a scalable, interpretable framework for universal time-series modeling with exceptional interpretability, consistent identification of topologically informative subspaces, and robust encoding of domain-invariant dynamics.

Abstract: This study introduces Universal Delay Embedding (UDE), a pretrained foundation model designed to revolutionize time-series forecasting through principled integration of delay embedding representation and Koopman operator prediction. Leveraging Takens’ embedding theorem, UDE as a dynamical representation of observed data constructs two-dimensional subspace patches from Hankel matrices, theoretically preserving dynamical and topological properties of underlying dynamical systems. Such patches are viewed as images, which can be efficiently processed by exploiting advanced deep learning technologies. Computationally, these patches further serve as tokens for learning a self-attention encoder, thus enabling accurate prediction of nonlinear time-series by a finite-dimensional Koopman operator in a linear manner in a latent space. Extensive evaluations across various benchmarks and real-world climate datasets demonstrate over 20% average reduction in mean squared error versus state-of-the-art foundation models, alongside superior generalization in fine-tuning scenarios. In particular, the learned dynamical representations and Koopman operator prediction forms from the patches exhibit exceptional interpretability, with consistent identification of topologically informative subspaces and robust encoding of domain-invariant dynamics, establishing UDE as a scalable, interpretable framework for universal time-series modeling and forecasting with broad scientific and industrial applicability.

Method: Used temporal social network analysis (SNA) on publication citation networks from scientists during dot-com era (1994-2001) and AI era (2017-2024), comparing patterns with financial market data. Applied multiple analysis techniques including LSTM, KNN, and AR X/GARCH models.

Result: Dot-com era patterns did not definitively predict AI bubble rise/fall. While yearly citation networks showed changes in publishing behavior, a subset of AI scientists mirrored dot-com patterns. Analysis suggests either unprecedented bubble form or no bubble in AI era.

Conclusion: Patterns from dot-com bubble do not effectively translate to predict AI market behavior, indicating limitations in using historical bubble patterns for forecasting future technology market disruptions.

Abstract: Financial bubbles often arrive without much warning, but create long-lasting economic effects. For example, during the dot-com bubble, innovative technologies created market disruptions through excitement for a promised bright future. Such technologies originated from research where scientists had developed them for years prior to their entry into the markets. That raises a question on the possibility of analyzing scientific publishing data (e.g. citation networks) leading up to a bubble for signals that may forecast the rise and fall of similar future bubbles. To that end, we utilized temporal SNAs to detect possible relationships between the publication citation networks of scientists and financial market data during two modern eras of rapidly shifting technology: 1) dot-com era from 1994 to 2001 and 2) AI era from 2017 to 2024. Results showed that the patterns from the dot-com era (which did end in a bubble) did not definitively predict the rise and fall of an AI bubble. While yearly citation networks reflected possible changes in publishing behavior of scientists between the two eras, there was a subset of AI era scientists whose publication influence patterns mirrored those during the dot-com era. Upon further analysis using multiple analysis techniques (LSTM, KNN, AR X/GARCH), the data seems to suggest two possibilities for the AI era: unprecedented form of financial bubble unseen or that no bubble exists. In conclusion, our findings imply that the patterns present in the dot-com era do not effectively translate in such a manner to apply them to the AI market.

Method: Deceptive Risk Minimization (DRM) - learns data representations that make training data appear iid to an observer, using a differentiable objective that simultaneously learns features to eliminate distribution shifts (via conformal martingales detector) while minimizing task-specific loss.

Result: DRM demonstrates efficacy in numerical experiments with concept shift and simulated imitation learning settings with covariate shift in robot deployment environments.

Conclusion: DRM provides a practical approach for OOD generalization that doesn’t require test data access or domain partitioning, effectively learning stable features through deceptive representation learning.

Abstract: This paper proposes deception as a mechanism for out-of-distribution (OOD) generalization: by learning data representations that make training data appear independent and identically distributed (iid) to an observer, we can identify stable features that eliminate spurious correlations and generalize to unseen domains. We refer to this principle as deceptive risk minimization (DRM) and instantiate it with a practical differentiable objective that simultaneously learns features that eliminate distribution shifts from the perspective of a detector based on conformal martingales while minimizing a task-specific loss. In contrast to domain adaptation or prior invariant representation learning methods, DRM does not require access to test data or a partitioning of training data into a finite number of data-generating domains. We demonstrate the efficacy of DRM on numerical experiments with concept shift and a simulated imitation learning setting with covariate shift in environments that a robot is deployed in.

Method: Develops low-rank orthogonalization technique that explicitly uses the low-rank property of gradients. Proposes low-rank matrix-signed gradient descent and a low-rank variant of the Muon optimizer.

Result: Numerical experiments show superior performance of low-rank orthogonalization. Low-rank Muon achieves promising results in GPT-2 and LLaMA pretraining, surpassing carefully tuned vanilla Muon.

Conclusion: The proposed low-rank orthogonalization effectively exploits gradient structure, leading to improved optimization performance with theoretical guarantees for convergence under various conditions.

Abstract: Neural network (NN) training is inherently a large-scale matrix optimization problem, yet the matrix structure of NN parameters has long been overlooked. Recently, the optimizer Muon \cite{jordanmuon}, which explicitly exploits this structure, has gained significant attention for its strong performance in foundation model training. A key component contributing to Muon’s success is matrix orthogonalization. In this paper, we propose {\it low-rank orthogonalization}, which explicitly leverages the low-rank nature of gradients during NN training. Building on this, we propose low-rank matrix-signed gradient descent and a low-rank variant of Muon. Our numerical experiments demonstrate the superior performance of low-rank orthogonalization, with the low-rank Muon achieving promising results in GPT-2 and LLaMA pretraining – surpassing the performance of the carefully tuned vanilla Muon. Theoretically, we establish the iteration complexity of the low-rank matrix-signed gradient descent for finding an approximate stationary solution, as well as that of low-rank Muon for finding an approximate stochastic stationary solution under heavy-tailed noise.

Method: Proposes USimUL framework with uncertain similarity annotations containing uncertainty components to reduce label leakage. Develops an unbiased risk estimator that learns from uncertain similarity and unlabeled data.

Result: Theoretical proof shows the estimator achieves statistically optimal parametric convergence rates. Extensive experiments on benchmark and real-world datasets demonstrate superior classification performance compared to conventional similarity-based approaches.

Conclusion: USimUL effectively addresses privacy concerns in similarity-based weakly supervised learning while maintaining strong classification performance through uncertain similarity annotations and theoretically sound estimation methods.

Abstract: Existing similarity-based weakly supervised learning approaches often rely on precise similarity annotations between data pairs, which may inadvertently expose sensitive label information and raise privacy risks. To mitigate this issue, we propose Uncertain Similarity and Unlabeled Learning (USimUL), a novel framework where each similarity pair is embedded with an uncertainty component to reduce label leakage. In this paper, we propose an unbiased risk estimator that learns from uncertain similarity and unlabeled data. Additionally, we theoretically prove that the estimator achieves statistically optimal parametric convergence rates. Extensive experiments on both benchmark and real-world datasets show that our method achieves superior classification performance compared to conventional similarity-based approaches.

Method: Use signature transform as an encoder for learning non-Markovian dynamics. Signature transform provides continuous-time alternative with strong theoretical foundations for summarizing multidimensional temporal information. Integrated into encoder-decoder dynamics models.

Result: Signature-based encoding outperforms RNN-based alternatives in test performance on synthetic benchmarks.

Conclusion: Signature transform offers an effective continuous-time encoding approach for capturing historical dependencies in dynamical systems, overcoming limitations of both neural ODEs and RNN-based methods.

Abstract: Neural ordinary differential equations offer an effective framework for modeling dynamical systems by learning a continuous-time vector field. However, they rely on the Markovian assumption - that future states depend only on the current state - which is often untrue in real-world scenarios where the dynamics may depend on the history of past states. This limitation becomes especially evident in settings involving the continuous control of complex systems with delays and memory effects. To capture historical dependencies, existing approaches often rely on recurrent neural network (RNN)-based encoders, which are inherently discrete and struggle with continuous modeling. In addition, they may exhibit poor training behavior. In this work, we investigate the use of the signature transform as an encoder for learning non-Markovian dynamics in a continuous-time setting. The signature transform offers a continuous-time alternative with strong theoretical foundations and proven efficiency in summarizing multidimensional information in time. We integrate a signature-based encoding scheme into encoder-decoder dynamics models and demonstrate that it outperforms RNN-based alternatives in test performance on synthetic benchmarks.

Method: K-Level Policy Gradient (KPG) recursively updates each agent against the updated policies of other agents, enabling faster discovery of effective coordinated policies. Applied to MAPPO, MADDPG, and FACMAC algorithms.

Result: Theoretical proof shows KPG with finite iterates achieves monotonic convergence to local Nash equilibrium. Empirical results demonstrate superior performance over existing deep MARL algorithms in StarCraft II and multi-agent MuJoCo.

Conclusion: KPG effectively addresses coordination issues in MARL by accounting for simultaneous policy updates, leading to improved performance and theoretical convergence guarantees.

Abstract: Actor-critic algorithms for deep multi-agent reinforcement learning (MARL) typically employ a policy update that responds to the current strategies of other agents. While being straightforward, this approach does not account for the updates of other agents at the same update step, resulting in miscoordination. In this paper, we introduce the $K$-Level Policy Gradient (KPG), a method that recursively updates each agent against the updated policies of other agents, speeding up the discovery of effective coordinated policies. We theoretically prove that KPG with finite iterates achieves monotonic convergence to a local Nash equilibrium under certain conditions. We provide principled implementations of KPG by applying it to the deep MARL algorithms MAPPO, MADDPG, and FACMAC. Empirically, we demonstrate superior performance over existing deep MARL algorithms in StarCraft II and multi-agent MuJoCo.

Method: Proposed Graph Neural Network framework with adaptive adjacency matrices using log-normal distributions and Coefficient of Variation values. Incorporated weather factors (temperature, wind speed, precipitation) to adjust edge weights. Used Adaptive Conformal Prediction for uncertainty quantification. Validated with SUMO traffic scenarios and Monte-Carlo simulation.

Result: The model demonstrated better prediction accuracy and uncertainty bounds compared to baseline methods. Simulated mean travel time fell within intervals defined by historical INRIX data, verifying robustness.

Conclusion: The proposed GNN framework effectively addresses traffic stochasticity through adaptive adjacency matrices and environmental factor integration, providing reliable uncertainty quantification and robust performance in real-world traffic scenarios.

Abstract: Traffic flow forecasting is essential for managing congestion, improving safety, and optimizing various transportation systems. However, it remains a prevailing challenge due to the stochastic nature of urban traffic and environmental factors. Better predictions require models capable of accommodating the traffic variability influenced by multiple dynamic and complex interdependent factors. In this work, we propose a Graph Neural Network (GNN) framework to address the stochasticity by leveraging adaptive adjacency matrices using log-normal distributions and Coefficient of Variation (CV) values to reflect real-world travel time variability. Additionally, weather factors such as temperature, wind speed, and precipitation adjust edge weights and enable GNN to capture evolving spatio-temporal dependencies across traffic stations. This enhancement over the static adjacency matrix allows the model to adapt effectively to traffic stochasticity and changing environmental conditions. Furthermore, we utilize the Adaptive Conformal Prediction (ACP) framework to provide reliable uncertainty quantification, achieving target coverage while maintaining acceptable prediction intervals. Experimental results demonstrate that the proposed model, in comparison with baseline methods, showed better prediction accuracy and uncertainty bounds. We, then, validate this method by constructing traffic scenarios in SUMO and applying Monte-Carlo simulation to derive a travel time distribution for a Vehicle Under Test (VUT) to reflect real-world variability. The simulated mean travel time of the VUT falls within the intervals defined by INRIX historical data, verifying the model’s robustness.

Method: Hierarchical multi-agent reinforcement learning with a meta-agent that analyzes market volatility and dynamically activates specialized Time Frame Agents for high-frequency or low-frequency trading.

Result: 25.17% cumulative return with Sharpe Ratio of 0.75 on AAPL stock (Jan 2024-May 2025), outperforming buy-and-hold AAPL (12.19%) and SPY (20.01%).

Conclusion: Dynamic hierarchical agents can achieve superior risk-adjusted returns while maintaining high computational efficiency.

Abstract: Conventional autonomous trading systems struggle to balance computational efficiency and market responsiveness due to their fixed operating frequency. We propose Hi-DARTS, a hierarchical multi-agent reinforcement learning framework that addresses this trade-off. Hi-DARTS utilizes a meta-agent to analyze market volatility and dynamically activate specialized Time Frame Agents for high-frequency or low-frequency trading as needed. During back-testing on AAPL stock from January 2024 to May 2025, Hi-DARTS yielded a cumulative return of 25.17% with a Sharpe Ratio of 0.75. This performance surpasses standard benchmarks, including a passive buy-and-hold strategy on AAPL (12.19% return) and the S&P 500 ETF (SPY) (20.01% return). Our work demonstrates that dynamic, hierarchical agents can achieve superior risk-adjusted returns while maintaining high computational efficiency.

Method: Building on the algorithmic foundations from Part I, the authors developed a hypersurface decision tree algorithm that doesn’t require external solvers. They tested it on synthetic datasets with varying tree size, data size, dimensionality, and noise levels, plus 30 real-world datasets.

Result: The HODT algorithm recovers ground truth more accurately than axis-parallel trees, shows greater noise robustness, and achieves up to 30% higher accuracy than state-of-the-art optimal axis-parallel decision trees when complexity is controlled.

Conclusion: The proposed hypersurface decision tree algorithm successfully generalizes beyond traditional hyperplane methods, demonstrating superior performance in both synthetic and real-world scenarios while maintaining computational independence from external solvers.

Abstract: Decision trees are a ubiquitous model for classification and regression tasks due to their interpretability and efficiency. However, solving the optimal decision tree (ODT) problem remains a challenging combinatorial optimization task. Even for the simplest splitting rules–axis-parallel hyperplanes–it is NP-hard to optimize. In Part I of this series, we rigorously defined the proper decision tree model through four axioms and, based on these, introduced four formal definitions of the ODT problem. From these definitions, we derived four generic algorithms capable of solving ODT problems for arbitrary decision trees satisfying the axioms. We also analyzed the combinatorial geometric properties of hypersurfaces, showing that decision trees defined by polynomial hypersurface splitting rules satisfy the proper axioms that we proposed. In this second paper (Part II) of this two-part series, building on the algorithmic and geometric foundations established in Part I, we introduce the first hypersurface decision tree (HODT) algorithm. To the best of our knowledge, existing optimal decision tree methods are, to date, limited to hyperplane splitting rules–a special case of hypersurfaces–and rely on general-purpose solvers. In contrast, our HODT algorithm addresses the general hypersurface decision tree model without requiring external solvers. Using synthetic datasets generated from ground-truth hyperplane decision trees, we vary tree size, data size, dimensionality, and label and feature noise. Results showing that our algorithm recovers the ground truth more accurately than axis-parallel trees and exhibits greater robustness to noise. We also analyzed generalization performance across 30 real-world datasets, showing that HODT can achieve up to 30% higher accuracy than the state-of-the-art optimal axis-parallel decision tree algorithm when tree complexity is properly controlled.

Method: NodePro combines data-centric signals (feature dissimilarity, label uncertainty, structural ambiguity) with model-centric measures (prediction confidence, training consistency) to assign interpretable profile scores to individual nodes.

Result: NodePro reveals systematic differences between models that aggregate metrics miss, generalizes to unseen nodes for reliability prediction without ground-truth labels, and effectively identifies semantically inconsistent or corrupted nodes in knowledge graphs.

Conclusion: NodePro enables fine-grained diagnosis of graph ML model behavior, providing interpretable node-level insights that support better model understanding, comparison, and real-world application in identifying problematic nodes.

Abstract: Graph machine learning models often achieve similar overall performance yet behave differently at the node level, failing on different subsets of nodes with varying reliability. Standard evaluation metrics such as accuracy obscure these fine grained differences, making it difficult to diagnose when and where models fail. We introduce NodePro, a node profiling framework that enables fine-grained diagnosis of model behavior by assigning interpretable profile scores to individual nodes. These scores combine data-centric signals, such as feature dissimilarity, label uncertainty, and structural ambiguity, with model-centric measures of prediction confidence and consistency during training. By aligning model behavior with these profiles, NodePro reveals systematic differences between models, even when aggregate metrics are indistinguishable. We show that node profiles generalize to unseen nodes, supporting prediction reliability without ground-truth labels. Finally, we demonstrate the utility of NodePro in identifying semantically inconsistent or corrupted nodes in a structured knowledge graph, illustrating its effectiveness in real-world settings.

Method: Adapted established OOD evaluation methodologies to climate data and systematically evaluated state-of-the-art ML-based climate models across diverse out-of-distribution scenarios using large-scale datasets.

Result: Experiments revealed notable performance variability across different OOD scenarios, highlighting both strengths and limitations of current ML climate models in handling distribution shifts.

Conclusion: The findings emphasize the importance of robust evaluation frameworks and provide actionable insights to guide reliable application of machine learning for climate risk forecasting in the face of distribution shifts.

Abstract: Climate change is accelerating the frequency and severity of unprecedented events, deviating from established patterns. Predicting these out-of-distribution (OOD) events is critical for assessing risks and guiding climate adaptation. While machine learning (ML) models have shown promise in providing precise, high-speed climate predictions, their ability to generalize under distribution shifts remains a significant limitation that has been underexplored in climate contexts. This research systematically evaluates state-of-the-art ML-based climate models in diverse OOD scenarios by adapting established OOD evaluation methodologies to climate data. Experiments on large-scale datasets reveal notable performance variability across scenarios, shedding light on the strengths and limitations of current models. These findings underscore the importance of robust evaluation frameworks and provide actionable insights to guide the reliable application of ML for climate risk forecasting.

Method: First part studies gradient flow evolution near sparse saddle points. Second part introduces Neuron Pursuit - an iterative algorithm that alternates between adding carefully chosen neurons and minimizing loss with the augmented network.

Result: Gradient flow remains near saddle points for extended periods, with small-norm weights converging in direction. Neuron Pursuit shows efficacy in numerical experiments for training deep networks.

Conclusion: The analysis of gradient flow near saddle points provides insights that motivate effective greedy training algorithms like Neuron Pursuit for deep neural networks.

Abstract: The first part of this paper studies the evolution of gradient flow for homogeneous neural networks near a class of saddle points exhibiting a sparsity structure. The choice of these saddle points is motivated from previous works on homogeneous networks, which identified the first saddle point encountered by gradient flow after escaping the origin. It is shown here that, when initialized sufficiently close to such saddle points, gradient flow remains near the saddle point for a sufficiently long time, during which the set of weights with small norm remain small but converge in direction. Furthermore, important empirical observations are made on the behavior of gradient descent after escaping these saddle points. The second part of the paper, motivated by these results, introduces a greedy algorithm to train deep neural networks called Neuron Pursuit (NP). It is an iterative procedure which alternates between expanding the network by adding neuron(s) with carefully chosen weights, and minimizing the training loss using this augmented network. The efficacy of the proposed algorithm is validated using numerical experiments.

Method: Proposed attention with dynamic relational priming (prime attention) that learns to modulate each token representation dynamically per interaction to optimize for specific token-pair relationships while maintaining computational complexity.

Result: Prime attention consistently outperforms standard attention across benchmarks with up to 6.5% improvement in forecasting accuracy and achieves comparable performance using up to 40% less sequence length.

Conclusion: Prime attention’s representational plasticity enables more effective extraction of relationship-specific information in multivariate time series while maintaining computational efficiency, demonstrating superior relational modeling capabilities.

Abstract: Standard attention mechanisms in transformers employ static token representations that remain unchanged across all pair-wise computations in each layer. This limits their representational alignment with the potentially diverse relational dynamics of each token-pair interaction. While they excel in domains with relatively homogeneous relationships, standard attention’s static relational learning struggles to capture the diverse, heterogeneous inter-channel dependencies of multivariate time series (MTS) data–where different channel-pair interactions within a single system may be governed by entirely different physical laws or temporal dynamics. To better align the attention mechanism for such domain phenomena, we propose attention with dynamic relational priming (prime attention). Unlike standard attention where each token presents an identical representation across all of its pair-wise interactions, prime attention tailors each token dynamically (or per interaction) through learnable modulations to best capture the unique relational dynamics of each token pair, optimizing each pair-wise interaction for that specific relationship. This representational plasticity of prime attention enables effective extraction of relationship-specific information in MTS while maintaining the same asymptotic computational complexity as standard attention. Our results demonstrate that prime attention consistently outperforms standard attention across benchmarks, achieving up to 6.5% improvement in forecasting accuracy. In addition, we find that prime attention achieves comparable or superior performance using up to 40% less sequence length compared to standard attention, further demonstrating its superior relational modeling capabilities.

Method: Adversarial learning with guided CycleGAN framework using spectral normalization for stable training, identity- and perceptual-guided losses, and landmark-weighted cycle constraints to maintain facial geometry.

Result: Improved realism (FID), perceptual quality (LPIPS), and identity preservation (ID-Sim) over autoencoders, with competitive cycle-reconstruction SSIM and practical inference times, approaching pix2pix performance on curated paired subsets.

Conclusion: Guided, spectrally normalized CycleGANs provide a practical path from autoencoders to robust unpaired face manipulation, achieving high quality without paired datasets.

Abstract: Human face synthesis and manipulation are increasingly important in entertainment and AI, with a growing demand for highly realistic, identity-preserving images even when only unpaired, unaligned datasets are available. We study unpaired face manipulation via adversarial learning, moving from autoencoder baselines to a robust, guided CycleGAN framework. While autoencoders capture coarse identity, they often miss fine details. Our approach integrates spectral normalization for stable training, identity- and perceptual-guided losses to preserve subject identity and high-level structure, and landmark-weighted cycle constraints to maintain facial geometry across pose and illumination changes. Experiments show that our adversarial trained CycleGAN improves realism (FID), perceptual quality (LPIPS), and identity preservation (ID-Sim) over autoencoders, with competitive cycle-reconstruction SSIM and practical inference times, which achieved high quality without paired datasets and approaching pix2pix on curated paired subsets. These results demonstrate that guided, spectrally normalized CycleGANs provide a practical path from autoencoders to robust unpaired face manipulation.

Method: The authors analyze common datasets (carbon-24 and perov-5), identify issues with duplicate structures and improper dataset splits, and propose solutions including deduplicated datasets, composition-based splits, and new evaluation metrics (METRe and cRMSE).

Result: Found that carbon-24 contains only ≈40% unique structures, identified issues with random splits in perov-5 dataset, and developed revised datasets with duplicates removed and proper splits. Proposed new metrics to address limitations of existing match rate evaluation.

Conclusion: Proper dataset curation and evaluation metrics are crucial for meaningful benchmarking of crystal structure prediction models. The proposed fixes and new metrics provide more robust standards for evaluating generative models in materials science.

Abstract: Generative models for materials, especially inorganic crystals, hold potential to transform the theoretical prediction of novel compounds and structures. Advancement in this field depends critically on robust benchmarks and minimal, information-rich datasets that enable meaningful model evaluation. This paper critically examines common datasets and reported metrics for a crystal structure prediction task$\unicode{x2014}$generating the most likely structures given the chemical composition of a material. We focus on three key issues: First, materials datasets should contain unique crystal structures; for example, we show that the widely-utilized carbon-24 dataset only contains $\approx$40% unique structures. Second, materials datasets should not be split randomly if polymorphs of many different compositions are numerous, which we find to be the case for the perov-5 dataset. Third, benchmarks can mislead if used uncritically, e.g., reporting a match rate metric without considering the structural variety exhibited by identical building blocks. To address these oft-overlooked issues, we introduce several fixes. We provide revised versions of the carbon-24 dataset: one with duplicates removed, one deduplicated and split by number of atoms $N$, and two containing only identical structures but with different unit cells. We also propose a new split for the perov-5 dataset which ensures polymorphs are grouped within each split subset, setting a more sensible standard for benchmarking model performance. Finally, we present METRe and cRMSE, new model evaluation metrics that can correct existing issues with the match rate metric.

Method: Comprehensive survey approach: defines model calibration, explains root causes of miscalibration, introduces key calibration metrics, and classifies calibration methods into four categories (post-hoc calibration, regularization, uncertainty estimation, composition methods). Also covers LLM calibration advancements.

Result: Systematic review and categorization of existing calibration techniques, providing a framework for understanding and implementing model calibration across different deep learning architectures including large language models.

Conclusion: Identifies open issues, challenges, and potential future directions in model calibration research, emphasizing the need for well-calibrated models in addition to high predictive performance for reliable AI systems.

Abstract: Calibrating deep neural models plays an important role in building reliable, robust AI systems in safety-critical applications. Recent work has shown that modern neural networks that possess high predictive capability are poorly calibrated and produce unreliable model predictions. Though deep learning models achieve remarkable performance on various benchmarks, the study of model calibration and reliability is relatively under-explored. Ideal deep models should have not only high predictive performance but also be well calibrated. There have been some recent advances in calibrating deep models. In this survey, we review the state-of-the-art calibration methods and their principles for performing model calibration. First, we start with the definition of model calibration and explain the root causes of model miscalibration. Then we introduce the key metrics that can measure this aspect. It is followed by a summary of calibration methods that we roughly classify into four categories: post-hoc calibration, regularization methods, uncertainty estimation, and composition methods. We also cover recent advancements in calibrating large models, particularly large language models (LLMs). Finally, we discuss some open issues, challenges, and potential directions.

Method: FairCoT employs iterative Chain of Thought refinement within multimodal generative large language models to systematically mitigate biases and dynamically adjust textual prompts in real time.

Result: Experimental evaluations on systems like DALLE and Stable Diffusion variants show FairCoT significantly enhances fairness and diversity while maintaining image quality and semantic fidelity.

Conclusion: FairCoT represents a promising step toward more socially responsible and transparent AI-driven content generation through robust reasoning, lightweight deployment, and extensibility to multiple models.

Abstract: In the domain of text-to-image generative models, biases inherent in training datasets often propagate into generated content, posing significant ethical challenges, particularly in socially sensitive contexts. We introduce FairCoT, a novel framework that enhances fairness in text to image models through Chain of Thought (CoT) reasoning within multimodal generative large language models. FairCoT employs iterative CoT refinement to systematically mitigate biases, and dynamically adjusts textual prompts in real time, ensuring diverse and equitable representation in generated images. By integrating iterative reasoning processes, FairCoT addresses the limitations of zero shot CoT in sensitive scenarios, balancing creativity with ethical responsibility. Experimental evaluations across popular text-to-image systems including DALLE and various Stable Diffusion variants, demonstrate that FairCoT significantly enhances fairness and diversity without sacrificing image quality or semantic fidelity. By combining robust reasoning, lightweight deployment, and extensibility to multiple models, FairCoT represents a promising step toward more socially responsible and transparent AI driven content generation.

Method: Proposed a computational framework separating extrinsic (environment interaction) and intrinsic (internal reflection/abstraction) functions, enabling scaffolded learning where agents initially learn in structured environments followed by open-ended generalization.

Result: Agent K achieved an Elo-MMR score of 1694, surpassing the median score of Kaggle Masters (top 2% of 200,000 users), with 9 gold, 8 silver, and 12 bronze medals including 4 gold and 4 silver in prize-awarding competitions.

Conclusion: This represents the first AI system to successfully integrate Kolb- and Vygotsky-inspired human cognitive learning, marking a significant advancement toward generalist AI capable of mastering complex domains that traditional methods cannot handle effectively.

Abstract: Human expertise emerges through iterative cycles of interaction, reflection, and internal model updating, which are central to cognitive theories such as Kolb’s experiential learning and Vygotsky’s zone of proximal development. In contrast, current AI systems, particularly LLM agents, rely on static pre-training or rigid workflows, lacking mechanisms for continual adaptation. Recent studies identified early cognitive traits in LLM agents (reflection, revision, and self-correction) suggesting foundational elements of human-like experiential learning. Thus the key question: Can we design LLM agents capable of structured, cognitively grounded learning similar to human processes? In response, we propose a computational framework of Kolb’s learning cycle with Vygotsky’s ZPD for autonomous agents. Our architecture separates extrinsic (environment interaction) and intrinsic (internal reflection/abstraction) functions, enabling cognitively grounded scaffolded learning, where the agent initially learns within structured environments, followed by open-ended generalisation. This approach empowers agents to master complex tasks ; domains that traditional fine-tuning or simple reflective methods could not tackle effectively. Its potential is powerfully demonstrated via direct comparison with humans in real-world Kaggle data science competitions. Learning fully automated data science code generation across 81 tasks, our system, Agent K, demonstrated the ability to perform the entire workflow autonomously, achieving an Elo-MMR score of 1694, beyond median score of the Kaggle Masters (the top 2% among 200,000 users) of our study. With 9 gold, 8 silver, and 12 bronze medals level performance - including 4 gold and 4 silver on prize-awarding competitions - Agent K is the 1st AI system to successfully integrate Kolb- and Vygotsky-inspired human cognitive learning, marking a major step toward generalist AI.

Method: CNRPO uses multi-objective optimization to separate true preferences from content-aware noises and leverages backdoor attack mechanisms to efficiently learn and control multiple noise sources within a single model.

Result: Theoretical analysis and experiments on synthetic noisy datasets show CNRPO significantly improves alignment with primary human preferences while controlling secondary noises like response length and harmfulness.

Conclusion: CNRPO effectively mitigates content-dependent noise in preference learning, providing a robust framework for aligning LLMs with human preferences in real-world noisy scenarios.

Abstract: Large Language Models (LLMs) have made significant strides in generating human-like responses, largely due to preference alignment techniques. However, these methods often assume unbiased human feedback, which is rarely the case in real-world scenarios. This paper introduces Content-Aware Noise-Resilient Preference Optimization (CNRPO), a novel framework that addresses multiple sources of content-dependent noise in preference learning. CNRPO employs a multi-objective optimization approach to separate true preferences from content-aware noises, effectively mitigating their impact. We leverage backdoor attack mechanisms to efficiently learn and control various noise sources within a single model. Theoretical analysis and extensive experiments on different synthetic noisy datasets demonstrate that CNRPO significantly improves alignment with primary human preferences while controlling for secondary noises and biases, such as response length and harmfulness.

Method: The authors developed Timing-Matters, a Transformer-Encoder based method specifically designed for predicting action timing. They also created a synthesized dataset of 11.6k sequences with precise timestamps based on human annotations of interaction patterns to address the lack of public datasets with fine-grained timestamps.

Result: Timing-Matters achieved 38.30% accuracy on the synthesized dataset, outperforming the best baseline by 6%, and showed 1-6% improvements on other open datasets.

Conclusion: The work successfully addresses the gap in predicting action timing in IoT environments, demonstrating significant performance improvements over existing baselines, and contributes both a novel method and a valuable dataset to the research community.

Abstract: The proliferation of IoT devices generates vast interaction data, offering insights into user behaviour. While prior work predicts what actions users perform, the timing of these actions – critical for enabling proactive and efficient smart systems – remains relatively underexplored. Addressing this gap, we focus on predicting the time of the next user action in smart environments. Due to the lack of public datasets with fine-grained timestamps suitable for this task and associated privacy concerns, we contribute a dataset of 11.6k sequences synthesized based on human annotations of interaction patterns, pairing actions with precise timestamps. To this end, we introduce Timing-Matters, a Transformer-Encoder based method that predicts action timing, achieving 38.30% accuracy on the synthesized dataset, outperforming the best baseline by 6%, and showing 1–6% improvements on other open datasets. Our code and dataset will be publicly released.

Method: Developed formalization in Lean 4 theorem prover using Mathlib 4 probability theory library. Formalized Rademacher complexity, symmetrization arguments, and topological assumptions on hypothesis classes.

Result: Successfully formalized the Rademacher complexity bound (uniform law of large numbers) for generalization error, which was previously unavailable. Demonstrated application to L²-regularization models.

Conclusion: This work establishes the foundation for formalizing advanced statistical learning theory results in theorem provers, enabling rigorous verification of generalization bounds for complex machine learning models.

Abstract: We formalize the generalization error bound using the Rademacher complexity for the Lean 4 theorem prover based on the probability theory in the Mathlib 4 library. Generalization error quantifies the gap between a learning machine’s performance on given training data versus unseen test data, and the Rademacher complexity is a powerful tool to upper-bound the generalization error of a variety of modern learning problems. Previous studies have only formalized extremely simple cases such as bounds by parameter counts and analyses for very simple models (decision stumps). Formalizing the Rademacher complexity bound, also known as the uniform law of large numbers, requires substantial development and is achieved for the first time in this study. In the course of development, we formalize the Rademacher complexity and its unique arguments such as symmetrization, and clarify the topological assumptions on hypothesis classes under which the bound holds. As an application, we also present the formalization of generalization error bound for $L^2$-regularization models.

Method: TSN combines pruning with different sparsity levels to identify performance-preserving weights, adaptive quantization to split weights for different tasks, and weight sharing to boost capacity exploitation and task similarity.

Result: Experimental results show TSN achieves better accuracy than state-of-the-art CL strategies and provides significantly improved model capacity exploitation on common benchmark datasets.

Conclusion: TSN efficiently leverages available capacity, enhances knowledge transfer, reduces computational resource consumption, and enables learning more tasks without capacity saturation.

Abstract: Continual Learning (CL) is a highly relevant setting gaining traction in recent machine learning research. Among CL works, architectural and hybrid strategies are particularly effective due to their potential to adapt the model architecture as new tasks are presented. However, many existing solutions do not efficiently exploit model sparsity, and are prone to capacity saturation due to their inefficient use of available weights, which limits the number of learnable tasks. In this paper, we propose TinySubNets (TSN), a novel architectural CL strategy that addresses the issues through the unique combination of pruning with different sparsity levels, adaptive quantization, and weight sharing. Pruning identifies a subset of weights that preserve model performance, making less relevant weights available for future tasks. Adaptive quantization allows a single weight to be separated into multiple parts which can be assigned to different tasks. Weight sharing between tasks boosts the exploitation of capacity and task similarity, allowing for the identification of a better trade-off between model accuracy and capacity. These features allow TSN to efficiently leverage the available capacity, enhance knowledge transfer, and reduce computational resource consumption. Experimental results involving common benchmark CL datasets and scenarios show that our proposed strategy achieves better results in terms of accuracy than existing state-of-the-art CL strategies. Moreover, our strategy is shown to provide a significantly improved model capacity exploitation. Code released at: https://github.com/lifelonglab/tinysubnets.

Method: STRICT benchmark systematically evaluates models across three dimensions: maximum readable text length, correctness/legibility of generated text, and instruction-following ratio.

Result: Evaluation of state-of-the-art models (both proprietary and open-source) shows persistent limitations in long-range consistency and instruction-following capabilities.

Conclusion: The findings reveal architectural bottlenecks and provide motivation for future research in multimodal generative modeling, with the evaluation pipeline publicly released.

Abstract: While diffusion models have revolutionized text-to-image generation with their ability to synthesize realistic and diverse scenes, they continue to struggle to generate consistent and legible text within images. This shortcoming is commonly attributed to the locality bias inherent in diffusion-based generation, which limits their ability to model long-range spatial dependencies. In this paper, we introduce $\textbf{STRICT}$, a benchmark designed to systematically stress-test the ability of diffusion models to render coherent and instruction-aligned text in images. Our benchmark evaluates models across multiple dimensions: (1) the maximum length of readable text that can be generated; (2) the correctness and legibility of the generated text, and (3) the ratio of not following instructions for generating text. We evaluate several state-of-the-art models, including proprietary and open-source variants, and reveal persistent limitations in long-range consistency and instruction-following capabilities. Our findings provide insights into architectural bottlenecks and motivate future research directions in multimodal generative modeling. We release our entire evaluation pipeline at https://github.com/tianyu-z/STRICT-Bench.

Method: Proposes Duo method that transfers Gaussian diffusion techniques to discrete setting: 1) Curriculum learning strategy guided by Gaussian process to reduce variance and double training speed, 2) Discrete Consistency Distillation adapting continuous consistency distillation for few-step generation.

Result: Models trained with curriculum learning surpass autoregressive models in zero-shot perplexity on 3 of 7 benchmarks. Consistency distillation accelerates sampling by two orders of magnitude, enabling few-step generation in diffusion language models.

Conclusion: The work demonstrates that leveraging insights from Gaussian diffusion can significantly improve both training efficiency and sampling speed of uniform-state discrete diffusion models, making them more competitive with state-of-the-art text generation approaches.

Abstract: Uniform-state discrete diffusion models hold the promise of fast text generation due to their inherent ability to self-correct. However, they are typically outperformed by autoregressive models and masked diffusion models. In this work, we narrow this performance gap by leveraging a key insight: Uniform-state diffusion processes naturally emerge from an underlying Gaussian diffusion. Our method, Duo, transfers powerful techniques from Gaussian diffusion to improve both training and sampling. First, we introduce a curriculum learning strategy guided by the Gaussian process, doubling training speed by reducing variance. Models trained with curriculum learning surpass autoregressive models in zero-shot perplexity on 3 of 7 benchmarks. Second, we present Discrete Consistency Distillation, which adapts consistency distillation from the continuous to the discrete setting. This algorithm unlocks few-step generation in diffusion language models by accelerating sampling by two orders of magnitude. We provide the code and model checkpoints on the project page: http://s-sahoo.github.io/duo

Method: Revisits variational dropout in LoRA context, placing rank-wise variational distributions over adapter components with ARD-style KL term for automatic rank pruning and Monte Carlo sampling for uncertainty.

Result: Empirically improves calibration while producing lighter, faster adapters that automatically prune redundant ranks, eliminating manual rank tuning.

Conclusion: BayesLoRA serves as a practical default for reliable and efficient PEFT by simultaneously addressing uncertainty estimation and rank selection through its Bayesian framework.

Abstract: Parameter-efficient fine-tuning (PEFT) methods such as LoRA adapt large language models by inserting low-rank adapters, but they leave open two key questions: how to give the adapted model calibrated uncertainty, and how to choose the adapter rank. Existing approaches to uncertainty are typically post-hoc, while rank selection is manual and task-specific. BayesLoRA revisits variational dropout in the LoRA setting and shows that the natural unit of stochasticity is not individual weights but entire ranks of the adapter. By placing rank-wise variational distributions over adapter components, BayesLoRA defines a posterior that (i) yields calibrated predictions through adapter-only Monte Carlo sampling and (ii) prunes redundant ranks automatically via an ARD-style KL term. Theoretical analysis shows that this rank-parameterized posterior localizes uncertainty to the adapted subspace and explains amplification under distribution shift. Empirically, BayesLoRA improves calibration while at the same time producing lighter, faster adapters, removing the need to tune ranks by hand. This dual role of uncertainty estimation and uncertainty-driven pruning suggests BayesLoRA may offer a practical default for reliable and efficient PEFT.

Method: Proposes Mixture of Expert Prompt Tuning (MEPT) that leverages Mixture of Experts architecture with multiple prompt experts to adaptively learn diverse and non-stationary data distributions.

Result: Outperforms state-of-the-art parameter efficient baselines on SuperGLUE with 1.94% mean accuracy improvement and 79.25% reduction in activated prompts.

Conclusion: MEPT provides an effective and efficient manifold-mapping framework supported by theoretical insights from manifold learning and validated through neural activation pathway visualization.

Abstract: Considering deep neural networks as manifold mappers, the pretrain-then-fine-tune paradigm can be interpreted as a two-stage process: pretrain establishes a broad knowledge base, and fine-tune adjusts the model parameters to activate specific neural pathways to align with the target manifold. Although prior fine-tuning approaches demonstrate success, their rigid parameter space limits their ability to dynamically activate appropriate neural pathways, rendering them ill-equipped to adapt flexibly to the diverse and evolving data distributions. In light of this view, we propose a novel approach, Mixture of Expert Prompt Tuning (MEPT), as an effective and efficient manifold-mapping framework. MEPT leverages the Mixture of Experts architecture by integrating multiple prompt experts to adaptively learn diverse and non-stationary data distributions. Empirical evaluations demonstrate that MEPT outperforms several state-of-the-art parameter efficient baselines on SuperGLUE, achieving notable improvements in mean accuracy (e.g., 1.94%) while significantly reducing activated prompts by 79.25%. The effectiveness of MEPT is further supported by theoretical insights from manifold learning and validated through neural activation pathway visualization results. Our code is avaliable at https://runjia.tech/emnlp_mept/.

Method: Dion replaces Newton-Schulz iteration with amortized power iteration on a momentum buffer, avoiding full-matrix reconstruction. It uses rank-fraction parameter with error feedback for low-rank updates that integrate cleanly with weight sharding.

Result: On language models from 160M to 3B parameters, Dion retains orthonormalized update benefits while significantly reducing wall-clock time at scale, making it practical for large foundation models.

Conclusion: Dion provides a scalable and efficient alternative to existing orthonormalized update methods, enabling practical use in next-generation foundation model training while maintaining stability and hyperparameter transfer advantages.

Abstract: Orthonormalized updates accelerate training, improve stability, and enable robust hyperparameter transfer, but existing methods like Muon rely on dense matrix operations that clash with sharded weights in large-scale LLM training, causing high compute and communication cost. We introduce Dion (Distributed Orthonormalization), a scalable and efficient update rule that replaces Newton-Schulz iteration with amortized power iteration on a momentum buffer, avoiding full-matrix reconstruction and integrating cleanly with weight sharding. The rank-fraction parameter with error feedback enables low-rank updates that balance quality with significant cost savings. On language models from 160M to 3B parameters, Dion retains the benefits of orthonormalized updates, while markedly reducing wall-clock time at scale, making it a practical optimizer for next-generation foundation models. Code is available at: https://github.com/microsoft/dion/

Method: FFTKF applies frequency-domain filtering to shift privacy noise into high-frequency components, preserving low-frequency gradient signals, and uses a scalar-gain Kalman filter with finite-difference Hessian approximation to refine denoised gradients.

Result: Achieves higher test accuracy than DP-SGD and DiSK on MNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet with various architectures (CNNs, Wide ResNets, Vision Transformers), with per-iteration complexity O(d log d).

Conclusion: FFTKF ensures equivalent privacy while delivering a stronger privacy-utility trade-off through reduced variance and controlled bias, making it an effective alternative to standard DP-SGD.

Abstract: Differential Privacy (DP) has emerged as a key framework for protecting sensitive data in machine learning, but standard DP-SGD often suffers from significant accuracy loss due to injected noise. To address this limitation, we introduce the FFT-Enhanced Kalman Filter (FFTKF), a differentially private optimization method that improves gradient quality while preserving $(\varepsilon, \delta)$-DP guarantees. FFTKF applies frequency-domain filtering to shift privacy noise into less informative high-frequency components, preserving the low-frequency gradient signals that carry most learning information. A scalar-gain Kalman filter with a finite-difference Hessian approximation further refines the denoised gradients. The method has per-iteration complexity $\mathcal{O}(d \log d)$ and achieves higher test accuracy than DP-SGD and DiSK on MNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet with CNNs, Wide ResNets, and Vision Transformers. Theoretical analysis shows that FFTKF ensures equivalent privacy while delivering a stronger privacy–utility trade-off through reduced variance and controlled bias.

Method: Extended robust UCB policy that generalizes Lattimore’s work to arbitrarily chosen p>q>1 moments with known controlled relationship, without requiring knowledge of reward distributions as long as p-th moments exist.

Result: Achieves optimal regret growth order O(log T) and provides broadened application area for UCB policies with heavy-tailed distributions. Also achieves near-optimal regret without distribution knowledge.

Conclusion: Demonstrates the simplicity and power of UCB policies for both heavy-tailed and light-tailed reward distributions, significantly expanding their practical applicability.

Abstract: The multi-armed bandit (MAB) problems are widely studied in fields of operations research, stochastic optimization, and reinforcement learning. In this paper, we consider the classical MAB model with heavy-tailed reward distributions and introduce the extended robust UCB policy, which is an extension of the results of Bubeck et al. [5] and Lattimore [22] that are further based on the pioneering idea of UCB policies [e.g. Auer et al. 3]. The previous UCB policies require some strict conditions on reward distributions, which can be difficult to guarantee in practical scenarios. Our extended robust UCB generalizes Lattimore’s seminary work (for moments of orders $p=4$ and $q=2$) to arbitrarily chosen $p>q>1$ as long as the two moments have a known controlled relationship, while still achieving the optimal regret growth order $O(log T)$, thus providing a broadened application area of UCB policies for heavy-tailed reward distributions. Furthermore, we achieve a near-optimal regret order without any knowledge of the reward distributions as long as their $p$-th moments exist for some $p>1$. Finally, we briefly present our earlier work on light-tailed reward distributions for a complete illustration of the amazing simplicity and power of UCB policies.

Method: Systematic categorization of attacks and defenses according to defined threat models and single- versus multi-agent settings, analyzing 86 recent studies on RL security.

Result: Provides a structured framework for understanding RL security vulnerabilities and defense mechanisms, with specific applicability analysis for autonomous driving contexts.

Conclusion: The survey offers insights to inform the design of robust RL systems, particularly for safety-critical applications like autonomous driving, by providing systematic guidance on attack and defense categorization.

Abstract: Reinforcement learning (RL) enables agents to learn optimal behaviors through interaction with their environment and has been increasingly deployed in safety-critical applications, including autonomous driving. Despite its promise, RL is susceptible to attacks designed either to compromise policy learning or to induce erroneous decisions by trained agents. Although the literature on RL security has grown rapidly and several surveys exist, existing categorizations often fall short in guiding the selection of appropriate defenses for specific systems. In this work, we present a comprehensive survey of 86 recent studies on RL security, addressing these limitations by systematically categorizing attacks and defenses according to defined threat models and single- versus multi-agent settings. Furthermore, we examine the relevance and applicability of state-of-the-art attacks and defense mechanisms within the context of autonomous driving, providing insights to inform the design of robust RL systems.

Method: Reformulate POMDPs as fully observable processes using fixed-length observation histories as augmented states, and propose a lightweight temporal encoder based on depthwise separable convolution and self-attention integrated into an actor-critic framework.

Result: Achieves superior performance on continuous control benchmarks under partial observability, demonstrating improved scalability and efficiency compared to recurrent and Transformer-based models.

Conclusion: Lightweight temporal encoding can significantly improve AI system scalability under uncertainty, advancing the development of agents capable of robust reasoning in real-world environments with incomplete or delayed information.

Abstract: Partially Observable Markov Decision Processes (POMDPs) remain a core challenge in reinforcement learning due to incomplete state information. We address this by reformulating POMDPs as fully observable processes with fixed-length observation histories as augmented states. To efficiently encode these histories, we propose a lightweight temporal encoder based on depthwise separable convolution and self-attention, avoiding the overhead of recurrent and Transformer-based models. Integrated into an actor-critic framework, our method achieves superior performance on continuous control benchmarks under partial observability. More broadly, this work shows that lightweight temporal encoding can improve the scalability of AI systems under uncertainty. It advances the development of agents capable of reasoning robustly in real-world environments where information is incomplete or delayed.

Method: SUDE uses a sampling-based approach: first seeks landmarks to construct the low-dimensional skeleton, then incorporates non-landmarks into the learned space using constrained locally linear embedding (CLLE).

Result: SUDE shows superior scalability with respect to data size and embedding dimension, excellent performance in cluster separation and structure preservation, and notable robustness as sampling rate decreases.

Conclusion: SUDE effectively addresses scalability and cluster distortion problems in manifold learning, demonstrating promising performance on synthetic datasets, real-world benchmarks, single-cell data analysis, and ECG anomaly detection.

Abstract: As a pivotal branch of machine learning, manifold learning uncovers the intrinsic low-dimensional structure within complex nonlinear manifolds in high-dimensional space for visualization, classification, clustering, and gaining key insights. Although existing techniques have achieved remarkable successes, they suffer from extensive distortions of cluster structure, which hinders the understanding of underlying patterns. Scalability issues also limit their applicability for handling large-scale data. We hence propose a sampling-based Scalable manifold learning technique that enables Uniform and Discriminative Embedding, namely SUDE, for large-scale and high-dimensional data. It starts by seeking a set of landmarks to construct the low-dimensional skeleton of the entire data, and then incorporates the non-landmarks into the learned space based on the constrained locally linear embedding (CLLE). We empirically validated the effectiveness of SUDE on synthetic datasets and real-world benchmarks, and applied it to analyze single-cell data and detect anomalies in electrocardiogram (ECG) signals. SUDE exhibits distinct advantage in scalability with respect to data size and embedding dimension, and has promising performance in cluster separation, integrity, and global structure preservation. The experiments also demonstrate notable robustness in embedding quality as the sampling rate decreases.

Method: Theoretical analysis of one hidden layer ReLU networks with small initialization, studying the early alignment phase where neurons align towards key directions during training.

Result: Small initialization induces neuron alignment that creates sparse network representations, directly related to gradient flow’s implicit bias. However, this alignment can prevent convergence to global minima in overparameterized networks.

Conclusion: The early alignment phase explains implicit bias but reveals limitations - sparse representations from small initialization may lead to convergence on spurious stationary points rather than optimal solutions.

Abstract: Training neural networks with first order optimisation methods is at the core of the empirical success of deep learning. The scale of initialisation is a crucial factor, as small initialisations are generally associated to a feature learning regime, for which gradient descent is implicitly biased towards simple solutions. This work provides a general and quantitative description of the early alignment phase, originally introduced by Maennel et al. (2018). For small initialisation and one hidden ReLU layer networks, the early stage of the training dynamics leads to an alignment of the neurons towards key directions. This alignment induces a sparse representation of the network, which is directly related to the implicit bias of gradient flow at convergence. This sparsity inducing alignment however comes at the expense of difficulties in minimising the training objective: we also provide a simple data example for which overparameterised networks fail to converge towards global minima and only converge to a spurious stationary point instead.

Method: Proposes Feature-Adaptive INR (FA-INR) with cross-attention to an augmented memory bank for flexible feature learning, plus coordinate-guided mixture of experts for improved scalability and specialization.

Result: Experiments on three large-scale ensemble simulation datasets show FA-INR achieves state-of-the-art fidelity while significantly reducing model size.

Conclusion: FA-INR establishes a new trade-off frontier between accuracy and compactness for INR-based surrogate models in scientific simulations.

Abstract: Effective surrogate models are critical for accelerating scientific simulations. Implicit neural representations (INRs) offer a compact and continuous framework for modeling spatially structured data, but they often struggle with complex scientific fields exhibiting localized, high-frequency variations. Recent approaches address this by introducing additional features along rigid geometric structures (e.g., grids), but at the cost of flexibility and increased model size. In this paper, we propose a simple yet effective alternative: Feature-Adaptive INR (FA-INR). FA-INR leverages cross-attention to an augmented memory bank to learn flexible feature representations, enabling adaptive allocation of model capacity based on data characteristics, rather than rigid structural assumptions. To further improve scalability, we introduce a coordinate-guided mixture of experts (MoE) that enhances the specialization and efficiency of feature representations. Experiments on three large-scale ensemble simulation datasets show that FA-INR achieves state-of-the-art fidelity while significantly reducing model size, establishing a new trade-off frontier between accuracy and compactness for INR-based surrogates.

Method: Uses Symbolic Descent to analyze noisy batch gradient sequences and dynamically assess weight importance scores. SEVEN specifically favors weights with consistently high sensitivity (small gradient noise) and preserves them during pruning.

Result: Extensive experiments show SEVEN achieves significant improvements across various Transformer models in natural language, QA, and image classification tasks. It performs well at different sparsity levels and under various fine-tuning strategies.

Conclusion: SEVEN provides an effective pruning approach that produces more robust and high-performing compressed Transformer models by focusing on weights with consistent sensitivity rather than noisy gradients.

Abstract: Large-scale Transformer models (TM) have demonstrated outstanding performance across various tasks. However, their considerable parameter size restricts their applicability, particularly on mobile devices. Due to the dynamic and intricate nature of gradients on TM compared to Convolutional Neural Networks, commonly used pruning methods tend to retain weights with larger gradient noise. This results in pruned models that are sensitive to sparsity and datasets, exhibiting suboptimal performance. Symbolic Descent (SD) is a general approach for training and fine-tuning TM. In this paper, we attempt to describe the noisy batch gradient sequences on TM through the cumulative process of SD. We utilize this design to dynamically assess the importance scores of weights.SEVEN is introduced by us, which particularly favors weights with consistently high sensitivity, i.e., weights with small gradient noise. These weights are tended to be preserved by SEVEN. Extensive experiments on various TM in natural language, question-answering, and image classification domains are conducted to validate the effectiveness of SEVEN. The results demonstrate significant improvements of SEVEN in multiple pruning scenarios and across different sparsity levels. Additionally, SEVEN exhibits robust performance under various fine-tuning strategies. The code is publicly available at https://github.com/xiaojinying/SEVEN.

Method: Proposes the concept of learning gap using feature maps from the penultimate layer, which accurately correlates with generalization. Introduces LNPT framework where mature cloud networks provide online guidance for pruning and learning on smart devices using unlabeled data.

Result: Experiments show that the learning gap aligns with generalization performance variations. The LNPT approach demonstrates superiority over supervised training methods.

Conclusion: The learning gap concept provides a more reliable indicator for network pruning than weight norm distances, enabling effective deployment of pruned neural networks on resource-constrained smart devices through the proposed LNPT framework.

Abstract: Pruning before training enables the deployment of neural networks on smart devices. By retaining weights conducive to generalization, pruned networks can be accommodated on resource-constrained smart devices. It is commonly held that the distance on weight norms between the initialized and the fully-trained networks correlates with generalization performance. However, as we have uncovered, inconsistency between this metric and generalization during training processes, which poses an obstacle to determine the pruned structures on smart devices in advance. In this paper, we introduce the concept of the learning gap, emphasizing its accurate correlation with generalization. Experiments show that the learning gap, in the form of feature maps from the penultimate layer of networks, aligns with variations of generalization performance. We propose a novel learning framework, LNPT, which enables mature networks on the cloud to provide online guidance for network pruning and learning on smart devices with unlabeled data. Our results demonstrate the superiority of this approach over supervised training.

Method: Graph-based state representation combining item attributes and warehouse topology. Novel neural network architecture integrating Graph Neural Network (GNN) for encoding topological/item information and Transformer for global priority assignments.

Result: Extensive numerical experiments show superiority over heuristic methods, demonstrating effective optimization of retrieval tardiness and strong generalization to diverse storage layouts.

Conclusion: The proposed GNN-Transformer framework effectively addresses retrieval challenges in multi-deep storage systems with heterogeneous items, offering improved flexibility and performance compared to conventional approaches.

Abstract: In modern industrial and logistics environments, the rapid expansion of fast delivery services has heightened the demand for storage systems that combine high efficiency with increased density. Multi-deep autonomous vehicle storage and retrieval systems (AVS/RS) present a viable solution for achieving greater storage density. However, these systems encounter significant challenges during retrieval operations due to lane blockages. A conventional approach to mitigate this issue involves storing items with homogeneous characteristics in a single lane, but this strategy restricts the flexibility and adaptability of multi-deep storage systems. In this study, we propose a deep reinforcement learning-based framework to address the retrieval problem in multi-deep storage systems with heterogeneous item configurations. Each item is associated with a specific due date, and the objective is to minimize total tardiness. To effectively capture the system’s topology, we introduce a graph-based state representation that integrates both item attributes and the local topological structure of the multi-deep warehouse. To process this representation, we design a novel neural network architecture that combines a Graph Neural Network (GNN) with a Transformer model. The GNN encodes topological and item-specific information into embeddings for all directly accessible items, while the Transformer maps these embeddings into global priority assignments. The Transformer’s strong generalization capability further allows our approach to be applied to storage systems with diverse layouts. Extensive numerical experiments, including comparisons with heuristic methods, demonstrate the superiority of the proposed neural network architecture and the effectiveness of the trained agent in optimizing retrieval tardiness.

Method: Proposes TED pruning using Internal Generalization (IG) and Internal Generalization Distance (IGD) optimization objective to measure model’s ability to improve on pruned data while fitting retained data, with fast estimation via Taylor approximation and progressive pruning strategy.

Result: Achieves lossless performance with 60-70% of data in experiments on image classification, natural language understanding, and large language model fine-tuning.

Conclusion: TED pruning effectively addresses overfitting under high pruning ratios through IGD optimization, providing an efficient dataset compression method with strong empirical results across multiple domains.

Abstract: Large language models have demonstrated strong performance in recent years, but the high cost of training drives the need for efficient methods to compress dataset sizes. We propose TED pruning, a method that addresses the challenge of overfitting under high pruning ratios by quantifying the model’s ability to improve performance on pruned data while fitting retained data, known as Internal Generalization (IG). TED uses an optimization objective based on Internal Generalization Distance (IGD), measuring changes in IG before and after pruning to align with true generalization performance and achieve implicit regularization. The IGD optimization objective was verified to allow the model to achieve the smallest upper bound on generalization error. The impact of small mask fluctuations on IG is studied through masks and Taylor approximation, and fast estimation of IGD is enabled. In analyzing continuous training dynamics, the prior effect of IGD is validated, and a progressive pruning strategy is proposed. Experiments on image classification, natural language understanding, and large language model fine-tuning show TED achieves lossless performance with 60-70% of the data. Upon acceptance, our code will be made publicly available.

Method: Created SIDED synthetic dataset using Digital Twin simulations across multiple industrial facilities and locations. Proposed AMDA method that intelligently scales appliance power contributions based on their relative impact for efficient data augmentation.

Result: AMDA-augmented models achieved Normalized Disaggregation Error of 0.093, significantly outperforming non-augmented (0.451) and random augmentation (0.290) models. Particularly effective for complex appliances like combined heat and power systems.

Conclusion: The combination of synthetic dataset generation and intelligent data augmentation effectively addresses data scarcity in industrial NILM, improving model generalization and performance in out-of-sample scenarios.

Abstract: Industrial Non-Intrusive Load Monitoring (NILM) is limited by the scarcity of high-quality datasets and the complex variability of industrial energy consumption patterns. To address data scarcity and privacy issues, we introduce the Synthetic Industrial Dataset for Energy Disaggregation (SIDED), an open-source dataset generated using Digital Twin simulations. SIDED includes three types of industrial facilities across three different geographic locations, capturing diverse appliance behaviors, weather conditions, and load profiles. We also propose the Appliance-Modulated Data Augmentation (AMDA) method, a computationally efficient technique that enhances NILM model generalization by intelligently scaling appliance power contributions based on their relative impact. We show in experiments that NILM models trained with AMDA-augmented data significantly improve the disaggregation of energy consumption of complex industrial appliances like combined heat and power systems. Specifically, in our out-of-sample scenarios, models trained with AMDA achieved a Normalized Disaggregation Error of 0.093, outperforming models trained without data augmentation (0.451) and those trained with random data augmentation (0.290). Data distribution analyses confirm that AMDA effectively aligns training and test data distributions, enhancing model generalization.

Method: EchoAlign consists of two components: EchoMod uses controllable generative models to modify instance features to align with noisy labels while preserving intrinsic characteristics; EchoSelect mitigates distribution shifts by retaining original instances with correct labels based on feature similarity distributions.

Result: Extensive experiments show EchoAlign significantly outperforms state-of-the-art methods, especially in high-noise environments. Under 30% instance-dependent noise, it retains nearly twice the correctly labeled samples with 99% selection accuracy.

Conclusion: EchoAlign represents a paradigm shift from label correction to instance modification, demonstrating superior accuracy and robustness in learning from noisy labels across multiple benchmark datasets.

Abstract: Noisy labels significantly hinder the accuracy and generalization of machine learning models, particularly when resulting from ambiguous instance features that complicate correct labeling. Traditional approaches, such as those relying on transition matrices for label correction, often struggle to effectively resolve such ambiguity, due to their inability to capture complex relationships between instances and noisy labels. In this paper, we propose EchoAlign, a paradigm shift in learning from noisy labels. Unlike previous methods that attempt to correct labels, EchoAlign treats noisy labels ($\tilde{Y}$) as accurate and modifies corresponding instances ($X$) to better align with these labels. The EchoAlign framework comprises two main components: (1) EchoMod leverages controllable generative models to selectively modify instance features, achieving alignment with noisy labels while preserving intrinsic instance characteristics such as shape, texture, and semantic identity. (2) EchoSelect mitigates distribution shifts introduced by instance modifications by strategically retaining a substantial subset of original instances with correct labels. Specifically, EchoSelect exploits feature similarity distributions between original and modified instances to accurately distinguish between correctly and incorrectly labeled samples. Extensive experiments across three benchmark datasets demonstrate that EchoAlign significantly outperforms state-of-the-art methods, particularly in high-noise environments, achieving superior accuracy and robustness. Notably, under 30% instance-dependent noise, EchoSelect retains nearly twice the number of correctly labeled samples compared to previous methods, maintaining 99% selection accuracy, thereby clearly illustrating the effectiveness of EchoAlign. The implementation of EchoAlign is publicly available at https://github.com/KevinCarpricorn/EchoAlign/tree/main.

Method: The authors formulate attention as a map between infinite-dimensional function spaces and prove it’s approximated by practical implementations. They introduce transformer neural operators with a slight modification of standard architecture, and develop a patching strategy generalization for efficient computation on multi-dimensional domains.

Result: The paper provides the first universal approximation result for transformer neural operators. Numerical experiments on various operator learning problems demonstrate the effectiveness of their function space attention formulations.

Conclusion: The attention mechanism can be successfully extended to function spaces, enabling the development of transformer neural operators that effectively learn mappings between infinite-dimensional function spaces while maintaining computational efficiency through novel patching strategies.

Abstract: Transformers, and the attention mechanism in particular, have become ubiquitous in machine learning. Their success in modeling nonlocal, long-range correlations has led to their widespread adoption in natural language processing, computer vision, and time series problems. Neural operators, which map spaces of functions into spaces of functions, are necessarily both nonlinear and nonlocal if they are universal; it is thus natural to ask whether the attention mechanism can be used in the design of neural operators. Motivated by this, we study transformers in the function space setting. We formulate attention as a map between infinite dimensional function spaces and prove that the attention mechanism as implemented in practice is a Monte Carlo or finite difference approximation of this operator. The function space formulation allows for the design of transformer neural operators, a class of architectures designed to learn mappings between function spaces. In this paper, we state and prove the first universal approximation result for transformer neural operators, using only a slight modification of the architecture implemented in practice. The prohibitive cost of applying the attention operator to functions defined on multi-dimensional domains leads to the need for more efficient attention-based architectures. For this reason we also introduce a function space generalization of the patching strategy from computer vision, and introduce a class of associated neural operators. Numerical results, on an array of operator learning problems, demonstrate the promise of our approaches to function space formulations of attention and their use in neural operators.

Method: LIVAR introduces: 1) variance-weighted classifier aggregation using naturally emergent feature statistics, and 2) explainability-driven LoRA merging technique based on SHAP analysis of existing update parameter patterns.

Result: The method achieves state-of-the-art performance on multiple benchmarks while maintaining seamless integration with existing FL methods, without any architectural overhead.

Conclusion: Effective model merging can be achieved solely through existing training signals, establishing a new paradigm for efficient federated model aggregation.

Abstract: Federated Learning (FL) enables collaborative model training across distributed clients while preserving data privacy. While existing approaches for aggregating client-specific classification heads and adapted backbone parameters require architectural modifications or loss function changes, our method uniquely leverages intrinsic training signals already available during standard optimization. We present LIVAR (Layer Importance and VARiance-based merging), which introduces: i) a variance-weighted classifier aggregation scheme using naturally emergent feature statistics, and ii) an explainability-driven LoRA merging technique based on SHAP analysis of existing update parameter patterns. Without any architectural overhead, LIVAR achieves state-of-the-art performance on multiple benchmarks while maintaining seamless integration with existing FL methods. This work demonstrates that effective model merging can be achieved solely through existing training signals, establishing a new paradigm for efficient federated model aggregation. The code is available at https://github.com/aimagelab/fed-mammoth.

Method: Proposes Rob-ReL algorithm that applies robust optimization to maximize performance with respect to the worst-case reward in the feasible set of plausible rewards.

Result: Developed theoretical guarantees on sample and iteration complexity for Rob-ReL, and provided proof-of-concept experimental validation.

Conclusion: The robust approach provides a principled way to handle identifiability issues in reward learning, ensuring better performance in downstream applications despite uncertainty about the true target reward.

Abstract: In Reward Learning (ReL), we are given feedback on an unknown target reward, and the goal is to use this information to recover it in order to carry out some downstream application, e.g., planning. When the feedback is not informative enough, the target reward is only partially identifiable, i.e., there exists a set of rewards, called the feasible set, that are equally plausible candidates for the target reward. In these cases, the ReL algorithm might recover a reward function different from the target reward, possibly leading to a failure in the application. In this paper, we introduce a general ReL framework that permits to quantify the drop in “performance” suffered in the considered application because of identifiability issues. Building on this, we propose a robust approach to address the identifiability problem in a principled way, by maximizing the “performance” with respect to the worst-case reward in the feasible set. We then develop Rob-ReL, a ReL algorithm that applies this robust approach to the subset of ReL problems aimed at assessing a preference between two policies, and we provide theoretical guarantees on sample and iteration complexity for Rob-ReL. We conclude with a proof-of-concept experiment to illustrate the considered setting.

Method: Adapted a general-purpose LLM (Gemma-3-1b-it) to nuclear domain using Low-Rank Adaptation fine-tuning, compared neuron activation patterns, and employed neuron silencing techniques to probe causal roles of specialized neurons.

Result: Silencing individual specialized neurons showed no significant effect, but collective deactivation caused statistically significant performance degradation and impaired generation of detailed technical information.

Conclusion: The methodology enhances transparency of black-box models by tracing domain expertise to verifiable neural circuits, providing a pathway for nuclear regulatory compliance and AI deployment in safety-critical operations.

Abstract: The integration of Large Language Models (LLMs) into safety-critical domains, such as nuclear engineering, necessitates a deep understanding of their internal reasoning processes. This paper presents a novel methodology for interpreting how an LLM encodes and utilizes domain-specific knowledge, using a Boiling Water Reactor system as a case study. We adapted a general-purpose LLM (Gemma-3-1b-it) to the nuclear domain using a parameter-efficient fine-tuning technique known as Low-Rank Adaptation. By comparing the neuron activation patterns of the base model to those of the fine-tuned model, we identified a sparse set of neurons whose behavior was significantly altered during the adaptation process. To probe the causal role of these specialized neurons, we employed a neuron silencing technique. Our results demonstrate that while silencing most of these specialized neurons individually did not produce a statistically significant effect, deactivating the entire group collectively led to a statistically significant degradation in task performance. Qualitative analysis further revealed that silencing these neurons impaired the model’s ability to generate detailed, contextually accurate technical information. This paper provides a concrete methodology for enhancing the transparency of an opaque black-box model, allowing domain expertise to be traced to verifiable neural circuits. This offers a pathway towards achieving nuclear-grade artificial intelligence (AI) assurance, addressing the verification and validation challenges mandated by nuclear regulatory frameworks (e.g., 10 CFR 50 Appendix B), which have limited AI deployment in safety-critical nuclear operations.

Method: Proposes SafeSwitch framework with prober-based internal state monitor to detect harmful intentions, and activates a safety head only when necessary. Uses less than 6% parameter tuning.

Result: Reduces harmful outputs by approximately 80% on harmful queries while maintaining strong utility, achieving Pareto optimal performance among several methods.

Conclusion: SafeSwitch demonstrates LLMs’ capacity for self-awareness and safety reflection, offering more informative, context-aware refusals with minimal parameter tuning.

Abstract: Large language models (LLMs) exhibit exceptional capabilities across various tasks but also pose risks by generating harmful content. Existing safety mechanisms, while improving model safety, often lead to overly cautious behavior and fail to fully leverage LLMs’ internal cognitive processes. Inspired by humans’ reflective thinking capability, we first show that LLMs can similarly perform internal assessments about safety in their internal states. Building on this insight, we propose SafeSwitch, a dynamic framework that regulates unsafe outputs by utilizing the prober-based internal state monitor that actively detects harmful intentions, and activates a safety head that leads to safer and more conservative responses only when necessary. SafeSwitch reduces harmful outputs by approximately 80% on harmful queries while maintaining strong utility, reaching a Pareto optimal among several methods. Our method is also advantageous over traditional methods in offering more informative, context-aware refusals, and achieves these benefits while only tuning less than 6% of the original parameters. SafeSwitch demonstrates large language models’ capacity for self-awareness and reflection regarding safety, offering a promising approach to more nuanced and effective safety controls. Codes for this work are available at https://github.com/Hanpx20/SafeSwitch.

Method: Combines multiple variational auto-encoders with a class classifier and adversarially trained modality classifier, using regularized relativistic GAN loss for training stability. Trains simultaneously on large reference database and small target dataset.

Result: High prediction accuracy on TCGA database (10-34 cancer types), robust performance with limited supervision, and stability to class imbalance. Validated on synthetic datasets first.

Conclusion: MODIS is a promising solution for challenging integration scenarios, particularly diagonal integration with small sample sizes typical of rare disease studies.

Abstract: An important objective in computational biology is the efficient integration of multi-omics data. The task of integration comes with challenges: multi-omics data are most often unpaired (requiring diagonal integration), partially labeled with information about biological conditions, and in some situations such as rare diseases, only very small datasets are available. We present MODIS, a semi supervised framework designed to account for these particular challenges. To address the challenge of very small datasets, we propose to exploit information contained in larger multi-omics databases by training our model on a large reference database and a small target dataset simultaneously, effectively turning the problem of transfer learning into a problem of learning with class imbalance. MODIS performs diagonal integration on unpaired samples, leveraging class-labels to align modalities despite class imbalance and data scarcity. The architecture combines multiple variational auto-encoders, a class classifier and an adversarially trained modality classifier. To ensure training stability, we adapted a regularized relativistic GAN loss to this setting. We first validate MODIS on a synthetic dataset to assess the level of supervision needed for accurate alignment and to quantify the impact of class imbalance on predictive performance. We then apply our approach to the large public TCGA database, considering between 10 and 34 classes (cancer types and normal tissue). MODIS demonstrates high prediction accuracy, robust performance with limited supervision, and stability to class imbalance. These results position MODIS as a promising solution for challenging integration scenarios, particularly diagonal integration with a small number of samples, typical of rare diseases studies. The code is available at https://github.com/VILLOUTREIXLab/MODIS.

Method: Implemented Permutation Decision Trees with strategic trailing stop-loss, using 5-minute candlestick data and technical indicators. Trained on 3-month Yahoo Finance dataset with hyperparameter optimization for risk management.

Result: PDT bot achieved 1.1802% profit, outperforming LSTM (0.557%), RNN (0.5896%), and significantly beating buy-and-hold strategy (-2.29%). All algorithmic approaches showed positive returns.

Conclusion: Permutation Decision Trees with strategic trailing provide an effective approach for short-term trading in regulated markets, demonstrating superior performance compared to neural network alternatives and traditional buy-and-hold strategy.

Abstract: In this paper, we explore the application of Permutation Decision Trees (PDT) and strategic trailing for predicting stock market movements and executing profitable trades in the Indian stock market. We focus on high-frequency data using 5-minute candlesticks for the top 50 stocks listed in the NIFTY 50 index and Forex pairs such as XAUUSD and EURUSD. We implement a trading strategy that aims to buy stocks at lower prices and sell them at higher prices, capitalizing on short-term market fluctuations. Due to regulatory constraints in India, short selling is not considered in our strategy. The model incorporates various technical indicators and employs hyperparameters such as the trailing stop-loss value and support thresholds to manage risk effectively. We trained and tested data on a 3 month dataset provided by Yahoo Finance. Our bot based on Permutation Decision Tree achieved a profit of 1.1802% over the testing period, where as a bot based on LSTM gave a return of 0.557% over the testing period and a bot based on RNN gave a return of 0.5896% over the testing period. All of the bots outperform the buy-and-hold strategy, which resulted in a loss of 2.29%.

Method: TFC-SR enhances standard experience replay with an Active Recall Probe - a periodic, task-aware evaluation mechanism that stabilizes past knowledge representations. It incorporates human-inspired strategies like Active Recall, Deliberate Practice, and Spaced Repetition.

Result: TFC-SR significantly outperforms leading regularization-based and replay-based baselines. On Split CIFAR-100, it achieved 13.17% final accuracy compared to 7.40% for Standard Experience Replay. The advantage comes from the stabilizing effect of the probe itself, not just replay volume.

Conclusion: TFC-SR is a robust and efficient continual learning approach that demonstrates the importance of integrating active memory retrieval mechanisms. It performs better in memory-constrained environments, though higher replay volume remains effective when memory is abundant.

Abstract: Deep neural networks often suffer from a critical limitation known as catastrophic forgetting, where performance on past tasks degrades after learning new ones. This paper introduces a novel continual learning approach inspired by human learning strategies like Active Recall, Deliberate Practice, and Spaced Repetition, named Task-Focused Consolidation with Spaced Recall (TFC-SR). TFC-SR enhances the standard experience replay framework with a mechanism we term the Active Recall Probe. It is a periodic, task-aware evaluation of the model’s memory that stabilizes the representations of past knowledge. We test TFC-SR on the Split MNIST and the Split CIFAR-100 benchmarks against leading regularization-based and replay-based baselines. Our results show that TFC-SR performs significantly better than these methods. For instance, on the Split CIFAR-100, it achieves a final accuracy of 13.17% compared to Standard Experience Replay’s 7.40%. We demonstrate that this advantage comes from the stabilizing effect of the probe itself, and not from the difference in replay volume. Additionally, we analyze the trade-off between memory size and performance and show that while TFC-SR performs better in memory-constrained environments, higher replay volume is still more effective when available memory is abundant. We conclude that TFC-SR is a robust and efficient approach, highlighting the importance of integrating active memory retrieval mechanisms into continual learning systems.

Method: Four-step framework: 1) Safety filtering to classify web data, 2) Safety rephrasing to recontextualize unsafe content, 3) Native refusal training with RefuseWeb and Moral Education datasets, 4) Harmfulness-tag annotated pretraining using special tokens to steer models away from unsafe generations.

Result: Safety-pretrained models reduce attack success rates from 38.8% to 8.4% on standard LLM safety benchmarks while maintaining performance on general tasks with no degradation.

Conclusion: Building safety into LLMs during pretraining through a comprehensive data-centric framework is effective at significantly reducing harmful content generation without compromising general model capabilities.

Abstract: As large language models (LLMs) are increasingly deployed in high-stakes settings, the risk of generating harmful or toxic content remains a central challenge. Post-hoc alignment methods are brittle: once unsafe patterns are learned during pretraining, they are hard to remove. In this work, we present a data-centric pretraining framework that builds safety into the model from the start. Our framework consists of four key steps: (i) Safety Filtering: building a safety classifier to classify webdata into safe and unsafe categories; (ii) Safety Rephrasing: we recontextualize unsafe webdata into safer narratives; (iii) Native Refusal: we develop RefuseWeb and Moral Education pretraining datasets that actively teach model to refuse on unsafe content and the moral reasoning behind it, and (iv) Harmfulness-Tag annotated pretraining: we flag unsafe content during pretraining using a special token, and use it to steer model away from unsafe generations at inference. Our safety-pretrained models reduce attack success rates from 38.8% to 8.4% on standard LLM safety benchmarks with no performance degradation on general tasks.

Method: Comprehensive convergence analysis across three FL frameworks: basic asynchronous FL (AsynFL), compressed asynchronous FL (AsynFLC), and compressed asynchronous FL with error feedback (AsynFLC-EF). Theoretical analysis decouples complex interactions between system constraints and compression mechanisms.

Result: Established improved convergence analysis with fewer assumptions and superior convergence rate for AsynFL. Derived sufficient conditions for AsynFLC convergence showing nonlinear interaction between delay and compression rate. Proved EF reduces gradient estimation variance, enabling AsynFLC-EF to match AsynFL’s convergence rate. Experimental results validate analytical findings.