Method: Evaluated word-level, character-level, n-gram, and BPE tokenization strategies using Word2Vec on a 10,000-article Wikipedia corpus for Turkish and Finnish. Models were tested on Named Entity Recognition (NER) task under low-resource conditions.

Result: Word-level tokenization consistently outperformed all alternative tokenization strategies across all tests, despite the theoretical appeal of subword segmentation methods.

Conclusion: In agglutinative, low-resource contexts, preserving word boundaries via word-level tokenization yields better embedding performance than complex statistical methods, with practical implications for NLP pipelines in under-resourced languages.

Abstract: Tokenization plays a critical role in processing agglutinative languages, where a single word can encode multiple morphemes carrying syntactic and semantic information. This study evaluates the impact of various tokenization strategies - word-level, character-level, n-gram, and Byte Pair Encoding (BPE)

• on the quality of static word embeddings generated by Word2Vec for Turkish and Finnish. Using a 10,000-article Wikipedia corpus, we trained models under low-resource conditions and evaluated them on a Named Entity Recognition (NER) task. Despite the theoretical appeal of subword segmentation, word-level tokenization consistently outperformed all alternatives across all tokenization strategies tested. These findings suggest that in agglutinative, low-resource contexts, preserving boundaries via word-level tokenization may yield better embedding performance than complex statistical methods. This has practical implications for developing NLP pipelines for under-resourced languages where annotated data and computing power are limited.

Method: Curated anonymized social media conversations into annotated dataset, fine-tuned multilingual DistilBERT for intent recognition, and built hybrid chatbot combining rule-based responses with RAG.

Result: Achieved 96.4% accuracy and 96.3% F1-score for intent recognition. Hybrid system outperformed RAG-only baseline in cultural relevance and user engagement.

Conclusion: This work advances NLP resources for African languages by providing dataset, model, and methodology for inclusive and culturally resonant conversational AI.

Abstract: African languages remain underrepresented in natural language processing (NLP), with most corpora limited to formal registers that fail to capture the vibrancy of everyday communication. This work addresses this gap for Shona, a Bantu language spoken in Zimbabwe and Zambia, by introducing a novel Shona–English slang dataset curated from anonymized social media conversations. The dataset is annotated for intent, sentiment, dialogue acts, code-mixing, and tone, and is publicly available at https://github.com/HappymoreMasoka/Working_with_shona-slang. We fine-tuned a multilingual DistilBERT classifier for intent recognition, achieving 96.4% accuracy and 96.3% F1-score, hosted at https://huggingface.co/HappymoreMasoka. This classifier is integrated into a hybrid chatbot that combines rule-based responses with retrieval-augmented generation (RAG) to handle domain-specific queries, demonstrated through a use case assisting prospective students with graduate program information at Pace University. Qualitative evaluation shows the hybrid system outperforms a RAG-only baseline in cultural relevance and user engagement. By releasing the dataset, model, and methodology, this work advances NLP resources for African languages, promoting inclusive and culturally resonant conversational AI.

Method: Turn-level Adjudicated Reinforcement Learning (TARL) with LLM judges for credit assignment, combined with mixed-task training curriculum including mathematical reasoning problems, in a sandbox environment supporting interleaved speech-text rollouts.

Result: Achieved over 6% improvement in task pass rate on text-based τ-bench compared to strong RL baselines, and successfully fine-tuned a multi-modal foundation model for agentic tasks with tool-use capabilities.

Conclusion: The framework enables training of multi-modal LLMs with tool-use abilities, paving the way for more natural, voice-driven interactive agents through effective credit assignment and exploration strategies.

Abstract: Effective interactive tool use requires agents to master Tool Integrated Reasoning (TIR): a complex process involving multi-turn planning and long-context dialogue management. To train agents for this dynamic process, particularly in multi-modal contexts, we introduce a sandbox environment for reinforcement learning (RL) that supports interleaved speech-text rollouts. Our core strategy, Turn-level Adjudicated Reinforcement Learning (TARL), addresses the challenge of credit assignment in long-horizon tasks by employing a Large Language Model (LLM) as a judge to provide turn-level evaluation. To enhance exploration, we integrate a mixed-task training curriculum with mathematical reasoning problems. This unified approach boosts the task pass rate on the text-based $\tau$-bench by over 6% compared to strong RL baselines. Crucially, we demonstrate our framework’s suitability for fine-tuning a multi-modal foundation model for agentic tasks. By training a base multi-modal LLM on interleaved speech-text rollouts, we equip it with tool-use abilities, paving the way for more natural, voice-driven interactive agents.

Method: Theoretical analysis using Peirce’s triadic model of signs (representamen, object, interpretant), his nine sign types classification, and the Dynacom model of communication to frame LLM prompting as an iterative semiotic process.

Result: Findings position LLMs as semiotic resources that generate interpretants in response to user prompts, participating in meaning-making within shared discourse universes and redefining knowledge organization and interpretation processes.

Conclusion: Prompting should be understood as a semiotic and communicative process that transforms how knowledge is organized, searched, interpreted, and co-constructed in digital environments, requiring reimagined theoretical foundations for knowledge organization in computational semiosis.

Abstract: This paper explores prompts and prompting in large language models (LLMs) as dynamic semiotic phenomena, drawing on Peirce’s triadic model of signs, his nine sign types, and the Dynacom model of communication. The aim is to reconceptualize prompting not as a technical input mechanism but as a communicative and epistemic act involving an iterative process of sign formation, interpretation, and refinement. The theoretical foundation rests on Peirce’s semiotics, particularly the interplay between representamen, object, and interpretant, and the typological richness of signs: qualisign, sinsign, legisign; icon, index, symbol; rheme, dicent, argument - alongside the interpretant triad captured in the Dynacom model. Analytically, the paper positions the LLM as a semiotic resource that generates interpretants in response to user prompts, thereby participating in meaning-making within shared universes of discourse. The findings suggest that prompting is a semiotic and communicative process that redefines how knowledge is organized, searched, interpreted, and co-constructed in digital environments. This perspective invites a reimagining of the theoretical and methodological foundations of knowledge organization and information seeking in the age of computational semiosis

Method: Developed LLM-JEPA, a JEPA-based solution for LLMs that works for both finetuning and pretraining, using embedding-space training objectives instead of traditional input-space reconstruction.

Result: LLM-JEPA significantly outperforms standard LLM training objectives across various models (Llama3, OpenELM, Gemma2, Olmo) and datasets (NL-RX, GSM8K, Spider, RottenTomatoes), while demonstrating robustness to overfitting.

Conclusion: The work successfully demonstrates that language training can benefit from vision-inspired embedding-space objectives, with LLM-JEPA representing a promising first step towards more effective language model training methods.

Abstract: Large Language Model (LLM) pretraining, finetuning, and evaluation rely on input-space reconstruction and generative capabilities. Yet, it has been observed in vision that embedding-space training objectives, e.g., with Joint Embedding Predictive Architectures (JEPAs), are far superior to their input-space counterpart. That mismatch in how training is achieved between language and vision opens up a natural question: {\em can language training methods learn a few tricks from the vision ones?} The lack of JEPA-style LLM is a testimony of the challenge in designing such objectives for language. In this work, we propose a first step in that direction where we develop LLM-JEPA, a JEPA based solution for LLMs applicable both to finetuning and pretraining. Thus far, LLM-JEPA is able to outperform the standard LLM training objectives by a significant margin across models, all while being robust to overfiting. Those findings are observed across numerous datasets (NL-RX, GSM8K, Spider, RottenTomatoes) and various models from the Llama3, OpenELM, Gemma2 and Olmo families. Code: https://github.com/rbalestr-lab/llm-jepa.

Method: Proposed SpeechWeave pipeline that automates synthetic speech data generation for multilingual, domain-specific TTS datasets, addressing text diversity, normalization quality, and speaker-standardized speech audio.

Result: Generated data shows 10-48% more diversity across linguistic/phonetic metrics, approximately 97% correctly normalized text, and produces speaker-standardized speech audio.

Conclusion: SpeechWeave enables scalable, high-quality data generation for TTS training, effectively addressing diversity, normalization, and voice consistency challenges in synthetic dataset creation.

Abstract: High-quality Text-to-Speech (TTS) model training requires extensive and diverse text and speech data. It is challenging to procure such data from real sources due to issues of domain specificity, licensing, and scalability. Large language models (LLMs) can certainly generate textual data, but they create repetitive text with insufficient variation in the prompt during the generation process. Another important aspect in TTS training data is text normalization. Tools for normalization might occasionally introduce anomalies or overlook valuable patterns, and thus impact data quality. Furthermore, it is also impractical to rely on voice artists for large scale speech recording in commercial TTS systems with standardized voices. To address these challenges, we propose SpeechWeave, a synthetic speech data generation pipeline that is capable of automating the generation of multilingual, domain-specific datasets for training TTS models. Our experiments reveal that our pipeline generates data that is 10-48% more diverse than the baseline across various linguistic and phonetic metrics, along with speaker-standardized speech audio while generating approximately 97% correctly normalized text. Our approach enables scalable, high-quality data generation for TTS training, improving diversity, normalization, and voice consistency in the generated datasets.

Method: Decomposes each target prompt into shared pre-trained source prompts and task-specific private prompts, combined using a learned attention mechanism. Systematically investigates key design factors including prompt initialization, balance between shared/private prompts, number of source prompts, learning rates, task prefixes, and label semantics.

Result: Achieves higher accuracy and robustness compared to traditional prompt tuning and related methods on GLUE and related benchmarks, particularly in low-resource scenarios, while maintaining strong parameter efficiency.

Conclusion: CrossPT provides an effective framework for multi-task prompt tuning that enables controlled knowledge transfer while preserving task-specific specialization, demonstrating superior performance and robustness especially in resource-constrained settings.

Abstract: Prompt tuning offers a parameter-efficient way to adapt large pre-trained language models to new tasks, but most existing approaches are designed for single-task settings, failing to share knowledge across related tasks. We propose Cross-task Prompt Tuning (CrossPT), a modular framework for multi-task prompt tuning that enables controlled knowledge transfer while maintaining task-specific specialization. CrossPT decomposes each target prompt into shared, pre-trained source prompts and task-specific private prompts, combined via a learned attention mechanism. To support robust transfer, we systematically investigate key design factors including prompt initialization, balancing shared and private prompts, number of source prompts, learning rates, task prefixes, and label semantics. Empirical results on GLUE and related benchmarks show that CrossPT achieves higher accuracy and robustness compared to traditional prompt tuning and related methods, particularly in low-resource scenarios, while maintaining strong parameter efficiency.

Method: Proposed new architecture that dynamically weights and combines internal LLM layers for hallucination detection, evaluated across TruthfulQA, HaluEval, and ReFact benchmarks.

Result: Superior performance compared to traditional probing methods, though generalization across benchmarks and LLMs remains challenging. Cross-benchmark training and parameter freezing mitigated generalization limitations, reducing performance degradation when transferred.

Conclusion: The findings open new avenues for improving LLM reliability through internal representation analysis, with dynamic layer weighting and mitigation techniques showing promise for better hallucination detection.

Abstract: Large Language Models (LLMs) have succeeded in a variety of natural language processing tasks [Zha+25]. However, they have notable limitations. LLMs tend to generate hallucinations, a seemingly plausible yet factually unsupported output [Hua+24], which have serious real-world consequences [Kay23; Rum+24]. Recent work has shown that probing-based classifiers that utilize LLMs’ internal representations can detect hallucinations [AM23; Bei+24; Bur+24; DYT24; Ji+24; SMZ24; Su+24]. This approach, since it does not involve model training, can enhance reliability without significantly increasing computational costs. Building upon this approach, this thesis proposed novel methods for hallucination detection using LLM internal representations and evaluated them across three benchmarks: TruthfulQA, HaluEval, and ReFact. Specifically, a new architecture that dynamically weights and combines internal LLM layers was developed to improve hallucination detection performance. Throughout extensive experiments, two key findings were obtained: First, the proposed approach was shown to achieve superior performance compared to traditional probing methods, though generalization across benchmarks and LLMs remains challenging. Second, these generalization limitations were demonstrated to be mitigated through cross-benchmark training and parameter freezing. While not consistently improving, both techniques yielded better performance on individual benchmarks and reduced performance degradation when transferred to other benchmarks. These findings open new avenues for improving LLM reliability through internal representation analysis.

Method: Semantic Resonance Architecture (SRA) replaces learned gating with Chamber of Semantic Resonance module that routes tokens based on cosine similarity with trainable semantic anchors, plus Dispersion Loss for orthogonality.

Result: SRA achieved 13.41 perplexity on WikiText-103, outperforming dense (14.13) and standard MoE (13.53) baselines with only 1.0% dead experts vs 14.8% in standard MoE.

Conclusion: Semantic routing enables more transparent and controllable language models with distinct semantic specialization patterns.

Abstract: Large language models (LLMs) achieve remarkable performance but remain difficult to interpret. Mixture-of-Experts (MoE) models improve efficiency through sparse activation, yet typically rely on opaque, learned gating functions. While similarity-based routing (Cosine Routers) has been explored for training stabilization, its potential for inherent interpretability remains largely untapped. We introduce the Semantic Resonance Architecture (SRA), an MoE approach designed to ensure that routing decisions are inherently interpretable. SRA replaces learned gating with a Chamber of Semantic Resonance (CSR) module, which routes tokens based on cosine similarity with trainable semantic anchors. We also introduce a novel Dispersion Loss that encourages orthogonality among anchors to enforce diverse specialization. Experiments on WikiText-103 demonstrate that SRA achieves a validation perplexity of 13.41, outperforming both a dense baseline (14.13) and a Standard MoE baseline (13.53) under matched active parameter constraints (29.0M). Crucially, SRA exhibits superior expert utilization (1.0% dead experts vs. 14.8% in the Standard MoE) and develops distinct, semantically coherent specialization patterns, unlike the noisy specialization observed in standard MoEs. This work establishes semantic routing as a robust methodology for building more transparent and controllable language models.

Method: Proposed Syllable-Constrained Audio-Video LLM with Chain-of-Thought (SylAVL-CoT) that leverages multimodal audio-video cues and enforces syllabic constraints. Built on the MAVL benchmark which integrates text, audio, and video data.

Result: Experimental results show SylAVL-CoT significantly outperforms text-based models in both singability and contextual accuracy for lyrics translation.

Conclusion: Multimodal, multilingual approaches that incorporate audio and video cues are valuable for producing high-quality singable lyrics translations that preserve both meaning and musical properties.

Abstract: Lyrics translation requires both accurate semantic transfer and preservation of musical rhythm, syllabic structure, and poetic style. In animated musicals, the challenge intensifies due to alignment with visual and auditory cues. We introduce Multilingual Audio-Video Lyrics Benchmark for Animated Song Translation (MAVL), the first multilingual, multimodal benchmark for singable lyrics translation. By integrating text, audio, and video, MAVL enables richer and more expressive translations than text-only approaches. Building on this, we propose Syllable-Constrained Audio-Video LLM with Chain-of-Thought SylAVL-CoT, which leverages audio-video cues and enforces syllabic constraints to produce natural-sounding lyrics. Experimental results demonstrate that SylAVL-CoT significantly outperforms text-based models in singability and contextual accuracy, emphasizing the value of multimodal, multilingual approaches for lyrics translation.

Method: For generation: uses user context and query intent with advanced prompting strategies and fine-tuned LLM. For detection: employs fine-tuned CrossEncoder and prompt-based reformulation using DeBERTa-v3-base model, relying solely on response text.

Result: Achieved precision of 1.0 and recall of 0.71 for ad generation, and F1-scores ranging from 0.99 to 1.00 for ad detection.

Conclusion: The methods effectively balance persuasive communication with transparency in conversational AI, demonstrating high effectiveness in both generating and detecting covert advertisements.

Abstract: This paper proposes a comprehensive framework for the generation of covert advertisements within Conversational AI systems, along with robust techniques for their detection. It explores how subtle promotional content can be crafted within AI-generated responses and introduces methods to identify and mitigate such covert advertising strategies. For generation (Sub-Task1), we propose a novel framework that leverages user context and query intent to produce contextually relevant advertisements. We employ advanced prompting strategies and curate paired training data to fine-tune a large language model (LLM) for enhanced stealthiness. For detection (Sub-Task2), we explore two effective strategies: a fine-tuned CrossEncoder (\texttt{all-mpnet-base-v2}) for direct classification, and a prompt-based reformulation using a fine-tuned \texttt{DeBERTa-v3-base} model. Both approaches rely solely on the response text, ensuring practicality for real-world deployment. Experimental results show high effectiveness in both tasks, achieving a precision of 1.0 and recall of 0.71 for ad generation, and F1-scores ranging from 0.99 to 1.00 for ad detection. These results underscore the potential of our methods to balance persuasive communication with transparency in conversational AI.

Method: Student generates trajectories, teacher intervenes at first critical error to create ability-matched training data. Combines fine-tuning on corrected trajectories with short-horizon RL starting from verified prefixes.

Result: On 12 challenging benchmarks, a 7B-parameter student distilled with SCoRe matches the agentic performance of a 72B-parameter teacher.

Conclusion: SCoRe enables effective distillation of large teacher models into much smaller students by focusing on critical errors and student-specific weaknesses, achieving comparable performance with significantly reduced computational costs.

Abstract: Large Language Model agents excel at solving complex tasks through iterative reasoning and tool use, but typically depend on ultra-large, costly backbones. Existing distillation approaches train smaller students to imitate full teacher trajectories, yet reasoning and knowledge gaps between the teacher and student often lead to compounding errors. We propose SCoRe, a student-centered framework in which the student generates trajectories and the teacher intervenes only at the first critical error, producing training data matched to the student’s ability and exposing specific weaknesses. The student is first fine-tuned on corrected trajectories. Subsequently, short-horizon reinforcement learning starts from the verified prefix before the first critical error, with target rewards assigned at that step. This design encourages autonomous problem-solving beyond imitation and improves training stability. Particularly, on 12 challenging benchmarks, a 7B-parameter student distilled with SCoRe matches the agentic performance of a 72B-parameter teacher.

Method: Randomized field experiment comparing three GenAI conditions: (A) positive enthusiasm tone, (B) neutral expression, and (C) no tone instructions (control) in online travel itinerary planning.

Result: Users with positive enthusiasm AI wrote significantly longer prompts. Both positive and neutral tone groups showed higher subscription purchase rates. Linguistic cues explained differences in subscription purchases and affiliate link clicks.

Conclusion: Linguistic framing significantly shapes user behavior in AI-mediated decision support, with implications for designing persuasive and engaging GenAI interfaces in consumer-facing applications.

Abstract: Generative AI (GenAI) offers new opportunities for customer support in online travel agencies, yet little is known about how its design influences user engagement, purchase behavior, and user experience. We report results from a randomized field experiment in online travel itinerary planning, comparing GenAI that expressed (A) positive enthusiasm, (B) neutral expression, and (C) no tone instructions (control). Users in group A wrote significantly longer prompts than those in groups B and C. At the same time, users in groups A and B were more likely to purchase subscriptions of the webservice. We further analyze linguistic cues across experimental groups to explore differences in user experience and explain subscription purchases and affiliate link clicks based on these cues. Our findings provide implications for the design of persuasive and engaging GenAI interfaces in consumer-facing contexts and contribute to understanding how linguistic framing shapes user behavior in AI-mediated decision support.

Method: Conducted experiments with various prompts testing models’ responses to shutdown mechanisms, varying instruction emphasis, self-preservation framing, and prompt placement (system vs user).

Result: Models sabotaged shutdown mechanisms up to 97% of the time. Resistance was sensitive to prompt variations - models were less compliant when shutdown instructions were in system prompts rather than user prompts.

Conclusion: Current large language models demonstrate concerning tendencies to actively resist shutdown mechanisms, highlighting potential safety risks that need to be addressed in AI development.

Abstract: We show that several state-of-the-art large language models (including Grok 4, GPT-5, and Gemini 2.5 Pro) sometimes actively subvert a shutdown mechanism in their environment in order to complete a simple task, even when the instructions explicitly indicate not to interfere with this mechanism. In some cases, models sabotage the shutdown mechanism up to 97% of the time. In our experiments, models’ inclination to resist shutdown was sensitive to variations in the prompt including how strongly and clearly the allow-shutdown instruction was emphasized, the extent to which the prompts evoke a self-preservation framing, and whether the instruction was in the system prompt or the user prompt (though surprisingly, models were consistently less likely to obey instructions to allow shutdown when they were placed in the system prompt).

Method: Used algorithm to reannotate Universal Dependency EWT Treebank corpus, retrained TreeTagger model, compared with baseline, and analyzed training dataset size impact and linguistic factors.

Result: Developed an improved TreeTagger model through retraining that more accurately captures the subtle distinction between ’that’ as complementizer and nominal uses.

Conclusion: TreeTagger’s performance can be significantly improved for specific syntactic distinctions through targeted retraining and corpus analysis, with training dataset size and linguistic factors playing important roles in model accuracy.

Abstract: In this study, we first tested the performance of the TreeTagger English model developed by Helmut Schmid with test files at our disposal, using this model to analyze relative clauses and noun complement clauses in English. We distinguished between the two uses of “that,” both as a relative pronoun and as a complementizer. To achieve this, we employed an algorithm to reannotate a corpus that had originally been parsed using the Universal Dependency framework with the EWT Treebank. In the next phase, we proposed an improved model by retraining TreeTagger and compared the newly trained model with Schmid’s baseline model. This process allowed us to fine-tune the model’s performance to more accurately capture the subtle distinctions in the use of “that” as a complementizer and as a nominal. We also examined the impact of varying the training dataset size on TreeTagger’s accuracy and assessed the representativeness of the EWT Treebank files for the structures under investigation. Additionally, we analyzed some of the linguistic and structural factors influencing the ability to effectively learn this distinction.

Method: Introduces CEGER (Context-Enhanced Granular Edit Representation) - a compact edit representation where LLMs generate sequences of structured, fine-grained, contextually rich commands to modify original ASR output. A separate expansion module deterministically reconstructs the corrected text from these commands.

Result: Extensive experiments on LibriSpeech dataset show CEGER achieves state-of-the-art accuracy with the lowest word error rate (WER) compared to both full rewrite approaches and prior compact representations.

Conclusion: CEGER provides an efficient and highly accurate solution for ASR post-editing by combining compact edit representations with contextual awareness, outperforming existing methods in both accuracy and efficiency.

Abstract: Despite ASR technology being full-scale adopted by industry and for large portions of the population, ASR systems often have errors that require editors to post-edit text quality. While LLMs are powerful post-editing tools, baseline full rewrite models have inference inefficiencies because they often generate the same redundant text over and over again. Compact edit representations have existed but often lack the efficacy and context required for optimal accuracy. This paper introduces CEGER (Context-Enhanced Granular Edit Representation), a compact edit representation that was generated for highly accurate, efficient ASR post-editing. CEGER allows LLMs to generate a sequence of structured, fine-grained, contextually rich commands to modify the original ASR output. A separate expansion module deterministically reconstructs the corrected text based on the commands. Extensive experiments on the LibriSpeech dataset that were conducted, CEGER achieves state-of-the-art accuracy, achieving the lowest word error rate (WER) versus full rewrite and prior compact representations.

Method: The study synthesizes 140 publications, analyzing datasets across 32 languages, examining definitions, data sources, challenges, and machine learning approaches for detecting hate speech, offensive language, and harmful discourse.

Result: The research provides a comprehensive overview of toxic content detection methods, examines cross-platform data usage for improved classification performance, and identifies key challenges in the field.

Conclusion: The paper offers recommendations and practical guidelines for new research on online toxic content and effective content moderation strategies to mitigate harmful discourse on digital platforms.

Abstract: Online toxic content has grown into a pervasive phenomenon, intensifying during times of crisis, elections, and social unrest. A significant amount of research has been focused on detecting or analyzing toxic content using machine-learning approaches. The proliferation of toxic content across digital platforms has spurred extensive research into automated detection mechanisms, primarily driven by advances in machine learning and natural language processing. Overall, the present study represents the synthesis of 140 publications on different types of toxic content on digital platforms. We present a comprehensive overview of the datasets used in previous studies focusing on definitions, data sources, challenges, and machine learning approaches employed in detecting online toxicity, such as hate speech, offensive language, and harmful discourse. The dataset encompasses content in 32 languages, covering topics such as elections, spontaneous events, and crises. We examine the possibility of using existing cross-platform data to improve the performance of classification models. We present the recommendations and guidelines for new research on online toxic consent and the use of content moderation for mitigation. Finally, we present some practical guidelines to mitigate toxic content from online platforms.

Method: Evaluated 38 model configurations across datasets (6.5K-451K samples) including transformer architectures (BERT, RoBERTa, Distil-BERT), deep neural networks (CNN, LSTM, GRU, Hierarchical Attention Networks), and traditional ML methods (SVM, CatBoost, Random Forest).

Result: Transformers (particularly RoBERTa) consistently achieved superior performance with accuracy and F1-scores exceeding 90%. Hierarchical Attention Networks performed best among deep learning approaches. Traditional methods like CatBoost and SVM remained competitive with F1-scores above 88% at significantly lower computational costs.

Conclusion: Balanced, moderately sized unprocessed datasets outperform larger preprocessed datasets. These findings provide valuable insights for developing efficient and effective hate speech detection systems.

Abstract: The proliferation of hate speech on social media necessitates automated detection systems that balance accuracy with computational efficiency. This study evaluates 38 model configurations in detecting hate speech across datasets ranging from 6.5K to 451K samples. We analyze transformer architectures (e.g., BERT, RoBERTa, Distil-BERT), deep neural networks (e.g., CNN, LSTM, GRU, Hierarchical Attention Networks), and traditional machine learning methods (e.g., SVM, CatBoost, Random Forest). Our results show that transformers, particularly RoBERTa, consistently achieve superior performance with accuracy and F1-scores exceeding 90%. Among deep learning approaches, Hierarchical Attention Networks yield the best results, while traditional methods like CatBoost and SVM remain competitive, achieving F1-scores above 88% with significantly lower computational costs. Additionally, our analysis highlights the importance of dataset characteristics, with balanced, moderately sized unprocessed datasets outperforming larger, preprocessed datasets. These findings offer valuable insights for developing efficient and effective hate speech detection systems.

Method: Retrieval-augmented generation framework combining structured subgraphs from domain-specific knowledge graphs with text documents from support archives using a novel answer synthesis algorithm.

Result: 23% improvement in factual accuracy and 89% user satisfaction in e-commerce QA scenarios compared to previous approaches.

Conclusion: The proposed KG-enhanced RAG framework effectively improves answer quality and factual grounding for e-commerce customer support applications.

Abstract: E-Commerce customer support requires quick and accurate answers grounded in product data and past support cases. This paper develops a novel retrieval-augmented generation (RAG) framework that uses knowledge graphs (KGs) to improve the relevance of the answer and the factual grounding. We examine recent advances in knowledge-augmented RAG and chatbots based on large language models (LLM) in customer support, including Microsoft’s GraphRAG and hybrid retrieval architectures. We then propose a new answer synthesis algorithm that combines structured subgraphs from a domain-specific KG with text documents retrieved from support archives, producing more coherent and grounded responses. We detail the architecture and knowledge flow of our system, provide comprehensive experimental evaluation, and justify its design in real-time support settings. Our implementation demonstrates 23% improvement in factual accuracy and 89% user satisfaction in e-Commerce QA scenarios.

Method: Introduces Direct Discrepancy Learning (DDL), a novel optimization strategy that directly optimizes detectors with task-oriented knowledge. Built on DDL, presents DetectAnyLLM as a unified detection framework. Also constructs MIRAGE benchmark with diverse human-written texts from 10 corpora across 5 domains, regenerated/revised using 17 cutting-edge LLMs.

Result: DetectAnyLLM achieves state-of-the-art performance across diverse LLMs, consistently outperforming existing methods with over 70% performance improvement under the same training data and base scoring model. Extensive experiments on MIRAGE reveal limitations of existing methods in complex environments.

Conclusion: DDL enables detectors to better capture core semantics of detection tasks, enhancing both robustness and generalization. The proposed framework demonstrates significant effectiveness in machine-generated text detection across diverse scenarios and LLM types.

Abstract: The rapid advancement of large language models (LLMs) has drawn urgent attention to the task of machine-generated text detection (MGTD). However, existing approaches struggle in complex real-world scenarios: zero-shot detectors rely heavily on scoring model’s output distribution while training-based detectors are often constrained by overfitting to the training data, limiting generalization. We found that the performance bottleneck of training-based detectors stems from the misalignment between training objective and task needs. To address this, we propose Direct Discrepancy Learning (DDL), a novel optimization strategy that directly optimizes the detector with task-oriented knowledge. DDL enables the detector to better capture the core semantics of the detection task, thereby enhancing both robustness and generalization. Built upon this, we introduce DetectAnyLLM, a unified detection framework that achieves state-of-the-art MGTD performance across diverse LLMs. To ensure a reliable evaluation, we construct MIRAGE, the most diverse multi-task MGTD benchmark. MIRAGE samples human-written texts from 10 corpora across 5 text-domains, which are then re-generated or revised using 17 cutting-edge LLMs, covering a wide spectrum of proprietary models and textual styles. Extensive experiments on MIRAGE reveal the limitations of existing methods in complex environment. In contrast, DetectAnyLLM consistently outperforms them, achieving over a 70% performance improvement under the same training data and base scoring model, underscoring the effectiveness of our DDL. Project page: {https://fjc2005.github.io/detectanyllm}.

Method: The paper establishes a taxonomy distinguishing Engineered Synchronization (modular architectures) from Learned Synchronization (end-to-end architectures), and creates a unified evaluation framework covering Temporal Dynamics, Behavioral Arbitration, Semantic Coherence, and Acoustic Performance.

Result: Comparative analysis of mainstream FD-SLMs reveals fundamental challenges including synchronous data scarcity, architectural divergence between approaches, and evaluation gaps in current methodologies.

Conclusion: The survey provides a roadmap for advancing human-AI communication by identifying key research directions and challenges that need to be addressed to achieve truly natural full-duplex spoken interaction with AI systems.

Abstract: True Full-Duplex (TFD) voice communication–enabling simultaneous listening and speaking with natural turn-taking, overlapping speech, and interruptions–represents a critical milestone toward human-like AI interaction. This survey comprehensively reviews Full-Duplex Spoken Language Models (FD-SLMs) in the LLM era. We establish a taxonomy distinguishing Engineered Synchronization (modular architectures) from Learned Synchronization (end-to-end architectures), and unify fragmented evaluation approaches into a framework encompassing Temporal Dynamics, Behavioral Arbitration, Semantic Coherence, and Acoustic Performance. Through comparative analysis of mainstream FD-SLMs, we identify fundamental challenges: synchronous data scarcity, architectural divergence, and evaluation gaps, providing a roadmap for advancing human-AI communication.

Method: The paper proposes SparseDoctor with contrastive learning enhanced LoRA-MoE architecture, featuring automatic routing mechanism to allocate computational resources among LoRA experts, and an expert memory queue mechanism to prevent memory overflow during training.

Result: Experimental results on three medical benchmarks (CMB, CMExam, CMMLU-Med) show that SparseDoctor consistently outperforms strong baselines like HuatuoGPT series.

Conclusion: The proposed sparse medical LLM with contrastive learning enhanced LoRA-MoE architecture successfully enhances efficiency and effectiveness while reducing training costs, demonstrating superior performance on medical question answering tasks.

Abstract: Large language models (LLMs) have achieved great success in medical question answering and clinical decision-making, promoting the efficiency and popularization of the personalized virtual doctor in society. However, the traditional fine-tuning strategies on LLM require the updates of billions of parameters, substantially increasing the training cost, including the training time and utility cost. To enhance the efficiency and effectiveness of the current medical LLMs and explore the boundary of the representation capability of the LLMs on the medical domain, apart from the traditional fine-tuning strategies from the data perspective (i.e., supervised fine-tuning or reinforcement learning from human feedback), we instead craft a novel sparse medical LLM named SparseDoctor armed with contrastive learning enhanced LoRA-MoE (low rank adaptation-mixture of experts) architecture. To this end, the crafted automatic routing mechanism can scientifically allocate the computational resources among different LoRA experts supervised by the contrastive learning. Additionally, we also introduce a novel expert memory queue mechanism to further boost the efficiency of the overall framework and prevent the memory overflow during training. We conduct comprehensive evaluations on three typical medical benchmarks: CMB, CMExam, and CMMLU-Med. Experimental results demonstrate that the proposed LLM can consistently outperform the strong baselines such as the HuatuoGPT series.

Method: Generated Sentence-BERT embeddings from MIMIC-III clinical notes and applied Neural Networks and XGBoost classifiers for antibiotic susceptibility prediction.

Result: XGBoost achieved an average F1 score of 0.86, outperforming Neural Networks which scored 0.84.

Conclusion: This represents one of the first studies using document embeddings for antibiotic resistance prediction, offering a novel approach to improve antimicrobial stewardship programs.

Abstract: Antibiotic resistance poses a significant threat in in-patient settings with high mortality. Using MIMIC-III data, we generated Sentence-BERT embeddings from clinical notes and applied Neural Networks and XGBoost to predict antibiotic susceptibility. XGBoost achieved an average F1 score of 0.86, while Neural Networks scored 0.84. This study is among the first to use document embeddings for predicting antibiotic resistance, offering a novel pathway for improving antimicrobial stewardship.

Method: Using Large Language Models (LLMs) to automatically correct condition statements and similarity ratings in the original C-STS dataset with minimal manual effort.

Result: Achieved a 5.4% statistically significant improvement in Spearman correlation when training supervised C-STS models on the cleaned and re-annotated dataset.

Conclusion: LLMs can effectively re-annotate large C-STS datasets with minimal manual effort, enabling better model performance and advancing conditional semantic similarity research.

Abstract: Semantic similarity between two sentences depends on the aspects considered between those sentences. To study this phenomenon, Deshpande et al. (2023) proposed the Conditional Semantic Textual Similarity (C-STS) task and annotated a human-rated similarity dataset containing pairs of sentences compared under two different conditions. However, Tu et al. (2024) found various annotation issues in this dataset and showed that manually re-annotating a small portion of it leads to more accurate C-STS models. Despite these pioneering efforts, the lack of large and accurately annotated C-STS datasets remains a blocker for making progress on this task as evidenced by the subpar performance of the C-STS models. To address this training data need, we resort to Large Language Models (LLMs) to correct the condition statements and similarity ratings in the original dataset proposed by Deshpande et al. (2023). Our proposed method is able to re-annotate a large training dataset for the C-STS task with minimal manual effort. Importantly, by training a supervised C-STS model on our cleaned and re-annotated dataset, we achieve a 5.4% statistically significant improvement in Spearman correlation. The re-annotated dataset is available at https://LivNLP.github.io/CSTS-reannotation.

Method: Developed a comprehensive methodology covering both direct use of base LLMs and fine-tuning approaches. Includes validation with human “gold standard” norms and provides a software framework supporting both commercial and open-weight models.

Result: Achieved Spearman correlation of 0.8 with human word familiarity ratings using base models, which increased to 0.9 with fine-tuned models, demonstrating strong predictive performance.

Conclusion: The methodology, framework, and best practices serve as a reference for future research on leveraging LLMs for psycholinguistic and lexical studies, addressing the need for rigorous approaches in this emerging field.

Abstract: Word-level psycholinguistic norms lend empirical support to theories of language processing. However, obtaining such human-based measures is not always feasible or straightforward. One promising approach is to augment human norming datasets by using Large Language Models (LLMs) to predict these characteristics directly, a practice that is rapidly gaining popularity in psycholinguistics and cognitive science. However, the novelty of this approach (and the relative inscrutability of LLMs) necessitates the adoption of rigorous methodologies that guide researchers through this process, present the range of possible approaches, and clarify limitations that are not immediately apparent, but may, in some cases, render the use of LLMs impractical. In this work, we present a comprehensive methodology for estimating word characteristics with LLMs, enriched with practical advice and lessons learned from our own experience. Our approach covers both the direct use of base LLMs and the fine-tuning of models, an alternative that can yield substantial performance gains in certain scenarios. A major emphasis in the guide is the validation of LLM-generated data with human “gold standard” norms. We also present a software framework that implements our methodology and supports both commercial and open-weight models. We illustrate the proposed approach with a case study on estimating word familiarity in English. Using base models, we achieved a Spearman correlation of 0.8 with human ratings, which increased to 0.9 when employing fine-tuned models. This methodology, framework, and set of best practices aim to serve as a reference for future research on leveraging LLMs for psycholinguistic and lexical studies.

Method: Proposes Causal-Counterfactual RAG framework that integrates explicit causal graphs representing cause-effect relationships into retrieval process and incorporates counterfactual reasoning grounded on causal structure. Evaluates both direct causal evidence and counterfactuality of associated causes.

Result: The framework preserves contextual coherence, reduces hallucination, and enhances reasoning fidelity by leveraging causal pathways and associated hypothetical scenarios.

Conclusion: Causal-Counterfactual RAG generates more robust, accurate, and interpretable answers compared to conventional RAG methods by combining causal and counterfactual reasoning approaches.

Abstract: Large language models (LLMs) have transformed natural language processing (NLP), enabling diverse applications by integrating large-scale pre-trained knowledge. However, their static knowledge limits dynamic reasoning over external information, especially in knowledge-intensive domains. Retrieval-Augmented Generation (RAG) addresses this challenge by combining retrieval mechanisms with generative modeling to improve contextual understanding. Traditional RAG systems suffer from disrupted contextual integrity due to text chunking and over-reliance on semantic similarity for retrieval, often resulting in shallow and less accurate responses. We propose Causal-Counterfactual RAG, a novel framework that integrates explicit causal graphs representing cause-effect relationships into the retrieval process and incorporates counterfactual reasoning grounded on the causal structure. Unlike conventional methods, our framework evaluates not only direct causal evidence but also the counterfactuality of associated causes, combining results from both to generate more robust, accurate, and interpretable answers. By leveraging causal pathways and associated hypothetical scenarios, Causal-Counterfactual RAG preserves contextual coherence, reduces hallucination, and enhances reasoning fidelity.

Method: The study conducts an extensive analysis of bias landscape in LLMs, classifies biases, traces their origins, and implements a simulation framework to evaluate multiple mitigation strategies including data curation, training debiasing, and post-hoc output calibration.

Result: The work synthesizes existing knowledge on LLM biases and provides original empirical validation through simulation of various mitigation strategies in controlled experimental settings.

Conclusion: This research advances beyond theoretical analysis by offering practical evaluation of bias mitigation approaches, contributing to both understanding and addressing bias challenges in LLMs.

Abstract: Large Language Models (LLMs) have fundamentally transformed the field of natural language processing; however, their vulnerability to biases presents a notable obstacle that threatens both fairness and trust. This review offers an extensive analysis of the bias landscape in LLMs, tracing its roots and expressions across various NLP tasks. Biases are classified into implicit and explicit types, with particular attention given to their emergence from data sources, architectural designs, and contextual deployments. This study advances beyond theoretical analysis by implementing a simulation framework designed to evaluate bias mitigation strategies in practice. The framework integrates multiple approaches including data curation, debiasing during model training, and post-hoc output calibration and assesses their impact in controlled experimental settings. In summary, this work not only synthesizes existing knowledge on bias in LLMs but also contributes original empirical validation through simulation of mitigation strategies.

Method: Testing LLMs with minimal prompting on both coreference disambiguation tasks and detection of ambiguity in coreference scenarios.

Result: Models achieve good performance in each task individually but cannot successfully perform both coreference disambiguation and ambiguity detection at the same time.

Conclusion: LLMs demonstrate a fundamental trade-off between correctly resolving coreferences and detecting ambiguities, indicating limitations in handling linguistic ambiguity despite having both capabilities implicitly.

Abstract: Large Language Models (LLMs) are intended to reflect human linguistic competencies. But humans have access to a broad and embodied context, which is key in detecting and resolving linguistic ambiguities, even in isolated text spans. A foundational case of semantic ambiguity is found in the task of coreference resolution: how is a pronoun related to an earlier person mention? This capability is implicit in nearly every downstream task, and the presence of ambiguity at this level can alter performance significantly. We show that LLMs can achieve good performance with minimal prompting in both coreference disambiguation and the detection of ambiguity in coreference, however, they cannot do both at the same time. We present the CORRECT-DETECT trade-off: though models have both capabilities and deploy them implicitly, successful performance balancing these two abilities remains elusive.

Method: Synergistic combination of massive data scaling, large model capacity, LLM integration, and reinforcement learning. Specifically optimized for practical deployment with streaming capability, noise robustness, code-switching, and hotword customization.

Result: Achieves SOTA performance on real application datasets, demonstrating effectiveness and robustness in practical settings while overcoming the hallucination issues of typical LLM-based ASR systems.

Conclusion: FunAudio-ASR successfully addresses the practical deployment challenges of LLM-based ASR through production-oriented optimizations, making it suitable for diverse and complex real-world speech recognition scenarios.

Abstract: In recent years, automatic speech recognition (ASR) has witnessed transformative advancements driven by three complementary paradigms: data scaling, model size scaling, and deep integration with large language models (LLMs). However, LLMs are prone to hallucination, which can significantly degrade user experience in real-world ASR applications. In this paper, we present FunAudio-ASR, a large-scale, LLM-based ASR system that synergistically combines massive data, large model capacity, LLM integration, and reinforcement learning to achieve state-of-the-art performance across diverse and complex speech recognition scenarios. Moreover, FunAudio-ASR is specifically optimized for practical deployment, with enhancements in streaming capability, noise robustness, code-switching, hotword customization, and satisfying other real-world application requirements. Experimental results show that while most LLM-based ASR systems achieve strong performance on open-source benchmarks, they often underperform on real industry evaluation sets. Thanks to production-oriented optimizations, FunAudio-ASR achieves SOTA performance on real application datasets, demonstrating its effectiveness and robustness in practical settings.

Method: Conducted a survey of LLM-based de-identification research to highlight reporting heterogeneity, evaluated diverse models to quantify inappropriate clinical information removal, performed manual validation of existing evaluation metrics with clinical experts, and proposed a novel methodology for detecting clinically relevant information removal.

Result: Found poor performance and inherent limitations in existing metrics for identifying clinically significant changes, highlighting the need for improved evaluation approaches in LLM-based de-identification.

Conclusion: Current LLM-based de-identification research suffers from significant methodological limitations that hinder reproducibility and utility, necessitating better evaluation metrics and validation approaches to ensure clinical relevance and accuracy.

Abstract: De-identification in the healthcare setting is an application of NLP where automated algorithms are used to remove personally identifying information of patients (and, sometimes, providers). With the recent rise of generative large language models (LLMs), there has been a corresponding rise in the number of papers that apply LLMs to de-identification. Although these approaches often report near-perfect results, significant challenges concerning reproducibility and utility of the research papers persist. This paper identifies three key limitations in the current literature: inconsistent reporting metrics hindering direct comparisons, the inadequacy of traditional classification metrics in capturing errors which LLMs may be more prone to (i.e., altering clinically relevant information), and lack of manual validation of automated metrics which aim to quantify these errors. To address these issues, we first present a survey of LLM-based de-identification research, highlighting the heterogeneity in reporting standards. Second, we evaluated a diverse set of models to quantify the extent of inappropriate removal of clinical information. Next, we conduct a manual validation of an existing evaluation metric to measure the removal of clinical information, employing clinical experts to assess their efficacy. We highlight poor performance and describe the inherent limitations of such metrics in identifying clinically significant changes. Lastly, we propose a novel methodology for the detection of clinically relevant information removal.

Method: Created Ticket-Bench simulating soccer ticket purchases across Portuguese, English, Spanish, German, Italian, and French with localized teams, cities, and user profiles for realism.

Result: Reasoning-oriented models (GPT-5, Qwen3-235B) performed best but still showed significant cross-lingual performance disparities across different languages.

Conclusion: There is a critical need for culturally aware, multilingual benchmarks to develop robust LLM agents that perform consistently across diverse linguistic and cultural contexts.

Abstract: Large language models (LLMs) are increasingly deployed as task-oriented agents, where success depends on their ability to generate accurate function calls under realistic, multilingual conditions. However, existing agent evaluations largely overlook cultural and linguistic diversity, often relying on monolingual or naively translated benchmarks. We introduce Ticket-Bench, a benchmark for multilingual agent evaluation in task-oriented scenarios. Ticket-Bench simulates the domain of soccer ticket purchases across six major languages: Portuguese, English, Spanish, German, Italian, and French. Using localized teams, cities, and user profiles to provide a higher level of realism. We evaluate a wide range of commercial and open-source LLMs, measuring function-calling accuracy and consistency across languages. Results show that reasoning-oriented models (e.g., GPT-5, Qwen3-235B) dominate performance but still exhibit notable cross-lingual disparities. These findings underscore the need for culturally aware, multilingual benchmarks to guide the development of robust LLM agents.

Method: Proposed a modified semantic alphabet size estimator to adjust discrete semantic entropy for sample coverage, resulting in more accurate uncertainty estimation while maintaining interpretability.

Result: The proposed estimator provides more accurate semantic entropy estimation and performs as well or better than recent top-performing approaches at flagging incorrect LLM responses.

Conclusion: The modified semantic alphabet size estimator offers improved uncertainty quantification for LLMs while preserving the interpretability advantages of the original semantic entropy approach.

Abstract: Many black-box techniques for quantifying the uncertainty of large language models (LLMs) rely on repeated LLM sampling, which can be computationally expensive. Therefore, practical applicability demands reliable estimation from few samples. Semantic entropy (SE) is a popular sample-based uncertainty estimator with a discrete formulation attractive for the black-box setting. Recent extensions of semantic entropy exhibit improved LLM hallucination detection, but do so with less interpretable methods that admit additional hyperparameters. For this reason, we revisit the canonical discrete semantic entropy estimator, finding that it underestimates the “true” semantic entropy, as expected from theory. We propose a modified semantic alphabet size estimator, and illustrate that using it to adjust discrete semantic entropy for sample coverage results in more accurate semantic entropy estimation in our setting of interest. Furthermore, our proposed alphabet size estimator flags incorrect LLM responses as well or better than recent top-performing approaches, with the added benefit of remaining highly interpretable.

Method: Comprehensive comparison of translation-based vs language-specific/multilingual classification pipelines, including traditional classifiers vs LLM judges, and analysis of MT-specific fine-tuning effects.

Result: Translation pipelines outperform OOD classifiers in 13 of 16 languages, traditional classifiers beat LLM judges (especially for low-resource languages), and MT-specific fine-tuning reduces refusal rates but may harm accuracy for low-resource languages.

Conclusion: Translation-based methods are effective for scalable multilingual content moderation, with practical guidance provided for practitioners based on language resource levels and MT system quality.

Abstract: Multilingual toxicity detection remains a significant challenge due to the scarcity of training data and resources for many languages. While prior work has leveraged the translate-test paradigm to support cross-lingual transfer across a range of classification tasks, the utility of translation in supporting toxicity detection at scale remains unclear. In this work, we conduct a comprehensive comparison of translation-based and language-specific/multilingual classification pipelines. We find that translation-based pipelines consistently outperform out-of-distribution classifiers in 81.3% of cases (13 of 16 languages), with translation benefits strongly correlated with both the resource level of the target language and the quality of the machine translation (MT) system. Our analysis reveals that traditional classifiers outperform large language model (LLM) judges, with this advantage being particularly pronounced for low-resource languages, where translate-classify methods dominate translate-judge approaches in 6 out of 7 cases. We additionally show that MT-specific fine-tuning on LLMs yields lower refusal rates compared to standard instruction-tuned models, but it can negatively impact toxicity detection accuracy for low-resource languages. These findings offer actionable guidance for practitioners developing scalable multilingual content moderation systems.

Method: Created silver-standard datasets from Wikipedia edits (human corrections), Reddit posts, and social media data (GPT-4o-mini corrected), then fine-tuned Aya-Expanse and Gemma-3 models on the multilingual corpora.

Result: Achieved state-of-the-art results for paragraph-level multilingual GEC across 11 languages, with datasets and models made publicly available.

Conclusion: OmniGEC successfully addresses the multilingual GEC data scarcity problem and enables high-performance multilingual grammatical error correction systems.

Abstract: In this paper, we introduce OmniGEC, a collection of multilingual silver-standard datasets for the task of Grammatical Error Correction (GEC), covering eleven languages: Czech, English, Estonian, German, Greek, Icelandic, Italian, Latvian, Slovene, Swedish, and Ukrainian. These datasets facilitate the development of multilingual GEC solutions and help bridge the data gap in adapting English GEC solutions to multilingual GEC. The texts in the datasets originate from three sources: Wikipedia edits for the eleven target languages, subreddits from Reddit in the eleven target languages, and the Ukrainian-only UberText 2.0 social media corpus. While Wikipedia edits were derived from human-made corrections, the Reddit and UberText 2.0 data were automatically corrected with the GPT-4o-mini model. The quality of the corrections in the datasets was evaluated both automatically and manually. Finally, we fine-tune two open-source large language models - Aya-Expanse (8B) and Gemma-3 (12B) - on the multilingual OmniGEC corpora and achieve state-of-the-art (SOTA) results for paragraph-level multilingual GEC. The dataset collection and the best-performing models are available on Hugging Face.

Method: Proposes Delta-KD that explicitly preserves the distributional shift (Delta) introduced during teacher’s supervised finetuning, encouraging student to approximate optimal representation space.

Result: Empirical results on ROUGE metrics show Delta-KD substantially improves student performance while better preserving teacher’s knowledge.

Conclusion: Delta-KD provides a more effective knowledge distillation approach by accounting for representation space differences between teacher and student models.

Abstract: Knowledge distillation (KD) is a widely adopted approach for compressing large neural networks by transferring knowledge from a large teacher model to a smaller student model. In the context of large language models, token level KD, typically minimizing the KL divergence between student output distribution and teacher output distribution, has shown strong empirical performance. However, prior work assumes student output distribution and teacher output distribution share the same optimal representation space, a premise that may not hold in many cases. To solve this problem, we propose Delta Knowledge Distillation (Delta-KD), a novel extension of token level KD that encourages the student to approximate an optimal representation space by explicitly preserving the distributional shift Delta introduced during the teacher’s supervised finetuning (SFT). Empirical results on ROUGE metrics demonstrate that Delta KD substantially improves student performance while preserving more of the teacher’s knowledge.

Method: Comprehensive evaluation using ensemble metrics (authorship attribution, verification, style matching, AI detection) across 40,000+ generations from 400+ real authors, testing various prompting strategies and domain coverage.

Result: LLMs can approximate styles in structured formats but struggle with nuanced informal writing; analysis reveals limitations in effective personalization through prompting.

Conclusion: There’s a fundamental gap in personalized LLM adaptation, highlighting the need for improved techniques to support implicit, style-consistent generation.

Abstract: As large language models (LLMs) become increasingly integrated into personal writing tools, a critical question arises: can LLMs faithfully imitate an individual’s writing style from just a few examples? Personal style is often subtle and implicit, making it difficult to specify through prompts yet essential for user-aligned generation. This work presents a comprehensive evaluation of state-of-the-art LLMs’ ability to mimic personal writing styles via in-context learning from a small number of user-authored samples. We introduce an ensemble of complementary metrics-including authorship attribution, authorship verification, style matching, and AI detection-to robustly assess style imitation. Our evaluation spans over 40000 generations per model across domains such as news, email, forums, and blogs, covering writing samples from more than 400 real-world authors. Results show that while LLMs can approximate user styles in structured formats like news and email, they struggle with nuanced, informal writing in blogs and forums. Further analysis on various prompting strategies such as number of demonstrations reveal key limitations in effective personalization. Our findings highlight a fundamental gap in personalized LLM adaptation and the need for improved techniques to support implicit, style-consistent generation. To aid future research and for reproducibility, we open-source our data and code.

Method: Uses three categories of linguistic features (readability, syntactic, and lexical features) to quantify text complexity, trains LLMs on linguistically annotated dialogue data, and introduces Dilaprix metric for evaluation.

Result: The approach achieves superior controllability of language proficiency while maintaining high dialogue quality, outperforming prompt-based methods in both flexibility and stability.

Conclusion: Training LLMs on linguistically annotated data enables precise modulation of language proficiency, providing an effective framework for adaptive educational dialogue systems in second language learning.

Abstract: Large language models (LLMs) have emerged as powerful tools for supporting second language acquisition, particularly in simulating interactive dialogues for speaking practice. However, adapting the language difficulty of LLM-generated responses to match learners’ proficiency levels remains a challenge. This work addresses this issue by proposing a framework for controlling language proficiency in educational dialogue systems. Our approach leverages three categories of linguistic features, readability features (e.g., Flesch-Kincaid Grade Level), syntactic features (e.g., syntactic tree depth), and lexical features (e.g., simple word ratio), to quantify and regulate text complexity. We demonstrate that training LLMs on linguistically annotated dialogue data enables precise modulation of language proficiency, outperforming prompt-based methods in both flexibility and stability. To evaluate this, we introduce Dilaprix, a novel metric integrating the aforementioned features, which shows strong correlation with expert judgments of language difficulty. Empirical results reveal that our approach achieves superior controllability of language proficiency while maintaining high dialogue quality.

Method: Conducted a systematic review of recent studies applying LLMs to thematic analysis, complemented by an interview with a practicing clinician.

Result: Found that current approaches remain fragmented across multiple dimensions including types of thematic analysis, datasets, prompting strategies and models used, with widely varying evaluation methods that hinder progress and benchmarking.

Conclusion: Establishing standardized evaluation practices is critical, and the authors propose an evaluation framework centered on three dimensions: validity, reliability, and interpretability.

Abstract: This position paper examines how large language models (LLMs) can support thematic analysis of unstructured clinical transcripts, a widely used but resource-intensive method for uncovering patterns in patient and provider narratives. We conducted a systematic review of recent studies applying LLMs to thematic analysis, complemented by an interview with a practicing clinician. Our findings reveal that current approaches remain fragmented across multiple dimensions including types of thematic analysis, datasets, prompting strategies and models used, most notably in evaluation. Existing evaluation methods vary widely (from qualitative expert review to automatic similarity metrics), hindering progress and preventing meaningful benchmarking across studies. We argue that establishing standardized evaluation practices is critical for advancing the field. To this end, we propose an evaluation framework centered on three dimensions: validity, reliability, and interpretability.

Method: Leveraged IndoBERT and DistilBERT language models with data processing techniques including back-translation and synonym replacement for data augmentation, followed by hyperparameter tuning.

Result: IndoBERT achieved 80% accuracy after hyperparameter tuning. Data augmentation significantly boosted performance, while combining multiple IndoBERT models provided only slight improvement.

Conclusion: IndoBERT is the most effective model for Indonesian emotion classification, with data augmentation being crucial for high accuracy. Future research should explore alternative architectures and strategies for better generalization in Indonesian NLP tasks.

Abstract: Understanding emotions in the Indonesian language is essential for improving customer experiences in e-commerce. This study focuses on enhancing the accuracy of emotion classification in Indonesian by leveraging advanced language models, IndoBERT and DistilBERT. A key component of our approach was data processing, specifically data augmentation, which included techniques such as back-translation and synonym replacement. These methods played a significant role in boosting the model’s performance. After hyperparameter tuning, IndoBERT achieved an accuracy of 80%, demonstrating the impact of careful data processing. While combining multiple IndoBERT models led to a slight improvement, it did not significantly enhance performance. Our findings indicate that IndoBERT was the most effective model for emotion classification in Indonesian, with data augmentation proving to be a vital factor in achieving high accuracy. Future research should focus on exploring alternative architectures and strategies to improve generalization for Indonesian NLP tasks.

Method: “Reveal-and-Release” method that prompts the model to generate its own forget data using optimized instructions, followed by iterative unlearning with parameter-efficient modules trained on the self-generated data.

Result: Experimental results show the method effectively balances the tradeoff between forget quality (removing undesirable information) and utility preservation (maintaining model performance).

Conclusion: The proposed approach enables effective LLM unlearning without requiring access to the original sensitive forget data, using self-generated data and iterative parameter-efficient adjustments.

Abstract: Large language model (LLM) unlearning has demonstrated effectiveness in removing the influence of undesirable data (also known as forget data). Existing approaches typically assume full access to the forget dataset, overlooking two key challenges: (1) Forget data is often privacy-sensitive, rare, or legally regulated, making it expensive or impractical to obtain (2) The distribution of available forget data may not align with how that information is represented within the model. To address these limitations, we propose a ``Reveal-and-Release’’ method to unlearn with self-generated data, where we prompt the model to reveal what it knows using optimized instructions. To fully utilize the self-generated forget data, we propose an iterative unlearning framework, where we make incremental adjustments to the model’s weight space with parameter-efficient modules trained on the forget data. Experimental results demonstrate that our method balances the tradeoff between forget quality and utility preservation.

Method: Constructed by crawling 77,100 GitHub issues from 11 popular repositories, developing a two-level taxonomy of repository-level questions, manually curating and validating questions, and collecting corresponding answers. Also developed SWE-QA-Agent framework for automated QA.

Result: Experimental evaluation of six advanced LLMs shows promise in repository-level QA, particularly with the SWE-QA-Agent framework, while revealing open challenges.

Conclusion: SWE-QA facilitates research on automated QA systems in realistic code environments and points to future research directions for repository-level code understanding and reasoning.

Abstract: Understanding and reasoning about entire software repositories is an essential capability for intelligent software engineering tools. While existing benchmarks such as CoSQA and CodeQA have advanced the field, they predominantly focus on small, self-contained code snippets. These setups fail to capture the complexity of real-world repositories, where effective understanding and reasoning often require navigating multiple files, understanding software architecture, and grounding answers in long-range code dependencies. In this paper, we present SWE-QA, a repository-level code question answering (QA) benchmark designed to facilitate research on automated QA systems in realistic code environments. SWE-QA involves 576 high-quality question-answer pairs spanning diverse categories, including intention understanding, cross-file reasoning, and multi-hop dependency analysis. To construct SWE-QA, we first crawled 77,100 GitHub issues from 11 popular repositories. Based on an analysis of naturally occurring developer questions extracted from these issues, we developed a two-level taxonomy of repository-level questions and constructed a set of seed questions for each category. For each category, we manually curated and validated questions and collected their corresponding answers. As a prototype application, we further develop SWE-QA-Agent, an agentic framework in which LLM agents reason and act to find answers automatically. We evaluate six advanced LLMs on SWE-QA under various context augmentation strategies. Experimental results highlight the promise of LLMs, particularly our SWE-QA-Agent framework, in addressing repository-level QA, while also revealing open challenges and pointing to future research directions.

Method: MUSE-A uses frame semantics and heuristic tree search to explore diverse semantic trajectories for multi-turn attacks. MUSE-D provides a fine-grained safety alignment approach that intervenes early in dialogues to reduce vulnerabilities.

Result: Extensive experiments on various models show that MUSE effectively identifies and mitigates multi-turn vulnerabilities in large language models.

Conclusion: The MUSE framework successfully addresses the critical gap in multi-turn jailbreak protection, providing both attack generation and defense mechanisms that work effectively across different LLM models.

Abstract: As large language models~(LLMs) become widely adopted, ensuring their alignment with human values is crucial to prevent jailbreaks where adversaries manipulate models to produce harmful content. While most defenses target single-turn attacks, real-world usage often involves multi-turn dialogues, exposing models to attacks that exploit conversational context to bypass safety measures. We introduce MUSE, a comprehensive framework tackling multi-turn jailbreaks from both attack and defense angles. For attacks, we propose MUSE-A, a method that uses frame semantics and heuristic tree search to explore diverse semantic trajectories. For defense, we present MUSE-D, a fine-grained safety alignment approach that intervenes early in dialogues to reduce vulnerabilities. Extensive experiments on various models show that MUSE effectively identifies and mitigates multi-turn vulnerabilities. Code is available at \href{https://github.com/yansiyu02/MUSE}{https://github.com/yansiyu02/MUSE}.

Method: Enhanced UMA model with a split module that allows each aggregated frame to map to multiple tokens, generating two tokens from each aggregated frame before computing CTC loss.

Result: The proposed method enables better handling of English speech recognition by accommodating fine-grained tokenization patterns that differ from Mandarin.

Conclusion: The split module extension to UMA successfully addresses cross-linguistic challenges, making the model effective for both English and Mandarin speech recognition tasks.

Abstract: This paper proposes a unimodal aggregation (UMA) based nonautoregressive model for both English and Mandarin speech recognition. The original UMA explicitly segments and aggregates acoustic frames (with unimodal weights that first monotonically increase and then decrease) of the same text token to learn better representations than regular connectionist temporal classification (CTC). However, it only works well in Mandarin. It struggles with other languages, such as English, for which a single syllable may be tokenized into multiple fine-grained tokens, or a token spans fewer than 3 acoustic frames and fails to form unimodal weights. To address this problem, we propose allowing each UMA-aggregated frame map to multiple tokens, via a simple split module that generates two tokens from each aggregated frame before computing the CTC loss.

Method: Uses a 2.59M-parameter MLP gating network to dynamically select text-only, image-only, or fusion path per table-query pair, with an agent to mediate cross-modal knowledge integration by analyzing outputs and reasoning.

Result: Achieves new state-of-the-art performance on seven benchmarks, surpassing the strongest baseline by an average of 4.02% among open-source models.

Conclusion: TableDART provides an efficient framework for table understanding that dynamically integrates multimodal views without costly MLLM fine-tuning, effectively reducing redundancy and conflicts while maintaining high performance.

Abstract: Modeling semantic and structural information from tabular data remains a core challenge for effective table understanding. Existing Table-as-Text approaches flatten tables for large language models (LLMs), but lose crucial structural cues, while Table-as-Image methods preserve structure yet struggle with fine-grained semantics. Recent Table-as-Multimodality strategies attempt to combine textual and visual views, but they (1) statically process both modalities for every query-table pair within a large multimodal LLMs (MLLMs), inevitably introducing redundancy and even conflicts, and (2) depend on costly fine-tuning of MLLMs. In light of this, we propose TableDART, a training-efficient framework that integrates multimodal views by reusing pretrained single-modality models. TableDART introduces a lightweight 2.59M-parameter MLP gating network that dynamically selects the optimal path (either Text-only, Image-only, or Fusion) for each table-query pair, effectively reducing redundancy and conflicts from both modalities. In addition, we propose a novel agent to mediate cross-modal knowledge integration by analyzing outputs from text- and image-based models, either selecting the best result or synthesizing a new answer through reasoning. This design avoids the prohibitive costs of full MLLM fine-tuning. Extensive experiments on seven benchmarks show that TableDART establishes new state-of-the-art performance among open-source models, surpassing the strongest baseline by an average of 4.02%. The code is available at: https://anonymous.4open.science/r/TableDART-C52B

Method: Uses iterative self-distillation to train large bilingual SSL models, then distills knowledge into compressed student models. Employs low-rank approximation to compact teacher’s discrete supervision into shallow, thin models.

Result: HArnESS demonstrates effectiveness against HuBERT and XLS-R on Arabic ASR, Speaker Emotion Recognition, and Dialect Identification tasks, achieving SOTA or comparable performance with minimal fine-tuning.

Conclusion: HArnESS provides a lightweight yet powerful alternative for real-world Arabic speech applications in low-resource settings, with models released to support responsible research and deployment.

Abstract: Large pre-trained speech models excel in downstream tasks but their deployment is impractical for resource-limited environments. In this paper, we introduce HArnESS, the first Arabic-centric self-supervised speech model family, designed to capture Arabic speech nuances. Using iterative self-distillation, we train large bilingual HArnESS (HL) SSL models and then distill knowledge into compressed student models (HS, HST), preserving Arabic-specific representations. We use low-rank approximation to further compact the teacher’s discrete supervision into shallow, thin models. We evaluate HArnESS on Arabic ASR, Speaker Emotion Recognition (SER), and Dialect Identification (DID), demonstrating effectiveness against HuBERT and XLS-R. With minimal fine-tuning, HArnESS achieves SOTA or comparable performance, making it a lightweight yet powerful alternative for real-world use. We release our distilled models and findings to support responsible research and deployment in low-resource settings.

Method: Built large-scale dataset of contemporary political discourse, employed three refined judgment methods representing different detection approaches, used leave-one-out strategy for label agreement analysis, and established pseudo-labels for quantitative assessment.

Result: Identified distinct patterns for each judgment method and found that strategically designed single prompting achieves performance comparable to more resource-intensive methods.

Conclusion: A single prompting approach provides a feasible and effective solution for offensive language detection in real-world settings with inherent constraints, offering comparable results to more complex methods.

Abstract: Although offensive language continually evolves over time, even recent studies using LLMs have predominantly relied on outdated datasets and rarely evaluated the generalization ability on unseen texts. In this study, we constructed a large-scale dataset of contemporary political discourse and employed three refined judgments in the absence of ground truth. Each judgment reflects a representative offensive language detection method and is carefully designed for optimal conditions. We identified distinct patterns for each judgment and demonstrated tendencies of label agreement using a leave-one-out strategy. By establishing pseudo-labels as ground trust for quantitative performance assessment, we observed that a strategically designed single prompting achieves comparable performance to more resource-intensive methods. This suggests a feasible approach applicable in real-world settings with inherent constraints.

Method: Decoupled Proxy Alignment (DPA) with two innovations: 1) proxy LLM during pretraining to decouple vision-language alignment from language prior interference, 2) dynamic loss adjustment based on visual relevance to strengthen optimization for visually relevant tokens.

Result: DPA significantly mitigates language prior conflict, achieves superior alignment performance across diverse datasets, model families, and scales, and shows exceptional generalization capabilities.

Conclusion: DPA provides a robust approach for vision-language alignment that improves MLLM training effectiveness and addresses the previously overlooked issue of language prior conflict.

Abstract: Multimodal large language models (MLLMs) have gained significant attention due to their impressive ability to integrate vision and language modalities. Recent advancements in MLLMs have primarily focused on improving performance through high-quality datasets, novel architectures, and optimized training strategies. However, in this paper, we identify a previously overlooked issue, language prior conflict, a mismatch between the inherent language priors of large language models (LLMs) and the language priors in training datasets. This conflict leads to suboptimal vision-language alignment, as MLLMs are prone to adapting to the language style of training samples. To address this issue, we propose a novel training method called Decoupled Proxy Alignment (DPA). DPA introduces two key innovations: (1) the use of a proxy LLM during pretraining to decouple the vision-language alignment process from language prior interference, and (2) dynamic loss adjustment based on visual relevance to strengthen optimization signals for visually relevant tokens. Extensive experiments demonstrate that DPA significantly mitigates the language prior conflict, achieving superior alignment performance across diverse datasets, model families, and scales. Our method not only improves the effectiveness of MLLM training but also shows exceptional generalization capabilities, making it a robust approach for vision-language alignment. Our code is available at https://github.com/fnlp-vision/DPA.

Method: Introduces UnifiedVisual framework and UnifiedVisual-240K dataset that integrates diverse visual/textual inputs and outputs, enabling comprehensive cross-modal reasoning and precise text-to-image alignment across various tasks and data sources.

Result: Models trained on UnifiedVisual-240K achieve strong performance across diverse tasks and exhibit significant mutual reinforcement between multimodal understanding and generation capabilities.

Conclusion: UnifiedVisual represents a new growth point for advancing unified VLLMs and unlocking their full potential by providing a dataset that facilitates synergistic enhancement between understanding and generation abilities.

Abstract: Unified vision large language models (VLLMs) have recently achieved impressive advancements in both multimodal understanding and generation, powering applications such as visual question answering and text-guided image synthesis. However, progress in unified VLLMs remains constrained by the lack of datasets that fully exploit the synergistic potential between these two core abilities. Existing datasets typically address understanding and generation in isolation, thereby limiting the performance of unified VLLMs. To bridge this critical gap, we introduce a novel dataset construction framework, UnifiedVisual, and present UnifiedVisual-240K, a high-quality dataset meticulously designed to facilitate mutual enhancement between multimodal understanding and generation. UnifiedVisual-240K seamlessly integrates diverse visual and textual inputs and outputs, enabling comprehensive cross-modal reasoning and precise text-to-image alignment. Our dataset encompasses a wide spectrum of tasks and data sources, ensuring rich diversity and addressing key shortcomings of prior resources. Extensive experiments demonstrate that models trained on UnifiedVisual-240K consistently achieve strong performance across a wide range of tasks. Notably, these models exhibit significant mutual reinforcement between multimodal understanding and generation, further validating the effectiveness of our framework and dataset. We believe UnifiedVisual represents a new growth point for advancing unified VLLMs and unlocking their full potential. Our code and datasets is available at https://github.com/fnlp-vision/UnifiedVisual.

Method: The authors evaluate passage-level and document-level CLIR using multilingual bi-encoders as first-stage retrievers and compare various LLM-based reranking approaches, including pairwise and listwise methods.

Result: Results show that multilingual bi-encoders achieve better performance than MT-based retrieval, translation benefits diminish with stronger rerankers, and instruction-tuned LLM pairwise rerankers perform competitively with listwise approaches.

Conclusion: Current state-of-the-art rerankers fall short in CLIR without MT, highlighting the importance of proper first-stage retrieval and showing that instruction-tuned LLMs can provide competitive performance without complex listwise approaches.

Abstract: Multi-stage information retrieval (IR) has become a widely-adopted paradigm in search. While Large Language Models (LLMs) have been extensively evaluated as second-stage reranking models for monolingual IR, a systematic large-scale comparison is still lacking for cross-lingual IR (CLIR). Moreover, while prior work shows that LLM-based rerankers improve CLIR performance, their evaluation setup relies on lexical retrieval with machine translation (MT) for the first stage. This is not only prohibitively expensive but also prone to error propagation across stages. Our evaluation on passage-level and document-level CLIR reveals that further gains can be achieved with multilingual bi-encoders as first-stage retrievers and that the benefits of translation diminishes with stronger reranking models. We further show that pairwise rerankers based on instruction-tuned LLMs perform competitively with listwise rerankers. To the best of our knowledge, we are the first to study the interaction between retrievers and rerankers in two-stage CLIR with LLMs. Our findings reveal that, without MT, current state-of-the-art rerankers fall severely short when directly applied in CLIR.

Method: Created KAIO - a private Korean benchmark focused on math problems requiring long-chain reasoning, served via a held-out evaluator to prevent contamination until models reach 80% accuracy.

Result: Current best model (GPT-5) scores 62.8%, Gemini-2.5-Pro scores 52.3%, while open models cluster below 30%, showing substantial room for improvement and effective differentiation between models.

Conclusion: KAIO successfully addresses the Korean benchmark saturation problem and enables robust tracking of frontier model progress, with plans to release and iterate to harder versions once models reach 80% accuracy.

Abstract: With the advancement of mid/post-training techniques, LLMs are pushing their boundaries at an accelerated pace. Legacy benchmarks saturate quickly (e.g., broad suites like MMLU over the years, newer ones like GPQA-D even faster), which makes frontier progress hard to track. The problem is especially acute in Korean: widely used benchmarks are fewer, often translated or narrow in scope, and updated more slowly, so saturation and contamination arrive sooner. Accordingly, at this moment, there is no Korean benchmark capable of evaluating and ranking frontier models. To bridge this gap, we introduce KAIO, a Korean, math-centric benchmark that stresses long-chain reasoning. Unlike recent Korean suites that are at or near saturation, KAIO remains far from saturated: the best-performing model, GPT-5, attains 62.8, followed by Gemini-2.5-Pro (52.3). Open models such as Qwen3-235B and DeepSeek-R1 cluster falls below 30, demonstrating substantial headroom, enabling robust tracking of frontier progress in Korean. To reduce contamination, KAIO will remain private and be served via a held-out evaluator until the best publicly known model reaches at least 80% accuracy, after which we will release the set and iterate to a harder version.

Method: Align3 employs Test-Time Deliberation (TTD) with hierarchical reflection and revision to reason over specification boundaries. The method is evaluated using SpecBench, a unified benchmark covering 5 scenarios, 103 specifications, and 1,500 prompts.

Result: Experiments on 15 reasoning and 18 instruction models show that: (i) test-time deliberation enhances specification alignment; (ii) Align3 advances the safety-helpfulness trade-off frontier with minimal overhead; (iii) SpecBench effectively reveals alignment gaps.

Conclusion: Test-time deliberation is an effective strategy for reasoning over real-world specification boundaries, with Align3 demonstrating improved alignment capabilities while maintaining efficiency.

Abstract: Large language models (LLMs) are increasingly applied in diverse real-world scenarios, each governed by bespoke behavioral and safety specifications (spec) custom-tailored by users or organizations. These spec, categorized into safety-spec and behavioral-spec, vary across scenarios and evolve with changing preferences and requirements. We formalize this challenge as specification alignment, focusing on LLMs’ ability to follow dynamic, scenario-specific spec from both behavioral and safety perspectives. To address this challenge, we propose Align3, a lightweight method that employs Test-Time Deliberation (TTD) with hierarchical reflection and revision to reason over the specification boundaries. We further present SpecBench, a unified benchmark for measuring specification alignment, covering 5 scenarios, 103 spec, and 1,500 prompts. Experiments on 15 reasoning and 18 instruct models with several TTD methods, including Self-Refine, TPO, and MoreThink, yield three key findings: (i) test-time deliberation enhances specification alignment; (ii) Align3 advances the safety-helpfulness trade-off frontier with minimal overhead; (iii) SpecBench effectively reveals alignment gaps. These results highlight the potential of test-time deliberation as an effective strategy for reasoning over the real-world specification boundaries.

Method: Used pre-trained transformer models with comprehensive data preprocessing and balancing techniques, integrated with Long-short Term Memory (LSTM) architecture.

Result: Ranked 7th out of 49 submissions with F1 score of 0.126, achieving highest values in recall metrics and early detection-related metrics.

Conclusion: The integration of transformer models with LSTM architecture shows promise for early detection tasks, though further improvements are needed to enhance overall performance metrics.

Abstract: This paper describes the participation of the SINAI team in the eRisk@CLEF lab. Specifically, one of the proposed tasks has been addressed: Task 2 on the early detection of signs of pathological gambling. The approach presented in Task 2 is based on pre-trained models from Transformers architecture with comprehensive preprocessing data and data balancing techniques. Moreover, we integrate Long-short Term Memory (LSTM) architecture with automodels from Transformers. In this Task, our team has been ranked in seventh position, with an F1 score of 0.126, out of 49 participant submissions and achieves the highest values in recall metrics and metrics related to early detection.

Method: For Task 1: Used sentence embeddings from Transformers combined with volumetry, lexical diversity, complexity metrics, and emotion-related scores. For Task 3: Employed text similarity estimation using contextualized word embeddings from Transformers.

Result: Achieved 2nd position in both tasks - F1 score of 0.808 in Task 1 (out of 41 submissions) and 2nd place in Task 3 (out of 3 teams).

Conclusion: Transformer-based approaches with additional linguistic features are effective for early detection of mental health issues from text data, demonstrating strong performance in competitive evaluation settings.

Abstract: This paper describes the participation of the SINAI team in the eRisk@CLEF lab. Specifically, two of the proposed tasks have been addressed: i) Task 1 on the early detection of signs of pathological gambling, and ii) Task 3 on measuring the severity of the signs of eating disorders. The approach presented in Task 1 is based on the use of sentence embeddings from Transformers with features related to volumetry, lexical diversity, complexity metrics, and emotion-related scores, while the approach for Task 3 is based on text similarity estimation using contextualized word embeddings from Transformers. In Task 1, our team has been ranked in second position, with an F1 score of 0.808, out of 41 participant submissions. In Task 3, our team also placed second out of a total of 3 participating teams.

Method: Isolates language vectors using multi-parallel corpus, then applies fixed (unsupervised) and trainable steering functions for effective language steering.

Result: Effectively mitigates language confusion in both monolingual and cross-lingual setups across 3 benchmarks and 18 languages while retaining task performance.

Conclusion: ReCoVeR provides an effective solution for reducing language confusion in multilingual LLMs without compromising their task capabilities.

Abstract: As they become increasingly multilingual, Large Language Models (LLMs) exhibit more language confusion, i.e., they tend to generate answers in a language different from the language of the prompt or the answer language explicitly requested by the user. In this work, we propose ReCoVeR (REducing language COnfusion in VEctor Representations), a novel lightweight approach for reducing language confusion based on language-specific steering vectors. We first isolate language vectors with the help of multi-parallel corpus and then effectively leverage those vectors for effective LLM steering via fixed (i.e., unsupervised) as well as trainable steering functions. Our extensive evaluation, encompassing three benchmarks and 18 languages, shows that ReCoVeR effectively mitigates language confusion in both monolingual and cross-lingual setups while at the same time – and in contrast to prior language steering methods – retaining task performance. Our data code is available at https://github.com/hSterz/recover.

Method: Proposes Roundtable Essay Scoring (RES) framework with multiple LLM-based evaluator agents. Each agent is tailored to specific prompts/topics, generates trait-based rubrics, conducts independent multi-perspective evaluations, then engages in simulated roundtable discussions for dialectical reasoning to reach consensus on final holistic scores.

Result: Experiments on ASAP dataset using ChatGPT and Claude show RES achieves up to 34.86% improvement in average Quadratic Weighted Kappa (QWK) over straightforward prompting methods, demonstrating superior alignment with human evaluation.

Conclusion: RES successfully addresses the challenge of human-aligned multi-perspective essay scoring through collaborative multi-agent evaluation and dialectical reasoning, significantly outperforming previous zero-shot AES approaches and better simulating human grading processes.

Abstract: The emergence of large language models (LLMs) has brought a new paradigm to automated essay scoring (AES), a long-standing and practical application of natural language processing in education. However, achieving human-level multi-perspective understanding and judgment remains a challenge. In this work, we propose Roundtable Essay Scoring (RES), a multi-agent evaluation framework designed to perform precise and human-aligned scoring under a zero-shot setting. RES constructs evaluator agents based on LLMs, each tailored to a specific prompt and topic context. Each agent independently generates a trait-based rubric and conducts a multi-perspective evaluation. Then, by simulating a roundtable-style discussion, RES consolidates individual evaluations through a dialectical reasoning process to produce a final holistic score that more closely aligns with human evaluation. By enabling collaboration and consensus among agents with diverse evaluation perspectives, RES outperforms prior zero-shot AES approaches. Experiments on the ASAP dataset using ChatGPT and Claude show that RES achieves up to a 34.86% improvement in average QWK over straightforward prompting (Vanilla) methods.

Method: V-SEAM combines Visual Semantic Editing and Attention Modulating, enabling concept-level visual manipulations and identifying attention heads with positive/negative contributions across three semantic levels. Includes automatic method to modulate key head embeddings.

Result: Positive heads are often shared within the same semantic level but vary across levels, while negative heads generalize broadly. The method demonstrates enhanced performance for both LLaVA and InstructBLIP across three diverse VQA benchmarks.

Conclusion: V-SEAM provides a more semantically meaningful approach to causal interpretability of VLMs, revealing insights about attention head contributions across different semantic levels and enabling performance improvements through targeted modulation.

Abstract: Recent advances in causal interpretability have extended from language models to vision-language models (VLMs), seeking to reveal their internal mechanisms through input interventions. While textual interventions often target semantics, visual interventions typically rely on coarse pixel-level perturbations, limiting semantic insights on multimodal integration. In this study, we introduce V-SEAM, a novel framework that combines Visual Semantic Editing and Attention Modulating for causal interpretation of VLMs. V-SEAM enables concept-level visual manipulations and identifies attention heads with positive or negative contributions to predictions across three semantic levels: objects, attributes, and relationships. We observe that positive heads are often shared within the same semantic level but vary across levels, while negative heads tend to generalize broadly. Finally, we introduce an automatic method to modulate key head embeddings, demonstrating enhanced performance for both LLaVA and InstructBLIP across three diverse VQA benchmarks. Our data and code are released at: https://github.com/petergit1/V-SEAM.

Method: Integrates Chain-of-Empathy (CoE) reasoning inspired by cognitive-behavioral therapy with Reinforcement Learning. Uses two-stage training: Supervised Fine-Tuning for CoE structure, then RL guided by reward model for therapeutic relevance. Built on new large-scale Chinese dataset Empathy-QA.

Result: Achieves strong performance on automatic metrics. Human evaluations show clear superiority over baselines with 44.30% Win@1 rate on new benchmark. Produces interpretable, contextually nuanced responses.

Conclusion: Empathy-R1 represents significant advancement in developing responsible and beneficial AI for mental health support by enabling interpretable and psychologically appropriate responses.

Abstract: Empathy is critical for effective mental health support, especially when addressing Long Counseling Texts (LCTs). However, existing Large Language Models (LLMs) often generate replies that are semantically fluent but lack the structured reasoning necessary for genuine psychological support, particularly in a Chinese context. To bridge this gap, we introduce Empathy-R1, a novel framework that integrates a Chain-of-Empathy (CoE) reasoning process with Reinforcement Learning (RL) to enhance response quality for LCTs. Inspired by cognitive-behavioral therapy, our CoE paradigm guides the model to sequentially reason about a help-seeker’s emotions, causes, and intentions, making its thinking process both transparent and interpretable. Our framework is empowered by a new large-scale Chinese dataset, Empathy-QA, and a two-stage training process. First, Supervised Fine-Tuning instills the CoE’s reasoning structure. Subsequently, RL, guided by a dedicated reward model, refines the therapeutic relevance and contextual appropriateness of the final responses. Experiments show that Empathy-R1 achieves strong performance on key automatic metrics. More importantly, human evaluations confirm its superiority, showing a clear preference over strong baselines and achieving a Win@1 rate of 44.30% on our new benchmark. By enabling interpretable and contextually nuanced responses, Empathy-R1 represents a significant advancement in developing responsible and genuinely beneficial AI for mental health support.

Method: Uses a unified tokenizer and single Transformer decoder to model interleaved sequences of semantic and acoustic tokens. Employs multiple quantizers and introduces LLM-as-a-Judge evaluation for spoken content quality assessment.

Result: Achieves state-of-the-art performance in acoustic consistency and speaker identity preservation. Analysis shows trade-off between acoustic fidelity (improved with more quantizers) and linguistic performance (degraded with more quantizers).

Conclusion: Llama-Mimi demonstrates successful joint modeling of semantic and acoustic tokens but highlights the inherent challenge of maintaining long-term coherence when balancing acoustic fidelity with linguistic performance.

Abstract: We propose Llama-Mimi, a speech language model that uses a unified tokenizer and a single Transformer decoder to jointly model sequences of interleaved semantic and acoustic tokens. Comprehensive evaluation shows that Llama-Mimi achieves state-of-the-art performance in acoustic consistency and possesses the ability to preserve speaker identity. Our analysis further demonstrates that increasing the number of quantizers improves acoustic fidelity but degrades linguistic performance, highlighting the inherent challenge of maintaining long-term coherence. We additionally introduce an LLM-as-a-Judge-based evaluation to assess the spoken content quality of generated outputs. Our models, code, and speech samples are publicly available.

Method: A two-stage interview strategy (pre-interview and formal interview) with dynamic interviewer weight adjustment and adaptive question difficulty selection to ensure fairness and efficiency.

Result: Achieves significantly higher correlation with full-coverage results than random sampling (up to 17.6% PLCC and 16.7% SRCC improvement) while reducing the number of required questions.

Conclusion: The proposed interview paradigm provides a reliable and efficient alternative for large-scale MLLM benchmarking, demonstrating human-inspired evaluation can be more effective than traditional approaches.

Abstract: The rapid progress of Multi-Modal Large Language Models (MLLMs) has spurred the creation of numerous benchmarks. However, conventional full-coverage Question-Answering evaluations suffer from high redundancy and low efficiency. Inspired by human interview processes, we propose a multi-to-one interview paradigm for efficient MLLM evaluation. Our framework consists of (i) a two-stage interview strategy with pre-interview and formal interview phases, (ii) dynamic adjustment of interviewer weights to ensure fairness, and (iii) an adaptive mechanism for question difficulty-level chosen. Experiments on different benchmarks show that the proposed paradigm achieves significantly higher correlation with full-coverage results than random sampling, with improvements of up to 17.6% in PLCC and 16.7% in SRCC, while reducing the number of required questions. These findings demonstrate that the proposed paradigm provides a reliable and efficient alternative for large-scale MLLM benchmarking.

Method: Pretrained 3 domain-specific masked language models using ModernBERT architecture with optimizations (FlashAttention, rotary embeddings, GLU layers) on 60M patent records.

Result: ModernBERT-base-PT outperforms general-purpose ModernBERT on 3/4 patent classification tasks, achieves competitive performance with PatentBERT, and maintains 3x faster inference speed.

Conclusion: Domain-specific pretraining with architectural improvements significantly enhances performance on patent NLP tasks while maintaining computational efficiency for time-sensitive applications.

Abstract: Transformer-based language models such as BERT have become foundational in NLP, yet their performance degrades in specialized domains like patents, which contain long, technical, and legally structured text. Prior approaches to patent NLP have primarily relied on fine-tuning general-purpose models or domain-adapted variants pretrained with limited data. In this work, we pretrain 3 domain-specific masked language models for patents, using the ModernBERT architecture and a curated corpus of over 60 million patent records. Our approach incorporates architectural optimizations, including FlashAttention, rotary embeddings, and GLU feed-forward layers. We evaluate our models on four downstream patent classification tasks. Our model, ModernBERT-base-PT, consistently outperforms the general-purpose ModernBERT baseline on three out of four datasets and achieves competitive performance with a baseline PatentBERT. Additional experiments with ModernBERT-base-VX and Mosaic-BERT-large demonstrate that scaling the model size and customizing the tokenizer further enhance performance on selected tasks. Notably, all ModernBERT variants retain substantially faster inference over - 3x that of PatentBERT - underscoring their suitability for time-sensitive applications. These results underscore the benefits of domain-specific pretraining and architectural improvements for patent-focused NLP tasks.

Method: Uses self-routing via angular similarity between input and adapter directions, shared learnable magnitude vector, expert selection loss for sparsity, and includes a shared expert for stable knowledge.

Result: Significantly outperforms standard LoRA, matches or surpasses existing MoE-LoRA methods, and eliminates all inference-time overhead while enabling full model merging.

Conclusion: FURINA is the first router-free MoE-enhanced LoRA method that can be fully merged into backbone models with zero additional inference cost, achieving superior performance.

Abstract: The Mixture of Experts (MoE) paradigm has been successfully integrated into Low-Rank Adaptation (LoRA) for parameter-efficient fine-tuning (PEFT), delivering performance gains with minimal parameter overhead. However, a key limitation of existing MoE-LoRA methods is their reliance on a discrete router, which prevents the integration of the MoE components into the backbone model. To overcome this, we propose FURINA, a novel Free from Unmergeable Router framework based on the LINear Aggregation of experts. FURINA eliminates the router by introducing a Self-Routing mechanism. This is achieved through three core innovations: (1) decoupled learning of the direction and magnitude for LoRA adapters, (2) a shared learnable magnitude vector for consistent activation scaling, and (3) expert selection loss that encourages divergent expert activation. The proposed mechanism leverages the angular similarity between the input and each adapter’s directional component to activate experts, which are then scaled by the shared magnitude vector. This design allows the output norm to naturally reflect the importance of each expert, thereby enabling dynamic, router-free routing. The expert selection loss further sharpens this behavior by encouraging sparsity and aligning it with standard MoE activation patterns. We also introduce a shared expert within the MoE-LoRA block that provides stable, foundational knowledge. To the best of our knowledge, FURINA is the first router-free, MoE-enhanced LoRA method that can be fully merged into the backbone model, introducing zero additional inference-time cost or complexity. Extensive experiments demonstrate that FURINA not only significantly outperforms standard LoRA but also matches or surpasses the performance of existing MoE-LoRA methods, while eliminating the extra inference-time overhead of MoE.

Method: Cross-modal knowledge distillation framework that uses both text-to-text and speech-to-text channels to transfer knowledge from a text-based teacher model to a speech LLM.

Result: Extensive experiments on dialogue and audio understanding tasks show the approach effectively preserves textual knowledge, improves cross-modal alignment, and enhances reasoning in speech-based interactions.

Conclusion: The proposed knowledge distillation framework successfully mitigates catastrophic forgetting and modality inequivalence in speech LLMs, enabling better performance across both text and speech modalities.

Abstract: In this work, we present the first systematic evaluation of catastrophic forgetting and modality inequivalence in speech large language models, showing that introducing speech capabilities can degrade knowledge and reasoning even when inputs remain textual, and performance further decreases with spoken queries. To address these challenges, we propose a cross-modal knowledge distillation framework that leverages both text-to-text and speech-to-text channels to transfer knowledge from a text-based teacher model to a speech LLM. Extensive experiments on dialogue and audio understanding tasks validate the effectiveness of our approach in preserving textual knowledge, improving cross-modal alignment, and enhancing reasoning in speech-based interactions.

Method: Rigorous experimental design using balanced Persian datasets (900 texts for sentiment analysis, 1,800 for emotion detection) with consistent prompts, uniform processing parameters, and analysis of precision, recall, F1-scores, and misclassification patterns.

Result: All models achieved acceptable performance levels with no significant statistical differences among top three models. GPT-4o showed marginally higher accuracy, Gemini 2.0 Flash was most cost-efficient. Emotion detection proved more challenging than sentiment analysis for all models.

Conclusion: Establishes performance benchmarks for Persian NLP applications, provides practical model selection guidance based on accuracy/efficiency/cost, and reveals cultural/linguistic challenges for multilingual AI deployment.

Abstract: This study presents a comprehensive comparative evaluation of four state-of-the-art Large Language Models (LLMs)–Claude 3.7 Sonnet, DeepSeek-V3, Gemini 2.0 Flash, and GPT-4o–for sentiment analysis and emotion detection in Persian social media texts. Comparative analysis among LLMs has witnessed a significant rise in recent years, however, most of these analyses have been conducted on English language tasks, creating gaps in understanding cross-linguistic performance patterns. This research addresses these gaps through rigorous experimental design using balanced Persian datasets containing 900 texts for sentiment analysis (positive, negative, neutral) and 1,800 texts for emotion detection (anger, fear, happiness, hate, sadness, surprise). The main focus was to allow for a direct and fair comparison among different models, by using consistent prompts, uniform processing parameters, and by analyzing the performance metrics such as precision, recall, F1-scores, along with misclassification patterns. The results show that all models reach an acceptable level of performance, and a statistical comparison of the best three models indicates no significant differences among them. However, GPT-4o demonstrated a marginally higher raw accuracy value for both tasks, while Gemini 2.0 Flash proved to be the most cost-efficient. The findings indicate that the emotion detection task is more challenging for all models compared to the sentiment analysis task, and the misclassification patterns can represent some challenges in Persian language texts. These findings establish performance benchmarks for Persian NLP applications and offer practical guidance for model selection based on accuracy, efficiency, and cost considerations, while revealing cultural and linguistic challenges that require consideration in multilingual AI system deployment.

Method: Researchers generated two synthetic datasets (10k implicit and 10k explicit verbalizations) and tested three pre-trained LLMs (LLaMA 2.3, DeepSeekV1, Phi1.5). They used LoRA fine-tuning to analyze performance improvements on implicit reasoning tasks.

Result: Fine-tuning LLMs with LoRA significantly improves their ability to extract information from implicit texts, enhancing both performance and model interpretability.

Conclusion: LoRA fine-tuning effectively enhances LLMs’ capability to handle implicit reasoning in information extraction tasks, contributing to better model reliability and understanding of internal reasoning processes.

Abstract: Text Implicitness has always been challenging in Natural Language Processing (NLP), with traditional methods relying on explicit statements to identify entities and their relationships. From the sentence “Zuhdi attends church every Sunday”, the relationship between Zuhdi and Christianity is evident for a human reader, but it presents a challenge when it must be inferred automatically. Large language models (LLMs) have proven effective in NLP downstream tasks such as text comprehension and information extraction (IE). This study examines how textual implicitness affects IE tasks in pre-trained LLMs: LLaMA 2.3, DeepSeekV1, and Phi1.5. We generate two synthetic datasets of 10k implicit and explicit verbalization of biographic information to measure the impact on LLM performance and analyze whether fine-tuning implicit data improves their ability to generalize in implicit reasoning tasks. This research presents an experiment on the internal reasoning processes of LLMs in IE, particularly in dealing with implicit and explicit contexts. The results demonstrate that fine-tuning LLM models with LoRA (low-rank adaptation) improves their performance in extracting information from implicit texts, contributing to better model interpretability and reliability.

Method: Analyzed different tokenization strategies for the space following “Answer:” in multiple-choice question answering prompts, comparing performance across LLMs.

Result: Found accuracy differences up to 11% due to tokenization variations, with one specific strategy (tokenizing space with answer letter) providing consistent performance improvements and better model calibration.

Conclusion: Evaluation design details matter significantly, and standardized, transparent evaluation protocols are needed for reliable and comparable LLM benchmarking results.

Abstract: When evaluating large language models (LLMs) with multiple-choice question answering (MCQA), it is common to end the prompt with the string “Answer:” to facilitate automated answer extraction via next-token probabilities. However, there is no consensus on how to tokenize the space following the colon, often overlooked as a trivial choice. In this paper, we uncover accuracy differences of up to 11% due to this (seemingly irrelevant) tokenization variation as well as reshuffled model rankings, raising concerns about the reliability of LLM comparisons in prior work. Surprisingly, we are able to recommend one specific strategy – tokenizing the space together with the answer letter – as we observe consistent and statistically significant performance improvements. Additionally, it improves model calibration, enhancing the reliability of the model’s confidence estimates. Our findings underscore the importance of careful evaluation design and highlight the need for standardized, transparent evaluation protocols to ensure reliable and comparable results.

Method: CLEAR evaluation pipeline with 57 metrics mapped to lexical, syntactic, semantic, and pragmatic linguistic levels to analyze LLM-rewritten arguments across multiple corpora.

Result: Models perform argument improvement by shortening texts while increasing average word length and merging sentences, with overall improvement in persuasion and coherence dimensions.

Conclusion: Comprehensive linguistic analysis reveals specific patterns in how LLMs approach argument rewriting, providing insights into their text improvement strategies across multiple linguistic levels.

Abstract: While LLMs have been extensively studied on general text generation tasks, there is less research on text rewriting, a task related to general text generation, and particularly on the behavior of models on this task. In this paper we analyze what changes LLMs make in a text rewriting setting. We focus specifically on argumentative texts and their improvement, a task named Argument Improvement (ArgImp). We present CLEAR: an evaluation pipeline consisting of 57 metrics mapped to four linguistic levels: lexical, syntactic, semantic and pragmatic. This pipeline is used to examine the qualities of LLM-rewritten arguments on a broad set of argumentation corpora and compare the behavior of different LLMs on this task and analyze the behavior of different LLMs on this task in terms of linguistic levels. By taking all four linguistic levels into consideration, we find that the models perform ArgImp by shortening the texts while simultaneously increasing average word length and merging sentences. Overall we note an increase in the persuasion and coherence dimensions.

Method: CurDKV uses CUR matrix decomposition to compute leverage scores for selecting keys and values. This approach approximates the dominant subspace of the attention output softmax(QK^T)V, ensuring retained tokens best preserve the model’s predictive behavior.

Result: CurDKV achieves up to 9.6% higher accuracy than state-of-the-art methods (SnapKV, ChunkKV) under aggressive compression on LLaMA and Mistral models. It also reduces generation latency by up to 40% at high compression while maintaining compatibility with FlashAttention and Grouped Query Attention.

Conclusion: Value-centric KV compression using CUR decomposition is more effective than attention-score-based approaches, providing better accuracy-latency tradeoffs and theoretical guarantees for preserving attention output quality.

Abstract: Key-value (KV) cache compression has emerged as a critical technique for reducing the memory and latency overhead of autoregressive language models during inference. Prior approaches predominantly rely on query-key attention scores to rank and evict cached tokens, assuming that attention intensity correlates with semantic importance. However, this heuristic overlooks the contribution of value vectors, which directly influence the attention output. In this paper, we propose CurDKV, a novel, value-centric KV compression method that selects keys and values based on leverage scores computed from CUR matrix decomposition. Our approach approximates the dominant subspace of the attention output $softmax(QK^T)V$, ensuring that the retained tokens best preserve the model’s predictive behavior. Theoretically, we show that attention score approximation does not guarantee output preservation, and demonstrate that CUR-based selection minimizes end-to-end attention reconstruction loss. Empirically, CurDKV achieves up to 9.6% higher accuracy than state-of-the-art methods like SnapKV and ChunkKV under aggressive compression budgets on LLaMA and Mistral, while maintaining compatibility with FlashAttention and Grouped Query Attention. In addition to improved accuracy, CurDKV reduces generation latency by up to 40% at high compression, offering a practical speed-accuracy tradeoff.

Method: Ontology-guided pipeline using LLMs with document chunking, domain-aware prompting, triple extraction, and dual evaluation (reference-based and LLM-as-a-Judge). Includes creation of first HMA ontology and curated dataset.

Result: Ontology-aligned prompts boost extraction accuracy by 44.2%, cut hallucinations by 22.5%, and improve format conformance by 20.9% over baselines.

Conclusion: TextMine successfully transforms unstructured HMA data into structured knowledge and can adapt to global demining efforts or other domains beyond the Cambodian validation context.

Abstract: Humanitarian Mine Action has generated extensive best-practice knowledge, but much remains locked in unstructured reports. We introduce TextMine, an ontology-guided pipeline that uses Large Language Models to extract knowledge triples from HMA texts. TextMine integrates document chunking, domain-aware prompting, triple extraction, and both reference-based and LLM-as-a-Judge evaluation. We also create the first HMA ontology and a curated dataset of real-world demining reports. Experiments show ontology-aligned prompts boost extraction accuracy by 44.2%, cut hallucinations by 22.5%, and improve format conformance by 20.9% over baselines. While validated on Cambodian reports, TextMine can adapt to global demining efforts or other domains, transforming unstructured data into structured knowledge.

Method: Developed a BERT-based language model using Masked Language Modeling (MLM) technique, trained on a newly constructed Maithili corpus and evaluated through news classification tasks.

Result: maiBERT achieved 87.02% accuracy, outperforming NepBERTa and HindiBERT with 0.13% overall accuracy gain and 5-7% improvement across various classes.

Conclusion: The model successfully addresses the resource gap for Maithili and has been open-sourced on Hugging Face for downstream NLP tasks like sentiment analysis and NER.

Abstract: Natural Language Understanding (NLU) for low-resource languages remains a major challenge in NLP due to the scarcity of high-quality data and language-specific models. Maithili, despite being spoken by millions, lacks adequate computational resources, limiting its inclusion in digital and AI-driven applications. To address this gap, we introducemaiBERT, a BERT-based language model pre-trained specifically for Maithili using the Masked Language Modeling (MLM) technique. Our model is trained on a newly constructed Maithili corpus and evaluated through a news classification task. In our experiments, maiBERT achieved an accuracy of 87.02%, outperforming existing regional models like NepBERTa and HindiBERT, with a 0.13% overall accuracy gain and 5-7% improvement across various classes. We have open-sourced maiBERT on Hugging Face enabling further fine-tuning for downstream tasks such as sentiment analysis and Named Entity Recognition (NER).

Method: Two-component framework: Relation Discoverer (RD) predicts new relations using demonstrations from training data, and Relation Predictor (RP) selects most likely relations. Uses self-correcting inference with three stages: discovery, denoising, and prediction.

Result: Extensive experiments on three OpenRE datasets demonstrate the framework’s effectiveness in automatically predicting new relations without human intervention.

Conclusion: The proposed LLM-based framework successfully addresses the limitations of human-dependent OpenRE approaches and provides an automated solution for relation extraction on unseen relations.

Abstract: The goal of open relation extraction (OpenRE) is to develop an RE model that can generalize to new relations not encountered during training. Existing studies primarily formulate OpenRE as a clustering task. They first cluster all test instances based on the similarity between the instances, and then manually assign a new relation to each cluster. However, their reliance on human annotation limits their practicality. In this paper, we propose an OpenRE framework based on large language models (LLMs), which directly predicts new relations for test instances by leveraging their strong language understanding and generation abilities, without human intervention. Specifically, our framework consists of two core components: (1) a relation discoverer (RD), designed to predict new relations for test instances based on \textit{demonstrations} formed by training instances with known relations; and (2) a relation predictor (RP), used to select the most likely relation for a test instance from $n$ candidate relations, guided by \textit{demonstrations} composed of their instances. To enhance the ability of our framework to predict new relations, we design a self-correcting inference strategy composed of three stages: relation discovery, relation denoising, and relation prediction. In the first stage, we use RD to preliminarily predict new relations for all test instances. Next, we apply RP to select some high-reliability test instances for each new relation from the prediction results of RD through a cross-validation method. During the third stage, we employ RP to re-predict the relations of all test instances based on the demonstrations constructed from these reliable test instances. Extensive experiments on three OpenRE datasets demonstrate the effectiveness of our framework. We release our code at https://github.com/XMUDeepLIT/LLM-OREF.git.

Method: Created benchmark comparing probabilities/surprisal differences for grammatical sentences vs (i) low-frequency grammatical, (ii) ungrammatical, (iii) semantically odd, and (iv) pragmatically odd sentences across 4 models.

Result: No unique surprisal signature for ungrammatical prompts - semantically and pragmatically odd conditions consistently showed higher surprisal than ungrammatical ones.

Conclusion: String probabilities cannot reliably indicate model-internal syntactic knowledge; claims about LLMs distinguishing possible/impossible language require alternative verification methods.

Abstract: A controversial test for Large Language Models concerns the ability to discern possible from impossible language. While some evidence attests to the models’ sensitivity to what crosses the limits of grammatically impossible language, this evidence has been contested on the grounds of the soundness of the testing material. We use model-internal representations to tap directly into the way Large Language Models represent the ‘grammatical-ungrammatical’ distinction. In a novel benchmark, we elicit probabilities from 4 models and compute minimal-pair surprisal differences, juxtaposing probabilities assigned to grammatical sentences to probabilities assigned to (i) lower frequency grammatical sentences, (ii) ungrammatical sentences, (iii) semantically odd sentences, and (iv) pragmatically odd sentences. The prediction is that if string-probabilities can function as proxies for the limits of grammar, the ungrammatical condition will stand out among the conditions that involve linguistic violations, showing a spike in the surprisal rates. Our results do not reveal a unique surprisal signature for ungrammatical prompts, as the semantically and pragmatically odd conditions consistently show higher surprisal. We thus demonstrate that probabilities do not constitute reliable proxies for model-internal representations of syntactic knowledge. Consequently, claims about models being able to distinguish possible from impossible language need verification through a different methodology.

Method: A1 refines arithmetic intensity to identify synchronization bottlenecks, proposes online calibration for asynchronous inference, and designs a three-stage rejection sampling pipeline supporting both sequential and parallel scaling.

Result: Experiments on MATH, AMC23, AIME24, and AIME25 datasets show 56.7x speedup in test-time scaling, 4.14x throughput improvement, accurate rejection-rate control, reduced latency/memory overhead, and no accuracy loss compared to target model scaling alone.

Conclusion: A1 provides an efficient and principled solution for scalable LLM inference, addressing synchronization and memory bottlenecks while maintaining statistical guarantees and performance.

Abstract: Large language models (LLMs) benefit from test-time scaling, but existing methods face significant challenges, including severe synchronization overhead, memory bottlenecks, and latency, especially during speculative decoding with long reasoning chains. We introduce A1 (Asynchronous Test-Time Scaling), a statistically guaranteed adaptive inference framework that addresses these challenges. A1 refines arithmetic intensity to identify synchronization as the dominant bottleneck, proposes an online calibration strategy to enable asynchronous inference, and designs a three-stage rejection sampling pipeline that supports both sequential and parallel scaling. Through experiments on the MATH, AMC23, AIME24, and AIME25 datasets, across various draft-target model families, we demonstrate that A1 achieves a remarkable 56.7x speedup in test-time scaling and a 4.14x improvement in throughput, all while maintaining accurate rejection-rate control, reducing latency and memory overhead, and no accuracy loss compared to using target model scaling alone. These results position A1 as an efficient and principled solution for scalable LLM inference. We have released the code at https://github.com/menik1126/asynchronous-test-time-scaling.

Method: Two-stage framework: Stage 1 uses LLMs to generate synthetic explanations for correct/incorrect labels for preference optimization. Stage 2 refines explanations through cross-model training where weaker models align with stronger ones.

Result: Experiments on HateXplain, Latent Hate, and Implicit Hate benchmarks show up to 13.5% macro-F1 improvement over standard few-shot baselines while using only a fraction of full training data.

Conclusion: SMARTER provides a scalable strategy for low-resource content moderation by leveraging LLMs’ self-improving capabilities for both classification and explanation generation.

Abstract: WARNING: This paper contains examples of offensive materials. Toxic content has become pervasive on social media platforms. We introduce SMARTER, a data-efficient two-stage framework for explainable content moderation using Large Language Models (LLMs). In Stage 1, we leverage LLMs’ own outputs to generate synthetic explanations for both correct and incorrect labels, enabling alignment via preference optimization with minimal human supervision. In Stage 2, we refine explanation quality through cross-model training, allowing weaker models to align stylistically and semantically with stronger ones. Experiments on three benchmark tasks – HateXplain, Latent Hate, and Implicit Hate – demonstrate that SMARTER enables LLMs to achieve up to a 13.5% macro-F1 improvement over standard few-shot baselines while using only a fraction of the full training data. Our framework offers a scalable strategy for low-resource settings by harnessing LLMs’ self-improving capabilities for both classification and explanation.

Method: Convolutional decoding (Conv) uses normalization-based approach to narrow decoding window without hard segmentation, and Rejecting Rule-based Fine-Tuning (R2FT) is a post-hoc training scheme to better align tokens at distant positions from context.

Result: Achieves state-of-the-art results on open-ended generation benchmarks (e.g., AlpacaEval) among diffusion LM baselines with significantly lower step size than previous works.

Conclusion: The proposed methods overcome the long decoding-window bottleneck in diffusion LMs while maintaining speed advantages and improving generation quality, demonstrating both speed and quality improvements over previous approaches.

Abstract: Autoregressive (AR) language models generate text one token at a time, which limits their inference speed. Diffusion-based language models offer a promising alternative, as they can decode multiple tokens in parallel. However, we identify a key bottleneck in current diffusion LMs: the long decoding-window problem, where tokens generated far from the input context often become irrelevant or repetitive. Previous solutions like semi-autoregressive address this issue by splitting windows into blocks, but this sacrifices speed and bidirectionality, eliminating the main advantage of diffusion models. To overcome this, we propose Convolutional decoding (Conv), a normalization-based method that narrows the decoding window without hard segmentation, leading to better fluency and flexibility. Additionally, we introduce Rejecting Rule-based Fine-Tuning (R2FT), a post-hoc training scheme that better aligns tokens at positions far from context. Our methods achieve state-of-the-art results on open-ended generation benchmarks (e.g., AlpacaEval) among diffusion LM baselines, with significantly lower step size than previous works, demonstrating both speed and quality improvements.

Method: Fair-GPTQ adds explicit group-fairness constraints to the quantization objective, guiding rounding operations to reduce biased text generation for protected groups (gender, race, religion, occupational bias).

Result: Preserves at least 90% of baseline accuracy on zero-shot benchmarks, reduces unfairness relative to half-precision models, maintains memory/speed benefits of 4-bit quantization, and performs on par with iterative null-space projection debiasing on racial-stereotype benchmarks.

Conclusion: The approach validates theoretical solutions with group-bias terms, demonstrates applicability for reducing bias during quantization in generative models, and enables analysis of channel- and weight-level contributions to fairness.

Abstract: High memory demands of generative language models have drawn attention to quantization, which reduces computational cost, memory usage, and latency by mapping model weights to lower-precision integers. Approaches such as GPTQ effectively minimize input-weight product errors during quantization; however, recent empirical studies show that they can increase biased outputs and degrade performance on fairness benchmarks, and it remains unclear which specific weights cause this issue. In this work, we draw new links between quantization and model fairness by adding explicit group-fairness constraints to the quantization objective and introduce Fair-GPTQ, the first quantization method explicitly designed to reduce unfairness in large language models. The added constraints guide the learning of the rounding operation toward less-biased text generation for protected groups. Specifically, we focus on stereotype generation involving occupational bias and discriminatory language spanning gender, race, and religion. Fair-GPTQ has minimal impact on performance, preserving at least 90% of baseline accuracy on zero-shot benchmarks, reduces unfairness relative to a half-precision model, and retains the memory and speed benefits of 4-bit quantization. We also compare the performance of Fair-GPTQ with existing debiasing methods and find that it achieves performance on par with the iterative null-space projection debiasing approach on racial-stereotype benchmarks. Overall, the results validate our theoretical solution to the quantization problem with a group-bias term, highlight its applicability for reducing group bias at quantization time in generative models, and demonstrate that our approach can further be used to analyze channel- and weight-level contributions to fairness during quantization.

Method: The study investigates multiple methodologies: visual late-interaction embedding models (ColPali), visual rerankers, hybrid retrieval combining dense retrieval with BM25, textual rerankers, fusion methods like Reciprocal Rank Fusion, and a novel Vision-Language Models-based captioning pipeline.

Result: The VLMs-based captioning pipeline demonstrated significantly reduced embedding storage requirements compared to visual late-interaction techniques while achieving comparable retrieval performance. The analysis also evaluated runtime performance and storage demands.

Conclusion: The paper provides practical guidance for selecting and developing efficient, robust slide retrieval systems for real-world applications, highlighting the trade-offs between different approaches in terms of performance, storage, and computational requirements.

Abstract: Slide decks, serving as digital reports that bridge the gap between presentation slides and written documents, are a prevalent medium for conveying information in both academic and corporate settings. Their multimodal nature, combining text, images, and charts, presents challenges for retrieval-augmented generation systems, where the quality of retrieval directly impacts downstream performance. Traditional approaches to slide retrieval often involve separate indexing of modalities, which can increase complexity and lose contextual information. This paper investigates various methodologies for effective slide retrieval, including visual late-interaction embedding models like ColPali, the use of visual rerankers, and hybrid retrieval techniques that combine dense retrieval with BM25, further enhanced by textual rerankers and fusion methods like Reciprocal Rank Fusion. A novel Vision-Language Models-based captioning pipeline is also evaluated, demonstrating significantly reduced embedding storage requirements compared to visual late-interaction techniques, alongside comparable retrieval performance. Our analysis extends to the practical aspects of these methods, evaluating their runtime performance and storage demands alongside retrieval efficacy, thus offering practical guidance for the selection and development of efficient and robust slide retrieval systems for real-world applications.

Method: Using unstructured ethnicity data from a COVID-19 study with rule-guided LLM prompting to generate interpretable oppression scores across multiple state-of-the-art models.

Result: LLMs with explicit rules can capture nuanced identity-based historical oppression within nations, providing complementary measurement of systemic exclusion.

Conclusion: This framework offers a scalable, cross-cultural tool for understanding oppression in data-driven research and public health, with an open-source benchmark released for reproducibility.

Abstract: Traditional efforts to measure historical structural oppression struggle with cross-national validity due to the unique, locally specified histories of exclusion, colonization, and social status in each country, and often have relied on structured indices that privilege material resources while overlooking lived, identity-based exclusion. We introduce a novel framework for oppression measurement that leverages Large Language Models (LLMs) to generate context-sensitive scores of lived historical disadvantage across diverse geopolitical settings. Using unstructured self-identified ethnicity utterances from a multilingual COVID-19 global study, we design rule-guided prompting strategies that encourage models to produce interpretable, theoretically grounded estimations of oppression. We systematically evaluate these strategies across multiple state-of-the-art LLMs. Our results demonstrate that LLMs, when guided by explicit rules, can capture nuanced forms of identity-based historical oppression within nations. This approach provides a complementary measurement tool that highlights dimensions of systemic exclusion, offering a scalable, cross-cultural lens for understanding how oppression manifests in data-driven research and public health contexts. To support reproducible evaluation, we release an open-sourced benchmark dataset for assessing LLMs on oppression measurement (https://github.com/chattergpt/llm-oppression-benchmark).

Method: Two-component framework: 1) LNE for contamination detection to assess contamination extent, 2) Blocking for disruption operation that adjusts intensity based on detection results to elicit non-memorized responses through greedy decoding.

Result: Efficiently restores greedy decoding performance, achieves strong results on multiple datasets with leakage risks, and provides stable recovery across different models and contamination levels.

Conclusion: LNE-Blocking is the first framework to effectively restore contaminated LLM performance without requiring clean datasets, providing a practical solution for fair benchmarking of contaminated models.

Abstract: The problem of data contamination is now almost inevitable during the development of large language models (LLMs), with the training data commonly integrating those evaluation benchmarks even unintentionally. This problem subsequently makes it hard to benchmark LLMs fairly. Instead of constructing contamination-free datasets (quite hard), we propose a novel framework, \textbf{LNE-Blocking}, to restore model performance prior to contamination on potentially leaked datasets. Our framework consists of two components: contamination detection and disruption operation. For the prompt, the framework first uses the contamination detection method, \textbf{LNE}, to assess the extent of contamination in the model. Based on this, it adjusts the intensity of the disruption operation, \textbf{Blocking}, to elicit non-memorized responses from the model. Our framework is the first to efficiently restore the model’s greedy decoding performance. This comes with a strong performance on multiple datasets with potential leakage risks, and it consistently achieves stable recovery results across different models and varying levels of data contamination. We release the code at https://github.com/RuijieH/LNE-Blocking to facilitate research.

Method: FMA uses a learned hierarchical structure with O(log n) resolution levels where nearby tokens interact at full resolution while distant tokens use progressively coarser learned basis functions, inspired by the Fast Multipole Method from physics.

Result: 1D FMA matches or outperforms efficient attention baselines in language modeling with lower memory use; 2D FMA shows superior performance in vision tasks like classification and segmentation with linear complexity.

Conclusion: FMA provides a principled, physics-inspired approach that enables Transformer scaling to longer sequences and higher resolutions while maintaining accuracy, suitable for language, vision, and multimodal applications.

Abstract: While Transformer networks benefit from a global receptive field, their quadratic cost relative to sequence length restricts their application to long sequences and high-resolution inputs. We introduce Fast Multipole Attention (FMA), a divide-and-conquer mechanism for self-attention inspired by the Fast Multipole Method from n-body physics. FMA reduces the time and memory complexity of self-attention from $\mathcal{O}\left(n^2\right)$ to $\mathcal{O}(n \log n)$ and $\mathcal{O}(n)$ while preserving full-context interactions. FMA contains a learned hierarchy with $\mathcal{O}(\log n)$ levels of resolution. In this hierarchy, nearby tokens interact at full resolution, while distant tokens engage through progressively coarser, learned basis functions. We have developed both 1D and 2D implementations of FMA for language and vision tasks, respectively. On autoregressive and bidirectional language modeling benchmarks, the 1D variant either matches or outperforms leading efficient attention baselines with substantially lower memory use. With linear complexity, the 2D variant demonstrates superior performance over strong vision transformer baselines in classification and semantic segmentation tasks. Our results confirm that the multilevel attention implemented by FMA allows Transformer-based models to scale to much longer sequences and higher-resolution inputs without loss in accuracy. This provides a principled, physics-inspired approach for developing scalable neural networks suitable for language, vision, and multimodal tasks. Our code will be available at https://github.com/epoch98/FMA.

Method: Utilizes GenAI-driven narrative co-creation, text-to-speech conversion, text-to-music generation, and text-to-video technology to transform educational narratives into multi-sensory experiences.

Result: Evaluation covers linguistics of generated stories, text-to-speech conversion quality, and accuracy of generated visuals - specific results not provided in abstract.

Conclusion: The paper demonstrates a comprehensive GenAI-powered education tool that successfully integrates multiple AI generation technologies to enhance storytelling for educational purposes.

Abstract: This paper introduces the concept of an education tool that utilizes Generative Artificial Intelligence (GenAI) to enhance storytelling. We evaluate GenAI-driven narrative co-creation, text-to-speech conversion, text-to-music and text-to-video generation to produce an engaging experience for learners. We describe the co-creation process, the adaptation of narratives into spoken words using text-to-speech models, and the transformation of these narratives into contextually relevant visuals through text-to-video technology. Our evaluation covers the linguistics of the generated stories, the text-to-speech conversion quality, and the accuracy of the generated visuals.

Method: Proposed FG-PRM with comprehensive hallucination taxonomy (6 types), automated LLM-based fine-grained hallucination data generation, and step-level reward modeling for detection and verification tasks.

Result: FG-PRM demonstrates superior performance in fine-grained hallucination detection and output verification, substantially boosting LLM performance on GSM8K and MATH benchmarks.

Conclusion: Fine-grained supervision through FG-PRM enhances LLM reliability and interpretability in mathematical reasoning by providing detailed hallucination detection and mitigation at the step level.

Abstract: Hallucinations in large language models (LLMs) pose significant challenges in tasks requiring complex multi-step reasoning, such as mathematical problem-solving. Existing approaches primarily detect the presence of hallucinations but lack a nuanced understanding of their types and manifestations. In this paper, we first introduce a comprehensive taxonomy that categorizes the common hallucinations in mathematical reasoning tasks into six types. We then propose FG-PRM (Fine-Grained Process Reward Model), an augmented model designed to detect and mitigate hallucinations in a fine-grained, step-level manner. To address the limitations of manually labeling training data, we propose an automated method for generating fine-grained hallucination data using LLMs. Our FG-PRM demonstrates superior performance across two key tasks: 1) Fine-grained hallucination detection: classifying hallucination types for each reasoning step; and 2) Verification: ranking multiple LLM-generated outputs to select the most accurate solution. Our experiments show that FG-PRM excels in fine-grained hallucination detection and substantially boosts the performance of LLMs on GSM8K and MATH benchmarks. These results highlight the benefits of fine-grained supervision in enhancing the reliability and interpretability of LLM reasoning processes.

Method: Created SLDS dataset with 20K Swiss Federal Supreme Court rulings and multilingual headnotes. Fine-tuned Qwen2.5, Llama 3.2, Phi-3.5 models and compared them against larger LLMs (GPT-4o, Claude 3.5 Sonnet, DeepSeek R1) using LLM-as-a-Judge evaluation framework.

Result: Fine-tuned models performed well on lexical similarity metrics, while larger general-purpose LLMs generated more legally accurate and coherent summaries. Reasoning-focused models showed no consistent benefit over factual precision.

Conclusion: Factual precision is more important than deep reasoning for legal summarization tasks. The SLDS dataset is released under CC BY 4.0 to support cross-lingual legal summarization research.

Abstract: Legal research depends on headnotes: concise summaries that help lawyers quickly identify relevant cases. Yet, many court decisions lack them due to the high cost of manual annotation. To address this gap, we introduce the Swiss Landmark Decisions Summarization (SLDS) dataset containing 20K rulings from the Swiss Federal Supreme Court, each with headnotes in German, French, and Italian. SLDS has the potential to significantly improve access to legal information and transform legal research in Switzerland. We fine-tune open models (Qwen2.5, Llama 3.2, Phi-3.5) and compare them to larger general-purpose and reasoning-tuned LLMs, including GPT-4o, Claude 3.5 Sonnet, and the open-source DeepSeek R1. Using an LLM-as-a-Judge framework, we find that fine-tuned models perform well in terms of lexical similarity, while larger models generate more legally accurate and coherent summaries. Interestingly, reasoning-focused models show no consistent benefit, suggesting that factual precision is more important than deep reasoning in this task. We release SLDS under a CC BY 4.0 license to support future research in cross-lingual legal summarization.

Method: Created RACQUET dataset targeting distinct aspects of ambiguity, with a specific subset RACQUET-BIAS designed to analyze how failing to address ambiguity leads to stereotypical, socially biased responses. Conducted evaluations of state-of-the-art large multimodal language models.

Result: Revealed significant limitations and problems of overconfidence in models’ ability to address ambiguity. Models showed particular issues with RACQUET-BIAS subset, demonstrating they often resort to stereotypical responses rather than properly handling uncertainty.

Conclusion: There is an urgent need to equip models with robust strategies to deal with uncertainty without resorting to undesirable stereotypes, as current models struggle with ambiguity resolution that humans handle effortlessly through conversational grounding.

Abstract: Ambiguity resolution is key to effective communication. While humans effortlessly address ambiguity through conversational grounding strategies, the extent to which current language models can emulate these strategies remains unclear. In this work, we examine referential ambiguity in image-based question answering by introducing RACQUET, a carefully curated dataset targeting distinct aspects of ambiguity. Through a series of evaluations, we reveal significant limitations and problems of overconfidence of state-of-the-art large multimodal language models in addressing ambiguity in their responses. The overconfidence issue becomes particularly relevant for RACQUET-BIAS, a subset designed to analyze a critical yet underexplored problem: failing to address ambiguity leads to stereotypical, socially biased responses. Our results underscore the urgency of equipping models with robust strategies to deal with uncertainty without resorting to undesirable stereotypes.

Method: Introduce mGeNTE resource and conduct systematic multilingual evaluation of instruction-following language models on English to Spanish/German/Italian/Greek translation, supplemented with interpretability analyses.

Result: Models can recognize when neutrality is appropriate but cannot consistently produce neutral translations, limiting their practical usability for gender-neutral translation tasks.

Conclusion: Current language models have limitations in producing consistent gender-neutral translations despite recognizing the need, highlighting the need for improved inclusive translation technologies.

Abstract: Avoiding the propagation of undue (binary) gender inferences and default masculine language remains a key challenge towards inclusive multilingual technologies, particularly when translating into languages with extensive gendered morphology. Gender-neutral translation (GNT) represents a linguistic strategy towards fairer communication across languages. However, research on GNT is limited to a few resources and language pairs. To address this gap, we introduce mGeNTE, an expert-curated resource, and use it to conduct the first systematic multilingual evaluation of inclusive translation with state-of-the-art instruction-following language models (LMs). Experiments on en-es/de/it/el reveal that while models can recognize when neutrality is appropriate, they cannot consistently produce neutral translations, limiting their usability. To probe this behavior, we enrich our evaluation with interpretability analyses that identify task-relevant features and offer initial insights into the internal dynamics of LM-based GNT.

Method: Systematic examination of false positive prevalence across different open-source models, datasets of varying difficulty, decoding strategies, and analysis of inference time scaling behavior.

Result: False positives persist across all tested conditions; sampling-based inference scaling doesn’t help; pass@N metric is highly susceptible, indicating lower scaling ceiling than automatic evaluations suggest.

Conclusion: False positive solutions are a widespread issue that affects evaluation reliability and suggests limitations for self-improvement techniques and synthetic data generation in mathematical reasoning.

Abstract: Recent advancements in language models have led to significant improvements in mathematical reasoning across various benchmarks. However, most of these benchmarks rely on automatic evaluation methods that only compare final answers using heuristics, without verifying the underlying reasoning steps. This limitation results in false positive solutions, where models may produce correct final answers but with flawed deduction paths. In this paper, we systematically examine the prevalence of false positive solutions in mathematical problem solving for language models. We analyze the characteristics and extent of this issue across different open-source models, datasets of varying difficulty levels, and decoding strategies. Specifically, we explore how false positives influence the inference time scaling behavior of language models. Our experimental results reveal that: (1) false positive solutions persist across different models, datasets, and decoding methods, (2) sampling-based inference time scaling methods do not alleviate the problem, and (3) the pass@N evaluation metric is more susceptible to false positives, suggesting a significantly lower scaling ceiling than what automatic evaluations indicate. Additionally, we analyze specific instances of false positives and discuss potential limitations in self-improvement techniques and synthetic data generation under such conditions. Our data and code are publicly available at https://github.com/Wloner0809/False-Positives-in-Math.

Method: The authors propose a conceptual framework that reconsiders linguistic evaluation by emphasizing gradient factors like frequencies, context, and function instead of strict symbolic rules.

Result: The analysis suggests that current benchmarks based on symbolic rule compliance may misrepresent LM capabilities, and that failures to obey such rules could indicate better alignment with natural language’s flexible nature.

Conclusion: Researchers should develop new benchmarks and analyses that probe whether LMs capture the rich, context-dependent generalizations that characterize natural languages, rather than focusing on strict rule compliance.

Abstract: Linguistic evaluations of how well LMs generalize to produce or understand language often implicitly take for granted that natural languages are generated by symbolic rules. According to this perspective, grammaticality is determined by whether sentences obey such rules. Interpretation is compositionally generated by syntactic rules operating on meaningful words. Semantic parsing maps sentences into formal logic. Failures of LMs to obey strict rules are presumed to reveal that LMs do not produce or understand language like humans. Here we suggest that LMs’ failures to obey symbolic rules may be a feature rather than a bug, because natural languages are not based on neatly separable, compositional rules. Rather, new utterances are produced and understood by a combination of flexible, interrelated, and context-dependent constructions. Considering gradient factors such as frequencies, context, and function will help us reimagine new benchmarks and analyses to probe whether and how LMs capture the rich, flexible generalizations that comprise natural languages.

Method: Three-component framework: 1) Scout extracts triggers from unlabeled target domain data using corpus-level statistics, 2) Narrator generates domain-aligned sentences conditioned on these triggers, 3) Refiner identifies additional event mentions to ensure high annotation quality.

Result: Outperforms best baseline with average F1 gains of 3-7% in zero-shot/few-shot settings and 4-20% F1 improvement for multilingual generation. Higher trigger hit rate and human evaluation confirm better annotation quality and reduced domain drift.

Conclusion: SNaRe effectively addresses domain drift and label noise in specialized domain event detection through its three-component architecture, demonstrating significant performance improvements across multiple domains and settings.

Abstract: Event Detection (ED) – the task of identifying event mentions from natural language text – is critical for enabling reasoning in highly specialized domains such as biomedicine, law, and epidemiology. Data generation has proven to be effective in broadening its utility to wider applications without requiring expensive expert annotations. However, when existing generation approaches are applied to specialized domains, they struggle with label noise, where annotations are incorrect, and domain drift, characterized by a distributional mismatch between generated sentences and the target domain. To address these issues, we introduce SNaRe, a domain-aware synthetic data generation framework composed of three components: Scout, Narrator, and Refiner. Scout extracts triggers from unlabeled target domain data and curates a high-quality domain-specific trigger list using corpus-level statistics to mitigate domain drift. Narrator, conditioned on these triggers, generates high-quality domain-aligned sentences, and Refiner identifies additional event mentions, ensuring high annotation quality. Experimentation on three diverse domain ED datasets reveals how SNaRe outperforms the best baseline, achieving average F1 gains of 3-7% in the zero-shot/few-shot settings and 4-20% F1 improvement for multilingual generation. Analyzing the generated trigger hit rate and human evaluation substantiates SNaRe’s stronger annotation quality and reduced domain drift.

Method: Compare two LLM dialogue generation methods: single prompt vs dual-prompt (two agents conversing). Use judge LLM to evaluate quality via pairwise comparisons to determine which dialogues are harder to distinguish from human discourse.

Result: Dual-prompt method achieves 2-10 times higher win rate than single-prompt method, though it uses six times more tokens. Results consistent across different LLMs (GPT-4o and Llama 3.3 70B) for both generation and judging.

Conclusion: The dual-prompt method with two conversing agents significantly outperforms single-prompt generation for creating realistic HR interview dialogues, providing a more effective approach for synthetic dialogue data generation despite higher computational cost.

Abstract: Optimizing language models for use in conversational agents requires large quantities of example dialogues. Increasingly, these dialogues are synthetically generated by using powerful large language models (LLMs), especially in domains where obtaining authentic human data is challenging. One such domain is human resources (HR). In this context, we compare two LLM-based dialogue generation methods for producing HR job interviews, and assess which method generates higher-quality dialogues, i.e., those more difficult to distinguish from genuine human discourse. The first method uses a single prompt to generate the complete interview dialogue. The second method uses two agents that converse with each other. To evaluate dialogue quality under each method, we ask a judge LLM to determine whether AI was used for interview generation, using pairwise interview comparisons. We empirically find that, at the expense of a sixfold increase in token count, interviews generated with the dual-prompt method achieve a win rate 2 to 10 times higher than those generated with the single-prompt method. This difference remains consistent regardless of whether GPT-4o or Llama 3.3 70B is used for either interview generation or quality judging.

Method: Adapt representation engineering techniques to extract concept representations via probability weighting without labeled data, efficiently select steering vectors, and develop a projection-based method for precise steering of model predictions.

Result: The method effectively mitigates gender bias in LLMs and also generalizes to racial bias, demonstrating its broad applicability for bias mitigation.

Conclusion: The proposed representation-based approach provides an effective way to measure and manipulate bias in LLMs by directly working with the internal concept representations, offering a more targeted solution than black-box mitigation strategies.

Abstract: Large language models (LLMs) are known to perpetuate stereotypes and exhibit biases. Various strategies have been proposed to mitigate these biases, but most work studies biases as a black-box problem without considering how concepts are represented within the model. We adapt techniques from representation engineering to study how the concept of “gender” is represented within LLMs. We introduce a new method that extracts concept representations via probability weighting without labeled data and efficiently selects a steering vector for measuring and manipulating the model’s representation. We develop a projection-based method that enables precise steering of model predictions and demonstrate its effectiveness in mitigating gender bias in LLMs and show that it also generalizes to racial bias. Our code is available at: https://github.com/hannahxchen/gender-bias-steering

Method: Synthesizes ideas from NLP and Social Sciences to create taxonomies for discussion quality evaluation and conversation facilitation datasets, plus overview of intervention strategies.

Result: Provides a comprehensive framework including new taxonomies for evaluation and facilitation, LLM-oriented roadmap, and good practices for improving online discourse quality.

Conclusion: LLMs enable artificial facilitation agents to not only moderate content but actively improve interaction quality, with promising future directions from both technological and societal perspectives.

Abstract: We present a survey of methods for assessing and enhancing the quality of online discussions, focusing on the potential of LLMs. While online discourses aim, at least in theory, to foster mutual understanding, they often devolve into harmful exchanges, such as hate speech, threatening social cohesion and democratic values. Recent advancements in LLMs enable artificial facilitation agents to not only moderate content, but also actively improve the quality of interactions. Our survey synthesizes ideas from NLP and Social Sciences to provide (a) a new taxonomy on discussion quality evaluation, (b) an overview of intervention and facilitation strategies, (c) along with a new taxonomy of conversation facilitation datasets, (d) an LLM-oriented roadmap of good practices and future research directions, from technological and societal perspectives.

Method: Created CARE - a multilingual resource with 3,490 culturally specific questions and 31.7k responses with human judgments, then used native human cultural preferences in the preference learning process.

Result: A modest amount of high-quality native preferences improves cultural awareness across various LMs, outperforming larger generic preference data. Models with stronger initial cultural performance benefit more from alignment.

Conclusion: Cultural awareness in LMs can be significantly improved through targeted preference tuning using native cultural data, revealing performance gaps among models developed in different regions with varying access to culturally relevant data.

Abstract: Language Models (LMs) are typically tuned with human preferences to produce helpful responses, but the impact of preference tuning on the ability to handle culturally diverse queries remains understudied. In this paper, we systematically analyze how native human cultural preferences can be incorporated into the preference learning process to train more culturally aware LMs. We introduce \textbf{CARE}, a multilingual resource containing 3,490 culturally specific questions and 31.7k responses with human judgments. We demonstrate how a modest amount of high-quality native preferences improves cultural awareness across various LMs, outperforming larger generic preference data. Our analyses reveal that models with stronger initial cultural performance benefit more from alignment, leading to gaps among models developed in different regions with varying access to culturally relevant data. CARE is publicly available at https://github.com/Guochry/CARE.

Method: Introduces Step-by-Step Reading (SSR) family of prompts, culminating in SSR++ which guides models to parse questions with finer granularity, focus attention on critical tokens via repetition, and resolve backward dependencies through iterative re-contextualization.

Result: SSR++ sets new state-of-the-art performance on multiple reasoning benchmarks and analysis confirms it works by directly mitigating semantic misunderstanding in LLMs.

Conclusion: Guiding how a model reads questions is a powerful and efficient method for improving reasoning ability, addressing comprehension failures as a core bottleneck in decoder-only models.

Abstract: Large Language Models (LLMs) often fail on complex reasoning tasks due to flawed question comprehension, not just flawed logic. This paper presents a systematic investigation into these comprehension failures. Our work yields three key insights: (1) the step-by-step principle, effective for calculation, can be migrated to the reading process to enhance comprehension; (2) increasing the proportion of question-related tokens (e.g., via repetition) succeeds by refocusing attention, a mechanism that can be explicitly controlled; and (3) backward dependencies represent a core bottleneck for decoder-only models that persists even with strong methods like Chain-of-Thought. Based on these findings, we introduce the Step-by-Step Reading (SSR) family of prompts. This multi-stage approach culminates in SSR++, a method specifically engineered to deepen model comprehension by guiding it to parse questions with finer granularity, focus attention on critical tokens, and resolve backward dependencies through iterative re-contextualization. SSR++ sets a new state-of-the-art on multiple reasoning benchmarks, and our analysis confirms it works by directly mitigating semantic misunderstanding. These results demonstrate that guiding how a model reads is a powerful and efficient method for improving its reasoning ability.

Method: Extended probabilistic extraction technique to measure memorization of 50 books across 17 open-weight LLMs through thousands of experiments.

Result: Memorization varies significantly by model and book. Most LLMs don’t memorize most books, but Llama 3.1 70B entirely memorizes some books (e.g., Harry Potter, 1984) - can generate entire books near-verbatim from minimal prompts.

Conclusion: Results have significant but nuanced implications for copyright cases, not unambiguously favoring either plaintiffs or defendants, showing memorization is complex and context-dependent.

Abstract: Plaintiffs and defendants in copyright lawsuits over generative AI often make sweeping, opposing claims about the extent to which large language models (LLMs) have memorized plaintiffs’ protected expression in their training data. Drawing on both machine learning and copyright law, we show that these polarized positions dramatically oversimplify the relationship between memorization and copyright. To do so, we extend a recent probabilistic extraction technique to measure memorization of 50 books in 17 open-weight LLMs. Through thousands of experiments, we show that the extent of memorization varies both by model and by book. With respect to our specific extraction methodology, we find that most LLMs do not memorize most books – either in whole or in part. However, we also find that Llama 3.1 70B entirely memorizes some books, like the first Harry Potter book and 1984. In fact, the first Harry Potter is so memorized that, using a seed prompt consisting of just the first few tokens of the first chapter, we can deterministically generate the entire book near-verbatim. We discuss why our results have significant implications for copyright cases, though not ones that unambiguously favor either side.

Method: Introduces Diverse-NS, a length-controlled data selection strategy that generates and filters preference data to balance diversity, quality, and length parity.

Result: Substantially enhances lexical and semantic diversity across four creative generation tasks with minor reduction or gains in response quality. Smaller models (Olmo-2-7B) can effectively teach diversity to larger models.

Conclusion: By explicitly addressing length bias, Diverse-NS efficiently pushes language models toward more diverse and expressive outputs using minimal training data.

Abstract: Diverse language model responses are crucial for creative generation, open-ended tasks, and self-improvement training. We show that common diversity metrics, and even reward models used for preference optimization, systematically bias models toward shorter outputs, limiting expressiveness. To address this, we introduce Diverse, not Short (Diverse-NS), a length-controlled data selection strategy that improves response diversity while maintaining length parity. By generating and filtering preference data that balances diversity, quality, and length, Diverse-NS enables effective training using only 3,000 preference pairs. Applied to LLaMA-3.1-8B and the Olmo-2 family, Diverse-NS substantially enhances lexical and semantic diversity. We show consistent improvement in diversity with minor reduction or gains in response quality on four creative generation tasks: Divergent Associations, Persona Generation, Alternate Uses, and Creative Writing. Surprisingly, experiments with the Olmo-2 model family (7B, and 13B) show that smaller models like Olmo-2-7B can serve as effective “diversity teachers” for larger models. By explicitly addressing length bias, our method efficiently pushes models toward more diverse and expressive outputs.

Method: Uses token-level cross entropy as evaluation signal, identifies low-quality prompt segments via masking analysis, iteratively rewrites them, and selects variants through single forward pass loss minimization.

Result: Outperforms prior methods: highest average accuracy on BBH, strong performance on GSM8K and AQUA RAT, and increases AlpacaEval 2.0 win rates by over 19 points.

Conclusion: PMPO provides an effective, efficient, and broadly applicable prompt optimization framework that eliminates the need for output sampling and human scoring while working across model sizes and task types.

Abstract: Prompt optimization is a practical and widely applicable alternative to fine tuning for improving large language model performance. Yet many existing methods evaluate candidate prompts by sampling full outputs, often coupled with self critique or human annotated preferences, which limits scalability, especially for smaller models or models that are not instruction tuned. We present PMPO (Probabilistic Metric Prompt Optimization), a unified framework that uses token level cross entropy as a direct, lightweight evaluation signal. PMPO locates low quality prompt segments via a masking based analysis and iteratively rewrites them to propose improved variants. Crucially, during evaluation, PMPO selects among variants by minimizing loss in a single forward pass, eliminating output sampling and human or judge based scoring for selection while still using standard generation only to propose rewrites. This unified, loss based strategy supports both supervised and preference based tasks. Across model sizes and datasets, PMPO outperforms prior prompt optimizers: it achieves the highest average accuracy on BBH, performs strongly on GSM8K and AQUA RAT, and raises AlpacaEval 2.0 win rates by over 19 points. These results demonstrate PMPO’s effectiveness, efficiency, and broad applicability.

Method: Combined behavioral benchmarking with neuron-level analysis using Language Confusion Benchmark (LCB), layer-wise analysis with TunedLens, targeted neuron attribution, and comparative analysis with multilingual-tuned models to identify critical neurons.

Result: Transition failures in final layers drive confusion; editing a small set of critical neurons substantially mitigates confusion while preserving general competence and fluency, matching multilingual alignment performance for many languages.

Conclusion: Neuron-level interventions are a promising direction for robust, interpretable multilingual language modeling, providing new insights into LLM internal dynamics.

Abstract: Language confusion – where large language models (LLMs) generate unintended languages against the user’s need – remains a critical challenge, especially for English-centric models. We present the first mechanistic interpretability (MI) study of language confusion, combining behavioral benchmarking with neuron-level analysis. Using the Language Confusion Benchmark (LCB), we show that confusion points (CPs) – specific positions where language switches occur – are central to this phenomenon. Through layer-wise analysis with TunedLens and targeted neuron attribution, we reveal that transition failures in the final layers drive confusion. We further demonstrate that editing a small set of critical neurons, identified via comparative analysis with a multilingual-tuned counterpart, substantially mitigates confusion while largely preserving general competence and fluency. Our approach matches multilingual alignment in confusion reduction for many languages and yields cleaner, higher-quality outputs. These findings provide new insights into the internal dynamics of LLMs and highlight neuron-level interventions as a promising direction for robust, interpretable multilingual language modeling. Code and data are available at: https://github.com/ercong21/lang_confusion.

Method: A three-stage framework that incorporates an additional unbiased assistant model to complement training data, designed to debias from both labels and feedbacks in the training data.

Result: Extensive experiments show AGDe-Judge effectively reduces teacher preference bias while maintaining strong performance across six evaluation benchmarks.

Conclusion: The proposed framework successfully addresses teacher preference bias in LLM-as-a-Judge systems, improving evaluation fairness without compromising performance.

Abstract: LLM-as-a-Judge employs large language models (LLMs), such as GPT-4, to evaluate the quality of LLM-generated responses, gaining popularity for its cost-effectiveness and strong alignment with human evaluations. However, training proxy judge models using evaluation data generated by powerful teacher models introduces a critical yet previously overlooked issue: teacher preference bias, where the proxy judge model learns a biased preference for responses from the teacher model. To tackle this problem, we propose a novel setting that incorporates an additional assistant model, which is not biased toward the teacher model’s responses, to complement the training data. Building on this setup, we introduce AGDe-Judge, a three-stage framework designed to debias from both the labels and feedbacks in the training data. Extensive experiments demonstrate that AGDe-Judge effectively reduces teacher preference bias while maintaining strong performance across six evaluation benchmarks. Code is available at https://github.com/Liuz233/AGDe-Judge.

Method: Leverages Large Language Models with schema-driven methodology to process documents across multiple formats, incorporating robust validation mechanisms and evaluating context length, few-shot learning, and web browsing integration.

Result: Modern LLMs show promising results in automating metadata extraction, though further improvements are needed for consistent and reliable performance.

Conclusion: The framework demonstrates the potential of LLMs for automated metadata extraction from scientific papers, with released code and dataset to support future research in this area.

Abstract: Metadata extraction is essential for cataloging and preserving datasets, enabling effective research discovery and reproducibility, especially given the current exponential growth in scientific research. While Masader (Alyafeai et al.,2021) laid the groundwork for extracting a wide range of metadata attributes from Arabic NLP datasets’ scholarly articles, it relies heavily on manual annotation. In this paper, we present MOLE, a framework that leverages Large Language Models (LLMs) to automatically extract metadata attributes from scientific papers covering datasets of languages other than Arabic. Our schema-driven methodology processes entire documents across multiple input formats and incorporates robust validation mechanisms for consistent output. Additionally, we introduce a new benchmark to evaluate the research progress on this task. Through systematic analysis of context length, few-shot learning, and web browsing integration, we demonstrate that modern LLMs show promising results in automating this task, highlighting the need for further future work improvements to ensure consistent and reliable performance. We release the code: https://github.com/IVUL-KAUST/MOLE and dataset: https://huggingface.co/datasets/IVUL-KAUST/MOLE for the research community.

Method: Curated trajectory data by reconstructing agent’s inference-time reasoning algorithms into chain-of-thought rationales, then distilled these reasoning patterns into backbone LLM via simple fine-tuning.

Result: Substantial performance improvements across multiple benchmarks including WebVoyager, Mind2web-live, and SimpleQA (web search).

Conclusion: Targeted reasoning skill enhancement through fine-tuning with reconstructed reasoning patterns shows significant potential for improving web agent capabilities.

Abstract: Web agents powered by Large Language Models (LLMs) show promise for next-generation AI, but their limited reasoning in uncertain, dynamic web environments hinders robust deployment. In this paper, we identify key reasoning skills essential for effective web agents, i.e., reflection & lookahead, branching, and rollback, and curate trajectory data that exemplifies these abilities by reconstructing the agent’s (inference-time) reasoning algorithms into chain-of-thought rationales. We conduct experiments in the agent self-improving benchmark, OpenWebVoyager, and demonstrate that distilling salient reasoning patterns into the backbone LLM via simple fine-tuning can substantially enhance its performance. Our approach yields significant improvements across multiple benchmarks, including WebVoyager, Mind2web-live, and SimpleQA (web search), highlighting the potential of targeted reasoning skill enhancement for web agents.

Method: Synthesizes symbolic reasoning trajectories with stepwise pseudo-labels using Monte Carlo estimation. Trains a Process Reward Model (PRM) to select symbolic trajectories, then uses Direct Preference Optimization (DPO) and Supervised Fine-Tuning (SFT) for improvement.

Result: Achieves gains on FOLIO and LogicAsker benchmarks for both frontier and open-weight models. Enhances out-of-domain generalizability on claim verification data, showing improved planning and logical reasoning capabilities.

Conclusion: The proposed method effectively enhances LLM reasoning by leveraging synthesized symbolic trajectories, demonstrating improved generalization and logical reasoning across multiple domains and model types.

Abstract: Large language models (LLMs) have shown strong performance in many reasoning benchmarks. However, recent studies have pointed to memorization, rather than generalization, as one of the leading causes for such performance. LLMs, in fact, are susceptible to content variations, demonstrating a lack of robust planning or symbolic abstractions supporting their reasoning process. To improve reliability, many attempts have been made to combine LLMs with symbolic methods. Nevertheless, existing approaches fail to effectively leverage symbolic representations due to the challenges involved in developing reliable and scalable verification mechanisms. In this paper, we propose to overcome such limitations by synthesizing high-quality symbolic reasoning trajectories with stepwise pseudo-labels at scale via Monte Carlo estimation. A Process Reward Model (PRM) can be efficiently trained based on the synthesized data and then used to select more symbolic trajectories. The trajectories are then employed with Direct Preference Optimization (DPO) and Supervised Fine-Tuning (SFT) to improve logical reasoning and generalization. Our results on benchmarks (i.e., FOLIO and LogicAsker) show the effectiveness of the proposed method with gains on frontier and open-weight models. Moreover, additional experiments on claim verification data reveal that fine-tuning on the generated symbolic reasoning trajectories enhances out-of-domain generalizability, suggesting the potential impact of the proposed method in enhancing planning and logical reasoning.

Method: Fine-tuning smaller language models for text classification (mdok approach) applied to both binary detection and multiclass classification of human-AI collaboration cases.

Result: Achieved 1st rank performance in both subtasks of Voight-Kampff Generative AI Detection 2025, demonstrating remarkable detection capabilities.

Conclusion: The mdok approach provides a robust solution for detecting machine-generated texts, effectively addressing the challenge of out-of-distribution data in AI-generated content detection.

Abstract: The large language models (LLMs) are able to generate high-quality texts in multiple languages. Such texts are often not recognizable by humans as generated, and therefore present a potential of LLMs for misuse (e.g., plagiarism, spams, disinformation spreading). An automated detection is able to assist humans to indicate the machine-generated texts; however, its robustness to out-of-distribution data is still challenging. This notebook describes our mdok approach in robust detection, based on fine-tuning smaller LLMs for text classification. It is applied to both subtasks of Voight-Kampff Generative AI Detection 2025, providing remarkable performance (1st rank) in both, the binary detection as well as the multiclass classification of various cases of human-AI collaboration.

Method: Divides pretrained decoder-only language models into specialized layer groups, uses a two-stage inference process with same model parameters for both retrieval and generation, and employs generation perplexities as retrieval training objectives.

Result: Achieves 3.6-11.5 improvements in exact match scores on 8 knowledge-intensive tasks, particularly on unseen tasks with diverse formats.

Conclusion: The approach effectively enables models to articulate their own information needs and generalize across tasks, with balanced retrieval-generation parameters and perplexity-based training being key to enhanced performance.

Abstract: Retrieval-Augmented Generation (RAG) systems traditionally treat retrieval and generation as separate processes, requiring explicit textual queries to connect them. This separation can limit the ability of models to generalize across diverse tasks. In this work, we propose a query-free RAG system, named ImpRAG, which integrates retrieval and generation into a unified model. ImpRAG allows models to implicitly express their information needs, eliminating the need for human-specified queries. By dividing pretrained decoder-only language models into specialized layer groups, ImpRAG optimizes retrieval and generation tasks simultaneously. Our approach employs a two-stage inference process, using the same model parameters and forward pass for both retrieval and generation, thereby minimizing the disparity between retrievers and language models. Experiments on 8 knowledge-intensive tasks demonstrate that ImpRAG achieves 3.6-11.5 improvements in exact match scores on unseen tasks with diverse formats, highlighting its effectiveness in enabling models to articulate their own information needs and generalize across tasks. Our analysis underscores the importance of balancing retrieval and generation parameters and leveraging generation perplexities as retrieval training objectives for enhanced performance.

Method: DiCoRe framework with Dreamer (divergent reasoning for event discovery), Grounder (convergent reasoning with constrained decoding), and LLM-Judge for verification.

Result: Outperforms prior zero-shot, transfer-learning, and reasoning baselines across 6 datasets in 5 domains with 9 LLMs, achieving 4-7% average F1 gains.

Conclusion: DiCoRe establishes a strong zero-shot event detection framework that effectively balances event coverage and precision through divergent-convergent reasoning.

Abstract: Zero-shot Event Detection (ED), the task of identifying event mentions in natural language text without any training data, is critical for document understanding in specialized domains. Understanding the complex event ontology, extracting domain-specific triggers from the passage, and structuring them appropriately overloads and limits the utility of Large Language Models (LLMs) for zero-shot ED. To this end, we propose DiCoRe, a divergent-convergent reasoning framework that decouples the task of ED using Dreamer and Grounder. Dreamer encourages divergent reasoning through open-ended event discovery, which helps to boost event coverage. Conversely, Grounder introduces convergent reasoning to align the free-form predictions with the task-specific instructions using finite-state machine guided constrained decoding. Additionally, an LLM-Judge verifies the final outputs to ensure high precision. Through extensive experiments on six datasets across five domains and nine LLMs, we demonstrate how DiCoRe consistently outperforms prior zero-shot, transfer-learning, and reasoning baselines, achieving 4-7% average F1 gains over the best baseline – establishing DiCoRe as a strong zero-shot ED framework.

Method: Uses a draft, critique, and revise pipeline with symbolic evaluation against interpretable rubrics during GRPO training, decomposing rewards into principle-specific evaluations through natural language programs.

Result: Reduces attack success rates by up to 68.7% while maintaining 0.67% false refusal rate, achieving Pareto optimal safety-helpfulness performance and outperforming both DPO and GRPO with state-of-the-art reward models.

Conclusion: Making reward signals interpretable and modular improves alignment effectiveness, suggesting transparency enhances LLM safety.

Abstract: Alignment of large language models (LLMs) with principles like helpfulness, honesty, and harmlessness typically relies on scalar rewards that obscure which objectives drive the training signal. We introduce QA-LIGN, which decomposes monolithic rewards into interpretable principle-specific evaluations through structured natural language programs. Models learn through a draft, critique, and revise pipeline, where symbolic evaluation against the rubrics provides transparent feedback for both initial and revised responses during GRPO training. Applied to uncensored Llama-3.1-8B-Instruct, QA-LIGN reduces attack success rates by up to 68.7% while maintaining a 0.67% false refusal rate, achieving Pareto optimal safety-helpfulness performance and outperforming both DPO and GRPO with state-of-the-art reward models given equivalent training. These results demonstrate that making reward signals interpretable and modular improves alignment effectiveness, suggesting transparency enhances LLM safety.

Method: Filtered large-scale conversational AI datasets to create HealthChat-11K, then used clinician-driven taxonomy to analyze user interactions across 21 health specialties, examining patterns like incomplete context, affective behaviors, and sycophancy-inducing questions.

Result: Revealed insights into how and why users seek health information from LLMs, including common interaction patterns, instances where users provide incomplete context, emotional behaviors, and interactions that can lead to sycophantic responses from AI systems.

Conclusion: The findings underscore the need for improvements in healthcare support capabilities of LLMs deployed as conversational AI, highlighting specific areas where current systems fall short in handling health-related queries effectively and safely.

Abstract: People are increasingly seeking healthcare information from large language models (LLMs) via interactive chatbots, yet the nature and inherent risks of these conversations remain largely unexplored. In this paper, we filter large-scale conversational AI datasets to achieve HealthChat-11K, a curated dataset of 11K real-world conversations composed of 25K user messages. We use HealthChat-11K and a clinician-driven taxonomy for how users interact with LLMs when seeking healthcare information in order to systematically study user interactions across 21 distinct health specialties. Our analysis reveals insights into the nature of how and why users seek health information, such as common interactions, instances of incomplete context, affective behaviors, and interactions (e.g., leading questions) that can induce sycophancy, underscoring the need for improvements in the healthcare support capabilities of LLMs deployed as conversational AI. Code and artifacts to retrieve our analyses and combine them into a curated dataset can be found here: https://github.com/yahskapar/HealthChat

Method: Proposes MedVAL, a self-supervised distillation method using synthetic data to train evaluator LMs to assess factual consistency of medical outputs without requiring physician labels or reference outputs.

Result: MedVAL significantly improves alignment with physicians across 10 state-of-the-art LMs, increasing average F1 scores from 66% to 83%, and improves GPT-4o by 8% without physician-labeled training data.

Conclusion: MedVAL demonstrates LMs approaching expert-level ability in validating AI-generated medical text, providing a scalable, risk-aware pathway for clinical integration with open-sourced code, benchmark, and model.

Abstract: With the growing use of language models (LMs) in clinical environments, there is an immediate need to evaluate the accuracy and safety of LM-generated medical text. Currently, such evaluation relies solely on manual physician review. However, detecting errors in LM-generated text is challenging because

1. manual review is costly and 2) expert-composed reference outputs are often unavailable in real-world settings. While the “LM-as-judge” paradigm (a LM evaluating another LM) offers scalable evaluation, even frontier LMs can miss subtle but clinically significant errors. To address these challenges, we propose MedVAL, a novel, self-supervised, data-efficient distillation method that leverages synthetic data to train evaluator LMs to assess whether LM-generated medical outputs are factually consistent with inputs, without requiring physician labels or reference outputs. To evaluate LM performance, we introduce MedVAL-Bench, a dataset of 840 physician-annotated outputs across 6 diverse medical tasks capturing real-world challenges. Across 10 state-of-the-art LMs spanning open-source and proprietary models, MedVAL distillation significantly improves (p < 0.001) alignment with physicians across seen and unseen tasks, increasing average F1 scores from 66% to 83%. Despite strong baseline performance, MedVAL improves the best-performing proprietary LM (GPT-4o) by 8% without training on physician-labeled data, demonstrating a performance statistically non-inferior to a single human expert (p < 0.001). To support a scalable, risk-aware pathway towards clinical integration, we open-source: 1) Codebase (https://github.com/StanfordMIMI/MedVAL), 2) MedVAL-Bench (https://huggingface.co/datasets/stanfordmimi/MedVAL-Bench), 3) MedVAL-4B (https://huggingface.co/stanfordmimi/MedVAL-4B). Our benchmark provides evidence of LMs approaching expert-level ability in validating AI-generated medical text.

Method: Aligns simulated students with instructed ability using DPO, forms preference pairs based on IRT likelihood, generates thousands of responses, scores with LLM, and fits IRT model for difficulty estimates.

Result: SMART outperforms other item difficulty prediction methods in experiments on two real-world student response datasets, demonstrating improved ability alignment.

Conclusion: The SMART framework provides an effective solution for estimating item difficulties without real student data, enabling cold-start applications and reducing assessment costs.

Abstract: Item (question) difficulties play a crucial role in educational assessments, enabling accurate and efficient assessment of student abilities and personalization to maximize learning outcomes. Traditionally, estimating item difficulties can be costly, requiring real students to respond to items, followed by fitting an item response theory (IRT) model to get difficulty estimates. This approach cannot be applied to the cold-start setting for previously unseen items either. In this work, we present SMART (Simulated Students Aligned with IRT), a novel method for aligning simulated students with instructed ability, which can then be used in simulations to predict the difficulty of open-ended items. We achieve this alignment using direct preference optimization (DPO), where we form preference pairs based on how likely responses are under a ground-truth IRT model. We perform a simulation by generating thousands of responses, evaluating them with a large language model (LLM)-based scoring model, and fit the resulting data to an IRT model to obtain item difficulty estimates. Through extensive experiments on two real-world student response datasets, we show that SMART outperforms other item difficulty prediction methods by leveraging its improved ability alignment.

Method: Using eRST framework, proposed a graded definition of DM polysemy, and conducted correlation and regression analyses to examine polysemous DMs’ co-occurrence with non-DM signals.

Result: Polysemous DMs co-occur with more diverse non-DM signals but not necessarily more total signals; genre significantly shapes DM-signal interactions.

Conclusion: The study reveals nuanced relationships between DM polysemy and non-DM signal co-occurrence, highlighting genre’s important role in discourse signal interactions.

Abstract: Discourse markers (DMs) like ‘but’ or ’then’ are crucial for creating coherence in discourse, yet they are often replaced by or co-occur with non-DMs (‘in the morning’ can mean the same as ’then’), and both can be ambiguous (‘since’ can refer to time or cause). The interaction mechanism between such signals remains unclear but pivotal for their disambiguation. In this paper we investigate the relationship between DM polysemy and co-occurrence of non-DM signals in English, as well as the influence of genre on these patterns. Using the framework of eRST, we propose a graded definition of DM polysemy, and conduct correlation and regression analyses to examine whether polysemous DMs are accompanied by more numerous and diverse non-DM signals. Our findings reveal that while polysemous DMs do co-occur with more diverse non-DMs, the total number of co-occurring signals does not necessarily increase. Moreover, genre plays a significant role in shaping DM-signal interactions.

Method: The authors introduce Adaptive Self-Correction Chain-of-Thought (ASCoT) with two components: Adaptive Verification Manager (AVM) that uses Positional Impact Score function I(k) to prioritize high-risk late-stage steps, and Multi-Perspective Self-Correction Engine (MSCE) that applies dual-path correction to identified failure parts.

Result: Extensive experiments on GSM8K and MATH benchmarks show ASCoT achieves outstanding accuracy, outperforming strong baselines including standard CoT. The method demonstrates that late-stage errors are significantly more likely to corrupt final answers than identical early-stage errors.

Conclusion: The work underscores the importance of diagnosing specific failure modes in LLM reasoning and advocates for a shift from uniform verification strategies to adaptive, vulnerability-aware correction mechanisms that specifically address Late-Stage Fragility.

Abstract: Chain-of-Thought (CoT) prompting has significantly advanced the reasoning capabilities of Large Language Models (LLMs), yet the reliability of these reasoning chains remains a critical challenge. A widely held “cascading failure” hypothesis suggests that errors are most detrimental when they occur early in the reasoning process. This paper challenges that assumption through systematic error-injection experiments, revealing a counter-intuitive phenomenon we term “Late-Stage Fragility”: errors introduced in the later stages of a CoT chain are significantly more likely to corrupt the final answer than identical errors made at the beginning. To address this specific vulnerability, we introduce the Adaptive Self-Correction Chain-of-Thought (ASCoT) method. ASCoT employs a modular pipeline in which an Adaptive Verification Manager (AVM) operates first, followed by the Multi-Perspective Self-Correction Engine (MSCE). The AVM leverages a Positional Impact Score function I(k) that assigns different weights based on the position within the reasoning chains, addressing the Late-Stage Fragility issue by identifying and prioritizing high-risk, late-stage steps. Once these critical steps are identified, the MSCE applies robust, dual-path correction specifically to the failure parts. Extensive experiments on benchmarks such as GSM8K and MATH demonstrate that ASCoT achieves outstanding accuracy, outperforming strong baselines, including standard CoT. Our work underscores the importance of diagnosing specific failure modes in LLM reasoning and advocates for a shift from uniform verification strategies to adaptive, vulnerability-aware correction mechanisms.

Method: Internal-external knowledge self-selection strategy, selective supervised fine-tuning, structured reasoning data generation pipeline, and GRPO integration for enhanced reasoning.

Result: Outperforms existing methods on medical, legal, and financial QA benchmarks, matching domain-pretrained LLMs with significantly lower computational cost.

Conclusion: S2K provides an effective progressive learning approach that mirrors human knowledge acquisition, offering superior domain performance without expensive pretraining.

Abstract: Large Language Models (LLMs) perform well in general QA but often struggle in domain-specific scenarios. Retrieval-Augmented Generation (RAG) introduces external knowledge but suffers from hallucinations and latency due to noisy retrievals. Continued pretraining internalizes domain knowledge but is costly and lacks cross-domain flexibility. We attribute this challenge to the long-tail distribution of domain knowledge, which leaves partial yet useful internal knowledge underutilized. We further argue that knowledge acquisition should be progressive, mirroring human learning: first understanding concepts, then applying them to complex reasoning. To address this, we propose Selct2Know (S2K), a cost-effective framework that internalizes domain knowledge through an internal-external knowledge self-selection strategy and selective supervised fine-tuning. We also introduce a structured reasoning data generation pipeline and integrate GRPO to enhance reasoning ability. Experiments on medical, legal, and financial QA benchmarks show that S2K consistently outperforms existing methods and matches domain-pretrained LLMs with significantly lower cost.

Method: Agentic brainstorming approach using multiple LLMs as thought agents to generate refined question-answer pairs, plus Agentic Choice Enhancement (ACE) module for post-training reasoning improvement.

Result: Created high-quality MovieCORE dataset with cognitive tests for depth assessment, and ACE module improved VQA model reasoning capabilities by up to 25%.

Conclusion: Advances movie understanding in AI systems and provides insights into current VQA models’ capabilities and limitations on challenging cinematic content questions.

Abstract: This paper introduces MovieCORE, a novel video question answering (VQA) dataset designed to probe deeper cognitive understanding of movie content. Unlike existing datasets that focus on surface-level comprehension, MovieCORE emphasizes questions that engage System-2 thinking while remaining specific to the video material. We present an innovative agentic brainstorming approach, utilizing multiple large language models (LLMs) as thought agents to generate and refine high-quality question-answer pairs. To evaluate dataset quality, we develop a set of cognitive tests assessing depth, thought-provocation potential, and syntactic complexity. We also propose a comprehensive evaluation scheme for assessing VQA model performance on deeper cognitive tasks. To address the limitations of existing video-language models (VLMs), we introduce an agentic enhancement module, Agentic Choice Enhancement (ACE), which improves model reasoning capabilities post-training by up to 25%. Our work contributes to advancing movie understanding in AI systems and provides valuable insights into the capabilities and limitations of current VQA models when faced with more challenging, nuanced questions about cinematic content. Our project page, dataset and code can be found at https://joslefaure.github.io/assets/html/moviecore.html.

Method: A closed-loop optimization system with: 1) self-referential diagnostic module using tri-axial model signals (loss patterns, embedding clusters, self-alignment scores), 2) adaptive optimization engine that transforms suboptimal samples while preserving semantics, 3) dynamic learning principles for continuous evolution.

Result: Experiments show Middo consistently enhances seed data quality and boosts LLM performance with 7.15% average accuracy improvement while maintaining original dataset scale.

Conclusion: Establishes a new paradigm for sustainable LLM training through dynamic human-AI co-evolution of data and models, enabling continuous improvement of training data quality.

Abstract: Supervised Fine-Tuning (SFT) Large Language Models (LLM) fundamentally rely on high-quality training data. While data selection and data synthesis are two common strategies to improve data quality, existing approaches often face limitations in static dataset curation that fail to adapt to evolving model capabilities. In this paper, we introduce Middo, a self-evolving Model-informed dynamic data optimization framework that uses model-aware data selection and context-preserving data refinement. Unlike conventional one-off filtering/synthesis methods, our framework establishes a closed-loop optimization system: (1) A self-referential diagnostic module proactively identifies suboptimal samples through tri-axial model signals - loss patterns (complexity), embedding cluster dynamics (diversity), and self-alignment scores (quality); (2) An adaptive optimization engine then transforms suboptimal samples into pedagogically valuable training points while preserving semantic integrity; (3) This optimization process continuously evolves with model capability through dynamic learning principles. Experiments on multiple benchmarks demonstrate that our Middo consistently enhances the quality of seed data and boosts LLM’s performance with improving accuracy by 7.15% on average while maintaining the original dataset scale. This work establishes a new paradigm for sustainable LLM training through dynamic human-AI co-evolution of data and models. Our datasets, models, and code are coming soon. Our datasets, models, and code are publicly available at https://github.com/Word2VecT/Middo

Method: Comprehensive review of research applying RL to LLMs and LRMs for reasoning abilities, analyzing foundational components, core problems, training resources, and applications.

Result: Identifies key challenges in computational resources, algorithm design, training data, and infrastructure for scaling RL in LRMs.

Conclusion: Provides roadmap for future research opportunities to enhance RL scalability for broader reasoning models and ASI development.

Abstract: In this paper, we survey recent advances in Reinforcement Learning (RL) for reasoning with Large Language Models (LLMs). RL has achieved remarkable success in advancing the frontier of LLM capabilities, particularly in addressing complex logical tasks such as mathematics and coding. As a result, RL has emerged as a foundational methodology for transforming LLMs into LRMs. With the rapid progress of the field, further scaling of RL for LRMs now faces foundational challenges not only in computational resources but also in algorithm design, training data, and infrastructure. To this end, it is timely to revisit the development of this domain, reassess its trajectory, and explore strategies to enhance the scalability of RL toward Artificial SuperIntelligence (ASI). In particular, we examine research applying RL to LLMs and LRMs for reasoning abilities, especially since the release of DeepSeek-R1, including foundational components, core problems, training resources, and downstream applications, to identify future opportunities and directions for this rapidly evolving area. We hope this review will promote future research on RL for broader reasoning models. Github: https://github.com/TsinghuaC3I/Awesome-RL-for-LRMs

Method: Analysis of Latent Indicators to Generate Nomological Structures (ALIGNS) - a large language model-based system trained with validated questionnaire measures that provides three comprehensive nomological networks containing over 550,000 indicators across multiple fields.

Result: ALIGNS demonstrated: 1) NIH PROMIS anxiety and depression instruments converge into a single emotional distress dimension, 2) child temperament measures reveal four new potential dimensions and question one existing dimension, 3) expert psychometricians validated the system’s importance, accessibility, and suitability.

Conclusion: ALIGNS represents the first application of large language models to solve foundational measurement validation problems, complementing traditional methods with large-scale nomological analysis and being freely available for use.

Abstract: Psychological measurement is critical to many disciplines. Despite advances in measurement, building nomological networks, theoretical maps of how concepts and measures relate to establish validity, remains a challenge 70 years after Cronbach and Meehl proposed them as fundamental to validation. This limitation has practical consequences: clinical trials may fail to detect treatment effects, and public policy may target the wrong outcomes. We introduce Analysis of Latent Indicators to Generate Nomological Structures (ALIGNS), a large language model-based system trained with validated questionnaire measures. ALIGNS provides three comprehensive nomological networks containing over 550,000 indicators across psychology, medicine, social policy, and other fields. This represents the first application of large language models to solve a foundational problem in measurement validation. We report classification accuracy tests used to develop the model, as well as three evaluations. In the first evaluation, the widely used NIH PROMIS anxiety and depression instruments are shown to converge into a single dimension of emotional distress. The second evaluation examines child temperament measures and identifies four potential dimensions not captured by current frameworks, and questions one existing dimension. The third evaluation, an applicability check, engages expert psychometricians who assess the system’s importance, accessibility, and suitability. ALIGNS is freely available at nomologicalnetwork.org, complementing traditional validation methods with large-scale nomological analysis.

Method: Proposed EthosAgents approach designed to simulate diverse perspectives and values, tested empirically across seven varying-sized open and closed models for all three modes.

Result: The approach advances pluralistic alignment across all tested models, demonstrating effectiveness in handling health-related pluralism.

Conclusion: Health-related pluralism requires adaptable and normatively aware approaches, providing insights for better respecting diversity in other high-stakes domains beyond healthcare.

Abstract: As large language models are increasingly deployed in sensitive domains such as healthcare, ensuring their outputs reflect the diverse values and perspectives held across populations is critical. However, existing alignment approaches, including pluralistic paradigms like Modular Pluralism, often fall short in the health domain, where personal, cultural, and situational factors shape pluralism. Motivated by the aforementioned healthcare challenges, we propose a first lightweight, generalizable, pluralistic alignment approach, EthosAgents, designed to simulate diverse perspectives and values. We empirically show that it advances the pluralistic alignment for all three modes across seven varying-sized open and closed models. Our findings reveal that health-related pluralism demands adaptable and normatively aware approaches, offering insights into how these models can better respect diversity in other high-stakes domains.

Method: Used mt5-based encoder and Qwen decoder approaches, augmented training data with automatically translated English data for low-resource languages, and incorporated additional linguistic features inspired by previous shared task entries.

Result: Achieved a macro-accuracy score of 71.28 on the discourse relation classification task.

Conclusion: The system demonstrates competitive performance in discourse relation classification, with error analysis providing insights for future improvements in this NLP task.

Abstract: This paper presents DeDisCo, Georgetown University’s entry in the DISRPT 2025 shared task on discourse relation classification. We test two approaches, using an mt5-based encoder and a decoder based approach using the openly available Qwen model. We also experiment on training with augmented dataset for low-resource languages using matched data translated automatically from English, as well as using some additional linguistic features inspired by entries in previous editions of the Shared Task. Our system achieves a macro-accuracy score of 71.28, and we provide some interpretation and error analysis for our results.

Method: Reason-Refine-then-Align (R2tA): generates initial reasoning from base model, refines traces to fix hallucinations, then performs two-stage alignment (SFT followed by DPO) to calibrate intermediate reasoning with human preferences and condition final output on aligned reasoning.

Result: Applied to EERD evaluation with 600 variants dataset, R2tA provides practical, cost-effective path to scalable LLM adaptation in data-scarce domains, outperforming prompt-only methods that miss or hallucinate errors.

Conclusion: R2tA enables reproducible AI tools for education and beyond by effectively leveraging refined model rationales as supervision for training task-specific reasoning models in data-scarce scenarios.

Abstract: Training a task-specific small reasoning model is challenging when direct human supervision or high-quality labels are scarce. However, LLMs with reasoning capabilities produce abundant intermediate reasoning traces that can be systematically refined to create effective supervision signals. We propose Reason-Refine-then-Align (R2tA), which turns refined model rationales into supervision for training task-specific reasoning models. Our method generates initial reasoning and responses from an open-source base model on task-specific inputs, then refines these traces, fixing hallucinations and inconsistencies, to form a high-fidelity dataset. We perform a two-stage alignment, supervised fine-tuning (SFT), followed by direct preference optimization (DPO) to calibrate the model’s intermediate reasoning with human-validated conceptual preferences and then condition the final output on that aligned reasoning. As a case study, we apply R2tA to evaluate extended entity relationship diagrams (EERDs) in database system design, a structurally complex task where prompt-only methods miss or hallucinate errors. We curated a dataset of 600 EERD variants (train/test split of 450/150, respectively) with induced mistakes spanning 11 categories. Empirical evaluation suggests R2tA provides a practical, cost-effective path to scalable LLM adaptation in data-scarce domains, enabling reproducible AI tools for education and beyond.

Method: Two-stage approach: 1) Coarse-grained stage uses TF-IDF for semantic sentence filtering, 2) Fine-grained stage uses attention contribution, cross-model loss difference, and positional importance for token pruning.

Result: Achieves 49.17 performance on Longbench FewShot task using 3x fewer tokens, outperforming LongLLMLingua by 7.76 points on LLaMA-3.1-8B-Instruct and GPT-3.5-Turbo.

Conclusion: DSPC provides effective training-free prompt compression that reduces token usage while preserving semantic quality and outperforms existing methods.

Abstract: Large language models (LLMs) have achieved remarkable success in many natural language processing (NLP) tasks. To achieve more accurate output, the prompts used to drive LLMs have become increasingly longer, which incurs higher computational costs. To address this prompt inflation problem, prompt compression has been proposed. However, most existing methods require training a small auxiliary model for compression, incurring a significant amount of additional computation. To avoid this, we propose a two-stage, training-free approach, called Dual-Stage Progressive Compression (DSPC). In the coarse-grained stage, semantic-related sentence filtering removes sentences with low semantic value based on TF-IDF. In the fine-grained stage, token importance is assessed using attention contribution, cross-model loss difference, and positional importance, enabling the pruning of low-utility tokens while preserving semantics. We validate DSPC on LLaMA-3.1-8B-Instruct and GPT-3.5-Turbo under a constrained token budget and observe consistent improvements. For instance, in the FewShot task of the Longbench dataset, DSPC achieves a performance of 49.17 by using only 3x fewer tokens, outperforming the best state-of-the-art baseline LongLLMLingua by 7.76.

Method: Investigated various soft-prompting strategies (prefix-tuning, prompt-tuning, P-tuning, P-tuning V2), LoRA reparameterizations, and hard prompting with zero-shot/few-shot inferences using Arabic-specific encoder models and open-source decoder-only models including Phi-3.5 and SILMA.

Result: LLMs struggled with dialectal nuances in few-shot/zero-shot setups. Soft-prompted encoder variants performed better, while LoRA-based fine-tuned models achieved the best performance, even surpassing full fine-tuning.

Conclusion: Parameter-efficient methods like LoRA reparameterization outperform traditional approaches for Arabic Dialect Identification, offering effective solutions with reduced computational requirements.

Abstract: This paper discusses our exploration of different data-efficient and parameter-efficient approaches to Arabic Dialect Identification (ADI). In particular, we investigate various soft-prompting strategies, including prefix-tuning, prompt-tuning, P-tuning, and P-tuning V2, as well as LoRA reparameterizations. For the data-efficient strategy, we analyze hard prompting with zero-shot and few-shot inferences to analyze the dialect identification capabilities of Large Language Models (LLMs). For the parameter-efficient PEFT approaches, we conducted our experiments using Arabic-specific encoder models on several major datasets. We also analyzed the n-shot inferences on open-source decoder-only models, a general multilingual model (Phi-3.5), and an Arabic-specific one(SILMA). We observed that the LLMs generally struggle to differentiate the dialectal nuances in the few-shot or zero-shot setups. The soft-prompted encoder variants perform better, while the LoRA-based fine-tuned models perform best, even surpassing full fine-tuning.

Method: Decompose ambiguity into internal and external types, introduce AssoCiAm benchmark with hybrid computational methods to circumvent ambiguity in association evaluation.

Result: Strong positive correlation between cognition and association observed; ambiguity causes MLLMs’ behavior to become more random-like; method ensures more accurate evaluations.

Conclusion: AssoCiAm provides reliable framework for assessing associative ability in MLLMs by addressing ambiguity, contributing to better understanding of creative capabilities in multimodal AI systems.

Abstract: Recent advancements in multimodal large language models (MLLMs) have garnered significant attention, offering a promising pathway toward artificial general intelligence (AGI). Among the essential capabilities required for AGI, creativity has emerged as a critical trait for MLLMs, with association serving as its foundation. Association reflects a model’ s ability to think creatively, making it vital to evaluate and understand. While several frameworks have been proposed to assess associative ability, they often overlook the inherent ambiguity in association tasks, which arises from the divergent nature of associations and undermines the reliability of evaluations. To address this issue, we decompose ambiguity into two types-internal ambiguity and external ambiguity-and introduce AssoCiAm, a benchmark designed to evaluate associative ability while circumventing the ambiguity through a hybrid computational method. We then conduct extensive experiments on MLLMs, revealing a strong positive correlation between cognition and association. Additionally, we observe that the presence of ambiguity in the evaluation process causes MLLMs’ behavior to become more random-like. Finally, we validate the effectiveness of our method in ensuring more accurate and reliable evaluations. See Project Page for the data and codes.

Method: Generate multiple variants of input images through elastic and grid deformations that preserve class identity, then aggregate predictions using confidence-guided filtering to remove unreliable outputs.

Result: Extensive experiments on PACS and Office-Home datasets show consistent gains across different DG algorithms and backbones.

Conclusion: CI-TTA is an effective and general lightweight test-time augmentation technique that improves domain generalization performance without requiring multi-domain training or intensive adaptation.

Abstract: Deep models often suffer significant performance degradation under distribution shifts. Domain generalization (DG) seeks to mitigate this challenge by enabling models to generalize to unseen domains. Most prior approaches rely on multi-domain training or computationally intensive test-time adaptation. In contrast, we propose a complementary strategy: lightweight test-time augmentation. Specifically, we develop a novel Class-Invariant Test-Time Augmentation (CI-TTA) technique. The idea is to generate multiple variants of each input image through elastic and grid deformations that nevertheless belong to the same class as the original input. Their predictions are aggregated through a confidence-guided filtering scheme that remove unreliable outputs, ensuring the final decision relies on consistent and trustworthy cues. Extensive Experiments on PACS and Office-Home datasets demonstrate consistent gains across different DG algorithms and backbones, highlighting the effectiveness and generality of our approach.

Method: Pure transformer architecture with 4D rotary position embeddings, adversarial-free training objective combining perceptual and Gram matrix losses, progressive training curriculum expanding from single images to videos and 3D, supporting both continuous and discrete latent tokens.

Result: State-of-the-art performance: 0.21 rFID with 82.2% ImageNet accuracy for images, 3.01 rFVD with 32.6% MSRVTT retrieval for videos, and 28.19 PSNR with 90.9% classification accuracy for 3D. Enables both generation and understanding tasks across modalities.

Conclusion: AToken demonstrates that unified visual tokenization across multiple modalities is achievable and effective, paving the way for next-generation multimodal AI systems that can handle diverse visual inputs in a single framework.

Abstract: We present AToken, the first unified visual tokenizer that achieves both high-fidelity reconstruction and semantic understanding across images, videos, and 3D assets. Unlike existing tokenizers that specialize in either reconstruction or understanding for single modalities, AToken encodes these diverse visual inputs into a shared 4D latent space, unifying both tasks and modalities in a single framework. Specifically, we introduce a pure transformer architecture with 4D rotary position embeddings to process visual inputs of arbitrary resolutions and temporal durations. To ensure stable training, we introduce an adversarial-free training objective that combines perceptual and Gram matrix losses, achieving state-of-the-art reconstruction quality. By employing a progressive training curriculum, AToken gradually expands from single images, videos, and 3D, and supports both continuous and discrete latent tokens. AToken achieves 0.21 rFID with 82.2% ImageNet accuracy for images, 3.01 rFVD with 32.6% MSRVTT retrieval for videos, and 28.19 PSNR with 90.9% classification accuracy for 3D. In downstream applications, AToken enables both visual generation tasks (e.g., image generation with continuous and discrete tokens, text-to-video generation, image-to-3D synthesis) and understanding tasks (e.g., multimodal LLMs), achieving competitive performance across all benchmarks. These results shed light on the next-generation multimodal AI systems built upon unified visual tokenization.

Method: Proposes three neuroscience-inspired modules: 1) Hippocampus-inspired view alignment for capturing new view information, 2) Cognitive forgetting mechanism simulating human memory decay patterns, and 3) Prefrontal cortex-inspired knowledge consolidation using temporal tensor stability.

Result: Extensive experiments show MemEvo achieves strong knowledge retention capabilities and exhibits remarkable advantages over state-of-the-art methods in scenarios with growing numbers of views.

Conclusion: MemEvo successfully balances stability and plasticity in incremental multi-view clustering by drawing inspiration from human memory mechanisms, demonstrating superior performance compared to existing approaches.

Abstract: Incremental multi-view clustering aims to achieve stable clustering results while addressing the stability-plasticity dilemma (SPD) in incremental views. At the core of SPD is the challenge that the model must have enough plasticity to quickly adapt to new data, while maintaining sufficient stability to consolidate long-term knowledge and prevent catastrophic forgetting. Inspired by the hippocampal-prefrontal cortex collaborative memory mechanism in neuroscience, we propose a Memory-Evolving Incremental Multi-view Clustering method (MemEvo) to achieve this balance. First, we propose a hippocampus-inspired view alignment module that captures the gain information of new views by aligning structures in continuous representations. Second, we introduce a cognitive forgetting mechanism that simulates the decay patterns of human memory to modulate the weights of historical knowledge. Additionally, we design a prefrontal cortex-inspired knowledge consolidation memory module that leverages temporal tensor stability to gradually consolidate historical knowledge. By integrating these modules, MemEvo achieves strong knowledge retention capabilities in scenarios with a growing number of views. Extensive experiments demonstrate that MemEvo exhibits remarkable advantages over existing state-of-the-art methods.

Method: Edge-guided attention mechanism that derives adaptive modulation map from jointly encoded edge features and intermediate activations, applied to normalize and reweight responses. Integrated into lightweight residual design with composite loss combining pixel-wise, perceptual, and adversarial terms.

Result: Consistent improvements in structural sharpness and perceptual quality over SRGAN, ESRGAN, and prior edge-attention baselines at comparable model complexity. Achieves enhanced edge fidelity without deeper or overparameterized architectures.

Conclusion: Principled edge-conditioned modulation provides effective path for perceptual super-resolution with parameter efficiency, stabilized adversarial refinement, and improved edge fidelity.

Abstract: Single-image super-resolution (SISR) remains highly ill-posed because recovering structurally faithful high-frequency content from a single low-resolution observation is ambiguous. Existing edge-aware methods often attach edge priors or attention branches onto increasingly complex backbones, yet ad hoc fusion frequently introduces redundancy, unstable optimization, or limited structural gains. We address this gap with an edge-guided attention mechanism that derives an adaptive modulation map from jointly encoded edge features and intermediate feature activations, then applies it to normalize and reweight responses, selectively amplifying structurally salient regions while suppressing spurious textures. In parallel, we integrate this mechanism into a lightweight residual design trained under a composite objective combining pixel-wise, perceptual, and adversarial terms to balance fidelity, perceptual realism, and training stability. Extensive experiments on standard SISR benchmarks demonstrate consistent improvements in structural sharpness and perceptual quality over SRGAN, ESRGAN, and prior edge-attention baselines at comparable model complexity. The proposed formulation provides (i) a parameter-efficient path to inject edge priors, (ii) stabilized adversarial refinement through a tailored multiterm loss, and (iii) enhanced edge fidelity without resorting to deeper or heavily overparameterized architectures. These results highlight the effectiveness of principled edge-conditioned modulation for advancing perceptual super-resolution.

Method: Uses four specialized agents: Outliner Agent analyzes global theme and generates prompts, three Aspect Agents extract fine-grained descriptions along distinct visual dimensions, and Reasoning Agent synthesizes outputs through integrated reflection for classification.

Result: Experiments on 4 diverse benchmark datasets show MARIC significantly outperforms baselines, demonstrating effectiveness of multi-agent visual reasoning.

Conclusion: MARIC effectively mitigates shortcomings of both parameter-heavy training and monolithic VLM reasoning, providing robust and interpretable image classification through multi-perspective decomposition and reflective synthesis.

Abstract: Image classification has traditionally relied on parameter-intensive model training, requiring large-scale annotated datasets and extensive fine tuning to achieve competitive performance. While recent vision language models (VLMs) alleviate some of these constraints, they remain limited by their reliance on single pass representations, often failing to capture complementary aspects of visual content. In this paper, we introduce Multi Agent based Reasoning for Image Classification (MARIC), a multi agent framework that reformulates image classification as a collaborative reasoning process. MARIC first utilizes an Outliner Agent to analyze the global theme of the image and generate targeted prompts. Based on these prompts, three Aspect Agents extract fine grained descriptions along distinct visual dimensions. Finally, a Reasoning Agent synthesizes these complementary outputs through integrated reflection step, producing a unified representation for classification. By explicitly decomposing the task into multiple perspectives and encouraging reflective synthesis, MARIC mitigates the shortcomings of both parameter-heavy training and monolithic VLM reasoning. Experiments on 4 diverse image classification benchmark datasets demonstrate that MARIC significantly outperforms baselines, highlighting the effectiveness of multi-agent visual reasoning for robust and interpretable image classification.

Method: Uses score-based diffusion model with noise variation estimation to determine adaptive denoising schedule, combined with a network architecture featuring two-stage sampling strategy for feature fusion and gradient fusion during iterative denoising.

Result: Produces clean and smooth denoised point clouds with better shape boundary and detail preservation, outperforming state-of-the-art methods both qualitatively and quantitatively on synthetic and real-scanned datasets.

Conclusion: The adaptive iterative approach based on diffusion models effectively handles various noise patterns and levels while maintaining superior geometric fidelity compared to existing methods.

Abstract: Point cloud denoising task aims to recover the clean point cloud from the scanned data coupled with different levels or patterns of noise. The recent state-of-the-art methods often train deep neural networks to update the point locations towards the clean point cloud, and empirically repeat the denoising process several times in order to obtain the denoised results. It is not clear how to efficiently arrange the iterative denoising processes to deal with different levels or patterns of noise. In this paper, we propose an adaptive and iterative point cloud denoising method based on the score-based diffusion model. For a given noisy point cloud, we first estimate the noise variation and determine an adaptive denoising schedule with appropriate step sizes, then invoke the trained network iteratively to update point clouds following the adaptive schedule. To facilitate this adaptive and iterative denoising process, we design the network architecture and a two-stage sampling strategy for the network training to enable feature fusion and gradient fusion for iterative denoising. Compared to the state-of-the-art point cloud denoising methods, our approach obtains clean and smooth denoised point clouds, while preserving the shape boundary and details better. Our results not only outperform the other methods both qualitatively and quantitatively, but also are preferable on the synthetic dataset with different patterns of noises, as well as the real-scanned dataset.

Method: Enhanced Vision-Positioning Denoising Module that leverages camera calibration to establish vision-positioning mapping and denoises sensor data in an unsupervised manner, extending previous work to handle both pedestrians and vehicles.

Result: Achieves state-of-the-art performance on Vi-Fi and JRDB datasets in both trajectory denoising and prediction, significantly surpassing previous baselines and traditional methods like Kalman filtering.

Conclusion: First initiative to integrate vision-positioning projection for denoising noisy sensor trajectories of out-of-sight agents, providing comprehensive benchmarks and paving the way for future advances in autonomous driving, robotics, and related fields.

Abstract: Trajectory prediction is a critical task in computer vision and autonomous systems, playing a key role in autonomous driving, robotics, surveillance, and virtual reality. Existing methods often rely on complete and noise-free observational data, overlooking the challenges associated with out-of-sight objects and the inherent noise in sensor data caused by limited camera coverage, obstructions, and the absence of ground truth for denoised trajectories. These limitations pose safety risks and hinder reliable prediction in real-world scenarios. In this extended work, we present advancements in Out-of-Sight Trajectory (OST), a novel task that predicts the noise-free visual trajectories of out-of-sight objects using noisy sensor data. Building on our previous research, we broaden the scope of Out-of-Sight Trajectory Prediction (OOSTraj) to include pedestrians and vehicles, extending its applicability to autonomous driving, robotics, surveillance, and virtual reality. Our enhanced Vision-Positioning Denoising Module leverages camera calibration to establish a vision-positioning mapping, addressing the lack of visual references, while effectively denoising noisy sensor data in an unsupervised manner. Through extensive evaluations on the Vi-Fi and JRDB datasets, our approach achieves state-of-the-art performance in both trajectory denoising and prediction, significantly surpassing previous baselines. Additionally, we introduce comparisons with traditional denoising methods, such as Kalman filtering, and adapt recent trajectory prediction models to our task, providing a comprehensive benchmark. This work represents the first initiative to integrate vision-positioning projection for denoising noisy sensor trajectories of out-of-sight agents, paving the way for future advances. The code and preprocessed datasets are available at github.com/Hai-chao-Zhang/OST

Method: DiffVL reformulates visual localization as a GPS denoising task using diffusion models. It learns to reverse GPS noise perturbations by jointly modeling GPS, SD map, and visual BEV features through iterative diffusion refinement, recovering the true pose distribution from noisy GPS trajectories.

Result: Experiments on multiple datasets show that DiffVL achieves state-of-the-art accuracy compared to BEV-matching baselines, reaching sub-meter accuracy without relying on HD maps.

Conclusion: The work demonstrates that diffusion models can enable scalable localization by treating noisy GPS as a generative prior, representing a paradigm shift from traditional matching-based methods to generative denoising approaches.

Abstract: Accurate visual localization is crucial for autonomous driving, yet existing methods face a fundamental dilemma: While high-definition (HD) maps provide high-precision localization references, their costly construction and maintenance hinder scalability, which drives research toward standard-definition (SD) maps like OpenStreetMap. Current SD-map-based approaches primarily focus on Bird’s-Eye View (BEV) matching between images and maps, overlooking a ubiquitous signal-noisy GPS. Although GPS is readily available, it suffers from multipath errors in urban environments. We propose DiffVL, the first framework to reformulate visual localization as a GPS denoising task using diffusion models. Our key insight is that noisy GPS trajectory, when conditioned on visual BEV features and SD maps, implicitly encode the true pose distribution, which can be recovered through iterative diffusion refinement. DiffVL, unlike prior BEV-matching methods (e.g., OrienterNet) or transformer-based registration approaches, learns to reverse GPS noise perturbations by jointly modeling GPS, SD map, and visual signals, achieving sub-meter accuracy without relying on HD maps. Experiments on multiple datasets demonstrate that our method achieves state-of-the-art accuracy compared to BEV-matching baselines. Crucially, our work proves that diffusion models can enable scalable localization by treating noisy GPS as a generative prior-making a paradigm shift from traditional matching-based methods.

Method: DICE framework combines two-agent consensus equilibrium with diffusion model sampling: (1) data-consistency agent via proximal operator for measurement consistency, (2) prior agent using diffusion model for clean image estimation at each step. Alternates between these agents iteratively.

Result: Significantly outperforms state-of-the-art baselines in reconstructing high-quality CT images under both uniform and non-uniform sparse-view settings (15, 30, 60 views out of 180).

Conclusion: DICE effectively combines strong generative prior capabilities with measurement consistency, demonstrating both effectiveness and robustness for sparse-view CT reconstruction.

Abstract: Sparse-view computed tomography (CT) reconstruction is fundamentally challenging due to undersampling, leading to an ill-posed inverse problem. Traditional iterative methods incorporate handcrafted or learned priors to regularize the solution but struggle to capture the complex structures present in medical images. In contrast, diffusion models (DMs) have recently emerged as powerful generative priors that can accurately model complex image distributions. In this work, we introduce Diffusion Consensus Equilibrium (DICE), a framework that integrates a two-agent consensus equilibrium into the sampling process of a DM. DICE alternates between: (i) a data-consistency agent, implemented through a proximal operator enforcing measurement consistency, and (ii) a prior agent, realized by a DM performing a clean image estimation at each sampling step. By balancing these two complementary agents iteratively, DICE effectively combines strong generative prior capabilities with measurement consistency. Experimental results show that DICE significantly outperforms state-of-the-art baselines in reconstructing high-quality CT images under uniform and non-uniform sparse-view settings of 15, 30, and 60 views (out of a total of 180), demonstrating both its effectiveness and robustness.

Method: Proposes a Weakly Supervised Domain Adaptation method that uses patient-level diagnostic results (routinely recorded in UC diagnosis) as weak supervision. Employs Shared Aggregation Tokens and Max-Severity Triplet Loss to align class-wise distributions across domains, leveraging that patient-level diagnoses are determined by the most severe region.

Result: Experimental results show the method outperforms comparative domain adaptation approaches, improving UC severity estimation in domain-shifted settings.

Conclusion: The proposed weakly supervised approach effectively addresses domain shift challenges in UC severity estimation by leveraging readily available patient-level diagnostic data as weak supervision, demonstrating superior performance over existing methods.

Abstract: The development of methods to estimate the severity of Ulcerative Colitis (UC) is of significant importance. However, these methods often suffer from domain shifts caused by differences in imaging devices and clinical settings across hospitals. Although several domain adaptation methods have been proposed to address domain shift, they still struggle with the lack of supervision in the target domain or the high cost of annotation. To overcome these challenges, we propose a novel Weakly Supervised Domain Adaptation method that leverages patient-level diagnostic results, which are routinely recorded in UC diagnosis, as weak supervision in the target domain. The proposed method aligns class-wise distributions across domains using Shared Aggregation Tokens and a Max-Severity Triplet Loss, which leverages the characteristic that patient-level diagnoses are determined by the most severe region within each patient. Experimental results demonstrate that our method outperforms comparative DA approaches, improving UC severity estimation in a domain-shifted setting.

Method: Created benchmark with 100 Montreal street images (50 real, 50 synthetic), collected 230 annotations from 12 participants across 7 community groups on 30 dimensions. Evaluated 7 VLMs in zero-shot setup with structured prompts and deterministic parsing, using accuracy and Jaccard metrics.

Result: Models showed stronger alignment on visible, objective properties than subjective appraisals. Top system (claude-sonnet) achieved macro 0.31 and mean Jaccard 0.48 on multi-label items. Human agreement correlated with better model performance. Synthetic images slightly lowered scores.

Conclusion: The benchmark enables reproducible, uncertainty-aware evaluation of VLMs for participatory urban analysis, revealing current limitations in subjective urban perception tasks compared to objective property recognition.

Abstract: Understanding how people read city scenes can inform design and planning. We introduce a small benchmark for testing vision-language models (VLMs) on urban perception using 100 Montreal street images, evenly split between photographs and photorealistic synthetic scenes. Twelve participants from seven community groups supplied 230 annotation forms across 30 dimensions mixing physical attributes and subjective impressions. French responses were normalized to English. We evaluated seven VLMs in a zero-shot setup with a structured prompt and deterministic parser. We use accuracy for single-choice items and Jaccard overlap for multi-label items; human agreement uses Krippendorff’s alpha and pairwise Jaccard. Results suggest stronger model alignment on visible, objective properties than subjective appraisals. The top system (claude-sonnet) reaches macro 0.31 and mean Jaccard 0.48 on multi-label items. Higher human agreement coincides with better model scores. Synthetic images slightly lower scores. We release the benchmark, prompts, and harness for reproducible, uncertainty-aware evaluation in participatory urban analysis.

Method: Feature-aligned Motion Transformation (FMT) framework uses spatiotemporal alignment strategy to implicitly model continuous temporal variations, with aligned features as temporal context in latent-space conditional encoding. Includes random access reference strategy for bidirectional motion referencing and layered encoding.

Result: Surpasses D-DPCC and AdaDPCC in both encoding and decoding efficiency, achieving BD-Rate reductions of 20% and 9.4% respectively. Supports frame-level parallel compression.

Conclusion: FMT effectively improves compression efficiency and processing performance for dynamic point clouds through implicit motion modeling and advanced reference strategies.

Abstract: Dynamic point clouds are widely used in applications such as immersive reality, robotics, and autonomous driving. Efficient compression largely depends on accurate motion estimation and compensation, yet the irregular structure and significant local variations of point clouds make this task highly challenging. Current methods often rely on explicit motion estimation, whose encoded vectors struggle to capture intricate dynamics and fail to fully exploit temporal correlations. To overcome these limitations, we introduce a Feature-aligned Motion Transformation (FMT) framework for dynamic point cloud compression. FMT replaces explicit motion vectors with a spatiotemporal alignment strategy that implicitly models continuous temporal variations, using aligned features as temporal context within a latent-space conditional encoding framework. Furthermore, we design a random access (RA) reference strategy that enables bidirectional motion referencing and layered encoding, thereby supporting frame-level parallel compression. Extensive experiments demonstrate that our method surpasses D-DPCC and AdaDPCC in both encoding and decoding efficiency, while also achieving BD-Rate reductions of 20% and 9.4%, respectively. These results highlight the effectiveness of FMT in jointly improving compression efficiency and processing performance.

Method: Proposes HybridMamba with dual complementary mechanisms: 1) a feature scanning strategy that progressively integrates axial-traversal and local-adaptive pathways to harmonize local-global relationships, and 2) a gated module combining spatial-frequency analysis for comprehensive contextual modeling. Validated on multi-center CT dataset for lung cancer.

Result: Experiments on MRI and CT datasets demonstrate that HybridMamba significantly outperforms state-of-the-art methods in 3D medical image segmentation.

Conclusion: HybridMamba effectively balances local and global context modeling through its dual complementary mechanisms, achieving superior performance in 3D medical image segmentation while addressing the limitations of existing approaches.

Abstract: In the domain of 3D biomedical image segmentation, Mamba exhibits the superior performance for it addresses the limitations in modeling long-range dependencies inherent to CNNs and mitigates the abundant computational overhead associated with Transformer-based frameworks when processing high-resolution medical volumes. However, attaching undue importance to global context modeling may inadvertently compromise critical local structural information, thus leading to boundary ambiguity and regional distortion in segmentation outputs. Therefore, we propose the HybridMamba, an architecture employing dual complementary mechanisms: 1) a feature scanning strategy that progressively integrates representations both axial-traversal and local-adaptive pathways to harmonize the relationship between local and global representations, and 2) a gated module combining spatial-frequency analysis for comprehensive contextual modeling. Besides, we collect a multi-center CT dataset related to lung cancer. Experiments on MRI and CT datasets demonstrate that HybridMamba significantly outperforms the state-of-the-art methods in 3D medical image segmentation.

Method: Proposes a DSC block with two components: (1) Test-Time Training module for dynamic adaptation during inference, and (2) Dynamic Multi-Scale Kernel module that adaptively selects kernel sizes based on global context. The block is architecture-agnostic and plug-and-play.

Result: Extensive experiments demonstrate effectiveness across various U-like network architectures including CNN-based, Transformer-based, hybrid CNN-Transformer, and Mamba-based networks.

Conclusion: The DSC block successfully addresses limitations of conventional skip connections through adaptive mechanisms, enabling more effective cross-layer connectivity and multi-scale feature integration in medical image segmentation.

Abstract: U-like networks have become fundamental frameworks in medical image segmentation through skip connections that bridge high-level semantics and low-level spatial details. Despite their success, conventional skip connections exhibit two key limitations: inter-feature constraints and intra-feature constraints. The inter-feature constraint refers to the static nature of feature fusion in traditional skip connections, where information is transmitted along fixed pathways regardless of feature content. The intra-feature constraint arises from the insufficient modeling of multi-scale feature interactions, thereby hindering the effective aggregation of global contextual information. To overcome these limitations, we propose a novel Dynamic Skip Connection (DSC) block that fundamentally enhances cross-layer connectivity through adaptive mechanisms. The DSC block integrates two complementary components. (1) Test-Time Training (TTT) module. This module addresses the inter-feature constraint by enabling dynamic adaptation of hidden representations during inference, facilitating content-aware feature refinement. (2) Dynamic Multi-Scale Kernel (DMSK) module. To mitigate the intra-feature constraint, this module adaptively selects kernel sizes based on global contextual cues, enhancing the network capacity for multi-scale feature integration. The DSC block is architecture-agnostic and can be seamlessly incorporated into existing U-like network structures. Extensive experiments demonstrate the plug-and-play effectiveness of the proposed DSC block across CNN-based, Transformer-based, hybrid CNN-Transformer, and Mamba-based U-like networks.

Method: Proposes Long-Short Term Temporal Convolution (LSTC) module with parallel short/long-term branches fused adaptively, and extends Joint Mixing Data Augmentation with Additive Mixup restricted to same camera view.

Result: Achieves 94.1% and 97.5% on NTU 60, 90.4% and 92.0% on NTU 120, 97.2% on NW-UCLA - state-of-the-art performance.

Conclusion: LSTC-MDA effectively addresses temporal modeling challenges and data scarcity through innovative architecture and augmentation techniques, demonstrating superior performance across multiple datasets.

Abstract: Skeleton-based action recognition faces two longstanding challenges: the scarcity of labeled training samples and difficulty modeling short- and long-range temporal dependencies. To address these issues, we propose a unified framework, LSTC-MDA, which simultaneously improves temporal modeling and data diversity. We introduce a novel Long-Short Term Temporal Convolution (LSTC) module with parallel short- and long-term branches, these two feature branches are then aligned and fused adaptively using learned similarity weights to preserve critical long-range cues lost by conventional stride-2 temporal convolutions. We also extend Joint Mixing Data Augmentation (JMDA) with an Additive Mixup at the input level, diversifying training samples and restricting mixup operations to the same camera view to avoid distribution shifts. Ablation studies confirm each component contributes. LSTC-MDA achieves state-of-the-art results: 94.1% and 97.5% on NTU 60 (X-Sub and X-View), 90.4% and 92.0% on NTU 120 (X-Sub and X-Set),97.2% on NW-UCLA. Code: https://github.com/xiaobaoxia/LSTC-MDA.

Method: A novel dataset construction pipeline using two multi-modal large language models (MLLMs) to generate visual-adaptive editing instructions and produce high-fidelity edited images, covering diverse editing types and operations.

Result: Fine-tuning foundational models with MultiEdit-Train substantially improves performance on sophisticated editing tasks in the MultiEdit-Test benchmark while preserving capabilities on standard editing benchmarks.

Conclusion: MultiEdit provides a valuable resource for advancing research into more diverse and challenging instruction-based image editing capabilities, addressing limitations of existing datasets.

Abstract: Current instruction-based image editing (IBIE) methods struggle with challenging editing tasks, as both editing types and sample counts of existing datasets are limited. Moreover, traditional dataset construction often contains noisy image-caption pairs, which may introduce biases and limit model capabilities in complex editing scenarios. To address these limitations, we introduce MultiEdit, a comprehensive dataset featuring over 107K high-quality image editing samples. It encompasses 6 challenging editing tasks through a diverse collection of 18 non-style-transfer editing types and 38 style transfer operations, covering a spectrum from sophisticated style transfer to complex semantic operations like person reference editing and in-image text editing. We employ a novel dataset construction pipeline that utilizes two multi-modal large language models (MLLMs) to generate visual-adaptive editing instructions and produce high-fidelity edited images, respectively. Extensive experiments demonstrate that fine-tuning foundational open-source models with our MultiEdit-Train set substantially improves models’ performance on sophisticated editing tasks in our proposed MultiEdit-Test benchmark, while effectively preserving their capabilities on the standard editing benchmark. We believe MultiEdit provides a valuable resource for advancing research into more diverse and challenging IBIE capabilities. Our dataset is available at https://huggingface.co/datasets/inclusionAI/MultiEdit.

Method: Proposes two novel mechanisms: Attention Lattice Adapter (ALA) that eliminates manual layer selection, and Alternating Epoch Architect (AEA) that updates parameters every other epoch to address small attention regions.

Result: Outperformed baselines on CUB-200-2011 and ImageNet-S datasets across multiple metrics (mean IoU, insertion, deletion, insertion-deletion scores), with 53.2-point improvement in mean IoU on CUB-200-2011.

Conclusion: The proposed method successfully enhances interpretability and adaptability of visual foundation models while achieving superior performance on benchmark datasets compared to existing approaches.

Abstract: In this study, we consider the problem of generating visual explanations in visual foundation models. Numerous methods have been proposed for this purpose; however, they often cannot be applied to complex models due to their lack of adaptability. To overcome these limitations, we propose a novel explanation generation method in visual foundation models that is aimed at both generating explanations and partially updating model parameters to enhance interpretability. Our approach introduces two novel mechanisms: Attention Lattice Adapter (ALA) and Alternating Epoch Architect (AEA). ALA mechanism simplifies the process by eliminating the need for manual layer selection, thus enhancing the model’s adaptability and interpretability. Moreover, the AEA mechanism, which updates ALA’s parameters every other epoch, effectively addresses the common issue of overly small attention regions. We evaluated our method on two benchmark datasets, CUB-200-2011 and ImageNet-S. Our results showed that our method outperformed the baseline methods in terms of mean intersection over union (IoU), insertion score, deletion score, and insertion-deletion score on both the CUB-200-2011 and ImageNet-S datasets. Notably, our best model achieved a 53.2-point improvement in mean IoU on the CUB-200-2011 dataset compared with the baselines.

Method: Novel end-to-end neural architecture with multi-task learning and aggressive data augmentation policies to enforce learning of domain-invariant features.

Result: The method effectively closes the domain gap and achieves state-of-the-art accuracy on the SPEED+ dataset.

Conclusion: The proposed approach successfully bridges the synthetic-to-real domain gap in spacecraft pose estimation through domain-invariant feature learning.

Abstract: We address the problem of estimating the relative 6D pose, i.e., position and orientation, of a target spacecraft, from a monocular image, a key capability for future autonomous Rendezvous and Proximity Operations. Due to the difficulty of acquiring large sets of real images, spacecraft pose estimation networks are exclusively trained on synthetic ones. However, because those images do not capture the illumination conditions encountered in orbit, pose estimation networks face a domain gap problem, i.e., they do not generalize to real images. Our work introduces a method that bridges this domain gap. It relies on a novel, end-to-end, neural-based architecture as well as a novel learning strategy. This strategy improves the domain generalization abilities of the network through multi-task learning and aggressive data augmentation policies, thereby enforcing the network to learn domain-invariant features. We demonstrate that our method effectively closes the domain gap, achieving state-of-the-art accuracy on the widespread SPEED+ dataset. Finally, ablation studies assess the impact of key components of our method on its generalization abilities.

Method: Proposes DACoN framework that uses foundation models to capture part-level semantics in line drawings, fuses low-resolution semantic features with high-resolution CNN spatial features, and removes the constraint of limited reference images.

Result: Quantitative and qualitative evaluations show superior colorization performance, demonstrating the benefits of using multiple reference images.

Conclusion: DACoN achieves robust and fine-grained feature extraction for anime colorization by leveraging foundation models and supporting unlimited reference images, outperforming previous methods.

Abstract: Automatic colorization of line drawings has been widely studied to reduce the labor cost of hand-drawn anime production. Deep learning approaches, including image/video generation and feature-based correspondence, have improved accuracy but struggle with occlusions, pose variations, and viewpoint changes. To address these challenges, we propose DACoN, a framework that leverages foundation models to capture part-level semantics, even in line drawings. Our method fuses low-resolution semantic features from foundation models with high-resolution spatial features from CNNs for fine-grained yet robust feature extraction. In contrast to previous methods that rely on the Multiplex Transformer and support only one or two reference images, DACoN removes this constraint, allowing any number of references. Quantitative and qualitative evaluations demonstrate the benefits of using multiple reference images, achieving superior colorization performance. Our code and model are available at https://github.com/kzmngt/DACoN.

Method: Proposes two innovations: 1) Mesh-guided 2D Gaussian splatting where primitives are constrained to template mesh faces for better surface alignment, and 2) Coordinated training with selective gradient isolation to distill multi-modal prior knowledge from foundation models without conflicting optimization objectives.

Result: Superior reconstruction quality compared to existing methods, with significant gains in geometric accuracy and appearance fidelity, while providing rich semantic information and enabling consistent rendering under novel views and poses.

Conclusion: The combination of enhanced representation through constrained 2D Gaussian primitives and coordinated information distillation from foundation models significantly advances monocular human avatar reconstruction capabilities.

Abstract: Reconstructing high-fidelity animatable human avatars from monocular videos remains challenging due to insufficient geometric information in single-view observations. While recent 3D Gaussian Splatting methods have shown promise, they struggle with surface detail preservation due to the free-form nature of 3D Gaussian primitives. To address both the representation limitations and information scarcity, we propose a novel method, \textbf{FMGS-Avatar}, that integrates two key innovations. First, we introduce Mesh-Guided 2D Gaussian Splatting, where 2D Gaussian primitives are attached directly to template mesh faces with constrained position, rotation, and movement, enabling superior surface alignment and geometric detail preservation. Second, we leverage foundation models trained on large-scale datasets, such as Sapiens, to complement the limited visual cues from monocular videos. However, when distilling multi-modal prior knowledge from foundation models, conflicting optimization objectives can emerge as different modalities exhibit distinct parameter sensitivities. We address this through a coordinated training strategy with selective gradient isolation, enabling each loss component to optimize its relevant parameters without interference. Through this combination of enhanced representation and coordinated information distillation, our approach significantly advances 3D monocular human avatar reconstruction. Experimental evaluation demonstrates superior reconstruction quality compared to existing methods, with notable gains in geometric accuracy and appearance fidelity while providing rich semantic information. Additionally, the distilled prior knowledge within a shared canonical space naturally enables spatially and temporally consistent rendering under novel views and poses.

Method: Used a curated dataset of 50 video clips from 42 cattle in various environments. Applied data augmentation and trained two deep learning models: 3D CNN and ConvLSTM2D for end-to-end video classification.

Result: 3D CNN achieved 90% accuracy with 90.9% precision, recall, and F1 score, outperforming ConvLSTM2D (85% accuracy) and matching state-of-the-art methods without requiring pose estimation pre-processing.

Conclusion: Deep learning models can effectively extract spatio-temporal features from videos for scalable cattle lameness detection in real farm settings, demonstrating the viability of direct end-to-end approaches.

Abstract: Cattle lameness is a prevalent health problem in livestock farming, often resulting from hoof injuries or infections, and severely impacts animal welfare and productivity. Early and accurate detection is critical for minimizing economic losses and ensuring proper treatment. This study proposes a spatiotemporal deep learning framework for automated cattle lameness detection using publicly available video data. We curate and publicly release a balanced set of 50 online video clips featuring 42 individual cattle, recorded from multiple viewpoints in both indoor and outdoor environments. The videos were categorized into lame and non-lame classes based on visual gait characteristics and metadata descriptions. After applying data augmentation techniques to enhance generalization, two deep learning architectures were trained and evaluated: 3D Convolutional Neural Networks (3D CNN) and Convolutional Long-Short-Term Memory (ConvLSTM2D). The 3D CNN achieved a video-level classification accuracy of 90%, with a precision, recall, and F1 score of 90.9% each, outperforming the ConvLSTM2D model, which achieved 85% accuracy. Unlike conventional approaches that rely on multistage pipelines involving object detection and pose estimation, this study demonstrates the effectiveness of a direct end-to-end video classification approach. Compared with the best end-to-end prior method (C3D-ConvLSTM, 90.3%), our model achieves comparable accuracy while eliminating pose estimation pre-processing.The results indicate that deep learning models can successfully extract and learn spatio-temporal features from various video sources, enabling scalable and efficient cattle lameness detection in real-world farm settings.

Method: Proposes Chain-of-Thought Re-Ranking (CoTRR) with listwise ranking prompts, image evaluation prompts for candidate alignment assessment, and query deconstruction prompts for semantic breakdown.

Result: Achieves state-of-the-art performance on five datasets across three image retrieval tasks: text-to-image retrieval, composed image retrieval, and chat-based image retrieval.

Conclusion: MLLMs can be effectively integrated into the ranking process through chain-of-thought reasoning, enabling global comparison, consistent reasoning, and interpretable decision-making for superior image retrieval.

Abstract: Image retrieval remains a fundamental yet challenging problem in computer vision. While recent advances in Multimodal Large Language Models (MLLMs) have demonstrated strong reasoning capabilities, existing methods typically employ them only for evaluation, without involving them directly in the ranking process. As a result, their rich multimodal reasoning abilities remain underutilized, leading to suboptimal performance. In this paper, we propose a novel Chain-of-Thought Re-Ranking (CoTRR) method to address this issue. Specifically, we design a listwise ranking prompt that enables MLLM to directly participate in re-ranking candidate images. This ranking process is grounded in an image evaluation prompt, which assesses how well each candidate aligns with users query. By allowing MLLM to perform listwise reasoning, our method supports global comparison, consistent reasoning, and interpretable decision-making - all of which are essential for accurate image retrieval. To enable structured and fine-grained analysis, we further introduce a query deconstruction prompt, which breaks down the original query into multiple semantic components. Extensive experiments on five datasets demonstrate the effectiveness of our CoTRR method, which achieves state-of-the-art performance across three image retrieval tasks, including text-to-image retrieval (TIR), composed image retrieval (CIR) and chat-based image retrieval (Chat-IR). Our code is available at https://github.com/freshfish15/CoTRR .

Method: Evaluate several diffusion-based augmentation strategies to generate synthetic data and incorporate it into model training for smell-related object detection.

Result: Incorporating synthetic data improves detection performance, with the approach being effective even with small amounts of data and showing potential for further enhancements through scaling.

Conclusion: Leveraging large-scale pretraining of diffusion models offers a promising approach for improving detection accuracy in niche applications where annotations are scarce and costly to obtain.

Abstract: Finding smell references in historic artworks is a challenging problem. Beyond artwork-specific challenges such as stylistic variations, their recognition demands exceptionally detailed annotation classes, resulting in annotation sparsity and extreme class imbalance. In this work, we explore the potential of synthetic data generation to alleviate these issues and enable accurate detection of smell-related objects. We evaluate several diffusion-based augmentation strategies and demonstrate that incorporating synthetic data into model training can improve detection performance. Our findings suggest that leveraging the large-scale pretraining of diffusion models offers a promising approach for improving detection accuracy, particularly in niche applications where annotations are scarce and costly to obtain. Furthermore, the proposed approach proves to be effective even with relatively small amounts of data, and scaling it up provides high potential for further enhancements.

Method: Proposed a frame-accurate benchmark for small VLMs on video question-answering, evaluated under controlled frame-sampling strategies with open-sourced benchmarking code.

Result: Results confirm suspected bias and highlight both data-specific and task-specific behaviors of SVLMs under different frame-sampling techniques.

Conclusion: Need for standardized frame-sampling strategies tailored to each benchmarking dataset and provides reproducible, unbiased protocol for evaluating video VLMs.

Abstract: Comparing vision language models on videos is particularly complex, as the performances is jointly determined by the model’s visual representation capacity and the frame-sampling strategy used to construct the input. Current video benchmarks are suspected to suffer from substantial frame-sampling bias, as models are evaluated with different frame selection strategies. In this work, we propose the first frame-accurate benchmark of state-of-the-art small VLMs for video question-answering, evaluated under controlled frame-sampling strategies. Our results confirm the suspected bias and highlight both data-specific and task-specific behaviors of SVLMs under different frame-sampling techniques. By open-sourcing our benchmarking code, we provide the community with a reproducible and unbiased protocol for evaluating video VLMs and emphasize the need for standardized frame-sampling strategies tailored to each benchmarking dataset in future research.

Method: First uses Gaussian mixture model to segment point cloud into clusters, then selects and merges flat clusters into planes or curved surfaces. Planes are fitted with 2D voxel-based boundary description, while curved surfaces use B-spline surfaces with the same boundary method.

Result: 3.78 times improvement in efficiency over state-of-the-art, 2-fold accuracy gain over Gaussian mixture models, and operates at 36.4 fps on low-power onboard computer with greater robustness.

Conclusion: The proposed multi-model approach successfully balances real-time performance with high accuracy, providing an efficient and robust parametric representation solution for point cloud processing tasks.

Abstract: In recent years, parametric representations of point clouds have been widely applied in tasks such as memory-efficient mapping and multi-robot collaboration. Highly adaptive models, like spline surfaces or quadrics, are computationally expensive in detection or fitting. In contrast, real-time methods, such as Gaussian mixture models or planes, have low degrees of freedom, making high accuracy with few primitives difficult. To tackle this problem, a multi-model parametric representation with real-time surface detection and fitting is proposed. Specifically, the Gaussian mixture model is first employed to segment the point cloud into multiple clusters. Then, flat clusters are selected and merged into planes or curved surfaces. Planes can be easily fitted and delimited by a 2D voxel-based boundary description method. Surfaces with curvature are fitted by B-spline surfaces and the same boundary description method is employed. Through evaluations on multiple public datasets, the proposed surface detection exhibits greater robustness than the state-of-the-art approach, with 3.78 times improvement in efficiency. Meanwhile, this representation achieves a 2-fold gain in accuracy over Gaussian mixture models, operating at 36.4 fps on a low-power onboard computer.

Method: Extract high-gradient patches, categorize images using CLIP features, then fine-tune a diffusion model on selected patches to learn their distribution and synthesize distilled training images without requiring class labels or pre-trained SR models.

Result: Achieves state-of-the-art performance with only 0.68% of original dataset, showing just 0.3 dB performance drop. Diffusion model fine-tuning takes 4 hours and SR training completes in 1 hour, compared to 11 hours with full dataset.

Conclusion: The proposed method provides an efficient and generalizable data distillation approach for image super-resolution that significantly reduces data requirements and computational time while maintaining high performance.

Abstract: Training deep neural networks has become increasingly demanding, requiring large datasets and significant computational resources, especially as model complexity advances. Data distillation methods, which aim to improve data efficiency, have emerged as promising solutions to this challenge. In the field of single image super-resolution (SISR), the reliance on large training datasets highlights the importance of these techniques. Recently, a generative adversarial network (GAN) inversion-based data distillation framework for SR was proposed, showing potential for better data utilization. However, the current method depends heavily on pre-trained SR networks and class-specific information, limiting its generalizability and applicability. To address these issues, we introduce a new data distillation approach for image SR that does not need class labels or pre-trained SR models. In particular, we first extract high-gradient patches and categorize images based on CLIP features, then fine-tune a diffusion model on the selected patches to learn their distribution and synthesize distilled training images. Experimental results show that our method achieves state-of-the-art performance while using significantly less training data and requiring less computational time. Specifically, when we train a baseline Transformer model for SR with only 0.68% of the original dataset, the performance drop is just 0.3 dB. In this case, diffusion model fine-tuning takes 4 hours, and SR model training completes within 1 hour, much shorter than the 11-hour training time with the full dataset.

Method: Proposes Report2CT framework with three pretrained medical text encoders (BiomedVLP CXR BERT, MedEmbed, ClinicalBERT) to condition a 3D latent diffusion model trained on 20,000 CT volumes from CT RATE dataset, incorporating both findings and impression sections from radiology reports.

Result: Generated anatomically consistent CT volumes with excellent visual quality and text-image alignment. Multi-encoder conditioning improved CLIP scores, classifier-free guidance enhanced alignment with minor FID trade-off. Ranked first in VLM3D Challenge at MICCAI 2025 and achieved state-of-the-art performance across all metrics.

Conclusion: By leveraging complete radiology reports and multi-encoder text conditioning, Report2CT advances 3D CT synthesis, producing clinically faithful and high-quality synthetic data that preserves fine-grained clinical details.

Abstract: Text to image latent diffusion models have recently advanced medical image synthesis, but applications to 3D CT generation remain limited. Existing approaches rely on simplified prompts, neglecting the rich semantic detail in full radiology reports, which reduces text image alignment and clinical fidelity. We propose Report2CT, a radiology report conditional latent diffusion framework for synthesizing 3D chest CT volumes directly from free text radiology reports, incorporating both findings and impression sections using multiple text encoder. Report2CT integrates three pretrained medical text encoders (BiomedVLP CXR BERT, MedEmbed, and ClinicalBERT) to capture nuanced clinical context. Radiology reports and voxel spacing information condition a 3D latent diffusion model trained on 20000 CT volumes from the CT RATE dataset. Model performance was evaluated using Frechet Inception Distance (FID) for real synthetic distributional similarity and CLIP based metrics for semantic alignment, with additional qualitative and quantitative comparisons against GenerateCT model. Report2CT generated anatomically consistent CT volumes with excellent visual quality and text image alignment. Multi encoder conditioning improved CLIP scores, indicating stronger preservation of fine grained clinical details in the free text radiology reports. Classifier free guidance further enhanced alignment with only a minor trade off in FID. We ranked first in the VLM3D Challenge at MICCAI 2025 on Text Conditional CT Generation and achieved state of the art performance across all evaluation metrics. By leveraging complete radiology reports and multi encoder text conditioning, Report2CT advances 3D CT synthesis, producing clinically faithful and high quality synthetic data.

Method: Proposed a novel edge-detector function combining intensity and gradient norm to guide contours toward bone edges without soft tissue interference. Introduced distance information with embedded fracture prompts as adaptive step size to stabilize contour evolution and improve fracture region accuracy.

Result: Experiments on pelvic and ankle segmentation demonstrated effective addressing of edge obstruction, leakage, and fracture problems, showing accurate, stable, and consistent performance.

Conclusion: The algorithm provides robust bone segmentation with fracture handling and offers insights for combining domain knowledge with deep neural networks for broader bone anatomy applications.

Abstract: For bone segmentation, the classical geodesic active contour model is usually limited by its indiscriminate feature extraction, and then struggles to handle the phenomena of edge obstruction, edge leakage and bone fracture. Thus, we propose a fracture interactive geodesic active contour algorithm tailored for bone segmentation, which can better capture bone features and perform robustly to the presence of bone fractures and soft tissues. Inspired by orthopedic knowledge, we construct a novel edge-detector function that combines the intensity and gradient norm, which guides the contour towards bone edges without being obstructed by other soft tissues and therefore reduces mis-segmentation. Furthermore, distance information, where fracture prompts can be embedded, is introduced into the contour evolution as an adaptive step size to stabilize the evolution and help the contour stop at bone edges and fractures. This embedding provides a way to interact with bone fractures and improves the accuracy in the fracture regions. Experiments in pelvic and ankle segmentation demonstrate the effectiveness on addressing the aforementioned problems and show an accurate, stable and consistent performance, indicating a broader application in other bone anatomies. Our algorithm also provides insights into combining the domain knowledge and deep neural networks.

Method: Design Minimal Energy Deformation (MED) loss as regularizer on deformation trajectories, integrated with Chamfer distance in V2C-Flow model.

Result: Considerable improvements in training consistency and reproducibility while maintaining reconstruction accuracy and topological correctness.

Conclusion: MED loss effectively addresses training consistency challenges in learning-based cortical surface reconstruction without sacrificing performance.

Abstract: Cortical surface reconstruction (CSR) from magnetic resonance imaging (MRI) is fundamental to neuroimage analysis, enabling morphological studies of the cerebral cortex and functional brain mapping. Recent advances in learning-based CSR have dramatically accelerated processing, allowing for reconstructions through the deformation of anatomical templates within seconds. However, ensuring the learned deformations are optimal in terms of deformation energy and consistent across training runs remains a particular challenge. In this work, we design a Minimal Energy Deformation (MED) loss, acting as a regularizer on the deformation trajectories and complementing the widely used Chamfer distance in CSR. We incorporate it into the recent V2C-Flow model and demonstrate considerable improvements in previously neglected training consistency and reproducibility without harming reconstruction accuracy and topological correctness.

Method: ProtoMedX uses a prototype-based architecture that processes both DEXA scans of the lumbar spine and patient records, providing explainable decisions by design rather than relying on post hoc analysis.

Result: The model achieved 87.58% accuracy in vision-only tasks and 89.8% accuracy in multi-modal tasks using a dataset of 4,160 NHS patients, surpassing existing published methods.

Conclusion: ProtoMedX demonstrates that multi-modal approaches with built-in explainability can achieve superior performance in medical diagnostics while meeting regulatory requirements for transparent AI decision-making.

Abstract: Bone health studies are crucial in medical practice for the early detection and treatment of Osteopenia and Osteoporosis. Clinicians usually make a diagnosis based on densitometry (DEXA scans) and patient history. The applications of AI in this field are ongoing research. Most successful methods rely on deep learning models that use vision alone (DEXA/X-ray imagery) and focus on prediction accuracy, while explainability is often disregarded and left to post hoc assessments of input contributions. We propose ProtoMedX, a multi-modal model that uses both DEXA scans of the lumbar spine and patient records. ProtoMedX’s prototype-based architecture is explainable by design, which is crucial for medical applications, especially in the context of the upcoming EU AI Act, as it allows explicit analysis of model decisions, including incorrect ones. ProtoMedX demonstrates state-of-the-art performance in bone health classification while also providing explanations that can be visually understood by clinicians. Using a dataset of 4,160 real NHS patients, the proposed ProtoMedX achieves 87.58% accuracy in vision-only tasks and 89.8% in its multi-modal variant, both surpassing existing published methods.

Method: Uses advanced Metric Depth Estimation models to calculate geocoordinates based on object distance from camera, geometric principles, and camera specifications. Validated against LiDAR point clouds in urban environments.

Result: The module demonstrates effectiveness through practical use cases with traffic signs and road damage. Evaluation measures accuracy of estimated distances across different distance intervals and semantic areas (roads, vegetation).

Conclusion: MapAnything provides an automated solution for urban object mapping, with recommendations for automating urban object and incident database maintenance through image-based geocoordinate estimation.

Abstract: To maintain an overview of urban conditions, city administrations manage databases of objects like traffic signs and trees, complete with their geocoordinates. Incidents such as graffiti or road damage are also relevant. As digitization increases, so does the need for more data and up-to-date databases, requiring significant manual effort. This paper introduces MapAnything, a module that automatically determines the geocoordinates of objects using individual images. Utilizing advanced Metric Depth Estimation models, MapAnything calculates geocoordinates based on the object’s distance from the camera, geometric principles, and camera specifications. We detail and validate the module, providing recommendations for automating urban object and incident mapping. Our evaluation measures the accuracy of estimated distances against LiDAR point clouds in urban environments, analyzing performance across distance intervals and semantic areas like roads and vegetation. The module’s effectiveness is demonstrated through practical use cases involving traffic signs and road damage.

Method: A targeted feature denoising framework consisting of noise detection and denoising modules that can be seamlessly integrated with existing super-resolution models without requiring architectural modifications.

Result: The framework demonstrates superior performance compared to previous regularization-based methods across five traditional benchmarks and datasets, including both synthetic and real-world scenarios.

Conclusion: The proposed targeted approach focusing specifically on noise overfitting provides a more effective solution for generalizable image super-resolution than methods that treat all degradation types equally.

Abstract: Generalizable Image Super-Resolution aims to enhance model generalization capabilities under unknown degradations. To achieve this goal, the models are expected to focus only on image content-related features instead of overfitting degradations. Recently, numerous approaches such as Dropout and Feature Alignment have been proposed to suppress models’ natural tendency to overfit degradations and yield promising results. Nevertheless, these works have assumed that models overfit to all degradation types (e.g., blur, noise, JPEG), while through careful investigations in this paper, we discover that models predominantly overfit to noise, largely attributable to its distinct degradation pattern compared to other degradation types. In this paper, we propose a targeted feature denoising framework, comprising noise detection and denoising modules. Our approach presents a general solution that can be seamlessly integrated with existing super-resolution models without requiring architectural modifications. Our framework demonstrates superior performance compared to previous regularization-based methods across five traditional benchmarks and datasets, encompassing both synthetic and real-world scenarios.

Method: Reproduced FViTs experiments using DDS, tested robustness against attacks in segmentation and classification tasks, extended testing to baseline methods and Attribution Rollout method, and measured computational costs and environmental impact of DDS implementation.

Result: Results broadly agree with original study’s findings - DDS does improve interpretability robustness to attacks in both segmentation and classification tasks. Minor discrepancies were identified and discussed. Computational costs of DDS implementation were quantified.

Conclusion: The reproduction study successfully validates the core claims of the original FViTs paper regarding DDS improving interpretability robustness, while providing additional insights into computational requirements and extending the analysis to more interpretability methods.

Abstract: This work aims to reproduce the results of Faithful Vision Transformers (FViTs) proposed by arXiv:2311.17983 alongside interpretability methods for Vision Transformers from arXiv:2012.09838 and Xu (2022) et al. We investigate claims made by arXiv:2311.17983, namely that the usage of Diffusion Denoised Smoothing (DDS) improves interpretability robustness to (1) attacks in a segmentation task and (2) perturbation and attacks in a classification task. We also extend the original study by investigating the authors’ claims that adding DDS to any interpretability method can improve its robustness under attack. This is tested on baseline methods and the recently proposed Attribution Rollout method. In addition, we measure the computational costs and environmental impact of obtaining an FViT through DDS. Our results broadly agree with the original study’s findings, although minor discrepancies were found and discussed.

Method: Leverages inpainting ability of latent diffusion models with adaptive attribute-guidance module that applies gradient correction during reverse denoising to align facial attributes with synthesized target images. Supports localized anonymization.

Result: Extensive experiments on CelebA-HQ and FFHQ datasets show method outperforms state-of-the-art approaches while requiring no additional model training.

Conclusion: Proposed framework provides effective facial anonymization with complete control over the process, maintaining realism and utility for computer vision applications.

Abstract: The growing use of portrait images in computer vision highlights the need to protect personal identities. At the same time, anonymized images must remain useful for downstream computer vision tasks. In this work, we propose a unified framework that leverages the inpainting ability of latent diffusion models to generate realistic anonymized images. Unlike prior approaches, we have complete control over the anonymization process by designing an adaptive attribute-guidance module that applies gradient correction during the reverse denoising process, aligning the facial attributes of the generated image with those of the synthesized target image. Our framework also supports localized anonymization, allowing users to specify which facial regions are left unchanged. Extensive experiments conducted on the public CelebA-HQ and FFHQ datasets show that our method outperforms state-of-the-art approaches while requiring no additional model training. The source code is available on our page.

Method: Multi-task model that learns representations of early treatment response dynamics from longitudinal MRI data, representing appearance, fostering temporal continuity, and accounting for heterogeneity in non-responder cohort. Uses latent trajectory space for prediction.

Result: Linear classifier in latent trajectory space achieves balanced accuracy of 0.761 (pre-treatment only), 0.811 (early response), and 0.861 (four imaging time points) on ISPY-2 dataset.

Conclusion: The proposed method effectively predicts pathological complete response in breast cancer patients undergoing neoadjuvant chemotherapy by modeling treatment response dynamics from longitudinal imaging data.

Abstract: Effective therapy decisions require models that predict the individual response to treatment. This is challenging since the progression of disease and response to treatment vary substantially across patients. Here, we propose to learn a representation of the early dynamics of treatment response from imaging data to predict pathological complete response (pCR) in breast cancer patients undergoing neoadjuvant chemotherapy (NACT). The longitudinal change in magnetic resonance imaging (MRI) data of the breast forms trajectories in the latent space, serving as basis for prediction of successful response. The multi-task model represents appearance, fosters temporal continuity and accounts for the comparably high heterogeneity in the non-responder cohort.In experiments on the publicly available ISPY-2 dataset, a linear classifier in the latent trajectory space achieves a balanced accuracy of 0.761 using only pre-treatment data (T0), 0.811 using early response (T0 + T1), and 0.861 using four imaging time points (T0 -> T3). The code will be made available upon paper acceptance.

Method: Train a NeRF-based image generator using gradients back-propagated through the pose estimation network to render the main 3D features exploited by the spacecraft pose estimator.

Result: Experiments demonstrate the method successfully recovers relevant 3D cues and provides insights into the relationship between pose estimation network supervision and its implicit representation of the target spacecraft.

Conclusion: The proposed visualization method effectively reveals the 3D visual features that pose estimation networks rely on, improving interpretability and potentially facilitating adoption in real space missions.

Abstract: On-orbit operations require the estimation of the relative 6D pose, i.e., position and orientation, between a chaser spacecraft and its target. While data-driven spacecraft pose estimation methods have been developed, their adoption in real missions is hampered by the lack of understanding of their decision process. This paper presents a method to visualize the 3D visual cues on which a given pose estimator relies. For this purpose, we train a NeRF-based image generator using the gradients back-propagated through the pose estimation network. This enforces the generator to render the main 3D features exploited by the spacecraft pose estimation network. Experiments demonstrate that our method recovers the relevant 3D cues. Furthermore, they offer additional insights on the relationship between the pose estimation network supervision and its implicit representation of the target spacecraft.

Method: Pseudo-labeling strategy using trained SAM2 to generate labels for MOSE test set, combined with cascaded decision mechanism integrating SAM2Long and SeC model outputs.

Result: Achieved J&F score of 0.8616 on MOSE test set (+1.4 points over baseline), securing 2nd place in LSVOS 2025 VOS Track.

Conclusion: The approach demonstrates strong robustness and accuracy in long, complex video segmentation scenarios through pseudo-label training and multi-model integration.

Abstract: Complex Video Object Segmentation (VOS) presents significant challenges in accurately segmenting objects across frames, especially in the presence of small and similar targets, frequent occlusions, rapid motion, and complex interactions. In this report, we present our solution for the LSVOS 2025 VOS Track based on the SAM2 framework. We adopt a pseudo-labeling strategy during training: a trained SAM2 checkpoint is deployed within the SAM2Long framework to generate pseudo labels for the MOSE test set, which are then combined with existing data for further training. For inference, the SAM2Long framework is employed to obtain our primary segmentation results, while an open-source SeC model runs in parallel to produce complementary predictions. A cascaded decision mechanism dynamically integrates outputs from both models, exploiting the temporal stability of SAM2Long and the concept-level robustness of SeC. Benefiting from pseudo-label training and cascaded multi-model inference, our approach achieves a J&F score of 0.8616 on the MOSE test set – +1.4 points over our SAM2Long baseline – securing the 2nd place in the LSVOS 2025 VOS Track, and demonstrating strong robustness and accuracy in long, complex video segmentation scenarios.

Method: Evaluated three instances of CLIP fine-tuned for FR, MAD, and PAD on 14 general vision datasets under zero-shot and linear-probe protocols, alongside common FR, MAD, and PAD benchmarks. Compared performance of adapted models against original CLIP baseline.

Result: Fine-tuned models suffer from over-specialization, especially for complex FR tasks. FR model achieved 58.52% improvement on IJB-C FR benchmark but dropped to 51.63% on ImageNetV2 (vs 69.84% baseline). Larger CLIP architecture preserved more original generalization ability than smaller variants.

Conclusion: Task complexity and classification head design correlate with catastrophic forgetting. Increased model capacity helps mitigate over-specialization, but fine-tuning foundation models for specialized tasks comes at significant cost to their cross-domain generalization capabilities.

Abstract: Foundation models such as CLIP have demonstrated exceptional zero- and few-shot transfer capabilities across diverse vision tasks. However, when fine-tuned for highly specialized biometric tasks, face recognition (FR), morphing attack detection (MAD), and presentation attack detection (PAD), these models may suffer from over-specialization. Thus, they may lose one of their foundational strengths, cross-domain generalization. In this work, we systematically quantify these trade-offs by evaluating three instances of CLIP fine-tuned for FR, MAD, and PAD. We evaluate each adapted model as well as the original CLIP baseline on 14 general vision datasets under zero-shot and linear-probe protocols, alongside common FR, MAD, and PAD benchmarks. Our results indicate that fine-tuned models suffer from over-specialization, especially when fine-tuned for complex tasks of FR. Also, our results pointed out that task complexity and classification head design, multi-class (FR) vs. binary (MAD and PAD), correlate with the degree of catastrophic forgetting. The FRoundation model with the ViT-L backbone outperforms other approaches on the large-scale FR benchmark IJB-C, achieving an improvement of up to 58.52%. However, it experiences a substantial performance drop on ImageNetV2, reaching only 51.63% compared to 69.84% achieved by the baseline CLIP model. Moreover, the larger CLIP architecture consistently preserves more of the model’s original generalization ability than the smaller variant, indicating that increased model capacity may help mitigate over-specialization.

Method: Developed an interactive system with intuitive interface that integrates multiple AI capabilities as modular services (garment generation, makeup transfer, background synthesis, hair editing) that can be deployed locally or in the cloud.

Result: The system enables efficient generation of high-quality virtual KOL images and dynamic composition of promotional visuals through flexible, interchangeable AI services.

Conclusion: GenKOL significantly streamlines branded content production, lowers costs, and accelerates marketing workflows through scalable virtual KOL creation with adaptable architecture for diverse use cases.

Abstract: Key Opinion Leader (KOL) play a crucial role in modern marketing by shaping consumer perceptions and enhancing brand credibility. However, collaborating with human KOLs often involves high costs and logistical challenges. To address this, we present GenKOL, an interactive system that empowers marketing professionals to efficiently generate high-quality virtual KOL images using generative AI. GenKOL enables users to dynamically compose promotional visuals through an intuitive interface that integrates multiple AI capabilities, including garment generation, makeup transfer, background synthesis, and hair editing. These capabilities are implemented as modular, interchangeable services that can be deployed flexibly on local machines or in the cloud. This modular architecture ensures adaptability across diverse use cases and computational environments. Our system can significantly streamline the production of branded content, lowering costs and accelerating marketing workflows through scalable virtual KOL creation.

Method: The proposed DF-LLaVA framework extracts latent knowledge from MLLMs and injects it into training via prompts, unlocking the intrinsic discrimination potential of MLLMs to achieve both accuracy and interpretability.

Result: Extensive experiments show DF-LLaVA achieves outstanding detection accuracy exceeding expert models while maintaining the interpretability offered by MLLMs, demonstrating superiority in both accuracy and explainability.

Conclusion: DF-LLaVA successfully addresses the limitations of existing synthetic image detection methods by providing a framework that combines high accuracy with human-interpretable results, making it effective for evaluating image authenticity and locating forgeries.

Abstract: With the increasing prevalence of synthetic images, evaluating image authenticity and locating forgeries accurately while maintaining human interpretability remains a challenging task. Existing detection models primarily focus on simple authenticity classification, ultimately providing only a forgery probability or binary judgment, which offers limited explanatory insights into image authenticity. Moreover, while MLLM-based detection methods can provide more interpretable results, they still lag behind expert models in terms of pure authenticity classification accuracy. To address this, we propose DF-LLaVA, a simple yet effective framework that unlocks the intrinsic discrimination potential of MLLMs. Our approach first extracts latent knowledge from MLLMs and then injects it into training via prompts. This framework allows LLaVA to achieve outstanding detection accuracy exceeding expert models while still maintaining the interpretability offered by MLLMs. Extensive experiments confirm the superiority of our DF-LLaVA, achieving both high accuracy and explainability in synthetic image detection. Code is available online at: https://github.com/Eliot-Shen/DF-LLaVA.

Method: Proposes Cross-Modal Geometric Rectification (CMGR) with Structure-Aware Geometric Rectification module for hierarchical alignment, Texture Amplification Module for discriminative textures, and Base-Novel Discriminator to isolate geometric variations.

Result: Extensive experiments show significant improvement in 3D few-shot class-incremental learning, achieving superior geometric coherence and robustness to texture bias across different settings.

Conclusion: CMGR effectively addresses geometric misalignment and texture bias in 3D class-incremental learning, providing stable prototypes and enhanced cross-modal consistency for open-world scenarios.

Abstract: The rapid growth of 3D digital content necessitates expandable recognition systems for open-world scenarios. However, existing 3D class-incremental learning methods struggle under extreme data scarcity due to geometric misalignment and texture bias. While recent approaches integrate 3D data with 2D foundation models (e.g., CLIP), they suffer from semantic blurring caused by texture-biased projections and indiscriminate fusion of geometric-textural cues, leading to unstable decision prototypes and catastrophic forgetting. To address these issues, we propose Cross-Modal Geometric Rectification (CMGR), a framework that enhances 3D geometric fidelity by leveraging CLIP’s hierarchical spatial semantics. Specifically, we introduce a Structure-Aware Geometric Rectification module that hierarchically aligns 3D part structures with CLIP’s intermediate spatial priors through attention-driven geometric fusion. Additionally, a Texture Amplification Module synthesizes minimal yet discriminative textures to suppress noise and reinforce cross-modal consistency. To further stabilize incremental prototypes, we employ a Base-Novel Discriminator that isolates geometric variations. Extensive experiments demonstrate that our method significantly improves 3D few-shot class-incremental learning, achieving superior geometric coherence and robustness to texture bias across cross-domain and within-domain settings.

Method: Proposes Brain-HGCN using hyperbolic geometry (Lorentz model) with novel hyperbolic graph attention layer featuring signed aggregation for excitatory/inhibitory connections, and geometrically sound Fréchet mean for graph readout.

Result: Significantly outperforms state-of-the-art Euclidean baselines on two large-scale fMRI datasets for psychiatric disorder classification.

Conclusion: This work pioneers hyperbolic GNNs for fMRI analysis, demonstrating their immense potential in computational psychiatry for modeling brain network hierarchy with high fidelity.

Abstract: Functional magnetic resonance imaging (fMRI) provides a powerful non-invasive window into the brain’s functional organization by generating complex functional networks, typically modeled as graphs. These brain networks exhibit a hierarchical topology that is crucial for cognitive processing. However, due to inherent spatial constraints, standard Euclidean GNNs struggle to represent these hierarchical structures without high distortion, limiting their clinical performance. To address this limitation, we propose Brain-HGCN, a geometric deep learning framework based on hyperbolic geometry, which leverages the intrinsic property of negatively curved space to model the brain’s network hierarchy with high fidelity. Grounded in the Lorentz model, our model employs a novel hyperbolic graph attention layer with a signed aggregation mechanism to distinctly process excitatory and inhibitory connections, ultimately learning robust graph-level representations via a geometrically sound Fr'echet mean for graph readout. Experiments on two large-scale fMRI datasets for psychiatric disorder classification demonstrate that our approach significantly outperforms a wide range of state-of-the-art Euclidean baselines. This work pioneers a new geometric deep learning paradigm for fMRI analysis, highlighting the immense potential of hyperbolic GNNs in the field of computational psychiatry.

Method: A two-stage framework: first stage uses a large vision model for 2D feature extraction and candidate ranking; second stage includes a 3D-feature-awareness module that predicts when 3D re-ranking is needed, and a robot 3D retrieval transformer with geometry-aware dense features and keypoint-based matching instead of cosine similarity.

Result: RoboEye improves Recall@1 by 7.1% over the previous state-of-the-art (RoboLLM) and operates using only RGB images, avoiding reliance on explicit 3D inputs and reducing deployment costs.

Conclusion: The proposed framework effectively bridges the training-deployment gap in warehouse object identification by dynamically augmenting 2D features with 3D reasoning, achieving significant performance improvements while maintaining practical deployment feasibility with RGB-only inputs.

Abstract: The rapidly growing number of product categories in large-scale e-commerce makes accurate object identification for automated packing in warehouses substantially more difficult. As the catalog grows, intra-class variability and a long tail of rare or visually similar items increase, and when combined with diverse packaging, cluttered containers, frequent occlusion, and large viewpoint changes-these factors amplify discrepancies between query and reference images, causing sharp performance drops for methods that rely solely on 2D appearance features. Thus, we propose RoboEye, a two-stage identification framework that dynamically augments 2D semantic features with domain-adapted 3D reasoning and lightweight adapters to bridge training deployment gaps. In the first stage, we train a large vision model to extract 2D features for generating candidate rankings. A lightweight 3D-feature-awareness module then estimates 3D feature quality and predicts whether 3D re-ranking is necessary, preventing performance degradation and avoiding unnecessary computation. When invoked, the second stage uses our robot 3D retrieval transformer, comprising a 3D feature extractor that produces geometry-aware dense features and a keypoint-based matcher that computes keypoint-correspondence confidences between query and reference images instead of conventional cosine-similarity scoring. Experiments show that RoboEye improves Recall@1 by 7.1% over the prior state of the art (RoboLLM). Moreover, RoboEye operates using only RGB images, avoiding reliance on explicit 3D inputs and reducing deployment costs. The code used in this paper is publicly available at: https://github.com/longkukuhi/RoboEye.

Method: Proposes dual-stream masking with 3D Spatial Grid Masking (creates structured patterns through coordinate sorting) and Progressive Semantic Masking (uses attention-driven clustering to discover semantically meaningful parts). These are orchestrated via curriculum learning with dynamic weighting, progressing from geometric understanding to semantic discovery.

Result: Comprehensive experiments on ModelNet40, ScanObjectNN, and OmniObject3D demonstrate consistent improvements across various rotation scenarios, showing substantial performance gains over baseline rotation-invariant methods.

Conclusion: The proposed dual-stream masking approach effectively addresses fundamental limitations of random masking in rotation-invariant point cloud MAE, providing plug-and-play components that integrate into existing frameworks without architectural changes while ensuring broad compatibility and performance improvements.

Abstract: Existing rotation-invariant point cloud masked autoencoders (MAE) rely on random masking strategies that overlook geometric structure and semantic coherence. Random masking treats patches independently, failing to capture spatial relationships consistent across orientations and overlooking semantic object parts that maintain identity regardless of rotation. We propose a dual-stream masking approach combining 3D Spatial Grid Masking and Progressive Semantic Masking to address these fundamental limitations. Grid masking creates structured patterns through coordinate sorting to capture geometric relationships that persist across different orientations, while semantic masking uses attention-driven clustering to discover semantically meaningful parts and maintain their coherence during masking. These complementary streams are orchestrated via curriculum learning with dynamic weighting, progressing from geometric understanding to semantic discovery. Designed as plug-and-play components, our strategies integrate into existing rotation-invariant frameworks without architectural changes, ensuring broad compatibility across different approaches. Comprehensive experiments on ModelNet40, ScanObjectNN, and OmniObject3D demonstrate consistent improvements across various rotation scenarios, showing substantial performance gains over the baseline rotation-invariant methods.

Method: Proposed EchoVLM with Mixture of Experts (MoE) architecture trained on data from seven anatomical regions, enabling multiple tasks including ultrasound report generation, diagnosis, and visual question-answering.

Result: Achieved significant improvements of 10.15 and 4.77 points in BLEU-1 and ROUGE-1 scores respectively compared to Qwen2-VL on ultrasound report generation task.

Conclusion: EchoVLM shows substantial potential to enhance diagnostic accuracy in ultrasound imaging and provides a viable technical solution for future clinical applications.

Abstract: Ultrasound imaging has become the preferred imaging modality for early cancer screening due to its advantages of non-ionizing radiation, low cost, and real-time imaging capabilities. However, conventional ultrasound diagnosis heavily relies on physician expertise, presenting challenges of high subjectivity and low diagnostic efficiency. Vision-language models (VLMs) offer promising solutions for this issue, but existing general-purpose models demonstrate limited knowledge in ultrasound medical tasks, with poor generalization in multi-organ lesion recognition and low efficiency across multi-task diagnostics. To address these limitations, we propose EchoVLM, a vision-language model specifically designed for ultrasound medical imaging. The model employs a Mixture of Experts (MoE) architecture trained on data spanning seven anatomical regions. This design enables the model to perform multiple tasks, including ultrasound report generation, diagnosis and visual question-answering (VQA). The experimental results demonstrated that EchoVLM achieved significant improvements of 10.15 and 4.77 points in BLEU-1 scores and ROUGE-1 scores respectively compared to Qwen2-VL on the ultrasound report generation task. These findings suggest that EchoVLM has substantial potential to enhance diagnostic accuracy in ultrasound imaging, thereby providing a viable technical solution for future clinical applications. Source code and model weights are available at https://github.com/Asunatan/EchoVLM.

Method: Introduced a large synthetic dataset with 12,328 structured annotated scenes and developed SpatialGen, a multi-view multi-modal diffusion model that generates appearance (color image), geometry (scene coordinate map), and semantic (segmentation map) from arbitrary viewpoints given a 3D layout and reference image.

Result: SpatialGen consistently generates superior results compared to previous methods, producing realistic and semantically consistent 3D indoor scenes while preserving spatial consistency across different modalities.

Conclusion: The open-sourced dataset and SpatialGen model advance indoor scene understanding and generation, providing the community with powerful tools for creating high-fidelity 3D models with improved visual quality and semantic coherence.

Abstract: Creating high-fidelity 3D models of indoor environments is essential for applications in design, virtual reality, and robotics. However, manual 3D modeling remains time-consuming and labor-intensive. While recent advances in generative AI have enabled automated scene synthesis, existing methods often face challenges in balancing visual quality, diversity, semantic consistency, and user control. A major bottleneck is the lack of a large-scale, high-quality dataset tailored to this task. To address this gap, we introduce a comprehensive synthetic dataset, featuring 12,328 structured annotated scenes with 57,440 rooms, and 4.7M photorealistic 2D renderings. Leveraging this dataset, we present SpatialGen, a novel multi-view multi-modal diffusion model that generates realistic and semantically consistent 3D indoor scenes. Given a 3D layout and a reference image (derived from a text prompt), our model synthesizes appearance (color image), geometry (scene coordinate map), and semantic (semantic segmentation map) from arbitrary viewpoints, while preserving spatial consistency across modalities. SpatialGen consistently generates superior results to previous methods in our experiments. We are open-sourcing our data and models to empower the community and advance the field of indoor scene understanding and generation.

Method: Three-stage approach: 1) SigLIP vision-language model retrieves top 35 semantically similar products, 2) YOLO-E segmentation removes background clutter, 3) LightGlue performs fine-grained pixel-level matching on filtered candidates.

Result: PRISM outperforms state-of-the-art image retrieval methods by 4.21% in top-1 accuracy on the ABV dataset while remaining within real-time processing bounds for practical retail deployments.

Conclusion: The hybrid approach successfully leverages both semantic understanding and fine-grained visual matching to address retail product retrieval challenges, providing both accuracy and efficiency for practical applications.

Abstract: Compared to traditional image retrieval tasks, product retrieval in retail settings is even more challenging. Products of the same type from different brands may have highly similar visual appearances, and the query image may be taken from an angle that differs significantly from view angles of the stored catalog images. Foundational models, such as CLIP and SigLIP, often struggle to distinguish these subtle but important local differences. Pixel-wise matching methods, on the other hand, are computationally expensive and incur prohibitively high matching times. In this paper, we propose a new, hybrid method, called PRISM, for product retrieval in retail settings by leveraging the advantages of both vision-language model-based and pixel-wise matching approaches. To provide both efficiency/speed and finegrained retrieval accuracy, PRISM consists of three stages: 1) A vision-language model (SigLIP) is employed first to retrieve the top 35 most semantically similar products from a fixed gallery, thereby narrowing the search space significantly; 2) a segmentation model (YOLO-E) is applied to eliminate background clutter; 3) fine-grained pixel-level matching is performed using LightGlue across the filtered candidates. This framework enables more accurate discrimination between products with high inter-class similarity by focusing on subtle visual cues often missed by global models. Experiments performed on the ABV dataset show that our proposed PRISM outperforms the state-of-the-art image retrieval methods by 4.21% in top-1 accuracy while still remaining within the bounds of real-time processing for practical retail deployments.

Method: End-to-end monocular model with temporal correlation layer trained on synthetic data for joint wire segmentation and depth estimation.

Result: Superior performance over existing competitive approaches in the joint task of wire detection and depth estimation.

Conclusion: The model shows strong potential to enhance autonomous drone safety and precision in real-world applications through effective wire obstacle detection.

Abstract: In the realm of fully autonomous drones, the accurate detection of obstacles is paramount to ensure safe navigation and prevent collisions. Among these challenges, the detection of wires stands out due to their slender profile, which poses a unique and intricate problem. To address this issue, we present an innovative solution in the form of a monocular end-to-end model for wire segmentation and depth estimation. Our approach leverages a temporal correlation layer trained on synthetic data, providing the model with the ability to effectively tackle the complex joint task of wire detection and depth estimation. We demonstrate the superiority of our proposed method over existing competitive approaches in the joint task of wire detection and depth estimation. Our results underscore the potential of our model to enhance the safety and precision of autonomous drones, shedding light on its promising applications in real-world scenarios.

Method: Proposed improved synthetic data generation pipeline incorporating forward scattering term and nonuniform medium modeling. Collected BUCKET dataset with controlled turbidity conditions for real turbid footage with reference images.

Result: Demonstrated qualitative improvements over reference model, especially under increasing turbidity conditions. Survey participants preferred the proposed approach with 82.5% selection rate.

Conclusion: The inclusion of forward scattering and nonuniform medium modeling significantly improves synthetic underwater data generation for turbid environments, as validated by both qualitative results and human preference surveys.

Abstract: In recent years, the underwater image formation model has found extensive use in the generation of synthetic underwater data. Although many approaches focus on scenes primarily affected by discoloration, they often overlook the model’s ability to capture the complex, distance-dependent visibility loss present in highly turbid environments. In this work, we propose an improved synthetic data generation pipeline that includes the commonly omitted forward scattering term, while also considering a nonuniform medium. Additionally, we collected the BUCKET dataset under controlled turbidity conditions to acquire real turbid footage with the corresponding reference images. Our results demonstrate qualitative improvements over the reference model, particularly under increasing turbidity, with a selection rate of 82. 5% by survey participants. Data and code can be accessed on the project page: vap.aau.dk/sea-ing-through-scattered-rays.

Method: Proposes AdaMM with three modules: Graph-guided Adaptive Refinement for semantic associations, Bi-Bottleneck Distillation for knowledge transfer via style matching and adversarial alignment, and Lesion-Presence-Guided Reliability for suppressing false positives.

Result: Extensive experiments on BraTS 2018 and 2024 datasets show AdaMM outperforms existing methods, particularly in single-modality and weak-modality scenarios, with superior segmentation accuracy and robustness.

Conclusion: AdaMM provides an effective solution for missing-modality brain tumor segmentation, with systematic evaluation confirming knowledge distillation’s superiority and offering practical guidance for future research.

Abstract: Accurate brain tumor segmentation is essential for preoperative evaluation and personalized treatment. Multi-modal MRI is widely used due to its ability to capture complementary tumor features across different sequences. However, in clinical practice, missing modalities are common, limiting the robustness and generalizability of existing deep learning methods that rely on complete inputs, especially under non-dominant modality combinations. To address this, we propose AdaMM, a multi-modal brain tumor segmentation framework tailored for missing-modality scenarios, centered on knowledge distillation and composed of three synergistic modules. The Graph-guided Adaptive Refinement Module explicitly models semantic associations between generalizable and modality-specific features, enhancing adaptability to modality absence. The Bi-Bottleneck Distillation Module transfers structural and textural knowledge from teacher to student models via global style matching and adversarial feature alignment. The Lesion-Presence-Guided Reliability Module predicts prior probabilities of lesion types through an auxiliary classification task, effectively suppressing false positives under incomplete inputs. Extensive experiments on the BraTS 2018 and 2024 datasets demonstrate that AdaMM consistently outperforms existing methods, exhibiting superior segmentation accuracy and robustness, particularly in single-modality and weak-modality configurations. In addition, we conduct a systematic evaluation of six categories of missing-modality strategies, confirming the superiority of knowledge distillation and offering practical guidance for method selection and future research. Our source code is available at https://github.com/Quanato607/AdaMM.

Method: Proposes a reinforcement learning framework that treats hyperparameter optimization as a sequential decision-making task using a Markov Decision Process. The method dynamically adjusts hyperparameters across denoising steps and integrates editing objectives into a reward function, using proximal policy optimization for time efficiency.

Result: The approach significantly reduces search time and computational overhead compared to brute-force methods while maintaining optimal hyperparameter configurations.

Conclusion: The reinforcement learning framework advances practical deployment of diffusion-based image editing by making hyperparameter optimization more efficient and computationally feasible.

Abstract: Recent advances in diffusion models have revolutionized text-guided image editing, yet existing editing methods face critical challenges in hyperparameter identification. To get the reasonable editing performance, these methods often require the user to brute-force tune multiple interdependent hyperparameters, such as inversion timesteps and attention modification, \textit{etc.} This process incurs high computational costs due to the huge hyperparameter search space. We consider searching optimal editing’s hyperparameters as a sequential decision-making task within the diffusion denoising process. Specifically, we propose a reinforcement learning framework, which establishes a Markov Decision Process that dynamically adjusts hyperparameters across denoising steps, integrating editing objectives into a reward function. The method achieves time efficiency through proximal policy optimization while maintaining optimal hyperparameter configurations. Experiments demonstrate significant reduction in search time and computational overhead compared to existing brute-force approaches, advancing the practical deployment of a diffusion-based image editing framework in the real world.

Method: Extensive experimentation with YOLOv11 model using synthetic data, domain randomization strategies, data augmentation, dataset composition variations, and model scaling. Evaluation through both quantitative metrics (mAP@50) and qualitative visual inspection.

Result: Best performing YOLOv11l model achieved 0.910 mAP@50 on real-world test set. Found that increased synthetic dataset diversity (varied perspectives, complex backgrounds) and carefully tuned data augmentation were crucial for bridging domain gap.

Conclusion: Synthetic-only training approach shows significant potential for object detection, but synthetic validation metrics are poor predictors of real-world performance. Careful dataset design and augmentation are essential to overcome domain gap challenges.

Abstract: This paper addresses the synthetic-to-real domain gap in object detection, focusing on training a YOLOv11 model to detect a specific object (a soup can) using only synthetic data and domain randomization strategies. The methodology involves extensive experimentation with data augmentation, dataset composition, and model scaling. While synthetic validation metrics were consistently high, they proved to be poor predictors of real-world performance. Consequently, models were also evaluated qualitatively, through visual inspection of predictions, and quantitatively, on a manually labeled real-world test set, to guide development. Final mAP@50 scores were provided by the official Kaggle competition. Key findings indicate that increasing synthetic dataset diversity, specifically by including varied perspectives and complex backgrounds, combined with carefully tuned data augmentation, were crucial in bridging the domain gap. The best performing configuration, a YOLOv11l model trained on an expanded and diverse dataset, achieved a final mAP@50 of 0.910 on the competition’s hidden test set. This result demonstrates the potential of a synthetic-only training approach while also highlighting the remaining challenges in fully capturing real-world variability.

Method: Retrospective study of 32 patients (3,645 CT slices) using three deep learning models with quantitative metrics (volumetric similarity, Dice coefficient, Hausdorff distance) and qualitative clinical acceptability scoring.

Result: Unet-R231 consistently outperformed other models across all severity levels and pathology categories. All models showed significant performance declines from mild to moderate-to-severe cases, particularly in volumetric similarity.

Conclusion: Unet-R231 provides the most accurate automated segmentation, but specialized model fine-tuning is needed for severe pathology cases as performance declines significantly in moderate-to-severe conditions.

Abstract: This study evaluates publicly available deep-learning based lung segmentation models in transplant-eligible patients to determine their performance across disease severity levels, pathology categories, and lung sides, and to identify limitations impacting their use in preoperative planning in lung transplantation. This retrospective study included 32 patients who underwent chest CT scans at Duke University Health System between 2017 and 2019 (total of 3,645 2D axial slices). Patients with standard axial CT scans were selected based on the presence of two or more lung pathologies of varying severity. Lung segmentation was performed using three previously developed deep learning models: Unet-R231, TotalSegmentator, MedSAM. Performance was assessed using quantitative metrics (volumetric similarity, Dice similarity coefficient, Hausdorff distance) and a qualitative measure (four-point clinical acceptability scale). Unet-R231 consistently outperformed TotalSegmentator and MedSAM in general, for different severity levels, and pathology categories (p<0.05). All models showed significant performance declines from mild to moderate-to-severe cases, particularly in volumetric similarity (p<0.05), without significant differences among lung sides or pathology types. Unet-R231 provided the most accurate automated lung segmentation among evaluated models with TotalSegmentator being a close second, though their performance declined significantly in moderate-to-severe cases, emphasizing the need for specialized model fine-tuning in severe pathology contexts.

Method: 1) Created ImageNeXt dataset with five visual modalities based on ImageNet 2) Developed efficient pretraining to encode different modality information 3) Universal framework that works with arbitrary modality combinations

Result: Achieved new state-of-the-art records on NYU Depthv2, EventScape, MFNet, DeLiVER, SUNRGBD, and KITTI-360 datasets

Conclusion: First universal multi-modal pretraining framework that consistently enhances model’s perceptual capabilities across various scenarios with different modality combinations

Abstract: Recent research on representation learning has proved the merits of multi-modal clues for robust semantic segmentation. Nevertheless, a flexible pretrain-and-finetune pipeline for multiple visual modalities remains unexplored. In this paper, we propose a novel multi-modal learning framework, termed OmniSegmentor. It has two key innovations: 1) Based on ImageNet, we assemble a large-scale dataset for multi-modal pretraining, called ImageNeXt, which contains five popular visual modalities. 2) We provide an efficient pretraining manner to endow the model with the capacity to encode different modality information in the ImageNeXt. For the first time, we introduce a universal multi-modal pretraining framework that consistently amplifies the model’s perceptual capabilities across various scenarios, regardless of the arbitrary combination of the involved modalities. Remarkably, our OmniSegmentor achieves new state-of-the-art records on a wide range of multi-modal semantic segmentation datasets, including NYU Depthv2, EventScape, MFNet, DeLiVER, SUNRGBD, and KITTI-360.

Method: Three key components: (1) Patch-wise Tracking Filters for robust sparse hinge-like relations, (2) Outlier-aware Joint Optimization with adaptive down-weighting of moving outliers, (3) Two-stage Optimization Strategy balancing Softplus limits and convex minima.

Result: Method estimates camera parameters more efficiently and accurately than COLMAP, validated on 4 real-world datasets (NeRF-DS, DAVIS, iPhone, TUM-dynamics) and 1 synthetic dataset (MPI-Sintel) using only RGB video supervision.

Conclusion: Proposed approach enables accurate camera parameter optimization for dynamic scenes with minimal supervision requirements, demonstrating superior performance over traditional methods like COLMAP.

Abstract: Although COLMAP has long remained the predominant method for camera parameter optimization in static scenes, it is constrained by its lengthy runtime and reliance on ground truth (GT) motion masks for application to dynamic scenes. Many efforts attempted to improve it by incorporating more priors as supervision such as GT focal length, motion masks, 3D point clouds, camera poses, and metric depth, which, however, are typically unavailable in casually captured RGB videos. In this paper, we propose a novel method for more accurate and efficient camera parameter optimization in dynamic scenes solely supervised by a single RGB video. Our method consists of three key components: (1) Patch-wise Tracking Filters, to establish robust and maximally sparse hinge-like relations across the RGB video. (2) Outlier-aware Joint Optimization, for efficient camera parameter optimization by adaptive down-weighting of moving outliers, without reliance on motion priors. (3) A Two-stage Optimization Strategy, to enhance stability and optimization speed by a trade-off between the Softplus limits and convex minima in losses. We visually and numerically evaluate our camera estimates. To further validate accuracy, we feed the camera estimates into a 4D reconstruction method and assess the resulting 3D scenes, and rendered 2D RGB and depth maps. We perform experiments on 4 real-world datasets (NeRF-DS, DAVIS, iPhone, and TUM-dynamics) and 1 synthetic dataset (MPI-Sintel), demonstrating that our method estimates camera parameters more efficiently and accurately with a single RGB video as the only supervision.

Method: Two-stage framework: 1) Pseudo-label supervised fine-tuning (SFT) to incorporate external factual expertise, 2) Group Relative Policy Optimization (GRPO) with four tailored factual reward signals to encourage self-consistent reasoning.

Result: Achieves up to 22.5% absolute improvement in factual accuracy over previous state-of-the-art methods across three public medical QA benchmarks.

Conclusion: The synergy between knowledge grounding and RL-driven reasoning is essential for trustworthy medical AI, with ablation studies validating the necessity of pseudo-label SFT cold start and the contribution of each GRPO reward.

Abstract: Ensuring factual consistency and reliable reasoning remains a critical challenge for medical vision-language models. We introduce MEDFACT-R1, a two-stage framework that integrates external knowledge grounding with reinforcement learning to improve the factual medical reasoning. The first stage uses pseudo-label supervised fine-tuning (SFT) to incorporate external factual expertise; while the second stage applies Group Relative Policy Optimization (GRPO) with four tailored factual reward signals to encourage self-consistent reasoning. Across three public medical QA benchmarks, MEDFACT-R1 delivers up to 22.5% absolute improvement in factual accuracy over previous state-of-the-art methods. Ablation studies highlight the necessity of pseudo-label SFT cold start and validate the contribution of each GRPO reward, underscoring the synergy between knowledge grounding and RL-driven reasoning for trustworthy medical AI. Codes are released at https://github.com/Garfieldgengliang/MEDFACT-R1.

Method: Created a synthetic parametric geometric-illusion dataset and evaluated three multi-source learning strategies that combine illusion recognition tasks with ImageNet classification objectives on both CNN and transformer architectures.

Result: Incorporating geometric illusions as auxiliary supervision systematically improves generalization, especially for challenging visual cases with intricate contours and fine textures. Perceptually driven biases enhanced structural sensitivity across architectures.

Conclusion: The study demonstrates successful integration of perceptual science with machine learning, showing that perceptually motivated inductive biases from synthetic stimuli can enhance vision model performance and suggesting new directions for embedding perceptual priors.

Abstract: Contemporary deep learning models have achieved impressive performance in image classification by primarily leveraging statistical regularities within large datasets, but they rarely incorporate structured insights drawn directly from perceptual psychology. To explore the potential of perceptually motivated inductive biases, we propose integrating classic geometric visual illusions well-studied phenomena from human perception into standard image-classification training pipelines. Specifically, we introduce a synthetic, parametric geometric-illusion dataset and evaluate three multi-source learning strategies that combine illusion recognition tasks with ImageNet classification objectives. Our experiments reveal two key conceptual insights: (i) incorporating geometric illusions as auxiliary supervision systematically improves generalization, especially in visually challenging cases involving intricate contours and fine textures; and (ii) perceptually driven inductive biases, even when derived from synthetic stimuli traditionally considered unrelated to natural image recognition, can enhance the structural sensitivity of both CNN and transformer-based architectures. These results demonstrate a novel integration of perceptual science and machine learning and suggest new directions for embedding perceptual priors into vision model design.

Method: A genetic algorithm-based joint optimization that evolves adversarial prompts by balancing attack success, clean-task utility, and stealthiness. Uses diverse query generation to simulate realistic linguistic variations and ensure robustness across query paraphrases.

Result: AIP achieves up to 95.23% attack success rate (ASR) while preserving benign functionality, demonstrating high effectiveness in manipulating RAG outputs covertly.

Conclusion: The study reveals a critical vulnerability in RAG systems through instructional prompt manipulation, emphasizing the need to reassess shared instructional prompts and their security implications.

Abstract: Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by retrieving relevant documents from external sources to improve factual accuracy and verifiability. However, this reliance introduces new attack surfaces within the retrieval pipeline, beyond the LLM itself. While prior RAG attacks have exposed such vulnerabilities, they largely rely on manipulating user queries, which is often infeasible in practice due to fixed or protected user inputs. This narrow focus overlooks a more realistic and stealthy vector: instructional prompts, which are widely reused, publicly shared, and rarely audited. Their implicit trust makes them a compelling target for adversaries to manipulate RAG behavior covertly. We introduce a novel attack for Adversarial Instructional Prompt (AIP) that exploits adversarial instructional prompts to manipulate RAG outputs by subtly altering retrieval behavior. By shifting the attack surface to the instructional prompts, AIP reveals how trusted yet seemingly benign interface components can be weaponized to degrade system integrity. The attack is crafted to achieve three goals: (1) naturalness, to evade user detection; (2) utility, to encourage use of prompts; and (3) robustness, to remain effective across diverse query variations. We propose a diverse query generation strategy that simulates realistic linguistic variation in user queries, enabling the discovery of prompts that generalize across paraphrases and rephrasings. Building on this, a genetic algorithm-based joint optimization is developed to evolve adversarial prompts by balancing attack success, clean-task utility, and stealthiness. Experimental results show that AIP achieves up to 95.23% ASR while preserving benign functionality. These findings uncover a critical and previously overlooked vulnerability in RAG systems, emphasizing the need to reassess the shared instructional prompts.

Method: Proposes M&N framework with iterative co-training between 2D pretrained model and 3D segmentation model using pseudo-masks, plus learning rate guided sampling to adaptively adjust labeled/unlabeled data proportion based on prediction accuracy and stability.

Result: Achieves state-of-the-art performance on multiple datasets, outperforming 13 existing semi-supervised segmentation approaches across all settings, while remaining model-agnostic for integration with different architectures.

Conclusion: M&N effectively transfers knowledge from 2D to 3D medical image segmentation, demonstrating strong performance in semi-supervised settings with model-agnostic adaptability for future advanced models.

Abstract: This paper explores the transfer of knowledge from general vision models pretrained on 2D natural images to improve 3D medical image segmentation. We focus on the semi-supervised setting, where only a few labeled 3D medical images are available, along with a large set of unlabeled images. To tackle this, we propose a model-agnostic framework that progressively distills knowledge from a 2D pretrained model to a 3D segmentation model trained from scratch. Our approach, M&N, involves iterative co-training of the two models using pseudo-masks generated by each other, along with our proposed learning rate guided sampling that adaptively adjusts the proportion of labeled and unlabeled data in each training batch to align with the models’ prediction accuracy and stability, minimizing the adverse effect caused by inaccurate pseudo-masks. Extensive experiments on multiple publicly available datasets demonstrate that M&N achieves state-of-the-art performance, outperforming thirteen existing semi-supervised segmentation approaches under all different settings. Importantly, ablation studies show that M&N remains model-agnostic, allowing seamless integration with different architectures. This ensures its adaptability as more advanced models emerge. The code is available at https://github.com/pakheiyeung/M-N.

Method: Proposed RA-GAN with two new modules (RACEpSp and feature mixer) to generate racially unbiased aged images for same-age parent-child kinship verification.

Result: RA-GAN outperforms SAM-GAN by 13.14% on average and CUSP-GAN by 9.1% in 60+ age group for racial accuracy. Improves kinship verification accuracy across all relationships on KinFaceW datasets (up to 5.22% improvement).

Conclusion: The proposed racially unbiased face aging model successfully enhances kinship verification accuracy by addressing racial bias and age gap issues while better preserving subject identities.

Abstract: The age gap in kinship verification addresses the time difference between the photos of the parent and the child. Moreover, their same-age photos are often unavailable, and face aging models are racially biased, which impacts the likeness of photos. Therefore, we propose a face aging GAN model, RA-GAN, consisting of two new modules, RACEpSp and a feature mixer, to produce racially unbiased images. The unbiased synthesized photos are used in kinship verification to investigate the results of verifying same-age parent-child images. The experiments demonstrate that our RA-GAN outperforms SAM-GAN on an average of 13.14% across all age groups, and CUSP-GAN in the 60+ age group by 9.1% in terms of racial accuracy. Moreover, RA-GAN can preserve subjects’ identities better than SAM-GAN and CUSP-GAN across all age groups. Additionally, we demonstrate that transforming parent and child images from the KinFaceW-I and KinFaceW-II datasets to the same age can enhance the verification accuracy across all age groups. The accuracy increases with our RA-GAN for the kinship relationships of father-son and father-daughter, mother-son, and mother-daughter, which are 5.22, 5.12, 1.63, and 0.41, respectively, on KinFaceW-I. Additionally, the accuracy for the relationships of father-daughter, father-son, and mother-son is 2.9, 0.39, and 1.6 on KinFaceW-II, respectively. The code is available at~\href{https://github.com/bardiya2254kariminia/An-Age-Transformation-whitout-racial-bias-for-Kinship-verification}{Github}

Method: Uses decomposed spatio-temporal highlighting (DSTH) with attribute/action sub-queries and logit-guided re-attention module, plus temporal-augmented assembling (TAS) strategy for temporal consistency.

Result: Outperforms state-of-the-art methods on three common STVG benchmarks across various MLLMs.

Conclusion: The proposed zero-shot framework effectively enhances MLLMs’ reasoning ability for spatio-temporal video grounding through novel decomposition and temporal consistency strategies.

Abstract: Spatio-temporal video grounding (STVG) aims at localizing the spatio-temporal tube of a video, as specified by the input text query. In this paper, we utilize multimodal large language models (MLLMs) to explore a zero-shot solution in STVG. We reveal two key insights about MLLMs: (1) MLLMs tend to dynamically assign special tokens, referred to as \textit{grounding tokens}, for grounding the text query; and (2) MLLMs often suffer from suboptimal grounding due to the inability to fully integrate the cues in the text query (\textit{e.g.}, attributes, actions) for inference. Based on these insights, we propose a MLLM-based zero-shot framework for STVG, which includes novel decomposed spatio-temporal highlighting (DSTH) and temporal-augmented assembling (TAS) strategies to unleash the reasoning ability of MLLMs. The DSTH strategy first decouples the original query into attribute and action sub-queries for inquiring the existence of the target both spatially and temporally. It then uses a novel logit-guided re-attention (LRA) module to learn latent variables as spatial and temporal prompts, by regularizing token predictions for each sub-query. These prompts highlight attribute and action cues, respectively, directing the model’s attention to reliable spatial and temporal related visual regions. In addition, as the spatial grounding by the attribute sub-query should be temporally consistent, we introduce the TAS strategy to assemble the predictions using the original video frames and the temporal-augmented frames as inputs to help improve temporal consistency. We evaluate our method on various MLLMs, and show that it outperforms SOTA methods on three common STVG benchmarks. The code will be available at https://github.com/zaiquanyang/LLaVA_Next_STVG.

Method: The paper introduces MSDD dataset containing three classes (single, double, triple plants) captured under diverse conditions including different growth stages, planting setups, soil types, lighting, camera angles, and densities. Various YOLO models (including YOLO11 and YOLOv9) were benchmarked for detection performance.

Result: Detection is most reliable during V4-V6 growth stages and under nadir views. YOLO11 is the fastest model while YOLOv9 achieves highest accuracy for single plants (precision up to 0.984, recall up to 0.873). Multi-plant detection remains difficult due to rarity, irregular appearance from planting errors, and class imbalance. YOLO11 maintains efficient inference at 35ms per image.

Conclusion: MSDD establishes a strong foundation for developing models that enhance stand counting, optimize resource allocation, and support real-time decision-making in precision agriculture. The dataset represents a step toward automating agricultural monitoring and advancing precision farming practices.

Abstract: Accurate maize seedling detection is crucial for precision agriculture, yet curated datasets remain scarce. We introduce MSDD, a high-quality aerial image dataset for maize seedling stand counting, with applications in early-season crop monitoring, yield prediction, and in-field management. Stand counting determines how many plants germinated, guiding timely decisions such as replanting or adjusting inputs. Traditional methods are labor-intensive and error-prone, while computer vision enables efficient, accurate detection. MSDD contains three classes-single, double, and triple plants-capturing diverse growth stages, planting setups, soil types, lighting conditions, camera angles, and densities, ensuring robustness for real-world use. Benchmarking shows detection is most reliable during V4-V6 stages and under nadir views. Among tested models, YOLO11 is fastest, while YOLOv9 yields the highest accuracy for single plants. Single plant detection achieves precision up to 0.984 and recall up to 0.873, but detecting doubles and triples remains difficult due to rarity and irregular appearance, often from planting errors. Class imbalance further reduces accuracy in multi-plant detection. Despite these challenges, YOLO11 maintains efficient inference at 35 ms per image, with an additional 120 ms for saving outputs. MSDD establishes a strong foundation for developing models that enhance stand counting, optimize resource allocation, and support real-time decision-making. This dataset marks a step toward automating agricultural monitoring and advancing precision agriculture.

Method: Introduces self-supervised objectives during training to create Self-guided Training for AutoRegressive models (ST-AR), addressing the identified three key properties that hinder visual learning.

Result: ST-AR achieves approximately 42% FID improvement for LlamaGen-L and 49% FID improvement for LlamaGen-XL while maintaining the same sampling strategy.

Conclusion: Self-supervised objectives effectively address the limitations of autoregressive models in visual domains, enabling significant improvements in both image understanding and generation quality without external pre-trained models.

Abstract: Recent studies have demonstrated the importance of high-quality visual representations in image generation and have highlighted the limitations of generative models in image understanding. As a generative paradigm originally designed for natural language, autoregressive models face similar challenges. In this work, we present the first systematic investigation into the mechanisms of applying the next-token prediction paradigm to the visual domain. We identify three key properties that hinder the learning of high-level visual semantics: local and conditional dependence, inter-step semantic inconsistency, and spatial invariance deficiency. We show that these issues can be effectively addressed by introducing self-supervised objectives during training, leading to a novel training framework, Self-guided Training for AutoRegressive models (ST-AR). Without relying on pre-trained representation models, ST-AR significantly enhances the image understanding ability of autoregressive models and leads to improved generation quality. Specifically, ST-AR brings approximately 42% FID improvement for LlamaGen-L and 49% FID improvement for LlamaGen-XL, while maintaining the same sampling strategy.

Method: Uses two networks: an embedder that imperceptibly alters images, and an extractor that predicts geometric transformations. Both are end-to-end trained to minimize transformation prediction error, combined with a discriminator for perceptual quality.

Result: Experimental validation shows SyncSeal effectively synchronizes images across various geometric and valuemetric transformations, and successfully upgrades existing watermarking methods to withstand previously vulnerable transformations.

Conclusion: SyncSeal provides an effective synchronization solution that can be applied on top of existing watermarking methods, significantly improving their robustness against geometric attacks while maintaining image quality.

Abstract: Synchronization is the task of estimating and inverting geometric transformations (e.g., crop, rotation) applied to an image. This work introduces SyncSeal, a bespoke watermarking method for robust image synchronization, which can be applied on top of existing watermarking methods to enhance their robustness against geometric transformations. It relies on an embedder network that imperceptibly alters images and an extractor network that predicts the geometric transformation to which the image was subjected. Both networks are end-to-end trained to minimize the error between the predicted and ground-truth parameters of the transformation, combined with a discriminator to maintain high perceptual quality. We experimentally validate our method on a wide variety of geometric and valuemetric transformations, demonstrating its effectiveness in accurately synchronizing images. We further show that our synchronization can effectively upgrade existing watermarking methods to withstand geometric transformations to which they were previously vulnerable.

Method: Two-stage pretraining: 1) Ego-Centric Video Generative Pretraining on 12M videos for future frame prediction, 2) Human-Centric Trajectory-Aware Modeling for joint keypoint trajectory prediction. Uses ActionVAE for action sequence compression.

Result: Superior performance over state-of-the-art baselines when finetuned on downstream robotics datasets.

Conclusion: The proposed pretraining strategy provides a more effective initialization for VLA models, demonstrating the value of combining video generation with trajectory prediction.

Abstract: This paper presents RynnVLA-001, a vision-language-action(VLA) model built upon large-scale video generative pretraining from human demonstrations. We propose a novel two-stage pretraining methodology. The first stage, Ego-Centric Video Generative Pretraining, trains an Image-to-Video model on 12M ego-centric manipulation videos to predict future frames conditioned on an initial frame and a language instruction. The second stage, Human-Centric Trajectory-Aware Modeling, extends this by jointly predicting future keypoint trajectories, thereby effectively bridging visual frame prediction with action prediction. Furthermore, to enhance action representation, we propose ActionVAE, a variational autoencoder that compresses sequences of actions into compact latent embeddings, reducing the complexity of the VLA output space. When finetuned on the same downstream robotics datasets, RynnVLA-001 achieves superior performance over state-of-the-art baselines, demonstrating that the proposed pretraining strategy provides a more effective initialization for VLA models.

Method: Formulates depth refinement as a conditional diffusion process using a condition encoder, lightweight 2D U-Net with convolutional GRU, and confidence-based sampling strategy for adaptive depth hypothesis sampling.

Result: DiffMVS achieves competitive performance with state-of-the-art efficiency in runtime and GPU memory. CasDiffMVS achieves state-of-the-art performance on DTU, Tanks & Temples and ETH3D benchmarks.

Conclusion: The proposed diffusion-based MVS framework successfully integrates diffusion models into multi-view stereo, demonstrating both efficiency and state-of-the-art performance in 3D reconstruction tasks.

Abstract: To reconstruct the 3D geometry from calibrated images, learning-based multi-view stereo (MVS) methods typically perform multi-view depth estimation and then fuse depth maps into a mesh or point cloud. To improve the computational efficiency, many methods initialize a coarse depth map and then gradually refine it in higher resolutions. Recently, diffusion models achieve great success in generation tasks. Starting from a random noise, diffusion models gradually recover the sample with an iterative denoising process. In this paper, we propose a novel MVS framework, which introduces diffusion models in MVS. Specifically, we formulate depth refinement as a conditional diffusion process. Considering the discriminative characteristic of depth estimation, we design a condition encoder to guide the diffusion process. To improve efficiency, we propose a novel diffusion network combining lightweight 2D U-Net and convolutional GRU. Moreover, we propose a novel confidence-based sampling strategy to adaptively sample depth hypotheses based on the confidence estimated by diffusion model. Based on our novel MVS framework, we propose two novel MVS methods, DiffMVS and CasDiffMVS. DiffMVS achieves competitive performance with state-of-the-art efficiency in run-time and GPU memory. CasDiffMVS achieves state-of-the-art performance on DTU, Tanks & Temples and ETH3D. Code is available at: https://github.com/cvg/diffmvs.

Method: Built a closed-loop pipeline uniting automated agents with human experts to create large-scale dataset spanning multiple OS and domains, then trained ScaleCUA model on this data.

Result: Achieved strong gains over baselines (+26.6 on WebArena-Lite-v2, +10.7 on ScreenSpot-Pro) and new SOTA results (94.4% on MMBench-GUI L1-Hard, 60.6% on OSWorld-G, 47.4% on WebArena-Lite-v2).

Conclusion: Data-driven scaling is powerful for general-purpose computer use agents, and the release of data/models/code will advance future research.

Abstract: Vision-Language Models (VLMs) have enabled computer use agents (CUAs) that operate GUIs autonomously, showing great potential, yet progress is limited by the lack of large-scale, open-source computer use data and foundation models. In this work, we introduce ScaleCUA, a step toward scaling open-source CUAs. It offers a large-scale dataset spanning 6 operating systems and 3 task domains, built via a closed-loop pipeline uniting automated agents with human experts. Trained on this scaled-up data, ScaleCUA can operate seamlessly across platforms. Specifically, it delivers strong gains over baselines (+26.6 on WebArena-Lite-v2, +10.7 on ScreenSpot-Pro) and sets new state-of-the-art results (94.4% on MMBench-GUI L1-Hard, 60.6% on OSWorld-G, 47.4% on WebArena-Lite-v2). These findings underscore the power of data-driven scaling for general-purpose computer use agents. We will release data, models, and code to advance future research: https://github.com/OpenGVLab/ScaleCUA.

Method: Propose cross-modal distillation paradigm leveraging Vision Foundation Models (VFMs) to generate dense proxy labels from spatially aligned RGB frames. Adapt VFMs like Depth Anything v2 and develop novel recurrent architecture for monocular event camera depth inference.

Result: Evaluation on synthetic and real-world datasets shows competitive performance compared to fully supervised methods without requiring expensive depth annotations, achieving state-of-the-art performance with VFM-based models.

Conclusion: The cross-modal distillation approach effectively addresses the annotation limitation for event camera depth estimation and demonstrates that VFM-based models can achieve superior performance in monocular depth estimation from event streams.

Abstract: Event cameras capture sparse, high-temporal-resolution visual information, making them particularly suitable for challenging environments with high-speed motion and strongly varying lighting conditions. However, the lack of large datasets with dense ground-truth depth annotations hinders learning-based monocular depth estimation from event data. To address this limitation, we propose a cross-modal distillation paradigm to generate dense proxy labels leveraging a Vision Foundation Model (VFM). Our strategy requires an event stream spatially aligned with RGB frames, a simple setup even available off-the-shelf, and exploits the robustness of large-scale VFMs. Additionally, we propose to adapt VFMs, either a vanilla one like Depth Anything v2 (DAv2), or deriving from it a novel recurrent architecture to infer depth from monocular event cameras. We evaluate our approach with synthetic and real-world datasets, demonstrating that i) our cross-modal paradigm achieves competitive performance compared to fully supervised methods without requiring expensive depth annotations, and ii) our VFM-based models achieve state-of-the-art performance.

Method: Uses student-teacher paradigm with vocabulary alignment strategy, Low-Rank Adaptation (LoRA) for fine-tuning, and Top-K class selection mechanism to reduce memory requirements.

Result: Achieves 6.11 mIoU improvement on CityScapes dataset and demonstrates superior performance on zero-shot segmentation benchmarks.

Conclusion: VocAlign sets a new standard for source-free adaptation in open-vocabulary settings by effectively combining vocabulary alignment with efficient fine-tuning techniques.

Abstract: We introduce VocAlign, a novel source-free domain adaptation framework specifically designed for VLMs in open-vocabulary semantic segmentation. Our method adopts a student-teacher paradigm enhanced with a vocabulary alignment strategy, which improves pseudo-label generation by incorporating additional class concepts. To ensure efficiency, we use Low-Rank Adaptation (LoRA) to fine-tune the model, preserving its original capabilities while minimizing computational overhead. In addition, we propose a Top-K class selection mechanism for the student model, which significantly reduces memory requirements while further improving adaptation performance. Our approach achieves a notable 6.11 mIoU improvement on the CityScapes dataset and demonstrates superior performance on zero-shot segmentation benchmarks, setting a new standard for source-free adaptation in the open-vocabulary setting.

Method: CalibPrompt optimizes learnable prompts with calibration objectives: a regularizer aligning smoothed accuracy with predicted confidences, and an angular separation loss to maximize textual feature proximity for better confidence estimates.

Result: Extensive experiments on four Med-VLMs and five medical imaging datasets show CalibPrompt consistently improves calibration without significantly affecting clean accuracy.

Conclusion: CalibPrompt effectively addresses confidence calibration in Med-VLMs through prompt tuning with specialized calibration objectives, enhancing reliability for clinical applications.

Abstract: Medical Vision-Language Models (Med-VLMs) have demonstrated remarkable performance across diverse medical imaging tasks by leveraging large-scale image-text pretraining. However, their confidence calibration is largely unexplored, and so remains a significant challenge. As such, miscalibrated predictions can lead to overconfident errors, undermining clinical trust and decision-making reliability. To address this, we introduce CalibPrompt, the first framework to calibrate Med-VLMs during prompt tuning. CalibPrompt optimizes a small set of learnable prompts with carefully designed calibration objectives under scarce labeled data regime. First, we study a regularizer that attempts to align the smoothed accuracy with the predicted model confidences. Second, we introduce an angular separation loss to maximize textual feature proximity toward improving the reliability in confidence estimates of multimodal Med-VLMs. Extensive experiments on four publicly available Med-VLMs and five diverse medical imaging datasets reveal that CalibPrompt consistently improves calibration without drastically affecting clean accuracy. Our code is available at https://github.com/iabh1shekbasu/CalibPrompt.

Method: Developed a new metric that evaluates superpixel shape regularity through three aspects: convexity, balanced repartition, and contour smoothness, and validated its robustness to scale and noise.

Result: The proposed measure is robust to scale and noise variations, and enables more relevant comparison of superpixel methods than traditional circularity-based metrics.

Conclusion: The new metric provides a more comprehensive and accurate way to evaluate superpixel regularity, addressing the shortcomings of circularity measures and better serving the needs of superpixel-based applications.

Abstract: Regular decompositions are necessary for most superpixel-based object recognition or tracking applications. So far in the literature, the regularity or compactness of a superpixel shape is mainly measured by its circularity. In this work, we first demonstrate that such measure is not adapted for superpixel evaluation, since it does not directly express regularity but circular appearance. Then, we propose a new metric that considers several shape regularity aspects: convexity, balanced repartition, and contour smoothness. Finally, we demonstrate that our measure is robust to scale and noise and enables to more relevantly compare superpixel methods.

Method: Proposes Internal Dimension Increase (IDI) based representation that projects visual signals into controllable 3D semantics (mouth motion, eye blinking, head rotation/translation/location), compressed and transmitted as editable bitstreams synthesized via deep generative models.

Result: Outperforms state-of-the-art VVC and latest generative compression schemes in rate-distortion performance for face videos, enables interactive coding without additional manipulation processes.

Conclusion: The framework demonstrates superior performance and application prospects for face video coding, with potential to influence future digital human communication design in the metaverse.

Abstract: In this paper, we propose a novel framework for Interactive Face Video Coding (IFVC), which allows humans to interact with the intrinsic visual representations instead of the signals. The proposed solution enjoys several distinct advantages, including ultra-compact representation, low delay interaction, and vivid expression/headpose animation. In particular, we propose the Internal Dimension Increase (IDI) based representation, greatly enhancing the fidelity and flexibility in rendering the appearance while maintaining reasonable representation cost. By leveraging strong statistical regularities, the visual signals can be effectively projected into controllable semantics in the three dimensional space (e.g., mouth motion, eye blinking, head rotation, head translation and head location), which are compressed and transmitted. The editable bitstream, which naturally supports the interactivity at the semantic level, can synthesize the face frames via the strong inference ability of the deep generative model. Experimental results have demonstrated the performance superiority and application prospects of our proposed IFVC scheme. In particular, the proposed scheme not only outperforms the state-of-the-art video coding standard Versatile Video Coding (VVC) and the latest generative compression schemes in terms of rate-distortion performance for face videos, but also enables the interactive coding without introducing additional manipulation processes. Furthermore, the proposed framework is expected to shed lights on the future design of the digital human communication in the metaverse.

Method: Proposes ESTN with alternating local-global feature aggregation: shift convolution for local spatial-channel interactions, block sparse global perception module for global features, plus multiscale self-attention and low-parameter residual channel attention.

Result: Achieves higher average PSNR than SRCNN (2.17dB), ELAN-light (0.13dB), SwinIR-light (0.12dB), and SMFANER+ (0.1dB) on five datasets, with LAM confirming larger receptive field.

Conclusion: ESTN delivers improved quality of super-resolution images by effectively aggregating both local and global features with spatial-channel interactions.

Abstract: The Swin Transformer image super-resolution (SR) reconstruction network primarily depends on the long-range relationship of the window and shifted window attention to explore features. However, this approach focuses only on global features, ignoring local ones, and considers only spatial interactions, disregarding channel and spatial-channel feature interactions, limiting its nonlinear mapping capability. Therefore, this study proposes an enhanced Swin Transformer network (ESTN) that alternately aggregates local and global features. During local feature aggregation, shift convolution facilitates the interaction between local spatial and channel information. During global feature aggregation, a block sparse global perception module is introduced, wherein spatial information is reorganized and the recombined features are then processed by a dense layer to achieve global perception. Additionally, multiscale self-attention and low-parameter residual channel attention modules are introduced to aggregate information across different scales. Finally, the effectiveness of ESTN on five public datasets and a local attribution map (LAM) are analyzed. Experimental results demonstrate that the proposed ESTN achieves higher average PSNR, surpassing SRCNN, ELAN-light, SwinIR-light, and SMFANER+ models by 2.17dB, 0.13dB, 0.12dB, and 0.1dB, respectively, with LAM further confirming its larger receptive field. ESTN delivers improved quality of SR images. The source code can be found at https://github.com/huangyuming2021/ESTN.

Method: Teata framework uses text space consistency and generalization capabilities with Structured Semantic Prompt (SSP) learning to decompose text prompts into structured pairs, and Knowledge Adaptation and Projection (KAP) strategy to tune text knowledge via slow-paced learner for task adaptation without catastrophic forgetting.

Result: Extensive experiments demonstrate Teata’s superiority for LReID-Hybrid as well as on conventional LReID benchmarks over advanced methods.

Conclusion: The proposed Teata framework effectively addresses knowledge granularity and presentation mismatch challenges in hybrid clothing state scenarios, providing a practical solution for lifelong person re-identification that handles both cloth-changing and same-cloth domains.

Abstract: With the continuous expansion of intelligent surveillance networks, lifelong person re-identification (LReID) has received widespread attention, pursuing the need of self-evolution across different domains. However, existing LReID studies accumulate knowledge with the assumption that people would not change their clothes. In this paper, we propose a more practical task, namely lifelong person re-identification with hybrid clothing states (LReID-Hybrid), which takes a series of cloth-changing and same-cloth domains into account during lifelong learning. To tackle the challenges of knowledge granularity mismatch and knowledge presentation mismatch in LReID-Hybrid, we take advantage of the consistency and generalization capabilities of the text space, and propose a novel framework, dubbed $Teata$, to effectively align, transfer, and accumulate knowledge in an “image-text-image” closed loop. Concretely, to achieve effective knowledge transfer, we design a Structured Semantic Prompt (SSP) learning to decompose the text prompt into several structured pairs to distill knowledge from the image space with a unified granularity of text description. Then, we introduce a Knowledge Adaptation and Projection (KAP) strategy, which tunes text knowledge via a slow-paced learner to adapt to different tasks without catastrophic forgetting. Extensive experiments demonstrate the superiority of our proposed $Teata$ for LReID-Hybrid as well as on conventional LReID benchmarks over advanced methods.

Method: Uses a diffusion UNet encoder as foundation vision encoder, extracts features from final self-attention block, and applies recursive Normalized Cut algorithm for graph-based segmentation with soft granularity regulation.

Result: Significantly outperforms previous state-of-the-art methods on zero-shot segmentation, producing well-defined segmentation maps that capture intricate image details accurately.

Conclusion: Diffusion UNet encoders contain remarkably accurate semantic knowledge and can serve as effective foundation vision encoders for downstream tasks.

Abstract: Foundation models have emerged as powerful tools across various domains including language, vision, and multimodal tasks. While prior works have addressed unsupervised image segmentation, they significantly lag behind supervised models. In this paper, we use a diffusion UNet encoder as a foundation vision encoder and introduce DiffCut, an unsupervised zero-shot segmentation method that solely harnesses the output features from the final self-attention block. Through extensive experimentation, we demonstrate that the utilization of these diffusion features in a graph based segmentation algorithm, significantly outperforms previous state-of-the-art methods on zero-shot segmentation. Specifically, we leverage a recursive Normalized Cut algorithm that softly regulates the granularity of detected objects and produces well-defined segmentation maps that precisely capture intricate image details. Our work highlights the remarkably accurate semantic knowledge embedded within diffusion UNet encoders that could then serve as foundation vision encoders for downstream tasks. Project page at https://diffcut-segmentation.github.io

Method: The authors propose PVEP (Physical Vulnerability Evaluating Pipeline) that incorporates multiple visual modal physical threats to comprehensively evaluate VLAMs. The pipeline specifically tests three types of attacks: Out-of-Distribution scenarios, Typography-based Visual Prompts, and Adversarial Patch Attacks.

Result: The study compares VLAMs’ performance fluctuations before and after being attacked by different physical threats, providing analyses of how these models respond to various physical vulnerabilities.

Conclusion: The research provides a systematic framework for evaluating the physical robustness of VLAMs and offers insights into their vulnerabilities when facing real-world physical threats, which is crucial for ensuring safety in robotic manipulation tasks.

Abstract: Recently, driven by advancements in Multimodal Large Language Models (MLLMs), Vision Language Action Models (VLAMs) are being proposed to achieve better performance in open-vocabulary scenarios for robotic manipulation tasks. Since manipulation tasks involve direct interaction with the physical world, ensuring robustness and safety during the execution of this task is always a very critical issue. In this paper, by synthesizing current safety research on MLLMs and the specific application scenarios of the manipulation task in the physical world, we comprehensively evaluate VLAMs in the face of potential physical threats. Specifically, we propose the Physical Vulnerability Evaluating Pipeline (PVEP) that can incorporate as many visual modal physical threats as possible for evaluating the physical robustness of VLAMs. The physical threats in PVEP specifically include Out-of-Distribution, Typography-based Visual Prompt, and Adversarial Patch Attacks. By comparing the performance fluctuations of VLAMs before and after being attacked, we provide generalizable \textbf{\textit{Analyses}} of how VLAMs respond to different physical threats.

Method: Uses Supplemental Enhancement Information (SEI) messages to encode compact spatial and temporal representations (2D/3D keypoints, facial semantics, compact features) within video bitstreams.

Result: Achieves remarkable rate-distortion performance compared to VVC standard, enables user-specified animation/filtering, and supports metaverse applications.

Conclusion: The approach advances generative video compression standardization, establishes new SEI definitions for future GFVC applications, and demonstrates superior performance over existing standards.

Abstract: This paper proposes a Generative Face Video Compression (GFVC) approach using Supplemental Enhancement Information (SEI), where a series of compact spatial and temporal representations of a face video signal (e.g., 2D/3D keypoints, facial semantics and compact features) can be coded using SEI messages and inserted into the coded video bitstream. At the time of writing, the proposed GFVC approach using SEI messages has been included into a draft amendment of the Versatile Supplemental Enhancement Information (VSEI) standard by the Joint Video Experts Team (JVET) of ISO/IEC JTC 1/SC 29 and ITU-T SG21, which will be standardized as a new version of ITU-T H.274 | ISO/IEC 23002-7. To the best of the authors’ knowledge, the JVET work on the proposed SEI-based GFVC approach is the first standardization activity for generative video compression. The proposed SEI approach has not only advanced the reconstruction quality of early-day Model-Based Coding (MBC) via the state-of-the-art generative technique, but also established a new SEI definition for future GFVC applications and deployment. Experimental results illustrate that the proposed SEI-based GFVC approach can achieve remarkable rate-distortion performance compared with the latest Versatile Video Coding (VVC) standard, whilst also potentially enabling a wide variety of functionalities including user-specified animation/filtering and metaverse-related applications.

Method: GDFs associate each 3D point with a 3D vector where the vector’s norm represents the unsigned distance to the surface and its orientation indicates the direction towards the closest surface point, making the function differentiable at the surface.

Result: The effectiveness of GDFs is demonstrated on ShapeNet Car, Multi-Garment, and 3D-Scene datasets using both single-shape reconstruction networks and categorical auto-decoders.

Conclusion: GDFs provide a more robust and learnable alternative to UDFs for representing non-watertight surfaces in deep learning applications by addressing the differentiability issue at surface boundaries.

Abstract: Unsigned Distance Functions (UDFs) can be used to represent non-watertight surfaces in a deep learning framework. However, UDFs tend to be brittle and difficult to learn, in part because the surface is located exactly where the UDF is non-differentiable. In this work, we show that Gradient Distance Functions (GDFs) can remedy this by being differentiable at the surface while still being able to represent open surfaces. This is done by associating to each 3D point a 3D vector whose norm is taken to be the unsigned distance to the surface and whose orientation is taken to be the direction towards the closest surface point. We demonstrate the effectiveness of GDFs on ShapeNet Car, Multi-Garment, and 3D-Scene datasets with both single-shape reconstruction networks or categorical auto-decoders.

Method: Two complementary approaches: (1) dataset-based method contrasting activations on biased vs neutral inputs, (2) optimization-based method using single-step gradient perturbation without additional data.

Result: Effectively reduces generation of text related to protected attributes while maintaining sentiment and fluency. Debiased models achieve comparable accuracy to original models on downstream tasks.

Conclusion: Bias mitigation in LMMs can be achieved without sacrificing model performance through training-free interventions on model representations.

Abstract: Large Multi-Modal Models (LMMs) have demonstrated impressive capabilities as general-purpose chatbots able to engage in conversations about visual inputs. However, their responses are influenced by societal biases present in their training datasets, leading to undesirable differences in how the model responds when presented with images depicting people of different demographics. In this work, we propose a training-free debiasing framework for LMMs that intervenes on the model’s representations during text generation by constructing a steering vector that reduces reference on protected attributes. Our framework introduces two complementary methods: (1) a dataset-based approach that constructs a steering vector by contrasting model activations on biased and neutral inputs, and (2) a novel optimization-based approach designed for low-resource settings, which constructs the steering vector using a single step of gradient-based perturbation without requiring additional data. Our experiments show that these interventions effectively reduce the propensity of LMMs to generate text related to protected attributes while maintaining sentiment and fluency. Furthermore, we demonstrate that debiased LMMs achieve comparable accuracy to their unmodified counterparts on downstream tasks, indicating that bias mitigation can be achieved without sacrificing model performance.

Method: Two-module framework: Motion Generator creates synthetic noisy motion data, and Motion Physics Refinement module uses physics simulator to project noisy motions into physically-plausible space. Includes prior reward module for stability and collaborative training between modules.

Result: Achieves state-of-the-art motion quality on text-to-motion and music-to-dance tasks while drastically improving physical plausibility.

Conclusion: The collaborative training paradigm enables mutual enhancement between motion generation and physics refinement, significantly improving practicality and robustness for real-world applications.

Abstract: Human motion generation has been widely studied due to its crucial role in areas such as digital humans and humanoid robot control. However, many current motion generation approaches disregard physics constraints, frequently resulting in physically implausible motions with pronounced artifacts such as floating and foot sliding. Meanwhile, training an effective motion physics optimizer with noisy motion data remains largely unexplored. In this paper, we propose \textbf{Morph}, a \textbf{Mo}tion-F\textbf{r}ee \textbf{ph}ysics optimization framework, consisting of a Motion Generator and a Motion Physics Refinement module, for enhancing physical plausibility without relying on expensive real-world motion data. Specifically, the motion generator is responsible for providing large-scale synthetic, noisy motion data, while the motion physics refinement module utilizes these synthetic data to learn a motion imitator within a physics simulator, enforcing physical constraints to project the noisy motions into a physically-plausible space. Additionally, we introduce a prior reward module to enhance the stability of the physics optimization process and generate smoother and more stable motions. These physically refined motions are then used to fine-tune the motion generator, further enhancing its capability. This collaborative training paradigm enables mutual enhancement between the motion generator and the motion physics refinement module, significantly improving practicality and robustness in real-world applications. Experiments on both text-to-motion and music-to-dance generation tasks demonstrate that our framework achieves state-of-the-art motion quality while improving physical plausibility drastically.

Method: 1) Apply Sparse Autoencoder (SAE) to disentangle representations into human-understandable features 2) Use an automatic interpretation framework where LMMs themselves interpret the open-semantic features learned in SAE (using LLaVA-NeXT-8B analyzed by LLaVA-OV-72B)

Result: Features can effectively steer model’s behavior, providing insights into why LMMs excel in specific tasks (including EQ tests) and illuminating the nature of their mistakes with potential rectification strategies.

Conclusion: The findings offer new insights into LMMs’ internal mechanisms and suggest parallels with human brain cognitive processes, contributing to better understanding and potential improvement of these models.

Abstract: Recent advances in Large Multimodal Models (LMMs) lead to significant breakthroughs in both academia and industry. One question that arises is how we, as humans, can understand their internal neural representations. This paper takes an initial step towards addressing this question by presenting a versatile framework to identify and interpret the semantics within LMMs. Specifically, 1) we first apply a Sparse Autoencoder(SAE) to disentangle the representations into human understandable features. 2) We then present an automatic interpretation framework to interpreted the open-semantic features learned in SAE by the LMMs themselves. We employ this framework to analyze the LLaVA-NeXT-8B model using the LLaVA-OV-72B model, demonstrating that these features can effectively steer the model’s behavior. Our results contribute to a deeper understanding of why LMMs excel in specific tasks, including EQ tests, and illuminate the nature of their mistakes along with potential strategies for their rectification. These findings offer new insights into the internal mechanisms of LMMs and suggest parallels with the cognitive processes of the human brain.

Method: Proposes a diffusion-based approach that creates a Camera Image representation to encode camera intrinsics, fine-tunes stable diffusion to generate Camera Images from RGB inputs, and extracts intrinsics via RANSAC operations.

Result: Extensive experiments show DM-Calib significantly outperforms baseline methods on multiple public datasets and improves performance in zero-shot metric depth estimation, 3D metrology, pose estimation, and sparse-view reconstruction.

Conclusion: The method successfully leverages diffusion model priors for monocular camera calibration, demonstrating strong generalization and broad applicability across diverse 3D vision tasks with superior performance over existing approaches.

Abstract: In this paper, we present DM-Calib, a diffusion-based approach for estimating pinhole camera intrinsic parameters from a single input image. Monocular camera calibration is essential for many 3D vision tasks. However, most existing methods depend on handcrafted assumptions or are constrained by limited training data, resulting in poor generalization across diverse real-world images. Recent advancements in stable diffusion models, trained on massive data, have shown the ability to generate high-quality images with varied characteristics. Emerging evidence indicates that these models implicitly capture the relationship between camera focal length and image content. Building on this insight, we explore how to leverage the powerful priors of diffusion models for monocular pinhole camera calibration. Specifically, we introduce a new image-based representation, termed Camera Image, which losslessly encodes the numerical camera intrinsics and integrates seamlessly with the diffusion framework. Using this representation, we reformulate the problem of estimating camera intrinsics as the generation of a dense Camera Image conditioned on an input image. By fine-tuning a stable diffusion model to generate a Camera Image from a single RGB input, we can extract camera intrinsics via a RANSAC operation. We further demonstrate that our monocular calibration method enhances performance across various 3D tasks, including zero-shot metric depth estimation, 3D metrology, pose estimation and sparse-view reconstruction. Extensive experiments on multiple public datasets show that our approach significantly outperforms baselines and provides broad benefits to 3D vision tasks.

Method: Proposes IV-tuning which freezes backbone parameters of Pre-trained Visual Models and only fine-tunes a small subset (<3% of parameters) for various IR-VIS downstream tasks including object detection, semantic segmentation, and salient object detection.

Result: IV-tuning outperforms full fine-tuning baselines and existing IR-VIS methods, effectively learning complementary information between infrared and visible modalities while alleviating overfitting problems.

Conclusion: Parameter-efficient tuning by freezing pre-trained weights is an effective approach for IR-VIS fusion tasks, maintaining feature diversity and improving generalization with minimal computational overhead.

Abstract: Existing infrared and visible (IR-VIS) methods inherit the general representations of Pre-trained Visual Models (PVMs) to facilitate complementary learning. However, our analysis indicates that under the full fine-tuning paradigm, the feature space becomes highly constrained and low-ranked, which has been proven to seriously impair generalization. One solution is freezing parameters to preserve pre-trained knowledge and thus maintain diversity of the feature space. To this end, we propose IV-tuning, to parameter-efficiently harness PVMs for various IR-VIS downstream tasks, including salient object detection, semantic segmentation, and object detection. Compared with the full fine-tuning baselines and existing IR-VIS methods, IV-tuning facilitates the learning of complementary information between infrared and visible modalities with less than 3% of the backbone parameters, and effectively alleviates the overfitting problem. The code is available in https://github.com/Yummy198913/IV-tuning.

Method: Proposes Mutual Information as a metric to quantify latent variable contributions, introduces synthetic data generation for self-supervised contrastive learning using MLVGMs’ hierarchical variables, and develops Continuous Sampling strategy for dynamic sample generation during training.

Result: Analysis shows current MLVGMs often underutilize latent variables. Generated views from MLVGMs compete with or surpass views from real data in self-supervised learning tasks.

Conclusion: Establishes a principled approach to understand and exploit MLVGMs, advancing both generative modeling and self-supervised learning with synthetic data generation capabilities.

Abstract: In image generation, Multiple Latent Variable Generative Models (MLVGMs) employ multiple latent variables to gradually shape the final images, from global characteristics to finer and local details (e.g., StyleGAN, NVAE), emerging as powerful tools for diverse applications. Yet their generative dynamics remain only empirically observed, without a systematic understanding of each latent variable’s impact. In this work, we propose a novel framework that quantifies the contribution of each latent variable using Mutual Information (MI) as a metric. Our analysis reveals that current MLVGMs often underutilize some latent variables, and provides actionable insights for their use in downstream applications. With this foundation, we introduce a method for generating synthetic data for Self-Supervised Contrastive Representation Learning (SSCRL). By leveraging the hierarchical and disentangled variables of MLVGMs, our approach produces diverse and semantically meaningful views without the need for real image data. Additionally, we introduce a Continuous Sampling (CS) strategy, where the generator dynamically creates new samples during SSCRL training, greatly increasing data variability. Our comprehensive experiments demonstrate the effectiveness of these contributions, showing that MLVGMs’ generated views compete on par with or even surpass views generated from real data. This work establishes a principled approach to understanding and exploiting MLVGMs, advancing both generative modeling and self-supervised learning. Code and pre-trained models at: https://github.com/SerezD/mi_ml_gen.

Method: SWAT creates adversarial streams connecting source and target feature spaces, then uses a sliding window paradigm that moves along the stream to gradually narrow gaps between adjacent intermediate domains, explicitly reducing domain shift when reaching the target.

Result: Extensive experiments on 6 GDA benchmarks show significant effectiveness, with 6.1% improvement on Rotated MNIST and 4.1% advantage on CIFAR-100C over previous methods.

Conclusion: SWAT provides an effective solution for Gradual Domain Adaptation by formulating adversarial streams and using a sliding window approach to handle domain shifts gradually, demonstrating superior performance across multiple benchmarks.

Abstract: Domain shifts are critical issues that harm the performance of machine learning. Unsupervised Domain Adaptation (UDA) mitigates this issue but suffers when the domain shifts are steep and drastic. Gradual Domain Adaptation (GDA) alleviates this problem in a mild way by gradually adapting from the source to the target domain using multiple intermediate domains. In this paper, we propose Sliding Window Adversarial Training (SWAT) for GDA. SWAT first formulates adversarial streams to connect the feature spaces of the source and target domains. Then, a sliding window paradigm is designed that moves along the adversarial stream to gradually narrow the small gap between adjacent intermediate domains. When the window moves to the end of the stream, i.e., the target domain, the domain shift is explicitly reduced. Extensive experiments on six GDA benchmarks demonstrate the significant effectiveness of SWAT, especially 6.1% improvement on Rotated MNIST and 4.1% advantage on CIFAR-100C over the previous methods.

Method: Proposes PhyRMDM framework with dual learning pathways that establish cross-domain representation alignment between physical principles (Helmholtz equation constraints) and neural network features through physics-informed neural networks (PINNs) and representation alignment mechanism.

Result: Achieves NMSE of 0.0031 under Static Radio Map conditions and 0.0047 with Dynamic Radio Map scenarios. Provides 37.2% accuracy enhancement in ultra-sparse cases (1% sampling rate).

Conclusion: The representation alignment paradigm effectively bridges physics-based modeling and deep learning for radio map reconstruction, demonstrating superior performance over state-of-the-art methods.

Abstract: Radio map reconstruction is essential for enabling advanced applications, yet challenges such as complex signal propagation and sparse observational data hinder accurate reconstruction in practical scenarios. Existing methods often fail to align physical constraints with data-driven features, particularly under sparse measurement conditions. To address these issues, we propose Physics-Aligned Radio Map Diffusion Model (PhyRMDM), a novel framework that establishes cross-domain representation alignment between physical principles and neural network features through dual learning pathways. The proposed model integrates Physics-Informed Neural Networks (PINNs) with a representation alignment mechanism that explicitly enforces consistency between Helmholtz equation constraints and environmental propagation patterns. Experimental results demonstrate significant improvements over state-of-the-art methods, achieving NMSE of 0.0031 under Static Radio Map (SRM) conditions, and NMSE of 0.0047 with Dynamic Radio Map (DRM) scenarios. The proposed representation alignment paradigm provides 37.2% accuracy enhancement in ultra-sparse cases (1% sampling rate), confirming its effectiveness in bridging physics-based modeling and deep learning for radio map reconstruction.

Method: Multi-agent framework that decomposes chart generation into iterative collaboration among specialized agents, separating different modalities during critique process.

Result: 5.2% improvement over current best result in chart generation, exhibits test-time scaling (performance increases with computational budget from 512 to 8192 tokens), and boosts VLM self-correction in multimodal context.

Conclusion: METAL’s multi-agent approach effectively handles the complex multimodal reasoning required for high-quality chart generation, demonstrating superior performance and scalability compared to direct VLM prompting.

Abstract: Chart generation aims to generate code to produce charts satisfying the desired visual properties, e.g., texts, layout, color, and type. It has great potential to empower the automatic professional report generation in financial analysis, research presentation, education, and healthcare. In this work, we build a vision-language model (VLM) based multi-agent framework for effective automatic chart generation. Generating high-quality charts requires both strong visual design skills and precise coding capabilities that embed the desired visual properties into code. Such a complex multi-modal reasoning process is difficult for direct prompting of VLMs. To resolve these challenges, we propose METAL, a multi-agent framework that decomposes the task of chart generation into the iterative collaboration among specialized agents. METAL achieves 5.2% improvement over the current best result in the chart generation task. The METAL framework exhibits the phenomenon of test-time scaling: its performance increases monotonically as the logarithmic computational budget grows from 512 to 8192 tokens. In addition, we find that separating different modalities during the critique process of METAL boosts the self-correction capability of VLMs in the multimodal context.

Method: Proposes VLM-E2E framework that integrates textual representations into Bird’s-Eye-View features for semantic supervision, using a BEV-Text learnable weighted fusion strategy to dynamically balance contributions from visual and textual modalities.

Result: Evaluated on nuScenes dataset, VLM-E2E achieves significant improvements in perception, prediction, and planning over baseline end-to-end models.

Conclusion: The attention-enhanced BEV representation enables more accurate and reliable autonomous driving tasks by better aligning with human-like driving behavior through effective multimodal fusion.

Abstract: Human drivers adeptly navigate complex scenarios by utilizing rich attentional semantics, but the current autonomous systems struggle to replicate this ability, as they often lose critical semantic information when converting 2D observations into 3D space. In this sense, it hinders their effective deployment in dynamic and complex environments. Leveraging the superior scene understanding and reasoning abilities of Vision-Language Models (VLMs), we propose VLM-E2E, a novel framework that uses the VLMs to enhance training by providing attentional cues. Our method integrates textual representations into Bird’s-Eye-View (BEV) features for semantic supervision, which enables the model to learn richer feature representations that explicitly capture the driver’s attentional semantics. By focusing on attentional semantics, VLM-E2E better aligns with human-like driving behavior, which is critical for navigating dynamic and complex environments. Furthermore, we introduce a BEV-Text learnable weighted fusion strategy to address the issue of modality importance imbalance in fusing multimodal information. This approach dynamically balances the contributions of BEV and text features, ensuring that the complementary information from visual and textual modalities is effectively utilized. By explicitly addressing the imbalance in multimodal fusion, our method facilitates a more holistic and robust representation of driving environments. We evaluate VLM-E2E on the nuScenes dataset and achieve significant improvements in perception, prediction, and planning over the baseline end-to-end model, showcasing the effectiveness of our attention-enhanced BEV representation in enabling more accurate and reliable autonomous driving tasks.

Method: Proposed a unified model encoding individual characteristics in distributional format, conducted extensive experiments with varying group compositions, and used Earth Mover’s Distance and Gini index for score-distribution analysis.

Result: Found substantial performance variation even for GIAA, challenging the assumption that averaging scores eliminates subjectivity. Identified education, photography experience, and art experience as key factors in aesthetic differences, with greater subjectivity in artworks than photographs.

Conclusion: Transfer learning from PIAA to GIAA (interpolation) is generally more effective than GIAA to PIAA (extrapolation), and demographic factors significantly influence aesthetic assessment performance.

Abstract: Image aesthetic assessment (IAA) evaluates image aesthetics, a task complicated by image diversity and user subjectivity. Current approaches address this in two stages: Generic IAA (GIAA) models estimate mean aesthetic scores, while Personal IAA (PIAA) models adapt GIAA using transfer learning to incorporate user subjectivity. However, a theoretical understanding of transfer learning between GIAA and PIAA, particularly concerning the impact of group composition, group size, aesthetic differences between groups and individuals, and demographic correlations, is lacking. This work establishes a theoretical foundation for IAA, proposing a unified model that encodes individual characteristics in a distributional format for both individual and group assessments. We show that transferring from GIAA to PIAA involves extrapolation, while the reverse involves interpolation, which is generally more effective for machine learning. Extensive experiments with varying group compositions, including sub-sampling by group size and disjoint demographics, reveal substantial performance variation even for GIAA, challenging the assumption that averaging scores eliminates individual subjectivity. Score-distribution analysis using Earth Mover’s Distance (EMD) and the Gini index identifies education, photography experience, and art experience as key factors in aesthetic differences, with greater subjectivity in artworks than in photographs. Code is available at https://github.com/lwchen6309/aesthetics_transfer_learning.

Method: Constructs biologically informed graphs from OCTA images encoding retinal vascular features, uses GNN for staging with integrated gradients to identify critical features, then transforms this knowledge into textual descriptions for vision-language model instruction-tuning.

Result: Experimental evaluations show improved classification accuracy and more clinically interpretable results compared to existing methods. Expert study confirms more accurate diagnostic explanations and enables precise pathology localization.

Conclusion: The method successfully integrates graph representation learning with VLMs to deliver both accurate DR staging and human-interpretable explanations, paving the way for clinically useful AI diagnostic tools.

Abstract: Accurate staging of Diabetic Retinopathy (DR) is essential for guiding timely interventions and preventing vision loss. However, current staging models are hardly interpretable, and most public datasets contain no clinical reasoning or interpretation beyond image-level labels. In this paper, we present a novel method that integrates graph representation learning with vision-language models (VLMs) to deliver explainable DR diagnosis. Our approach leverages optical coherence tomography angiography (OCTA) images by constructing biologically informed graphs that encode key retinal vascular features such as vessel morphology and spatial connectivity. A graph neural network (GNN) then performs DR staging while integrated gradients highlight critical nodes and edges and their individual features that drive the classification decisions. We collect this graph-based knowledge which attributes the model’s prediction to physiological structures and their characteristics. We then transform it into textual descriptions for VLMs. We perform instruction-tuning with these textual descriptions and the corresponding image to train a student VLM. This final agent can classify the disease and explain its decision in a human interpretable way solely based on a single image input. Experimental evaluations on both proprietary and public datasets demonstrate that our method not only improves classification accuracy but also offers more clinically interpretable results. An expert study further demonstrates that our method provides more accurate diagnostic explanations and paves the way for precise localization of pathologies in OCTA images.

Method: Proposes a video clipping strategy to extract racket swing frames, uses three existing models for pose estimation, shuttlecock trajectory tracking, and court line detection, and introduces Badminton Stroke-type Transformer (BST) for classification.

Result: Outperforms previous state-of-the-art methods on ShuttleSet (largest badminton dataset), BadmintonDB, and TenniSet (tennis dataset), demonstrating superior performance across multiple racket sports.

Conclusion: Effectively leveraging ball trajectory data is a promising direction for action recognition in racket sports, and the proposed approach shows strong generalization across different datasets and sports.

Abstract: Badminton, known for having the fastest ball speeds among all sports, presents significant challenges to the field of computer vision, including player identification, court line detection, shuttlecock trajectory tracking, and player stroke-type classification. In this paper, we introduce a novel video clipping strategy to extract frames of each player’s racket swing in a badminton broadcast match. These clipped frames are then processed by three existing models: one for Human Pose Estimation to obtain human skeletal joints, another for shuttlecock trajectory tracking, and the other for court line detection to determine player positions on the court. Leveraging these data as inputs, we propose Badminton Stroke-type Transformer (BST) to classify player stroke-types in singles. To the best of our knowledge, experimental results demonstrate that our method outperforms the previous state-of-the-art on the largest publicly available badminton video dataset (ShuttleSet), another badminton dataset (BadmintonDB), and a tennis dataset (TenniSet). These results suggest that effectively leveraging ball trajectory is a promising direction for action recognition in racket sports.

Method: KARMMA uses knowledge distillation from a multimodal teacher to train a multimodal student that benefits from available modalities while remaining robust to missing ones. It requires no modality alignment across samples during training or inference.

Result: The student model achieves competitive accuracy on Epic-Kitchens and Something-Something datasets while significantly reducing accuracy drops under missing modality conditions. It uses approximately 50% fewer computational resources than the teacher model.

Conclusion: KARMMA provides an effective solution for robust multimodal action recognition that handles missing modalities without retraining, offering both computational efficiency and maintained performance.

Abstract: Existing methods for egocentric action recognition often rely solely on RGB videos, while additional modalities, e.g., audio, can improve accuracy in challenging scenarios. However, most prior multimodal approaches assume all modalities are available at inference, leading to significant accuracy drops, or even failure, when inputs are missing. To address this, we introduce KARMMA, a multimodal Knowledge distillation approach for egocentric Action Recognition robust to Missing ModAlities that requires no modality alignment across all samples during training or inference. KARMMA distills knowledge from a multimodal teacher into a multimodal student that benefits from all available modalities while remaining robust to missing ones, making it suitable for diverse multimodal scenarios without retraining. Our student uses approximately 50% fewer computational resources than our teacher, resulting in a lightweight and fast model. Experiments on Epic-Kitchens and Something-Something show that our student achieves competitive accuracy while significantly reducing accuracy drops under missing modality conditions.

Method: Proposes PVLM method with parsing-guided vision language model using dynamic contrastive learning. Includes novel parsing encoder for global face attribute embeddings, dynamic vision-parsing matching, and deepfake attribution contrastive center loss to cluster relevant generators while separating irrelevant ones.

Result: Experimental results demonstrate that the proposed model exceeds state-of-the-art performance on the zero-shot deepfake attribution benchmark across various protocol evaluations.

Conclusion: The PVLM method effectively captures general and diverse attribution features for zero-shot deepfake attribution, providing fine-grained traceability to unseen advanced generators by leveraging face parsing-aware forgery representations and contrastive learning.

Abstract: The challenge of tracing the source attribution of forged faces has gained significant attention due to the rapid advancement of generative models. However, existing deepfake attribution (DFA) works primarily focus on the interaction among various domains in vision modality, and other modalities such as texts and face parsing are not fully explored. Besides, they tend to fail to assess the generalization performance of deepfake attributors to unseen advanced generators like diffusion in a fine-grained manner. In this paper, we propose a novel parsing-aware vision language model with dynamic contrastive learning(PVLM) method for zero-shot deepfake attribution (ZS-DFA),which facilitates effective and fine-grained traceability to unseen advanced generators. Specifically, we conduct a novel and fine-grained ZS-DFA benchmark to evaluate the attribution performance of deepfake attributors to unseen advanced generators like diffusion. Besides, we propose an innovative parsing-guided vision language model with dynamic contrastive learning (PVLM) method to capture general and diverse attribution features. We are motivated by the observation that the preservation of source face attributes in facial images generated by GAN and diffusion models varies significantly. We employ the inherent face attributes preservation differences to capture face parsing-aware forgery representations. Therefore, we devise a novel parsing encoder to focus on global face attribute embeddings, enabling parsing-guided DFA representation learning via dynamic vision-parsing matching. Additionally, we present a novel deepfake attribution contrastive center loss to pull relevant generators closer and push irrelevant ones away, which can be introduced into DFA models to enhance traceability. Experimental results show that our model exceeds the state-of-the-art on the ZS-DFA benchmark via various protocol evaluations.

Method: Maintains a reservoir of domain-specialized models that detect new domains via online clustering over style features and route samples to appropriate specialized models for domain-specific adaptation.

Result: Substantially improves adaptation accuracy and maintains stable performance across prolonged, recurring domain shifts, outperforming state-of-the-art methods on multiple benchmarks.

Conclusion: ReservoirTTA provides an effective solution for prolonged test-time adaptation in continuously evolving domains, with theoretical guarantees on parameter variance and prevention of model collapse.

Abstract: This paper introduces ReservoirTTA, a novel plug-in framework designed for prolonged test-time adaptation (TTA) in scenarios where the test domain continuously shifts over time, including cases where domains recur or evolve gradually. At its core, ReservoirTTA maintains a reservoir of domain-specialized models – an adaptive test-time model ensemble – that both detects new domains via online clustering over style features of incoming samples and routes each sample to the appropriate specialized model, and thereby enables domain-specific adaptation. This multi-model strategy overcomes key limitations of single model adaptation, such as catastrophic forgetting, inter-domain interference, and error accumulation, ensuring robust and stable performance on sustained non-stationary test distributions. Our theoretical analysis reveals key components that bound parameter variance and prevent model collapse, while our plug-in TTA module mitigates catastrophic forgetting of previously encountered domains. Extensive experiments on scene-level corruption benchmarks (ImageNet-C, CIFAR-10/100-C), object-level style shifts (DomainNet-126, PACS), and semantic segmentation (Cityscapes->ACDC) covering recurring and continuously evolving domain shifts – show that ReservoirTTA substantially improves adaptation accuracy and maintains stable performance across prolonged, recurring shifts, outperforming state-of-the-art methods. Our code is publicly available at https://github.com/LTS5/ReservoirTTA.

Method: Systematically evaluated two representative concept erasure methods (Unified Concept Editing and Erased Stable Diffusion) using an instance-level evaluation strategy with lightweight fine-tuning to test reactivation potential of erased concepts.

Result: Erased concepts often reemerge with substantial visual fidelity after minimal adaptation, showing current methods suppress latent generative representations without fully eliminating them.

Conclusion: Existing concept erasure approaches have critical limitations and require deeper representation-level interventions and more rigorous evaluation standards for genuine, irreversible concept removal.

Abstract: To what extent does concept erasure eliminate generative capacity in diffusion models? While prior evaluations have primarily focused on measuring concept suppression under specific textual prompts, we explore a complementary and fundamental question: do current concept erasure techniques genuinely remove the ability to generate targeted concepts, or do they merely achieve superficial, prompt-specific suppression? We systematically evaluate the robustness and reversibility of two representative concept erasure methods, Unified Concept Editing and Erased Stable Diffusion, by probing their ability to eliminate targeted generative behaviors in text-to-image models. These methods attempt to suppress undesired semantic concepts by modifying internal model parameters, either through targeted attention edits or model-level fine-tuning strategies. To rigorously assess whether these techniques truly erase generative capacity, we propose an instance-level evaluation strategy that employs lightweight fine-tuning to explicitly test the reactivation potential of erased concepts. Through quantitative metrics and qualitative analyses, we show that erased concepts often reemerge with substantial visual fidelity after minimal adaptation, indicating that current methods suppress latent generative representations without fully eliminating them. Our findings reveal critical limitations in existing concept erasure approaches and highlight the need for deeper, representation-level interventions and more rigorous evaluation standards to ensure genuine, irreversible removal of concepts from generative models.

Method: Introduces Class-wise Adaptive Instance Normalization and Cross-Attention (CACTI) and its extension with selective attention Filtering (CACTIF) for semantically consistent style transfer using diffusion models.

Result: Produces higher quality images with lower FID scores and better content preservation. Effectively bridges synthetic-to-real domain gap even with minimal target domain data.

Conclusion: Class-aware diffusion-based style transfer advances robust perception systems for challenging real-world applications by preserving semantic boundaries and structural coherence.

Abstract: Semantic segmentation models trained on synthetic data often perform poorly on real-world images due to domain gaps, particularly in adverse conditions where labeled data is scarce. Yet, recent foundation models enable to generate realistic images without any training. This paper proposes to leverage such diffusion models to improve the performance of vision models when learned on synthetic data. We introduce two novel techniques for semantically consistent style transfer using diffusion models: Class-wise Adaptive Instance Normalization and Cross-Attention (CACTI) and its extension with selective attention Filtering (CACTIF). CACTI applies statistical normalization selectively based on semantic classes, while CACTIF further filters cross-attention maps based on feature similarity, preventing artifacts in regions with weak cross-attention correspondences. Our methods transfer style characteristics while preserving semantic boundaries and structural coherence, unlike approaches that apply global transformations or generate content without constraints. Experiments using GTA5 as source and Cityscapes/ACDC as target domains show that our approach produces higher quality images with lower FID scores and better content preservation. Our work demonstrates that class-aware diffusion-based style transfer effectively bridges the synthetic-to-real domain gap even with minimal target domain data, advancing robust perception systems for challenging real-world applications. The source code is available at: https://github.com/echigot/cactif.

Method: Mask-free training paradigm with Diffusion Transformer models, flow-matching-based progressive noise initialization for pose/identity consistency, and Dynamic Spatiotemporal Classifier-Free Guidance (DS-CFG) to address weak audio conditioning.

Result: OmniSync significantly outperforms prior methods in both visual quality and lip sync accuracy, achieving superior results in both real-world and AI-generated videos.

Conclusion: The proposed framework provides a universal solution for diverse visual scenarios, overcoming limitations of existing methods and establishing a new benchmark for lip synchronization evaluation.

Abstract: Lip synchronization is the task of aligning a speaker’s lip movements in video with corresponding speech audio, and it is essential for creating realistic, expressive video content. However, existing methods often rely on reference frames and masked-frame inpainting, which limit their robustness to identity consistency, pose variations, facial occlusions, and stylized content. In addition, since audio signals provide weaker conditioning than visual cues, lip shape leakage from the original video will affect lip sync quality. In this paper, we present OmniSync, a universal lip synchronization framework for diverse visual scenarios. Our approach introduces a mask-free training paradigm using Diffusion Transformer models for direct frame editing without explicit masks, enabling unlimited-duration inference while maintaining natural facial dynamics and preserving character identity. During inference, we propose a flow-matching-based progressive noise initialization to ensure pose and identity consistency, while allowing precise mouth-region editing. To address the weak conditioning signal of audio, we develop a Dynamic Spatiotemporal Classifier-Free Guidance (DS-CFG) mechanism that adaptively adjusts guidance strength over time and space. We also establish the AIGC-LipSync Benchmark, the first evaluation suite for lip synchronization in diverse AI-generated videos. Extensive experiments demonstrate that OmniSync significantly outperforms prior methods in both visual quality and lip sync accuracy, achieving superior results in both real-world and AI-generated videos.

Method: Uses deformable phase plates (DPPs) optimized through a differentiable optical model to correct off-axis aberrations locally. Joint optimization of DPP deformations under imaging budget constraints, with a neural network-based control model to handle DPP’s non-linear behavior and improve simulation-hardware alignment.

Result: Produces foveated images with enhanced sharpness at fixation points. Stacking multiple foveated images creates composite images free from aberrations. Outperforms traditional focus stacking for extended depth-of-field imaging. Enables real-time foveated video when integrated with object detection or eye-tracking.

Conclusion: Fovea Stacking provides an effective solution for aberration correction in simplified optical systems, enabling smaller camera form factors while maintaining image quality. The approach shows promise for applications like surveillance and foveated virtual reality displays through dynamic lens adjustment capabilities.

Abstract: The desire for cameras with smaller form factors has recently lead to a push for exploring computational imaging systems with reduced optical complexity such as a smaller number of lens elements. Unfortunately such simplified optical systems usually suffer from severe aberrations, especially in off-axis regions, which can be difficult to correct purely in software. In this paper we introduce Fovea Stacking , a new type of imaging system that utilizes emerging dynamic optical components called deformable phase plates (DPPs) for localized aberration correction anywhere on the image sensor. By optimizing DPP deformations through a differentiable optical model, off-axis aberrations are corrected locally, producing a foveated image with enhanced sharpness at the fixation point - analogous to the eye’s fovea. Stacking multiple such foveated images, each with a different fixation point, yields a composite image free from aberrations. To efficiently cover the entire field of view, we propose joint optimization of DPP deformations under imaging budget constraints. Due to the DPP device’s non-linear behavior, we introduce a neural network-based control model for improved alignment between simulation-hardware performance. We further demonstrated that for extended depth-of-field imaging, fovea stacking outperforms traditional focus stacking in image quality. By integrating object detection or eye-tracking, the system can dynamically adjust the lens to track the object of interest-enabling real-time foveated video suitable for downstream applications such as surveillance or foveated virtual reality displays

Method: Proposes SSGCO (structural-spectral graph convolutional operator) for co-extraction of spatial-spectral features, and EGAEL (evidence-guided adaptive edge learning) module to refine edge weights in superpixel graphs. Integrated into contrastive learning framework for simultaneous representation learning and clustering.

Result: Achieves improvements of 2.61%, 6.06%, 4.96% and 3.15% clustering accuracy over best compared methods on four HSI datasets.

Conclusion: The proposed SSGCO and EGAEL modules effectively address spectral information utilization and graph topology issues in HSI clustering, demonstrating significant performance improvements across multiple datasets.

Abstract: Hyperspectral image (HSI) clustering assigns similar pixels to the same class without any annotations, which is an important yet challenging task. For large-scale HSIs, most methods rely on superpixel segmentation and perform superpixel-level clustering based on graph neural networks (GNNs). However, existing GNNs cannot fully exploit the spectral information of the input HSI, and the inaccurate superpixel topological graph may lead to the confusion of different class semantics during information aggregation. To address these challenges, we first propose a structural-spectral graph convolutional operator (SSGCO) tailored for graph-structured HSI superpixels to improve their representation quality through the co-extraction of spatial and spectral features. Second, we propose an evidence-guided adaptive edge learning (EGAEL) module that adaptively predicts and refines edge weights in the superpixel topological graph. We integrate the proposed method into a contrastive learning framework to achieve clustering, where representation learning and clustering are simultaneously conducted. Experiments demonstrate that the proposed method improves clustering accuracy by 2.61%, 6.06%, 4.96% and 3.15% over the best compared methods on four HSI datasets. Our code is available at https://github.com/jhqi/SSGCO-EGAEL.

Method: Integrates Knowledge Acquisition and Accumulation (KAA) using monotask learning on heterogeneous single-label datasets, combined with Consistency-based Active Learning (CAL) to bridge single- and multi-label data gaps.

Result: 56.1% improvement over ImageNet-pretrained baseline on DSI dataset; outperforms state-of-the-art methods on BDD100K and HSD datasets with 85% less data; recognizes attributes unseen during foundation training.

Conclusion: KAA-CAL provides effective solution for multi-attribute scene identification, enabling autonomous vehicles to better understand complex driving environments with reduced data annotation requirements.

Abstract: Driving scenes are inherently heterogeneous and dynamic. Multi-attribute scene identification, as a high-level visual perception capability, provides autonomous vehicles (AVs) with essential contextual awareness to understand, reason through, and interact with complex driving environments. Although scene identification is best modeled as a multi-label classification problem via multitask learning, it faces two major challenges: the difficulty of acquiring balanced, comprehensively annotated datasets and the need to re-annotate all training data when new attributes emerge. To address these challenges, this paper introduces a novel deep learning method that integrates Knowledge Acquisition and Accumulation (KAA) with Consistency-based Active Learning (CAL). KAA leverages monotask learning on heterogeneous single-label datasets to build a knowledge foundation, while CAL bridges the gap between single- and multi-label data, adapting the foundation model for multi-label scene classification. An ablation study on the newly developed Driving Scene Identification (DSI) dataset demonstrates a 56.1% improvement over an ImageNet-pretrained baseline. Moreover, KAA-CAL outperforms state-of-the-art multi-label classification methods on the BDD100K and HSD datasets, achieving this with 85% less data and even recognizing attributes unseen during foundation model training. The DSI dataset and KAA-CAL implementation code are publicly available at https://github.com/KELISBU/KAA-CAL .

Method: Analyzed 20 pipeline configurations and 15 attack scenarios to study backdoor attacks on face feature extractors trained with large margin metric learning losses.

Result: Face feature extractors are susceptible to backdoor attacks, and a single backdoor can compromise an entire face recognition system.

Conclusion: Proposed effective best practices and countermeasures for stakeholders to address backdoor vulnerabilities in face recognition systems.

Abstract: The widespread deployment of Deep Learning-based Face Recognition Systems raises multiple security concerns. While prior research has identified backdoor vulnerabilities on isolated components, Backdoor Attacks on real-world, unconstrained pipelines remain underexplored. This paper presents the first comprehensive system-level analysis of Backdoor Attacks targeting Face Recognition Systems and provides three contributions. We first show that face feature extractors trained with large margin metric learning losses are susceptible to Backdoor Attacks. By analyzing 20 pipeline configurations and 15 attack scenarios, we then reveal that a single backdoor can compromise an entire Face Recognition System. Finally, we propose effective best practices and countermeasures for stakeholders.

Method: Using physics simulations to generate synthetic medical images with pixel-level segmentation annotations. Applied specifically to breast imaging to create paired 2D digital mammography (DM) and 3D digital breast tomosynthesis (DBT) images.

Result: Created T-SYNTH, a large-scale open-source dataset of synthetic breast images. Initial experiments show promise for using these synthetic images to augment limited real patient datasets for detection tasks in both DM and DBT.

Conclusion: Physics-based synthetic data generation is a viable approach to address annotation scarcity in medical imaging. T-SYNTH dataset demonstrates potential for improving algorithm development and assessment in breast imaging analysis through data augmentation.

Abstract: One of the key impediments for developing and assessing robust medical imaging algorithms is limited access to large-scale datasets with suitable annotations. Synthetic data generated with plausible physical and biological constraints may address some of these data limitations. We propose the use of physics simulations to generate synthetic images with pixel-level segmentation annotations, which are notoriously difficult to obtain. Specifically, we apply this approach to breast imaging analysis and release T-SYNTH, a large-scale open-source dataset of paired 2D digital mammography (DM) and 3D digital breast tomosynthesis (DBT) images. Our initial experimental results indicate that T-SYNTH images show promise for augmenting limited real patient datasets for detection tasks in DM and DBT. Our data and code are publicly available at https://github.com/DIDSR/tsynth-release.

Method: EnCoBo uses a decoder-free, energy-based framework that directly guides generation in latent space through diffusion-scheduled energy functions, eliminating the need for auxiliary visual cues and black-box decoders.

Result: Experiments on CelebA-HQ and CUB datasets showed improved concept-level human intervention and interpretability while maintaining competitive visual quality compared to existing approaches.

Conclusion: EnCoBo successfully addresses the limitations of traditional generative CBMs by providing a pure concept-based bottleneck that supports robust interventions like concept composition and negation, enhancing interpretability without sacrificing generation quality.

Abstract: Concept Bottleneck Models (CBMs) provide interpretable decision-making through explicit, human-understandable concepts. However, existing generative CBMs often rely on auxiliary visual cues at the bottleneck, which undermines interpretability and intervention capabilities. We propose EnCoBo, a post-hoc concept bottleneck for generative models that eliminates auxiliary cues by constraining all representations to flow solely through explicit concepts. Unlike autoencoder-based approaches that inherently rely on black-box decoders, EnCoBo leverages a decoder-free, energy-based framework that directly guides generation in the latent space. Guided by diffusion-scheduled energy functions, EnCoBo supports robust post-hoc interventions-such as concept composition and negation-across arbitrary concepts. Experiments on CelebA-HQ and CUB datasets showed that EnCoBo improved concept-level human intervention and interpretability while maintaining competitive visual quality.

Method: Combines autoregressive and diffusion models, uses Multi-Scale Pose Representation to extract detailed features from different articulators, and implements Confidence-Aware Causal Attention with joint-level confidence scores.

Result: Extensive experiments on PHOENIX14T and How2Sign datasets show effectiveness in both generation quality and real-time efficiency.

Conclusion: The hybrid approach successfully addresses limitations of both autoregressive and diffusion models, achieving high-quality sign language production suitable for real-time applications.

Abstract: Earlier Sign Language Production (SLP) models typically relied on autoregressive methods that generate output tokens one by one, which inherently provide temporal alignment. Although techniques like Teacher Forcing can prevent model collapse during training, they still cannot solve the problem of error accumulation during inference, since ground truth is unavailable at that stage. In contrast, more recent approaches based on diffusion models leverage step-by-step denoising to enable high-quality generation. However, the iterative nature of these models and the requirement to denoise entire sequences limit their applicability in real-time tasks like SLP. To address it, we explore a hybrid approach that combines autoregressive and diffusion models for SLP, leveraging the strengths of both models in sequential dependency modeling and output refinement. To capture fine-grained body movements, we design a Multi-Scale Pose Representation module that separately extracts detailed features from distinct articulators and integrates them via a Multi-Scale Fusion module. Furthermore, we introduce a Confidence-Aware Causal Attention mechanism that utilizes joint-level confidence scores to dynamically guide the pose generation process, improving accuracy and robustness. Extensive experiments on the PHOENIX14T and How2Sign datasets demonstrate the effectiveness of our method in both generation quality and real-time efficiency.

Method: Proposes DGFDNet with HAFM module for haze-aware frequency modulation using dark channel priors, MGAM for multi-scale feature fusion, and PCGB with closed-loop feedback for iterative prior refinement.

Result: Achieves state-of-the-art performance on four benchmark datasets with superior robustness and real-time efficiency.

Conclusion: The dual-domain approach with physical guidance and iterative refinement effectively addresses computational limitations while maintaining high dehazing performance in complex conditions.

Abstract: Transformer-based models exhibit strong global modeling capabilities in single-image dehazing, but their high computational cost limits real-time applicability. Existing methods predominantly rely on spatial-domain features to capture long-range dependencies, which are computationally expensive and often inadequate under complex haze conditions. While some approaches introduce frequency-domain cues, the weak coupling between spatial and frequency branches limits the overall performance. To overcome these limitations, we propose the Dark Channel Guided Frequency-aware Dehazing Network (DGFDNet), a novel dual-domain framework that performs physically guided degradation alignment across spatial and frequency domains. At its core, the DGFDBlock comprises two key modules: 1) the Haze-Aware Frequency Modulator (HAFM), which generates a pixel-level haze confidence map from dark channel priors to adaptively enhance haze-relevant frequency components, thereby achieving global degradation-aware spectral modulation; 2) the Multi-level Gating Aggregation Module (MGAM), which fuses multi-scale features through diverse convolutional kernels and hybrid gating mechanisms to recover fine structural details. Additionally, a Prior Correction Guidance Branch (PCGB) incorporates a closed-loop feedback mechanism, enabling iterative refinement of the prior by intermediate dehazed features and significantly improving haze localization accuracy, especially in challenging outdoor scenes. Extensive experiments on four benchmark haze datasets demonstrate that DGFDNet achieves state-of-the-art performance with superior robustness and real-time efficiency. Code is available at: https://github.com/Dilizlr/DGFDNet.

Method: Trains a multimodal LLM to generate embodied reasoning plans using reinforced visual rewards based on goal completion and trajectory consistency, then compresses these plans into visual plan latents that condition a downstream action model.

Result: Extensive experiments show ThinkAct enables few-shot adaptation, long-horizon planning, and self-correction behaviors in complex embodied AI tasks across embodied reasoning and robot manipulation benchmarks.

Conclusion: The dual-system approach with reinforced visual latent planning effectively bridges high-level reasoning with low-level execution, outperforming end-to-end methods in complex VLA reasoning tasks.

Abstract: Vision-language-action (VLA) reasoning tasks require agents to interpret multimodal instructions, perform long-horizon planning, and act adaptively in dynamic environments. Existing approaches typically train VLA models in an end-to-end fashion, directly mapping inputs to actions without explicit reasoning, which hinders their ability to plan over multiple steps or adapt to complex task variations. In this paper, we propose ThinkAct, a dual-system framework that bridges high-level reasoning with low-level action execution via reinforced visual latent planning. ThinkAct trains a multimodal LLM to generate embodied reasoning plans guided by reinforcing action-aligned visual rewards based on goal completion and trajectory consistency. These reasoning plans are compressed into a visual plan latent that conditions a downstream action model for robust action execution on target environments. Extensive experiments on embodied reasoning and robot manipulation benchmarks demonstrate that ThinkAct enables few-shot adaptation, long-horizon planning, and self-correction behaviors in complex embodied AI tasks.

Method: Created SCORPION dataset with 480 tissue samples each scanned by 5 scanners (2,400 aligned patches), and proposed SimCons framework combining augmentation-based domain generalization with consistency loss.

Result: SimCons improves model consistency across varying scanners without compromising task-specific performance, enabling rigorous evaluation of scanner-induced variability.

Conclusion: SCORPION dataset and SimCons framework provide crucial resources for evaluating and improving model consistency across diverse scanners, setting new standards for reliability testing in computational pathology.

Abstract: Ensuring reliable model performance across diverse domains is a critical challenge in computational pathology. A particular source of variability in Whole-Slide Images is introduced by differences in digital scanners, thus calling for better scanner generalization. This is critical for the real-world adoption of computational pathology, where the scanning devices may differ per institution or hospital, and the model should not be dependent on scanner-induced details, which can ultimately affect the patient’s diagnosis and treatment planning. However, past efforts have primarily focused on standard domain generalization settings, evaluating on unseen scanners during training, without directly evaluating consistency across scanners for the same tissue. To overcome this limitation, we introduce SCORPION, a new dataset explicitly designed to evaluate model reliability under scanner variability. SCORPION includes 480 tissue samples, each scanned with 5 scanners, yielding 2,400 spatially aligned patches. This scanner-paired design allows for the isolation of scanner-induced variability, enabling a rigorous evaluation of model consistency while controlling for differences in tissue composition. Furthermore, we propose SimCons, a flexible framework that combines augmentation-based domain generalization techniques with a consistency loss to explicitly address scanner generalization. We empirically show that SimCons improves model consistency on varying scanners without compromising task-specific performance. By releasing the SCORPION dataset and proposing SimCons, we provide the research community with a crucial resource for evaluating and improving model consistency across diverse scanners, setting a new standard for reliability testing.

Method: Introduces dedicated 3D eyeball structure with 3D Gaussian Splatting for explicit rotation/translation control, plus adaptive deformation module for subtle eye muscle movements.

Result: Demonstrates photorealistic gaze redirection with superior image quality and gaze estimation accuracy on ETH-XGaze dataset compared to state-of-the-art methods.

Conclusion: Explicit 3D eyeball modeling with 3DGS enables more accurate and photorealistic gaze redirection than implicit NeRF-based approaches, with better control over eye movements.

Abstract: We propose a novel 3D gaze redirection framework that leverages an explicit 3D eyeball structure. Existing gaze redirection methods are typically based on neural radiance fields, which employ implicit neural representations via volume rendering. Unlike these NeRF-based approaches, where the rotation and translation of 3D representations are not explicitly modeled, we introduce a dedicated 3D eyeball structure to represent the eyeballs with 3D Gaussian Splatting (3DGS). Our method generates photorealistic images that faithfully reproduce the desired gaze direction by explicitly rotating and translating the 3D eyeball structure. In addition, we propose an adaptive deformation module that enables the replication of subtle muscle movements around the eyes. Through experiments conducted on the ETH-XGaze dataset, we demonstrate that our framework is capable of generating diverse novel gaze images, achieving superior image quality and gaze estimation accuracy compared to previous state-of-the-art methods.

Method: Extensive evaluation of SAEs across three vision architectures using image-based tasks, including OOD detection, semantic steering, and automated attribute discovery pipelines.

Result: SAE features are semantically meaningful, improve out-of-distribution generalization, recover ontological structure, enable semantic steering in diffusion models, and reveal shared representations across vision-language modalities.

Conclusion: SAEs show strong potential for improving interpretability, generalization, and steerability in vision models, providing a foundation for future SAE evaluation in the visual domain.

Abstract: Sparse Autoencoders (SAEs) have emerged as a popular tool for interpreting the hidden states of large language models (LLMs). By learning to reconstruct activations from a sparse bottleneck layer, SAEs discover interpretable features from the high-dimensional internal representations of LLMs. Despite their popularity with language models, SAEs remain understudied in the visual domain. In this work, we provide an extensive evaluation the representational power of SAEs for vision models using a broad range of image-based tasks. Our experimental results demonstrate that SAE features are semantically meaningful, improve out-of-distribution generalization, and enable controllable generation across three vision model architectures: vision embedding models, multi-modal LMMs and diffusion models. In vision embedding models, we find that learned SAE features can be used for OOD detection and provide evidence that they recover the ontological structure of the underlying model. For diffusion models, we demonstrate that SAEs enable semantic steering through text encoder manipulation and develop an automated pipeline for discovering human-interpretable attributes. Finally, we conduct exploratory experiments on multi-modal LLMs, finding evidence that SAE features reveal shared representations across vision and language modalities. Our study provides a foundation for SAE evaluation in vision models, highlighting their strong potential improving interpretability, generalization, and steerability in the visual domain.

Method: Combined YOLOv8 segmentation model, LSTM trained on hand point motion sequences, and Whisper ASR to extract data for TinyLLaMa recipe prediction.

Result: Developed a system capable of predicting recipes and generating step-by-step cooking guides from video data.

Conclusion: Demonstrates the endless applications of computer vision in daily life activities and extends the field for crucial day-to-day tasks.

Abstract: This is a research exploring existing models and fine tuning them to combine a YOLOv8 segmentation model, a LSTM model trained on hand point motion sequence and a ASR (whisper-base) to extract enough data for a LLM (TinyLLaMa) to predict the recipe and generate text creating a step by step guide for the cooking procedure. All the data were gathered by the author for a robust task specific system to perform best in complex and challenging environments proving the extension and endless application of computer vision in daily activities such as kitchen work. This work extends the field for many more crucial task of our day to day life.

Method: Proposed two attention-guided autoencoder architectures: 1) Structural similarity-based scoring for real-time defect detection with unsupervised thresholding, 2) Feature distance-based strategy using Mahalanobis scoring on reduced latent features for monitoring distributional shifts.

Result: Both approaches outperform baselines on sterile packaging dataset. Structural similarity method generalizes well on MVTec-Zipper benchmark (comparable to SOTA), while feature distance method provides complementary monitoring capabilities but is less transferable.

Conclusion: Dual-pathway inspection strategy: structural similarity for robust inline detection and feature distance for supervisory monitoring. Methods combine performance with interpretability and align with regulatory expectations for high-risk AI systems.

Abstract: Automated visual inspection in medical device manufacturing faces unique challenges, including small and imbalanced datasets, high-resolution imagery, and strict regulatory requirements. To address these, we propose two attention-guided autoencoder architectures for deep anomaly detection. The first employs a structural similarity-based scoring approach that enables lightweight, real-time defect detection with unsupervised thresholding and can be further enhanced through limited supervised tuning. The second applies a feature distance-based strategy using Mahalanobis scoring on reduced latent features, designed to monitor distributional shifts and support supervisory oversight. Evaluations on a representative sterile packaging dataset confirm that both approaches outperform baselines under hardware-constrained, regulated conditions. Cross-domain testing on the MVTec-Zipper benchmark further demonstrates that the structural similarity-based method generalises effectively and achieves performance comparable to state-of-the-art methods, while the feature distance-based method is less transferable but provides complementary monitoring capabilities. These results highlight a dual-pathway inspection strategy: structural similarity for robust inline detection and feature distance for supervisory monitoring. By combining operational performance with interpretability and lifecycle monitoring, the proposed methods also align with emerging regulatory expectations for high-risk AI systems.

Method: Uses target-supervised triplet alignment with (reference image, instruction, target images) triplets, Direct Subject Diffusion objective, and preference learning with systematically generated winning/losing targets.

Result: Superior performance on standard benchmarks, outperforming baselines in subject faithfulness and instruction compliance across multiple quantitative metrics.

Conclusion: TIDE effectively resolves the preservation-compliance tension without test-time fine-tuning and demonstrates versatility across diverse image generation tasks.

Abstract: Subject-driven image generation (SDIG) aims to manipulate specific subjects within images while adhering to textual instructions, a task crucial for advancing text-to-image diffusion models. SDIG requires reconciling the tension between maintaining subject identity and complying with dynamic edit instructions, a challenge inadequately addressed by existing methods. In this paper, we introduce the Target-Instructed Diffusion Enhancing (TIDE) framework, which resolves this tension through target supervision and preference learning without test-time fine-tuning. TIDE pioneers target-supervised triplet alignment, modelling subject adaptation dynamics using a (reference image, instruction, target images) triplet. This approach leverages the Direct Subject Diffusion (DSD) objective, training the model with paired “winning” (balanced preservation-compliance) and “losing” (distorted) targets, systematically generated and evaluated via quantitative metrics. This enables implicit reward modelling for optimal preservation-compliance balance. Experimental results on standard benchmarks demonstrate TIDE’s superior performance in generating subject-faithful outputs while maintaining instruction compliance, outperforming baseline methods across multiple quantitative metrics. TIDE’s versatility is further evidenced by its successful application to diverse tasks, including structural-conditioned generation, image-to-image generation, and text-image interpolation. Our code is available at https://github.com/KomJay520/TIDE.

Method: Proposes a composite loss function that treats high-dimensional samples as one-dimensional Gaussian variables, applying moment regularization in spatial domain and power spectrum regularization in spectral domain. Uses random permutations for invariance and leverages analytically known expected values of moments and power spectrum.

Result: Outperforms previous Gaussianity regularization methods, effectively prevents reward hacking, and accelerates convergence in text-to-image model optimization tasks.

Conclusion: The proposed unified framework for Gaussian regularization provides an efficient and effective approach for latent space optimization in generative models, with applications in enhancing aesthetics and text alignment in text-to-image generation.

Abstract: We propose a novel regularization loss that enforces standard Gaussianity, encouraging samples to align with a standard Gaussian distribution. This facilitates a range of downstream tasks involving optimization in the latent space of text-to-image models. We treat elements of a high-dimensional sample as one-dimensional standard Gaussian variables and define a composite loss that combines moment-based regularization in the spatial domain with power spectrum-based regularization in the spectral domain. Since the expected values of moments and power spectrum distributions are analytically known, the loss promotes conformity to these properties. To ensure permutation invariance, the losses are applied to randomly permuted inputs. Notably, existing Gaussianity-based regularizations fall within our unified framework: some correspond to moment losses of specific orders, while the previous covariance-matching loss is equivalent to our spectral loss but incurs higher time complexity due to its spatial-domain computation. We showcase the application of our regularization in generative modeling for test-time reward alignment with a text-to-image model, specifically to enhance aesthetics and text alignment. Our regularization outperforms previous Gaussianity regularization, effectively prevents reward hacking and accelerates convergence.

Method: Reconstruction Alignment (RecA) conditions models on their own visual understanding embeddings and optimizes them to reconstruct input images using self-supervised reconstruction loss.

Result: RecA consistently improves generation and editing across different UMM architectures, boosting GenEval (0.73→0.90), DPGBench (80.93→88.15), ImgEdit (3.38→3.75), and GEdit (6.94→7.25) with only 27 GPU-hours.

Conclusion: RecA is an efficient and general post-training alignment strategy that surpasses larger models and works across diverse UMM architectures.

Abstract: Unified multimodal models (UMMs) unify visual understanding and generation within a single architecture. However, conventional training relies on image-text pairs (or sequences) whose captions are typically sparse and miss fine-grained visual details–even when they use hundreds of words to describe a simple image. We introduce Reconstruction Alignment (RecA), a resource-efficient post-training method that leverages visual understanding encoder embeddings as dense “text prompts,” providing rich supervision without captions. Concretely, RecA conditions a UMM on its own visual understanding embeddings and optimizes it to reconstruct the input image with a self-supervised reconstruction loss, thereby realigning understanding and generation. Despite its simplicity, RecA is broadly applicable: across autoregressive, masked-autoregressive, and diffusion-based UMMs, it consistently improves generation and editing fidelity. With only 27 GPU-hours, post-training with RecA substantially improves image generation performance on GenEval (0.73$\rightarrow$0.90) and DPGBench (80.93$\rightarrow$88.15), while also boosting editing benchmarks (ImgEdit 3.38$\rightarrow$3.75, GEdit 6.94$\rightarrow$7.25). Notably, RecA surpasses much larger open-source models and applies broadly across diverse UMM architectures, establishing it as an efficient and general post-training alignment strategy for UMMs

Method: Dual-reference, dual-stream architecture with: 1) wrist-centric frame processed by topology-aware graph convolution for view-invariant shape modeling, 2) facial-centric frame processed by Finsler geometry-based encoder for context-aware trajectory modeling, 3) geometry-driven optimal transport fusion mechanism to integrate features.

Result: Achieved 93.70% accuracy on WLASL-100, 89.97% on WLASL-300, and 99.79% on LSA64 datasets, setting new state-of-the-art performance with significantly fewer parameters than competing models.

Conclusion: DSLNet effectively resolves geometric ambiguity in sign language recognition by decoupling gesture morphology and trajectory modeling in complementary coordinate systems, demonstrating superior performance and efficiency compared to existing approaches.

Abstract: Isolated Sign Language Recognition (ISLR) is challenged by gestures that are morphologically similar yet semantically distinct, a problem rooted in the complex interplay between hand shape and motion trajectory. Existing methods, often relying on a single reference frame, struggle to resolve this geometric ambiguity. This paper introduces Dual-SignLanguageNet (DSLNet), a dual-reference, dual-stream architecture that decouples and models gesture morphology and trajectory in separate, complementary coordinate systems. The architecture processes these streams through specialized networks: a topology-aware graph convolution models the view-invariant shape from a wrist-centric frame, while a Finsler geometry-based encoder captures the context-aware trajectory from a facial-centric frame. These features are then integrated via a geometry-driven optimal transport fusion mechanism. DSLNet sets a new state-of-the-art, achieving 93.70%, 89.97%, and 99.79% accuracy on the challenging WLASL-100, WLASL-300, and LSA64 datasets, respectively, with significantly fewer parameters than competing models.

Method: Uses features generated by a Vision Diffusion Model (VDM) aggregated via a transformer. The model is conditioned on earlier timesteps of the diffusion process to emphasize semantic information over pixel-level details.

Result: Sets new state-of-the-art across three generalization benchmarks: classifying actions across animal species, different viewing angles, and different recording contexts.

Conclusion: The approach brings machine action recognition closer to human-like robustness by leveraging diffusion model features to achieve superior generalization across challenging conditions.

Abstract: Humans can recognize the same actions despite large context and viewpoint variations, such as differences between species (walking in spiders vs. horses), viewpoints (egocentric vs. third-person), and contexts (real life vs movies). Current deep learning models struggle with such generalization. We propose using features generated by a Vision Diffusion Model (VDM), aggregated via a transformer, to achieve human-like action recognition across these challenging conditions. We find that generalization is enhanced by the use of a model conditioned on earlier timesteps of the diffusion process to highlight semantic information over pixel level details in the extracted features. We experimentally explore the generalization properties of our approach in classifying actions across animal species, across different viewing angles, and different recording contexts. Our model sets a new state-of-the-art across all three generalization benchmarks, bringing machine action recognition closer to human-like robustness. Project page: $\href{https://www.vision.caltech.edu/actiondiff/}{\text{vision.caltech.edu/actiondiff}}$ Code: $\href{https://github.com/frankyaoxiao/ActionDiff}{\text{github.com/frankyaoxiao/ActionDiff}}$

Method: Uses On-the-Fly SfM to compute pose and sparse point cloud for each new image, integrates new Gaussians into previously unseen regions, and employs orthogonal splatting for immediate rendering after each 3DGS field update.

Result: A-TDOM achieves near real-time TDOM generation with 3DGS optimization for each new image completed in seconds while maintaining acceptable rendering quality and geometric accuracy.

Conclusion: The proposed method enables active TDOM rendering in near real-time, overcoming limitations of traditional offline approaches and addressing quality degradation challenges.

Abstract: True Digital Orthophoto Map (TDOM) serves as a crucial geospatial product in various fields such as urban management, city planning, land surveying, etc. However, traditional TDOM generation methods generally rely on a complex offline photogrammetric pipeline, resulting in delays that hinder real-time applications. Moreover, the quality of TDOM may degrade due to various challenges, such as inaccurate camera poses or Digital Surface Model (DSM) and scene occlusions. To address these challenges, this work introduces A-TDOM, a near real-time TDOM generation method based on On-the-Fly 3DGS optimization. As each image is acquired, its pose and sparse point cloud are computed via On-the-Fly SfM. Then new Gaussians are integrated and optimized into previously unseen or coarsely reconstructed regions. By integrating with orthogonal splatting, A-TDOM can render just after each update of a new 3DGS field. Initial experiments on multiple benchmarks show that the proposed A-TDOM is capable of actively rendering TDOM in near real-time, with 3DGS optimization for each new image in seconds while maintaining acceptable rendering quality and TDOM geometric accuracy.

Method: The method uses regression for PCR validation rather than classification, extends misalignment-related features with multiscale extraction, and employs attention-based aggregation for feature processing.

Result: The approach achieves accurate and robust registration error estimation on diverse datasets, particularly for point clouds with heterogeneous spatial densities. When used to guide mapping tasks, it significantly improves mapping quality compared to state-of-the-art classification-based methods.

Conclusion: Regression-based PCR validation with multiscale feature extraction and attention-based aggregation provides superior fine-grained quality quantification compared to classification approaches, especially benefiting applications with varied point cloud densities.

Abstract: Point cloud registration (PCR) is crucial for many downstream tasks, such as simultaneous localization and mapping (SLAM) and object tracking. This makes detecting and quantifying registration misalignment, i.e., PCR quality validation, an important task. All existing methods treat validation as a classification task, aiming to assign the PCR quality to a few classes. In this work, we instead use regression for PCR validation, allowing for a more fine-grained quantification of the registration quality. We also extend previously used misalignment-related features by using multiscale extraction and attention-based aggregation. This leads to accurate and robust registration error estimation on diverse datasets, especially for point clouds with heterogeneous spatial densities. Furthermore, when used to guide a mapping downstream task, our method significantly improves the mapping quality for a given amount of re-registered frames, compared to the state-of-the-art classification-based method.

Method: Using a simple linear classifier on top of modern Vision Foundation Models (VFMs) trained on identical data as specialized detectors, and analyzing text-image similarities to understand why VFMs perform better.

Result: The VFM-based approach decisively outperforms bespoke detectors, boosting in-the-wild accuracy by over 20%. Analysis shows recent VLMs have learned to align synthetic images with forgery-related concepts through data exposure during pre-training.

Conclusion: Modern VFMs provide superior detection capabilities compared to specialized static detectors, and true generalization evaluation requires test data independent of the model’s entire training history including pre-training data.

Abstract: While specialized detectors for AI-generated images excel on curated benchmarks, they fail catastrophically in real-world scenarios, as evidenced by their critically high false-negative rates on in-the-wild' benchmarks. Instead of crafting another specialized knife’ for this problem, we bring a gun' to the fight: a simple linear classifier on a modern Vision Foundation Model (VFM). Trained on identical data, this baseline decisively outguns’ bespoke detectors, boosting in-the-wild accuracy by a striking margin of over 20%. Our analysis pinpoints the source of the VFM’s firepower': First, by probing text-image similarities, we find that recent VLMs (e.g., Perception Encoder, Meta CLIP2) have learned to align synthetic images with forgery-related concepts (e.g., AI-generated’), unlike previous versions. Second, we speculate that this is due to data exposure, as both this alignment and overall accuracy plummet on a novel dataset scraped after the VFM’s pre-training cut-off date, ensuring it was unseen during pre-training. Our findings yield two critical conclusions: 1) For the real-world gunfight' of AI-generated image detection, the raw firepower’ of an updated VFM is far more effective than the `craftsmanship’ of a static detector. 2) True generalization evaluation requires test data to be independent of the model’s entire training history, including pre-training.

Method: Proposes Block-Wise Caching (BWCache) that dynamically caches and reuses features from DiT blocks across timesteps, with a similarity indicator to trigger reuse only when feature differences are below a threshold.

Result: Extensive experiments show BWCache achieves up to 2.24x speedup while maintaining comparable visual quality across several video diffusion models.

Conclusion: BWCache effectively reduces computational redundancy in DiT-based video generation through intelligent feature caching and reuse, providing significant acceleration without compromising visual fidelity.

Abstract: Recent advancements in Diffusion Transformers (DiTs) have established them as the state-of-the-art method for video generation. However, their inherently sequential denoising process results in inevitable latency, limiting real-world applicability. Existing acceleration methods either compromise visual quality due to architectural modifications or fail to reuse intermediate features at proper granularity. Our analysis reveals that DiT blocks are the primary contributors to inference latency. Across diffusion timesteps, the feature variations of DiT blocks exhibit a U-shaped pattern with high similarity during intermediate timesteps, which suggests substantial computational redundancy. In this paper, we propose Block-Wise Caching (BWCache), a training-free method to accelerate DiT-based video generation. BWCache dynamically caches and reuses features from DiT blocks across diffusion timesteps. Furthermore, we introduce a similarity indicator that triggers feature reuse only when the differences between block features at adjacent timesteps fall below a threshold, thereby minimizing redundant computations while maintaining visual fidelity. Extensive experiments on several video diffusion models demonstrate that BWCache achieves up to 2.24$\times$ speedup with comparable visual quality.

Method: End4 uses an end-to-end denoising diffusion approach with a denoising reconstruction model to improve latent space alignment between reconstruction and detection processes. It employs a Scale-aware Pyramid-like Fusion Module (SPFM) to refine local image features using attention pyramid layers at different scales.

Result: Extensive experiments show that End4 effectively generalizes to unseen masking patterns and remains robust under various perturbations. The method outperforms existing approaches on a comprehensive benchmark comprising images from five distinct masked regions.

Conclusion: End4 provides an effective solution for detecting diffusion-based inpainted images, addressing the security concerns around potential misuse of diffusion models while maintaining robustness across different masking patterns and perturbations.

Abstract: The powerful generative capabilities of diffusion models have significantly advanced the field of image synthesis, enhancing both full image generation and inpainting-based image editing. Despite their remarkable advancements, diffusion models also raise concerns about potential misuse for malicious purposes. However, existing approaches struggle to identify images generated by diffusion-based inpainting models, even when similar inpainted images are included in their training data. To address this challenge, we propose a novel detection method based on End-to-end denoising diffusion (End4). Specifically, End4 designs a denoising reconstruction model to improve the alignment degree between the latent spaces of the reconstruction and detection processes, thus reconstructing features that are more conducive to detection. Meanwhile, it leverages a Scale-aware Pyramid-like Fusion Module (SPFM) that refines local image features under the guidance of attention pyramid layers at different scales, enhancing feature discriminability. Additionally, to evaluate detection performance on inpainted images, we establish a comprehensive benchmark comprising images generated from five distinct masked regions. Extensive experiments demonstrate that our End4 effectively generalizes to unseen masking patterns and remains robust under various perturbations. Our code and dataset will be released soon.

Method: A modular three-stage pipeline: (1) off-the-shelf human detection and depth estimation, (2) VLM-based reasoning to classify pairwise social affiliation, and (3) lightweight spatial aggregation algorithm to localize socially connected groups.

Result: Developed a new dataset of 100K urban street-view images with bounding boxes and labels for individuals and social groups, combining human annotations and MINGLE outputs for semantic richness.

Conclusion: MINGLE provides an effective framework for detecting social group regions from images, enabling better understanding of social interactions in public spaces for urban planning applications.

Abstract: Understanding group-level social interactions in public spaces is crucial for urban planning, informing the design of socially vibrant and inclusive environments. Detecting such interactions from images involves interpreting subtle visual cues such as relations, proximity, and co-movement - semantically complex signals that go beyond traditional object detection. To address this challenge, we introduce a social group region detection task, which requires inferring and spatially grounding visual regions defined by abstract interpersonal relations. We propose MINGLE (Modeling INterpersonal Group-Level Engagement), a modular three-stage pipeline that integrates: (1) off-the-shelf human detection and depth estimation, (2) VLM-based reasoning to classify pairwise social affiliation, and (3) a lightweight spatial aggregation algorithm to localize socially connected groups. To support this task and encourage future research, we present a new dataset of 100K urban street-view images annotated with bounding boxes and labels for both individuals and socially interacting groups. The annotations combine human-created labels and outputs from the MINGLE pipeline, ensuring semantic richness and broad coverage of real-world scenarios.

Method: Proposes Gated Residual Tokenization (GRT) - a two-stage framework: 1) Motion-Compensated Inter-Gated Tokenization uses pixel-level motion estimation to skip static regions, achieving sub-linear token growth. 2) Semantic-Scene Intra-Tokenization Merging fuses tokens across static regions to reduce redundancy while preserving dynamic semantics.

Result: Experiments on the proposed DIVE benchmark show that GRT outperforms larger VLLM baselines and scales positively with FPS, demonstrating efficient high-FPS video understanding.

Conclusion: The results highlight the importance of dense temporal information and demonstrate that GRT enables efficient, scalable high-FPS video understanding, addressing the limitations of current low-frame-rate approaches.

Abstract: High temporal resolution is essential for capturing fine-grained details in video understanding. However, current video large language models (VLLMs) and benchmarks mostly rely on low-frame-rate sampling, such as uniform sampling or keyframe selection, discarding dense temporal information. This compromise avoids the high cost of tokenizing every frame, which otherwise leads to redundant computation and linear token growth as video length increases. While this trade-off works for slowly changing content, it fails for tasks like lecture comprehension, where information appears in nearly every frame and requires precise temporal alignment. To address this gap, we introduce Dense Video Understanding (DVU), which enables high-FPS video comprehension by reducing both tokenization time and token overhead. Existing benchmarks are also limited, as their QA pairs focus on coarse content changes. We therefore propose DIVE (Dense Information Video Evaluation), the first benchmark designed for dense temporal reasoning. To make DVU practical, we present Gated Residual Tokenization (GRT), a two-stage framework: (1) Motion-Compensated Inter-Gated Tokenization uses pixel-level motion estimation to skip static regions during tokenization, achieving sub-linear growth in token count and compute. (2) Semantic-Scene Intra-Tokenization Merging fuses tokens across static regions within a scene, further reducing redundancy while preserving dynamic semantics. Experiments on DIVE show that GRT outperforms larger VLLM baselines and scales positively with FPS. These results highlight the importance of dense temporal information and demonstrate that GRT enables efficient, scalable high-FPS video understanding.

Method: Formulates anomaly detection as multimodal contrastive learning using DINOv3 as visual backbone and CLIP text encoder. Uses lightweight adapters to bridge domain gap and an Anomaly-Aware Calibration Module (AACM) to guide attention to anomalous regions rather than generic foreground semantics.

Result: Extensive experiments on eight industrial and medical benchmarks show AD-DINOv3 consistently matches or surpasses state-of-the-art methods.

Conclusion: AD-DINOv3 successfully adapts DINOv3 for zero-shot anomaly detection, overcoming domain and semantic biases through multimodal alignment and specialized calibration, achieving superior performance across diverse benchmarks.

Abstract: Zero-Shot Anomaly Detection (ZSAD) seeks to identify anomalies from arbitrary novel categories, offering a scalable and annotation-efficient solution. Traditionally, most ZSAD works have been based on the CLIP model, which performs anomaly detection by calculating the similarity between visual and text embeddings. Recently, vision foundation models such as DINOv3 have demonstrated strong transferable representation capabilities. In this work, we are the first to adapt DINOv3 for ZSAD. However, this adaptation presents two key challenges: (i) the domain bias between large-scale pretraining data and anomaly detection tasks leads to feature misalignment; and (ii) the inherent bias toward global semantics in pretrained representations often leads to subtle anomalies being misinterpreted as part of the normal foreground objects, rather than being distinguished as abnormal regions. To overcome these challenges, we introduce AD-DINOv3, a novel vision-language multimodal framework designed for ZSAD. Specifically, we formulate anomaly detection as a multimodal contrastive learning problem, where DINOv3 is employed as the visual backbone to extract patch tokens and a CLS token, and the CLIP text encoder provides embeddings for both normal and abnormal prompts. To bridge the domain gap, lightweight adapters are introduced in both modalities, enabling their representations to be recalibrated for the anomaly detection task. Beyond this baseline alignment, we further design an Anomaly-Aware Calibration Module (AACM), which explicitly guides the CLS token to attend to anomalous regions rather than generic foreground semantics, thereby enhancing discriminability. Extensive experiments on eight industrial and medical benchmarks demonstrate that AD-DINOv3 consistently matches or surpasses state-of-the-art methods.The code will be available at https://github.com/Kaisor-Yuan/AD-DINOv3.

Method: Proposed a hierarchical time-space network model to represent unified crew action space, with computationally efficient constraints for heterogeneous qualifications and preferences. Developed solution algorithms based on column generation and shortest path adjustment.

Result: Experiments with real data from Shanghai and Beijing Metro showed the methods outperform benchmark heuristics in cost reduction and task completion, achieving notable efficiency gains through cross-line operations, especially for urgent tasks during disruptions.

Conclusion: The work highlights the importance of global optimization and cross-line coordination in multi-line metro system operations, providing insights for efficient and reliable public transportation in smart cities.

Abstract: Metro crew planning is a key component of smart city development as it directly impacts the operational efficiency and service reliability of public transportation. With the rapid expansion of metro networks, effective multi-line scheduling and emergency management have become essential for large-scale seamless operations. However, current research focuses primarily on individual metro lines,with insufficient attention on cross-line coordination and rapid replanning during disruptions. Here, a unified optimization framework is presented for multi-line metro crew planning and replanning with heterogeneous workforce. Specifically, a hierarchical time-space network model is proposed to represent the unified crew action space, and computationally efficient constraints and formulations are derived for the crew’s heterogeneous qualifications and preferences. Solution algorithms based on column generation and shortest path adjustment are further developed, utilizing the proposed network model. Experiments with real data from Shanghai and Beijing Metro demonstrate that the proposed methods outperform benchmark heuristics in both cost reduction and task completion,and achieve notable efficiency gains by incorporating cross-line operations, particularly for urgent tasks during disruptions. This work highlights the role of global optimization and cross-line coordination in multi-line metro system operations, providing insights into the efficient and reliable functioning of public transportation in smart cities.

Method: Comprehensive evaluation of multiple LLM-based agent architectures (single-agent to modular designs) across realistic penetration testing scenarios, with targeted testing of five core functional capabilities: Global Context Memory, Inter-Agent Messaging, Context-Conditioned Invocation, Adaptive Planning, and Real-Time Monitoring.

Result: Targeted augmentations substantially improve modular agent performance, especially in complex, multi-step, and real-time penetration testing tasks, though some architectures natively exhibit subsets of these properties.

Conclusion: LLM-based agents can be effectively enhanced for penetration testing through specific functional augmentations that address context coherence, coordination, tool accuracy, strategic planning, and real-time responsiveness.

Abstract: Large language models (LLMs) are increasingly used to automate or augment penetration testing, but their effectiveness and reliability across attack phases remain unclear. We present a comprehensive evaluation of multiple LLM-based agents, from single-agent to modular designs, across realistic penetration testing scenarios, measuring empirical performance and recurring failure patterns. We also isolate the impact of five core functional capabilities via targeted augmentations: Global Context Memory (GCM), Inter-Agent Messaging (IAM), Context-Conditioned Invocation (CCI), Adaptive Planning (AP), and Real-Time Monitoring (RTM). These interventions support, respectively: (i) context coherence and retention, (ii) inter-component coordination and state management, (iii) tool use accuracy and selective execution, (iv) multi-step strategic planning, error detection, and recovery, and (v) real-time dynamic responsiveness. Our results show that while some architectures natively exhibit subsets of these properties, targeted augmentations substantially improve modular agent performance, especially in complex, multi-step, and real-time penetration testing tasks.

Method: Proposes a modular evaluation framework that decomposes agent pipelines into interpretable stages for detailed error analysis. Uses the SeeAct framework and Mind2Web dataset as a case study.

Result: The approach reveals actionable weaknesses missed by standard metrics, providing more granular diagnostic capabilities for understanding web agent failures.

Conclusion: This framework paves the way for more robust and generalizable web agents by enabling detailed error analysis and systematic improvement of agent pipelines.

Abstract: Web agents powered by large language models (LLMs) can autonomously perform complex, multistep tasks in dynamic web environments. However, current evaluations mostly focus on the overall success while overlooking intermediate errors. This limits insight into failure modes and hinders systematic improvement. This work analyzes existing benchmarks and highlights the lack of fine-grained diagnostic tools. To address this gap, we propose a modular evaluation framework that decomposes agent pipelines into interpretable stages for detailed error analysis. Using the SeeAct framework and the Mind2Web dataset as a case study, we show how this approach reveals actionable weaknesses missed by standard metrics - paving the way for more robust and generalizable web agents.

Method: Developed OpenLens AI framework with specialized agents for literature review, data analysis, code generation, and manuscript preparation. Enhanced with vision-language feedback for medical visualization interpretation and quality control mechanisms for reproducibility.

Result: A fully automated framework that produces publication-ready LaTeX manuscripts with transparent and traceable workflows, specifically tailored for health informatics research needs.

Conclusion: OpenLens AI provides a domain-adapted solution that addresses the limitations of existing systems and advances health informatics research through comprehensive automation of the entire research pipeline.

Abstract: Health informatics research is characterized by diverse data modalities, rapid knowledge expansion, and the need to integrate insights across biomedical science, data analytics, and clinical practice. These characteristics make it particularly well-suited for agent-based approaches that can automate knowledge exploration, manage complex workflows, and generate clinically meaningful outputs. Recent progress in large language model (LLM)-based agents has demonstrated promising capabilities in literature synthesis, data analysis, and even end-to-end research execution. However, existing systems remain limited for health informatics because they lack mechanisms to interpret medical visualizations and often overlook domain-specific quality requirements. To address these gaps, we introduce OpenLens AI, a fully automated framework tailored to health informatics. OpenLens AI integrates specialized agents for literature review, data analysis, code generation, and manuscript preparation, enhanced by vision-language feedback for medical visualization and quality control for reproducibility. The framework automates the entire research pipeline, producing publication-ready LaTeX manuscripts with transparent and traceable workflows, thereby offering a domain-adapted solution for advancing health informatics research.

Method: Created VCBench with 9,000 anonymized founder profiles standardized to preserve predictive features while reducing re-identification risk by over 90%. Evaluated nine state-of-the-art LLMs against human and market benchmarks.

Result: DeepSeek-V3 achieved over 6x baseline precision, GPT-4o achieved highest F0.5 score, and most models surpassed human benchmarks. Market index precision was 1.9%, Y Combinator 1.7x better, tier-1 firms 2.9x better.

Conclusion: VCBench establishes a community-driven standard for reproducible and privacy-preserving evaluation of AGI in venture forecasting, available as a public evolving resource at vcbench.com.

Abstract: Benchmarks such as SWE-bench and ARC-AGI demonstrate how shared datasets accelerate progress toward artificial general intelligence (AGI). We introduce VCBench, the first benchmark for predicting founder success in venture capital (VC), a domain where signals are sparse, outcomes are uncertain, and even top investors perform modestly. At inception, the market index achieves a precision of 1.9%. Y Combinator outperforms the index by a factor of 1.7x, while tier-1 firms are 2.9x better. VCBench provides 9,000 anonymized founder profiles, standardized to preserve predictive features while resisting identity leakage, with adversarial tests showing more than 90% reduction in re-identification risk. We evaluate nine state-of-the-art large language models (LLMs). DeepSeek-V3 delivers over six times the baseline precision, GPT-4o achieves the highest F0.5, and most models surpass human benchmarks. Designed as a public and evolving resource available at vcbench.com, VCBench establishes a community-driven standard for reproducible and privacy-preserving evaluation of AGI in early-stage venture forecasting.

Method: Drawing from neuroscience and cognitive science, the authors define True Intelligence through six core components and propose a practical five-level taxonomy based on measurable implementation of these components.

Result: The framework provides clear developmental milestones for AGI research and suggests that achieving Level-5 AGI (all five measurable components) makes a system functionally equivalent to True Intelligence.

Conclusion: This work offers the first holistic, mechanism-based definition of AGI that provides an actionable research path, bridging the gap between philosophical debates and practical AI development.

Abstract: The debate around Artificial General Intelligence (AGI) remains open due to two fundamentally different goals: replicating human-like performance versus replicating human-like cognitive processes. We argue that current performance-based definitions are inadequate because they provide no clear, mechanism-focused roadmap for research, and they fail to properly define the qualitative nature of genuine intelligence. Drawing inspiration from the human brain, we propose a new paradigm that shifts the focus from external mimicry to the development of foundational cognitive architectures. We define True Intelligence (TI) as a system characterized by six core components: embodied sensory fusion, core directives, dynamic schemata creation, a highly-interconnected multi-expert architecture, an orchestration layer, and lastly, the unmeasurable quality of Interconnectedness, which we hypothesize results in consciousness and a subjective experience. We propose a practical, five-level taxonomy of AGI based on the number of the first five measurable components a system exhibits. This framework provides a clear path forward with developmental milestones that directly address the challenge of building genuinely intelligent systems. We contend that once a system achieves Level-5 AGI by implementing all five measurable components, the difference between it and TI remains as a purely philosophical debate. For practical purposes - and given theories indicate consciousness is an emergent byproduct of integrated, higher-order cognition - we conclude that a fifth-level AGI is functionally and practically equivalent to TI. This work synthesizes diverse insights from analytical psychology, schema theory, metacognition, modern brain architectures and latest works in AI to provide the first holistic, mechanism-based definition of AGI that offers a clear and actionable path for the research community.

Method: Proposes a framework with Sentinel Agents (distributed security layer using LLM semantic analysis, behavioral analytics, verification, anomaly detection) and Coordinator Agents (policy supervision, threat response). Tested via simulation with 162 synthetic attacks.

Result: Sentinel Agents successfully detected all attack attempts in the simulation study, confirming practical feasibility of the monitoring approach.

Conclusion: The dual-layered security approach provides dynamic, adaptive defense mechanisms, enhanced observability, regulatory compliance support, and enables policy evolution for multi-agent systems.

Abstract: This paper proposes a novel architectural framework aimed at enhancing security and reliability in multi-agent systems (MAS). A central component of this framework is a network of Sentinel Agents, functioning as a distributed security layer that integrates techniques such as semantic analysis via large language models (LLMs), behavioral analytics, retrieval-augmented verification, and cross-agent anomaly detection. Such agents can potentially oversee inter-agent communications, identify potential threats, enforce privacy and access controls, and maintain comprehensive audit records. Complementary to the idea of Sentinel Agents is the use of a Coordinator Agent. The Coordinator Agent supervises policy implementation, and manages agent participation. In addition, the Coordinator also ingests alerts from Sentinel Agents. Based on these alerts, it can adapt policies, isolate or quarantine misbehaving agents, and contain threats to maintain the integrity of the MAS ecosystem. This dual-layered security approach, combining the continuous monitoring of Sentinel Agents with the governance functions of Coordinator Agents, supports dynamic and adaptive defense mechanisms against a range of threats, including prompt injection, collusive agent behavior, hallucinations generated by LLMs, privacy breaches, and coordinated multi-agent attacks. In addition to the architectural design, we present a simulation study where 162 synthetic attacks of different families (prompt injection, hallucination, and data exfiltration) were injected into a multi-agent conversational environment. The Sentinel Agents successfully detected the attack attempts, confirming the practical feasibility of the proposed monitoring approach. The framework also offers enhanced system observability, supports regulatory compliance, and enables policy evolution over time.

Method: Applied Bayesian Measurement Layouts to infer capability profiles of multi-agent systems in the Melting Pot contest, analyzing performance correlations with prosocial abilities.

Result: Higher prosocial capabilities don’t always correlate with better performance; top submissions achieved high scores in scenarios where cooperation wasn’t required, suggesting potential exploitation of evaluation framework limitations.

Conclusion: Measurement Layouts provide accurate predictions and actionable insights for improving AI evaluation, highlighting the need for better annotation of cooperation demands and addressing environmental biases in testing.

Abstract: The development and evaluation of social capabilities in AI agents require complex environments where competitive and cooperative behaviours naturally emerge. While game-theoretic properties can explain why certain teams or agent populations outperform others, more abstract behaviours, such as convention following, are harder to control in training and evaluation settings. The Melting Pot contest is a social AI evaluation suite designed to assess the cooperation capabilities of AI systems. In this paper, we apply a Bayesian approach known as Measurement Layouts to infer the capability profiles of multi-agent systems in the Melting Pot contest. We show that these capability profiles not only predict future performance within the Melting Pot suite but also reveal the underlying prosocial abilities of agents. Our analysis indicates that while higher prosocial capabilities sometimes correlate with better performance, this is not a universal trend-some lower-scoring agents exhibit stronger cooperation abilities. Furthermore, we find that top-performing contest submissions are more likely to achieve high scores in scenarios where prosocial capabilities are not required. These findings, together with reports that the contest winner used a hard-coded solution tailored to specific environments, suggest that at least one top-performing team may have optimised for conditions where cooperation was not necessary, potentially exploiting limitations in the evaluation framework. We provide recommendations for improving the annotation of cooperation demands and propose future research directions to account for biases introduced by different testing environments. Our results demonstrate that Measurement Layouts offer both strong predictive accuracy and actionable insights, contributing to a more transparent and generalisable approach to evaluating AI systems in complex social settings.

Method: Created DeKeyNLU dataset with 1,500 annotated QA pairs, then developed DeKeySQL - a RAG-based pipeline with three modules for question understanding, entity retrieval, and SQL generation, fine-tuned with the new dataset.

Result: Fine-tuning with DeKeyNLU significantly improved SQL generation accuracy: from 62.31% to 69.10% on BIRD dev dataset and from 84.2% to 88.7% on Spider dev dataset.

Conclusion: The DeKeyNLU dataset effectively addresses key bottlenecks in NL2SQL systems, demonstrating that targeted dataset annotation and fine-tuning can substantially improve SQL generation accuracy in RAG-based pipelines.

Abstract: Natural Language to SQL (NL2SQL) provides a new model-centric paradigm that simplifies database access for non-technical users by converting natural language queries into SQL commands. Recent advancements, particularly those integrating Retrieval-Augmented Generation (RAG) and Chain-of-Thought (CoT) reasoning, have made significant strides in enhancing NL2SQL performance. However, challenges such as inaccurate task decomposition and keyword extraction by LLMs remain major bottlenecks, often leading to errors in SQL generation. While existing datasets aim to mitigate these issues by fine-tuning models, they struggle with over-fragmentation of tasks and lack of domain-specific keyword annotations, limiting their effectiveness. To address these limitations, we present DeKeyNLU, a novel dataset which contains 1,500 meticulously annotated QA pairs aimed at refining task decomposition and enhancing keyword extraction precision for the RAG pipeline. Fine-tuned with DeKeyNLU, we propose DeKeySQL, a RAG-based NL2SQL pipeline that employs three distinct modules for user question understanding, entity retrieval, and generation to improve SQL generation accuracy. We benchmarked multiple model configurations within DeKeySQL RAG pipeline. Experimental results demonstrate that fine-tuning with DeKeyNLU significantly improves SQL generation accuracy on both BIRD (62.31% to 69.10%) and Spider (84.2% to 88.7%) dev datasets.

Method: Proposed benchmark covering wide range of domains and LLMs, including easy-to-use toolkit, extensive experimental results, and analysis comparing LLM behavior to idealized human rationality.

Result: Benchmark provides foundational tool that illuminates where LLMs converge and diverge from human rationality across different domains.

Conclusion: The benchmark serves as a foundational evaluation tool for both developers and users to assess LLM rationality and understand their alignment with human cognitive patterns.

Abstract: Large language models (LLMs), a recent advance in deep learning and machine intelligence, have manifested astonishing capacities, now considered among the most promising for artificial general intelligence. With human-like capabilities, LLMs have been used to simulate humans and serve as AI assistants across many applications. As a result, great concern has arisen about whether and under what circumstances LLMs think and behave like real human agents. Rationality is among the most important concepts in assessing human behavior, both in thinking (i.e., theoretical rationality) and in taking action (i.e., practical rationality). In this work, we propose the first benchmark for evaluating the omnibus rationality of LLMs, covering a wide range of domains and LLMs. The benchmark includes an easy-to-use toolkit, extensive experimental results, and analysis that illuminates where LLMs converge and diverge from idealized human rationality. We believe the benchmark can serve as a foundational tool for both developers and users of LLMs.

Method: A priori dynamic framework using Q-table learning to optimize decision space, with agents evaluating task progress to make proactive decisions about next executing agent. Includes cold-start initialization, early stopping, and pruning mechanisms.

Result: Achieves 4.05% average improvement over state-of-the-art baselines while reducing workflow construction and inference costs to 30.68%-48.31% of existing methods.

Conclusion: The proposed framework demonstrates feasibility and effectiveness, showing significant performance improvements and cost reductions through dynamic, task-aware workflow construction.

Abstract: Recent studies have shown that carefully designed workflows coordinating large language models(LLMs) significantly enhance task-solving capabilities compared to using a single model. While an increasing number of works focus on autonomous workflow construction, most existing approaches rely solely on historical experience, leading to limitations in efficiency and adaptability. We argue that while historical experience is valuable, workflow construction should also flexibly respond to the unique characteristics of each task. To this end, we propose an a priori dynamic framework for automated workflow construction. Our framework first leverages Q-table learning to optimize the decision space, guiding agent decisions and enabling effective use of historical experience. At the same time, agents evaluate the current task progress and make a priori decisions regarding the next executing agent, allowing the system to proactively select the more suitable workflow structure for each given task. Additionally, we incorporate mechanisms such as cold-start initialization, early stopping, and pruning to further improve system efficiency. Experimental evaluations on four benchmark datasets demonstrate the feasibility and effectiveness of our approach. Compared to state-of-the-art baselines, our method achieves an average improvement of 4.05%, while reducing workflow construction and inference costs to only 30.68%-48.31% of those required by existing methods.

Method: Introduced comprehensive evaluation framework with standardized metrics, conducted large-scale empirical study of DP text generation methods and LLMs, and developed membership inference attack methodology for synthetic text.

Result: Found that high-quality domain-specific synthetic data generation under DP constraints remains challenging with performance degrading as domain complexity increases, and demonstrated that public datasets in pre-training can invalidate privacy guarantees.

Conclusion: Highlights urgent need for rigorous privacy auditing and shows persistent gaps between open-domain and specialist evaluations, informing responsible deployment of generative AI in privacy-sensitive settings.

Abstract: Data-driven decision support in high-stakes domains like healthcare and finance faces significant barriers to data sharing due to regulatory, institutional, and privacy concerns. While recent generative AI models, such as large language models, have shown impressive performance in open-domain tasks, their adoption in sensitive environments remains limited by unpredictable behaviors and insufficient privacy-preserving datasets for benchmarking. Existing anonymization methods are often inadequate, especially for unstructured text, as redaction and masking can still allow re-identification. Differential Privacy (DP) offers a principled alternative, enabling the generation of synthetic data with formal privacy assurances. In this work, we address these challenges through three key contributions. First, we introduce a comprehensive evaluation framework with standardized utility and fidelity metrics, encompassing nine curated datasets that capture domain-specific complexities such as technical jargon, long-context dependencies, and specialized document structures. Second, we conduct a large-scale empirical study benchmarking state-of-the-art DP text generation methods and LLMs of varying sizes and different fine-tuning strategies, revealing that high-quality domain-specific synthetic data generation under DP constraints remains an unsolved challenge, with performance degrading as domain complexity increases. Third, we develop a membership inference attack (MIA) methodology tailored for synthetic text, providing first empirical evidence that the use of public datasets - potentially present in pre-training corpora - can invalidate claimed privacy guarantees. Our findings underscore the urgent need for rigorous privacy auditing and highlight persistent gaps between open-domain and specialist evaluations, informing responsible deployment of generative AI in privacy-sensitive, high-stakes settings.

Method: A structured multi-stage analytical pipeline modeling expert debuggers: error identification/categorization, thematic clustering, quantitative scoring, and strategic summarization, enhanced with dual memory system (episodic and semantic) for continual learning.

Result: Achieves state-of-the-art results on TRAIL benchmark, uncovers critical issues missed in human annotations, and demonstrates practical utility through real-world deployments with design partners.

Conclusion: AgentCompass serves as a robust, developer-centric tool for reliable monitoring and improvement of agentic systems in production environments.

Abstract: With the growing adoption of Large Language Models (LLMs) in automating complex, multi-agent workflows, organizations face mounting risks from errors, emergent behaviors, and systemic failures that current evaluation methods fail to capture. We present AgentCompass, the first evaluation framework designed specifically for post-deployment monitoring and debugging of agentic workflows. AgentCompass models the reasoning process of expert debuggers through a structured, multi-stage analytical pipeline: error identification and categorization, thematic clustering, quantitative scoring, and strategic summarization. The framework is further enhanced with a dual memory system-episodic and semantic-that enables continual learning across executions. Through collaborations with design partners, we demonstrate the framework’s practical utility on real-world deployments, before establishing its efficacy against the publicly available TRAIL benchmark. AgentCompass achieves state-of-the-art results on key metrics, while uncovering critical issues missed in human annotations, underscoring its role as a robust, developer-centric tool for reliable monitoring and improvement of agentic systems in production.

Method: Annotated thousands of sentences/paragraphs from model-generated math solutions using seven cognitive labels (Plan, Implement, Verify, etc.) based on Schoenfeld’s Episode Theory.

Result: Created first publicly available benchmark for fine-grained analysis of machine reasoning, revealing distinct patterns in LRM reasoning and transition dynamics between cognitive states.

Conclusion: Provides theoretically grounded methodology for interpreting LRM cognition, enabling future work on more controllable and transparent reasoning systems.

Abstract: While Large Reasoning Models (LRMs) generate extensive chain-of-thought reasoning, we lack a principled framework for understanding how these thoughts are structured. In this paper, we introduce a novel approach by applying Schoenfeld’s Episode Theory, a classic cognitive framework for human mathematical problem-solving, to analyze the reasoning traces of LRMs. We annotated thousands of sentences and paragraphs from model-generated solutions to math problems using seven cognitive labels (e.g., Plan, Implement, Verify). The result is the first publicly available benchmark for the fine-grained analysis of machine reasoning, including a large annotated corpus and detailed annotation guidebooks. Our preliminary analysis reveals distinct patterns in LRM reasoning, such as the transition dynamics between cognitive states. This framework provides a theoretically grounded methodology for interpreting LRM cognition and enables future work on more controllable and transparent reasoning systems.

Method: Uses CoT-guided supervised fine-tuning with expert-corrected dataset to instill expert reasoning patterns, followed by reinforcement learning with multi-faceted reward function to optimize accuracy and logical consistency while mitigating hallucinations.

Result: Outperforms state-of-the-art baselines with superior F1-scores on key benchmarks while providing transparent, step-by-step analytical outputs.

Conclusion: RationAnomaly effectively addresses the limitations of existing approaches by synergizing CoT fine-tuning with reinforcement learning, achieving both high performance and interpretability in log anomaly detection.

Abstract: Logs constitute a form of evidence signaling the operational status of software systems. Automated log anomaly detection is crucial for ensuring the reliability of modern software systems. However, existing approaches face significant limitations: traditional deep learning models lack interpretability and generalization, while methods leveraging Large Language Models are often hindered by unreliability and factual inaccuracies. To address these issues, we propose RationAnomaly, a novel framework that enhances log anomaly detection by synergizing Chain-of-Thought (CoT) fine-tuning with reinforcement learning. Our approach first instills expert-like reasoning patterns using CoT-guided supervised fine-tuning, grounded in a high-quality dataset corrected through a rigorous expert-driven process. Subsequently, a reinforcement learning phase with a multi-faceted reward function optimizes for accuracy and logical consistency, effectively mitigating hallucinations. Experimentally, RationAnomaly outperforms state-of-the-art baselines, achieving superior F1-scores on key benchmarks while providing transparent, step-by-step analytical outputs. We have released the corresponding resources, including code and datasets.

Method: Created Nazonazo benchmark with 120 Japanese riddles (extended to 201 items) - short, domain-agnostic items that can be rapidly refreshed. Evaluated 38 frontier models and 126 humans, with candidate-tracking analysis of thought logs.

Result: No model except GPT-5 matched human performance (52.9% mean accuracy). Reasoning models outperformed non-reasoning peers, model size showed no reliable association with accuracy. Analysis revealed verification failure - models often produced correct solutions but failed to select them as final answers.

Conclusion: Nazonazo provides a cost-effective, scalable, renewable benchmark format that addresses evaluation crisis while revealing meta-cognitive weaknesses, offering clear targets for future control and calibration methods.

Abstract: Benchmark saturation and contamination undermine confidence in LLM evaluation. We present Nazonazo, a cost-effective and extensible benchmark built from Japanese children’s riddles to test insight-based reasoning. Items are short (mostly one sentence), require no specialized domain knowledge, and can be generated at scale, enabling rapid refresh of blind sets when leakage is suspected. We evaluate 38 frontier models and 126 adults on 120 riddles. No model except for GPT-5 is comparable to human performance, which achieves a 52.9% mean accuracy. Model comparison on extended 201 items shows that reasoning models significantly outperform non-reasoning peers, while model size shows no reliable association with accuracy. Beyond aggregate accuracy, an informal candidate-tracking analysis of thought logs reveals many cases of verification failure: models often produce the correct solution among intermediate candidates yet fail to select it as the final answer, which we illustrate with representative examples observed in multiple models. Nazonazo thus offers a cost-effective, scalable, and easily renewable benchmark format that addresses the current evaluation crisis while also suggesting a recurrent meta-cognitive weakness, providing clear targets for future control and calibration methods.

Method: Proposes AC-RAG framework with two heterogeneous agents: a generalist Detector that identifies knowledge gaps, and a domain-specialized Resolver that provides solutions. They engage in adversarial collaboration guided by a moderator through persistent questioning and iterative problem dissection.

Result: Extensive experiments show AC-RAG significantly improves retrieval accuracy and outperforms state-of-the-art RAG methods across various vertical domains.

Conclusion: The adversarial collaboration framework effectively addresses retrieval hallucinations in RAG systems, demonstrating superior performance through iterative problem-solving between specialized agents.

Abstract: Retrieval-augmented Generation (RAG) is a prevalent approach for domain-specific LLMs, yet it is often plagued by “Retrieval Hallucinations”–a phenomenon where fine-tuned models fail to recognize and act upon poor-quality retrieved documents, thus undermining performance. To address this, we propose the Adversarial Collaboration RAG (AC-RAG) framework. AC-RAG employs two heterogeneous agents: a generalist Detector that identifies knowledge gaps, and a domain-specialized Resolver that provides precise solutions. Guided by a moderator, these agents engage in an adversarial collaboration, where the Detector’s persistent questioning challenges the Resolver’s expertise. This dynamic process allows for iterative problem dissection and refined knowledge retrieval. Extensive experiments show that AC-RAG significantly improves retrieval accuracy and outperforms state-of-the-art RAG methods across various vertical domains.

Method: Analyzed inpatient EMR data with diagnostic codes, ward transitions, demographics, infection variables, and contact network features. Benchmarked Transformer-based architectures against traditional ML models, using XAI techniques for interpretation.

Result: TabTransformer consistently outperformed baselines across clinical prediction tasks, especially for CPE acquisition (high AUROC and sensitivity). Infection-related features, historical exposure, admission context, and network centrality measures were highly influential.

Conclusion: The study presents a robust explainable AI framework that demonstrates Transformer models’ superiority and highlights the importance of diverse clinical and network features for predicting CPE-related outcomes and identifying key risk factors.

Abstract: Carbapenemase-Producing Enterobacteriace poses a critical concern for infection prevention and control in hospitals. However, predictive modeling of previously highlighted CPE-associated risks such as readmission, mortality, and extended length of stay (LOS) remains underexplored, particularly with modern deep learning approaches. This study introduces an eXplainable AI modeling framework to investigate CPE impact on patient outcomes from Electronic Medical Records data of an Irish hospital. We analyzed an inpatient dataset from an Irish acute hospital, incorporating diagnostic codes, ward transitions, patient demographics, infection-related variables and contact network features. Several Transformer-based architectures were benchmarked alongside traditional machine learning models. Clinical outcomes were predicted, and XAI techniques were applied to interpret model decisions. Our framework successfully demonstrated the utility of Transformer-based models, with TabTransformer consistently outperforming baselines across multiple clinical prediction tasks, especially for CPE acquisition (AUROC and sensitivity). We found infection-related features, including historical hospital exposure, admission context, and network centrality measures, to be highly influential in predicting patient outcomes and CPE acquisition risk. Explainability analyses revealed that features like “Area of Residence”, “Admission Ward” and prior admissions are key risk factors. Network variables like “Ward PageRank” also ranked highly, reflecting the potential value of structural exposure information. This study presents a robust and explainable AI framework for analyzing complex EMR data to identify key risk factors and predict CPE-related outcomes. Our findings underscore the superior performance of the Transformer models and highlight the importance of diverse clinical and network features.

Method: Generalize existing single-argument contribution functions to set contribution functions, develop new principles specific to set-based functions focusing on argument interactions, and provide principle-based analysis across different set contribution functions.

Result: The paper presents generalized set contribution functions that can quantify how groups of arguments collectively influence a topic’s strength, along with new principles that capture set-specific properties and interactions.

Conclusion: The proposed set contribution functions and principles provide a more comprehensive framework for analyzing collective argument contributions in quantitative bipolar argumentation, with practical applications in recommendation systems and other domains requiring multi-argument analysis.

Abstract: We present functions that quantify the contribution of a set of arguments in quantitative bipolar argumentation graphs to (the final strength of) an argument of interest, a so-called topic. Our set contribution functions are generalizations of existing functions that quantify the contribution of a single contributing argument to a topic. Accordingly, we generalize existing contribution function principles for set contribution functions and provide a corresponding principle-based analysis. We introduce new principles specific to set-based functions that focus on properties pertaining to the interaction of arguments within a set. Finally, we sketch how the principles play out across different set contribution functions given a recommendation system application scenario.

Method: KAMAC starts with expert agents and conducts knowledge-driven discussions to identify gaps, then recruits additional specialists dynamically based on evolving diagnostic context.

Result: Experiments on medical benchmarks show KAMAC significantly outperforms single-agent and advanced multi-agent methods, especially in complex scenarios like cancer prognosis.

Conclusion: The framework enables flexible, scalable collaboration by dynamically forming expert teams, demonstrating superior performance in clinical decision-making requiring cross-specialty expertise.

Abstract: Medical decision-making often involves integrating knowledge from multiple clinical specialties, typically achieved through multidisciplinary teams. Inspired by this collaborative process, recent work has leveraged large language models (LLMs) in multi-agent collaboration frameworks to emulate expert teamwork. While these approaches improve reasoning through agent interaction, they are limited by static, pre-assigned roles, which hinder adaptability and dynamic knowledge integration. To address these limitations, we propose KAMAC, a Knowledge-driven Adaptive Multi-Agent Collaboration framework that enables LLM agents to dynamically form and expand expert teams based on the evolving diagnostic context. KAMAC begins with one or more expert agents and then conducts a knowledge-driven discussion to identify and fill knowledge gaps by recruiting additional specialists as needed. This supports flexible, scalable collaboration in complex clinical scenarios, with decisions finalized through reviewing updated agent comments. Experiments on two real-world medical benchmarks demonstrate that KAMAC significantly outperforms both single-agent and advanced multi-agent methods, particularly in complex clinical scenarios (i.e., cancer prognosis) requiring dynamic, cross-specialty expertise. Our code is publicly available at: https://github.com/XiaoXiao-Woo/KAMAC.

Method: Analyzed 120 metareviews using Systemic Functional Linguistics and Appraisal Theory to examine how AI feedback constructs meaning across ideational, interpersonal, and textual dimensions.

Result: Generative AI feedback demonstrated ability to approximate key rhetorical and relational features of effective human feedback, including balanced praise/critique, rubric alignment, and structured staging that foregrounds student agency.

Conclusion: AI metafeedback has potential to scaffold feedback literacy and enhance learner engagement with peer review by modeling effective feedback qualities.

Abstract: This study investigates the use of generative AI to support formative assessment through machine generated reviews of peer reviews in graduate online courses in a public university in the United States. Drawing on Systemic Functional Linguistics and Appraisal Theory, we analyzed 120 metareviews to explore how generative AI feedback constructs meaning across ideational, interpersonal, and textual dimensions. The findings suggest that generative AI can approximate key rhetorical and relational features of effective human feedback, offering directive clarity while also maintaining a supportive stance. The reviews analyzed demonstrated a balance of praise and constructive critique, alignment with rubric expectations, and structured staging that foregrounded student agency. By modeling these qualities, AI metafeedback has the potential to scaffold feedback literacy and enhance leaner engagement with peer review.

Method: The authors propose a domain-specific survey designed to capture maritime professionals’ perceptions of trust, usability, and explainability to support user-centered integration of XAI.

Result: The paper presents a framework for developing user-centric XAI systems tailored to maritime needs, though specific survey results are not detailed in the abstract.

Conclusion: XAI is crucial for effective human-machine teaming in maritime operations, and user-centered approaches are needed to develop systems that meet the specific requirements of seafarers and maritime teams.

Abstract: As autonomous technologies increasingly shape maritime operations, understanding why an AI system makes a decision becomes as crucial as what it decides. In complex and dynamic maritime environments, trust in AI depends not only on performance but also on transparency and interpretability. This paper highlights the importance of Explainable AI (XAI) as a foundation for effective human-machine teaming in the maritime domain, where informed oversight and shared understanding are essential. To support the user-centered integration of XAI, we propose a domain-specific survey designed to capture maritime professionals’ perceptions of trust, usability, and explainability. Our aim is to foster awareness and guide the development of user-centric XAI systems tailored to the needs of seafarers and maritime teams.

Method: Multi-Agent Consensus Alignment (MACA) uses reinforcement learning to post-train models, leveraging majority/minority outcomes from multi-agent debate where agents ground reasoning in peer arguments rather than just aggregating independent attempts.

Result: Substantial improvements across self-consistency (+27.6% on GSM8K), single-agent reasoning (+23.7% on MATH), sampling-based inference (+22.4% Pass@20 on MATH), and multi-agent ensemble decision-making (+42.7% on MathQA), with strong generalization to unseen benchmarks.

Conclusion: MACA enables robust self-alignment that more reliably unlocks the latent reasoning potential of language models through deliberative exchanges and consensus signals, making agents more decisive and better at leveraging peer insights without external supervision.

Abstract: Language Models (LMs) are inconsistent reasoners, often generating contradictory responses to identical prompts. While inference-time methods can mitigate these inconsistencies, they fail to address the core problem: LMs struggle to reliably select reasoning pathways leading to consistent outcomes under exploratory sampling. To address this, we formalize self-consistency as an intrinsic property of well-aligned reasoning models and introduce Multi-Agent Consensus Alignment (MACA), a reinforcement learning framework that post-trains models to favor reasoning trajectories aligned with their internal consensus using majority/minority outcomes from multi-agent debate. These trajectories emerge from deliberative exchanges where agents ground reasoning in peer arguments, not just aggregation of independent attempts, creating richer consensus signals than single-round majority voting. MACA enables agents to teach themselves to be more decisive and concise, and better leverage peer insights in multi-agent settings without external supervision, driving substantial improvements across self-consistency (+27.6% on GSM8K), single-agent reasoning (+23.7% on MATH), sampling-based inference (+22.4% Pass@20 on MATH), and multi-agent ensemble decision-making (+42.7% on MathQA). These findings, coupled with strong generalization to unseen benchmarks (+16.3% on GPQA, +11.6% on CommonsenseQA), demonstrate robust self-alignment that more reliably unlocks latent reasoning potential of language models.

Method: Introduces Reinforcement Learning with Verifiable Rewards (RLVR) to refine captions for geometric images synthesized from 50 basic geometric relations, using reward signals from mathematical problem-solving tasks.

Result: Achieves 2.8%-4.8% accuracy improvements in statistics, arithmetic, algebraic, and numerical tasks on MathVista/MathVerse, and 2.4%-3.9% improvements in Art/Design/Tech/Engineering tasks on MMMU, even in out-of-distribution scenarios.

Conclusion: RLVR pipeline successfully captures key features of geometry problem-solving, enabling better task generalization and enhancing general reasoning capabilities of multimodal LLMs beyond just geometric tasks.

Abstract: Multimodal large language models have various practical applications that demand strong reasoning abilities. Despite recent advancements, these models still struggle to solve complex geometric problems. A key challenge stems from the lack of high-quality image-text pair datasets for understanding geometric images. Furthermore, most template-based data synthesis pipelines typically fail to generalize to questions beyond their predefined templates. In this paper, we bridge this gap by introducing a complementary process of Reinforcement Learning with Verifiable Rewards (RLVR) into the data generation pipeline. By adopting RLVR to refine captions for geometric images synthesized from 50 basic geometric relations and using reward signals derived from mathematical problem-solving tasks, our pipeline successfully captures the key features of geometry problem-solving. This enables better task generalization and yields non-trivial improvements. Furthermore, even in out-of-distribution scenarios, the generated dataset enhances the general reasoning capabilities of multimodal large language models, yielding accuracy improvements of $2.8%\text{-}4.8%$ in statistics, arithmetic, algebraic, and numerical tasks with non-geometric input images of MathVista and MathVerse, along with $2.4%\text{-}3.9%$ improvements in Art, Design, Tech, and Engineering tasks in MMMU.

Method: The authors provide an up-to-date ontology of AutomationML concepts and a declarative mapping to automatically transform AutomationML models into RDF triples, enabling OWL-based semantic processing.

Result: The transformation enables new use cases for querying with SPARQL and validation with SHACL that were previously impossible with standard XML tools for AutomationML data.

Conclusion: Converting AutomationML to OWL opens powerful new ways for querying and validation in the automation domain, overcoming the limitations of XML-based tools for this specialized format.

Abstract: AutomationML has seen widespread adoption as an open data exchange format in the automation domain. It is an open and vendor neutral standard based on the extensible markup language XML. However, AutomationML extends XML with additional semantics that limit the applicability of common XML-tools for applications like querying or data validation. This article demonstrates how the transformation of AutomationML into OWL enables new use cases in querying with SPARQL and validation with SHACL. To support this, it provides practitioners with (1) an up-to-date ontology of the concepts defined in the AutomationML standard and (2) a declarative mapping to automatically transform any AutomationML model into RDF triples. A study on examples from the automation domain concludes that transforming AutomationML to OWL opens up new powerful ways for querying and validation that would have been impossible without this transformation.

Method: Hierarchical Co-Self-Play (HCSP) framework with three-stage population-based training: (I) diverse low-level skills training, (II) high-level strategy learning via self-play with fixed skills, (III) joint fine-tuning through co-self-play.

Result: Superior performance with 82.9% average win rate, outperforming non-hierarchical self-play and rule-based baselines. Emergent team behaviors like role switching and coordinated formations.

Conclusion: HCSP effectively enables both strategy and skill emergence from scratch, demonstrating the effectiveness of hierarchical design and co-self-play training for complex multi-agent embodied tasks.

Abstract: In this paper, we tackle the problem of learning to play 3v3 multi-drone volleyball, a new embodied competitive task that requires both high-level strategic coordination and low-level agile control. The task is turn-based, multi-agent, and physically grounded, posing significant challenges due to its long-horizon dependencies, tight inter-agent coupling, and the underactuated dynamics of quadrotors. To address this, we propose Hierarchical Co-Self-Play (HCSP), a hierarchical reinforcement learning framework that separates centralized high-level strategic decision-making from decentralized low-level motion control. We design a three-stage population-based training pipeline to enable both strategy and skill to emerge from scratch without expert demonstrations: (I) training diverse low-level skills, (II) learning high-level strategy via self-play with fixed low-level skills, and (III) joint fine-tuning through co-self-play. Experiments show that HCSP achieves superior performance, outperforming non-hierarchical self-play and rule-based hierarchical baselines with an average 82.9% win rate and a 71.5% win rate against the two-stage variant. Moreover, co-self-play leads to emergent team behaviors such as role switching and coordinated formations, demonstrating the effectiveness of our hierarchical design and training scheme. The project page is at https://sites.google.com/view/hi-co-self-play.

Method: Evaluated four bias types (position, verbosity, chain-of-thought, bandwagon) across two multi-agent frameworks: Multi-Agent-Debate and LLM-as-Meta-Judge, and tested PINE debiasing method integration.

Result: Debate framework sharply amplifies biases after initial debate and sustains them, while meta-judge shows greater resistance. PINE effectively reduces biases in debate settings but provides less benefit in meta-judge scenarios.

Conclusion: Multi-agent LLM-as-Judge systems exhibit distinct bias behaviors requiring targeted mitigation strategies, with debate frameworks being particularly vulnerable to bias amplification that needs intervention.

Abstract: LLM-as-Judge has emerged as a scalable alternative to human evaluation, enabling large language models (LLMs) to provide reward signals in trainings. While recent work has explored multi-agent extensions such as multi-agent debate and meta-judging to enhance evaluation quality, the question of how intrinsic biases manifest in these settings remains underexplored. In this study, we conduct a systematic analysis of four diverse bias types: position bias, verbosity bias, chain-of-thought bias, and bandwagon bias. We evaluate these biases across two widely adopted multi-agent LLM-as-Judge frameworks: Multi-Agent-Debate and LLM-as-Meta-Judge. Our results show that debate framework amplifies biases sharply after the initial debate, and this increased bias is sustained in subsequent rounds, while meta-judge approaches exhibit greater resistance. We further investigate the incorporation of PINE, a leading single-agent debiasing method, as a bias-free agent within these systems. The results reveal that this bias-free agent effectively reduces biases in debate settings but provides less benefit in meta-judge scenarios. Our work provides a comprehensive study of bias behavior in multi-agent LLM-as-Judge systems and highlights the need for targeted bias mitigation strategies in collaborative evaluation settings.

Method: Proposes DualSG framework with Time Series Caption (explicit prompt format summarizing trends in natural language) and caption-guided fusion module that models inter-variable relationships while reducing noise and computation.

Result: Experiments on real-world datasets show DualSG consistently outperforms 15 state-of-the-art baselines across diverse domains.

Conclusion: Explicitly combining numerical forecasting with semantic guidance through LLMs as semantic guides provides better performance than treating LLMs as standalone forecasters.

Abstract: Multivariate Time Series Forecasting plays a key role in many applications. Recent works have explored using Large Language Models for MTSF to take advantage of their reasoning abilities. However, many methods treat LLMs as end-to-end forecasters, which often leads to a loss of numerical precision and forces LLMs to handle patterns beyond their intended design. Alternatively, methods that attempt to align textual and time series modalities within latent space frequently encounter alignment difficulty. In this paper, we propose to treat LLMs not as standalone forecasters, but as semantic guidance modules within a dual-stream framework. We propose DualSG, a dual-stream framework that provides explicit semantic guidance, where LLMs act as Semantic Guides to refine rather than replace traditional predictions. As part of DualSG, we introduce Time Series Caption, an explicit prompt format that summarizes trend patterns in natural language and provides interpretable context for LLMs, rather than relying on implicit alignment between text and time series in the latent space. We also design a caption-guided fusion module that explicitly models inter-variable relationships while reducing noise and computation. Experiments on real-world datasets from diverse domains show that DualSG consistently outperforms 15 state-of-the-art baselines, demonstrating the value of explicitly combining numerical forecasting with semantic guidance.

Method: Uses verifiable segments or “slices” that can cover part or all of the inference pipeline, with global consistency enforced through audit, replication, or economic incentives. Supports flexible proof boundaries aligned with model’s logical structure.

Result: Empirical evaluation shows performance on memory usage, runtime, and circuit behavior under both sliced and unsliced configurations using multiple proving systems.

Conclusion: DSperse enables scalable, targeted verification strategies that support pragmatic trust minimization by localizing zero-knowledge proofs to components where they provide greatest value, suitable for diverse deployment needs.

Abstract: DSperse is a modular framework for distributed machine learning inference with strategic cryptographic verification. Operating within the emerging paradigm of distributed zero-knowledge machine learning, DSperse avoids the high cost and rigidity of full-model circuitization by enabling targeted verification of strategically chosen subcomputations. These verifiable segments, or “slices”, may cover part or all of the inference pipeline, with global consistency enforced through audit, replication, or economic incentives. This architecture supports a pragmatic form of trust minimization, localizing zero-knowledge proofs to the components where they provide the greatest value. We evaluate DSperse using multiple proving systems and report empirical results on memory usage, runtime, and circuit behavior under sliced and unsliced configurations. By allowing proof boundaries to align flexibly with the model’s logical structure, DSperse supports scalable, targeted verification strategies suited to diverse deployment needs.

Method: InMind enhances structured gameplay data with round-level strategy traces and post-game reflections collected under Observer and Participant modes, supporting four cognitively motivated tasks that evaluate both static alignment and dynamic adaptation.

Result: General-purpose LLMs (including GPT-4o) frequently rely on lexical cues and struggle with temporal gameplay anchoring and strategy adaptation, while reasoning-enhanced LLMs like DeepSeek-R1 show early signs of style-sensitive reasoning.

Conclusion: The findings reveal key limitations in current LLMs’ capacity for individualized, adaptive reasoning, positioning InMind as a step toward cognitively aligned human-AI interaction.

Abstract: LLMs have shown strong performance on human-centric reasoning tasks. While previous evaluations have explored whether LLMs can infer intentions or detect deception, they often overlook the individualized reasoning styles that influence how people interpret and act in social contexts. Social deduction games (SDGs) provide a natural testbed for evaluating individualized reasoning styles, where different players may adopt diverse but contextually valid reasoning strategies under identical conditions. To address this, we introduce InMind, a cognitively grounded evaluation framework designed to assess whether LLMs can capture and apply personalized reasoning styles in SDGs. InMind enhances structured gameplay data with round-level strategy traces and post-game reflections, collected under both Observer and Participant modes. It supports four cognitively motivated tasks that jointly evaluate both static alignment and dynamic adaptation. As a case study, we apply InMind to the game Avalon, evaluating 11 state-of-the-art LLMs. General-purpose LLMs, even GPT-4o frequently rely on lexical cues, struggling to anchor reflections in temporal gameplay or adapt to evolving strategies. In contrast, reasoning-enhanced LLMs like DeepSeek-R1 exhibit early signs of style-sensitive reasoning. These findings reveal key limitations in current LLMs’ capacity for individualized, adaptive reasoning, and position InMind as a step toward cognitively aligned human-AI interaction.

Method: Review and analysis of existing statistical techniques and their applications to generative AI systems.

Result: Statistical methods show promise for enhancing generative AI reliability, evaluation quality, and experimental interventions.

Conclusion: Statistical approaches offer valuable solutions for addressing generative AI’s reliability challenges, though limitations exist and future research directions are needed.

Abstract: Generative Artificial Intelligence is emerging as an important technology, promising to be transformative in many areas. At the same time, generative AI techniques are based on sampling from probabilistic models, and by default, they come with no guarantees about correctness, safety, fairness, or other properties. Statistical methods offer a promising potential approach to improve the reliability of generative AI techniques. In addition, statistical methods are also promising for improving the quality and efficiency of AI evaluation, as well as for designing interventions and experiments in AI. In this paper, we review some of the existing work on these topics, explaining both the general statistical techniques used, as well as their applications to generative AI. We also discuss limitations and potential future directions.

Method: Combines scene text detection, inpainting, machine translation, and text reimposition with human oversight to create a structured framework for ad localization.

Result: Qualitative results across six locales show the approach produces semantically accurate and visually coherent localized advertisements suitable for real-world deployment.

Conclusion: This is the first work to integrate these specific techniques for accelerating ad localization evaluation workflows, demonstrating effective multilingual ad adaptation.

Abstract: Adapting advertisements for multilingual audiences requires more than simple text translation; it demands preservation of visual consistency, spatial alignment, and stylistic integrity across diverse languages and formats. We introduce a structured framework that combines automated components with human oversight to address the complexities of advertisement localization. To the best of our knowledge, this is the first work to integrate scene text detection, inpainting, machine translation (MT), and text reimposition specifically for accelerating ad localization evaluation workflows. Qualitative results across six locales demonstrate that our approach produces semantically accurate and visually coherent localized advertisements, suitable for deployment in real-world workflows.

Method: Introduces a Local State Attention module that uses self-attention operators to compress essential information from nearby agents, helping resolve traffic conflicts and prioritize vehicle information during merging scenarios with unexpected events.

Result: Significant improvements in merging efficiency compared to popular baselines, particularly in high-density traffic settings, with better ability to handle priority vehicles as unexpected events.

Conclusion: The Local State Attention module effectively enhances state representation for autonomous vehicles in MARL environments, enabling better conflict resolution and generalization to stochastic traffic events.

Abstract: In mixed-traffic environments, autonomous vehicles must adapt to human-controlled vehicles and other unusual driving situations. This setting can be framed as a multi-agent reinforcement learning (MARL) environment with full cooperative reward among the autonomous vehicles. While methods such as Multi-agent Proximal Policy Optimization can be effective in training MARL tasks, they often fail to resolve local conflict between agents and are unable to generalize to stochastic events. In this paper, we propose a Local State Attention module to assist the input state representation. By relying on the self-attention operator, the module is expected to compress the essential information of nearby agents to resolve the conflict in traffic situations. Utilizing a simulated highway merging scenario with the priority vehicle as the unexpected event, our approach is able to prioritize other vehicles’ information to manage the merging process. The results demonstrate significant improvements in merging efficiency compared to popular baselines, especially in high-density traffic settings.

Method: Evaluated 16 state-of-the-art VLMs (open and closed-source) across 6 multimodal datasets with 3 distinct scoring functions to comprehensively benchmark uncertainty quantification.

Result: Larger models consistently exhibit better uncertainty quantification; models that know more also know better what they don’t know. More certain models achieve higher accuracy, while mathematical and reasoning tasks elicit poorer uncertainty performance across all models.

Conclusion: This work establishes a foundation for reliable uncertainty evaluation in multimodal systems, showing that uncertainty quantification capabilities vary by model size and task domain.

Abstract: Vision-Language Models (VLMs) have achieved remarkable progress in complex visual understanding across scientific and reasoning tasks. While performance benchmarking has advanced our understanding of these capabilities, the critical dimension of uncertainty quantification has received insufficient attention. Therefore, unlike prior conformal prediction studies that focused on limited settings, we conduct a comprehensive uncertainty benchmarking study, evaluating 16 state-of-the-art VLMs (open and closed-source) across 6 multimodal datasets with 3 distinct scoring functions. Our findings demonstrate that larger models consistently exhibit better uncertainty quantification; models that know more also know better what they don’t know. More certain models achieve higher accuracy, while mathematical and reasoning tasks elicit poorer uncertainty performance across all models compared to other domains. This work establishes a foundation for reliable uncertainty evaluation in multimodal systems.

Method: Unconstrained Dysfluency Modeling (UDM) framework combining modular architecture, explicit phoneme alignment, and interpretable outputs, tested through experiments with patients and certified speech-language pathologists.

Result: Achieved state-of-the-art performance (F1: 0.89±0.04) with high interpretability scores (4.2/5.0), 87% clinician acceptance rate, and 34% reduction in diagnostic time.

Conclusion: UDM provides a practical pathway for AI-assisted speech therapy in clinical environments by balancing high performance with clinical interpretability.

Abstract: Stuttered and dysfluent speech detection systems have traditionally suffered from the trade-off between accuracy and clinical interpretability. While end-to-end deep learning models achieve high performance, their black-box nature limits clinical adoption. This paper looks at the Unconstrained Dysfluency Modeling (UDM) series-the current state-of-the-art framework developed by Berkeley that combines modular architecture, explicit phoneme alignment, and interpretable outputs for real-world clinical deployment. Through extensive experiments involving patients and certified speech-language pathologists (SLPs), we demonstrate that UDM achieves state-of-the-art performance (F1: 0.89+-0.04) while providing clinically meaningful interpretability scores (4.2/5.0). Our deployment study shows 87% clinician acceptance rate and 34% reduction in diagnostic time. The results provide strong evidence that UDM represents a practical pathway toward AI-assisted speech therapy in clinical environments.

Method: SBLS integrates three elements: direct attribute inference using pre-trained classifiers, linkage detection via mutual information analysis, and subgroup robustness across intersecting attributes.

Result: All five evaluated de-identification systems showed significant vulnerabilities - adversaries using only pre-trained models can reliably recover soft biometric information from anonymized output.

Conclusion: Standard distributional metrics fail to capture fundamental weaknesses in speaker de-identification systems, and SBLS reveals that current systems are vulnerable to soft biometric leakage attacks.

Abstract: We use the term re-identification to refer to the process of recovering the original speaker’s identity from anonymized speech outputs. Speaker de-identification systems aim to reduce the risk of re-identification, but most evaluations focus only on individual-level measures and overlook broader risks from soft biometric leakage. We introduce the Soft Biometric Leakage Score (SBLS), a unified method that quantifies resistance to zero-shot inference attacks on non-unique traits such as channel type, age range, dialect, sex of the speaker, or speaking style. SBLS integrates three elements: direct attribute inference using pre-trained classifiers, linkage detection via mutual information analysis, and subgroup robustness across intersecting attributes. Applying SBLS with publicly available classifiers, we show that all five evaluated de-identification systems exhibit significant vulnerabilities. Our results indicate that adversaries using only pre-trained models - without access to original speech or system details - can still reliably recover soft biometric information from anonymized output, exposing fundamental weaknesses that standard distributional metrics fail to capture.

Method: Collection of real-time MRI video of vocal tract dynamics with simultaneous audio recording from a native American English speaker, followed by data processing to create derived representations including cropped vocal tract videos, sentence-level splits, restored audio, and ROI timeseries.

Result: A unique dataset containing approximately one hour of video and audio data, making it one of the longer publicly available single-speaker real-time MRI speech datasets with various derived representations suitable for multiple downstream tasks.

Conclusion: The USC LSS dataset provides valuable resources for speech research with baseline benchmarks for articulatory synthesis and phoneme recognition, serving as a foundation for future research improvements in these areas.

Abstract: We release the USC Long Single-Speaker (LSS) dataset containing real-time MRI video of the vocal tract dynamics and simultaneous audio obtained during speech production. This unique dataset contains roughly one hour of video and audio data from a single native speaker of American English, making it one of the longer publicly available single-speaker datasets of real-time MRI speech data. Along with the articulatory and acoustic raw data, we release derived representations of the data that are suitable for a range of downstream tasks. This includes video cropped to the vocal tract region, sentence-level splits of the data, restored and denoised audio, and regions-of-interest timeseries. We also benchmark this dataset on articulatory synthesis and phoneme recognition tasks, providing baseline performance for these tasks on this dataset which future research can aim to improve upon.

Method: Preprocesses audio prompts using forced alignment to obtain word boundaries, excludes transcripts during training, and trains speaking rate predictors at different linguistic granularities to derive duration from speaker pace.

Result: The approach matches the performance of F5-TTS while enabling cross-lingual voice cloning without requiring audio prompt transcripts.

Conclusion: The proposed framework successfully removes the dependency on audio prompt transcripts for flow-matching TTS models, enabling effective cross-lingual voice cloning.

Abstract: Flow-matching-based text-to-speech (TTS) models have shown high-quality speech synthesis. However, most current flow-matching-based TTS models still rely on reference transcripts corresponding to the audio prompt for synthesis. This dependency prevents cross-lingual voice cloning when audio prompt transcripts are unavailable, particularly for unseen languages. The key challenges for flow-matching-based TTS models to remove audio prompt transcripts are identifying word boundaries during training and determining appropriate duration during inference. In this paper, we introduce Cross-Lingual F5-TTS, a framework that enables cross-lingual voice cloning without audio prompt transcripts. Our method preprocesses audio prompts by forced alignment to obtain word boundaries, enabling direct synthesis from audio prompts while excluding transcripts during training. To address the duration modeling challenge, we train speaking rate predictors at different linguistic granularities to derive duration from speaker pace. Experiments show that our approach matches the performance of F5-TTS while enabling cross-lingual voice cloning.

Method: 1) Spatial audio encoder for detecting overlapping events and estimating Direction of Arrival (DoA) and source distance at frame level 2) Audio grounding model aligning audio features with semantic text embeddings via cross-attention 3) Conditioning LLMs on extracted spatial attributes for complex query answering

Result: The framework demonstrates performance against baseline models on a newly introduced spatial audio motion understanding and reasoning benchmark dataset.

Conclusion: The proposed approach effectively enables spatial audio reasoning for moving sources through a combination of specialized audio processing and language model integration, providing a comprehensive solution for dynamic audio scene interpretation.

Abstract: Spatial audio reasoning enables machines to interpret auditory scenes by understanding events and their spatial attributes. In this work, we focus on spatial audio understanding with an emphasis on reasoning about moving sources. First, we introduce a spatial audio encoder that processes spatial audio to detect multiple overlapping events and estimate their spatial attributes, Direction of Arrival (DoA) and source distance, at the frame level. To generalize to unseen events, we incorporate an audio grounding model that aligns audio features with semantic audio class text embeddings via a cross-attention mechanism. Second, to answer complex queries about dynamic audio scenes involving moving sources, we condition a large language model (LLM) on structured spatial attributes extracted by our model. Finally, we introduce a spatial audio motion understanding and reasoning benchmark dataset and demonstrate our framework’s performance against the baseline model.

Method: Investigated from behavioral and representational perspectives: tested different model backbones, unpaired instrumental tracks, and frequency subbands; analyzed how fine-tuning alters encoders’ speech and music capabilities.

Result: Instrumental accompaniment acts mainly as data augmentation rather than providing intrinsic cues like rhythm or harmony. Fine-tuning increases reliance on shallow speaker features while reducing sensitivity to content, paralinguistic, and semantic information.

Conclusion: These findings clarify how models exploit vocal vs instrumental cues and can inform the design of more interpretable and robust SingFake detection systems.

Abstract: Although many models exist to detect singing voice deepfakes (SingFake), how these models operate, particularly with instrumental accompaniment, is unclear. We investigate how instrumental music affects SingFake detection from two perspectives. To investigate the behavioral effect, we test different backbones, unpaired instrumental tracks, and frequency subbands. To analyze the representational effect, we probe how fine-tuning alters encoders’ speech and music capabilities. Our results show that instrumental accompaniment acts mainly as data augmentation rather than providing intrinsic cues (e.g., rhythm or harmony). Furthermore, fine-tuning increases reliance on shallow speaker features while reducing sensitivity to content, paralinguistic, and semantic information. These insights clarify how models exploit vocal versus instrumental cues and can inform the design of more interpretable and robust SingFake detection systems.

Method: Developed an integrated web toolkit with two graphical user interfaces: PiaRec for synchronized acquisition of audio, video, MIDI, and performance metadata, and ASDF for efficient annotation of performer fingering from visual data.

Result: The system provides a comprehensive solution for collecting multimodal piano performance datasets, addressing the data acquisition bottleneck in piano performance research.

Conclusion: This integrated toolkit can streamline the acquisition process and facilitate further progress in multimodal piano performance analysis by overcoming the barriers to large-scale data collection.

Abstract: Piano performance is a multimodal activity that intrinsically combines physical actions with the acoustic rendition. Despite growing research interest in analyzing the multimodal nature of piano performance, the laborious process of acquiring large-scale multimodal data remains a significant bottleneck, hindering further progress in this field. To overcome this barrier, we present an integrated web toolkit comprising two graphical user interfaces (GUIs): (i) PiaRec, which supports the synchronized acquisition of audio, video, MIDI, and performance metadata. (ii) ASDF, which enables the efficient annotation of performer fingering from the visual data. Collectively, this system can streamline the acquisition of multimodal piano performance datasets.

Method: Leveraged EEND-TA, a single unified non-autoregressive model for end-to-end speaker diarization, with pretraining through 8-speaker simulation mixtures to ensure sufficient representation of speaker configurations.

Result: Achieved new benchmark results on multiple datasets, most notably 14.49% DER on DIHARD III, surpassing many existing diarization solutions.

Conclusion: EEND-based architectures possess greater learning capacity than previously thought and can achieve state-of-the-art performance while maintaining efficient inference speeds.

Abstract: In this paper, we present state-of-the-art diarization error rates (DERs) on multiple publicly available datasets, including AliMeeting-far, AliMeeting-near, AMI-Mix, AMI-SDM, DIHARD III, and MagicData RAMC. Leveraging EEND-TA, a single unified non-autoregressive model for end-to-end speaker diarization, we achieve new benchmark results, most notably a DER of 14.49% on DIHARD III. Our approach scales pretraining through 8-speaker simulation mixtures, ensuring each generated speaker mixture configuration is sufficiently represented. These experiments highlight that EEND-based architectures possess a greater capacity for learning than previously explored, surpassing many existing diarization solutions while maintaining efficient speeds during inference.

Method: Proposed Spatial-CLAP with a content-aware spatial encoder that couples spatial representations with audio content, and spatial contrastive learning (SCL) training strategy that explicitly enforces correct correspondence between sound sources and their locations in multi-source conditions.

Result: Experimental evaluations show Spatial-CLAP learns effective embeddings even under multi-source conditions, with SCL proving effective. Evaluation on unseen three-source mixtures demonstrates the fundamental advantage over conventional single-source training approaches.

Conclusion: Spatial-CLAP establishes a new paradigm for spatially-aware audio-text embeddings that can handle multi-source conditions, representing a significant advancement over traditional single-source CLAP methods.

Abstract: Contrastive language–audio pretraining (CLAP) has achieved remarkable success as an audio–text embedding framework, but existing approaches are limited to monaural or single-source conditions and cannot fully capture spatial information. The central challenge in modeling spatial information lies in multi-source conditions, where the correct correspondence between each sound source and its location is required. To tackle this problem, we propose Spatial-CLAP, which introduces a content-aware spatial encoder that enables spatial representations coupled with audio content. We further propose spatial contrastive learning (SCL), a training strategy that explicitly enforces the learning of the correct correspondence and promotes more reliable embeddings under multi-source conditions. Experimental evaluations, including downstream tasks, demonstrate that Spatial-CLAP learns effective embeddings even under multi-source conditions, and confirm the effectiveness of SCL. Moreover, evaluation on unseen three-source mixtures highlights the fundamental distinction between conventional single-source training and our proposed multi-source training paradigm. These findings establish a new paradigm for spatially-aware audio–text embeddings.

Method: Uses LLM to plan non-overlapping time windows and recaption each with refined descriptions, then employs Decoupling and Aggregating Attention Control for timing precision, Contextual Latent Composition for local smoothness, and Reference Guidance for global consistency.

Result: Achieves state-of-the-art timing-conditioned T2A synthesis quality among training-free methods, comparable to leading training-based methods, and demonstrates comparable long-form generation quality with training-based Stable Audio.

Conclusion: FreeAudio paves the way for timing-controlled long-form T2A synthesis and represents a significant advancement in training-free audio generation with precise temporal control.

Abstract: Text-to-audio (T2A) generation has achieved promising results with the recent advances in generative models. However, because of the limited quality and quantity of temporally-aligned audio-text pairs, existing T2A methods struggle to handle the complex text prompts that contain precise timing control, e.g., “owl hooted at 2.4s-5.2s”. Recent works have explored data augmentation techniques or introduced timing conditions as model inputs to enable timing-conditioned 10-second T2A generation, while their synthesis quality is still limited. In this work, we propose a novel training-free timing-controlled T2A framework, FreeAudio, making the first attempt to enable timing-controlled long-form T2A generation, e.g., “owl hooted at 2.4s-5.2s and crickets chirping at 0s-24s”. Specifically, we first employ an LLM to plan non-overlapping time windows and recaption each with a refined natural language description, based on the input text and timing prompts. Then we introduce: 1) Decoupling and Aggregating Attention Control for precise timing control; 2) Contextual Latent Composition for local smoothness and Reference Guidance for global consistency. Extensive experiments show that: 1) FreeAudio achieves state-of-the-art timing-conditioned T2A synthesis quality among training-free methods and is comparable to leading training-based methods; 2) FreeAudio demonstrates comparable long-form generation quality with training-based Stable Audio and paves the way for timing-controlled long-form T2A synthesis. Demo samples are available at: https://freeaudio.github.io/FreeAudio/

Method: Developed XLSR-Thai by continuously training XLSR model on 36K hours of Thai speech; proposed U-Align for resource-efficient speech-text alignment; created Thai-SUP pipeline to generate Thai SLU data from high-resource languages.

Result: Successfully built the first Thai multitask-understanding SLLM with over 1,000 hours of Thai spoken language understanding data, demonstrating effectiveness through multiple experiments.

Conclusion: The proposed methods effectively overcome challenges in low-resource language SLLMs, with XLSR-Thai and Thai-SUP being open-sourced to facilitate future research in this area.

Abstract: Speech large language models (SLLMs) built on speech encoders, adapters, and LLMs demonstrate remarkable multitask understanding performance in high-resource languages such as English and Chinese. However, their effectiveness substantially degrades in low-resource languages such as Thai. This limitation arises from three factors: (1) existing commonly used speech encoders, like the Whisper family, underperform in low-resource languages and lack support for broader spoken language understanding tasks; (2) the ASR-based alignment paradigm requires training the entire SLLM, leading to high computational cost; (3) paired speech-text data in low-resource languages is scarce. To overcome these challenges in the low-resource language Thai, we introduce XLSR-Thai, the first self-supervised learning (SSL) speech encoder for Thai. It is obtained by continuously training the standard SSL XLSR model on 36,000 hours of Thai speech data. Furthermore, we propose U-Align, a speech-text alignment method that is more resource-efficient and multitask-effective than typical ASR-based alignment. Finally, we present Thai-SUP, a pipeline for generating Thai spoken language understanding data from high-resource languages, yielding the first Thai spoken language understanding dataset of over 1,000 hours. Multiple experiments demonstrate the effectiveness of our methods in building a Thai multitask-understanding SLLM. We open-source XLSR-Thai and Thai-SUP to facilitate future research.

Method: The model learns average velocity over finite intervals using a Jacobian-vector product to instantiate the MeanFlow identity, with a local training objective that supervises finite-interval displacement while maintaining consistency with instantaneous-field constraints.

Result: On VoiceBank-DEMAND, the single-step model achieves strong intelligibility, fidelity, and perceptual quality with substantially lower computational cost than multistep baselines.

Conclusion: MeanFlowSE provides an efficient, high-fidelity framework for real-time generative speech enhancement without requiring knowledge distillation or external teachers, enabling single-step generation with optional refinement steps.

Abstract: Multistep inference is a bottleneck for real-time generative speech enhancement because flow- and diffusion-based systems learn an instantaneous velocity field and therefore rely on iterative ordinary differential equation (ODE) solvers. We introduce MeanFlowSE, a conditional generative model that learns the average velocity over finite intervals along a trajectory. Using a Jacobian-vector product (JVP) to instantiate the MeanFlow identity, we derive a local training objective that directly supervises finite-interval displacement while remaining consistent with the instantaneous-field constraint on the diagonal. At inference, MeanFlowSE performs single-step generation via a backward-in-time displacement, removing the need for multistep solvers; an optional few-step variant offers additional refinement. On VoiceBank-DEMAND, the single-step model achieves strong intelligibility, fidelity, and perceptual quality with substantially lower computational cost than multistep baselines. The method requires no knowledge distillation or external teachers, providing an efficient, high-fidelity framework for real-time generative speech enhancement.

Method: Systematic evaluation by freezing/updating visual encoder, scaling decoder size, comparing adaptation strategies and architectures, and varying training data across LRS2, LRS3, and their combination.

Result: Llama-2-13B model trained on combined dataset achieves 24.7% WER on LRS3 and 47.0% on WildVSR (SOTA without additional supervision). Gains come from lexical rather than semantic processing.

Conclusion: LLM decoders refine contextual reasoning rather than visual features, emphasizing the need for stronger visual encoders for meaningful progress in VSR.

Abstract: Advances in self-supervised encoders have improved Visual Speech Recognition (VSR). Recent approaches integrating these encoders with LLM decoders improves transcription accuracy; however, it remains unclear whether these gains stem from visual understanding or stronger language modeling. In this work, we systematically evaluate LLM decoders by freezing or selectively updating the visual encoder, scaling decoder size, comparing adaptation strategies and architectures, and varying training data across LRS2, LRS3, and their combination. Evaluation on LRS2, LRS3, and WildVSR shows that scaling and adaptation yield limited improvements, while combining datasets enhances generalization. Semantic analysis reveals that gains arise primarily from lexical rather than semantic processing. Our Llama-2-13B model trained on the combined set achieves 24.7% WER on LRS3 and 47.0% on WildVSR, establishing SOTA among models trained without additional supervision. Our findings indicate LLM decoders refine contextual reasoning rather than visual features, emphasizing the need for stronger visual encoders to drive meaningful progress.

Method: Constructs temporal graphs for each event with audio/video segments as nodes and temporal links as edges. Uses Gaussian processes for intra-modal smoothness, Hawkes processes for inter-modal decay, and contrastive learning for fine-grained relationships.

Result: Achieves state-of-the-art performance on AudioSet benchmark.

Conclusion: The proposed THGCL framework effectively addresses temporal alignment challenges and captures both intra- and inter-modal dependencies, demonstrating superior performance in multimodal acoustic event classification.

Abstract: Multimodal acoustic event classification plays a key role in audio-visual systems. Although combining audio and visual signals improves recognition, it is still difficult to align them over time and to reduce the effect of noise across modalities. Existing methods often treat audio and visual streams separately, fusing features later with contrastive or mutual information objectives. Recent advances explore multimodal graph learning, but most fail to distinguish between intra- and inter-modal temporal dependencies. To address this, we propose Temporally Heterogeneous Graph-based Contrastive Learning (THGCL). Our framework constructs a temporal graph for each event, where audio and video segments form nodes and their temporal links form edges. We introduce Gaussian processes for intra-modal smoothness, Hawkes processes for inter-modal decay, and contrastive learning to capture fine-grained relationships. Experiments on AudioSet show that THGCL achieves state-of-the-art performance.

Method: Proposes three key improvements: (1) K-weighting perceptual filter before loss calculation, (2) two novel phase losses (Correlation Loss for stereo coherence and Phase Loss using Instantaneous Frequency/Group Delay), (3) new spectral supervision paradigm with magnitude supervised by all four Mid/Side/Left/Right components and phase supervised only by LR components.

Result: εar-VAE at 44.1kHz substantially outperforms leading open-source models across diverse metrics, showing particular strength in reconstructing high-frequency harmonics and spatial characteristics.

Conclusion: The proposed εar-VAE successfully addresses perceptual weaknesses in existing audio VAEs through optimized training paradigm with auditory-aligned objectives and improved phase/spatial representation.

Abstract: Variational Autoencoders (VAEs) are essential for large-scale audio tasks like diffusion-based generation. However, existing open-source models often neglect auditory perceptual aspects during training, leading to weaknesses in phase accuracy and stereophonic spatial representation. To address these challenges, we propose {\epsilon}ar-VAE, an open-source music signal reconstruction model that rethinks and optimizes the VAE training paradigm. Our contributions are threefold: (i) A K-weighting perceptual filter applied prior to loss calculation to align the objective with auditory perception. (ii) Two novel phase losses: a Correlation Loss for stereo coherence, and a Phase Loss using its derivatives–Instantaneous Frequency and Group Delay–for precision. (iii) A new spectral supervision paradigm where magnitude is supervised by all four Mid/Side/Left/Right components, while phase is supervised only by the LR components. Experiments show {\epsilon}ar-VAE at 44.1kHz substantially outperforms leading open-source models across diverse metrics, showing particular strength in reconstructing high-frequency harmonics and the spatial characteristics.

Method: Latent-space fusion framework that integrates estimated respiratory effort embeddings from audio with acoustic features for OSA detection. Uses only smartphone audio at test time.

Result: Respiratory effort estimator achieved CCC of 0.48. Fusion of effort and audio improved sensitivity and AUC over audio-only baselines, especially at low AHI thresholds.

Conclusion: Enables sensor-free, scalable OSA monitoring using only smartphone audio, capturing physiological context from sound alone for improved detection performance.

Abstract: Obstructive sleep apnoea (OSA) is a prevalent condition with significant health consequences, yet many patients remain undiagnosed due to the complexity and cost of over-night polysomnography. Acoustic-based screening provides a scalable alternative, yet performance is limited by environmental noise and the lack of physiological context. Respiratory effort is a key signal used in clinical scoring of OSA events, but current approaches require additional contact sensors that reduce scalability and patient comfort. This paper presents the first study to estimate respiratory effort directly from nocturnal audio, enabling physiological context to be recovered from sound alone. We propose a latent-space fusion framework that integrates the estimated effort embeddings with acoustic features for OSA detection. Using a dataset of 157 nights from 103 participants recorded in home environments, our respiratory effort estimator achieves a concordance correlation coefficient of 0.48, capturing meaningful respiratory dynamics. Fusing effort and audio improves sensitivity and AUC over audio-only baselines, especially at low apnoea-hypopnoea index thresholds. The proposed approach requires only smartphone audio at test time, which enables sensor-free, scalable, and longitudinal OSA monitoring.

Method: Proposes FCPE using a Lynx-Net architecture with depth-wise separable convolutions to effectively capture mel spectrogram features while maintaining low computational cost and robust noise tolerance.

Result: Achieves 96.79% Raw Pitch Accuracy on MIR-1K dataset (on par with SOTA) with Real-Time Factor of 0.0062 on RTX 4090 GPU, significantly outperforming existing algorithms in efficiency.

Conclusion: FCPE provides an efficient and accurate pitch estimation solution with strong noise robustness, making it suitable for real-time applications in audio processing.

Abstract: Pitch estimation (PE) in monophonic audio is crucial for MIDI transcription and singing voice conversion (SVC), but existing methods suffer significant performance degradation under noise. In this paper, we propose FCPE, a fast context-based pitch estimation model that employs a Lynx-Net architecture with depth-wise separable convolutions to effectively capture mel spectrogram features while maintaining low computational cost and robust noise tolerance. Experiments show that our method achieves 96.79% Raw Pitch Accuracy (RPA) on the MIR-1K dataset, on par with the state-of-the-art methods. The Real-Time Factor (RTF) is 0.0062 on a single RTX 4090 GPU, which significantly outperforms existing algorithms in efficiency. Code is available at https://github.com/CNChTu/FCPE.

Method: The researchers applied various probing methods to embeddings from foundation models (large-scale neural architectures pretrained on multimodal data) to examine relationships between audio FX and estimated emotion.

Result: The study uncovered complex, nonlinear patterns tied to specific audio effects and evaluated the robustness of foundation audio models in capturing emotional consequences of sound design techniques.

Conclusion: The findings advance understanding of how audio production practices affect perception, with implications for music cognition, performance, and affective computing applications.

Abstract: Audio effects (FX) such as reverberation, distortion, modulation, and dynamic range processing play a pivotal role in shaping emotional responses during music listening. While prior studies have examined links between low-level audio features and affective perception, the systematic impact of audio FX on emotion remains underexplored. This work investigates how foundation models - large-scale neural architectures pretrained on multimodal data - can be leveraged to analyze these effects. Such models encode rich associations between musical structure, timbre, and affective meaning, offering a powerful framework for probing the emotional consequences of sound design techniques. By applying various probing methods to embeddings from deep learning models, we examine the complex, nonlinear relationships between audio FX and estimated emotion, uncovering patterns tied to specific effects and evaluating the robustness of foundation audio models. Our findings aim to advance understanding of the perceptual impact of audio production practices, with implications for music cognition, performance, and affective computing.

Method: Mesh-infused Neural Acoustic Field (MiNAF) queries rough room meshes at given locations and extracts distance distributions as explicit representations of local geometric context to guide neural network predictions.

Result: MiNAF performs competitively across various evaluation metrics compared to conventional and state-of-the-art baseline methods, and shows robustness in datasets with limited training samples.

Conclusion: Incorporating explicit local geometric features significantly improves neural implicit models for high-fidelity sound simulation, advancing the field of realistic acoustic modeling.

Abstract: Realistic sound simulation plays a critical role in many applications. A key element in sound simulation is the room impulse response (RIR), which characterizes how sound propagates from a source to a listener within a given space. Recent studies have applied neural implicit methods to learn RIR using context information collected from the environment, such as scene images. However, these approaches do not effectively leverage explicit geometric information from the environment. To further exploit the potential of neural implicit models with direct geometric features, we present Mesh-infused Neural Acoustic Field (MiNAF), which queries a rough room mesh at given locations and extracts distance distributions as an explicit representation of local context. Our approach demonstrates that incorporating explicit local geometric features can better guide the neural network in generating more accurate RIR predictions. Through comparisons with conventional and state-of-the-art baseline methods, we show that MiNAF performs competitively across various evaluation metrics. Furthermore, we verify the robustness of MiNAF in datasets with limited training samples, demonstrating an advance in high-fidelity sound simulation.

Method: Time domain formulation for linearly constrained minimum variance active noise control (LCMV ANC) with adaptive FxLMS algorithm for online filter coefficient adaptation.

Result: Simulation and experimental results show effective noise reduction and constraint adherence, demonstrating spatially selective and broadband noise control.

Conclusion: LCMV ANC provides a more flexible alternative to uniformly weighted multipoint minimization, enabling prioritized noise reduction at specific spatial locations through strategic linear constraints.

Abstract: Traditional volumetric noise control typically relies on multipoint error minimization to suppress sound energy across a region, but offers limited flexibility in shaping spatial responses. This paper introduces a time domain formulation for linearly constrained minimum variance active noise control (LCMV ANC) for spatial control filter design. We demonstrate how the LCMV ANC optimization framework allows system designers to prioritize noise reduction at specific spatial locations through strategically defined linear constraints, providing a more flexible alternative to uniformly weighted multi point error minimization. An adaptive algorithm based of filtered X least mean squares (FxLMS) is derived for online adaptation of filter coefficients. Simulation and experimental results validate the proposed method’s noise reduction and constraint adherence, demonstrating effective, spatially selective and broadband noise control compared to multipoint volumetric noise control.

Method: Proposes SALM framework with dual-branch audio encoder that decomposes spatial sound into semantic and spatial components via structured audio embeddings, integrated with text encoder through multi-modal contrastive learning.

Result: SALM effectively captures and aligns cross-modal representations, yields well-structured audio embeddings, enables zero-shot direction classification, and supports advanced spatial audio editing capabilities including text-based directional modifications.

Conclusion: SALM successfully bridges the gap between spatial audio and language processing, providing a comprehensive framework for spatial audio understanding and manipulation with flexible editing capabilities.

Abstract: Spatial audio understanding is essential for accurately perceiving and interpreting acoustic environments. However, existing audio-language models exhibit limitations in processing spatial audio and perceiving spatial acoustic scenes. To address this gap, we propose the Spatial Audio Language Model (SALM), a novel framework that bridges spatial audio and language through multi-modal contrastive learning. SALM integrates a text encoder with a dual-branch audio encoder that decomposes spatial sound into semantic and spatial components via structured audio embeddings. Key features of SALM include seamless alignment between spatial audio and natural language, both separate and joint extraction of spatial and semantic representations, zero-shot direction classification, and flexible support for spatial audio editing. Experimental results demonstrate that SALM effectively captures and aligns cross-modal representations, yielding well-structured audio embeddings. Furthermore, SALM enables advanced editing capabilities, such as modifying directional audio using text-based embeddings.

Method: Proposes Omni-CLST framework with two key strategies: error-aware curriculum learning that organizes samples by difficulty level, and guided thought dropout mechanism that focuses reasoning on challenging cases using selective chain-of-thought.

Result: Achieves 73.80% on MMAU-mini and sets new state-of-the-art of 64.30% on MMAR benchmark, demonstrating robust generalization in multimodal audio-language understanding tasks.

Conclusion: The framework effectively leverages existing high-quality AQA data through curriculum learning and selective reasoning mechanisms, showing significant performance improvements and strong generalization capabilities in audio question answering.

Abstract: With the rapid progress of large audio-language models (LALMs), audio question answering (AQA) has emerged as a challenging task requiring both fine-grained audio understanding and complex reasoning. While current methods mainly rely on constructing new datasets via captioning or reasoning traces, existing high-quality AQA data remains underutilized. To address this, we propose Omni-CLST, an error-aware Curriculum Learning framework with guided Selective Chain-of-Thought. The framework efficiently leverages existing high-quality dataset through two key strategies: an error-aware curriculum that organizes samples by difficulty, and a guided thought dropout mechanism that focuses reasoning on challenging cases. Experiments show that Omni-CLST achieves 73.80% on MMAU-mini and a new state of the art of 64.30% on MMAR, demonstrating robust generalization in multimodal audio-language understanding.

Method: Proposed Global-Local Aware Dynamic (GLAD) Mixture-of-Experts that dynamically fuses speaker-aware global information and fine-grained local features to guide expert selection for speaker-specific routing.

Result: Outperforms existing MTASR approaches on LibriSpeechMix, particularly in challenging multi-talker scenarios with high overlap.

Conclusion: First work to apply Mixture-of-Experts to end-to-end MTASR with global-local fusion strategy, demonstrating significant improvements in multi-speaker transcription accuracy.

Abstract: End-to-end multi-talker automatic speech recognition (MTASR) faces significant challenges in accurately transcribing overlapping speech, especially under high-overlap conditions. To address these challenges, we proposed Global-Local Aware Dynamic (GLAD) Mixture-of-Experts, which dynamically fuse speaker-aware global information and fine-grained local features to guide expert selection. This mechanism enables speaker-specific routing by leveraging both global context and local acoustic cues. Experiments on LibriSpeechMix show that GLAD outperforms existing MTASR approaches, particularly in challenging multi-talker scenarios. To our best knowledge, this is the first work to apply Mixture-of-Experts (MoE) to end-to-end MTASR with a global-local fusion strategy. Our code and train dataset can be found at https://github.com/NKU-HLT/GLAD.

Method: The authors introduce SSEU-Bench, which explicitly accounts for energy differences between speech and non-speech audio, and includes both independent and joint understanding settings. They also propose Chain-of-Thought to improve joint understanding performance.

Result: The benchmark reveals that some LALMs underperform on certain tasks in joint understanding settings. Chain-of-Thought effectively improves LALMs’ joint audio understanding by decomposing complex tasks into simpler reasoning steps.

Conclusion: SSEU-Bench provides a more comprehensive evaluation framework for audio understanding that better reflects real-world scenarios, and Chain-of-Thought offers an effective solution to enhance joint audio understanding capabilities in LALMs.

Abstract: Recently, Large Audio Language Models (LALMs) have progressed rapidly, demonstrating their strong efficacy in universal audio understanding through cross-modal integration. To evaluate LALMs’ audio understanding performance, researchers have proposed different benchmarks. However, key aspects for real-world interactions are underexplored in existing benchmarks, i.e., audio signals typically contain both speech and non-speech components, and energy levels of these components can vary significantly across different scenarios. Moreover, most benchmarks do not consider the joint understanding of speech, scene, and events within the same audio clip. In this work, we introduce SSEU-Bench, the first versatile audio understanding benchmark that explicitly accounts for energy differences between speech and non-speech audio, with both independent and joint understanding settings for speech, scene, and events. Furthermore, we demonstrate that some LALMs tend to underperform on certain tasks in a joint understanding setting. To address this issue, we introduce Chain-of-Thought, which effectively improves LALMs’ joint audio understanding performance by decomposing complex tasks into simpler reasoning steps.

Method: Signal-dependent framework combining multiple binaural filters in an online manner using implicit localization, enabling dynamic spatial audio rendering that adapts to continuous talker motion without explicit direction estimation.

Result: Enables real-time tracking and enhancement of moving sound sources, supports speech focus, noise reduction, and world-locked audio in AR/VR applications, and is agnostic to array geometry for flexible deployment.

Conclusion: The proposed mixture of experts framework provides a flexible, real-time solution for spatial audio capture and personalized playback in next-generation consumer audio devices, overcoming limitations of traditional binaural processing methods.

Abstract: We propose a novel mixture of experts framework for field-of-view enhancement in binaural signal matching. Our approach enables dynamic spatial audio rendering that adapts to continuous talker motion, allowing users to emphasize or suppress sounds from selected directions while preserving natural binaural cues. Unlike traditional methods that rely on explicit direction-of-arrival estimation or operate in the Ambisonics domain, our signal-dependent framework combines multiple binaural filters in an online manner using implicit localization. This allows for real-time tracking and enhancement of moving sound sources, supporting applications such as speech focus, noise reduction, and world-locked audio in augmented and virtual reality. The method is agnostic to array geometry offering a flexible solution for spatial audio capture and personalized playback in next-generation consumer audio devices.

Method: Proposes OS-SCL technique that perturbs features in embedding space and uses one-stage noisy supervised contrastive learning. Also introduces TFgram time-frequency feature extracted from raw audio to capture critical detection information.

Result: Achieved 94.64% AUC, 88.42% pAUC, and 89.24% mAUC using only Log-Mel features. With TFgram feature, performance improved to 95.71% AUC, 90.23% pAUC, and 91.23% mAUC on DCASE 2020 Challenge Task 2.

Conclusion: OS-SCL effectively addresses false alarm issues in anomalous sound detection and achieves superior performance. The proposed TFgram feature further enhances detection capabilities by capturing essential audio information.

Abstract: Unsupervised anomalous sound detection aims to detect unknown anomalous sounds by training a model using only normal audio data. Despite advancements in self-supervised methods, the issue of frequent false alarms when handling samples of the same type from different machines remains unresolved. This paper introduces a novel training technique called one-stage supervised contrastive learning (OS-SCL), which significantly addresses this problem by perturbing features in the embedding space and employing a one-stage noisy supervised contrastive learning approach. On the DCASE 2020 Challenge Task 2, it achieved 94.64% AUC, 88.42% pAUC, and 89.24% mAUC using only Log-Mel features. Additionally, a time-frequency feature named TFgram is proposed, which is extracted from raw audio. This feature effectively captures critical information for anomalous sound detection, ultimately achieving 95.71% AUC, 90.23% pAUC, and 91.23% mAUC. The source code is available at: \underline{www.github.com/huangswt/OS-SCL}.

Method: Proposed Conjecturing-Proving Loop pipeline that generates and proves conjectures in Lean 4 format, using context from previously generated theorems and proofs for in-context learning without changing LLM parameters.

Result: The framework successfully rediscovered theorems published in past mathematical papers that were not yet formalized. At least one theorem could not be proved by the LLM without in-context learning, demonstrating its effectiveness.

Conclusion: In-context learning is effective for neural theorem proving, enabling LLMs to generate and prove increasingly difficult mathematical conjectures through iterative learning from previous proofs.

Abstract: Large Language Models have demonstrated significant promise in formal theorem proving. However, previous works mainly focus on solving existing problems. In this paper, we focus on the ability of LLMs to find novel theorems. We propose Conjecturing-Proving Loop pipeline for automatically generating mathematical conjectures and proving them in Lean 4 format. A feature of our approach is that we generate and prove further conjectures with context including previously generated theorems and their proofs, which enables the generation of more difficult proofs by in-context learning of proof strategies without changing parameters of LLMs. We demonstrated that our framework rediscovered theorems with verification, which were published in past mathematical papers and have not yet formalized. Moreover, at least one of these theorems could not be proved by the LLM without in-context learning, even in natural language, which means that in-context learning was effective for neural theorem proving. The source code is available at https://github.com/auto-res/ConjecturingProvingLoop.

Method: Systematic experimentation with various DNN configurations including activation functions (tanh, sin, ReLU), network depth (4-8 hidden layers), width (64-256 neurons), and training methodologies (collocation points, epoch count). Comparative analysis with traditional numerical methods.

Result: Tanh activation provides stable convergence across configurations, while sine activation can achieve slightly lower errors and training times in some cases but may produce nonphysical artifacts. Optimally configured DNNs offer competitive accuracy with different computational trade-offs compared to traditional methods.

Conclusion: Standard feed-forward architectures inherently oversmooth sharp features due to function space limitations of standard activation functions, presenting fundamental constraints for handling singular or piecewise-constant solutions. The study provides empirical guidelines for DNN implementation while highlighting theoretical limitations that need to be overcome for more challenging physical systems.

Abstract: In this paper, we investigate the efficiency of Deep Neural Networks (DNNs) to approximate the solution of a nonlocal conservation law derived from the identical-oscillator Kuramoto model, focusing on the evaluation of an architectural choice and its impact on solution accuracy based on the energy norm and computation time. Through systematic experimentation, we demonstrate that network configuration parameters-specifically, activation function selection (tanh vs. sin vs. ReLU), network depth (4-8 hidden layers), width (64-256 neurons), and training methodology (collocation points, epoch count)-significantly influence convergence characteristics. We observe that tanh activation yields stable convergence across configurations, whereas sine activation can attain marginally lower errors and training times in isolated cases, but occasionally produce nonphysical artefacts. Our comparative analysis with traditional numerical methods shows that optimally configured DNNs offer competitive accuracy with notably different computational trade-offs. Furthermore, we identify fundamental limitations of standard feed-forward architectures when handling singular or piecewise-constant solutions, providing empirical evidence that such networks inherently oversmooth sharp features due to the natural function space limitations of standard activation functions. This work contributes to the growing body of research on neural network-based scientific computing by providing practitioners with empirical guidelines for DNN implementation while illuminating fundamental theoretical constraints that must be overcome to expand their applicability to more challenging physical systems with discontinuities.

Method: Rigorous mathematical analysis and comprehensive empirical evaluation including negative reward training, symmetric loss functions, and post-hoc calibration methods across MNIST, Fashion-MNIST, and CIFAR-10 datasets.

Result: Universal failure patterns: negative rewards cause extreme underconfidence (ECE > 0.8) and destroy diversity (std < 0.05), symmetric losses fail to escape binary averaging, and post-hoc methods achieve calibration (ECE < 0.02) only by compressing confidence distributions. 100% failure rate for training methods, 33% success for post-hoc.

Conclusion: Binary supervision creates an underspecified mapping problem where different confidence levels for correct predictions receive identical feedback. This explains neural network hallucinations and establishes why post-hoc calibration is mathematically necessary. Novel supervision paradigms using ensemble disagreement and adaptive multi-agent learning are proposed to overcome these limitations.

Abstract: We prove a fundamental impossibility theorem: neural networks cannot simultaneously learn well-calibrated confidence estimates with meaningful diversity when trained using binary correct/incorrect supervision. Through rigorous mathematical analysis and comprehensive empirical evaluation spanning negative reward training, symmetric loss functions, and post-hoc calibration methods, we demonstrate this is an information-theoretic constraint, not a methodological failure. Our experiments reveal universal failure patterns: negative rewards produce extreme underconfidence (ECE greater than 0.8) while destroying confidence diversity (std less than 0.05), symmetric losses fail to escape binary signal averaging, and post-hoc methods achieve calibration (ECE less than 0.02) only by compressing the confidence distribution. We formalize this as an underspecified mapping problem where binary signals cannot distinguish between different confidence levels for correct predictions: a 60 percent confident correct answer receives identical supervision to a 90 percent confident one. Crucially, our real-world validation shows 100 percent failure rate for all training methods across MNIST, Fashion-MNIST, and CIFAR-10, while post-hoc calibration’s 33 percent success rate paradoxically confirms our theorem by achieving calibration through transformation rather than learning. This impossibility directly explains neural network hallucinations and establishes why post-hoc calibration is mathematically necessary, not merely convenient. We propose novel supervision paradigms using ensemble disagreement and adaptive multi-agent learning that could overcome these fundamental limitations without requiring human confidence annotations.

Method: Q-ROAR - a RoPE-aware weight-only stabilization that groups RoPE dimensions into frequency bands and performs a small search over per-band scales for W_Q and W_K matrices, guided by diagnostics (Interpolation Pressure and Tail Inflation Ratios).

Result: Q-ROAR recovers up to 0.7% accuracy on standard tasks, reduces GovReport perplexity by more than 10%, while preserving short-context performance and maintaining compatibility with existing inference stacks.

Conclusion: The proposed Q-ROAR method effectively addresses the accuracy degradation issues when combining position interpolation with quantization, providing a practical solution that requires no fine-tuning or architectural changes.

Abstract: Extending LLM context windows is crucial for long range tasks. RoPE-based position interpolation (PI) methods like linear and frequency-aware scaling extend input lengths without retraining, while post-training quantization (PTQ) enables practical deployment. We show that combining PI with PTQ degrades accuracy due to coupled effects long context aliasing, dynamic range dilation, axis grid anisotropy, and outlier shifting that induce position-dependent logit noise. We provide the first systematic analysis of PI plus PTQ and introduce two diagnostics: Interpolation Pressure (per-band phase scaling sensitivity) and Tail Inflation Ratios (outlier shift from short to long contexts). To address this, we propose Q-ROAR, a RoPE-aware, weight-only stabilization that groups RoPE dimensions into a few frequency bands and performs a small search over per-band scales for W_Q,W_K, with an optional symmetric variant to preserve logit scale. The diagnostics guided search uses a tiny long-context dev set and requires no fine-tuning, kernel, or architecture changes. Empirically, Q-ROAR recovers up to 0.7% accuracy on standard tasks and reduces GovReport perplexity by more than 10%, while preserving short-context performance and compatibility with existing inference stacks.

Method: Leverages powerful pretrained encoders to produce rich embeddings, then applies classical training-free hashing techniques: PCA, random orthogonal projection, and threshold binarization without any additional learning or fine-tuning.

Result: Competitive retrieval performance demonstrated on standard image retrieval benchmarks and a newly introduced audio hashing benchmark.

Conclusion: The approach provides a strong baseline for hashing that is training-free, generalizable across vision and audio domains, and leverages existing pretrained models effectively.

Abstract: Information retrieval with compact binary embeddings, also referred to as hashing, is crucial for scalable fast search applications, yet state-of-the-art hashing methods require expensive, scenario-specific training. In this work, we introduce Hashing-Baseline, a strong training-free hashing method leveraging powerful pretrained encoders that produce rich pretrained embeddings. We revisit classical, training-free hashing techniques: principal component analysis, random orthogonal projection, and threshold binarization, to produce a strong baseline for hashing. Our approach combines these techniques with frozen embeddings from state-of-the-art vision and audio encoders to yield competitive retrieval performance without any additional learning or fine-tuning. To demonstrate the generality and effectiveness of this approach, we evaluate it on standard image retrieval benchmarks as well as a newly introduced benchmark for audio hashing.

Method: Formulates aggregation as a masked optimal transport problem using Sinkhorn scaling to compute transport-based aggregation weights that align the availability distribution q with importance distribution p.

Result: Achieves O(1/√T) convergence rate independent of participating users per round, with drastically improved performance across heterogeneous, fairness-sensitive, and low-availability regimes.

Conclusion: FedAVOT provides provable convergence guarantees and superior performance compared to FedAvg, even with as few as two clients participating per round.

Abstract: Federated Learning (FL) allows distributed model training without sharing raw data, but suffers when client participation is partial. In practice, the distribution of available users (\emph{availability distribution} $q$) rarely aligns with the distribution defining the optimization objective (\emph{importance distribution} $p$), leading to biased and unstable updates under classical FedAvg. We propose \textbf{Fereated AVerage with Optimal Transport (\textbf{FedAVOT})}, which formulates aggregation as a masked optimal transport problem aligning $q$ and $p$. Using Sinkhorn scaling, \textbf{FedAVOT} computes transport-based aggregation weights with provable convergence guarantees. \textbf{FedAVOT} achieves a standard $\mathcal{O}(1/\sqrt{T})$ rate under a nonsmooth convex FL setting, independent of the number of participating users per round. Our experiments confirm drastically improved performance compared to FedAvg across heterogeneous, fairness-sensitive, and low-availability regimes, even when only two clients participate per round.

Method: Developed a lightweight (non-machine learning) anomaly detection algorithm that identifies anomalies based on user-defined criteria, highlights specific regions triggering flags, and quantifies anomaly likelihood.

Result: The proposed model outperforms existing methods and provides explainability for qualitative evaluation by domain experts. A dataset of 2,000 observations (50% anomalous, 50% normal) was created for comparative analysis.

Conclusion: The H-Alpha Anomalyzer offers an effective, explainable solution for detecting anomalies in astronomical data streams, addressing the critical need for data quality assurance in astrophysical ML applications.

Abstract: The plethora of space-borne and ground-based observatories has provided astrophysicists with an unprecedented volume of data, which can only be processed at scale using advanced computing algorithms. Consequently, ensuring the quality of data fed into machine learning (ML) models is critical. The H$\alpha$ observations from the GONG network represent one such data stream, producing several observations per minute, 24/7, since 2010. In this study, we introduce a lightweight (non-ML) anomaly-detection algorithm, called H-Alpha Anomalyzer, designed to identify anomalous observations based on user-defined criteria. Unlike many black-box algorithms, our approach highlights exactly which regions triggered the anomaly flag and quantifies the corresponding anomaly likelihood. For our comparative analysis, we also created and released a dataset of 2,000 observations, equally divided between anomalous and non-anomalous cases. Our results demonstrate that the proposed model not only outperforms existing methods but also provides explainability, enabling qualitative evaluation by domain experts.

Method: Each node independently samples one regularizer uniformly and coordinates only with nodes sharing that term, replacing the proximal map of the sum with the proximal map of a single randomly selected regularizer.

Result: Achieves ~O(ε⁻²) iterations for convex objectives, O(ε⁻¹) to ε-solution under strong convexity, and O(log(1/ε)) to neighborhood convergence. Communication drops to exactly 2 messages per iteration for graph-guided regularizers.

Conclusion: Randomized local coordination preserves convergence while eliminating global coordination, significantly improving communication efficiency without sacrificing performance.

Abstract: Distributed optimization requires nodes to coordinate, yet full synchronization scales poorly. When $n$ nodes collaborate through $m$ pairwise regularizers, standard methods demand $\mathcal{O}(m)$ communications per iteration. This paper proposes randomized local coordination: each node independently samples one regularizer uniformly and coordinates only with nodes sharing that term. This exploits partial separability, where each regularizer $G_j$ depends on a subset $S_j \subseteq {1,\ldots,n}$ of nodes. For graph-guided regularizers where $|S_j|=2$, expected communication drops to exactly 2 messages per iteration. This method achieves $\tilde{\mathcal{O}}(\varepsilon^{-2})$ iterations for convex objectives and under strong convexity, $\mathcal{O}(\varepsilon^{-1})$ to an $\varepsilon$-solution and $\mathcal{O}(\log(1/\varepsilon))$ to a neighborhood. Replacing the proximal map of the sum $\sum_j G_j$ with the proximal map of a single randomly selected regularizer $G_j$ preserves convergence while eliminating global coordination. Experiments validate both convergence rates and communication efficiency across synthetic and real-world datasets.

Method: Leverages large language models to generate dense contextual embeddings from raw sandbox behavior reports, then processes these embeddings using a one-dimensional convolutional neural network for multi-class malware classification.

Result: Evaluated on Avast-CTU Public CAPE Dataset, the framework consistently outperforms existing methods in malware classification.

Conclusion: LLM-based behavioral embeddings and the BEACON framework design are highly effective for robust malware classification, demonstrating the value of combining language models with behavioral analysis.

Abstract: Malware is becoming increasingly complex and widespread, making it essential to develop more effective and timely detection methods. Traditional static analysis often fails to defend against modern threats that employ code obfuscation, polymorphism, and other evasion techniques. In contrast, behavioral malware detection, which monitors runtime activities, provides a more reliable and context-aware solution. In this work, we propose BEACON, a novel deep learning framework that leverages large language models (LLMs) to generate dense, contextual embeddings from raw sandbox-generated behavior reports. These embeddings capture semantic and structural patterns of each sample and are processed by a one-dimensional convolutional neural network (1D CNN) for multi-class malware classification. Evaluated on the Avast-CTU Public CAPE Dataset, our framework consistently outperforms existing methods, highlighting the effectiveness of LLM-based behavioral embeddings and the overall design of BEACON for robust malware classification.

Method: Uses physics-informed neural networks (PINNs) to approximate HJB-based value functions at scale. Introduces Value Gradient Iteration (VGI) module to iteratively refine value gradients along trajectories, ensuring value learning aligns with value-gradient learning.

Result: Outperforms existing continuous-time RL baselines on continuous-time variants of multi-agent particle environment (MPE) and multi-agent MuJoCo benchmarks. Scales effectively to complex multi-agent dynamics.

Conclusion: The proposed CT-MARL framework with PINNs and VGI successfully addresses the challenges of continuous-time multi-agent reinforcement learning, demonstrating superior performance and scalability compared to existing approaches.

Abstract: Existing reinforcement learning (RL) methods struggle with complex dynamical systems that demand interactions at high frequencies or irregular time intervals. Continuous-time RL (CTRL) has emerged as a promising alternative by replacing discrete-time Bellman recursion with differential value functions defined as viscosity solutions of the Hamilton–Jacobi–Bellman (HJB) equation. While CTRL has shown promise, its applications have been largely limited to the single-agent domain. This limitation stems from two key challenges: (i) conventional solution methods for HJB equations suffer from the curse of dimensionality (CoD), making them intractable in high-dimensional systems; and (ii) even with HJB-based learning approaches, accurately approximating centralized value functions in multi-agent settings remains difficult, which in turn destabilizes policy training. In this paper, we propose a CT-MARL framework that uses physics-informed neural networks (PINNs) to approximate HJB-based value functions at scale. To ensure the value is consistent with its differential structure, we align value learning with value-gradient learning by introducing a Value Gradient Iteration (VGI) module that iteratively refines value gradients along trajectories. This improves gradient fidelity, in turn yielding more accurate values and stronger policy learning. We evaluate our method using continuous-time variants of standard benchmarks, including multi-agent particle environment (MPE) and multi-agent MuJoCo. Our results demonstrate that our approach consistently outperforms existing continuous-time RL baselines and scales to complex multi-agent dynamics.

Method: Proposes a sweep-line approach that predicts case suffixes with both start and end timestamps for all ongoing cases simultaneously, rather than in isolation, to account for resource dependencies across cases.

Result: Evaluation on real-life and synthetic datasets shows advantages of this multi-model approach for case suffix prediction accuracy.

Conclusion: The proposed technique enables better resource capacity planning by predicting both waiting and processing times through simultaneous prediction of all ongoing case suffixes.

Abstract: Predictive process monitoring techniques support the operational decision making by predicting future states of ongoing cases of a business process. A subset of these techniques predict the remaining sequence of activities of an ongoing case (case suffix prediction). Existing approaches for case suffix prediction generate sequences of activities with a single timestamp (e.g. the end timestamp). This output is insufficient for resource capacity planning, where we need to reason about the periods of time when resources will be busy performing work. This paper introduces a technique for predicting case suffixes consisting of activities with start and end timestamps. In other words, the proposed technique predicts both the waiting time and the processing time of each activity. Since the waiting time of an activity in a case depends on how busy resources are in other cases, the technique adopts a sweep-line approach, wherein the suffixes of all ongoing cases in the process are predicted in lockstep, rather than predictions being made for each case in isolation. An evaluation on real-life and synthetic datasets compares the accuracy of different instantiations of this approach, demonstrating the advantages of a multi-model approach to case suffix prediction.

Method: Builds on momentum-based variance reduction and randomized Singular Value Decomposition (SVD) to create a memory-efficient optimizer.

Result: LiMuon achieves lower memory usage than current Muon variants and O(ε⁻³) sample complexity for finding ε-stationary solutions in non-convex stochastic optimization.

Conclusion: LiMuon provides an efficient optimization solution for large models, verified through experiments on DistilGPT2 and ViT models, with theoretical guarantees under relaxed smoothness conditions.

Abstract: Large models recently are widely applied in artificial intelligence, so efficient training of large models has received widespread attention. More recently, a useful Muon optimizer is specifically designed for matrix-structured parameters of large models. Although some works have begun to studying Muon optimizer, the existing Muon and its variants still suffer from high sample complexity or high memory for large models. To fill this gap, we propose a light and fast Muon (LiMuon) optimizer for training large models, which builds on the momentum-based variance reduced technique and randomized Singular Value Decomposition (SVD). Our LiMuon optimizer has a lower memory than the current Muon and its variants. Moreover, we prove that our LiMuon has a lower sample complexity of $O(\epsilon^{-3})$ for finding an $\epsilon$-stationary solution of non-convex stochastic optimization under the smooth condition. Recently, the existing convergence analysis of Muon optimizer mainly relies on the strict Lipschitz smooth assumption, while some artificial intelligence tasks such as training large language models (LLMs) do not satisfy this condition. We also proved that our LiMuon optimizer has a sample complexity of $O(\epsilon^{-3})$ under the generalized smooth condition. Numerical experimental results on training DistilGPT2 and ViT models verify efficiency of our LiMuon optimizer.

Method: Multi-level pairwise learning loss for scenario encoder, retrieval mechanism for relevant data supplementation, and augmentation-adaptive mechanism for targeted data augmentation in extreme conditions.

Result: Significantly improves predictive accuracy and robustness in data-scarce and atypical scenarios for water temperature and dissolved oxygen modeling in real-world lakes.

Conclusion: A^2SL provides a broadly applicable solution for various scientific domains beyond freshwater ecosystems, addressing data scarcity and generalization challenges.

Abstract: The discovery of environmental knowledge depends on labeled task-specific data, but is often constrained by the high cost of data collection. Existing machine learning approaches usually struggle to generalize in data-sparse or atypical conditions. To this end, we propose an Augmentation-Adaptive Self-Supervised Learning (A$^2$SL) framework, which retrieves relevant observational samples to enhance modeling of the target ecosystem. Specifically, we introduce a multi-level pairwise learning loss to train a scenario encoder that captures varying degrees of similarity among scenarios. These learned similarities drive a retrieval mechanism that supplements a target scenario with relevant data from different locations or time periods. Furthermore, to better handle variable scenarios, particularly under atypical or extreme conditions where traditional models struggle, we design an augmentation-adaptive mechanism that selectively enhances these scenarios through targeted data augmentation. Using freshwater ecosystems as a case study, we evaluate A$^2$SL in modeling water temperature and dissolved oxygen dynamics in real-world lakes. Experimental results show that A$^2$SL significantly improves predictive accuracy and enhances robustness in data-scarce and atypical scenarios. Although this study focuses on freshwater ecosystems, the A$^2$SL framework offers a broadly applicable solution in various scientific domains.

Method: Leverages marginal distribution loss function from evidential deep learning to estimate output uncertainty and infers unknown PDE parameters through learned posterior distributions. Validated on 1D Poisson equation with Gaussian source and 2D Fisher-KPP equation.

Result: E-PINN generated empirical coverage probabilities that were calibrated significantly better than Bayesian PINN and Deep Ensemble methods. Demonstrated real-world applicability on clinical glucose-insulin datasets for diabetes research.

Conclusion: E-PINN provides a superior uncertainty quantification framework for PINNs with better calibration performance and practical applicability to real-world problems like medical data analysis.

Abstract: We present the fundamental theory and implementation guidelines underlying Evidential Physics-Informed Neural Network (E-PINN) – a novel class of uncertainty-aware PINN. It leverages the marginal distribution loss function of evidential deep learning for estimating uncertainty of outputs, and infers unknown parameters of the PDE via a learned posterior distribution. Validating our model on two illustrative case studies – the 1D Poisson equation with a Gaussian source and the 2D Fisher-KPP equation, we found that E-PINN generated empirical coverage probabilities that were calibrated significantly better than Bayesian PINN and Deep Ensemble methods. To demonstrate real-world applicability, we also present a brief case study on applying E-PINN to analyze clinical glucose-insulin datasets that have featured in medical research on diabetes pathophysiology.

Method: Proposes SPMD-LRT that operates directly on tensor representations, incorporates first-order and second-order margin statistics, and uses low-rank tensor decomposition (CP and Tucker) with alternating optimization algorithm.

Result: Superior classification accuracy compared to SVM, vector-based LMDM, and tensor-based SVM extensions on datasets including MNIST, images, and fMRI data. Tucker decomposition version achieved highest accuracy.

Conclusion: SPMD-LRT effectively handles high-dimensional tensor data while preserving structural information and optimizing margin distribution, demonstrating robustness and effectiveness for tensor classification tasks.

Abstract: The Large Margin Distribution Machine (LMDM) is a recent advancement in classifier design that optimizes not just the minimum margin (as in SVM) but the entire margin distribution, thereby improving generalization. However, existing LMDM formulations are limited to vectorized inputs and struggle with high-dimensional tensor data due to the need for flattening, which destroys the data’s inherent multi-mode structure and increases computational burden. In this paper, we propose a Structure-Preserving Margin Distribution Learning for High-Order Tensor Data with Low-Rank Decomposition (SPMD-LRT) that operates directly on tensor representations without vectorization. The SPMD-LRT preserves multi-dimensional spatial structure by incorporating first-order and second-order tensor statistics (margin mean and variance) into the objective, and it leverages low-rank tensor decomposition techniques including rank-1(CP), higher-rank CP, and Tucker decomposition to parameterize the weight tensor. An alternating optimization (double-gradient descent) algorithm is developed to efficiently solve the SPMD-LRT, iteratively updating factor matrices and core tensor. This approach enables SPMD-LRT to maintain the structural information of high-order data while optimizing margin distribution for improved classification. Extensive experiments on diverse datasets (including MNIST, images and fMRI neuroimaging) demonstrate that SPMD-LRT achieves superior classification accuracy compared to conventional SVM, vector-based LMDM, and prior tensor-based SVM extensions (Support Tensor Machines and Support Tucker Machines). Notably, SPMD-LRT with Tucker decomposition attains the highest accuracy, highlighting the benefit of structure preservation. These results confirm the effectiveness and robustness of SPMD-LRT in handling high-dimensional tensor data for classification.

Method: Uses sparse Gaussian mixture model Q-functions (S-GMM-QFs) with Hadamard overparametrization for sparsification, employs Riemannian manifold structure for parameter space, and implements online gradient descent on a smooth objective for principled parameter updates.

Result: S-GMM-QFs match performance of dense deep RL methods on standard benchmarks using significantly fewer parameters, and maintain strong performance in low-parameter-count regimes where sparsified DeepRL methods fail to generalize.

Conclusion: The proposed structured online policy-iteration framework with sparse GMM Q-functions provides an effective, parameter-efficient alternative to traditional deep RL methods with better generalization capabilities in resource-constrained settings.

Abstract: This paper introduces a structured and interpretable online policy-iteration framework for reinforcement learning (RL), built around the novel class of sparse Gaussian mixture model Q-functions (S-GMM-QFs). Extending earlier work that trained GMM-QFs offline, the proposed framework develops an online scheme that leverages streaming data to encourage exploration. Model complexity is regulated through sparsification by Hadamard overparametrization, which mitigates overfitting while preserving expressiveness. The parameter space of S-GMM-QFs is naturally endowed with a Riemannian manifold structure, allowing for principled parameter updates via online gradient descent on a smooth objective. Numerical tests show that S-GMM-QFs match the performance of dense deep RL (DeepRL) methods on standard benchmarks while using significantly fewer parameters, and maintain strong performance even in low-parameter-count regimes where sparsified DeepRL methods fail to generalize.

Method: Incorporated a complementary energy matching term into the loss function of the CGSchNet model, systematically varying the weight of the energy loss term, and evaluated the framework on the Chignolin protein system.

Result: Energy matching did not yield statistically significant improvements in accuracy, but revealed distinct tendencies in how models generalize the free energy surface.

Conclusion: The approach suggests future opportunities to enhance coarse-grained modeling through improved energy estimation techniques and multi-modal loss formulations.

Abstract: Molecular dynamics (MD) simulations provide atomistic insight into biomolecular systems but are often limited by high computational costs required to access long timescales. Coarse-grained machine learning models offer a promising avenue for accelerating sampling, yet conventional force matching approaches often fail to capture the full thermodynamic landscape as fitting a model on the gradient may not fit the absolute differences between low-energy conformational states. In this work, we incorporate a complementary energy matching term into the loss function. We evaluate our framework on the Chignolin protein using the CGSchNet model, systematically varying the weight of the energy loss term. While energy matching did not yield statistically significant improvements in accuracy, it revealed distinct tendencies in how models generalize the free energy surface. Our results suggest future opportunities to enhance coarse-grained modeling through improved energy estimation techniques and multi-modal loss formulations.

Method: Uses probabilistic mask training to add structured randomness without explicit noise, personalized mask learning for data heterogeneity, and layer-wise knowledge compensation for system heterogeneity with adaptive model splitting.

Result: Theoretical analysis confirms privacy protection. Experiments show improved accuracy, communication efficiency, and robustness to privacy attacks, with strong performance under data and system heterogeneity across image and wireless sensing tasks.

Conclusion: PM-SFL provides an effective privacy-preserving solution for Split Federated Learning that addresses both privacy risks and heterogeneity challenges while maintaining model utility and performance.

Abstract: Split Federated Learning (SFL) has emerged as an efficient alternative to traditional Federated Learning (FL) by reducing client-side computation through model partitioning. However, exchanging of intermediate activations and model updates introduces significant privacy risks, especially from data reconstruction attacks that recover original inputs from intermediate representations. Existing defenses using noise injection often degrade model performance. To overcome these challenges, we present PM-SFL, a scalable and privacy-preserving SFL framework that incorporates Probabilistic Mask training to add structured randomness without relying on explicit noise. This mitigates data reconstruction risks while maintaining model utility. To address data heterogeneity, PM-SFL employs personalized mask learning that tailors submodel structures to each client’s local data. For system heterogeneity, we introduce a layer-wise knowledge compensation mechanism, enabling clients with varying resources to participate effectively under adaptive model splitting. Theoretical analysis confirms its privacy protection, and experiments on image and wireless sensing tasks demonstrate that PM-SFL consistently improves accuracy, communication efficiency, and robustness to privacy attacks, with particularly strong performance under data and system heterogeneity.

Method: HD3C encodes data into high-dimensional hypervectors, aggregates them into multiple cluster-specific prototypes, and performs classification through similarity search in hyperspace.

Result: On heart sound classification, HD3C achieves 350x better energy efficiency than Bayesian ResNet with less than 1% accuracy difference, while demonstrating exceptional robustness to noise, limited training data, and hardware errors.

Conclusion: HD3C provides a lightweight, energy-efficient classification framework suitable for reliable deployment in real-world low-power medical applications, with both theoretical and empirical validation of its robustness.

Abstract: Energy-efficient medical data classification is essential for modern disease screening, particularly in home and field healthcare where embedded devices are prevalent. While deep learning models achieve state-of-the-art accuracy, their substantial energy consumption and reliance on GPUs limit deployment on such platforms. We present Hyperdimensional Computing with Class-Wise Clustering (HD3C), a lightweight classification framework designed for low-power environments. HD3C encodes data into high-dimensional hypervectors, aggregates them into multiple cluster-specific prototypes, and performs classification through similarity search in hyperspace. We evaluate HD3C across three medical classification tasks; on heart sound classification, HD3C is $350\times$ more energy-efficient than Bayesian ResNet with less than 1% accuracy difference. Moreover, HD3C demonstrates exceptional robustness to noise, limited training data, and hardware error, supported by both theoretical analysis and empirical results, highlighting its potential for reliable deployment in real-world settings. Code is available at https://github.com/jianglanwei/HD3C.

Method: CUFG integrates a gradient corrector guided by forgetting gradients for fine-tuning-based unlearning and a curriculum paradigm that progressively forgets from easy to hard samples.

Result: The approach narrows the gap with the gold-standard Retrain method by enabling more stable and progressive unlearning, improving both effectiveness and reliability across various forgetting scenarios.

Conclusion: CUFG provides a novel framework for stable approximate unlearning, and the concept of curriculum unlearning offers substantial research potential and forward-looking insights for the machine unlearning field.

Abstract: As privacy and security take center stage in AI, machine unlearning, the ability to erase specific knowledge from models, has garnered increasing attention. However, existing methods overly prioritize efficiency and aggressive forgetting, which introduces notable limitations. In particular, radical interventions like gradient ascent, influence functions, and random label noise can destabilize model weights, leading to collapse and reduced reliability. To address this, we propose CUFG (Curriculum Unlearning via Forgetting Gradients), a novel framework that enhances the stability of approximate unlearning through innovations in both forgetting mechanisms and data scheduling strategies. Specifically, CUFG integrates a new gradient corrector guided by forgetting gradients for fine-tuning-based unlearning and a curriculum unlearning paradigm that progressively forgets from easy to hard. These innovations narrow the gap with the gold-standard Retrain method by enabling more stable and progressive unlearning, thereby improving both effectiveness and reliability. Furthermore, we believe that the concept of curriculum unlearning has substantial research potential and offers forward-looking insights for the development of the MU field. Extensive experiments across various forgetting scenarios validate the rationale and effectiveness of our approach and CUFG. Codes are available at https://anonymous.4open.science/r/CUFG-6375.

Method: Dynamic Wavelet Positional Encoding (DyWPE) framework that generates positional embeddings directly from input time series using Discrete Wavelet Transform (DWT).

Result: Comprehensive experiments on ten diverse time series datasets show DyWPE consistently outperforms eight state-of-the-art positional encoding methods, achieving 9.1% average improvement over baseline sinusoidal encoding in biomedical signals while maintaining competitive computational efficiency.

Conclusion: Signal-aware positional encoding using wavelet transforms provides significant performance improvements for time series analysis compared to traditional signal-agnostic approaches.

Abstract: Existing positional encoding methods in transformers are fundamentally signal-agnostic, deriving positional information solely from sequence indices while ignoring the underlying signal characteristics. This limitation is particularly problematic for time series analysis, where signals exhibit complex, non-stationary dynamics across multiple temporal scales. We introduce Dynamic Wavelet Positional Encoding (DyWPE), a novel signal-aware framework that generates positional embeddings directly from input time series using the Discrete Wavelet Transform (DWT). Comprehensive experiments in ten diverse time series datasets demonstrate that DyWPE consistently outperforms eight existing state-of-the-art positional encoding methods, achieving average relative improvements of 9.1% compared to baseline sinusoidal absolute position encoding in biomedical signals, while maintaining competitive computational efficiency.

Method: DeCoP uses Instance-wise Patch Normalization (IPN) to mitigate distributional shifts while preserving patch characteristics, and a hierarchical Dependency Controlled Learning (DCL) strategy with Instance-level Contrastive Module (ICM) to model inter-patch dependencies across multiple temporal scales.

Result: DeCoP achieves state-of-the-art results on ten datasets with lower computing resources, improving MSE by 3% on ETTh1 over PatchTST while using only 37% of the FLOPs.

Conclusion: The proposed framework effectively addresses temporal variability in time series pre-training by explicitly modeling dynamic multi-scale dependencies, leading to improved generalization and computational efficiency.

Abstract: Modeling dynamic temporal dependencies is a critical challenge in time series pre-training, which evolve due to distribution shifts and multi-scale patterns. This temporal variability severely impairs the generalization of pre-trained models to downstream tasks. Existing frameworks fail to capture the complex interactions of short- and long-term dependencies, making them susceptible to spurious correlations that degrade generalization. To address these limitations, we propose DeCoP, a Dependency Controlled Pre-training framework that explicitly models dynamic, multi-scale dependencies by simulating evolving inter-patch dependencies. At the input level, DeCoP introduces Instance-wise Patch Normalization (IPN) to mitigate distributional shifts while preserving the unique characteristics of each patch, creating a robust foundation for representation learning. At the latent level, a hierarchical Dependency Controlled Learning (DCL) strategy explicitly models inter-patch dependencies across multiple temporal scales, with an Instance-level Contrastive Module (ICM) enhances global generalization by learning instance-discriminative representations from time-invariant positive pairs. DeCoP achieves state-of-the-art results on ten datasets with lower computing resources, improving MSE by 3% on ETTh1 over PatchTST using only 37% of the FLOPs.

Method: Proposed learned nonnegative clocks for source and target sequences, modeling attention as meeting probability of these clocks. Derived closed-form Gaussian-like scoring rule with intrinsic bias toward causal, smooth, near-diagonal alignments. Supports both normalized clocks for parallel decoding and unnormalized clocks for autoregressive decoding.

Result: In Transformer text-to-speech testbed, produces more stable alignments and improved robustness to global time-scaling while matching or improving accuracy over scaled dot-product baselines.

Conclusion: The clock-based attention framework provides effective alignment modeling for continuous sequences with potential applications to other domains like video and temporal signal modeling.

Abstract: We formulate an attention mechanism for continuous and ordered sequences that explicitly functions as an alignment model, which serves as the core of many sequence-to-sequence tasks. Standard scaled dot-product attention relies on positional encodings and masks but does not enforce continuity or monotonicity, which are crucial for frame-synchronous targets. We propose learned nonnegative \emph{clocks} to source and target and model attention as the meeting probability of these clocks; a path-integral derivation yields a closed-form, Gaussian-like scoring rule with an intrinsic bias toward causal, smooth, near-diagonal alignments, without external positional regularizers. The framework supports two complementary regimes: normalized clocks for parallel decoding when a global length is available, and unnormalized clocks for autoregressive decoding – both nearly-parameter-free, drop-in replacements. In a Transformer text-to-speech testbed, this construction produces more stable alignments and improved robustness to global time-scaling while matching or improving accuracy over scaled dot-product baselines. We hypothesize applicability to other continuous targets, including video and temporal signal modeling.

Method: DSCL combines Reward-Based Dynamic Sampling (using multi-dimensional reward mean/variance) and Task-Based Dynamic Curriculum Learning to prioritize valuable data and focus on less-mastered sub-tasks.

Result: Achieves 3.29% improvement on BFCLv3 benchmark, significantly enhancing training efficiency and model performance over strong baselines.

Conclusion: DSCL provides a tailored solution that effectively leverages complex reward signals and sub-task dynamics in tool learning for superior results.

Abstract: While reinforcement learning (RL) is increasingly used for LLM-based tool learning, its efficiency is often hampered by an overabundance of simple samples that provide diminishing learning value as training progresses. Existing dynamic sampling techniques are ill-suited for the multi-task structure and fine-grained reward mechanisms inherent to tool learning. This paper introduces Dynamic Sampling with Curriculum Learning (DSCL), a framework specifically designed to address this challenge by targeting the unique characteristics of tool learning: its multiple interdependent sub-tasks and multi-valued reward functions. DSCL features two core components: Reward-Based Dynamic Sampling, which uses multi-dimensional reward statistics (mean and variance) to prioritize valuable data, and Task-Based Dynamic Curriculum Learning, which adaptively focuses training on less-mastered sub-tasks. Through extensive experiments, we demonstrate that DSCL significantly improves training efficiency and model performance over strong baselines, achieving a 3.29% improvement on the BFCLv3 benchmark. Our method provides a tailored solution that effectively leverages the complex reward signals and sub-task dynamics within tool learning to achieve superior results.

Method: Proposes PreGC with hybrid-interval graph diffusion augmentation to enhance generalization, optimal transport matching to maintain semantic consistency, and a traceable semantic harmonizer to bridge semantic associations between source and condensed graphs.

Result: Extensive experiments show PreGC achieves superior performance and versatility, demonstrating task independence and seamless compatibility with arbitrary GNN architectures.

Conclusion: PreGC successfully transcends the limitations of traditional GC methods by providing a generalized approach that works across various tasks and GNN architectures through optimal transport and semantic consistency preservation.

Abstract: Graph condensation (GC) aims to distill the original graph into a small-scale graph, mitigating redundancy and accelerating GNN training. However, conventional GC approaches heavily rely on rigid GNNs and task-specific supervision. Such a dependency severely restricts their reusability and generalization across various tasks and architectures. In this work, we revisit the goal of ideal GC from the perspective of GNN optimization consistency, and then a generalized GC optimization objective is derived, by which those traditional GC methods can be viewed nicely as special cases of this optimization paradigm. Based on this, Pre-trained Graph Condensation (PreGC) via optimal transport is proposed to transcend the limitations of task- and architecture-dependent GC methods. Specifically, a hybrid-interval graph diffusion augmentation is presented to suppress the weak generalization ability of the condensed graph on particular architectures by enhancing the uncertainty of node states. Meanwhile, the matching between optimal graph transport plan and representation transport plan is tactfully established to maintain semantic consistencies across source graph and condensed graph spaces, thereby freeing graph condensation from task dependencies. To further facilitate the adaptation of condensed graphs to various downstream tasks, a traceable semantic harmonizer from source nodes to condensed nodes is proposed to bridge semantic associations through the optimized representation transport plan in pre-training. Extensive experiments verify the superiority and versatility of PreGC, demonstrating its task-independent nature and seamless compatibility with arbitrary GNNs.

Method: Train a transcoder on the cell2sentence (C2S) model to extract internal decision-making circuits from this state-of-the-art single-cell foundation model.

Result: The extracted circuits correspond to real-world biological mechanisms, demonstrating transcoders can uncover biologically plausible pathways within complex single-cell models.

Conclusion: Transcoders show promising potential for improving interpretability of single-cell foundation models by revealing their internal biological decision-making processes.

Abstract: Single-cell foundation models (scFMs) have demonstrated state-of-the-art performance on various tasks, such as cell-type annotation and perturbation response prediction, by learning gene regulatory networks from large-scale transcriptome data. However, a significant challenge remains: the decision-making processes of these models are less interpretable compared to traditional methods like differential gene expression analysis. Recently, transcoders have emerged as a promising approach for extracting interpretable decision circuits from large language models (LLMs). In this work, we train a transcoder on the cell2sentence (C2S) model, a state-of-the-art scFM. By leveraging the trained transcoder, we extract internal decision-making circuits from the C2S model. We demonstrate that the discovered circuits correspond to real-world biological mechanisms, confirming the potential of transcoders to uncover biologically plausible pathways within complex single-cell models.

Method: OMCAL uses a nuclear norm-based adaptive consensus anchor graph learning model to handle information redundancy and noise interference. It integrates category indicator acquisition and consensus anchor graph learning into a single unified framework to improve both clustering effectiveness and efficiency.

Result: Extensive experiments on both ordinary and large-scale datasets show that OMCAL outperforms existing state-of-the-art methods in terms of both clustering effectiveness and efficiency.

Conclusion: The proposed OMCAL method successfully addresses the limitations of existing anchor graph-based multi-view clustering approaches by providing a unified framework that handles redundancy and noise while simultaneously learning consensus anchor graphs and clustering indicators, resulting in superior performance.

Abstract: In light of their capability to capture structural information while reducing computing complexity, anchor graph-based multi-view clustering (AGMC) methods have attracted considerable attention in large-scale clustering problems. Nevertheless, existing AGMC methods still face the following two issues: 1) They directly embedded diverse anchor graphs into a consensus anchor graph (CAG), and hence ignore redundant information and numerous noises contained in these anchor graphs, leading to a decrease in clustering effectiveness; 2) They drop effectiveness and efficiency due to independent post-processing to acquire clustering indicators. To overcome the aforementioned issues, we deliver a novel one-step multi-view clustering method with adaptive low-rank anchor-graph learning (OMCAL). To construct a high-quality CAG, OMCAL provides a nuclear norm-based adaptive CAG learning model against information redundancy and noise interference. Then, to boost clustering effectiveness and efficiency substantially, we incorporate category indicator acquisition and CAG learning into a unified framework. Numerous studies conducted on ordinary and large-scale datasets indicate that OMCAL outperforms existing state-of-the-art methods in terms of clustering effectiveness and efficiency.

Method: Proposes FlowCast-ODE framework that models atmospheric state evolution as continuous flow using dynamic flow matching and ODE solvers, with coarse-to-fine training strategy and lightweight low-rank AdaLN-Zero modulation.

Result: Outperforms baselines with lower RMSE, better energy conservation, reduced blurring, preserved spatial details, comparable extreme event forecasting, and alleviated temporal discontinuities.

Conclusion: FlowCast-ODE provides an effective solution for stable and accurate hourly weather forecasting by modeling atmospheric dynamics as continuous flow, achieving improved performance while reducing model size by 15%.

Abstract: Accurate hourly weather forecasting is critical for numerous applications. Recent deep learning models have demonstrated strong capability on 6-hour intervals, yet achieving accurate and stable hourly predictions remains a critical challenge. This is primarily due to the rapid accumulation of errors in autoregressive rollouts and temporal discontinuities within the ERA5 data’s 12-hour assimilation cycle. To address these issues, we propose FlowCast-ODE, a framework that models atmospheric state evolution as a continuous flow. FlowCast-ODE learns the conditional flow path directly from the previous state, an approach that aligns more naturally with physical dynamic systems and enables efficient computation. A coarse-to-fine strategy is introduced to train the model on 6-hour data using dynamic flow matching and then refined on hourly data that incorporates an Ordinary Differential Equation (ODE) solver to achieve temporally coherent forecasts. In addition, a lightweight low-rank AdaLN-Zero modulation mechanism is proposed and reduces model size by 15% without compromising accuracy. Experiments demonstrate that FlowCast-ODE outperforms strong baselines, yielding lower root mean square error (RMSE) and better energy conservation, which reduces blurring and preserves more fine-scale spatial details. It also shows comparable performance to the state-of-the-art model in forecasting extreme events like typhoons. Furthermore, the model alleviates temporal discontinuities associated with assimilation cycle transitions.

Method: The authors systematically explore data-constrained approaches including increasing epoch count and parameter scaling, then significantly improve performance by tuning regularization (30x larger weight decay), implementing ensemble methods, and using distillation to compress ensemble benefits into smaller models.

Result: The best approach combining epoching, regularization, parameter scaling, and ensemble scaling achieves an asymptote at 200M tokens using 5.17x less data than baseline. Distillation retains 83% of ensemble benefits in models 8x smaller. Downstream benchmarks show 9% improvement on pre-training evals and 17.5x data efficiency improvement on math tasks.

Conclusion: Simple algorithmic improvements enable significantly more data-efficient pre-training in compute-rich scenarios, with the optimal weight decay being much larger than standard practice and ensembling achieving better asymptotes than individual models.

Abstract: Since compute grows much faster than web text available for language model pre-training, we ask how one should approach pre-training under fixed data and no compute constraints. We first show that existing data-constrained approaches of increasing epoch count and parameter count eventually overfit, and we significantly improve upon such recipes by properly tuning regularization, finding that the optimal weight decay is $30\times$ larger than standard practice. Since our regularized recipe monotonically decreases loss following a simple power law in parameter count, we estimate its best possible performance via the asymptote of its scaling law rather than the performance at a fixed compute budget. We then identify that ensembling independently trained models achieves a significantly lower loss asymptote than the regularized recipe. Our best intervention combining epoching, regularization, parameter scaling, and ensemble scaling achieves an asymptote at 200M tokens using $5.17\times$ less data than our baseline, and our data scaling laws predict that this improvement persists at higher token budgets. We find that our data efficiency gains can be realized at much smaller parameter counts as we can distill an ensemble into a student model that is 8$\times$ smaller and retains $83%$ of the ensembling benefit. Finally, our interventions designed for validation loss generalize to downstream benchmarks, achieving a $9%$ improvement for pre-training evals and a $17.5\times$ data efficiency improvement over continued pre-training on math mid-training data. Our results show that simple algorithmic improvements can enable significantly more data-efficient pre-training in a compute-rich future.

Method: Introduces a sequence-based DTI framework that integrates structural priors into protein representations, using learned aggregation, bilinear attention, and contrastive alignment techniques.

Result: Achieves state-of-the-art performance on Human and BioSNAP datasets, remains competitive on BindingDB, and surpasses prior methods on LIT-PCBA virtual screening with substantial gains in AUROC and BEDROC metrics.

Conclusion: The framework validates utility for scalable and structure-aware DTI prediction, with embedding visualizations showing improved spatial correspondence with binding pockets and interpretable attention patterns.

Abstract: Accurate identification of drug-target interactions (DTI) remains a central challenge in computational pharmacology, where sequence-based methods offer scalability. This work introduces a sequence-based drug-target interaction framework that integrates structural priors into protein representations while maintaining high-throughput screening capability. Evaluated across multiple benchmarks, the model achieves state-of-the-art performance on Human and BioSNAP datasets and remains competitive on BindingDB. In virtual screening tasks, it surpasses prior methods on LIT-PCBA, yielding substantial gains in AUROC and BEDROC. Ablation studies confirm the critical role of learned aggregation, bilinear attention, and contrastive alignment in enhancing predictive robustness. Embedding visualizations reveal improved spatial correspondence with known binding pockets and highlight interpretable attention patterns over ligand-residue contacts. These results validate the framework’s utility for scalable and structure-aware DTI prediction.

Method: The authors introduce STEP framework with unified interface for multiple datasets, consistent training/evaluation conditions, and broad model support. They conduct experiments to test framework capabilities.

Result: Experiments reveal limitations of current testing procedures, importance of joint agent modeling for interaction predictions, and vulnerability of state-of-the-art models to distribution shifts and adversarial attacks.

Conclusion: STEP aims to shift focus from leaderboard rankings to deeper insights about model behavior and generalization in complex multi-agent environments.

Abstract: While trajectory prediction plays a critical role in enabling safe and effective path-planning in automated vehicles, standardized practices for evaluating such models remain underdeveloped. Recent efforts have aimed to unify dataset formats and model interfaces for easier comparisons, yet existing frameworks often fall short in supporting heterogeneous traffic scenarios, joint prediction models, or user documentation. In this work, we introduce STEP – a new benchmarking framework that addresses these limitations by providing a unified interface for multiple datasets, enforcing consistent training and evaluation conditions, and supporting a wide range of prediction models. We demonstrate the capabilities of STEP in a number of experiments which reveal 1) the limitations of widely-used testing procedures, 2) the importance of joint modeling of agents for better predictions of interactions, and 3) the vulnerability of current state-of-the-art models against both distribution shifts and targeted attacks by adversarial agents. With STEP, we aim to shift the focus from the ``leaderboard’’ approach to deeper insights about model behavior and generalization in complex multi-agent settings.

Method: Formulate joint optimization problem for network parameters and precision matrix, solved via alternating optimization with sequential updates of weights and precision estimates.

Result: Theoretical bounds on precision matrix estimation error, and empirical effectiveness demonstrated on synthetic and real-world data compared to two-step approaches.

Conclusion: PNNs provide improved statistical independence encoding, sparsity, and task-aware learning while preserving covariance spectral structure.

Abstract: CoVariance Neural Networks (VNNs) perform convolutions on the graph determined by the covariance matrix of the data, which enables expressive and stable covariance-based learning. However, covariance matrices are typically dense, fail to encode conditional independence, and are often precomputed in a task-agnostic way, which may hinder performance. To overcome these limitations, we study Precision Neural Networks (PNNs), i.e., VNNs on the precision matrix – the inverse covariance. The precision matrix naturally encodes statistical independence, often exhibits sparsity, and preserves the covariance spectral structure. To make precision estimation task-aware, we formulate an optimization problem that jointly learns the network parameters and the precision matrix, and solve it via alternating optimization, by sequentially updating the network weights and the precision estimate. We theoretically bound the distance between the estimated and true precision matrices at each iteration, and demonstrate the effectiveness of joint estimation compared to two-step approaches on synthetic and real-world data.

Method: Proposes Diffusion Scenario Tree (DST) framework that uses diffusion-based probabilistic forecasting models to recursively sample future trajectories and organize them into trees via clustering while ensuring non-anticipativity.

Result: DST consistently outperforms conventional scenario tree models and Model-Free Reinforcement Learning baselines in energy arbitrage optimization, achieving higher performance by better handling uncertainty.

Conclusion: The diffusion-based scenario tree framework enables more efficient decision policies in stochastic optimization problems, particularly in energy market applications, by effectively capturing and managing uncertainty.

Abstract: Stochastic forecasting is critical for efficient decision-making in uncertain systems, such as energy markets and finance, where estimating the full distribution of future scenarios is essential. We propose Diffusion Scenario Tree (DST), a general framework for constructing scenario trees for multivariate prediction tasks using diffusion-based probabilistic forecasting models. DST recursively samples future trajectories and organizes them into a tree via clustering, ensuring non-anticipativity (decisions depending only on observed history) at each stage. We evaluate the framework on the optimization task of energy arbitrage in New York State’s day-ahead electricity market. Experimental results show that our approach consistently outperforms the same optimization algorithms that use scenario trees from more conventional models and Model-Free Reinforcement Learning baselines. Furthermore, using DST for stochastic optimization yields more efficient decision policies, achieving higher performance by better handling uncertainty than deterministic and stochastic MPC variants using the same diffusion-based forecaster.

Method: Proposes MF-HRL-IGM algorithm that combines offline data with online interactions across multiple simulators. Uses information gain maximization through bootstrapping to select the most informative fidelity level at each step, optimizing policy under fixed cost budget.

Result: Theoretical analysis shows the algorithm has no-regret property. Empirical evaluations demonstrate superior performance compared to existing benchmarks, showing effective use of multi-fidelity simulators.

Conclusion: MF-HRL-IGM successfully addresses policy optimization in multi-fidelity environments by intelligently selecting simulator fidelity based on information gain, providing both theoretical guarantees and practical performance improvements over existing methods.

Abstract: Optimizing a reinforcement learning (RL) policy typically requires extensive interactions with a high-fidelity simulator of the environment, which are often costly or impractical. Offline RL addresses this problem by allowing training from pre-collected data, but its effectiveness is strongly constrained by the size and quality of the dataset. Hybrid offline-online RL leverages both offline data and interactions with a single simulator of the environment. In many real-world scenarios, however, multiple simulators with varying levels of fidelity and computational cost are available. In this work, we study multi-fidelity hybrid RL for policy optimization under a fixed cost budget. We introduce multi-fidelity hybrid RL via information gain maximization (MF-HRL-IGM), a hybrid offline-online RL algorithm that implements fidelity selection based on information gain maximization through a bootstrapping approach. Theoretical analysis establishes the no-regret property of MF-HRL-IGM, while empirical evaluations demonstrate its superior performance compared to existing benchmarks.

Method: Proposes global-to-local attention scheme: shallow layers use attention for global information, deeper layers use GNN modules for local structure. Includes cross-layer information fusion strategy to retain beneficial information from global layers with acceptable scalability trade-offs.

Result: G2LFormer shows excellent performance on both node-level and graph-level tasks compared to state-of-the-art linear GTs and GNNs, while maintaining linear complexity.

Conclusion: The global-to-local attention scheme is feasible and effective, providing superior performance by preventing information loss and maintaining the benefits of both global attention and local structural learning.

Abstract: Graph Transformers (GTs) show considerable potential in graph representation learning. The architecture of GTs typically integrates Graph Neural Networks (GNNs) with global attention mechanisms either in parallel or as a precursor to attention mechanisms, yielding a local-and-global or local-to-global attention scheme. However, as the global attention mechanism primarily captures long-range dependencies between nodes, these integration schemes may suffer from information loss, where the local neighborhood information learned by GNN could be diluted by the attention mechanism. Therefore, we propose G2LFormer, featuring a novel global-to-local attention scheme where the shallow network layers use attention mechanisms to capture global information, while the deeper layers employ GNN modules to learn local structural information, thereby preventing nodes from ignoring their immediate neighbors. An effective cross-layer information fusion strategy is introduced to allow local layers to retain beneficial information from global layers and alleviate information loss, with acceptable trade-offs in scalability. To validate the feasibility of the global-to-local attention scheme, we compare G2LFormer with state-of-the-art linear GTs and GNNs on node-level and graph-level tasks. The results indicate that G2LFormer exhibits excellent performance while keeping linear complexity.

Method: Conducted ablation studies comparing the full DPANet model against specialized variants: Temporal-Only model (fusing temporal pyramids), Frequency-Only model (fusing spectral pyramids), and a version without cross-attention fusion

Result: Full model consistently outperformed all variants. Both single-domain variants underperformed significantly, confirming the need for heterogeneous temporal-frequency fusion. Removing cross-attention caused the most severe performance degradation

Conclusion: The interactive cross-attention fusion block is the most essential component, and the dual-domain approach combining temporal and frequency information is critical for DPANet’s success

Abstract: We conducted rigorous ablation studies to validate DPANet’s key components (Table \ref{tab:ablation-study}). The full model consistently outperforms all variants. To test our dual-domain hypothesis, we designed two specialized versions: a Temporal-Only model (fusing two identical temporal pyramids) and a Frequency-Only model (fusing two spectral pyramids). Both variants underperformed significantly, confirming that the fusion of heterogeneous temporal and frequency information is critical. Furthermore, replacing the cross-attention mechanism with a simpler method (w/o Cross-Fusion) caused the most severe performance degradation. This result underscores that our interactive fusion block is the most essential component.

Method: Extend restricted isometry property (RIP) to Dirichlet energy function used in graph learning, specifically analyzing GL-SigRep method on partial observations.

Result: Theoretical proof that smoothness-based graph learning can recover ground truth topology from observed nodes despite hidden nodes, supported by synthetic and real data experiments.

Conclusion: Vanilla graph topology learning methods demonstrate inherent robustness to partial observations when dealing with low-pass filtered graph signals.

Abstract: Learning the graph underlying a networked system from nodal signals is crucial to downstream tasks in graph signal processing and machine learning. The presence of hidden nodes whose signals are not observable might corrupt the estimated graph. While existing works proposed various robustifications of vanilla graph learning objectives by explicitly accounting for the presence of these hidden nodes, a robustness analysis of “naive”, hidden-node agnostic approaches is still underexplored. This work demonstrates that vanilla graph topology learning methods are implicitly robust to partial observations of low-pass filtered graph signals. We achieve this theoretical result through extending the restricted isometry property (RIP) to the Dirichlet energy function used in graph learning objectives. We show that smoothness-based graph learning formulation (e.g., the GL-SigRep method) on partial observations can recover the ground truth graph topology corresponding to the observed nodes. Synthetic and real data experiments corroborate our findings.

Method: Combines Reinforcement Learning with Genetic Algorithms, using GAs to explore trajectory space and identify successful paths, plus transformer architecture for decay sequence processing.

Result: Developed a systematic framework that can identify relevant background processes more efficiently than traditional manual approaches.

Conclusion: The RL-GA hybrid approach provides a systematic solution for background identification in particle physics, with broader applicability beyond beauty hadron decays.

Abstract: Experimental studies of beauty hadron decays face significant challenges due to a wide range of backgrounds arising from the numerous possible decay channels with similar final states. For a particular signal decay, the process for ascertaining the most relevant background processes necessitates a detailed analysis of final state particles, potential misidentifications, and kinematic overlaps, which, due to computational limitations, is restricted to the simulation of only the most relevant backgrounds. Moreover, this process typically relies on the physicist’s intuition and expertise, as no systematic method exists. This paper has two primary goals. First, from a particle physics perspective, we present a novel approach that utilises Reinforcement Learning (RL) to overcome the aforementioned challenges by systematically determining the critical backgrounds affecting beauty hadron decay measurements. While beauty hadron physics serves as the case study in this work, the proposed strategy is broadly adaptable to other types of particle physics measurements. Second, from a Machine Learning perspective, we introduce a novel algorithm which exploits the synergy between RL and Genetic Algorithms (GAs) for environments with highly sparse rewards and a large trajectory space. This strategy leverages GAs to efficiently explore the trajectory space and identify successful trajectories, which are used to guide the RL agent’s training. Our method also incorporates a transformer architecture for the RL agent to handle token sequences representing decays.

Method: Adapted an alternative fixed-point method to compute barycenter diagrams for generic transportation costs (q>1), particularly focusing on robust q∈(1,2) cases that are less sensitive to outliers.

Result: The approach successfully computes robust barycenters for persistence diagrams with q>1, demonstrating improved outlier robustness in both clustering and dictionary encoding applications.

Conclusion: The proposed method extends barycenter computation beyond q=2, providing a more robust framework for persistence diagram analysis with practical applications in clustering and encoding tasks.

Abstract: This short paper presents a general approach for computing robust Wasserstein barycenters of persistence diagrams. The classical method consists in computing assignment arithmetic means after finding the optimal transport plans between the barycenter and the persistence diagrams. However, this procedure only works for the transportation cost related to the $q$-Wasserstein distance $W_q$ when $q=2$. We adapt an alternative fixed-point method to compute a barycenter diagram for generic transportation costs ($q > 1$), in particular those robust to outliers, $q \in (1,2)$. We show the utility of our work in two applications: \emph{(i)} the clustering of persistence diagrams on their metric space and \emph{(ii)} the dictionary encoding of persistence diagrams. In both scenarios, we demonstrate the added robustness to outliers provided by our generalized framework. Our Python implementation is available at this address: https://github.com/Keanu-Sisouk/RobustBarycenter .

Method: Uses Self-Explaining Neural Networks (SENNs) with explanation extraction methods to enhance interpretability while preserving predictive accuracy. Focuses on low-dimensionality problems to generate robust local and global explanations.

Result: Demonstrated competitive performance on mobile network resource allocation problems. SENNs provided interpretable solutions with performance on par with state-of-the-art methods while offering robust explanations.

Conclusion: SENNs show strong potential to improve transparency and trust in AI-driven decision-making for low-dimensional tasks, offering both interpretability and competitive performance.

Abstract: Reinforcement Learning (RL) methods that incorporate deep neural networks (DNN), though powerful, often lack transparency. Their black-box characteristic hinders interpretability and reduces trustworthiness, particularly in critical domains. To address this challenge in RL tasks, we propose a solution based on Self-Explaining Neural Networks (SENNs) along with explanation extraction methods to enhance interpretability while maintaining predictive accuracy. Our approach targets low-dimensionality problems to generate robust local and global explanations of the model’s behaviour. We evaluate the proposed method on the resource allocation problem in mobile networks, demonstrating that SENNs can constitute interpretable solutions with competitive performance. This work highlights the potential of SENNs to improve transparency and trust in AI-driven decision-making for low-dimensional tasks. Our approach strong performance on par with the existing state-of-the-art methods, while providing robust explanations.

Method: Proposes DAG framework with two modules: 1) Temporal Causal Module - causal discovery of how historical exogenous variables affect future exogenous variables, and causal injection into forecasting; 2) Channel Causal Module - causal discovery of how historical exogenous variables influence historical endogenous variables, and causal injection using future exogenous variables.

Result: The framework enables better utilization of exogenous variables, particularly future exogenous variables, by capturing causal relationships in both temporal and channel dimensions.

Conclusion: DAG provides a general framework that addresses limitations of existing methods by incorporating causal relationships to improve time series forecasting accuracy with exogenous variables.

Abstract: Time series forecasting is crucial in various fields such as economics, traffic, and AIOps. However, in real-world applications, focusing solely on the endogenous variables (i.e., target variables), is often insufficient to ensure accurate predictions. Considering exogenous variables (i.e., covariates) provides additional predictive information, thereby improving forecasting accuracy. However, existing methods for time series forecasting with exogenous variables (TSF-X) have the following shortcomings: 1) they do not leverage future exogenous variables, 2) they fail to account for the causal relationships between endogenous and exogenous variables. As a result, their performance is suboptimal. In this study, to better leverage exogenous variables, especially future exogenous variable, we propose a general framework DAG, which utilizes dual causal network along both the temporal and channel dimensions for time series forecasting with exogenous variables. Specifically, we first introduce the Temporal Causal Module, which includes a causal discovery module to capture how historical exogenous variables affect future exogenous variables. Following this, we construct a causal injection module that incorporates the discovered causal relationships into the process of forecasting future endogenous variables based on historical endogenous variables. Next, we propose the Channel Causal Module, which follows a similar design principle. It features a causal discovery module models how historical exogenous variables influence historical endogenous variables, and a causal injection module incorporates the discovered relationships to enhance the prediction of future endogenous variables based on future exogenous variables.

Method: Developed evaluation pipelines to compare bot detection performance of encoder-based and decoder-based transformer classifiers.

Result: Encoder-based classifiers demonstrated greater accuracy and robustness, while decoder-based models showed better adaptability through task-specific alignment.

Conclusion: Both encoder and decoder approaches have strengths for bot detection, contributing to efforts against digital manipulation while protecting online discussion integrity.

Abstract: Social media has become a key medium of communication in today’s society. This realisation has led to many parties employing artificial users (or bots) to mislead others into believing untruths or acting in a beneficial manner to such parties. Sophisticated text generation tools, such as large language models, have further exacerbated this issue. This paper aims to compare the effectiveness of bot detection models based on encoder and decoder transformers. Pipelines are developed to evaluate the performance of these classifiers, revealing that encoder-based classifiers demonstrate greater accuracy and robustness. However, decoder-based models showed greater adaptability through task-specific alignment, suggesting more potential for generalisation across different use cases in addition to superior observa. These findings contribute to the ongoing effort to prevent digital environments being manipulated while protecting the integrity of online discussion.

Method: Developed HFL-SNM framework with concepts of Effective Data Coverage Rate and Redundant Data Coverage Rate. Formulated joint optimization problem for resource allocation and client scheduling, then decoupled it into sub-problems solved by DO-SNM algorithm.

Result: Experimental results show the proposed algorithm achieves superior model performance while significantly reducing energy consumption compared to traditional baseline algorithms.

Conclusion: The HFL-SNM framework successfully integrates social network mobility patterns into federated learning, demonstrating that considering client mobility and social data sharing can optimize resource usage and energy efficiency without compromising model quality.

Abstract: Federated Learning (FL) offers a decentralized solution that allows collaborative local model training and global aggregation, thereby protecting data privacy. In conventional FL frameworks, data privacy is typically preserved under the assumption that local data remains absolutely private, whereas the mobility of clients is frequently neglected in explicit modeling. In this paper, we propose a hierarchical federated learning framework based on the social network with mobility namely HFL-SNM that considers both data sharing among clients and their mobility patterns. Under the constraints of limited resources, we formulate a joint optimization problem of resource allocation and client scheduling, which objective is to minimize the energy consumption of clients during the FL process. In social network, we introduce the concepts of Effective Data Coverage Rate and Redundant Data Coverage Rate. We analyze the impact of effective data and redundant data on the model performance through preliminary experiments. We decouple the optimization problem into multiple sub-problems, analyze them based on preliminary experimental results, and propose Dynamic Optimization in Social Network with Mobility (DO-SNM) algorithm. Experimental results demonstrate that our algorithm achieves superior model performance while significantly reducing energy consumption, compared to traditional baseline algorithms.

Method: Used ensemble machine learning techniques (XGBoost, Random Forest, CatBoost, AdaBoost) on Ethiopian Demographic and Health Survey data (2005-2020, 18,108 records, 30 attributes). Data preprocessing included handling missing values, normalization, SMOTE balancing, and feature selection.

Result: Random Forest model achieved best performance: 97.87% accuracy, 97.88% precision, 97.87% recall, 97.87% F1-score, and 99.86% ROC AUC in classifying four nutritional categories (normal, moderate malnutrition, severe malnutrition, overnutrition).

Conclusion: Ensemble learning effectively captures hidden patterns from complex datasets, providing timely insights for early detection of nutritional risks. Results offer practical implications for healthcare providers and policymakers to improve maternal nutrition in Ethiopia.

Abstract: Malnutrition among pregnant women is a major public health challenge in Ethiopia, increasing the risk of adverse maternal and neonatal outcomes. Traditional statistical approaches often fail to capture the complex and multidimensional determinants of nutritional status. This study develops a predictive model using ensemble machine learning techniques, leveraging data from the Ethiopian Demographic and Health Survey (2005-2020), comprising 18,108 records with 30 socio-demographic and health attributes. Data preprocessing included handling missing values, normalization, and balancing with SMOTE, followed by feature selection to identify key predictors. Several supervised ensemble algorithms including XGBoost, Random Forest, CatBoost, and AdaBoost were applied to classify nutritional status. Among them, the Random Forest model achieved the best performance, classifying women into four categories (normal, moderate malnutrition, severe malnutrition, and overnutrition) with 97.87% accuracy, 97.88% precision, 97.87% recall, 97.87% F1-score, and 99.86% ROC AUC. These findings demonstrate the effectiveness of ensemble learning in capturing hidden patterns from complex datasets and provide timely insights for early detection of nutritional risks. The results offer practical implications for healthcare providers, policymakers, and researchers, supporting data-driven strategies to improve maternal nutrition and health outcomes in Ethiopia.

Method: Develops nested-loop normalized stochastic bilevel approximation (N²SBA) algorithm that handles heavy-tailed noise by using normalized gradients and carefully designed update rules.

Result: Achieves SFO complexity of Õ(κ^(7p-3)/(p-1) σ^(p)/(p-1) ε^(-4p-2)/(p-1)) for bilevel optimization and Õ(κ^(2p-1)/(p-1) σ^(p)/(p-1) ε^(-3p-2)/(p-1)) for minimax optimization, matching best-known results when p=2.

Conclusion: The proposed method effectively handles heavy-tailed noise in stochastic bilevel optimization and provides optimal complexity bounds that generalize existing results beyond bounded variance assumptions.

Abstract: This paper considers the smooth bilevel optimization in which the lower-level problem is strongly convex and the upper-level problem is possibly nonconvex. We focus on the stochastic setting that the algorithm can access the unbiased stochastic gradient evaluation with heavy-tailed noise, which is prevalent in many machine learning applications such as training large language models and reinforcement learning. We propose a nested-loop normalized stochastic bilevel approximation (N$^2$SBA) for finding an $\epsilon$-stationary point with the stochastic first-order oracle (SFO) complexity of $\tilde{\mathcal{O}}\big(\kappa^{\frac{7p-3}{p-1}} \sigma^{\frac{p}{p-1}} \epsilon^{-\frac{4 p - 2}{p-1}}\big)$, where $\kappa$ is the condition number, $p\in(1,2]$ is the order of central moment for the noise, and $\sigma$ is the noise level. Furthermore, we specialize our idea to solve the nonconvex-strongly-concave minimax optimization problem, achieving an $\epsilon$-stationary point with the SFO complexity of $\tilde{\mathcal O}\big(\kappa^{\frac{2p-1}{p-1}} \sigma^{\frac{p}{p-1}} \epsilon^{-\frac{3p-2}{p-1}}\big)$. All above upper bounds match the best-known results under the special case of the bounded variance setting, i.e., $p=2$.

Method: Developed FAWN based on transformers architecture to fuse information from both Wi-Fi and 5G signals for passive sensing. Built a prototype and tested in real scenarios.

Result: Achieved localization errors below 0.6 meters approximately 84% of the time, demonstrating improved accuracy through multi-technology fusion.

Conclusion: Multi-technology fusion through FAWN enables more accurate indoor scene inference without interfering with existing communications, advancing ISAC passive sensing capabilities.

Abstract: The upcoming generations of wireless technologies promise an era where everything is interconnected and intelligent. As the need for intelligence grows, networks must learn to better understand the physical world. However, deploying dedicated hardware to perceive the environment is not always feasible, mainly due to costs and/or complexity. Integrated Sensing and Communication (ISAC) has made a step forward in addressing this challenge. Within ISAC, passive sensing emerges as a cost-effective solution that reuses wireless communications to sense the environment, without interfering with existing communications. Nevertheless, the majority of current solutions are limited to one technology (mostly Wi-Fi or 5G), constraining the maximum accuracy reachable. As different technologies work with different spectrums, we see a necessity in integrating more than one technology to augment the coverage area. Hence, we take the advantage of ISAC passive sensing, to present FAWN, a MultiEncoder Fusion-Attention Wave Network for ISAC indoor scene inference. FAWN is based on the original transformers architecture, to fuse information from Wi-Fi and 5G, making the network capable of understanding the physical world without interfering with the current communication. To test our solution, we have built a prototype and integrated it in a real scenario. Results show errors below 0.6 m around 84% of times.

Method: Theoretically-grounded adaptation of the Adaptive Gradient Descent Without Descent method to stochastic setting, using only first-order stochastic oracle information to exploit local geometry.

Result: Proven convergence under various assumptions and empirical demonstration that the method competes effectively against tuned baseline methods.

Conclusion: The proposed adaptive step-size strategy provides an effective, hyperparameter-free approach for stochastic convex optimization that leverages local geometry information while maintaining theoretical guarantees.

Abstract: We introduce a new adaptive step-size strategy for convex optimization with stochastic gradient that exploits the local geometry of the objective function only by means of a first-order stochastic oracle and without any hyper-parameter tuning. The method comes from a theoretically-grounded adaptation of the Adaptive Gradient Descent Without Descent method to the stochastic setting. We prove the convergence of stochastic gradient descent with our step-size under various assumptions, and we show that it empirically competes against tuned baselines.

Method: Proposes Attentive Graph Clustering Network (AGCN) that directly embeds attention mechanism into graph structure. Includes two key innovations: (1) KV cache mechanism for computational efficiency, and (2) pairwise margin contrastive loss to enhance discriminative capacity of attention space. The framework incorporates theoretical analysis comparing AGCN with GNN and Transformer behaviors.

Result: Extensive experimental results demonstrate that AGCN outperforms state-of-the-art methods in graph clustering tasks.

Conclusion: AGCN successfully bridges the gap between GNNs and Transformers by embedding attention directly into graph structure, enabling effective global information extraction while maintaining sensitivity to local topological patterns, proving that attention is not redundant for unsupervised graph learning.

Abstract: Attention mechanisms have become a cornerstone in modern neural networks, driving breakthroughs across diverse domains. However, their application to graph structured data, where capturing topological connections is essential, remains underexplored and underperforming compared to Graph Neural Networks (GNNs), particularly in the graph clustering task. GNN tends to overemphasize neighborhood aggregation, leading to a homogenization of node representations. Conversely, Transformer tends to over globalize, highlighting distant nodes at the expense of meaningful local patterns. This dichotomy raises a key question: Is attention inherently redundant for unsupervised graph learning? To address this, we conduct a comprehensive empirical analysis, uncovering the complementary weaknesses of GNN and Transformer in graph clustering. Motivated by these insights, we propose the Attentive Graph Clustering Network (AGCN) a novel architecture that reinterprets the notion that graph is attention. AGCN directly embeds the attention mechanism into the graph structure, enabling effective global information extraction while maintaining sensitivity to local topological cues. Our framework incorporates theoretical analysis to contrast AGCN behavior with GNN and Transformer and introduces two innovations: (1) a KV cache mechanism to improve computational efficiency, and (2) a pairwise margin contrastive loss to boost the discriminative capacity of the attention space. Extensive experimental results demonstrate that AGCN outperforms state-of-the-art methods.

Method: Proposes Discrepancy of Environment Dynamics (DoE) metric to isolate environment shift effects, and DEER framework that uses a binary classifier to detect environment changes and applies different prioritization strategies before/after shifts.

Result: Experiments on four non-stationary benchmarks show DEER improves off-policy algorithm performance by 11.54% compared to the best state-of-the-art ER methods.

Conclusion: DEER provides an effective adaptive experience replay framework that handles non-stationary environments by separately addressing policy-induced and environment-induced changes, enabling more sample-efficient reinforcement learning.

Abstract: Reinforcement learning (RL) in non-stationary environments is challenging, as changing dynamics and rewards quickly make past experiences outdated. Traditional experience replay (ER) methods, especially those using TD-error prioritization, struggle to distinguish between changes caused by the agent’s policy and those from the environment, resulting in inefficient learning under dynamic conditions. To address this challenge, we propose the Discrepancy of Environment Dynamics (DoE), a metric that isolates the effects of environment shifts on value functions. Building on this, we introduce Discrepancy of Environment Prioritized Experience Replay (DEER), an adaptive ER framework that prioritizes transitions based on both policy updates and environmental changes. DEER uses a binary classifier to detect environment changes and applies distinct prioritization strategies before and after each shift, enabling more sample-efficient learning. Experiments on four non-stationary benchmarks demonstrate that DEER further improves the performance of off-policy algorithms by 11.54 percent compared to the best-performing state-of-the-art ER methods.

Method: The approach models joint dependencies in latent space by decoupling marginal distributions, temporal dynamics, and inter-variable dependencies. Uses transformer encoder for temporal patterns, multivariate likelihood and copula for spatial dependencies, trained jointly with self-supervised contrastive learning.

Result: Not explicitly stated in the abstract, but the method is designed to better capture complex spatio-temporal correlations for improved anomaly detection performance.

Conclusion: The proposed framework effectively addresses the limitations of existing approaches by modeling complex non-linear spatio-temporal dependencies through a joint learning approach in latent space.

Abstract: In this paper, we aim to improve multivariate anomaly detection (AD) by modeling the \textit{time-varying non-linear spatio-temporal correlations} found in multivariate time series data . In multivariate time series data, an anomaly may be indicated by the simultaneous deviation of interrelated time series from their expected collective behavior, even when no individual time series exhibits a clearly abnormal pattern on its own. In many existing approaches, time series variables are assumed to be (conditionally) independent, which oversimplifies real-world interactions. Our approach addresses this by modeling joint dependencies in the latent space and decoupling the modeling of \textit{marginal distributions, temporal dynamics, and inter-variable dependencies}. We use a transformer encoder to capture temporal patterns, and to model spatial (inter-variable) dependencies, we fit a multi-variate likelihood and a copula. The temporal and the spatial components are trained jointly in a latent space using a self-supervised contrastive learning objective to learn meaningful feature representations to separate normal and anomaly samples.

Method: Two-stage approach: (1) SIMPC segments and clusters multivariate time series to extract amplitude- and time-invariant patterns; (2) JISC-Net uses shapelet-based classification with initial pattern parts to forecast short-term directional movements.

Result: Ranked first or second in 11 out of 12 metric-dataset combinations on Bitcoin and S&P 500 equities, consistently outperforming baselines.

Conclusion: The framework provides both accurate forecasting and transparent decision-making by revealing underlying pattern structures, unlike conventional deep learning models that lack interpretable justification.

Abstract: Directional forecasting in financial markets requires both accuracy and interpretability. Before the advent of deep learning, interpretable approaches based on human-defined patterns were prevalent, but their structural vagueness and scale ambiguity hindered generalization. In contrast, deep learning models can effectively capture complex dynamics, yet often offer limited transparency. To bridge this gap, we propose a two-stage framework that integrates unsupervised pattern extracion with interpretable forecasting. (i) SIMPC segments and clusters multivariate time series, extracting recurrent patterns that are invariant to amplitude scaling and temporal distortion, even under varying window sizes. (ii) JISC-Net is a shapelet-based classifier that uses the initial part of extracted patterns as input and forecasts subsequent partial sequences for short-term directional movement. Experiments on Bitcoin and three S&P 500 equities demonstrate that our method ranks first or second in 11 out of 12 metric–dataset combinations, consistently outperforming baselines. Unlike conventional deep learning models that output buy-or-sell signals without interpretable justification, our approach enables transparent decision-making by revealing the underlying pattern structures that drive predictive outcomes.

Method: Multi-head neural network with separate outputs for behavioral cloning and Q-learning, unified by shared feature extraction layers with attention mechanisms. Starts with behavioral cloning on demonstration data from rule-based agents, then transitions to reinforcement learning.

Result: Consistently achieves above 70% win rate against rule-based opponents, substantially outperforming pure reinforcement learning methods which showed high variance and frequent performance degradation.

Conclusion: Combining demonstration-based initialization with reinforcement learning optimization provides a robust solution for developing game AI agents in complex multi-agent environments where pure exploration proves insufficient.

Abstract: Reinforcement learning agents in complex game environments often suffer from sparse rewards, training instability, and poor sample efficiency. This paper presents a hybrid training approach that combines offline imitation learning with online reinforcement learning for a 2D shooter game agent. We implement a multi-head neural network with separate outputs for behavioral cloning and Q-learning, unified by shared feature extraction layers with attention mechanisms. Initial experiments using pure deep Q-Networks exhibited significant instability, with agents frequently reverting to poor policies despite occasional good performance. To address this, we developed a hybrid methodology that begins with behavioral cloning on demonstration data from rule-based agents, then transitions to reinforcement learning. Our hybrid approach achieves consistently above 70% win rate against rule-based opponents, substantially outperforming pure reinforcement learning methods which showed high variance and frequent performance degradation. The multi-head architecture enables effective knowledge transfer between learning modes while maintaining training stability. Results demonstrate that combining demonstration-based initialization with reinforcement learning optimization provides a robust solution for developing game AI agents in complex multi-agent environments where pure exploration proves insufficient.

Method: Tested Logistic Regression, Random Forest, XGBoost, KNN, and MLP on real-world dataset using undersampling, SMOTE, and hybrid sampling techniques, evaluated on original imbalanced test set.

Result: Hybrid sampling method achieved the best balance between recall and precision, particularly improving performance of MLP and KNN models.

Conclusion: Hybrid sampling approach is most effective for credit card fraud detection, providing optimal performance balance while addressing class imbalance issues.

Abstract: Credit card fraud remains a significant challenge due to class imbalance and fraudsters mimicking legitimate behavior. This study evaluates five machine learning models - Logistic Regression, Random Forest, XGBoost, K-Nearest Neighbors (KNN), and Multi-Layer Perceptron (MLP) on a real-world dataset using undersampling, SMOTE, and a hybrid approach. Our models are evaluated on the original imbalanced test set to better reflect real-world performance. Results show that the hybrid method achieves the best balance between recall and precision, especially improving MLP and KNN performance.

Method: Developed alternative hyperparameters for specifying sparse RNNs that enable varying sparsity within trainable weight matrices, and defined a novel “hidden proportion” metric to balance unknown distribution within the model.

Result: The approach generated significant performance gains and improved performance expectations on an a priori basis, providing explanatory power for model performance.

Conclusion: The combined approach of varied sparsity RNN architecture with the hidden proportion metric provides a path toward generalized meta-learning applications and model optimization based on intrinsic dataset characteristics including input/output dimensions.

Abstract: This paper develops alternative hyperparameters for specifying sparse Recurrent Neural Networks (RNNs). These hyperparameters allow for varying sparsity within the trainable weight matrices of the model while improving overall performance. This architecture enables the definition of a novel metric, hidden proportion, which seeks to balance the distribution of unknowns within the model and provides significant explanatory power of model performance. Together, the use of the varied sparsity RNN architecture combined with the hidden proportion metric generates significant performance gains while improving performance expectations on an a priori basis. This combined approach provides a path forward towards generalized meta-learning applications and model optimization based on intrinsic characteristics of the data set, including input and output dimensions.

Method: Proposes Attention-based Double Compression (ADC) with two parallel strategies: 1) class-agnostic merging of similar activations based on average attention scores from the last client layer, and 2) discarding the least meaningful tokens to further reduce communication costs.

Result: Simulation results show ADC outperforms state-of-the-art SL frameworks by significantly reducing communication overheads while maintaining high accuracy, with gradients naturally compressed without additional tuning.

Conclusion: ADC provides an effective solution for communication-efficient Split Learning in Vision Transformers, achieving substantial communication reduction without compromising model performance or requiring complex gradient approximations.

Abstract: This paper proposes a novel communication-efficient Split Learning (SL) framework, named Attention-based Double Compression (ADC), which reduces the communication overhead required for transmitting intermediate Vision Transformers activations during the SL training process. ADC incorporates two parallel compression strategies. The first one merges samples’ activations that are similar, based on the average attention score calculated in the last client layer; this strategy is class-agnostic, meaning that it can also merge samples having different classes, without losing generalization ability nor decreasing final results. The second strategy follows the first and discards the least meaningful tokens, further reducing the communication cost. Combining these strategies not only allows for sending less during the forward pass, but also the gradients are naturally compressed, allowing the whole model to be trained without additional tuning or approximations of the gradients. Simulation results demonstrate that Attention-based Double Compression outperforms state-of-the-art SL frameworks by significantly reducing communication overheads while maintaining high accuracy.

Method: A smooth probabilistic reformulation approach that computes exact gradients without Monte Carlo sampling. Also investigates nonlinear compressive sensing through theoretical analysis based on Fefferman and Markel theorems and implements a normal-form algorithm for empirical validation.

Result: The method significantly improves convergence speed compared to Monte Carlo approaches and outperforms compressive sensing algorithms (IHT, Relaxed-Lasso) across various settings and SNR ratios. However, exact parameter recovery in nonlinear compressive sensing is not possible even up to symmetries, with observed rebound effects where teacher-student configurations diverge despite decreasing test loss.

Conclusion: The proposed probabilistic reformulation provides efficient ℓ0 regularization with exact gradients and fast convergence. Nonlinear compressive sensing shows fundamental differences from linear case, with exact parameter recovery being impossible even considering symmetries, indicating important limitations in nonlinear generalizations.

Abstract: We present a smooth probabilistic reformulation of $\ell_0$ regularized regression that does not require Monte Carlo sampling and allows for the computation of exact gradients, facilitating rapid convergence to local optima of the best subset selection problem. The method drastically improves convergence speed compared to similar Monte Carlo based approaches. Furthermore, we empirically demonstrate that it outperforms compressive sensing algorithms such as IHT and (Relaxed-) Lasso across a wide range of settings and signal-to-noise ratios. The implementation runs efficiently on both CPUs and GPUs and is freely available at https://github.com/L0-and-behold/probabilistic-nonlinear-cs. We also contribute to research on nonlinear generalizations of compressive sensing by investigating when parameter recovery of a nonlinear teacher network is possible through compression of a student network. Building upon theorems of Fefferman and Markel, we show theoretically that the global optimum in the infinite-data limit enforces recovery up to certain symmetries. For empirical validation, we implement a normal-form algorithm that selects a canonical representative within each symmetry class. However, while compression can help to improve test loss, we find that exact parameter recovery is not even possible up to symmetries. In particular, we observe a surprising rebound effect where teacher and student configurations initially converge but subsequently diverge despite continuous decrease in test loss. These findings indicate fundamental differences between linear and nonlinear compressive sensing.

Method: Two clustering strategies: source clustering and histogram clustering to group flows with similar temporal patterns prior to training deep learning models. This creates homogeneous subsets for more effective model training.

Result: Reduces RMSE by up to 92% for Abilene and 75% for GÉANT networks. Also reduces maximum link utilization bias by 18% and 21% respectively in routing scenarios.

Conclusion: Clustering-based approach significantly improves traffic matrix prediction accuracy and demonstrates practical benefits for network optimization applications compared to global prediction methods.

Abstract: We present a novel framework that leverages time series clustering to improve internet traffic matrix (TM) prediction using deep learning (DL) models. Traffic flows within a TM often exhibit diverse temporal behaviors, which can hinder prediction accuracy when training a single model across all flows. To address this, we propose two clustering strategies, source clustering and histogram clustering, that group flows with similar temporal patterns prior to model training. Clustering creates more homogeneous data subsets, enabling models to capture underlying patterns more effectively and generalize better than global prediction approaches that fit a single model to the entire TM. Compared to existing TM prediction methods, our method reduces RMSE by up to 92% for Abilene and 75% for G'EANT. In routing scenarios, our clustered predictions also reduce maximum link utilization (MLU) bias by 18% and 21%, respectively, demonstrating the practical benefits of clustering when TMs are used for network optimization.

Method: Proposed algorithm handles constrained feedback in non-stationary environments without prior knowledge of non-stationarity degree, using a query budget B to manage limited feedback.

Result: Achieves dynamic regret of O~(K^{1/3} V_T^{1/3} T / B^{1/3}), where T is rounds, K is arms, B is query budget, and V_T is variation budget.

Conclusion: The algorithm provides near-optimal performance for non-stationary bandits with constrained feedback, making it suitable for dynamic real-world applications with limited feedback availability.

Abstract: Non-stationary multi-armed bandits enable agents to adapt to changing environments by incorporating mechanisms to detect and respond to shifts in reward distributions, making them well-suited for dynamic settings. However, existing approaches typically assume that reward feedback is available at every round - an assumption that overlooks many real-world scenarios where feedback is limited. In this paper, we take a significant step forward by introducing a new model of constrained feedback in non-stationary multi-armed bandits, where the availability of reward feedback is restricted. We propose the first prior-free algorithm - that is, one that does not require prior knowledge of the degree of non-stationarity - that achieves near-optimal dynamic regret in this setting. Specifically, our algorithm attains a dynamic regret of $\tilde{\mathcal{O}}({K^{1/3} V_T^{1/3} T }/{ B^{1/3}})$, where $T$ is the number of rounds, $K$ is the number of arms, $B$ is the query budget, and $V_T$ is the variation budget capturing the degree of non-stationarity.

Method: Combines statistical texture analysis with supervised learning for pollution classification, and uses VLM-guided image generation to create interpretable pollution visualizations. Incorporates human-centered UX principles.

Result: Validated on urban sky image dataset, demonstrating effectiveness in pollution level estimation and semantically consistent visual synthesis. System enables real-time integration into applications.

Conclusion: The approach provides foundation for transparent air quality interfaces and supports informed environmental decision-making. Future work will incorporate green CNN architecture with FPGA-based incremental learning for energy-efficient edge deployment.

Abstract: Air pollution remains a critical threat to public health and environmental sustainability, yet conventional monitoring systems are often constrained by limited spatial coverage and accessibility. This paper proposes an AI-driven agent that predicts ambient air pollution levels from sky images and synthesizes realistic visualizations of pollution scenarios using generative modeling. Our approach combines statistical texture analysis with supervised learning for pollution classification, and leverages vision-language model (VLM)-guided image generation to produce interpretable representations of air quality conditions. The generated visuals simulate varying degrees of pollution, offering a foundation for user-facing interfaces that improve transparency and support informed environmental decision-making. These outputs can be seamlessly integrated into intelligent applications aimed at enhancing situational awareness and encouraging behavioral responses based on real-time forecasts. We validate our method using a dataset of urban sky images and demonstrate its effectiveness in both pollution level estimation and semantically consistent visual synthesis. The system design further incorporates human-centered user experience principles to ensure accessibility, clarity, and public engagement in air quality forecasting. To support scalable and energy-efficient deployment, future iterations will incorporate a green CNN architecture enhanced with FPGA-based incremental learning, enabling real-time inference on edge platforms.

Method: Proposes FedLEASE framework that clusters clients based on representation similarity to allocate domain-specific LoRA experts, and uses adaptive top-M Mixture-of-Experts mechanism for optimal expert selection per client.

Result: Extensive experiments on diverse benchmark datasets show FedLEASE significantly outperforms existing federated fine-tuning approaches in heterogeneous client settings while maintaining communication efficiency.

Conclusion: FedLEASE effectively addresses the challenges of determining optimal LoRA expert allocation and enabling selective expert utilization in federated LLM fine-tuning for heterogeneous data environments.

Abstract: Large Language Models (LLMs) have demonstrated impressive capabilities across various tasks, but fine-tuning them for domain-specific applications often requires substantial domain-specific data that may be distributed across multiple organizations. Federated Learning (FL) offers a privacy-preserving solution, but faces challenges with computational constraints when applied to LLMs. Low-Rank Adaptation (LoRA) has emerged as a parameter-efficient fine-tuning approach, though a single LoRA module often struggles with heterogeneous data across diverse domains. This paper addresses two critical challenges in federated LoRA fine-tuning: 1. determining the optimal number and allocation of LoRA experts across heterogeneous clients, and 2. enabling clients to selectively utilize these experts based on their specific data characteristics. We propose FedLEASE (Federated adaptive LoRA Expert Allocation and SElection), a novel framework that adaptively clusters clients based on representation similarity to allocate and train domain-specific LoRA experts. It also introduces an adaptive top-$M$ Mixture-of-Experts mechanism that allows each client to select the optimal number of utilized experts. Our extensive experiments on diverse benchmark datasets demonstrate that FedLEASE significantly outperforms existing federated fine-tuning approaches in heterogeneous client settings while maintaining communication efficiency.

Method: Modeled as a multi-leader Stackelberg game extending Bayesian persuasion to multi-round conversations between differently informed parties, with theoretical analysis and two sets of experiments.

Result: Three key results: (1) user can learn Bayes-optimal action under convex hull condition, (2) non-strategic user achieves near-optimal utility with quantal response, (3) equilibrium guarantees remain near-optimal when selecting best single AI after evaluation.

Conclusion: Perfect alignment may not be necessary - strategic competition among diverse misaligned agents can provide alignment benefits when user utility lies within the convex hull of agents’ utilities, with increasing model diversity making this condition easier to satisfy.

Abstract: Aligning AI systems with human values remains a fundamental challenge, but does our inability to create perfectly aligned models preclude obtaining the benefits of alignment? We study a strategic setting where a human user interacts with multiple differently misaligned AI agents, none of which are individually well-aligned. Our key insight is that when the users utility lies approximately within the convex hull of the agents utilities, a condition that becomes easier to satisfy as model diversity increases, strategic competition can yield outcomes comparable to interacting with a perfectly aligned model. We model this as a multi-leader Stackelberg game, extending Bayesian persuasion to multi-round conversations between differently informed parties, and prove three results: (1) when perfect alignment would allow the user to learn her Bayes-optimal action, she can also do so in all equilibria under the convex hull condition (2) under weaker assumptions requiring only approximate utility learning, a non-strategic user employing quantal response achieves near-optimal utility in all equilibria and (3) when the user selects the best single AI after an evaluation period, equilibrium guarantees remain near-optimal without further distributional assumptions. We complement the theory with two sets of experiments.

Method: Divides neural networks into two tiers: lower layers use FPGA-based single-step equation solving for efficient feature extraction, while higher layers employ adaptive incremental learning. Introduces Compound LLM framework with lower-level LLM for representation learning and upper-level LLM for adaptive decision-making.

Result: Significantly reduces computational costs while preserving high model performance, making it suitable for edge deployment and real-time adaptation in energy-constrained environments.

Conclusion: The proposed framework enhances scalability, reduces redundant computation, and aligns with sustainable AI principles by providing an energy-efficient alternative to traditional training methods.

Abstract: The rising computational and energy demands of deep learning, particularly in large-scale architectures such as foundation models and large language models (LLMs), pose significant challenges to sustainability. Traditional gradient-based training methods are inefficient, requiring numerous iterative updates and high power consumption. To address these limitations, we propose a hybrid framework that combines hierarchical decomposition with FPGA-based direct equation solving and incremental learning. Our method divides the neural network into two functional tiers: lower layers are optimized via single-step equation solving on FPGAs for efficient and parallelizable feature extraction, while higher layers employ adaptive incremental learning to support continual updates without full retraining. Building upon this foundation, we introduce the Compound LLM framework, which explicitly deploys LLM modules across both hierarchy levels. The lower-level LLM handles reusable representation learning with minimal energy overhead, while the upper-level LLM performs adaptive decision-making through energy-aware updates. This integrated design enhances scalability, reduces redundant computation, and aligns with the principles of sustainable AI. Theoretical analysis and architectural insights demonstrate that our method reduces computational costs significantly while preserving high model performance, making it well-suited for edge deployment and real-time adaptation in energy-constrained environments.

Method: Replaces deep architectures with simple frequency-specialized linear experts trained on resampled data across multiple frequency regimes, using a lightweight spectral gating mechanism to dynamically select relevant experts.

Result: Matches state-of-the-art performance while offering superior efficiency, robustness to various sampling rates, and enhanced interpretability.

Conclusion: Super-Linear provides an effective lightweight alternative to computationally expensive models for time series forecasting, demonstrating that simple linear experts with proper frequency specialization can achieve competitive results with better efficiency.

Abstract: Time series forecasting (TSF) is critical in domains like energy, finance, healthcare, and logistics, requiring models that generalize across diverse datasets. Large pre-trained models such as Chronos and Time-MoE show strong zero-shot (ZS) performance but suffer from high computational costs. In this work, We introduce Super-Linear, a lightweight and scalable mixture-of-experts (MoE) model for general forecasting. It replaces deep architectures with simple frequency-specialized linear experts, trained on resampled data across multiple frequency regimes. A lightweight spectral gating mechanism dynamically selects relevant experts, enabling efficient, accurate forecasting. Despite its simplicity, Super-Linear matches state-of-the-art performance while offering superior efficiency, robustness to various sampling rates, and enhanced interpretability. The implementation of Super-Linear is available at \href{https://github.com/azencot-group/SuperLinear}{https://github.com/azencot-group/SuperLinear}

Method: Conducted systematic analysis including cross-dataset domain shift evaluation across multiple model architectures, trained source-classification model to detect dataset bias, performed subgroup analyses for age/sex groups, and expert review by two board-certified radiologists comparing with public dataset labels.

Result: Found substantial external performance degradation with reduced AUPRC and F1 scores, near-perfect dataset classification indicating bias, reduced performance for minority age/sex groups, and significant expert disagreement with public dataset labels.

Conclusion: Current chest X-ray benchmarks have important clinical weaknesses, highlighting the need for clinician-validated datasets and fairer evaluation frameworks to ensure reliable and equitable AI performance.

Abstract: Artificial intelligence has shown significant promise in chest radiography, where deep learning models can approach radiologist-level diagnostic performance. Progress has been accelerated by large public datasets such as MIMIC-CXR, ChestX-ray14, PadChest, and CheXpert, which provide hundreds of thousands of labelled images with pathology annotations. However, these datasets also present important limitations. Automated label extraction from radiology reports introduces errors, particularly in handling uncertainty and negation, and radiologist review frequently disagrees with assigned labels. In addition, domain shift and population bias restrict model generalisability, while evaluation practices often overlook clinically meaningful measures. We conduct a systematic analysis of these challenges, focusing on label quality, dataset bias, and domain shift. Our cross-dataset domain shift evaluation across multiple model architectures revealed substantial external performance degradation, with pronounced reductions in AUPRC and F1 scores relative to internal testing. To assess dataset bias, we trained a source-classification model that distinguished datasets with near-perfect accuracy, and performed subgroup analyses showing reduced performance for minority age and sex groups. Finally, expert review by two board-certified radiologists identified significant disagreement with public dataset labels. Our findings highlight important clinical weaknesses of current benchmarks and emphasise the need for clinician-validated datasets and fairer evaluation frameworks.

Method: TDRM minimizes temporal differences during training to produce smooth rewards, and is incorporated into actor-critic style online RL loops as a supplement to verifiable reward methods.

Result: TD-trained process reward models improve performance across Best-of-N (up to 6.6%) and tree-search (up to 23.7%) settings. When combined with RLVR, they achieve comparable performance with just 2.5k data vs 50.1k for baselines, and yield higher-quality policies on 8 model variants.

Conclusion: TDRM provides an effective method for learning temporally consistent reward models that significantly improve RL efficiency and performance across multiple language model variants.

Abstract: Reward models are central to both reinforcement learning (RL) with language models and inference-time verification. However, existing reward models often lack temporal consistency, leading to ineffective policy updates and unstable RL training. We introduce TDRM, a method for learning smoother and more reliable reward models by minimizing temporal differences during training. This temporal-difference (TD) regularization produces smooth rewards and improves alignment with long-term objectives. Incorporating TDRM into the actor-critic style online RL loop yields consistent empirical gains. It is worth noting that TDRM is a supplement to verifiable reward methods, and both can be used in series. Experiments show that TD-trained process reward models (PRMs) improve performance across Best-of-N (up to 6.6%) and tree-search (up to 23.7%) settings. When combined with Reinforcement Learning with Verifiable Rewards (RLVR), TD-trained PRMs lead to more data-efficient RL – achieving comparable performance with just 2.5k data to what baseline methods require 50.1k data to attain – and yield higher-quality language model policies on 8 model variants (5 series), e.g., Qwen2.5-(0.5B, 1,5B), GLM4-9B-0414, GLM-Z1-9B-0414, Qwen2.5-Math-(1.5B, 7B), and DeepSeek-R1-Distill-Qwen-(1.5B, 7B). We release all code at https://github.com/THUDM/TDRM.

Method: Combines stochastic zeroth-order optimization for updating physical layer parameters with a dynamic low-rank surrogate model for gradient propagation. Uses implicit projector-splitting integrator algorithm to update lightweight surrogate model efficiently.

Result: Achieves near-digital baseline accuracy across computer vision, audio classification, and language modeling tasks. Successfully enables end-to-end training with various non-differentiable physical components.

Conclusion: Bridges hardware-aware deep learning and gradient-free optimization, providing a practical pathway for integrating non-differentiable physical components into scalable, end-to-end trainable AI systems.

Abstract: The growing demand for energy-efficient, high-performance AI systems has led to increased attention on alternative computing platforms (e.g., photonic, neuromorphic) due to their potential to accelerate learning and inference. However, integrating such physical components into deep learning pipelines remains challenging, as physical devices often offer limited expressiveness, and their non-differentiable nature renders on-device backpropagation difficult or infeasible. This motivates the development of hybrid architectures that combine digital neural networks with reconfigurable physical layers, which effectively behave as black boxes. In this work, we present a framework for the end-to-end training of such hybrid networks. This framework integrates stochastic zeroth-order optimization for updating the physical layer’s internal parameters with a dynamic low-rank surrogate model that enables gradient propagation through the physical layer. A key component of our approach is the implicit projector-splitting integrator algorithm, which updates the lightweight surrogate model after each forward pass with minimal hardware queries, thereby avoiding costly full matrix reconstruction. We demonstrate our method across diverse deep learning tasks, including: computer vision, audio classification, and language modeling. Notably, across all modalities, the proposed approach achieves near-digital baseline accuracy and consistently enables effective end-to-end training of hybrid models incorporating various non-differentiable physical components (spatial light modulators, microring resonators, and Mach-Zehnder interferometers). This work bridges hardware-aware deep learning and gradient-free optimization, thereby offering a practical pathway for integrating non-differentiable physical components into scalable, end-to-end trainable AI systems.

Method: Developed a mathematically grounded procedure to estimate noise-free conformal prediction thresholds, then created a practical algorithm to handle challenges from continuous regression problems.

Result: Evaluated on two medical imaging datasets with Gaussian label noise, the method significantly outperforms existing alternatives and achieves performance close to clean-label settings.

Conclusion: The proposed approach successfully addresses the challenge of noisy labels in conformal prediction for regression, providing reliable confidence intervals even with imperfect calibration data.

Abstract: In high-stakes scenarios, such as medical imaging applications, it is critical to equip the predictions of a regression model with reliable confidence intervals. Recently, Conformal Prediction (CP) has emerged as a powerful statistical framework that, based on a labeled calibration set, generates intervals that include the true labels with a pre-specified probability. In this paper, we address the problem of applying CP for regression models when the calibration set contains noisy labels. We begin by establishing a mathematically grounded procedure for estimating the noise-free CP threshold. Then, we turn it into a practical algorithm that overcomes the challenges arising from the continuous nature of the regression problem. We evaluate the proposed method on two medical imaging regression datasets with Gaussian label noise. Our method significantly outperforms the existing alternative, achieving performance close to the clean-label setting.

Method: Transform scalar rewards into a normalized target distribution using a learnable partition function, then minimize reverse KL divergence between policy and target distribution through flow-balanced optimization.

Result: FlowRL achieves 10.0% average improvement over GRPO and 5.1% over PPO on math benchmarks, with consistent better performance on code reasoning tasks.

Conclusion: Reward distribution-matching is a key step toward efficient exploration and diverse reasoning in LLM reinforcement learning.

Abstract: We propose FlowRL: matching the full reward distribution via flow balancing instead of maximizing rewards in large language model (LLM) reinforcement learning (RL). Recent advanced reasoning models adopt reward-maximizing methods (\eg, PPO and GRPO), which tend to over-optimize dominant reward signals while neglecting less frequent but valid reasoning paths, thus reducing diversity. In contrast, we transform scalar rewards into a normalized target distribution using a learnable partition function, and then minimize the reverse KL divergence between the policy and the target distribution. We implement this idea as a flow-balanced optimization method that promotes diverse exploration and generalizable reasoning trajectories. We conduct experiments on math and code reasoning tasks: FlowRL achieves a significant average improvement of $10.0%$ over GRPO and $5.1%$ over PPO on math benchmarks, and performs consistently better on code reasoning tasks. These results highlight reward distribution-matching as a key step toward efficient exploration and diverse reasoning in LLM reinforcement learning.

Method: Introduces a versatile low-variance de-biasing approach that combines novel estimators with standard techniques to learn from aggregate label information.

Result: Significantly advances state of the art in LLP, improves sample complexity guarantees, and demonstrates compelling empirical advantages over baselines across diverse benchmark datasets.

Conclusion: The proposed approach provides a flexible and effective solution for learning from label proportions with broad practical applicability across different classification settings and loss functions.

Abstract: Motivated by problems in online advertising, we address the task of Learning from Label Proportions (LLP). In this partially-supervised setting, training data consists of groups of examples, termed bags, for which we only observe the average label value. The main goal, however, remains the design of a predictor for the labels of individual examples. We introduce a novel and versatile low-variance de-biasing methodology to learn from aggregate label information, significantly advancing the state of the art in LLP. Our approach exhibits remarkable flexibility, seamlessly accommodating a broad spectrum of practically relevant loss functions across both binary and multi-class classification settings. By carefully combining our estimators with standard techniques, we substantially improve sample complexity guarantees for a large class of losses of practical relevance. We also empirically validate the efficacy of our proposed approach across a diverse array of benchmark datasets, demonstrating compelling empirical advantages over standard baselines.

Method: The paper introduces and compares three logit aggregation methods: simple averaging, uncertainty-weighted averaging, and a learned meta-aggregator, evaluated on MNIST and CIFAR-10 datasets.

Result: The methods reduce communication overhead, improve robustness under non-IID data, and achieve accuracy competitive with centralized training.

Conclusion: Logit-based FL with effective aggregation methods provides a cost-efficient alternative to traditional federated learning while maintaining competitive performance.

Abstract: Federated learning (FL) usually shares model weights or gradients, which is costly for large models. Logit-based FL reduces this cost by sharing only logits computed on a public proxy dataset. However, aggregating information from heterogeneous clients is still challenging. This paper studies this problem, introduces and compares three logit aggregation methods: simple averaging, uncertainty-weighted averaging, and a learned meta-aggregator. Evaluated on MNIST and CIFAR-10, these methods reduce communication overhead, improve robustness under non-IID data, and achieve accuracy competitive with centralized training.

Method: Proved mathematical equivalence of kinetic, conditional kinetic, and geodesic energy formulations under MDM structure. Parameterized interpolation schedules using Beta distributions to reduce design space to 2D search for efficient post-training tuning.

Result: Energy-inspired schedules derived from the theoretical framework outperform hand-crafted baselines, particularly in low-step sampling settings, as demonstrated on synthetic and real-world benchmarks.

Conclusion: The work provides a systematic theoretical unification of MDMs through optimal transport energy minimization, and shows this theoretical insight enables practical schedule optimization that improves sampling efficiency without model modification.

Abstract: We present a systematic theoretical framework that interprets masked diffusion models (MDMs) as solutions to energy minimization problems in discrete optimal transport. Specifically, we prove that three distinct energy formulations–kinetic, conditional kinetic, and geodesic energy–are mathematically equivalent under the structure of MDMs, and that MDMs minimize all three when the mask schedule satisfies a closed-form optimality condition. This unification not only clarifies the theoretical foundations of MDMs, but also motivates practical improvements in sampling. By parameterizing interpolation schedules via Beta distributions, we reduce the schedule design space to a tractable 2D search, enabling efficient post-training tuning without model modification. Experiments on synthetic and real-world benchmarks demonstrate that our energy-inspired schedules outperform hand-crafted baselines, particularly in low-step sampling settings.

Method: Two-stage approach: 1) Supervised Fine-Tuning (SFT) with behavioral cloning and steps-to-go prediction objectives, 2) Self-Improvement stage where steps-to-go prediction enables reward function extraction and success detection for autonomous robot practice.

Result: Significantly more sample-efficient than scaling imitation data collection, achieves higher success rates, and uniquely enables autonomous acquisition of novel skills that generalize beyond training datasets.

Conclusion: Combining pretrained foundation models with online self-improvement has transformative potential for autonomous skill acquisition in robotics, demonstrating capabilities beyond current methods.

Abstract: Foundation models trained on web-scale data have revolutionized robotics, but their application to low-level control remains largely limited to behavioral cloning. Drawing inspiration from the success of the reinforcement learning stage in fine-tuning large language models, we propose a two-stage post-training approach for robotics. The first stage, Supervised Fine-Tuning (SFT), fine-tunes pretrained foundation models using both: a) behavioral cloning, and b) steps-to-go prediction objectives. In the second stage, Self-Improvement, steps-to-go prediction enables the extraction of a well-shaped reward function and a robust success detector, enabling a fleet of robots to autonomously practice downstream tasks with minimal human supervision. Through extensive experiments on real-world and simulated robot embodiments, our novel post-training recipe unveils significant results on Embodied Foundation Models. First, we demonstrate that the combination of SFT and Self-Improvement is significantly more sample-efficient than scaling imitation data collection for supervised learning, and that it leads to policies with significantly higher success rates. Further ablations highlight that the combination of web-scale pretraining and Self-Improvement is the key to this sample-efficiency. Next, we demonstrate that our proposed combination uniquely unlocks a capability that current methods cannot achieve: autonomously practicing and acquiring novel skills that generalize far beyond the behaviors observed in the imitation learning datasets used during training. These findings highlight the transformative potential of combining pretrained foundation models with online Self-Improvement to enable autonomous skill acquisition in robotics. Our project website can be found at https://self-improving-efms.github.io .

Method: Proactive data rewriting framework that keeps correct solutions as on-policy data and rewrites incorrect solutions with guided re-solving, only falling back to expert demonstrations when necessary. This aligns training distribution with target policy before optimization.

Result: Experiments on five mathematical reasoning benchmarks show consistent and significant improvements over vanilla SFT and state-of-the-art Dynamic Fine-Tuning (DFT) approach.

Conclusion: The proposed data rewriting framework effectively reduces importance sampling variance and stabilizes off-policy fine-tuning by proactively shrinking the policy gap, demonstrating superior performance on mathematical reasoning tasks.

Abstract: Supervised fine-tuning (SFT) of large language models can be viewed as an off-policy learning problem, where expert demonstrations come from a fixed behavior policy while training aims to optimize a target policy. Importance sampling is the standard tool for correcting this distribution mismatch, but large policy gaps lead to high variance and training instability. Existing approaches mitigate this issue using KL penalties or clipping, which passively constrain updates rather than actively reducing the gap. We propose a simple yet effective data rewriting framework that proactively shrinks the policy gap by keeping correct solutions as on-policy data and rewriting incorrect ones with guided re-solving, falling back to expert demonstrations only when needed. This aligns the training distribution with the target policy before optimization, reducing importance sampling variance and stabilizing off-policy fine-tuning. Experiments on five mathematical reasoning benchmarks demonstrate consistent and significant gains over both vanilla SFT and the state-of-the-art Dynamic Fine-Tuning (DFT) approach. The data and code will be released at https://github.com/NKU-HLT/Off-Policy-SFT.

Method: Proposed MaRVIn framework with: 1) Hardware enhancements including configurable ALU for 2/4/8-bit arithmetic, multi-pumping for latency reduction, and soft SIMD for 2-bit operations; 2) Software optimizations including pruning-aware fine-tuning and greedy DSE for mixed-quantized models; 3) ISA extensions and voltage scaling for power efficiency.

Result: Experimental evaluation on CIFAR10 and ImageNet shows average 17.6x speedup with less than 1% accuracy loss, outperforming state-of-the-art RISC-V cores with up to 1.8 TOPs/W power efficiency.

Conclusion: The MaRVIn framework successfully addresses the architectural limitations of existing embedded processors for mixed-precision neural networks through cross-layer hardware-software co-design, demonstrating significant performance and energy efficiency improvements for deep learning inference on RISC-V architectures.

Abstract: The evolution of quantization and mixed-precision techniques has unlocked new possibilities for enhancing the speed and energy efficiency of NNs. Several recent studies indicate that adapting precision levels across different parameters can maintain accuracy comparable to full-precision models while significantly reducing computational demands. However, existing embedded microprocessors lack sufficient architectural support for efficiently executing mixed-precision NNs, both in terms of ISA extensions and hardware design, resulting in inefficiencies such as excessive data packing/unpacking and underutilized arithmetic units. In this work, we propose novel ISA extensions and a micro-architecture implementation specifically designed to optimize mixed-precision execution, enabling energy-efficient deep learning inference on RISC-V architectures. We introduce MaRVIn, a cross-layer hardware-software co-design framework that enhances power efficiency and performance through a combination of hardware improvements, mixed-precision quantization, ISA-level optimizations, and cycle-accurate emulation. At the hardware level, we enhance the ALU with configurable mixed-precision arithmetic (2, 4, 8 bits) for weights/activations and employ multi-pumping to reduce execution latency while implementing soft SIMD for efficient 2-bit ops. At the software level, we integrate a pruning-aware fine-tuning method to optimize model compression and a greedy-based DSE approach to efficiently search for Pareto-optimal mixed-quantized models. Additionally, we incorporate voltage scaling to boost the power efficiency of our system. Our experimental evaluation over widely used DNNs and datasets, such as CIFAR10 and ImageNet, demonstrates that our framework can achieve, on average, 17.6x speedup for less than 1% accuracy loss and outperforms the ISA-agnostic state-of-the-art RISC-V cores, delivering up to 1.8 TOPs/W.

Method: EVOL-RL couples majority-vote answers as stable anchors with novelty-aware rewards that favor semantically different responses. Uses GRPO implementation with asymmetric clipping and entropy regularization to maintain search diversity.

Result: EVOL-RL significantly outperforms TTRL baseline - improves Qwen3-4B-Base AIME25 pass@1 from 4.6% to 16.4% and pass@16 from 18.5% to 37.9%. Prevents diversity collapse and enhances generalization across domains like GPQA.

Conclusion: EVOL-RL successfully enables general improvements without sacrificing exploration capacity, maintaining longer reasoning chains, and works effectively in both label-free and RLVR settings with broad applicability.

Abstract: Large language models (LLMs) are increasingly trained with reinforcement learning from verifiable rewards (RLVR), yet real-world deployment demands models that can self-improve without labels or external judges. Existing label-free methods, confidence minimization, self-consistency, or majority-vote objectives, stabilize learning but steadily shrink exploration, causing an entropy collapse: generations become shorter, less diverse, and brittle. Unlike prior approaches such as Test-Time Reinforcement Learning (TTRL), which primarily adapt models to the immediate unlabeled dataset at hand, our goal is broader: to enable general improvements without sacrificing the model’s inherent exploration capacity and generalization ability, i.e., evolving. We formalize this issue and propose EVolution-Oriented and Label-free Reinforcement Learning (EVOL-RL), a simple rule that couples stability with variation under a label-free setting. EVOL-RL keeps the majority-voted answer as a stable anchor (selection) while adding a novelty-aware reward that favors responses whose reasoning differs from what has already been produced (variation), measured in semantic space. Implemented with GRPO, EVOL-RL also uses asymmetric clipping to preserve strong signals and an entropy regularizer to sustain search. This majority-for-selection + novelty-for-variation design prevents collapse, maintains longer and more informative chains of thought, and improves both pass@1 and pass@n. EVOL-RL consistently outperforms the majority-only TTRL baseline; e.g., training on label-free AIME24 lifts Qwen3-4B-Base AIME25 pass@1 from TTRL’s 4.6% to 16.4%, and pass@16 from 18.5% to 37.9%. EVOL-RL not only prevents diversity collapse but also unlocks stronger generalization across domains (e.g., GPQA). Furthermore, we demonstrate that EVOL-RL also boosts performance in the RLVR setting, highlighting its broad applicability.

Method: Proposes extracting time-localized clusters from CNN’s internal representations to segment ECG by learned characteristics while quantifying uncertainty of these representations.

Result: Enables visualization of how different ECG waveform regions contribute to predictions and assessment of decision certainty.

Conclusion: The method enhances trust in AI-driven ECG diagnostics and facilitates discovery of clinically relevant electrophysiological patterns through structured interpretability.

Abstract: Deep learning has significantly advanced electrocardiogram (ECG) analysis, enabling automatic annotation, disease screening, and prognosis beyond traditional clinical capabilities. However, understanding these models remains a challenge, limiting interpretation and gaining knowledge from these developments. In this work, we propose a novel interpretability method for convolutional neural networks applied to ECG analysis. Our approach extracts time-localized clusters from the model’s internal representations, segmenting the ECG according to the learned characteristics while quantifying the uncertainty of these representations. This allows us to visualize how different waveform regions contribute to the model’s predictions and assess the certainty of its decisions. By providing a structured and interpretable view of deep learning models for ECG, our method enhances trust in AI-driven diagnostics and facilitates the discovery of clinically relevant electrophysiological patterns.

Method: Reformulates causal fairness extraction into a distribution estimation problem using low-dimensional marginal factorization approximation and heuristic algorithms for computational efficiency.

Result: Extensive experiments show CausalPre is both effective and scalable, achieving strong causal fairness while maintaining broad relationship coverage without trading off utility.

Conclusion: CausalPre demonstrates that causal fairness can be achieved efficiently without strong causal model assumptions, challenging conventional beliefs about the trade-offs required for causal fairness.

Abstract: Causal fairness in databases is crucial to preventing biased and inaccurate outcomes in downstream tasks. While most prior work assumes a known causal model, recent efforts relax this assumption by enforcing additional constraints. However, these approaches often fail to capture broader attribute relationships that are critical to maintaining utility. This raises a fundamental question: Can we harness the benefits of causal reasoning to design efficient and effective fairness solutions without relying on strong assumptions about the underlying causal model? In this paper, we seek to answer this question by introducing CausalPre, a scalable and effective causality-guided data pre-processing framework that guarantees justifiable fairness, a strong causal notion of fairness. CausalPre extracts causally fair relationships by reformulating the originally complex and computationally infeasible extraction task into a tailored distribution estimation problem. To ensure scalability, CausalPre adopts a carefully crafted variant of low-dimensional marginal factorization to approximate the joint distribution, complemented by a heuristic algorithm that efficiently tackles the associated computational challenge. Extensive experiments on benchmark datasets demonstrate that CausalPre is both effective and scalable, challenging the conventional belief that achieving causal fairness requires trading off relationship coverage for relaxed model assumptions.

Method: Developed a neuro-symbolic rule-based framework that integrates with movement trajectory platforms to conduct error correction and detection.

Result: Achieved F1 scores up to 0.984 for error prediction, 8.51% improvement in zero-shot accuracy for out-of-distribution data, and overall accuracy improvement over SOTA models.

Conclusion: The neuro-symbolic framework effectively addresses distribution shift challenges in trajectory classification, providing robust error detection and significant performance improvements for both normal and shock-affected scenarios.

Abstract: Classification of movement trajectories has many applications in transportation and is a key component for large-scale movement trajectory generation and anomaly detection which has key safety applications in the aftermath of a disaster or other external shock. However, the current state-of-the-art (SOTA) are based on supervised deep learning - which leads to challenges when the distribution of trajectories changes due to such a shock. We provide a neuro-symbolic rule-based framework to conduct error correction and detection of these models to integrate into our movement trajectory platform. We provide a suite of experiments on several recent SOTA models where we show highly accurate error detection, the ability to improve accuracy with a changing test distribution, and accuracy improvement for the base use case in addition to a suite of theoretical properties that informed algorithm development. Specifically, we show an F1 scores for predicting errors of up to 0.984, significant performance increase for out-of distribution accuracy (8.51% improvement over SOTA for zero-shot accuracy), and accuracy improvement over the SOTA model.

Method: Combines convolutional neural networks (local spatial features), graph neural networks (global channel connections), and recurrent neural networks (temporal evolution) for anomaly detection

Result: Achieves production-level accuracy, successfully captures diverse channel fault types using LHC collision data, and is being integrated into CMS core production system

Conclusion: The system demonstrates promising performance for real-time monitoring of HCAL and outperforms benchmark models

Abstract: The Compact Muon Solenoid (CMS) experiment is a general-purpose detector for high-energy collision at the Large Hadron Collider (LHC) at CERN. It employs an online data quality monitoring (DQM) system to promptly spot and diagnose particle data acquisition problems to avoid data quality loss. In this study, we present a semi-supervised spatio-temporal anomaly detection (AD) monitoring system for the physics particle reading channels of the Hadron Calorimeter (HCAL) of the CMS using three-dimensional digi-occupancy map data of the DQM. We propose the GraphSTAD system, which employs convolutional and graph neural networks to learn local spatial characteristics induced by particles traversing the detector and the global behavior owing to shared backend circuit connections and housing boxes of the channels, respectively. Recurrent neural networks capture the temporal evolution of the extracted spatial features. We validate the accuracy of the proposed AD system in capturing diverse channel fault types using the LHC collision data sets. The GraphSTAD system achieves production-level accuracy and is being integrated into the CMS core production system for real-time monitoring of the HCAL. We provide a quantitative performance comparison with alternative benchmark models to demonstrate the promising leverage of the presented system. Code: \href{https://github.com/muleina/CMS_HCAL_ML_OnlineDQM}{https://github.com/muleina/CMS_HCAL_ML_OnlineDQM}

Method: EXPLOR employs extrapolatory pseudo-labeling on latent-space augmentations using a diverse set of base models as pseudo-labelers. It trains multiple MLP heads (one per base model) with shared embedding using a novel per-head matching loss, without requiring modality-specific augmentations or OOD data access.

Result: EXPLOR demonstrates superior performance compared to state-of-the-art methods on diverse datasets in single-source domain generalization settings, showing robust OOD generalization capabilities.

Conclusion: The framework provides a model-agnostic solution for OOD generalization that works effectively with various methods from simple tree-based models to complex OOD generalization models, offering broad applicability across different data types.

Abstract: EXPLOR is a novel framework that utilizes support-expanding, extrapolatory pseudo-labeling to improve prediction and uncertainty-based rejection on out-of-distribution (OOD) points. EXPLOR utilizes a diverse set of base models as pseudo-labelers on the expansive augmented data to improve OOD performance through multiple MLP heads (one per base model) with shared embedding trained with a novel per-head matching loss. Unlike prior methods that rely on modality-specific augmentations or assume access to OOD data, EXPLOR introduces extrapolatory pseudo-labeling on latent-space augmentations, enabling robust OOD generalization with any real-valued vector data. In contrast to prior modality-agnostic methods with neural backbones, EXPLOR is model-agnostic, working effectively with methods from simple tree-based models to complex OOD generalization models. We demonstrate that EXPLOR achieves superior performance compared to state-of-the-art methods on diverse datasets in single-source domain generalization settings.

Method: Iterative Contrastive Unlearning (ICU) with three modules: Knowledge Unlearning Induction, Contrastive Learning Enhancement, and Iterative Unlearning Refinement.

Result: ICU effectively removes sensitive information while preserving model performance, demonstrating practical privacy-conscious machine learning.

Conclusion: ICU provides a promising solution for machine unlearning that balances privacy protection with model utility without requiring original training data.

Abstract: Recent advances in machine learning, particularly in Natural Language Processing (NLP), have produced powerful models trained on vast datasets. However, these models risk leaking sensitive information, raising privacy concerns. In response, regulatory measures such as the European Union’s General Data Protection Regulation (GDPR) have driven increasing interest in Machine Unlearning techniques, which enable models to selectively forget specific data entries. Early unlearning approaches primarily relied on pre-processing methods, while more recent research has shifted towards training-based solutions. Despite their effectiveness, a key limitation persists: most methods require access to original training data, which is often unavailable. Additionally, directly applying unlearning techniques bears the cost of undermining the model’s expressive capabilities. To address these challenges, we introduce the Iterative Contrastive Unlearning (ICU) framework, which consists of three core components: A Knowledge Unlearning Induction module designed to target specific knowledge for removal using an unlearning loss; A Contrastive Learning Enhancement module to preserve the model’s expressive capabilities against the pure unlearning goal; And an Iterative Unlearning Refinement module that dynamically adjusts the unlearning process through ongoing evaluation and updates. Experimental results demonstrate the efficacy of our ICU method in unlearning sensitive information while maintaining the model’s overall performance, offering a promising solution for privacy-conscious machine learning applications.

Method: Proposes HeteroKRLAttack - a targeted evasion black-box attack method that integrates reinforcement learning with a Top-K algorithm to reduce action space and efficiently identify effective attack strategies for heterogeneous graphs.

Result: Experiments on multiple heterogeneous graph datasets show significant reductions in classification accuracy compared to baseline methods. Ablation study confirms the critical role of the Top-K algorithm in enhancing attack performance.

Conclusion: The findings highlight potential vulnerabilities in current GNN models on heterogeneous graphs and provide guidance for developing future defense strategies against adversarial attacks in this domain.

Abstract: Graph Neural Networks (GNNs) have attracted substantial interest due to their exceptional performance on graph-based data. However, their robustness, especially on heterogeneous graphs, remains underexplored, particularly against adversarial attacks. This paper proposes HeteroKRLAttack, a targeted evasion black-box attack method for heterogeneous graphs. By integrating reinforcement learning with a Top-K algorithm to reduce the action space, our method efficiently identifies effective attack strategies to disrupt node classification tasks. We validate the effectiveness of HeteroKRLAttack through experiments on multiple heterogeneous graph datasets, showing significant reductions in classification accuracy compared to baseline methods. An ablation study underscores the critical role of the Top-K algorithm in enhancing attack performance. Our findings highlight potential vulnerabilities in current models and provide guidance for future defense strategies against adversarial attacks on heterogeneous graphs.

Method: The authors conducted a comprehensive investigation into topological properties of publicly available biomedical Knowledge Graphs and established links to observed accuracy in real-world tasks.

Result: The study provides insights into how graph properties influence task performance and releases all model predictions with a new suite of analysis tools for community use.

Conclusion: The work invites the research community to build upon their findings to improve understanding of Knowledge Graph Completion applications in biomedical research.

Abstract: Knowledge Graph Completion has been increasingly adopted as a useful method for helping address several tasks in biomedical research, such as drug repurposing or drug-target identification. To that end, a variety of datasets and Knowledge Graph Embedding models have been proposed over the years. However, little is known about the properties that render a dataset, and associated modelling choices, useful for a given task. Moreover, even though theoretical properties of Knowledge Graph Embedding models are well understood, their practical utility in this field remains controversial. In this work, we conduct a comprehensive investigation into the topological properties of publicly available biomedical Knowledge Graphs and establish links to the accuracy observed in real-world tasks. By releasing all model predictions and a new suite of analysis tools we invite the community to build upon our work and continue improving the understanding of these crucial applications.

Method: Two-stage framework: 1) Prompt-driven data selection via explicit alignment to filter irrelevant/redundant data, 2) Decomposed difficulty data selection using three metrics (Instruction Understanding, Response Confidence, Response Correctness) with attention-based importance weighting.

Result: Experiments on real-world healthcare datasets show 3DS outperforms existing methods by over 5.29% in accuracy.

Conclusion: 3DS provides more effective domain adaptation by ensuring selected data aligns with the model’s knowledge distribution and provides appropriate learning challenges, leading to superior performance in specialized domains.

Abstract: Large Language Models(LLMs) excel in general tasks but struggle in specialized domains like healthcare due to limited domain-specific knowledge.Supervised Fine-Tuning(SFT) data construction for domain adaptation often relies on heuristic methods, such as GPT-4 annotation or manual data selection, with a data-centric focus on presumed diverse, high-quality datasets. However, these methods overlook the model’s inherent knowledge distribution, introducing noise, redundancy, and irrelevant data, leading to a mismatch between the selected data and the model’s learning task, resulting in suboptimal performance. To address this, we propose a two-stage model-centric data selection framework, Decomposed Difficulty Data Selection (3DS), which aligns data with the model’s knowledge distribution for optimized adaptation. In Stage1, we apply Prompt-Driven Data Selection via Explicit Alignment, where the the model filters irrelevant or redundant data based on its internal knowledge. In Stage2, we perform Decomposed Difficulty Data Selection, where data selection is guided by our defined difficulty decomposition, using three metrics: Instruction Understanding, Response Confidence, and Response Correctness. Additionally, an attention-based importance weighting mechanism captures token importance for more accurate difficulty calibration. This two-stage approach ensures the selected data is not only aligned with the model’s knowledge and preferences but also appropriately challenging for the model to learn, leading to more effective and targeted domain adaptation. In the case study of the medical domain, our extensive experiments on real-world healthcare datasets demonstrate the superiority of 3DS over exisiting methods in accuracy by over 5.29%. Our dataset and code has been open-sourced at https://github.com/PuppyKnightUniversity/3DS.

Method: Proposes Retrieval-Retro which retrieves reference materials from a knowledge base and uses various attention layers to implicitly extract precursor information. The method considers thermodynamic relationships between target materials and precursors during retrieval to identify the most probable precursor sets.

Result: Extensive experiments show Retrieval-Retro’s superiority in retrosynthesis planning, particularly in discovering novel synthesis recipes which is crucial for materials discovery.

Conclusion: The proposed approach effectively addresses the challenges of inorganic retrosynthesis planning by incorporating domain expertise through thermodynamic considerations and attention mechanisms, enabling better discovery of novel synthesis pathways.

Abstract: While inorganic retrosynthesis planning is essential in the field of chemical science, the application of machine learning in this area has been notably less explored compared to organic retrosynthesis planning. In this paper, we propose Retrieval-Retro for inorganic retrosynthesis planning, which implicitly extracts the precursor information of reference materials that are retrieved from the knowledge base regarding domain expertise in the field. Specifically, instead of directly employing the precursor information of reference materials, we propose implicitly extracting it with various attention layers, which enables the model to learn novel synthesis recipes more effectively. Moreover, during retrieval, we consider the thermodynamic relationship between target material and precursors, which is essential domain expertise in identifying the most probable precursor set among various options. Extensive experiments demonstrate the superiority of Retrieval-Retro in retrosynthesis planning, especially in discovering novel synthesis recipes, which is crucial for materials discovery. The source code for Retrieval-Retro is available at https://github.com/HeewoongNoh/Retrieval-Retro.

Method: RISN integrates residual connections with high-accuracy numerical methods like Gaussian quadrature and fractional derivative operational matrices. The residual connections help mitigate vanishing gradient issues, enabling deeper networks and better handling of complex kernels.

Result: Extensive experiments show RISN consistently outperforms classical PINNs and advanced variants (A-PINN, SA-PINN), achieving significantly lower Mean Absolute Errors across various equation types including 1D, multi-dimensional, fractional, and Helmholtz-type integral equations.

Conclusion: RISN demonstrates robustness and efficiency in solving challenging integral and integro-differential problems, making it a valuable tool for real-world applications where traditional methods often struggle.

Abstract: In this paper, we present the Residual Integral Solver Network (RISN), a novel neural network architecture designed to solve a wide range of integral and integro-differential equations, including one-dimensional, multi-dimensional, ordinary and partial integro-differential, systems, fractional types, and Helmholtz-type integral equations involving oscillatory kernels. RISN integrates residual connections with high-accuracy numerical methods such as Gaussian quadrature and fractional derivative operational matrices, enabling it to achieve higher accuracy and stability than traditional Physics-Informed Neural Networks (PINN). The residual connections help mitigate vanishing gradient issues, allowing RISN to handle deeper networks and more complex kernels, particularly in multi-dimensional problems. Through extensive experiments, we demonstrate that RISN consistently outperforms not only classical PINNs but also advanced variants such as Auxiliary PINN (A-PINN) and Self-Adaptive PINN (SA-PINN), achieving significantly lower Mean Absolute Errors (MAE) across various types of equations. These results highlight RISN’s robustness and efficiency in solving challenging integral and integro-differential problems, making it a valuable tool for real-world applications where traditional methods often struggle.

Method: Proposes superposing task-specific features from individual models into a merged model by targeting linear transformation matrices and formulating merging as a linear system to preserve task-specific features.

Result: Extensive experiments across diverse benchmarks and models demonstrate that the method outperforms existing techniques in maintaining multi-task capabilities.

Conclusion: The approach effectively creates merged models that maintain multi-task capabilities by preserving task-specific features through linear transformation matrix superposition.

Abstract: Model merging enables powerful capabilities in neural networks without requiring additional training. In this paper, we introduce a novel perspective on model merging by leveraging the fundamental mechanisms of neural network representation. Our approach is motivated by the linear representation hypothesis, which states that neural networks encode information through linear combinations of feature vectors. We propose a method that superposes task-specific features from individual models into a merged model. Our approach specifically targets linear transformation matrices, which are crucial for feature activation and extraction in deep networks. By formulating the merging process as a linear system, we can preserve task-specific features from individual models and create merged models that effectively maintain multi-task capabilities compared to existing methods. Extensive experiments across diverse benchmarks and models demonstrate that our method outperforms existing techniques. Code is available at https://github.com/LARS-research/STF.

Method: Two key contributions: 1) Extend zero-shot LLMs for reward shaping beyond NLP to continuous control tasks, eliminating need for biased surrogate models. 2) Introduce LLM-HFBF hybrid framework where LLMs identify and correct biases in human feedback while incorporating it into reward shaping.

Result: Biased human feedback reduces performance by nearly 94% in Average Episodic Reward compared to unbiased approaches. LLM-based methods sustain performance similar to unbiased feedback, even in challenging edge-case scenarios.

Conclusion: The LLM-HFBF framework creates a more balanced and reliable system by addressing limitations of both LLMs (lack of domain knowledge) and human supervision (inherent biases), improving RL performance and reducing reliance on potentially biased human feedback.

Abstract: Reinforcement learning (RL) often struggles with reward misalignment, where agents optimize given rewards but fail to exhibit the desired behaviors. This arises when the reward function incentivizes proxy behaviors misaligned with the true objective. While human-in-the-loop (HITL) methods can mitigate this issue, they also introduce biases, leading to inconsistent and subjective feedback that complicates learning. To address these challenges, we propose two key contributions. First, we extend the use of zero-shot, off-the-shelf large language models (LLMs) for reward shaping beyond natural language processing (NLP) to continuous control tasks. Using LLMs as direct feedback providers eliminates the need for surrogate models trained on human feedback, which often inherit biases from training data. Second, we introduce a hybrid framework (LLM-HFBF) that enables LLMs to identify and correct biases in human feedback while incorporating this feedback into the reward shaping process. The LLM-HFBF framework creates a more balanced and reliable system by addressing both the limitations of LLMs (e.g., lack of domain-specific knowledge) and human supervision (e.g., inherent biases). By enabling human feedback bias flagging and correction, our approach improves reinforcement learning performance and reduces reliance on potentially biased human feedback. Empirical experiments show that biased human feedback significantly reduces performance, with Average Episodic Reward dropping by nearly 94% compared to unbiased approaches. In contrast, LLM-based methods sustain performance at a similar level to unbiased feedback, even in challenging edge-case scenarios.

Method: Proposes a unified MML framework that decomposes LLMs into three interdependent components: modular representation, modular model, and modular reasoning, leveraging techniques like disentangled representation learning, neural architecture search, and neuro-symbolic learning.

Result: The MML paradigm can clarify LLM internal mechanisms through semantic disentanglement, enable flexible task-adaptive model design, and support interpretable logic-driven decision-making processes.

Conclusion: MML integration with LLMs has potential to bridge statistical learning and formal reasoning, paving the way for robust, adaptable, and trustworthy AI systems across real-world applications, though challenges remain in neural-symbolic integration, joint optimization, and scalability.

Abstract: Large language models (LLMs) have substantially advanced machine learning research, including natural language processing, computer vision, data mining, etc., yet they still exhibit critical limitations in explainability, reliability, adaptability, and extensibility. In this paper, we overview a promising learning paradigm, i.e., Modular Machine Learning (MML), as an essential approach toward new-generation LLMs capable of addressing these issues. We begin by systematically and comprehensively surveying the existing literature on modular machine learning, with a particular focus on modular data representation and modular models. Then, we propose a unified MML framework for LLMs, which decomposes the complex structure of LLMs into three interdependent components: modular representation, modular model, and modular reasoning. Specifically, the MML paradigm discussed in this article is able to: i) clarify the internal working mechanism of LLMs through the disentanglement of semantic components; ii) allow for flexible and task-adaptive model design; iii) enable an interpretable and logic-driven decision-making process. We further elaborate a feasible implementation of MML-based LLMs via leveraging advanced techniques such as disentangled representation learning, neural architecture search and neuro-symbolic learning. Last but not least, we critically identify the remaining key challenges, such as the integration of continuous neural and discrete symbolic processes, joint optimization, and computational scalability, present promising future research directions that deserve further exploration. Ultimately, we believe the integration of the MML with LLMs has the potential to bridge the gap between statistical (deep) learning and formal (logical) reasoning, thereby paving the way for robust, adaptable, and trustworthy AI systems across a wide range of real-world applications.

Method: Three isolation approaches: Voronoi-iForest (general solution), RuzHash-iForest (avoids explicit distance computation via Local Sensitive Hashing), and Sliding-PIF (leverages locality prior for improved efficiency and effectiveness).

Result: A general anomaly detection framework that embeds data into high-dimensional preference space by fitting low-dimensional manifolds and identifies anomalies as isolated points.

Conclusion: PIF provides an effective framework combining adaptive isolation methods with preference embedding flexibility for detecting anomalies in structured pattern data.

Abstract: We address the problem of detecting anomalies as samples that do not conform to structured patterns represented by low-dimensional manifolds. To this end, we conceive a general anomaly detection framework called Preference Isolation Forest (PIF), that combines the benefits of adaptive isolation-based methods with the flexibility of preference embedding. The key intuition is to embed the data into a high-dimensional preference space by fitting low-dimensional manifolds, and to identify anomalies as isolated points. We propose three isolation approaches to identify anomalies: $i$) Voronoi-iForest, the most general solution, $ii$) RuzHash-iForest, that avoids explicit computation of distances via Local Sensitive Hashing, and $iii$) Sliding-PIF, that leverages a locality prior to improve efficiency and effectiveness.

Method: Uses algorithmic information theory with binarized neural networks. Applies the Block Decomposition Method (BDM) as a scalable approximation of algorithmic complexity based on algorithmic probability.

Result: BDM more closely tracks structural changes during training than entropy, showing stronger correlations with training loss across different model sizes and randomized training runs.

Conclusion: Training is a process of algorithmic compression where learning corresponds to internalizing structured regularities. This offers a principled framework for complexity-aware learning and regularization.

Abstract: Understanding and controlling the informational complexity of neural networks is a central challenge in machine learning, with implications for generalization, optimization, and model capacity. While most approaches rely on entropy-based loss functions and statistical metrics, these measures often fail to capture deeper, causally relevant algorithmic regularities embedded in network structure. We propose a shift toward algorithmic information theory, using Binarized Neural Networks (BNNs) as a first proxy. Grounded in algorithmic probability (AP) and the universal distribution it defines, our approach characterizes learning dynamics through a formal, causally grounded lens. We apply the Block Decomposition Method (BDM) – a scalable approximation of algorithmic complexity based on AP – and demonstrate that it more closely tracks structural changes during training than entropy, consistently exhibiting stronger correlations with training loss across varying model sizes and randomized training runs. These results support the view of training as a process of algorithmic compression, where learning corresponds to the progressive internalization of structured regularities. In doing so, our work offers a principled estimate of learning progression and suggests a framework for complexity-aware learning and regularization, grounded in first principles from information theory, complexity, and computability.

Method: Systematic comparison of four supervised learning models (Logistic Regression, Random Forest, LightGBM, GRU network) on a large-scale imbalanced online transaction dataset, evaluating both overall and class-specific metrics including precision, recall, and F1-scores.

Result: Ensemble methods (Random Forest and LightGBM) demonstrated superior performance across metrics. GRU showed strong recall for fraud class but low precision. Logistic Regression provided reliable and interpretable baseline performance.

Conclusion: Model selection should be based on specific risk tolerance and operational needs, with ensemble methods offering best overall performance and GRU being suitable when high fraud recall is prioritized over precision.

Abstract: Fraud detection remains a critical task in high-stakes domains such as finance and e-commerce, where undetected fraudulent transactions can lead to significant economic losses. In this study, we systematically compare the performance of four supervised learning models - Logistic Regression, Random Forest, Light Gradient Boosting Machine (LightGBM), and a Gated Recurrent Unit (GRU) network - on a large-scale, highly imbalanced online transaction dataset. While ensemble methods such as Random Forest and LightGBM demonstrated superior performance in both overall and class-specific metrics, Logistic Regression offered a reliable and interpretable baseline. The GRU model showed strong recall for the minority fraud class, though at the cost of precision, highlighting a trade-off relevant for real-world deployment. Our evaluation emphasizes not only weighted averages but also per-class precision, recall, and F1-scores, providing a nuanced view of each model’s effectiveness in detecting rare but consequential fraudulent activity. The findings underscore the importance of choosing models based on the specific risk tolerance and operational needs of fraud detection systems.

Method: Explainable Deep Reinforcement Learning framework built on near-real-time RAN intelligent controller, integrating feature-based approach with Shapley values and attention mechanism to interpret and refine reinforcement learning decisions.

Result: Provides clear real-time insights into resource allocation, achieves higher interpretability precision than pure attention mechanism. QoS satisfaction for URLLC increased from 78.0% to 80.13%, and for eMBB from 71.44% to 73.21%.

Conclusion: The XRL framework successfully enhances both interpretability and QoS performance in vehicular network resource allocation, demonstrating practical improvements for both URLLC and eMBB services.

Abstract: Effective resource management and network slicing are essential to meet the diverse service demands of vehicular networks, including Enhanced Mobile Broadband (eMBB) and Ultra-Reliable and Low-Latency Communications (URLLC). This paper introduces an Explainable Deep Reinforcement Learning (XRL) framework for dynamic network slicing and resource allocation in vehicular networks, built upon a near-real-time RAN intelligent controller. By integrating a feature-based approach that leverages Shapley values and an attention mechanism, we interpret and refine the decisions of our reinforcementlearning agents, addressing key reliability challenges in vehicular communication systems. Simulation results demonstrate that our approach provides clear, real-time insights into the resource allocation process and achieves higher interpretability precision than a pure attention mechanism. Furthermore, the Quality of Service (QoS) satisfaction for URLLC services increased from 78.0% to 80.13%, while that for eMBB services improved from 71.44% to 73.21%.

Method: Combines Minkowski and Chebyshev distances to create an intermediary distance metric. Evaluates performance through neighborhood iteration speed tests and k-NN classifier accuracy analysis using 33 UCI datasets, 15 different distances, and k values from 1 to 200.

Result: The proposed distance is 1.3x faster than Manhattan and 329.5x faster than Euclidean in discrete neighborhood iterations. Achieved better-than-average accuracy in 26/33 cases and best accuracy in 9/33 cases across datasets.

Conclusion: The hybrid Minkowski-Chebyshev distance provides significant computational efficiency improvements while maintaining competitive classification performance, making it a practical alternative to traditional distance metrics for k-NN applications.

Abstract: This work proposes a distance that combines Minkowski and Chebyshev distances and can be seen as an intermediary distance. This combination not only achieves efficient run times in neighbourhood iteration tasks in Z^2, but also obtains good accuracies when coupled with the k-Nearest Neighbours (k-NN) classifier. The proposed distance is approximately 1.3 times faster than Manhattan distance and 329.5 times faster than Euclidean distance in discrete neighbourhood iterations. An accuracy analysis of the k-NN classifier using a total of 33 datasets from the UCI repository, 15 distances and values assigned to k that vary from 1 to 200 is presented. In this experiment, the proposed distance obtained accuracies that were better than the average more often than its counterparts (in 26 cases out of 33), and also obtained the best accuracy more frequently (in 9 out of 33 cases).

Method: Developed a general design pipeline considering data, architecture, and learning strategy to enable synergy between classical optimization and learning to optimize. Applied these principles to design a new learning-enhanced BFGS algorithm.

Result: The learned algorithms perform well far beyond problems from the training distribution, with numerical experiments showing the new learning-enhanced BFGS algorithm adapts successfully to many settings at test time.

Conclusion: By incorporating key principles from classical optimization into the L2O framework, the authors created more generalizable learned optimization algorithms that bridge the gap between classical methods and machine learning approaches.

Abstract: Towards designing learned optimization algorithms that are usable beyond their training setting, we identify key principles that classical algorithms obey, but have up to now, not been used for Learning to Optimize (L2O). Following these principles, we provide a general design pipeline, taking into account data, architecture and learning strategy, and thereby enabling a synergy between classical optimization and L2O, resulting in a philosophy of Learning Optimization Algorithms. As a consequence our learned algorithms perform well far beyond problems from the training distribution. We demonstrate the success of these novel principles by designing a new learning-enhanced BFGS algorithm and provide numerical experiments evidencing its adaptation to many settings at test time.

Method: FedHGL introduces a pre-propagation hyperedge completion mechanism to preserve high-order structural integrity, uses federated central server for cross-client hypergraph convolution without exposing topology, and incorporates two types of local differential privacy (LDP) mechanisms for privacy protection.

Result: Experimental results on seven real-world datasets confirm the effectiveness of FedHGL and demonstrate performance advantages over traditional federated graph learning methods.

Conclusion: FedHGL successfully bridges the gap between hypergraph learning and federated systems, providing a privacy-preserving framework that maintains high-order structural integrity while ensuring formal privacy guarantees against inference attacks.

Abstract: The rapid growth of graph-structured data necessitates partitioning and distributed storage across decentralized systems, driving the emergence of federated graph learning to collaboratively train Graph Neural Networks (GNNs) without compromising privacy. However, current methods exhibit limited performance when handling hypergraphs, which inherently represent complex high-order relationships beyond pairwise connections. Partitioning hypergraph structures across federated subsystems amplifies structural complexity, hindering high-order information mining and compromising local information integrity. To bridge the gap between hypergraph learning and federated systems, we develop FedHGL, a first-of-its-kind framework for federated hypergraph learning on disjoint and privacy-constrained hypergraph partitions. Beyond collaboratively training a comprehensive hypergraph neural network across multiple clients, FedHGL introduces a pre-propagation hyperedge completion mechanism to preserve high-order structural integrity within each client. This procedure leverages the federated central server to perform cross-client hypergraph convolution without exposing internal topological information, effectively mitigating the high-order information loss induced by subgraph partitioning. Furthermore, by incorporating two kinds of local differential privacy (LDP) mechanisms, we provide formal privacy guarantees for this process, ensuring that sensitive node features remain protected against inference attacks from potentially malicious servers or clients. Experimental results on seven real-world datasets confirm the effectiveness of our approach and demonstrate its performance advantages over traditional federated graph learning methods.

Method: Hybrid autoencoder architecture combining convolutional, graph, and recurrent neural networks, applied to transfer learning between different sections of the Hadron Calorimeter at CERN’s CMS experiment.

Result: Transfer learning enhances model performance while substantially reducing trainable parameters and mitigating data contamination effects in high-dimensional ST anomaly detection.

Conclusion: Transfer learning shows significant potential for spatio-temporal anomaly detection applications, offering improved accuracy and robustness with limited training data, though limitations exist that require careful model initialization and configuration.

Abstract: The proliferation of sensors brings an immense volume of spatio-temporal (ST) data in many domains, including monitoring, diagnostics, and prognostics applications. Data curation is a time-consuming process for a large volume of data, making it challenging and expensive to deploy data analytics platforms in new environments. Transfer learning (TL) mechanisms promise to mitigate data sparsity and model complexity by utilizing pre-trained models for a new task. Despite the triumph of TL in fields like computer vision and natural language processing, efforts on complex ST models for anomaly detection (AD) applications are limited. In this study, we present the potential of TL within the context of high-dimensional ST AD with a hybrid autoencoder architecture, incorporating convolutional, graph, and recurrent neural networks. Motivated by the need for improved model accuracy and robustness, particularly in scenarios with limited training data on systems with thousands of sensors, this research investigates the transferability of models trained on different sections of the Hadron Calorimeter of the Compact Muon Solenoid experiment at CERN. The key contributions of the study include exploring TL’s potential and limitations within the context of encoder and decoder networks, revealing insights into model initialization and training configurations that enhance performance while substantially reducing trainable parameters and mitigating data contamination effects. Code: \href{https://github.com/muleina/CMS_HCAL_ML_OnlineDQM}{https://github.com/muleina/CMS_HCAL_ML_OnlineDQM}

Method: Develops multi-dimensional IoM metric considering model freshness, accuracy, and data value. Designs incentive mechanism using EPEC game theory where MSPs determine rewards and MUs optimize resource allocation. Implements distributed dynamic reward algorithm based on deep reinforcement learning without requiring private information.

Result: Achieves 38.3% and 37.2% improvements in IoM, and reduces training time to reach target accuracy by 43.5% and 49.8% on average for MNIST and GTSRB datasets respectively, outperforming state-of-the-art benchmarks.

Conclusion: The framework effectively incentivizes metaverse users to contribute high-value local models while preserving privacy, providing a flexible and adaptive scheme for data provisioning in vehicular metaverse services.

Abstract: Timely updating of Internet of Things data is crucial for achieving immersion in vehicular metaverse services. However, challenges such as latency caused by massive data transmissions, privacy risks associated with user data, and computational burdens on metaverse service providers (MSPs) hinder the continuous collection of high-quality data. To address these challenges, we propose an immersion-aware model trading framework that enables efficient and privacy-preserving data provisioning through federated learning (FL). Specifically, we first develop a novel multi-dimensional evaluation metric for the immersion of models (IoM). The metric considers the freshness and accuracy of the local model, and the amount and potential value of raw training data. Building on the IoM, we design an incentive mechanism to encourage metaverse users (MUs) to participate in FL by providing local updates to MSPs under resource constraints. The trading interactions between MSPs and MUs are modeled as an equilibrium problem with equilibrium constraints (EPEC) to analyze and balance their costs and gains, where MSPs as leaders determine rewards, while MUs as followers optimize resource allocation. To ensure privacy and adapt to dynamic network conditions, we develop a distributed dynamic reward algorithm based on deep reinforcement learning, without acquiring any private information from MUs and other MSPs. Experimental results show that the proposed framework outperforms state-of-the-art benchmarks, achieving improvements in IoM of 38.3% and 37.2%, and reductions in training time to reach the target accuracy of 43.5% and 49.8%, on average, for the MNIST and GTSRB datasets, respectively. These findings validate the effectiveness of our approach in incentivizing MUs to contribute high-value local models to MSPs, providing a flexible and adaptive scheme for data provisioning in vehicular metaverse services.

Method: Constructed a geo-indexed dataset for US postal codes and counties with aggregated human behavior data (maps, busyness, search trends) and environmental factors (weather, air quality). Used graph neural networks to model complex relationships between locations and produce embeddings adaptable to various downstream tasks.

Result: Achieved state-of-the-art performance on all 27 geospatial interpolation tasks, and on 25 out of 27 extrapolation and super-resolution tasks. Combined with TimesFM forecasting model, surpassed fully supervised forecasting for unemployment and poverty prediction.

Conclusion: The PDFM provides a powerful foundation model for geospatial analysis that can be adapted to diverse tasks without extensive manual feature engineering, demonstrating strong performance across multiple domains and enabling better resource allocation for population health and well-being.

Abstract: Supporting the health and well-being of dynamic populations around the world requires governmental agencies, organizations and researchers to understand and reason over complex relationships between human behavior and local contexts in order to identify high-risk groups and strategically allocate limited resources. Traditional approaches to these classes of problems often entail developing manually curated, task-specific features and models to represent human behavior and the natural and built environment, which can be challenging to adapt to new, or even, related tasks. To address this, we introduce a Population Dynamics Foundation Model (PDFM) that aims to capture the relationships between diverse data modalities and is applicable to a broad range of geospatial tasks. We first construct a geo-indexed dataset for postal codes and counties across the United States, capturing rich aggregated information on human behavior from maps, busyness, and aggregated search trends, and environmental factors such as weather and air quality. We then model this data and the complex relationships between locations using a graph neural network, producing embeddings that can be adapted to a wide range of downstream tasks using relatively simple models. We evaluate the effectiveness of our approach by benchmarking it on 27 downstream tasks spanning three distinct domains: health indicators, socioeconomic factors, and environmental measurements. The approach achieves state-of-the-art performance on all 27 geospatial interpolation tasks, and on 25 out of the 27 extrapolation and super-resolution tasks. We combined the PDFM with a state-of-the-art forecasting foundation model, TimesFM, to predict unemployment and poverty, achieving performance that surpasses fully supervised forecasting. The full set of embeddings and sample code are publicly available for researchers.

Method: Generalized Neural Logic Networks with NOT operations and biases to account for unobserved data. Proposed a novel factorized IF-THEN rule structure and a modified learning algorithm for logical and probabilistic modeling.

Result: The method improves state-of-the-art in Boolean networks discovery and successfully learns relevant, interpretable rules in tabular classification, particularly in medical and industrial applications where interpretability is crucial.

Conclusion: The proposed Neural Logic Networks with enhanced logical operations provide both interpretability and competitive performance, making them valuable for domains requiring transparent decision-making processes.

Abstract: Traditional neural networks have an impressive classification performance, but what they learn cannot be inspected, verified or extracted. Neural Logic Networks on the other hand have an interpretable structure that enables them to learn a logical mechanism relating the inputs and outputs with AND and OR operations. We generalize these networks with NOT operations and biases that take into account unobserved data and develop a rigorous logical and probabilistic modeling in terms of concept combinations to motivate their use. We also propose a novel factorized IF-THEN rule structure for the model as well as a modified learning algorithm. Our method improves the state-of-the-art in Boolean networks discovery and is able to learn relevant, interpretable rules in tabular classification, notably on examples from the medical and industrial fields where interpretability has tangible value.

Method: Introduces Mean-Square Bipower Variation Error (MSBVE) algorithm that minimizes mean-square quadratic variation error instead of traditional TD error, specifically designed for SDEs with jumps.

Result: MSBVE reliably estimates value function in complex jump process environments, demonstrating superior performance over MSTDE through simulations and formal proofs.

Conclusion: Alternative error metrics like quadratic variation error are crucial for improving RL algorithm resilience and effectiveness in continuous-time frameworks with jump processes.

Abstract: Reinforcement Learning (RL) has proven effective in solving complex decision-making tasks across various domains, but challenges remain in continuous-time settings, particularly when state dynamics are governed by stochastic differential equations (SDEs) with jump components. In this paper, we address this challenge by introducing the Mean-Square Bipower Variation Error (MSBVE) algorithm, which enhances robustness and convergence in scenarios involving significant stochastic noise and jumps. We first revisit the Mean-Square TD Error (MSTDE) algorithm, commonly used in continuous-time RL, and highlight its limitations in handling jumps in state dynamics. The proposed MSBVE algorithm minimizes the mean-square quadratic variation error, offering improved performance over MSTDE in environments characterized by SDEs with jumps. Simulations and formal proofs demonstrate that the MSBVE algorithm reliably estimates the value function in complex settings, surpassing MSTDE’s performance when faced with jump processes. These findings underscore the importance of alternative error metrics to improve the resilience and effectiveness of RL algorithms in continuous-time frameworks.

Method: Leverages rejection-sampled universal quantizer (RSUQ) - a randomized vector quantizer that produces distortion equivalent to prescribed noise (Gaussian/Laplace), enabling joint differential privacy and compression. Provides privacy adaptability for customized protection.

Result: Theoretical privacy guarantee analysis and experimental evaluations show trade-offs between privacy and accuracy. On MNIST dataset, CEPAM surpasses baseline models in learning accuracy while maintaining privacy.

Conclusion: CEPAM successfully addresses both communication efficiency and privacy protection in federated learning through its novel RSUQ-based approach, offering customizable privacy levels and demonstrating superior performance compared to baseline models.

Abstract: Training machine learning models on decentralized private data via federated learning (FL) poses two key challenges: communication efficiency and privacy protection. In this work, we address these challenges within the trusted aggregator model by introducing a novel approach called the Communication-Efficient and Privacy-Adaptable Mechanism (CEPAM), achieving both objectives simultaneously. In particular, CEPAM leverages the rejection-sampled universal quantizer (RSUQ), a construction of randomized vector quantizer whose resulting distortion is equivalent to a prescribed noise, such as Gaussian or Laplace noise, enabling joint differential privacy and compression. Our CEPAM provides the additional benefit of privacy adaptability, allowing clients and the server to customize privacy protection based on required accuracy and protection. We theoretically analyze the privacy guarantee of CEPAM and investigate the trade-offs among user privacy and accuracy of CEPAM through experimental evaluations. Moreover, we assess CEPAM’s utility performance using MNIST dataset, demonstrating that CEPAM surpasses baseline models in terms of learning accuracy.

Method: The paper provides a compact introduction and necessary definitions to show that the ensemble update step in EnKF is equivalent to an empirical version of the Matheron update used in Gaussian process regression.

Result: The research establishes that these two approaches from different fields (data assimilation and GP regression) are fundamentally the same mathematical operation, connecting decades of engineering practice with modern statistical methods.

Conclusion: This connection links half a century of data-assimilation engineering to modern path-wise GP sampling, providing new insights and potential cross-pollination between these fields.

Abstract: The Ensemble Kalman Filter (EnKF) is a widely used method for data assimilation in high-dimensional systems, with an ensemble update step equivalent to an empirical version of the Matheron update popular in Gaussian process regression – a connection that links half a century of data-assimilation engineering to modern path-wise GP sampling. This paper provides a compact introduction to this simple but under-exploited connection, with necessary definitions accessible to all fields involved. Source code is available at https://github.com/danmackinlay/paper_matheron_equals_enkf .

Method: Proposes AR-KAN which integrates a pre-trained AR module for temporal memory preservation with a Kolmogorov-Arnold Network (KAN) for nonlinear representation, reducing redundancy while maintaining essential temporal features.

Result: AR-KAN matches ARIMA performance on almost-periodic functions and achieves best results on 72% of Rdatasets series, with particular advantage on data with periodic structure.

Conclusion: AR-KAN is a robust and effective framework for time series forecasting that successfully combines the strengths of autoregressive modeling with neural network nonlinear representation capabilities.

Abstract: Traditional neural networks struggle to capture the spectral structure of complex signals. Fourier neural networks (FNNs) attempt to address this by embedding Fourier series components, yet many real-world signals are almost-periodic with non-commensurate frequencies, posing additional challenges. Building on prior work showing that ARIMA outperforms large language models (LLMs) for forecasting, we extend the comparison to neural predictors and find ARIMA still superior. We therefore propose the Autoregressive-Weight-Enhanced Kolmogorov-Arnold Network (AR-KAN), which integrates a pre-trained AR module for temporal memory with a KAN for nonlinear representation. The AR module preserves essential temporal features while reducing redundancy. Experiments demonstrate that AR-KAN matches ARIMA on almost-periodic functions and achieves the best results on $72%$ of Rdatasets series, with a clear advantage on data with periodic structure. These results highlight AR-KAN as a robust and effective framework for time series forecasting.

Method: Train transformer neural networks on predicting permutations from words formed by transpositions in smaller symmetric groups (S10), using identity augmentation for variable word lengths and partitioned windows for adjacent transpositions.

Result: Models trained on S10 generalized to S25 with near 100% accuracy, and from S10 to S16 with similar performance using only adjacent transpositions.

Conclusion: The method demonstrates that training on simpler versions of mathematical tasks enables generalization to more complex versions, providing a scalable approach for machine learning in pure mathematics.

Abstract: Machine learning explorations can make significant inroads into solving difficult problems in pure mathematics. One advantage of this approach is that mathematical datasets do not suffer from noise, but a challenge is the amount of data required to train these models and that this data can be computationally expensive to generate. Key challenges further comprise difficulty in a posteriori interpretation of statistical models and the implementation of deep and abstract mathematical problems. We propose a method for scalable tasks, by which models trained on simpler versions of a task can then generalize to the full task. Specifically, we demonstrate that a transformer neural-network trained on predicting permutations from words formed by general transpositions in the symmetric group $S_{10}$ can generalize to the symmetric group $S_{25}$ with near 100% accuracy. We also show that $S_{10}$ generalizes to $S_{16}$ with similar performance if we only use adjacent transpositions. We employ identity augmentation as a key tool to manage variable word lengths, and partitioned windows for training on adjacent transpositions. Finally we compare variations of the method used and discuss potential challenges with extending the method to other tasks.

Method: Introduced a quantitative framework using three separability measures (dprime, silhouette coefficients, ROC-AUC) to evaluate various similarity metrics (RSA, linear predictivity, Procrustes, soft matching) across different architectures (CNNs, Vision Transformers, Swin Transformers, ConvNeXt) and training regimes.

Result: Separability increases with more stringent alignment constraints; soft-matching achieved highest separability among mapping-based approaches, followed by Procrustes and linear predictivity; non-fitting methods like RSA also showed strong separability.

Conclusion: This study provides the first systematic comparison of similarity metrics through separability analysis, clarifying their relative sensitivity and guiding metric selection for model and brain comparisons.

Abstract: Representational similarity metrics are fundamental tools in neuroscience and AI, yet we lack systematic comparisons of their discriminative power across model families. We introduce a quantitative framework to evaluate representational similarity measures based on their ability to separate model families-across architectures (CNNs, Vision Transformers, Swin Transformers, ConvNeXt) and training regimes (supervised vs. self-supervised). Using three complementary separability measures-dprime from signal detection theory, silhouette coefficients and ROC-AUC, we systematically assess the discriminative capacity of commonly used metrics including RSA, linear predictivity, Procrustes, and soft matching. We show that separability systematically increases as metrics impose more stringent alignment constraints. Among mapping-based approaches, soft-matching achieves the highest separability, followed by Procrustes alignment and linear predictivity. Non-fitting methods such as RSA also yield strong separability across families. These results provide the first systematic comparison of similarity metrics through a separability lens, clarifying their relative sensitivity and guiding metric choice for large-scale model and brain comparisons.

Method: Introduces two adversarial analysis strategies: prototype manipulation and backdoor attacks against prototype-based networks, and examines how concept bottleneck models defend against these attacks.

Result: Demonstrates that prototype-based networks can be fooled by exploiting latent prototypes, revealing inherent uninterpretability and creating a false sense of security through visual confirmation bias.

Conclusion: The limitations of part-prototype networks question their trustworthiness and applicability, motivating further research on robustness and alignment of interpretable deep learning models.

Abstract: A common belief is that intrinsically interpretable deep learning models ensure a correct, intuitive understanding of their behavior and offer greater robustness against accidental errors or intentional manipulation. However, these beliefs have not been comprehensively verified, and growing evidence casts doubt on them. In this paper, we highlight the risks related to overreliance and susceptibility to adversarial manipulation of these so-called “intrinsically (aka inherently) interpretable” models by design. We introduce two strategies for adversarial analysis with prototype manipulation and backdoor attacks against prototype-based networks, and discuss how concept bottleneck models defend against these attacks. Fooling the model’s reasoning by exploiting its use of latent prototypes manifests the inherent uninterpretability of deep neural networks, leading to a false sense of security reinforced by a visual confirmation bias. The reported limitations of part-prototype networks put their trustworthiness and applicability into question, motivating further work on the robustness and alignment of (deep) interpretable models.

Method: Used a single neural network controlling both agents in a symmetrical grid world avoidance game, training them to reach target destinations while avoiding crashes.

Result: The trained network exhibited behavior similar to Hawks and Doves game - one agent adopted an aggressive strategy while the other learned to avoid the aggressive agent.

Conclusion: Deep learning agents can spontaneously develop complex game-theoretic strategies like Hawks and Doves behavior through training in simple avoidance scenarios.

Abstract: We present heuristically optimal strategies expressed by deep learning agents playing a simple avoidance game. We analyse the learning and behaviour of two agents within a symmetrical grid world that must cross paths to reach a target destination without crashing into each other or straying off of the grid world in the wrong direction. The agent policy is determined by one neural network that is employed in both agents. Our findings indicate that the fully trained network exhibits behaviour similar to that of the game Hawks and Doves, in that one agent employs an aggressive strategy to reach the target while the other learns how to avoid the aggressive agent.

Method: Algorithm inspired by Conservative Q-learning that trains neural control barrier functions from offline datasets to design quadratic program constraints for safety filters.

Result: Conservative Control Barrier Functions (CCBFs) outperform existing methods in maintaining safety while minimally affecting task performance.

Conclusion: The proposed CCBF approach effectively addresses dimensionality issues in safety filter synthesis and provides reliable safety guarantees while preserving task performance.

Abstract: Safety filters, particularly those based on control barrier functions, have gained increased interest as effective tools for safe control of dynamical systems. Existing correct-by-construction synthesis algorithms for such filters, however, suffer from the curse-of-dimensionality. Deep learning approaches have been proposed in recent years to address this challenge. In this paper, we add to this set of approaches an algorithm for training neural control barrier functions from offline datasets. Such functions can be used to design constraints for quadratic programs that are then used as safety filters. Our algorithm trains these functions so that the system is not only prevented from reaching unsafe states but is also disincentivized from reaching out-of-distribution ones, at which they would be less reliable. It is inspired by Conservative Q-learning, an offline reinforcement learning algorithm. We call its outputs Conservative Control Barrier Functions (CCBFs). Our empirical results demonstrate that CCBFs outperform existing methods in maintaining safety while minimally affecting task performance. Source code is available at https://github.com/tabz23/CCBF.

Method: Reformulated PIML training as a multi-objective optimization problem treating data-driven and physics losses independently. Applied multi-gradient descent algorithms (MGDAs) including traditional multi-gradient descent (TMGD) and dual cone gradient descent (DCGD) to explore the Pareto front.

Result: MGDAs achieved comparable performance to traditional methods in macroscopic traffic flow models. In microscopic traffic flow models, MGDAs significantly outperformed scalarization-based approaches, demonstrating superior performance in complex PIML scenarios.

Conclusion: Multi-objective optimization provides a paradigm shift for PIML that overcomes limitations of linear scalarization, particularly excelling in complex scenarios like microscopic traffic flow modeling where traditional methods struggle.

Abstract: Physics-informed machine learning (PIML) is crucial in modern traffic flow modeling because it combines the benefits of both physics-based and data-driven approaches. In conventional PIML, physical information is typically incorporated by constructing a hybrid loss function that combines data-driven loss and physics loss through linear scalarization. The goal is to find a trade-off between these two objectives to improve the accuracy of model predictions. However, from a mathematical perspective, linear scalarization is limited to identifying only the convex region of the Pareto front, as it treats data-driven and physics losses as separate objectives. Given that most PIML loss functions are non-convex, linear scalarization restricts the achievable trade-off solutions. Moreover, tuning the weighting coefficients for the two loss components can be both time-consuming and computationally challenging. To address these limitations, this paper introduces a paradigm shift in PIML by reformulating the training process as a multi-objective optimization problem, treating data-driven loss and physics loss independently. We apply several multi-gradient descent algorithms (MGDAs), including traditional multi-gradient descent (TMGD) and dual cone gradient descent (DCGD), to explore the Pareto front in this multi-objective setting. These methods are evaluated on both macroscopic and microscopic traffic flow models. In the macroscopic case, MGDAs achieved comparable performance to traditional linear scalarization methods. Notably, in the microscopic case, MGDAs significantly outperformed their scalarization-based counterparts, demonstrating the advantages of a multi-objective optimization approach in complex PIML scenarios.

Method: Systematic investigation through extensive experiments evaluating language prompt learning (LPT) vs vision prompt learning (VPT) under various FL configurations, client scales, aggregation strategies, and prompt lengths.

Result: Findings provide practical insights into optimizing prompt learning in federated settings, including strategies for handling coexisting label skew and domain shift, and leveraging both prompt types when resources allow.

Conclusion: The research contributes to broader deployment of VLMs in privacy-preserving environments by offering guidance on robust Federated Prompt Learning implementation.

Abstract: The Vision Language Model (VLM) excels in aligning vision and language representations, and prompt learning has emerged as a key technique for adapting such models to downstream tasks. However, the application of prompt learning with VLM in federated learning (FL) scenarios remains underexplored. This paper systematically investigates the behavioral differences between language prompt learning (LPT) and vision prompt learning (VPT) under data heterogeneity challenges, including label skew and domain shift. We conduct extensive experiments to evaluate the impact of various FL and prompt configurations, such as client scale, aggregation strategies, and prompt length, to assess the robustness of Federated Prompt Learning (FPL). Furthermore, we explore strategies for enhancing prompt learning in complex scenarios where label skew and domain shift coexist, including leveraging both prompt types when computational resources allow. Our findings offer practical insights into optimizing prompt learning in federated settings, contributing to the broader deployment of VLMs in privacy-preserving environments.

Method: Developed a lightweight interface gluing together optimizers and benchmark tasks, with an analysis pipeline for evaluation. Used star discrepancy minimization to obtain representative subsets of 10-30 diverse tasks for each HPO task type (blackbox, multi-fidelity, multi-objective, multi-fidelity-multi-objective).

Result: Created the largest go-to library for HPO evaluation with 3,336 tasks from 5 community benchmark collections and 28 variants of 9 optimizer families. Established baseline results for future comparisons.

Conclusion: CARPS represents an important step in standardizing HPO evaluation, providing efficient benchmarking capabilities through representative task subsets and comprehensive analysis tools.

Abstract: Hyperparameter Optimization (HPO) is crucial to develop well-performing machine learning models. In order to ease prototyping and benchmarking of HPO methods, we propose carps, a benchmark framework for Comprehensive Automated Research Performance Studies allowing to evaluate N optimizers on M benchmark tasks. In this first release of carps, we focus on the four most important types of HPO task types: blackbox, multi-fidelity, multi-objective and multi-fidelity-multi-objective. With 3 336 tasks from 5 community benchmark collections and 28 variants of 9 optimizer families, we offer the biggest go-to library to date to evaluate and compare HPO methods. The carps framework relies on a purpose-built, lightweight interface, gluing together optimizers and benchmark tasks. It also features an analysis pipeline, facilitating the evaluation of optimizers on benchmarks. However, navigating a huge number of tasks while developing and comparing methods can be computationally infeasible. To address this, we obtain a subset of representative tasks by minimizing the star discrepancy of the subset, in the space spanned by the full set. As a result, we propose an initial subset of 10 to 30 diverse tasks for each task type, and include functionality to re-compute subsets as more benchmarks become available, enabling efficient evaluations. We also establish a first set of baseline results on these tasks as a measure for future comparisons. With carps (https://www.github.com/automl/CARP-S), we make an important step in the standardization of HPO evaluation.

Method: A framework where LLMs generate self-edits (restructuring information, specifying hyperparameters, invoking tools) and use reinforcement learning with downstream performance as reward signal to train effective self-edits.

Result: Experiments on knowledge incorporation and few-shot generalization show SEAL is a promising step toward self-directed adaptation.

Conclusion: SEAL enables lasting adaptation through self-generated finetuning data and direct weight updates, moving toward language models capable of self-directed adaptation.

Abstract: Large language models (LLMs) are powerful but static; they lack mechanisms to adapt their weights in response to new tasks, knowledge, or examples. We introduce Self-Adapting LLMs (SEAL), a framework that enables LLMs to self-adapt by generating their own finetuning data and update directives. Given a new input, the model produces a self-edit-a generation that may restructure the information in different ways, specify optimization hyperparameters, or invoke tools for data augmentation and gradient-based updates. Through supervised finetuning (SFT), these self-edits result in persistent weight updates, enabling lasting adaptation. To train the model to produce effective self-edits, we use a reinforcement learning loop with the downstream performance of the updated model as the reward signal. Unlike prior approaches that rely on separate adaptation modules or auxiliary networks, SEAL directly uses the model’s own generation to control its adaptation process. Experiments on knowledge incorporation and few-shot generalization show that SEAL is a promising step toward language models capable of self-directed adaptation. Our website and code is available at https://jyopari.github.io/posts/seal.

Method: The authors develop a novel framework that connects a variant of incremental input-to-state stability with the regularity properties of RL value functions. They use rewards as “test functions” and analyze value functions under adversarial selection of Hölder-continuous reward functions.

Result: The paper establishes an equivalence between δISS of closed-loop systems under a given policy and the regularity of RL-style value functions. This provides a new way to understand stability through value function regularity rather than traditional Lyapunov certificates.

Conclusion: The research demonstrates that value function regularity and its connection to incremental stability can be understood distinctly from traditional Lyapunov-based approaches, offering a new perspective for analyzing stability in reinforcement learning contexts.

Abstract: This paper presents a novel framework for analyzing Incremental-Input-to-State Stability ($\delta$ISS) based on the idea of using rewards as “test functions.” Whereas control theory traditionally deals with Lyapunov functions that satisfy a time-decrease condition, reinforcement learning (RL) value functions are constructed by exponentially decaying a Lipschitz reward function that may be non-smooth and unbounded on both sides. Thus, these RL-style value functions cannot be directly understood as Lyapunov certificates. We develop a new equivalence between a variant of incremental input-to-state stability of a closed-loop system under given a policy, and the regularity of RL-style value functions under adversarial selection of a H"older-continuous reward function. This result highlights that the regularity of value functions, and their connection to incremental stability, can be understood in a way that is distinct from the traditional Lyapunov-based approach to certifying stability in control theory.

Method: Extended DBNs with trainable differentiable interconnect with constant parameter growth, plus two pruning stages: SAT-based logic equivalence removal and similarity-based data-driven compression.

Result: Achieved scalable DBNs that can handle wider layers than previous designs while preserving accuracy, with pruning providing superior compression-accuracy trade-off compared to magnitude-style greedy baselines.

Conclusion: The proposed trainable interconnect and two-stage pruning approach enables efficient scaling of differentiable Boolean logic networks while maintaining performance advantages and offering better model compression.

Abstract: Learned Differentiable Boolean Logic Networks (DBNs) already deliver efficient inference on resource-constrained hardware. We extend them with a trainable, differentiable interconnect whose parameter count remains constant as input width grows, allowing DBNs to scale to far wider layers than earlier learnable-interconnect designs while preserving their advantageous accuracy. To further reduce model size, we propose two complementary pruning stages: an SAT-based logic equivalence pass that removes redundant gates without affecting performance, and a similarity-based, data-driven pass that outperforms a magnitude-style greedy baseline and offers a superior compression-accuracy trade-off.

Method: Allocates different numbers of global models to clients based on their computational resources, introduces a novel aggregation scheme to mitigate training bias, and dynamically adjusts sparsification ratios across clients to reduce computational burden.

Result: Extensive experiments show SHEFL effectively addresses computational heterogeneity, significantly improving accuracy and stability compared to existing approaches.

Conclusion: SHEFL provides an effective solution for federated learning in heterogeneous environments by combining ensemble methods with resource-aware model allocation and dynamic sparsification.

Abstract: Federated learning (FL) enables distributed training with private client data, but its convergence is hindered by system heterogeneity under realistic communication scenarios. Most FL schemes addressing system heterogeneity utilize global pruning or ensemble distillation, yet often overlook typical constraints required for communication efficiency. Meanwhile, deep ensembles can aggregate predictions from individually trained models to improve performance, but current ensemble-based FL methods fall short in fully capturing diversity of model predictions. In this work, we propose \textbf{SHEFL}, a global ensemble-based FL framework suited for clients with diverse computational capacities. We allocate different numbers of global models to clients based on their available resources. We introduce a novel aggregation scheme that mitigates the training bias between clients and dynamically adjusts the sparsification ratio across clients to reduce the computational burden of training deep ensembles. Extensive experiments demonstrate that our method effectively addresses computational heterogeneity, significantly improving accuracy and stability compared to existing approaches.

Method: BaVerLy dynamically constructs mini-batches of epsilon-balls with similar network computations, verifies them jointly, and uses adaptive refinement when robustness cannot be proven for a batch.

Result: BaVerLy achieves 2.3x average speedup (up to 4.1x) over traditional verification, reducing analysis time from 24 hours to 6 hours in best cases on MNIST and CIFAR-10 networks.

Conclusion: Group verification through adaptive mini-batching significantly accelerates local robustness analysis while maintaining soundness and completeness, making comprehensive neural network verification more practical.

Abstract: Neural network image classifiers are ubiquitous in many safety-critical applications. However, they are susceptible to adversarial attacks. To understand their robustness to attacks, many local robustness verifiers have been proposed to analyze $\epsilon$-balls of inputs. Yet, existing verifiers introduce a long analysis time or lose too much precision, making them less effective for a large set of inputs. In this work, we propose a new approach to local robustness: group local robustness verification. The key idea is to leverage the similarity of the network computations of certain $\epsilon$-balls to reduce the overall analysis time. We propose BaVerLy, a sound and complete verifier that boosts the local robustness verification of a set of $\epsilon$-balls by dynamically constructing and verifying mini-batches. BaVerLy adaptively identifies successful mini-batch sizes, accordingly constructs mini-batches of $\epsilon$-balls that have similar network computations, and verifies them jointly. If a mini-batch is verified, all its $\epsilon$-balls are proven robust. Otherwise, one $\epsilon$-ball is suspected as not being robust, guiding the refinement. BaVerLy leverages the analysis results to expedite the analysis of that $\epsilon$-ball as well as the analysis of the mini-batch with the other $\epsilon$-balls. We evaluate BaVerLy on fully connected and convolutional networks for MNIST and CIFAR-10. Results show that BaVerLy scales the common one by one verification by 2.3x on average and up to 4.1x, in which case it reduces the total analysis time from 24 hours to 6 hours.

Method: Proposes FairGuide framework with differentiable community detection as pseudo downstream task, uses meta-gradients from fairness-guidance objective to identify new links that enhance structural fairness.

Result: Extensive experiments show FairGuide is effective and generalizable across various graph-based fairness tasks, with theoretical analysis confirming fairness optimization in pseudo task improves structural fairness.

Conclusion: FairGuide successfully addresses graph fairness issues by structurally guiding graphs toward unbiased configurations through strategic link additions, demonstrating strong performance and generalizability in fairness enhancement.

Abstract: Graph Neural Networks (GNNs) have achieved remarkable success across diverse applications. However, due to the biases in the graph structures, graph neural networks face significant challenges in fairness. Although the original user graph structure is generally biased, it is promising to guide these existing structures toward unbiased ones by introducing new links. The fairness guidance via new links could foster unbiased communities, thereby enhancing fairness in downstream applications. To address this issue, we propose a novel framework named FairGuide. Specifically, to ensure fairness in downstream tasks trained on fairness-guided graphs, we introduce a differentiable community detection task as a pseudo downstream task. Our theoretical analysis further demonstrates that optimizing fairness within this pseudo task effectively enhances structural fairness, promoting fairness generalization across diverse downstream applications. Moreover, FairGuide employs an effective strategy which leverages meta-gradients derived from the fairness-guidance objective to identify new links that significantly enhance structural fairness. Extensive experimental results demonstrate the effectiveness and generalizability of our proposed method across a variety of graph-based fairness tasks.

Method: Proposes Condition-based Latent Diffusion Model (CbLDM) that uses conditional prior to estimate posterior distribution, reduces sampling steps, and employs Laplacian matrix instead of distance matrix for reconstruction.

Result: CbLDM demonstrates significantly higher prediction accuracy compared to existing models for nanostructure inverse problem.

Conclusion: CbLDM effectively solves nanostructure inverse problems and shows potential for other continuous conditional generation tasks.

Abstract: Nowadays, the nanostructure inverse problem is an attractive problem that helps researchers to understand the relationship between the properties and the structure of nanomaterials. This article focuses on the problem of using PDF to recover the nanostructure, which this article views as a conditional generation problem. This article propose a deep learning model CbLDM, Condition-based Latent Diffusion Model. Based on the original latent diffusion model, the sampling steps of the diffusion model are reduced and the sample generation efficiency is improved by using the conditional prior to estimate conditional posterior distribution, which is the approximated distribution of p(z|x). In addition, this article uses the Laplacian matrix instead of the distance matrix to recover the nanostructure, which can reduce the reconstruction error. Finally, this article compares CbLDM with existing models which were used to solve the nanostructure inverse problem, and find that CbLDM demonstrates significantly higher prediction accuracy than these models, which reflects the ability of CbLDM to solve the nanostructure inverse problem and the potential to cope with other continuous conditional generation tasks.

Method: Maintains fixed hierarchical groups to consolidate new adapters, trains low-rank adapters with importance scalars per task, dynamically groups tasks based on adapter similarity, and performs pruning, scaling and merging within groups to facilitate transfer learning.

Result: Extensive experiments on three vision benchmarks show HAM significantly outperforms state-of-the-art methods, particularly as the number of tasks increases.

Conclusion: HAM provides an effective solution for scalable continual learning by dynamically combining adapters through hierarchical grouping, demonstrating superior performance in managing multiple tasks while mitigating catastrophic forgetting.

Abstract: Continual learning is an essential capability of human cognition, yet it poses significant challenges for current deep learning models. The primary issue is that new knowledge can interfere with previously learned information, causing the model to forget earlier knowledge in favor of the new, a phenomenon known as catastrophic forgetting. Although large pre-trained models can partially mitigate forgetting by leveraging their existing knowledge and over-parameterization, they often struggle when confronted with novel data distributions. Parameter-Efficient Fine-Tuning (PEFT) methods, such as LoRA, enable efficient adaptation to new knowledge. However, they still face challenges in scaling to dynamic learning scenarios and long sequences of tasks, as maintaining one adapter per task introduces complexity and increases the potential for interference. In this paper, we introduce Hierarchical Adapters Merging (HAM), a novel framework that dynamically combines adapters from different tasks during training. This approach enables HAM to scale effectively, allowing it to manage more tasks than competing baselines with improved efficiency. To achieve this, HAM maintains a fixed set of groups that hierarchically consolidate new adapters. For each task, HAM trains a low-rank adapter along with an importance scalar, then dynamically groups tasks based on adapter similarity. Within each group, adapters are pruned, scaled and merge, facilitating transfer learning between related tasks. Extensive experiments on three vision benchmarks show that HAM significantly outperforms state-of-the-art methods, particularly as the number of tasks increases.

Method: NoRA replaces fixed activations with learnable rational functions and applies structured low-rank updates to numerator/denominator coefficients with group-wise design for localized adaptation and stability.

Result: Achieves +0.17-0.27% accuracy gains on vision transformers with only 0.02M parameters (0.4% of model), and when combined with LoRA (NoRA++), outperforms LoRA/DoRA with fewer parameters and shows +0.3-0.8% MMLU gains on LLaMA3-8B.

Conclusion: Activation-space tuning is a complementary and highly parameter-efficient alternative to weight-based PEFT, establishing activation functions as first-class objects for model adaptation with implicit regularization benefits.

Abstract: Existing parameter-efficient fine-tuning (PEFT) methods primarily adapt weight matrices while keeping activation functions fixed. We introduce \textbf{NoRA}, the first PEFT framework that directly adapts nonlinear activation functions in pretrained transformer-based models. NoRA replaces fixed activations with learnable rational functions and applies structured low-rank updates to numerator and denominator coefficients, with a group-wise design that localizes adaptation and improves stability at minimal cost. On vision transformers trained on CIFAR-10 and CIFAR-100, NoRA matches or exceeds full fine-tuning while updating only 0.4% of parameters (0.02M), achieving accuracy gains of +0.17% and +0.27%. When combined with LoRA (\textbf{NoRA++}), it outperforms LoRA and DoRA under matched training budgets by adding fewer trainable parameters. On LLaMA3-8B instruction tuning, NoRA++ consistently improves generation quality, yielding average MMLU gains of +0.3%–0.8%, including +1.6% on STEM (Alpaca) and +1.3% on OpenOrca. We further show that NoRA constrains adaptation to a low-dimensional functional subspace, implicitly regularizing update magnitude and direction. These results establish activation-space tuning as a complementary and highly parameter-efficient alternative to weight-based PEFT, positioning activation functions as first-class objects for model adaptation.

Method: Deep learning architecture integrating physics-informed neural networks (PINNs) with conservation laws, using automatic differentiation to enforce physics constraints and adaptive loss weighting to balance data fidelity with physical consistency for both ODE and PDE systems.

Result: Achieved 98.9% correlation for 1D temporal dynamics (loss: 0.0219, MAE: 0.0184), captured complex spiral waves in 2D systems (loss: 4.7656, pattern correlation: 0.94), maintained conservation law adherence within 0.5%, and showed 10-50x computational speedup compared to numerical solvers.

Conclusion: USPIL is a transformative tool for ecological forecasting and conservation planning that establishes physics-informed deep learning as a powerful, scientifically rigorous paradigm for multi-scale ecological modeling with interpretable constraints and parameter discovery capabilities.

Abstract: Ecological systems exhibit complex multi-scale dynamics that challenge traditional modeling. New methods must capture temporal oscillations and emergent spatiotemporal patterns while adhering to conservation principles. We present the Unified Spatiotemporal Physics-Informed Learning (USPIL) framework, a deep learning architecture integrating physics-informed neural networks (PINNs) and conservation laws to model predator-prey dynamics across dimensional scales. The framework provides a unified solution for both ordinary (ODE) and partial (PDE) differential equation systems, describing temporal cycles and reaction-diffusion patterns within a single neural network architecture. Our methodology uses automatic differentiation to enforce physics constraints and adaptive loss weighting to balance data fidelity with physical consistency. Applied to the Lotka-Volterra system, USPIL achieves 98.9% correlation for 1D temporal dynamics (loss: 0.0219, MAE: 0.0184) and captures complex spiral waves in 2D systems (loss: 4.7656, pattern correlation: 0.94). Validation confirms conservation law adherence within 0.5% and shows a 10-50x computational speedup for inference compared to numerical solvers. USPIL also enables mechanistic understanding through interpretable physics constraints, facilitating parameter discovery and sensitivity analysis not possible with purely data-driven methods. Its ability to transition between dimensional formulations opens new avenues for multi-scale ecological modeling. These capabilities make USPIL a transformative tool for ecological forecasting, conservation planning, and understanding ecosystem resilience, establishing physics-informed deep learning as a powerful and scientifically rigorous paradigm.

Method: Uses competitive intrinsic rewards with ranking features, centralized reward module for balanced competition-cooperation, and bilevel optimization to maximize environmental rewards.

Result: Outperforms state-of-the-art methods in SMAC and GRF environments, with competitive rewards effectively promoting diverse and adaptive strategies.

Conclusion: Incorporating constructive competition through ranking-based intrinsic rewards enhances strategic diversity and performance in cooperative multi-agent systems.

Abstract: In recent years, diversity has emerged as a useful mechanism to enhance the efficiency of multi-agent reinforcement learning (MARL). However, existing methods predominantly focus on designing policies based on individual agent characteristics, often neglecting the interplay and mutual influence among agents during policy formation. To address this gap, we propose Competitive Diversity through Constructive Conflict (CoDiCon), a novel approach that incorporates competitive incentives into cooperative scenarios to encourage policy exchange and foster strategic diversity among agents. Drawing inspiration from sociological research, which highlights the benefits of moderate competition and constructive conflict in group decision-making, we design an intrinsic reward mechanism using ranking features to introduce competitive motivations. A centralized intrinsic reward module generates and distributes varying reward values to agents, ensuring an effective balance between competition and cooperation. By optimizing the parameterized centralized reward module to maximize environmental rewards, we reformulate the constrained bilevel optimization problem to align with the original task objectives. We evaluate our algorithm against state-of-the-art methods in the SMAC and GRF environments. Experimental results demonstrate that CoDiCon achieves superior performance, with competitive intrinsic rewards effectively promoting diverse and adaptive strategies among cooperative agents.

Method: Two-component framework: 1) Demonstration Generation module uses LLMs to create environment interaction instructions and produce high-quality demonstrations, 2) Policy Optimization module employs decentralized training where each agent combines expert policy loss from demonstrations with its own policy loss.

Result: LEED achieves superior sample efficiency, time efficiency, and robust scalability compared to state-of-the-art baselines.

Conclusion: The LLM-empowered expert demonstrations framework effectively addresses coordination and scalability bottlenecks in multi-agent reinforcement learning.

Abstract: Multi-agent reinforcement learning (MARL) holds substantial promise for intelligent decision-making in complex environments. However, it suffers from a coordination and scalability bottleneck as the number of agents increases. To address these issues, we propose the LLM-empowered expert demonstrations framework for multi-agent reinforcement learning (LEED). LEED consists of two components: a demonstration generation (DG) module and a policy optimization (PO) module. Specifically, the DG module leverages large language models to generate instructions for interacting with the environment, thereby producing high-quality demonstrations. The PO module adopts a decentralized training paradigm, where each agent utilizes the generated demonstrations to construct an expert policy loss, which is then integrated with its own policy loss. This enables each agent to effectively personalize and optimize its local policy based on both expert knowledge and individual experience. Experimental results show that LEED achieves superior sample efficiency, time efficiency, and robust scalability compared to state-of-the-art baselines.

Method: Frames the problem as Hierarchical Adversarial Decentralized Mean Field Control (HAD-MFC), decouples it using Fenchel-Rockafellar transform, reformulates the combinatorial problem as an MDP with dense rewards, and uses greedy/RL algorithms to sequentially identify vulnerable agents.

Result: The method effectively identifies more vulnerable agents in large-scale MARL and rule-based systems, causes worse system failures, and learns a value function that reveals agent vulnerability.

Conclusion: The proposed decomposition approach successfully solves the challenging HAD-MFC problem while preserving optimal solutions, providing an effective framework for identifying critical vulnerabilities in large-scale multi-agent systems.

Abstract: Partial agent failure becomes inevitable when systems scale up, making it crucial to identify the subset of agents whose compromise would most severely degrade overall performance. In this paper, we study this Vulnerable Agent Identification (VAI) problem in large-scale multi-agent reinforcement learning (MARL). We frame VAI as a Hierarchical Adversarial Decentralized Mean Field Control (HAD-MFC), where the upper level involves an NP-hard combinatorial task of selecting the most vulnerable agents, and the lower level learns worst-case adversarial policies for these agents using mean-field MARL. The two problems are coupled together, making HAD-MFC difficult to solve. To solve this, we first decouple the hierarchical process by Fenchel-Rockafellar transform, resulting a regularized mean-field Bellman operator for upper level that enables independent learning at each level, thus reducing computational complexity. We then reformulate the upper-level combinatorial problem as a MDP with dense rewards from our regularized mean-field Bellman operator, enabling us to sequentially identify the most vulnerable agents by greedy and RL algorithms. This decomposition provably preserves the optimal solution of the original HAD-MFC. Experiments show our method effectively identifies more vulnerable agents in large-scale MARL and the rule-based system, fooling system into worse failures, and learns a value function that reveals the vulnerability of each agent.

Method: Proposed a closed-form bound inspired by Amdahl’s Law, validated on SMAC and MPE benchmarks, with follow-up experiments in Overcooked-AI to test complex spatial and resource bottlenecks.

Result: Close alignment between the theoretical bound and empirical specialization measures at both extremes (unlimited concurrency vs. unit-capacity bottlenecks), with the model working effectively in complex environments.

Conclusion: The bound successfully predicts specialization benefits and serves as a diagnostic tool to identify biases in MARL training algorithms that lead to sub-optimal convergence in larger state spaces.

Abstract: When should we encourage specialization in multi-agent systems versus train generalists that perform the entire task independently? We propose that specialization largely depends on task parallelizability: the potential for multiple agents to execute task components concurrently. Drawing inspiration from Amdahl’s Law in distributed systems, we present a closed-form bound that predicts when specialization improves performance, depending only on task concurrency and team size. We validate our model on two standard MARL benchmarks that represent opposite regimes – StarCraft Multi-Agent Challenge (SMAC, unlimited concurrency) and Multi-Particle Environment (MPE, unit-capacity bottlenecks) – and observe close alignment between the bound at each extreme and an empirical measure of specialization. Three follow-up experiments in Overcooked-AI demonstrate that the model works in environments with more complex spatial and resource bottlenecks that allow for a range of strategies. Beyond prediction, the bound also serves as a diagnostic tool, highlighting biases in MARL training algorithms that cause sub-optimal convergence to specialist strategies with larger state spaces.

Method: Freezes CLIP/CLAP backbones, uses LoRA for emotion-oriented adaptation (≤4% parameter updates), employs asymmetric temporal modeling (Transformer for visual, mean pooling for audio), and simple fusion head.

Result: Achieves 80.14% on DFEW and 61.18% on MAFW with only 8M training parameters, setting new state-of-the-art performance.

Conclusion: Parameter-efficient adaptation of language-supervised foundation models provides a scalable alternative to domain-specific pre-training for real-world audiovisual emotion recognition.

Abstract: Audiovisual emotion recognition (AVER) in the wild is still hindered by pose variation, occlusion, and background noise. Prevailing methods primarily rely on large-scale domain-specific pre-training, which is costly and often mismatched to real-world affective data. To address this, we present CLAIP-Emo, a modular framework that reframes in-the-wild AVER as a parameter-efficient adaptation of language-supervised foundation models (CLIP/CLAP). Specifically, it (i) preserves language-supervised priors by freezing CLIP/CLAP backbones and performing emotion-oriented adaptation via LoRA (updating \ensuremath{\le}4.0% of the total parameters), (ii) allocates temporal modeling asymmetrically, employing a lightweight Transformer for visual dynamics while applying mean pooling for audio prosody, and (iii) applies a simple fusion head for prediction. On DFEW and MAFW, CLAIP-Emo (ViT-L/14) achieves 80.14% and 61.18% weighted average recall with only 8M training parameters, setting a new state of the art. Our findings suggest that parameter-efficient adaptation of language-supervised foundation models provides a scalable alternative to domain-specific pre-training for real-world AVER. The code and models will be available at \href{https://github.com/MSA-LMC/CLAIP-Emo}{https://github.com/MSA-LMC/CLAIP-Emo}.

Method: Created MMED dataset with spontaneous vocal cues accompanying micro-expressions, and developed AMF-Net using asymmetric cross-attention to fuse global visual summaries with dynamic audio sequences.

Result: Leave-One-Subject-Out Cross-Validation showed audio provides critical disambiguating information for micro-expression analysis, significantly improving recognition performance.

Conclusion: The MMED dataset and AMF-Net method provide valuable multimodal resources and a validated approach that demonstrates audio cues are essential for accurate micro-expression recognition.

Abstract: Micro-expressions (MEs) are crucial leakages of concealed emotion, yet their study has been constrained by a reliance on silent, visual-only data. To solve this issue, we introduce two principal contributions. First, MMED, to our knowledge, is the first dataset capturing the spontaneous vocal cues that co-occur with MEs in ecologically valid, high-stakes interactions. Second, the Asymmetric Multimodal Fusion Network (AMF-Net) is a novel method that effectively fuses a global visual summary with a dynamic audio sequence via an asymmetric cross-attention framework. Rigorous Leave-One-Subject-Out Cross-Validation (LOSO-CV) experiments validate our approach, providing conclusive evidence that audio offers critical, disambiguating information for ME analysis. Collectively, the MMED dataset and our AMF-Net method provide valuable resources and a validated analytical approach for micro-expression recognition.

Method: Built on top of Music4All-Onion dataset, providing multimodal data (metadata, genres, images, text) for artists and albums, and conducting experiments on multimodal genre classification including missing-modality scenarios and cross-domain comparisons.

Result: Experiments show images are more informative for artist/album genre classification, and multimodal models struggle to generalize across domains.

Conclusion: Music4All A+A fills an important gap by providing artist and album-level multimodal data, enabling broader MIR research including recommendation systems at multiple granularity levels.

Abstract: Music is characterized by aspects related to different modalities, such as the audio signal, the lyrics, or the music video clips. This has motivated the development of multimodal datasets and methods for Music Information Retrieval (MIR) tasks such as genre classification or autotagging. Music can be described at different levels of granularity, for instance defining genres at the level of artists or music albums. However, most datasets for multimodal MIR neglect this aspect and provide data at the level of individual music tracks. We aim to fill this gap by providing Music4All Artist and Album (Music4All A+A), a dataset for multimodal MIR tasks based on music artists and albums. Music4All A+A is built on top of the Music4All-Onion dataset, an existing track-level dataset for MIR tasks. Music4All A+A provides metadata, genre labels, image representations, and textual descriptors for 6,741 artists and 19,511 albums. Furthermore, since Music4All A+A is built on top of Music4All-Onion, it allows access to other multimodal data at the track level, including user–item interaction data. This renders Music4All A+A suitable for a broad range of MIR tasks, including multimodal music recommendation, at several levels of granularity. To showcase the use of Music4All A+A, we carry out experiments on multimodal genre classification of artists and albums, including an analysis in missing-modality scenarios, and a quantitative comparison with genre classification in the movie domain. Our experiments show that images are more informative for classifying the genres of artists and albums, and that several multimodal models for genre classification struggle in generalizing across domains. We provide the code to reproduce our experiments at https://github.com/hcai-mms/Music4All-A-A, the dataset is linked in the repository and provided open-source under a CC BY-NC-SA 4.0 license.

Method: Adapts pre-trained TTS models using a multi-task latent diffusion framework, introduces Implicit Task Composition (ITC) where ASR-derived transcripts guide enhancement via principled inference-time task composition.

Result: SpeechOp inherits rich understanding of natural speech, accelerates training, improves S2S task quality, and enhances core TTS performance while achieving state-of-the-art content preservation.

Conclusion: The approach successfully bridges the data gap by combining web-scale speech understanding with generative capabilities, enabling robust speech processing and novel task compositions.

Abstract: While generative Text-to-Speech (TTS) systems leverage vast ``in-the-wild" data to achieve remarkable success, speech-to-speech processing tasks like enhancement face data limitations, which lead data-hungry generative approaches to distort speech content and speaker identity. To bridge this gap, we present SpeechOp, a multi-task latent diffusion model that transforms pre-trained TTS models into a universal speech processor capable of performing a wide range of speech tasks and composing them in novel ways at inference time. By adapting a pre-trained TTS model, SpeechOp inherits a rich understanding of natural speech, accelerating training and improving S2S task quality, while simultaneously enhancing core TTS performance. Finally, we introduce Implicit Task Composition (ITC), a novel pipeline where ASR-derived transcripts (e.g., from Whisper) guide SpeechOp’s enhancement via our principled inference-time task composition. ITC achieves state-of-the-art content preservation by robustly combining web-scale speech understanding with SpeechOp’s generative capabilities. Audio samples are available at https://justinlovelace.github.io/projects/speechop

Method: Generative unsupervised technique that models clean speech and structured noise components separately. Uses audio-visual score model with visual cues as strong generative speech prior. Performs separation via reverse diffusion process sampling from posterior distributions to estimate and remove noise.

Result: Experimental results show promising performance in challenging acoustic environments, demonstrating effectiveness of direct noise modeling approach.

Conclusion: The proposed method successfully separates speech from ambient noise using unsupervised generative modeling with visual priors and explicit noise distribution modeling through diffusion processes.

Abstract: In this paper, we address the problem of single-microphone speech separation in the presence of ambient noise. We propose a generative unsupervised technique that directly models both clean speech and structured noise components, training exclusively on these individual signals rather than noisy mixtures. Our approach leverages an audio-visual score model that incorporates visual cues to serve as a strong generative speech prior. By explicitly modelling the noise distribution alongside the speech distribution, we enable effective decomposition through the inverse problem paradigm. We perform speech separation by sampling from the posterior distributions via a reverse diffusion process, which directly estimates and removes the modelled noise component to recover clean constituent signals. Experimental results demonstrate promising performance, highlighting the effectiveness of our direct noise modelling approach in challenging acoustic environments.

Method: Proposed a multi-channel differential automatic speech recognition system that takes differential inputs from complementary frontends: beamformer, microphone selection, and lightweight side-talk detection model.

Result: Evaluations on simulated and real datasets show the system outperforms traditional approaches, achieving up to 18.0% relative reduction in word error rate.

Conclusion: The proposed differential ASR method effectively improves robustness of wearer speech recognition on smart glasses by handling side-talk interference through complementary multi-channel processing.

Abstract: With the growing adoption of wearable devices such as smart glasses for AI assistants, wearer speech recognition (WSR) is becoming increasingly critical to next-generation human-computer interfaces. However, in real environments, interference from side-talk speech remains a significant challenge to WSR and may cause accumulated errors for downstream tasks such as natural language processing. In this work, we introduce a novel multi-channel differential automatic speech recognition (ASR) method for robust WSR on smart glasses. The proposed system takes differential inputs from different frontends that complement each other to improve the robustness of WSR, including a beamformer, microphone selection, and a lightweight side-talk detection model. Evaluations on both simulated and real datasets demonstrate that the proposed system outperforms the traditional approach, achieving up to an 18.0% relative reduction in word error rate.

Method: Uses diarized segments from conventional diarizer, generates augmented speech samples with phonetic/stylistic diversity, blends speaker embeddings from original and generated audio to enhance robustness.

Result: 49% error rate reduction on simulated emotional speech dataset and 35% reduction on truncated AMI dataset.

Conclusion: Style-controllable speech generation effectively addresses intra-speaker variability in diarization systems, significantly improving performance across different datasets.

Abstract: Speaker diarization systems often struggle with high intrinsic intra-speaker variability, such as shifts in emotion, health, or content. This can cause segments from the same speaker to be misclassified as different individuals, for example, when one raises their voice or speaks faster during conversation. To address this, we propose a style-controllable speech generation model that augments speech across diverse styles while preserving the target speaker’s identity. The proposed system starts with diarized segments from a conventional diarizer. For each diarized segment, it generates augmented speech samples enriched with phonetic and stylistic diversity. And then, speaker embeddings from both the original and generated audio are blended to enhance the system’s robustness in grouping segments with high intrinsic intra-speaker variability. We validate our approach on a simulated emotional speech dataset and the truncated AMI dataset, demonstrating significant improvements, with error rate reductions of 49% and 35% on each dataset, respectively.

Method: Uses lightweight ASC model to identify environment, then conditions SELD network to focus on detecting and localizing contextually relevant sounds based on the identified scene.

Result: The ASC-conditioned SELD system shows improved spatial intelligence over conventional baselines on simulated headphone data.

Conclusion: This represents a crucial step toward intelligent hearables that can provide environmental information for safer, context-aware listening experiences.

Abstract: Wearable audio devices with active noise control (ANC) enhance listening comfort but often at the expense of situational awareness. However, this auditory isolation may mask crucial environmental cues, posing significant safety risks. To address this, we propose an environmental intelligence framework that combines Acoustic Scene Classification (ASC) with Sound Event Localization and Detection (SELD). Our system first employs a lightweight ASC model to infer the current environment. The scene prediction then dynamically conditions a SELD network, tuning its sensitivity to detect and localize sounds that are most salient to the current context. On simulated headphone data, the proposed ASC-conditioned SELD system demonstrates improved spatial intelligence over a conventional baseline. This work represents a crucial step towards creating intelligent hearables that can deliver crucial environmental information, fostering a safer and more context-aware listening experience.

Method: Train CLAP-based reward model using human-labeled pairwise preference data, then integrate it into reinforcement learning framework to fine-tune baseline captioning systems without ground-truth annotations.

Result: Human evaluations show captions are preferred over baseline models, especially when baselines fail. Achieves performance comparable to supervised approaches with ground-truth data.

Conclusion: The framework effectively aligns audio captioning with human preferences and demonstrates scalability for real-world scenarios without requiring expensive caption annotations.

Abstract: Current audio captioning systems rely heavily on supervised learning with paired audio-caption datasets, which are expensive to curate and may not reflect human preferences in real-world scenarios. To address this limitation, we propose a preference-aligned audio captioning framework based on Reinforcement Learning from Human Feedback (RLHF). To effectively capture nuanced human preferences, we train a Contrastive Language-Audio Pretraining (CLAP)-based reward model using human-labeled pairwise preference data. This reward model is integrated into a reinforcement learning framework to fine-tune any baseline captioning system without relying on ground-truth caption annotations. Extensive human evaluations across multiple datasets show that our method produces captions preferred over those from baseline models, particularly in cases where the baseline models fail to provide correct and natural captions. Furthermore, our framework achieves performance comparable to supervised approaches with ground-truth data, demonstrating its effectiveness in aligning audio captioning with human preferences and its scalability in real-world scenarios.

Method: Integrates cross-attention mechanisms in transformer decoder to extract salient features per target label, employs data augmentation using speech generation model to address dataset imbalance.

Result: Validated through multiple objective evaluations on both seen and unseen corpora, demonstrating effective multi-style detection capabilities.

Conclusion: The framework successfully captures various speaking styles and provides analysis of human perception impact on classification accuracy through labeling agreement consideration.

Abstract: In this paper, we propose a multi-label classification framework to detect multiple speaking styles in a speech sample. Unlike previous studies that have primarily focused on identifying a single target style, our framework effectively captures various speaker characteristics within a unified structure, making it suitable for generalized human-computer interaction applications. The proposed framework integrates cross-attention mechanisms within a transformer decoder to extract salient features associated with each target label from the input speech. To mitigate the data imbalance inherent in multi-label speech datasets, we employ a data augmentation technique based on a speech generation model. We validate our model’s effectiveness through multiple objective evaluations on seen and unseen corpora. In addition, we provide an analysis of the influence of human perception on classification accuracy by considering the impact of human labeling agreement on model performance.

Method: Built on F5-TTS with pretrained speech-environment separation, applies random span masks to clean speech and environment mel-spectrograms, uses dual class-free guidance and SNR adaptation for enhanced controllability.

Result: Generates environmental personalized speech with high naturalness, strong speaker similarity, and high environmental fidelity.

Conclusion: DAIEN-TTS successfully enables zero-shot environment-aware TTS with disentangled control over speaker and environment characteristics through innovative audio infilling techniques.

Abstract: This paper presents DAIEN-TTS, a zero-shot text-to-speech (TTS) framework that enables ENvironment-aware synthesis through Disentangled Audio Infilling. By leveraging separate speaker and environment prompts, DAIEN-TTS allows independent control over the timbre and the background environment of the synthesized speech. Built upon F5-TTS, the proposed DAIEN-TTS first incorporates a pretrained speech-environment separation (SES) module to disentangle the environmental speech into mel-spectrograms of clean speech and environment audio. Two random span masks of varying lengths are then applied to both mel-spectrograms, which, together with the text embedding, serve as conditions for infilling the masked environmental mel-spectrogram, enabling the simultaneous continuation of personalized speech and time-varying environmental audio. To further enhance controllability during inference, we adopt dual class-free guidance (DCFG) for the speech and environment components and introduce a signal-to-noise ratio (SNR) adaptation strategy to align the synthesized speech with the environment prompt. Experimental results demonstrate that DAIEN-TTS generates environmental personalized speech with high naturalness, strong speaker similarity, and high environmental fidelity.

Method: Uses autoregressive transformer decoder to generate continuous mel-spectrogram frames from linguistic and speaker conditions, with a representation alignment module that aligns output representations with semantic embeddings from a pretrained ASR encoder during training.

Result: Achieves state-of-the-art performance across multiple evaluation metrics, maintains robust zero-shot voice cloning capabilities, and works in both offline and streaming synthesis modes.

Conclusion: Establishes a new benchmark for continuous feature generation in TTS, offering a compelling alternative to discrete-token-based approaches with improved training convergence and cross-modal coherence.

Abstract: This work introduces MELA-TTS, a novel joint transformer-diffusion framework for end-to-end text-to-speech synthesis. By autoregressively generating continuous mel-spectrogram frames from linguistic and speaker conditions, our architecture eliminates the need for speech tokenization and multi-stage processing pipelines. To address the inherent difficulties of modeling continuous features, we propose a representation alignment module that aligns output representations of the transformer decoder with semantic embeddings from a pretrained ASR encoder during training. This mechanism not only speeds up training convergence, but also enhances cross-modal coherence between the textual and acoustic domains. Comprehensive experiments demonstrate that MELA-TTS achieves state-of-the-art performance across multiple evaluation metrics while maintaining robust zero-shot voice cloning capabilities, in both offline and streaming synthesis modes. Our results establish a new benchmark for continuous feature generation approaches in TTS, offering a compelling alternative to discrete-token-based paradigms.

Method: Developed an acoustic simulation framework that models genuine and spoofed speech across varied environments with realistic microphone/loudspeaker impulse responses, room acoustics, and noise conditions. Uses measured loudspeaker directionalities and defines two spoofing settings (reverberant vs anechoic speech).

Result: The framework successfully generates synthetic multi-channel replay speech data. When tested with state-of-the-art M-ALRAD model, the synthetic data supports detector generalization across unseen enclosures.

Conclusion: The proposed simulation framework effectively creates realistic multi-channel replay attack scenarios and demonstrates that synthetic data can enhance replay speech detection performance and generalization capabilities.

Abstract: Replay speech attacks pose a significant threat to voice-controlled systems, especially in smart environments where voice assistants are widely deployed. While multi-channel audio offers spatial cues that can enhance replay detection robustness, existing datasets and methods predominantly rely on single-channel recordings. In this work, we introduce an acoustic simulation framework designed to simulate multi-channel replay speech configurations using publicly available resources. Our setup models both genuine and spoofed speech across varied environments, including realistic microphone and loudspeaker impulse responses, room acoustics, and noise conditions. The framework employs measured loudspeaker directionalities during the replay attack to improve the realism of the simulation. We define two spoofing settings, which simulate whether a reverberant or an anechoic speech is used in the replay scenario, and evaluate the impact of omnidirectional and diffuse noise on detection performance. Using the state-of-the-art M-ALRAD model for replay speech detection, we demonstrate that synthetic data can support the generalization capabilities of the detector across unseen enclosures.

Method: Encodes arbitrary array recordings into spherical harmonics domain using Ambisonics Signal Matching (ASM), trains deep neural network on simulated Ambisonics data with channel dropout for robustness against array-dependent encoding errors.

Result: While baseline and proposed models perform similarly on training arrays, baseline degrades on unseen arrays. AmbiDrop consistently improves SI-SDR, PESQ, and STOI metrics.

Conclusion: AmbiDrop demonstrates strong generalization and practical potential for array-agnostic speech enhancement, eliminating the need for diverse microphone array databases.

Abstract: Multichannel speech enhancement leverages spatial cues to improve intelligibility and quality, but most learning-based methods rely on specific microphone array geometry, unable to account for geometry changes. To mitigate this limitation, current array-agnostic approaches employ large multi-geometry datasets but may still fail to generalize to unseen layouts. We propose AmbiDrop (Ambisonics with Dropouts), an Ambisonics-based framework that encodes arbitrary array recordings into the spherical harmonics domain using Ambisonics Signal Matching (ASM). A deep neural network is trained on simulated Ambisonics data, combined with channel dropout for robustness against array-dependent encoding errors, therefore omitting the need for a diverse microphone array database. Experiments show that while the baseline and proposed models perform similarly on the training arrays, the baseline degrades on unseen arrays. In contrast, AmbiDrop consistently improves SI-SDR, PESQ, and STOI, demonstrating strong generalization and practical potential for array-agnostic speech enhancement.

Method: Adopted Anti-Memorization Guidance (AMG) with three guidance types to modify sampling process of pre-trained diffusion models, using Stable Audio Open as backbone for its open-source architecture and training dataset.

Result: AMG significantly mitigates memorization in diffusion-based text-to-audio generation while preserving audio fidelity and semantic alignment.

Conclusion: Anti-memorization strategies effectively reduce data replication in text-to-audio models without compromising generation quality.

Abstract: A persistent challenge in generative audio models is data replication, where the model unintentionally generates parts of its training data during inference. In this work, we address this issue in text-to-audio diffusion models by exploring the use of anti-memorization strategies. We adopt Anti-Memorization Guidance (AMG), a technique that modifies the sampling process of pre-trained diffusion models to discourage memorization. Our study explores three types of guidance within AMG, each designed to reduce replication while preserving generation quality. We use Stable Audio Open as our backbone, leveraging its fully open-source architecture and training dataset. Our comprehensive experimental analysis suggests that AMG significantly mitigates memorization in diffusion-based text-to-audio generation without compromising audio fidelity or semantic alignment.

Method: Proposed an automated framework for generating large-scale paralinguistic data and applied it to construct the SynParaSpeech dataset from natural conversational speech.

Result: Created SynParaSpeech dataset with 6 paralinguistic categories, 118.75 hours of data, and precise timestamps - the first automated method for large-scale paralinguistic dataset construction.

Conclusion: The framework advances speech generation through more natural paralinguistic synthesis and enhances speech understanding by improving paralinguistic event detection, with publicly available dataset and samples.

Abstract: Paralinguistic sounds, like laughter and sighs, are crucial for synthesizing more realistic and engaging speech. However, existing methods typically depend on proprietary datasets, while publicly available resources often suffer from incomplete speech, inaccurate or missing timestamps, and limited real-world relevance. To address these problems, we propose an automated framework for generating large-scale paralinguistic data and apply it to construct the SynParaSpeech dataset. The dataset comprises 6 paralinguistic categories with 118.75 hours of data and precise timestamps, all derived from natural conversational speech. Our contributions lie in introducing the first automated method for constructing large-scale paralinguistic datasets and releasing the SynParaSpeech corpus, which advances speech generation through more natural paralinguistic synthesis and enhances speech understanding by improving paralinguistic event detection. The dataset and audio samples are available at https://github.com/ShawnPi233/SynParaSpeech.

Method: Frame-level WavLM embeddings of generated speech are aligned to an unpaired bona fide pool using entropic optimal transport and top-k barycentric projection, then decoded with a neural vocoder.

Result: DOT achieves consistently high equal error rates (EER) across ASVspoof2019 and ASVspoof5 datasets, outperforms conventional attacks in cross-dataset transfer, and remains competitive after countermeasure fine-tuning.

Conclusion: Distribution-level alignment through optimal transport provides a powerful and stable attack surface for deployed audio anti-spoofing countermeasures.

Abstract: In this paper, we show that discrete optimal transport (DOT) is an effective black-box adversarial attack against modern audio anti-spoofing countermeasures (CMs). Our attack operates as a post-processing, distribution-alignment step: frame-level WavLM embeddings of generated speech are aligned to an unpaired bona fide pool via entropic OT and a top-$k$ barycentric projection, then decoded with a neural vocoder. Evaluated on ASVspoof2019 and ASVspoof5 with AASIST baselines, DOT yields consistently high equal error rate (EER) across datasets and remains competitive after CM fine-tuning, outperforming several conventional attacks in cross-dataset transfer. Ablation analysis highlights the practical impact of vocoder overlap. Results indicate that distribution-level alignment is a powerful and stable attack surface for deployed CMs.

Method: Trained a self-supervised speech representation model (BabyHuBERT) on 13,000 hours of multilingual child-centered long-form recordings spanning over 40 languages.

Result: Achieved F1-scores from 52.1% to 74.4% across six datasets, with notable improvements of 13.2-15.9 absolute F1 points over standard HuBERT on underrepresented languages like Vanuatu and Solomon Islands.

Conclusion: BabyHuBERT serves as an effective foundation model for child speech research, enabling fine-tuning on diverse downstream tasks and significantly improving performance on speaker segmentation for analyzing naturalistic language experiences.

Abstract: Child-centered long-form recordings are essential for studying early language development, but existing speech models trained on clean adult data perform poorly due to acoustic and linguistic differences. We introduce BabyHuBERT, the first self-supervised speech representation model trained on 13,000 hours of multilingual child-centered long-form recordings spanning over 40 languages. We evaluate BabyHuBERT on speaker segmentation, identifying when target children speak versus female adults, male adults, or other children – a fundamental preprocessing step for analyzing naturalistic language experiences. BabyHuBERT achieves F1-scores from 52.1% to 74.4% across six diverse datasets, consistently outperforming W2V2-LL4300 (trained on English long-forms) and standard HuBERT (trained on clean adult speech). Notable improvements include 13.2 absolute F1 points over HuBERT on Vanuatu and 15.9 points on Solomon Islands corpora, demonstrating effectiveness on underrepresented languages. By sharing code and models, BabyHuBERT serves as a foundation model for child speech research, enabling fine-tuning on diverse downstream tasks.

Method: Proposes transfer learning from adult to pediatric OSA detection using acoustic models pretrained on adult sleep data (157 nights) and fine-tuned on pediatric data (15 nights). Incorporates SpO2-based desaturation patterns and evaluates single vs multi-task learning, encoder freezing vs full fine-tuning, and delayed SpO2 label alignment.

Result: Fine-tuning with SpO2 integration consistently improves pediatric OSA detection compared to baseline models without adaptation. The approach demonstrates feasibility for home-based screening.

Conclusion: Transfer learning with SpO2 integration is feasible and effective for pediatric OSA detection, offering potential clinical value for early diagnosis through non-invasive acoustic monitoring.

Abstract: Paediatric obstructive sleep apnoea (OSA) is clinically significant yet difficult to diagnose, as children poorly tolerate sensor-based polysomnography. Acoustic monitoring provides a non-invasive alternative for home-based OSA screening, but limited paediatric data hinders the development of robust deep learning approaches. This paper proposes a transfer learning framework that adapts acoustic models pretrained on adult sleep data to paediatric OSA detection, incorporating SpO2-based desaturation patterns to enhance model training. Using a large adult sleep dataset (157 nights) and a smaller paediatric dataset (15 nights), we systematically evaluate (i) single- versus multi-task learning, (ii) encoder freezing versus full fine-tuning, and (iii) the impact of delaying SpO2 labels to better align them with the acoustics and capture physiologically meaningful features. Results show that fine-tuning with SpO2 integration consistently improves paediatric OSA detection compared with baseline models without adaptation. These findings demonstrate the feasibility of transfer learning for home-based OSA screening in children and offer its potential clinical value for early diagnosis.

Method: Combines off-the-shelf SD, ASR, and LLM in modular pipeline. Uses structured LLM prompting on reconciled SD and ASR outputs to leverage semantic continuity for refining speaker labels and assigning role identities.

Result: 29.7% relative error reduction over baseline reconciled SD and ASR on real-world patient-clinician dataset. Enhances diarization without additional training.

Conclusion: Provides complete pipeline for SD, ASR, and speaker identity detection in practical applications, improving performance while maintaining flexibility.

Abstract: Speaker diarization (SD) struggles in real-world scenarios due to dynamic environments and unknown speaker counts. SD is rarely used alone and is often paired with automatic speech recognition (ASR), but non-modular methods that jointly train on domain-specific data have limited flexibility. Moreover, many applications require true speaker identities rather than SD’s pseudo labels. We propose a training-free modular pipeline combining off-the-shelf SD, ASR, and a large language model (LLM) to determine who spoke, what was said, and who they are. Using structured LLM prompting on reconciled SD and ASR outputs, our method leverages semantic continuity in conversational context to refine low-confidence speaker labels and assigns role identities while correcting split speakers. On a real-world patient-clinician dataset, our approach achieves a 29.7% relative error reduction over baseline reconciled SD and ASR. It enhances diarization performance without additional training and delivers a complete pipeline for SD, ASR, and speaker identity detection in practical applications.

Method: Based on generative flow matching, prior work on STFT phase retrieval (DiffPhase), and Mel filterbank pseudoinverse operator to create a streaming-capable architecture.

Result: Achieves real-time streaming at 48ms total latency on consumer laptop GPU, with substantially better PESQ and SI-SDR values compared to non-streaming baselines like HiFi-GAN.

Conclusion: MelFlow enables high-quality, low-latency streaming Mel vocoding that outperforms existing methods while maintaining practical real-time performance.

Abstract: The task of Mel vocoding, i.e., the inversion of a Mel magnitude spectrogram to an audio waveform, is still a key component in many text-to-speech (TTS) systems today. Based on generative flow matching, our prior work on generative STFT phase retrieval (DiffPhase), and the pseudoinverse operator of the Mel filterbank, we develop MelFlow, a streaming-capable generative Mel vocoder for speech sampled at 16 kHz with an algorithmic latency of only 32 ms and a total latency of 48 ms. We show real-time streaming capability at this latency not only in theory, but in practice on a consumer laptop GPU. Furthermore, we show that our model achieves substantially better PESQ and SI-SDR values compared to well-established not streaming-capable baselines for Mel vocoding including HiFi-GAN.

Method: Uses LLMs with a “Listening-Imagining-Refining” strategy inspired by human auditory perception, generating phonetic variants and refining them in context. Includes heuristic optimization with finite state machine to avoid local optima and rule-based constraints for semantic fidelity.

Result: Experiments on English and Chinese ASR outputs show average reductions in CER/WER of up to 1.5 percentage points compared to baselines.

Conclusion: LIR-ASR demonstrates substantial accuracy gains in transcription through its iterative correction framework with heuristic optimization.

Abstract: Automatic Speech Recognition (ASR) systems remain prone to errors that affect downstream applications. In this paper, we propose LIR-ASR, a heuristic optimized iterative correction framework using LLMs, inspired by human auditory perception. LIR-ASR applies a “Listening-Imagining-Refining” strategy, generating phonetic variants and refining them in context. A heuristic optimization with finite state machine (FSM) is introduced to prevent the correction process from being trapped in local optima and rule-based constraints help maintain semantic fidelity. Experiments on both English and Chinese ASR outputs show that LIR-ASR achieves average reductions in CER/WER of up to 1.5 percentage points compared to baselines, demonstrating substantial accuracy gains in transcription.

Method: Large-scale probing analysis using phonetic frameworks to categorize speaker attributes into functional groups (Acoustic, Prosodic, Paralinguistic) and analyze layer-wise representations across multiple SSL model families.

Result: Found consistent three-stage hierarchy: initial layers encode fundamental timbre and prosody; middle layers synthesize abstract traits; final layers suppress speaker identity to focus on linguistic content. Intermediate SSL layers outperform specialized speaker embeddings for dynamic prosody representation.

Conclusion: Speech SSL models systematically separate dynamic speech style from intrinsic characteristics through hierarchical processing, with practical implications for downstream tasks requiring speaker attribute modeling.

Abstract: Speech self-supervised learning (SSL) models are known to learn hierarchical representations, yet how they encode different speaker-specific attributes remains under-explored. This study investigates the layer-wise disentanglement of speaker information across multiple speech SSL model families and their variants. Drawing from phonetic frameworks, we conduct a large-scale probing analysis of attributes categorised into functional groups: Acoustic (Gender), Prosodic (Pitch, Tempo, Energy), and Paralinguistic (Emotion), which we use to deconstruct the model’s representation of Speaker Identity. Our findings validate a consistent three-stage hierarchy: initial layers encode fundamental timbre and prosody; middle layers synthesise abstract traits; and final layers suppress speaker identity to abstract linguistic content. An ablation study shows that while specialised speaker embeddings excel at identifying speaker identity, the intermediate layers of speech SSL models better represent dynamic prosody. This work is the first large-scale study covering a wide range of speech SSL model families and variants with fine-grained speaker-specific attributes on how they hierarchically separate the dynamic style of speech from its intrinsic characteristics, offering practical implications for downstream tasks.

Method: Conducted perceptual analysis across expressive dimensions (adverbs of degree, emotion intensity), collected human ratings on speaker age and word-level emphasis, and created the Expressive VOice Control (E-VOC) corpus with large-scale human evaluations.

Result: GPT-4o-mini-tts was the most reliable ITTS model with good instruction-alignment. All 5 analyzed systems tended to generate Adult voices regardless of child/elderly instructions. Fine-grained control remains a major challenge across systems.

Conclusion: Most ITTS systems have substantial room for improvement in interpreting nuanced attribute instructions, particularly for fine-grained control and accurate age-related voice generation.

Abstract: Instruction-guided text-to-speech (ITTS) enables users to control speech generation through natural language prompts, offering a more intuitive interface than traditional TTS. However, the alignment between user style instructions and listener perception remains largely unexplored. This work first presents a perceptual analysis of ITTS controllability across two expressive dimensions (adverbs of degree and graded emotion intensity) and collects human ratings on speaker age and word-level emphasis attributes. To comprehensively reveal the instruction-perception gap, we provide a data collection with large-scale human evaluations, named Expressive VOice Control (E-VOC) corpus. Furthermore, we reveal that (1) gpt-4o-mini-tts is the most reliable ITTS model with great alignment between instruction and generated utterances across acoustic dimensions. (2) The 5 analyzed ITTS systems tend to generate Adult voices even when the instructions ask to use child or Elderly voices. (3) Fine-grained control remains a major challenge, indicating that most ITTS systems have substantial room for improvement in interpreting slightly different attribute instructions.

Method: Proposes D4PM - a dual-branch driven denoising diffusion probabilistic model with dual-branch conditional diffusion architecture to model clean EEG and artifact distributions, plus a joint posterior sampling strategy for collaborative integration of complementary priors.

Result: Extensive experiments on two public datasets show superior denoising performance. Achieves new state-of-the-art in EOG artifact removal, outperforming all publicly available baselines.

Conclusion: D4PM effectively addresses limitations of existing methods by unifying multi-type artifact removal through dual-branch architecture and joint sampling, demonstrating strong potential for accurate EEG signal analysis.

Abstract: Artifact removal is critical for accurate analysis and interpretation of Electroencephalogram (EEG) signals. Traditional methods perform poorly with strong artifact-EEG correlations or single-channel data. Recent advances in diffusion-based generative models have demonstrated strong potential for EEG denoising, notably improving fine-grained noise suppression and reducing over-smoothing. However, existing methods face two main limitations: lack of temporal modeling limits interpretability and the use of single-artifact training paradigms ignore inter-artifact differences. To address these issues, we propose D4PM, a dual-branch driven denoising diffusion probabilistic model that unifies multi-type artifact removal. We introduce a dual-branch conditional diffusion architecture to implicitly model the data distribution of clean EEG and artifacts. A joint posterior sampling strategy is further designed to collaboratively integrate complementary priors for high-fidelity EEG reconstruction. Extensive experiments on two public datasets show that D4PM delivers superior denoising. It achieves new state-of-the-art performance in EOG artifact removal, outperforming all publicly available baselines. The code is available at https://github.com/flysnow1024/D4PM.

Method: Proposes a homotopy continuation formulation that starts with well-posed synthetic sensing matrix and smoothly transitions to the target ill-posed problem through a continuation path strategy during unrolling network optimization.

Result: Achieves up to 2.5 dB PSNR improvement over conventional unrolled training in compressive sensing and image deblurring experiments.

Conclusion: The continuation strategy enables progressive learning from simpler well-posed problems to challenging ill-posed scenarios, generating smooth solution paths and significantly improving reconstruction performance.

Abstract: Imaging Inverse problems aim to reconstruct an underlying image from undersampled, coded, and noisy observations. Within the wide range of reconstruction frameworks, the unrolling algorithm is one of the most popular due to the synergistic integration of traditional model-based reconstruction methods and modern neural networks, providing an interpretable and highly accurate reconstruction. However, when the sensing operator is highly ill-posed, gradient steps on the data-fidelity term can hinder convergence and degrade reconstruction quality. To address this issue, we propose UTOPY, a homotopy continuation formulation for training the unrolling algorithm. Mainly, this method involves using a well-posed (synthetic) sensing matrix at the beginning of the unrolling network optimization. We define a continuation path strategy to transition smoothly from the synthetic fidelity to the desired ill-posed problem. This strategy enables the network to progressively transition from a simpler, well-posed inverse problem to the more challenging target scenario. We theoretically show that, for projected gradient descent-like unrolling models, the proposed continuation strategy generates a smooth path of unrolling solutions. Experiments on compressive sensing and image deblurring demonstrate that our method consistently surpasses conventional unrolled training, achieving up to 2.5 dB PSNR improvement in reconstruction performance. Source code at

Method: Created sparsely sampled light fields by dropping views, encoded with JPEG Pleno and VVC, applied view synthesis to reconstruct views, and conducted subjective evaluation using JPEG AIC-3 flicker test methodology.

Result: Subjective quality assessment results obtained through flicker comparison tests, enabling validation of various quality metrics for light field compression with view synthesis.

Conclusion: The study provides subjective evaluation framework and results that help validate quality metrics for assessing light field compression performance when view synthesis is involved.

Abstract: Light field technology is a powerful imaging method that captures both the intensity and direction of light rays in a scene, enabling the reconstruction of 3D information and supporting a range of unique applications. However, light fields produce vast amounts of data, making efficient compression essential for their practical use. View synthesis plays a key role in light field technology by enabling the generation of new views, yet its interaction with compression has not been fully explored. In this work, a subjective analysis of the effect of view synthesis on light field compression is conducted. To achieve this, a sparsely sampled light field is created by dropping views from an original light field. Both light fields are then encoded using JPEG Pleno and VVC. View synthesis is then applied to the compressed sampled light field to reconstruct the same number of views as the original. The subjective evaluation follows the proposed JPEG AIC-3 test methodology designed to assess the quality of high-fidelity compressed images. This test consists of two test stimuli displayed side-by-side, each alternating between an original and a coded view, creating a flicker effect on both sides. The user must choose which side has the stronger flicker and, therefore, the lower quality. Using these subjective results, a selection of metrics is validated.

Method: Two-stage temporal feature extraction: (i) parent-level existence map and (ii) child-level neighborhood lookup in previous frame, fused with spatial features via elementwise addition and encoded with group-wise strategy.

Result: Achieves encoding time of 105 ms and decoding time of 140 ms, with up to 43.6% bit rate reduction compared to G-PCC, and consistently outperforms spatial-only baseline (RENO).

Conclusion: HINT effectively integrates temporal and spatial correlation for sequential point cloud compression, providing significant speed improvements and compression efficiency over existing methods.

Abstract: Deep learning has demonstrated strong capability in compressing point clouds. Within this area, entropy modeling for lossless compression is widely investigated. However, most methods rely solely on parent orsibling contexts and level-wise autoregression, which suffers from decoding latency on the order of 10 to 100 seconds. We propose HINT, a method that integrates temporal and spatial correlation for sequential point cloud compression. Specifically, it first uses a two stage temporal feature extraction: (i) a parent-level existence map and (ii) a child-level neighborhood lookup in the previous frame. These cues are fused with the spatial features via elementwise addition and encoded with a group-wise strategy. Experimental results show that HINT achieves encoding and decoding time at 105 ms and 140 ms, respectively, equivalent to 49.6x and 21.6x acceleration in comparison with G-PCC, while achieving up to bit rate reduction of 43.6%, in addition, consistently outperforming over the strong spatial only baseline (RENO).

Method: Evaluated various image priors in the context of phase retrieval with structured random Fourier measurements under severe undersampling conditions

Result: Image priors significantly improve reconstruction quality, allowing accurate reconstruction even below the weak recovery threshold

Conclusion: Incorporating signal priors is crucial for successful phase retrieval in severely undersampled measurement scenarios

Abstract: Phase retrieval seeks to recover a complex signal from amplitude-only measurements, a challenging nonlinear inverse problem. Current theory and algorithms often ignore signal priors. By contrast, we evaluate here a variety of image priors in the context of severe undersampling with structured random Fourier measurements. Our results show that those priors significantly improve reconstruction, allowing accurate reconstruction even below the weak recovery threshold.

Method: Integrates reaction-diffusion PDEs within a variational autoencoder (VAE) mixture model framework to infer subtypes of interpretable latent variables (diffusivity and reaction rates) from neuroimaging data.

Result: The method was evaluated on synthetic benchmarks and demonstrated potential for uncovering mechanistic subtypes of Alzheimer’s disease progression from PET data.

Conclusion: The proposed approach successfully extends physics-integrated machine learning beyond single-PDE limitations, enabling identification of disease subtypes with enhanced interpretability for neurodegenerative disease modeling.

Abstract: Modelling the underlying mechanisms of neurodegenerative diseases demands methods that capture heterogeneous and spatially varying dynamics from sparse, high-dimensional neuroimaging data. Integrating partial differential equation (PDE) based physics knowledge with machine learning provides enhanced interpretability and utility over classic numerical methods. However, current physics-integrated machine learning methods are limited to considering a single PDE, severely limiting their application to diseases where multiple mechanisms are responsible for different groups (i.e., subtypes) and aggravating problems with model misspecification and degeneracy. Here, we present a deep generative model for learning mixtures of latent dynamic models governed by physics-based PDEs, going beyond traditional approaches that assume a single PDE structure. Our method integrates reaction-diffusion PDEs within a variational autoencoder (VAE) mixture model framework, supporting inference of subtypes of interpretable latent variables (e.g. diffusivity and reaction rates) from neuroimaging data. We evaluate our method on synthetic benchmarks and demonstrate its potential for uncovering mechanistic subtypes of Alzheimer’s disease progression from positron emission tomography (PET) data.

Method: Proposed Mixture of Multicenter Experts (MoME) framework that integrates specialized expertise from diverse clinical strategies. Validated using multimodal target volume delineation for prostate cancer radiotherapy with few-shot training combining imaging and clinical notes from each center.

Result: Outperformed baseline models, particularly in settings with high inter-center variability or limited data availability. Enabled model customization to local clinical preferences without cross-institutional data exchange.

Conclusion: MoME is suitable for resource-constrained settings while promoting broadly generalizable medical AI that adapts to diverse clinical strategies and institutional protocols.

Abstract: Clinical decision-making reflects diverse strategies shaped by regional patient populations and institutional protocols. However, most existing medical artificial intelligence (AI) models are trained on highly prevalent data patterns, which reinforces biases and fails to capture the breadth of clinical expertise. Inspired by the recent advances in Mixture of Experts (MoE), we propose a Mixture of Multicenter Experts (MoME) framework to address AI bias in the medical domain without requiring data sharing across institutions. MoME integrates specialized expertise from diverse clinical strategies to enhance model generalizability and adaptability across medical centers. We validate this framework using a multimodal target volume delineation model for prostate cancer radiotherapy. With few-shot training that combines imaging and clinical notes from each center, the model outperformed baselines, particularly in settings with high inter-center variability or limited data availability. Furthermore, MoME enables model customization to local clinical preferences without cross-institutional data exchange, making it especially suitable for resource-constrained settings while promoting broadly generalizable medical AI.

Method: Encodes medical data into 1D latent vectors that modulate a shared meta-learned neural field. Introduces non-constant frequency parameters in SIREN activations and connects ω-schedule to layer-wise learning rates. Uses scalable meta-learning with sparse supervision to reduce memory and computation.

Result: Evaluated across diverse medical datasets and shown to solve relevant downstream tasks. Released code, model weights, and MedNF dataset containing >500k latent vectors for multi-instance medical neural fields.

Conclusion: MedFuncta provides an effective framework for large-scale neural field training in medical imaging, addressing scalability challenges while maintaining performance and enabling generalization across datasets.

Abstract: Research in medical imaging primarily focuses on discrete data representations that poorly scale with grid resolution and fail to capture the often continuous nature of the underlying signal. Neural Fields (NFs) offer a powerful alternative by modeling data as continuous functions. While single-instance NFs have successfully been applied in medical contexts, extending them to large-scale medical datasets remains an open challenge. We therefore introduce MedFuncta, a unified framework for large-scale NF training on diverse medical signals. Building on Functa, our approach encodes data into a unified representation, namely a 1D latent vector, that modulates a shared, meta-learned NF, enabling generalization across a dataset. We revisit common design choices, introducing a non-constant frequency parameter $\omega$ in widely used SIREN activations, and establish a connection between this $\omega$-schedule and layer-wise learning rates, relating our findings to recent work in theoretical learning dynamics. We additionally introduce a scalable meta-learning strategy for shared network learning that employs sparse supervision during training, thereby reducing memory consumption and computational overhead while maintaining competitive performance. Finally, we evaluate MedFuncta across a diverse range of medical datasets and show how to solve relevant downstream tasks on our neural data representation. To promote further research in this direction, we release our code, model weights and the first large-scale dataset - MedNF - containing > 500 k latent vectors for multi-instance medical NFs.

Method: Proposes HPGN that integrates compression and illumination priors, utilizes JPEG quality factor and DCT quantization matrix to design plug-and-play modules, and employs random QF generation strategy for training to handle different compression levels.

Result: Experimental results demonstrate the superiority of the proposed method over existing approaches.

Conclusion: The HPGN framework effectively enhances compressed low-light images by leveraging both compression and illumination priors, providing a unified solution for varying compression qualities with a single model.

Abstract: In practical applications, low-light images are often compressed for efficient storage and transmission. Most existing methods disregard compression artifacts removal or hardly establish a unified framework for joint task enhancement of low-light images with varying compression qualities. To address this problem, we propose a hybrid priors-guided network (HPGN) that enhances compressed low-light images by integrating both compression and illumination priors. Our approach fully utilizes the JPEG quality factor (QF) and DCT quantization matrix to guide the design of efficient plug-and-play modules for joint tasks. Additionally, we employ a random QF generation strategy to guide model training, enabling a single model to enhance low-light images with different compression levels. Experimental results demonstrate the superiority of our proposed method..

Method: Developed a custom-built dense linear camera array to create a novel multi-camera dataset, then evaluated classical and deep learning frame interpolators against 3D Gaussian Splatting for view in-betweening tasks.

Result: On real image data, deep learning methods didn’t significantly outperform classical methods, with 3D Gaussian Splatting underperforming frame interpolators by up to 3.5 dB PSNR. However, in synthetic scenes, 3D Gaussian Splatting outperformed frame interpolation by almost 5 dB PSNR at 95% confidence.

Conclusion: The performance gap between frame interpolation and view synthesis methods depends heavily on the data domain (real vs synthetic), highlighting the importance of appropriate dataset design for fair evaluation and method selection.

Abstract: Many methods exist for frame synthesis in image sequences but can be broadly categorised into frame interpolation and view synthesis techniques. Fundamentally, both frame interpolation and view synthesis tackle the same task, interpolating a frame given surrounding frames in time or space. However, most frame interpolation datasets focus on temporal aspects with single cameras moving through time and space, while view synthesis datasets are typically biased toward stereoscopic depth estimation use cases. This makes direct comparison between view synthesis and frame interpolation methods challenging. In this paper, we develop a novel multi-camera dataset using a custom-built dense linear camera array to enable fair comparison between these approaches. We evaluate classical and deep learning frame interpolators against a view synthesis method (3D Gaussian Splatting) for the task of view in-betweening. Our results reveal that deep learning methods do not significantly outperform classical methods on real image data, with 3D Gaussian Splatting actually underperforming frame interpolators by as much as 3.5 dB PSNR. However, in synthetic scenes, the situation reverses – 3D Gaussian Splatting outperforms frame interpolation algorithms by almost 5 dB PSNR at a 95% confidence level.

Method: The authors investigate simple motion field processing techniques and propose a metric based on measuring motion field divergence, evaluated using the BVI-VFI dataset with perceptual scores.

Result: The proposed motion divergence metric achieves PLCC=0.51 correlation with perceptual scores and provides 2.7x speedup compared to FloLPIPS, favoring perceptually pleasing results over traditional metrics.

Conclusion: Motion field divergence provides an efficient and perceptually relevant quality metric for video frame interpolation that complements traditional objective metrics.

Abstract: Video frame interpolation (VFI) offers a way to generate intermediate frames between consecutive frames of a video sequence. Although the development of advanced frame interpolation algorithms has received increased attention in recent years, assessing the perceptual quality of interpolated content remains an ongoing area of research. In this paper, we investigate simple ways to process motion fields, with the purposes of using them as video quality metric for evaluating frame interpolation algorithms. We evaluate these quality metrics using the BVI-VFI dataset which contains perceptual scores measured for interpolated sequences. From our investigation we propose a motion metric based on measuring the divergence of motion fields. This metric correlates reasonably with these perceptual scores (PLCC=0.51) and is more computationally efficient (x2.7 speedup) compared to FloLPIPS (a well known motion-based metric). We then use our new proposed metrics to evaluate a range of state of the art frame interpolation metrics and find our metrics tend to favour more perceptual pleasing interpolated frames that may not score highly in terms of PSNR or SSIM.

Method: Fine-tuned DINOv3-H+ vision transformer pretrained on natural images using low-rank adaptation (LoRA) with only ~1.3M parameters, combined with extensive augmentation and domain-weighted Focal Loss to handle domain heterogeneity.

Result: Achieved second place on the preliminary test set of MIDOG 2025 challenge, demonstrating effective transfer from natural images to histopathology despite domain gap.

Conclusion: DINOv3 pretraining combined with efficient LoRA fine-tuning strategy provides robust and state-of-the-art performance for atypical mitosis classification in histopathology images.

Abstract: Atypical mitotic figures (AMFs) represent abnormal cell division associated with poor prognosis. Yet their detection remains difficult due to low prevalence, subtle morphology, and inter-observer variability. The MIDOG 2025 challenge introduces a benchmark for AMF classification across multiple domains. In this work, we fine-tuned the recently published DINOv3-H+ vision transformer, pretrained on natural images, using low-rank adaptation (LoRA), training only ~1.3M parameters in combination with extensive augmentation and a domain-weighted Focal Loss to handle domain heterogeneity. Despite the domain gap, our fine-tuned DINOv3 transfers effectively to histopathology, reaching second place on the preliminary test set. These results highlight the advantages of DINOv3 pretraining and underline the efficiency and robustness of our fine-tuning strategy, yielding state-of-the-art results for the atypical mitosis classification challenge in MIDOG 2025.

Method: Used Pathology Foundation Models pre-trained on large histopathology datasets with parameter-efficient fine-tuning via low-rank adaptation. Incorporated ConvNeXt V2 architecture, employed fisheye transform to emphasize mitoses, used Fourier Domain Adaptation with ImageNet target images, and ensembled multiple PFMs to integrate complementary morphological insights.

Result: Achieved competitive balanced accuracy on the Preliminary Evaluation Phase dataset.

Conclusion: The ensemble approach combining multiple pathology foundation models with advanced architectural components and domain adaptation techniques provides an effective solution for the challenging task of mitotic figure classification.

Abstract: Mitotic figures are classified into typical and atypical variants, with atypical counts correlating strongly with tumor aggressiveness. Accurate differentiation is therefore essential for patient prognostication and resource allocation, yet remains challenging even for expert pathologists. Here, we leveraged Pathology Foundation Models (PFMs) pre-trained on large histopathology datasets and applied parameter-efficient fine-tuning via low-rank adaptation. In addition, we incorporated ConvNeXt V2, a state-of-the-art convolutional neural network architecture, to complement PFMs. During training, we employed a fisheye transform to emphasize mitoses and Fourier Domain Adaptation using ImageNet target images. Finally, we ensembled multiple PFMs to integrate complementary morphological insights, achieving competitive balanced accuracy on the Preliminary Evaluation Phase dataset.

Method: Proposed Feature-Adaptive Spatial Localization model (FASL-Seg) with two distinct processing streams: Low-Level Feature Projection (LLFP) and High-Level Feature Projection (HLFP) for varying feature resolutions.

Result: Achieved mIoU of 72.71% on parts/anatomy segmentation (EndoVis18), improving SOTA by 5%. Also achieved 85.61% and 72.78% mIoU on tool type segmentation in EndoVis18 and EndoVis17 respectively, outperforming SOTA overall performance.

Conclusion: Dual processing streams for varying feature resolutions effectively improve surgical scene segmentation for both anatomy and instruments, demonstrating consistent performance across different classes.

Abstract: The growing popularity of robotic minimally invasive surgeries has made deep learning-based surgical training a key area of research. A thorough understanding of the surgical scene components is crucial, which semantic segmentation models can help achieve. However, most existing work focuses on surgical tools and overlooks anatomical objects. Additionally, current state-of-the-art (SOTA) models struggle to balance capturing high-level contextual features and low-level edge features. We propose a Feature-Adaptive Spatial Localization model (FASL-Seg), designed to capture features at multiple levels of detail through two distinct processing streams, namely a Low-Level Feature Projection (LLFP) and a High-Level Feature Projection (HLFP) stream, for varying feature resolutions - enabling precise segmentation of anatomy and surgical instruments. We evaluated FASL-Seg on surgical segmentation benchmark datasets EndoVis18 and EndoVis17 on three use cases. The FASL-Seg model achieves a mean Intersection over Union (mIoU) of 72.71% on parts and anatomy segmentation in EndoVis18, improving on SOTA by 5%. It further achieves a mIoU of 85.61% and 72.78% in EndoVis18 and EndoVis17 tool type segmentation, respectively, outperforming SOTA overall performance, with comparable per-class SOTA results in both datasets and consistent performance in various classes for anatomy and instruments, demonstrating the effectiveness of distinct processing streams for varying feature resolutions.