Method: Proposes a unified multi-task learning framework with dynamic prompt scheduling mechanism, including a prompt pool, task-aware scheduling strategy, task embeddings with gating mechanism for prompt fusion, and joint multi-task optimization with automatic learning of scheduling weights.

Result: The prompt scheduling method significantly improves performance on language understanding and knowledge reasoning tasks, demonstrating advantages in maintaining model stability and enhancing transferability across different task numbers and prompt temperature parameters.

Conclusion: The proposed framework effectively addresses task interference and negative transfer, demonstrating strong applicability and effectiveness in unified multi-task modeling and cross-domain adaptation for large language models.

Abstract: This study addresses the generalization limitations commonly observed in large language models under multi-task and cross-domain settings. Unlike prior methods such as SPoT, which depends on fixed prompt templates, our study introduces a unified multi-task learning framework with dynamic prompt scheduling mechanism. By introducing a prompt pool and a task-aware scheduling strategy, the method dynamically combines and aligns prompts for different tasks. This enhances the model’s ability to capture semantic differences across tasks. During prompt fusion, the model uses task embeddings and a gating mechanism to finely control the prompt signals. This ensures alignment between prompt content and task-specific demands. At the same time, it builds flexible sharing pathways across tasks. In addition, the proposed optimization objective centers on joint multi-task learning. It incorporates an automatic learning strategy for scheduling weights, which effectively mitigates task interference and negative transfer. To evaluate the effectiveness of the method, a series of sensitivity experiments were conducted. These experiments examined the impact of prompt temperature parameters and task number variation. The results confirm the advantages of the proposed mechanism in maintaining model stability and enhancing transferability. Experimental findings show that the prompt scheduling method significantly improves performance on a range of language understanding and knowledge reasoning tasks. These results fully demonstrate its applicability and effectiveness in unified multi-task modeling and cross-domain adaptation.

Method: Developed a benchmark framework that categorizes mathematical problems into 12 skill dimensions across three domains, allowing for multidimensional, skill-based evaluation of LLMs.

Result: The benchmark successfully constructed skill profiles for models like GPT-5-thinking, revealing their strengths, weaknesses, and differences compared to other models like o4-mini-high.

Conclusion: GAUSS provides a valuable multidimensional evaluation approach that faithfully represents LLMs’ underlying mathematical intelligence through skill-based profiling.

Abstract: We introduce \textbf{GAUSS} (\textbf{G}eneral \textbf{A}ssessment of \textbf{U}nderlying \textbf{S}tructured \textbf{S}kills in Mathematics), a benchmark that evaluates LLMs’ mathematical abilities across twelve core skill dimensions, grouped into three domains: knowledge and understanding, problem solving and communication, and meta-skills and creativity. By categorizing problems according to cognitive skills and designing tasks that isolate specific abilities, GAUSS constructs comprehensive, fine-grained, and interpretable profiles of models’ mathematical abilities. These profiles faithfully represent their underlying mathematical intelligence. To exemplify how to use the \textsc{GAUSS} benchmark, we have derived the skill profile of \textsc{GPT-5-thinking}, revealing its strengths and weaknesses as well as its differences relative to \textsc{o4-mini-high}, thereby underscoring the value of multidimensional, skill-based evaluation.

Method: The approach frames event causality using the Rubin Causal Model: the first event is treatment, the second is outcome. Since direct manipulation isn’t possible in text, they use synthetic control methods to generate ’twin’ comparisons from existing corpora, employing text embedding synthesis and inversion techniques.

Result: The method demonstrates superior performance on the causality benchmark COPES-hard, outperforming previous methods including GPT-4, showing more robust identification of causal relations.

Conclusion: The synthetic control approach based on the Rubin Causal Model provides a more reliable framework for event causality identification compared to traditional pattern-based methods, effectively addressing limitations of previous approaches.

Abstract: Event causality identification (ECI), a process that extracts causal relations between events from text, is crucial for distinguishing causation from correlation. Traditional approaches to ECI have primarily utilized linguistic patterns and multi-hop relational inference, risking false causality identification due to informal usage of causality and specious graphical inference. In this paper, we adopt the Rubin Causal Model to identify event causality: given two temporally ordered events, we see the first event as the treatment and the second one as the observed outcome. Determining their causality involves manipulating the treatment and estimating the resultant change in the likelihood of the outcome. Given that it is only possible to implement manipulation conceptually in the text domain, as a work-around, we try to find a twin for the protagonist from existing corpora. This twin should have identical life experiences with the protagonist before the treatment but undergoes an intervention of treatment. However, the practical difficulty of locating such a match limits its feasibility. Addressing this issue, we use the synthetic control method to generate such a twin’ from relevant historical data, leveraging text embedding synthesis and inversion techniques. This approach allows us to identify causal relations more robustly than previous methods, including GPT-4, which is demonstrated on a causality benchmark, COPES-hard.

Method: ZERA scores prompts using eight generalizable criteria with automatically inferred weights, and revises prompts based on these structured critiques to enable fast convergence using minimal examples.

Result: Experimental evaluation across five LLMs and nine diverse datasets shows consistent improvements over strong baselines in reasoning, summarization, and code generation tasks.

Conclusion: ZERA enables more effective prompt construction through principled, low-overhead refinement, with ablation studies confirming the contribution of each component.

Abstract: Automatic Prompt Optimization (APO) improves large language model (LLM) performance by refining prompts for specific tasks. However, prior APO methods typically focus only on user prompts, rely on unstructured feedback, and require large sample sizes and long iteration cycles-making them costly and brittle. We propose ZERA (Zero-init Instruction Evolving Refinement Agent), a novel framework that jointly optimizes both system and user prompts through principled, low-overhead refinement. ZERA scores prompts using eight generalizable criteria with automatically inferred weights, and revises prompts based on these structured critiques. This enables fast convergence to high-quality prompts using minimal examples and short iteration cycles. We evaluate ZERA across five LLMs and nine diverse datasets spanning reasoning, summarization, and code generation tasks. Experimental results demonstrate consistent improvements over strong baselines. Further ablation studies highlight the contribution of each component to more effective prompt construction. Our implementation including all prompts is publicly available at https://github.com/younatics/zera-agent.

Method: Proposes model kinship concept and Top-k Greedy Merging with Model Kinship strategy, using comprehensive empirical analysis to study model evolution through iterative merging with biological evolution analogy.

Result: Shows that model kinship is closely linked to performance improvements in merging, providing a useful criterion for selecting candidate models and enabling continuous merging while mitigating performance degradation.

Conclusion: Model kinship serves as an effective guiding criterion for model merging, facilitating more effective model evolution by preventing local optima issues and improving benchmark performance.

Abstract: Model merging has emerged as a key technique for enhancing the capabilities and efficiency of Large Language Models (LLMs). The open-source community has driven model evolution by iteratively merging existing models, yet a principled understanding of the gains and underlying factors in model merging remains limited. In this work, we study model evolution through iterative merging, drawing an analogy to biological evolution, and introduce the concept of model kinship, the degree of similarity or relatedness between LLMs. Through comprehensive empirical analysis, we show that model kinship is closely linked to the performance improvements achieved by merging, providing a useful criterion for selecting candidate models. Building on this insight, we propose a new model merging strategy: Top-k Greedy Merging with Model Kinship, which can improve benchmark performance. Specifically, we discover that incorporating model kinship as a guiding criterion enables continuous merging while mitigating performance degradation caused by local optima, thereby facilitating more effective model evolution. Code is available at https://github.com/zjunlp/ModelKinship.

Method: Introduce SciAux dataset derived from ScienceQA to systematically test model robustness against different types of external information.

Result: Thinking mode is a double-edged sword: helpful context improves accuracy but misleading information causes catastrophic performance drop that is amplified by the thinking process.

Conclusion: The challenge is not just making models think, but endowing them with critical evaluation skills to assess the information their reasoning is based on.

Abstract: The capacity of Large Language Models (LLMs) to reason is fundamental to their application in complex, knowledge-intensive domains. In real-world scenarios, LLMs are often augmented with external information that can be helpful, irrelevant, or even misleading. This paper investigates the causal impact of such auxiliary information on the reasoning process of LLMs with explicit step-by-step thinking capabilities. We introduce SciAux, a new dataset derived from ScienceQA, to systematically test the robustness of the model against these types of information. Our findings reveal a critical vulnerability: the model’s deliberative “thinking mode” is a double-edged sword. While helpful context improves accuracy, misleading information causes a catastrophic drop in performance, which is amplified by the thinking process. Instead of conferring robustness, thinking reinforces the degree of error when provided with misinformation. This highlights that the challenge is not merely to make models “think”, but to endow them with the critical faculty to evaluate the information upon which their reasoning is based. The SciAux dataset is available at https://huggingface.co/datasets/billhdzhao/SciAux.

Method: Proposes a process-supervised multi-agent framework with Decision Maker and Knowledge Selector agents, using LLM-as-a-Judge for process-level rewards, tree-structured rollouts, and PPO training for end-to-end optimization.

Result: Achieves higher accuracy, more stable convergence, and more interpretable reasoning on single-hop and multi-hop QA benchmarks compared to standard RAG baselines.

Conclusion: The modular framework effectively bridges retriever-generator gaps without modifying existing components, making it practical for real-world RAG applications.

Abstract: Retrieval-Augmented Generation (RAG) enables large language models (LLMs) to access external knowledge sources, but the effectiveness of RAG relies on the coordination between the retriever and the generator. Since these components are developed independently, their interaction is often suboptimal: the retriever may return irrelevant or redundant documents, while the generator may fail to fully leverage retrieved evidence. In this work, we propose a process-supervised multi-agent framework to bridge the gap between retriever and generator. The framework introduces two lightweight agents: a Decision Maker, which determines when to continue retrieval or stop for answer generation, and a Knowledge Selector, which filters retrieved documents to retain only the most useful evidence. To provide fine-grained supervision, we employ an LLM-as-a-Judge that evaluates each intermediate action with process-level rewards, ensuring more accurate credit assignment than relying solely on final answer correctness. We further adopt a tree-structured rollout strategy to explore diverse reasoning paths, and train both agents with Proximal Policy Optimization (PPO) in an end-to-end manner. Experiments on single-hop and multi-hop question answering benchmarks show that our approach achieves higher accuracy, more stable convergence, and produces more interpretable reasoning trajectories compared with standard RAG baselines. Importantly, the proposed framework is modular and plug-and-play, requiring no modification to the retriever or generator, making it practical for real-world RAG applications.

Method: ERFC architecture considers multi-modalities, different emotion attributes, context, and interdependencies between speakers’ utterances in conversations.

Result: Intensive experiments on IEMOCAP dataset demonstrated the feasibility of the proposed ERFC approach.

Conclusion: The ERFC approach provides significant business value for call center applications where customer happiness is paramount, enabling agents to transform unhappy customers into happy ones through timely emotional insights.

Abstract: Emotion Recognition in Conversation has been seen to be widely applicable in call center analytics, opinion mining, finance, retail, healthcare, and other industries. In a call center scenario, the role of the call center agent is not just confined to receiving calls but to also provide good customer experience by pacifying the frustration or anger of the customers. This can be achieved by maintaining neutral and positive emotion from the agent. As in any conversation, the emotion of one speaker is usually dependent on the emotion of other speaker. Hence the positive emotion of an agent, accompanied with the right resolution will help in enhancing customer experience. This can change an unhappy customer to a happy one. Imparting the right resolution at right time becomes easier if the agent has the insight of the emotion of future utterances. To predict the emotions of the future utterances we propose a novel architecture, Emotion Recognition and Forecasting in Conversation. Our proposed ERFC architecture considers multi modalities, different attributes of emotion, context and the interdependencies of the utterances of the speakers in the conversation. Our intensive experiments on the IEMOCAP dataset have shown the feasibility of the proposed ERFC. This approach can provide a tremendous business value for the applications like call center, where the happiness of customer is utmost important.

Method: Developed Guided-CoT prompting technique to handle in-context policy definitions, evaluated 8 open-source LLMs with various prompting strategies, and introduced metrics to quantify semantic divergence in model rationales.

Result: Guided-CoT significantly improved performance across all models regardless of size or reasoning capability. Llama 3.1 70B outperformed fine-tuned GPT-3.5. Analysis revealed notable differences in LLM utility, explainability, and reliability.

Conclusion: LLMs can be effectively guided by in-context policy definitions for hate speech detection, with Guided-CoT proving particularly effective. However, models show paradoxical behaviors and semantic divergence in rationales, highlighting the need for careful evaluation of their utility and reliability.

Abstract: Detecting hateful content is a challenging and important problem. Automated tools, like machine-learning models, can help, but they require continuous training to adapt to the ever-changing landscape of social media. In this work, we evaluate eight open-source LLMs’ capability to detect antisemitic content, specifically leveraging in-context definition as a policy guideline. We explore various prompting techniques and design a new CoT-like prompt, Guided-CoT. Guided-CoT handles the in-context policy well, increasing performance across all evaluated models, regardless of decoding configuration, model sizes, or reasoning capability. Notably, Llama 3.1 70B outperforms fine-tuned GPT-3.5. Additionally, we examine LLM errors and introduce metrics to quantify semantic divergence in model-generated rationales, revealing notable differences and paradoxical behaviors among LLMs. Our experiments highlight the differences observed across LLMs’ utility, explainability, and reliability.

Method: Created a comprehensive dataset spanning all Indian states and union territories, covering cultural themes like festivals, attire, cuisines, art forms, and historical heritage. Evaluated various vision-language models including open-source, proprietary, reasoning-specialized, and Indic-focused models in zero-shot and chain-of-thought settings.

Result: Revealed significant limitations in current models’ ability to reason over culturally grounded multimodal inputs, especially for low-resource languages and less-documented traditions.

Conclusion: DRISHTIKON fills a critical gap in inclusive AI research and provides a robust benchmark to advance culturally aware, multimodally competent language technologies.

Abstract: We introduce DRISHTIKON, a first-of-its-kind multimodal and multilingual benchmark centered exclusively on Indian culture, designed to evaluate the cultural understanding of generative AI systems. Unlike existing benchmarks with a generic or global scope, DRISHTIKON offers deep, fine-grained coverage across India’s diverse regions, spanning 15 languages, covering all states and union territories, and incorporating over 64,000 aligned text-image pairs. The dataset captures rich cultural themes including festivals, attire, cuisines, art forms, and historical heritage amongst many more. We evaluate a wide range of vision-language models (VLMs), including open-source small and large models, proprietary systems, reasoning-specialized VLMs, and Indic-focused models, across zero-shot and chain-of-thought settings. Our results expose key limitations in current models’ ability to reason over culturally grounded, multimodal inputs, particularly for low-resource languages and less-documented traditions. DRISHTIKON fills a vital gap in inclusive AI research, offering a robust testbed to advance culturally aware, multimodally competent language technologies.

Method: Prefix-to-Tree (P2T) converts response groups into prefix trees to compute nonparametric prefix values. TEMPO builds on P2T by adding branch-gated temporal-difference corrections to GRPO’s group-relative outcome signal, providing precise token-level credit at branching points without extra models.

Result: TEMPO outperforms PPO and GRPO on Qwen3-1.7B/4B models across in-distribution (MATH, MedQA) and out-of-distribution (GSM-HARD, AMC23, MedMCQA, MMLU-Medical) benchmarks, achieving higher validation accuracy with similar training time.

Conclusion: TEMPO provides an effective critic-free alternative for LLM reasoning tasks, handling token-level credit assignment through tree-based prefix value estimation and branch-gated TD corrections, demonstrating superior performance over existing methods.

Abstract: Reinforcement learning improves LLM reasoning, yet sparse delayed reward over long sequences makes token-level credit assignment the key bottleneck. We study the verifiable-reward setting, where the final answer is checkable and multiple responses can be drawn per prompt. Reasoning tasks in math and medical QA align with this setup, where only a few decision tokens significantly impact the outcome. PPO offers token-level advantages with a learned value model, but it is complex to train both the actor and critic models simultaneously, and it is not easily generalizable, as the token-level values from the critic model can make training prone to overfitting. GRPO is critic-free and supports verifiable rewards, but spreads a single sequence-level return across tokens and ignores branching. We introduce \textbf{Prefix-to-Tree (P2T)}, a simple procedure that converts a group of responses into a prefix tree and computes \emph{nonparametric} prefix values (V(s)) by aggregating descendant outcomes. Built on P2T, we propose \textbf{TEMPO} (\emph{\textbf{T}ree-\textbf{E}stimated \textbf{M}ean Prefix Value for \textbf{P}olicy \textbf{O}ptimization}), a critic-free algorithm that augments the group-relative outcome signal of GRPO with \emph{branch-gated} temporal-difference corrections derived from the tree. At non-branch tokens, the temporal-difference (TD) term is zero, so TEMPO reduces to GRPO; at branching tokens, it supplies precise token-level credit without a learned value network or extra judges/teachers. On Qwen3-1.7B/4B, TEMPO outperforms PPO and GRPO on in-distribution (MATH, MedQA) and out-of-distribution (GSM-HARD, AMC23, MedMCQA, MMLU-Medical) benchmarks, and reaches higher validation accuracy with roughly the same wall-clock time.

Method: Treat LLMs as reward models of KG reasoning paths, where models learn to judge whether candidate paths lead to correct diagnoses. Evaluated five task formulations and eight training paradigms, testing generalization to downstream diagnostic tasks.

Result: Experiments with three open-source LLMs show promise and brittleness: specific reward optimization and distillation lead to strong path-judging performance, but transferability to downstream tasks remains weak.

Conclusion: Provides the first systematic assessment of ‘reward model style’ reasoning over clinical KGs, offering insights into how structured, reward-based supervision influences diagnostic reasoning in GenAI healthcare systems.

Abstract: Large language models (LLMs) show promise for diagnostic reasoning but often lack reliable, knowledge grounded inference. Knowledge graphs (KGs), such as the Unified Medical Language System (UMLS), offer structured biomedical knowledge that can support trustworthy reasoning. Prior approaches typically integrate KGs via retrieval augmented generation or fine tuning, inserting KG content into prompts rather than enabling structured reasoning. We explore an alternative paradigm: treating the LLM as a reward model of KG reasoning paths, where the model learns to judge whether a candidate path leads to correct diagnosis for a given patient input. This approach is inspired by recent work that leverages reward training to enhance model reasoning abilities, and grounded in computational theory, which suggests that verifying a solution is often easier than generating one from scratch. It also parallels physicians’ diagnostic assessment, where they judge which sequences of findings and intermediate conditions most plausibly support a diagnosis. We first systematically evaluate five task formulation for knowledge path judging and eight training paradigm. Second, we test whether the path judging abilities generalize to downstream diagnostic tasks, including diagnosis summarization and medical question answering. Experiments with three open source instruct-tuned LLMs reveal both promise and brittleness: while specific reward optimization and distillation lead to strong path-judging performance, the transferability to downstream tasks remain weak. Our finding provides the first systematic assessment of “reward model style” reasoning over clinical KGs, offering insights into how structured, reward-based supervision influences diagnostic reasoning in GenAI systems for healthcare.

Method: Proposes a hybrid architecture integrating Mamba and Transformer blocks, introduces data construction methods capturing temporal and spatial dependencies, and employs progressive training strategies.

Result: LongLLaVA achieves competitive results across various benchmarks while maintaining high throughput and low memory consumption, capable of processing nearly 1,000 images on a single A100 80GB GPU.

Conclusion: The model demonstrates an effective balance between efficiency and performance, showing strong potential for wide-ranging multi-modal applications requiring long-context capabilities.

Abstract: Expanding the long-context capabilities of Multi-modal Large Language Models~(MLLMs) is critical for advancing video understanding and high-resolution image analysis. Achieving this requires systematic improvements in model architecture, data construction, and training strategies, particularly to address challenges such as performance degradation with increasing image counts and high computational costs. In this paper, we propose a hybrid architecture that integrates Mamba and Transformer blocks, introduce data construction methods that capture both temporal and spatial dependencies, and employ a progressive training strategy. Our released model, LongLLaVA (\textbf{Long}-Context \textbf{L}arge \textbf{L}anguage \textbf{a}nd \textbf{V}ision \textbf{A}ssistant), demonstrates an effective balance between efficiency and performance. LongLLaVA achieves competitive results across various benchmarks while maintaining high throughput and low memory consumption. Notably, it can process nearly one thousand images on a single A100 80GB GPU, underscoring its potential for a wide range of multi-modal applications.

Method: SubSpec constructs a highly aligned draft model by generating low-bit quantized substitute layers from offloaded target LLM portions. It shares remaining GPU-resident layers and KV-Cache to reduce memory overhead and enhance alignment, enabling parallel verification of multiple draft tokens with a single forward pass.

Result: SubSpec achieves 9.1x speedup for Qwen2.5 7B on MT-Bench (8GB VRAM limit) and an average of 12.5x speedup for Qwen2.5 32B on popular generation benchmarks (24GB VRAM limit), with high average acceptance length.

Conclusion: SubSpec provides a plug-and-play, training-free solution that significantly accelerates parameter offloading while maintaining lossless quality, addressing key limitations of existing methods for deploying large language models on memory-constrained hardware.

Abstract: The immense model sizes of large language models (LLMs) challenge deployment on memory-limited consumer GPUs. Although model compression and parameter offloading are common strategies to address memory limitations, compression can degrade quality, and offloading maintains quality but suffers from slow inference. Speculative decoding presents a promising avenue to accelerate parameter offloading, utilizing a fast draft model to propose multiple draft tokens, which are then verified by the target LLM in parallel with a single forward pass. This method reduces the time-consuming data transfers in forward passes that involve offloaded weight transfers. Existing methods often rely on pretrained weights of the same family, but require additional training to align with custom-trained models. Moreover, approaches that involve draft model training usually yield only modest speedups. This limitation arises from insufficient alignment with the target model, preventing higher token acceptance lengths. To address these challenges and achieve greater speedups, we propose SubSpec, a plug-and-play method to accelerate parameter offloading that is lossless and training-free. SubSpec constructs a highly aligned draft model by generating low-bit quantized substitute layers from offloaded target LLM portions. Additionally, our method shares the remaining GPU-resident layers and the KV-Cache, further reducing memory overhead and enhance alignment. SubSpec achieves a high average acceptance length, delivering 9.1x speedup for Qwen2.5 7B on MT-Bench (8GB VRAM limit) and an average of 12.5x speedup for Qwen2.5 32B on popular generation benchmarks (24GB VRAM limit).

Method: A parallel speech document alignment method that monotonically aligns speech segment embeddings, comparing against Global Mining and Local Mining variants of speech mining.

Result: Applied to 3,000 hours of unlabeled parallel English-German speech, yielding 1,000 hours of high-quality alignments. Speech Vecalign improved En-to-De and De-to-En performance by 0.37 and 0.18 ASR-BLEU respectively over Global Mining, matching or outperforming SpeechMatrix with 8x fewer raw documents.

Conclusion: Speech Vecalign is an effective text-free speech alignment method that produces superior alignments and enables training high-quality speech-to-speech translation models with significantly less raw data.

Abstract: We present Speech Vecalign, a parallel speech document alignment method that monotonically aligns speech segment embeddings and does not depend on text transcriptions. Compared to the baseline method Global Mining, a variant of speech mining, Speech Vecalign produces longer speech-to-speech alignments. It also demonstrates greater robustness than Local Mining, another speech mining variant, as it produces less noise. We applied Speech Vecalign to 3,000 hours of unlabeled parallel English-German (En-De) speech documents from VoxPopuli, yielding about 1,000 hours of high-quality alignments. We then trained En-De speech-to-speech translation models on the aligned data. Speech Vecalign improves the En-to-De and De-to-En performance over Global Mining by 0.37 and 0.18 ASR-BLEU, respectively. Moreover, our models match or outperform SpeechMatrix model performance, despite using 8 times fewer raw speech documents.

Method: Trains models to compress verbose thought steps into compact representations using data construction, hidden state mapping to gist tokens, and specialized attention masks. Introduces Dependency metric to quantify compression.

Result: Extensive experiments on four datasets and two models show reduced peak memory usage and inference time while maintaining competitive accuracy.

Conclusion: Provides a new direction for improving LLM efficiency in complex reasoning without sacrificing performance.

Abstract: Large language models (LLMs) have shown remarkable performance in complex reasoning tasks, but their efficiency is hindered by the substantial memory and computational costs associated with generating lengthy tokens. In this paper, we propose LightThinker, a novel method that enables LLMs to dynamically compress intermediate thoughts during reasoning. Inspired by human cognitive processes, LightThinker compresses verbose thought steps into compact representations and discards the original reasoning chains, thereby significantly reducing the number of tokens stored in the context window. This is achieved by training the model on when and how to perform compression through data construction, mapping hidden states to condensed gist tokens, and creating specialized attention masks. Additionally, we introduce the Dependency (Dep) metric to quantify the degree of compression by measuring the reliance on historical tokens during generation. Extensive experiments on four datasets and two models show that LightThinker reduces peak memory usage and inference time, while maintaining competitive accuracy. Our work provides a new direction for improving the efficiency of LLMs in complex reasoning tasks without sacrificing performance. Code is released at https://github.com/zjunlp/LightThinker.

Method: Collected 114 hours of spontaneous Thai dialogue with three participants, recorded simultaneously by nine single-channel devices at distances from 0.12m to 10m. Provided standard data splits and benchmarked Whisper variants under zero-shot and fine-tuned conditions.

Result: Fine-tuning on LOTUSDIS dramatically improved ASR robustness: reduced overall WER from 64.3 to 38.3 and far-field WER from 81.6 to 49.5, with largest gains on distant microphones.

Conclusion: Distance-diverse training data is crucial for robust far-field ASR. The corpus and baseline system are publicly available to promote reproducible research in this field.

Abstract: We present LOTUSDIS, a publicly available Thai meeting corpus designed to advance far-field conversational ASR. The dataset comprises 114 hours of spontaneous, unscripted dialogue collected in 15-20 minute sessions with three participants, where overlapping speech is frequent and natural. Speech was recorded simultaneously by nine independent single-channel devices spanning six microphone types at distances from 0.12 m to 10 m, preserving the authentic effects of reverberation, noise, and device coloration without relying on microphone arrays. We provide standard train, dev, test splits and release a reproducible baseline system. We benchmarked several Whisper variants under zero-shot and fine-tuned conditions. Off-the-shelf models showed strong degradation with distance, confirming a mismatch between pre-training data and Thai far-field speech. Fine-tuning on LOTUSDIS dramatically improved robustness: a Thai Whisper baseline reduced overall WER from 64.3 to 38.3 and far-field WER from 81.6 to 49.5, with especially large gains on the most distant microphones. These results underscore the importance of distance-diverse training data for robust ASR. The corpus is available under CC-BY-SA 4.0. We also release training and evaluation scripts as a baseline system to promote reproducible research in this field.

Method: The system performs streaming ASR and diarization, uses an LLM to deliver concise summaries, and accepts brief verbal feedback. It employs split-when-merged (SWM) technique to detect multi-speaker segments and online speaker enrollments based on user corrections.

Result: LLM-driven simulations on AMI test set show substantial reduction in DER by 9.92% and speaker confusion error by 44.23%. The system was analyzed under different settings including summary vs full transcript display and correction frequency.

Conclusion: The proposed LLM-assisted correction system effectively reduces speaker diarization errors through real-time user feedback and online enrollment techniques, demonstrating significant improvements in diarization accuracy.

Abstract: Most automatic speech processing systems operate in “open loop” mode without user feedback about who said what; yet, human-in-the-loop workflows can potentially enable higher accuracy. We propose an LLM-assisted speaker diarization correction system that lets users fix speaker attribution errors in real time. The pipeline performs streaming ASR and diarization, uses an LLM to deliver concise summaries to the users, and accepts brief verbal feedback that is immediately incorporated without disrupting interactions. Moreover, we develop techniques to make the workflow more effective: First, a split-when-merged (SWM) technique detects and splits multi-speaker segments that the ASR erroneously attributes to just a single speaker. Second, online speaker enrollments are collected based on users’ diarization corrections, thus helping to prevent speaker diarization errors from occurring in the future. LLM-driven simulations on the AMI test set indicate that our system substantially reduces DER by 9.92% and speaker confusion error by 44.23%. We further analyze correction efficacy under different settings, including summary vs full transcript display, the number of online enrollments limitation, and correction frequency.

Method: Introduces two simple strategies that modify training data by previewing partial answer prefixes to minimize perturbations to the model’s initial token distributions and maintain built-in safety mechanisms.

Result: Comprehensive experiments show that LookAhead Tuning effectively maintains model safety without sacrificing robust performance on downstream tasks.

Conclusion: LookAhead Tuning is positioned as a reliable and efficient solution for safe and effective adaptation of LLMs.

Abstract: Fine-tuning enables large language models (LLMs) to adapt to specific domains, but often compromises their previously established safety alignment. To mitigate the degradation of model safety during fine-tuning, we introduce LookAhead Tuning, a lightweight and effective data-driven approach that preserves safety during fine-tuning. The method introduces two simple strategies that modify training data by previewing partial answer prefixes, thereby minimizing perturbations to the model’s initial token distributions and maintaining its built-in safety mechanisms. Comprehensive experiments demonstrate that LookAhead Tuning effectively maintains model safety without sacrificing robust performance on downstream tasks. Our findings position LookAhead Tuning as a reliable and efficient solution for the safe and effective adaptation of LLMs.

Method: Proposes V2R dialogue type modeling norm violation progression, implements exemplar-based iterative refinement for pragmatic consistency, and constructs a dataset of 10,800 multi-turn dialogues with turn-level annotations for norm adherence, speaker intent, and emotional response.

Result: Human and LLM-based evaluations show NormGenesis significantly outperforms existing datasets in refinement quality, dialogue naturalness, and generalization performance. Models trained on V2R-augmented data exhibit improved pragmatic competence in ethically sensitive contexts.

Conclusion: Establishes a new benchmark for culturally adaptive dialogue modeling and provides a scalable methodology for norm-aware generation across linguistically and culturally diverse languages.

Abstract: Social norms govern culturally appropriate behavior in communication, enabling dialogue systems to produce responses that are not only coherent but also socially acceptable. We present NormGenesis, a multicultural framework for generating and annotating socially grounded dialogues across English, Chinese, and Korean. To model the dynamics of social interaction beyond static norm classification, we propose a novel dialogue type, Violation-to-Resolution (V2R), which models the progression of conversations following norm violations through recognition and socially appropriate repair. To improve pragmatic consistency in underrepresented languages, we implement an exemplar-based iterative refinement early in the dialogue synthesis process. This design introduces alignment with linguistic, emotional, and sociocultural expectations before full dialogue generation begins. Using this framework, we construct a dataset of 10,800 multi-turn dialogues annotated at the turn level for norm adherence, speaker intent, and emotional response. Human and LLM-based evaluations demonstrate that NormGenesis significantly outperforms existing datasets in refinement quality, dialogue naturalness, and generalization performance. We show that models trained on our V2R-augmented data exhibit improved pragmatic competence in ethically sensitive contexts. Our work establishes a new benchmark for culturally adaptive dialogue modeling and provides a scalable methodology for norm-aware generation across linguistically and culturally diverse languages.

Method: Built a dataset of Persian literary texts across 20 topics, assessed creativity along four dimensions (originality, fluency, flexibility, elaboration) using adapted Torrance Tests, employed LLM as automated judge with validation against human evaluations.

Result: Strong agreement between LLM and human judgments, identified both strengths and limitations in LLMs’ Persian literary generation, including analysis of literary device usage (simile, metaphor, hyperbole, antithesis).

Conclusion: LLMs show promise but need further refinement for Persian literary text generation, highlighting the importance of culturally-aware evaluation frameworks.

Abstract: Large language models (LLMs) have demonstrated notable creative abilities in generating literary texts, including poetry and short stories. However, prior research has primarily centered on English, with limited exploration of non-English literary traditions and without standardized methods for assessing creativity. In this paper, we evaluate the capacity of LLMs to generate Persian literary text enriched with culturally relevant expressions. We build a dataset of user-generated Persian literary spanning 20 diverse topics and assess model outputs along four creativity dimensions-originality, fluency, flexibility, and elaboration-by adapting the Torrance Tests of Creative Thinking. To reduce evaluation costs, we adopt an LLM as a judge for automated scoring and validate its reliability against human judgments using intraclass correlation coefficients, observing strong agreement. In addition, we analyze the models' ability to understand and employ four core literary devices: simile, metaphor, hyperbole, and antithesis. Our results highlight both the strengths and limitations of LLMs in Persian literary text generation, underscoring the need for further refinement.

Method: AutoSteer uses three components: Safety Awareness Score to identify safety-relevant layers, adaptive safety prober to detect toxic outputs from intermediate representations, and lightweight Refusal Head to intervene when risks are detected.

Result: Experiments on LLaVA-OV and Chameleon show AutoSteer significantly reduces Attack Success Rate for textual, visual, and cross-modal threats while maintaining general model abilities.

Conclusion: AutoSteer provides a practical, interpretable, and effective framework for safer deployment of multimodal AI systems.

Abstract: Recent progress in Multimodal Large Language Models (MLLMs) has unlocked powerful cross-modal reasoning abilities, but also raised new safety concerns, particularly when faced with adversarial multimodal inputs. To improve the safety of MLLMs during inference, we introduce a modular and adaptive inference-time intervention technology, AutoSteer, without requiring any fine-tuning of the underlying model. AutoSteer incorporates three core components: (1) a novel Safety Awareness Score (SAS) that automatically identifies the most safety-relevant distinctions among the model’s internal layers; (2) an adaptive safety prober trained to estimate the likelihood of toxic outputs from intermediate representations; and (3) a lightweight Refusal Head that selectively intervenes to modulate generation when safety risks are detected. Experiments on LLaVA-OV and Chameleon across diverse safety-critical benchmarks demonstrate that AutoSteer significantly reduces the Attack Success Rate (ASR) for textual, visual, and cross-modal threats, while maintaining general abilities. These findings position AutoSteer as a practical, interpretable, and effective framework for safer deployment of multimodal AI systems.

Method: Used 157 video-recorded conversations (42,559 sentences) to train deep learning and fine-tuned BERT models via next sentence prediction task, then calculated conversational alignment scores and assessed their association with SDM outcomes.

Result: Fine-tuned BERTbase achieved highest performance (recall@1: 0.640). Conversational alignment scores from both DL and BERT models showed significant associations with established SDM measurement tools (OPTION12 and DCS scores).

Conclusion: The study successfully introduces an automated, scalable methodology to measure SDM using explainable conversational alignment scores, enabling large-scale evaluation of SDM strategies.

Abstract: Shared decision-making (SDM) is necessary to achieve patient-centred care. Currently no methodology exists to automatically measure SDM at scale. This study aimed to develop an automated approach to measure SDM by using language modelling and the conversational alignment (CA) score. A total of 157 video-recorded patient-doctor conversations from a randomized multi-centre trial evaluating SDM decision aids for anticoagulation in atrial fibrillations were transcribed and segmented into 42,559 sentences. Context-response pairs and negative sampling were employed to train deep learning (DL) models and fine-tuned BERT models via the next sentence prediction (NSP) task. Each top-performing model was used to calculate four types of CA scores. A random-effects analysis by clinician, adjusting for age, sex, race, and trial arm, assessed the association between CA scores and SDM outcomes: the Decisional Conflict Scale (DCS) and the Observing Patient Involvement in Decision-Making 12 (OPTION12) scores. p-values were corrected for multiple comparisons with the Benjamini-Hochberg method. Among 157 patients (34% female, mean age 70 SD 10.8), clinicians on average spoke more words than patients (1911 vs 773). The DL model without the stylebook strategy achieved a recall@1 of 0.227, while the fine-tuned BERTbase (110M) achieved the highest recall@1 with 0.640. The AbsMax (18.36 SE7.74 p=0.025) and Max CA (21.02 SE7.63 p=0.012) scores generated with the DL without stylebook were associated with OPTION12. The Max CA score generated with the fine-tuned BERTbase (110M) was associated with the DCS score (-27.61 SE12.63 p=0.037). BERT model sizes did not have an impact the association between CA scores and SDM. This study introduces an automated, scalable methodology to measure SDM in patient-doctor conversations through explainable CA scores, with potential to evaluate SDM strategies at scale.

Method: Created SloPalSpeech dataset (2,806 hours of parliamentary speech), developed processing pipeline for alignment and segmentation, and fine-tuned multiple OpenAI Whisper models (small, medium, large-v3, large-v3-turbo) on this data.

Result: Achieved significant WER reductions (up to 70% for Whisper-small) on Slovak benchmarks like Common Voice and FLEURS, with fine-tuned small model approaching baseline performance of much larger models.

Conclusion: The SloPalSpeech dataset enables effective ASR for low-resource languages, and the authors publicly release the dataset, transcripts, and fine-tuned models to foster future research.

Abstract: Automatic Speech Recognition (ASR) for low-resource languages like Slovak is hindered by the scarcity of training data. To address this, we introduce SloPalSpeech, a new, large-scale Slovak ASR dataset containing 2,806 hours of speech from parliamentary proceedings. We developed a robust processing pipeline to align and segment long-form recordings into clean, 30-second audio-transcript pairs suitable for model training. We use this dataset to fine-tune several OpenAI Whisper models (small, medium, large-v3, and large-v3-turbo), achieving significant Word Error Rate (WER) reductions on standard Slovak benchmarks like Common Voice and FLEURS. For instance, the fine-tuned Whisper-small model’s WER dropped by up to 70%, approaching the baseline performance of the much larger Whisper-large-v3 model. To foster future research in low-resource speech recognition, we publicly release the complete SloPalSpeech dataset, the fully segmented transcripts (60 million words), and all our fine-tuned models.

Method: CogniLoad generates natural-language logic puzzles with tunable parameters: intrinsic difficulty (d) for intrinsic load, distractor-to-signal ratio (ρ) for extraneous load, and task length (N) as proxy for germane load demands.

Result: Evaluation of 22 state-of-the-art reasoning LLMs revealed task length as the dominant constraint, varied tolerances to intrinsic complexity, and U-shaped responses to distractor ratios.

Conclusion: CogniLoad provides a reproducible, scalable, and diagnostically rich tool for systematically analyzing LLM reasoning limitations and guiding future model development.

Abstract: Current benchmarks for long-context reasoning in Large Language Models (LLMs) often blur critical factors like intrinsic task complexity, distractor interference, and task length. To enable more precise failure analysis, we introduce CogniLoad, a novel synthetic benchmark grounded in Cognitive Load Theory (CLT). CogniLoad generates natural-language logic puzzles with independently tunable parameters that reflect CLT’s core dimensions: intrinsic difficulty ($d$) controls intrinsic load; distractor-to-signal ratio ($\rho$) regulates extraneous load; and task length ($N$) serves as an operational proxy for conditions demanding germane load. Evaluating 22 SotA reasoning LLMs, CogniLoad reveals distinct performance sensitivities, identifying task length as a dominant constraint and uncovering varied tolerances to intrinsic complexity and U-shaped responses to distractor ratios. By offering systematic, factorial control over these cognitive load dimensions, CogniLoad provides a reproducible, scalable, and diagnostically rich tool for dissecting LLM reasoning limitations and guiding future model development.

Method: LAWCAT integrates causal Conv1D layers for local dependency modeling and uses normalized gated linear attention to improve generalization across context lengths, enabling efficient knowledge distillation from pre-trained transformers.

Result: Distilling Mistral-7B with only 1K-length sequences achieved over 90% passkey retrieval accuracy up to 22K tokens. Llama3.2-1B LAWCAT variant showed competitive performance on long-context benchmarks while requiring less than 0.1% pre-training tokens compared to full pre-training.

Conclusion: LAWCAT provides an efficient pathway to high-performance linear models suitable for edge deployment, reducing reliance on extensive long-sequence training data and computational resources while offering faster inference for sequences over 8K tokens.

Abstract: Although transformer architectures have achieved state-of-the-art performance across diverse domains, their quadratic computational complexity with respect to sequence length remains a significant bottleneck, particularly for latency-sensitive long-context applications. While recent linear-complexity alternatives are increasingly powerful, effectively training them from scratch is still resource-intensive. To overcome these limitations, we propose LAWCAT (Linear Attention with Convolution Across Time), a novel linearization framework designed to efficiently transfer the capabilities of pre-trained transformers into a performant linear attention architecture. LAWCAT integrates causal Conv1D layers to enhance local dependency modeling and employs normalized gated linear attention to improve generalization across varying context lengths. Our comprehensive evaluations demonstrate that, distilling Mistral-7B with only 1K-length sequences yields over 90% passkey retrieval accuracy up to 22K tokens, significantly extending its effective context window. Similarly, Llama3.2-1B LAWCAT variant achieves competitive performance on S-NIAH 1&2&3 tasks (1K-8K context length) and BABILong benchmark (QA2&QA3, 0K-16K context length), requiring less than 0.1% pre-training tokens compared with pre-training models. Furthermore, LAWCAT exhibits faster prefill speeds than FlashAttention-2 for sequences exceeding 8K tokens. LAWCAT thus provides an efficient pathway to high-performance, long-context linear models suitable for edge deployment, reducing reliance on extensive long-sequence training data and computational resources.

Method: Uses compositional phoneme approximation (CPA), a feature-based representation technique that approximates L2 sounds with sequences of L1 phonemes.

Result: CPA-based training achieves 76% in-box formant rate, over 20% relative improvement in phoneme recognition accuracy, and over 80% of speech rated as more native-like with minimal training.

Conclusion: L1-grounded pronunciation training using CPA is effective for improving L2 pronunciation with minimal training effort.

Abstract: Learners of a second language (L2) often map non-native phonemes with similar native-language (L1) phonemes, making conventional L2-focused training slow and effortful. To address this, we propose an L1-grounded pronunciation training method based on compositional phoneme approximation (CPA), a feature-based representation technique that approximates L2 sounds with sequences of L1 phonemes. Evaluations with 20 Korean non-native English speakers show that CPA-based training achieves a 76% in-box formant rate in acoustic analysis, over 20% relative improvement in phoneme recognition accuracy, and over 80% of speech being rated as more native-like, with minimal training.

Method: Large-scale controlled evaluation across multiple axes: LLM-graph interaction modes (prompting, tool-use, code generation), dataset domains (citation, web-link, e-commerce, social networks), structural regimes (homophilic vs heterophilic graphs), feature characteristics (short vs long text), and model configurations. Also analyzed dependencies by truncating features, deleting edges, and removing labels.

Result: 1) Code generation achieves strongest overall performance, especially on long-text/high-degree graphs where prompting exceeds token limits. 2) All interaction strategies remain effective on heterophilic graphs. 3) Code generation adapts reliance between structure, features, and labels to leverage most informative input type.

Conclusion: The findings provide comprehensive understanding of LLM-graph interaction strengths/limitations and highlight key design principles for future approaches, showing code generation as the most flexible and effective method.

Abstract: Large language models (LLMs) are increasingly used for text-rich graph machine learning tasks such as node classification in high-impact domains like fraud detection and recommendation systems. Yet, despite a surge of interest, the field lacks a principled understanding of the capabilities of LLMs in their interaction with graph data. In this work, we conduct a large-scale, controlled evaluation across several key axes of variability to systematically assess the strengths and weaknesses of LLM-based graph reasoning methods in text-based applications. The axes include the LLM-graph interaction mode, comparing prompting, tool-use, and code generation; dataset domains, spanning citation, web-link, e-commerce, and social networks; structural regimes contrasting homophilic and heterophilic graphs; feature characteristics involving both short- and long-text node attributes; and model configurations with varying LLM sizes and reasoning capabilities. We further analyze dependencies by methodically truncating features, deleting edges, and removing labels to quantify reliance on input types. Our findings provide practical and actionable guidance. (1) LLMs as code generators achieve the strongest overall performance on graph data, with especially large gains on long-text or high-degree graphs where prompting quickly exceeds the token budget. (2) All interaction strategies remain effective on heterophilic graphs, challenging the assumption that LLM-based methods collapse under low homophily. (3) Code generation is able to flexibly adapt its reliance between structure, features, or labels to leverage the most informative input type. Together, these findings provide a comprehensive view of the strengths and limitations of current LLM-graph interaction modes and highlight key design principles for future approaches.

Method: Using auto-regressive LLMs trained on text tokens, the architecture processes image patches and audio waveforms directly as input to produce embeddings or classification labels, leveraging the model’s internal representations.

Result: Successfully applied text LLM weights to audio classification (FSD-50K, GTZAN) and image classification (CIFAR-10, Fashion-MNIST, image patches) without multimodal fine-tuning.

Conclusion: Text LLMs learn powerful internal circuits that can be activated for various applications, suggesting a more efficient approach than training specialized models from scratch for each modality.

Abstract: This paper presents a fascinating find: By training an auto-regressive LLM model on text tokens, the text model inherently develops internally an ability to understand images and audio, thereby developing the ability to see and hear just by reading. Popular audio and visual LLM models fine-tune text LLM models to give text output conditioned on images and audio embeddings. On the other hand, our architecture takes in patches of images, audio waveforms or tokens as input. It gives us the embeddings or category labels typical of a classification pipeline. We show the generality of text weights in aiding audio classification for datasets FSD-50K and GTZAN. Further, we show this working for image classification on CIFAR-10 and Fashion-MNIST, as well on image patches. This pushes the notion of text-LLMs learning powerful internal circuits that can be utilized by activating necessary connections for various applications rather than training models from scratch every single time.

Method: Uses ByT5 transformer model with rule-based grapheme-to-beat transformation for rhythm extraction, conditional denoising objective for fine-tuning, curriculum learning (pre-training on general Arabic data then poetic data), and explores cross-lingual transfer from English to Arabic.

Result: Experimental results show high rhythmic alignment while maintaining semantic coherence in the generated poetic phrases.

Conclusion: The proposed model successfully achieves rhythmic alignment in Arabic poetry generation and has potential for co-creative applications in classical Arabic poem composition.

Abstract: This paper presents a methodology for inserting phrases in Arabic poems to conform to a specific rhythm using ByT5, a byte-level multilingual transformer-based model. Our work discusses a rule-based grapheme-to-beat transformation tailored for extracting the rhythm from fully diacritized Arabic script. Our approach employs a conditional denoising objective to fine-tune ByT5, where the model reconstructs masked words to match a target rhythm. We adopt a curriculum learning strategy, pre-training on a general Arabic dataset before fine-tuning on poetic dataset, and explore cross-lingual transfer from English to Arabic. Experimental results demonstrate that our models achieve high rhythmic alignment while maintaining semantic coherence. The proposed model has the potential to be used in co-creative applications in the process of composing classical Arabic poems.

Method: Encodes sentence embeddings from pre-trained language models and models their relationships via attention. Uses contrastive learning to mitigate embedding biases and incorporates a causal graph with counterfactual methods to isolate structural features from topic-related biases.

Result: Experiments on two curated datasets (abstract comparisons and revised life FAQs) validate the effectiveness of the method in detecting both original and paraphrased AI-generated texts.

Conclusion: The proposed framework successfully detects AI-generated text by focusing on invariant structural features that remain consistent under word-level changes, providing a robust solution to current detection limitations.

Abstract: The widespread adoption of ChatGPT has raised concerns about its misuse, highlighting the need for robust detection of AI-generated text. Current word-level detectors are vulnerable to paraphrasing or simple prompts (PSP), suffer from biases induced by ChatGPT’s word-level patterns (CWP) and training data content, degrade on modified text, and often require large models or online LLM interaction. To tackle these issues, we introduce a novel task to detect both original and PSP-modified AI-generated texts, and propose a lightweight framework that classifies texts based on their internal structure, which remains invariant under word-level changes. Our approach encodes sentence embeddings from pre-trained language models and models their relationships via attention. We employ contrastive learning to mitigate embedding biases from autoregressive generation and incorporate a causal graph with counterfactual methods to isolate structural features from topic-related biases. Experiments on two curated datasets, including abstract comparisons and revised life FAQs, validate the effectiveness of our method.

Method: CCQA generates a question from each reasoning path and answer, evaluates them by similarity to the original question, and selects the candidate with highest similarity. It uses a lightweight Flan-T5 model for question generation since SLMs struggle with this task.

Result: CCQA consistently outperforms existing SOTA methods across eight models on mathematical and commonsense reasoning benchmarks, establishing a new practical baseline for efficient reasoning in SLMs.

Conclusion: The proposed CCQA method effectively addresses the limitations of conventional reasoning approaches for small language models and demonstrates superior performance across multiple reasoning benchmarks.

Abstract: Recently, inference-time reasoning strategies have further improved the accuracy of large language models (LLMs), but their effectiveness on smaller models remains unclear. Based on the observation that conventional approaches often fail to improve performance in this context, we propose \textbf{C}ycle-\textbf{C}onsistency in \textbf{Q}uestion \textbf{A}nswering (CCQA), a novel reasoning method that can be effectively applied to SLMs. Inspired by cycle consistency, CCQA generates a question from each reasoning path and answer, evaluates each by its similarity to the original question, and then selects the candidate solution with the highest similarity score as the final response. Since conventional SLMs struggle to generate accurate questions from their own reasoning paths and answers, we employ a lightweight Flan-T5 model specialized for question generation to support this process efficiently. From the experimental results, it is verified that CCQA consistently outperforms existing state-of-the-art (SOTA) methods across eight models on mathematical and commonsense reasoning benchmarks. Furthermore, our method establishes a new practical baseline for efficient reasoning in SLMs. Source code can be found at https://github.com/scai-research/ccqa_official.

Method: Estimates token priors using corpus-level term frequency statistics, filters documents based on mean and standard deviation of token priors without requiring model inference.

Result: Achieves highest average performance across 20 downstream benchmarks while reducing time cost by over 1000x compared to PPL-based filtering.

Conclusion: The prior-based filter is a simple yet powerful alternative that works effectively across different domains (code, math) and adapts dynamically to multilingual corpora without supervision.

Abstract: As large language models (LLMs) are pretrained on massive web corpora, careful selection of data becomes essential to ensure effective and efficient learning. While perplexity (PPL)-based filtering has shown strong performance, it suffers from drawbacks: substantial time costs and inherent unreliability of the model when handling noisy or out-of-distribution samples. In this work, we propose a simple yet powerful alternative: a prior-based data filtering method that estimates token priors using corpus-level term frequency statistics, inspired by linguistic insights on word roles and lexical density. Our approach filters documents based on the mean and standard deviation of token priors, serving as a fast proxy to PPL while requiring no model inference. Despite its simplicity, the prior-based filter achieves the highest average performance across 20 downstream benchmarks, while reducing time cost by over 1000x compared to PPL-based filtering. We further demonstrate its applicability to symbolic languages such as code and math, and its dynamic adaptability to multilingual corpora without supervision

Method: TsqLoRA integrates two components: quality-aware sampling for selecting informative training data, and dynamic rank allocation that adjusts each layer’s rank based on sensitivity to parameter updates.

Result: Experimental results show TsqLoRA improves fine-tuning efficiency while maintaining or improving performance on various NLP tasks.

Conclusion: TsqLoRA provides an effective solution for resource-efficient fine-tuning of large language models by addressing both data quality and layer sensitivity considerations.

Abstract: Fine-tuning large pre-trained models for downstream tasks has become a fundamental approach in natural language processing. Fully fine-tuning all model parameters is computationally expensive and memory-intensive, especially in resource-constrained environments. Existing parameter-efficient fine-tuning methods reduce the number of trainable parameters but typically overlook the varying sensitivity of different model layers and the importance of training data. In this work, we propose TsqLoRA, a novel method that integrates data-quality-driven selection with sensitivity-aware low-rank adaptation, consisted of two main components: a quality-aware sampling mechanism for selecting the most informative training data, and a dynamic rank allocation module that adjusts the rank of each layer based on its sensitivity to parameter updates. The experimental results demonstrate that TsqLoRA improves fine-tuning efficiency while maintaining or even improving performance on a variety of NLP tasks. Our code will be available at https://github.com/Benjamin-Ricky/TsqLoRA.

Method: A decoupled two-stage training approach: first learns evidence-based ECG interpretation (ECG-to-Text), then injects ECG generation capabilities (Text-to-ECG) through latent space alignment.

Result: UniECG can autonomously choose to interpret or generate ECG based on user input, significantly extending the capability boundaries of current ECG models.

Conclusion: The proposed UniECG model successfully addresses the limitations of existing unified models in ECG understanding and generation, representing a significant advancement in medical AI capabilities for ECG analysis.

Abstract: Recent unified models such as GPT-5 have achieved encouraging progress on vision-language tasks. However, these unified models typically fail to correctly understand ECG signals and provide accurate medical diagnoses, nor can they correctly generate ECG signals. To address these limitations, we propose UniECG, the first unified model for ECG capable of concurrently performing evidence-based ECG interpretation and text-conditioned ECG generation tasks. Through a decoupled two-stage training approach, the model first learns evidence-based interpretation skills (ECG-to-Text), and then injects ECG generation capabilities (Text-to-ECG) via latent space alignment. UniECG can autonomously choose to interpret or generate an ECG based on user input, significantly extending the capability boundaries of current ECG models. Our code and checkpoints will be made publicly available at https://github.com/PKUDigitalHealth/UniECG upon acceptance.

Method: Created Planorama interface where 126 users answered 300 multi-step questions with LLM plans, collecting 4388 plan executions and 5584 comparisons to measure plan helpfulness (QA success) and user preferences.

Result: 1) User/model preferences and agent success do not predict which plans actually help users; 2) Gap is not due to user-specific preferences; 3) Surface-level cues like brevity and question similarity strongly influence preferences but fail to predict helpfulness.

Conclusion: Aligning helpful LLMs requires feedback from real user interactions, not just preferences of what looks helpful. The paper discusses actionable steps for NLP researchers to address this problem.

Abstract: To assist users in complex tasks, LLMs generate plans: step-by-step instructions towards a goal. While alignment methods aim to ensure LLM plans are helpful, they train (RLHF) or evaluate (ChatbotArena) on what users prefer, assuming this reflects what helps them. We test this with Planorama: an interface where 126 users answer 300 multi-step questions with LLM plans. We get 4388 plan executions and 5584 comparisons to measure plan helpfulness (QA success) and user preferences on plans, and recreate the setup in agents and reward models to see if they simulate or prefer what helps users. We expose: 1) user/model preferences and agent success do not accurately predict which plans help users, so common alignment feedback can misalign with helpfulness; 2) this gap is not due to user-specific preferences, as users are similarly successful when using plans they prefer/disprefer; 3) surface-level cues like brevity and question similarity strongly link to preferences, but such biases fail to predict helpfulness. In all, we argue aligning helpful LLMs needs feedback from real user interactions, not just preferences of what looks helpful, so we discuss the plan NLP researchers can execute to solve this problem.

Method: CAPE-KG framework ensures KG construction, update, and retrieval are always aligned with MHQA task requirements, maintaining coherent reasoning over both unedited and edited knowledge through consistency-aware parameter-preserving editing.

Result: Extensive experiments on MQuAKE benchmark show accuracy improvements in PPKE performance for MHQA, demonstrating the effectiveness of addressing consistency in PPKE.

Conclusion: CAPE-KG successfully addresses consistency issues in PPKE for multi-hop QA, improving reliability and performance through alignment of KG operations with MHQA requirements.

Abstract: Parameter-Preserving Knowledge Editing (PPKE) enables updating models with new or corrected information without retraining or parameter adjustment. Recent PPKE approaches based on knowledge graphs (KG) to extend knowledge editing (KE) capabilities to multi-hop question answering (MHQA). However, these methods often lack consistency, leading to knowledge contamination, unstable updates, and retrieval behaviors that fail to reflect the intended edits. Such inconsistencies undermine the reliability of PPKE in multi- hop reasoning. We present CAPE-KG, Consistency-Aware Parameter-Preserving Editing with Knowledge Graphs, a novel consistency-aware framework for PPKE on MHQA. CAPE-KG ensures KG construction, update, and retrieval are always aligned with the requirements of the MHQA task, maintaining coherent reasoning over both unedited and edited knowledge. Extensive experiments on the MQuAKE benchmark show accuracy improvements in PPKE performance for MHQA, demonstrating the effectiveness of addressing consistency in PPKE.

Method: The framework uses conformal prediction to construct continuous prediction intervals from single evaluation runs, with ordinal boundary adjustment for discrete rating tasks. It also proposes a midpoint-based score as a low-bias alternative.

Result: Extensive experiments show that conformal prediction provides valid prediction intervals with coverage guarantees. The interval midpoint and judge reprompting are explored for improved judgment quality.

Conclusion: The proposed framework successfully addresses uncertainty in LLM-as-a-judge evaluations, offering reliable prediction intervals and improved scoring methods for more trustworthy NLG assessment.

Abstract: LLM-as-a-judge has become a promising paradigm for using large language models (LLMs) to evaluate natural language generation (NLG), but the uncertainty of its evaluation remains underexplored. This lack of reliability may limit its deployment in many applications. This work presents the first framework to analyze the uncertainty by offering a prediction interval of LLM-based scoring via conformal prediction. Conformal prediction constructs continuous prediction intervals from a single evaluation run, and we design an ordinal boundary adjustment for discrete rating tasks. We also suggest a midpoint-based score within the interval as a low-bias alternative to raw model score and weighted average. We perform extensive experiments and analysis, which show that conformal prediction can provide valid prediction interval with coverage guarantees. We also explore the usefulness of interval midpoint and judge reprompting for better judgment.

Method: Proposes MemOrb - a lightweight, plug-and-play verbal reinforcement memory layer that distills multi-turn interactions into compact strategy reflections stored in a shared memory bank, retrieved to guide decision-making without requiring fine-tuning.

Result: MemOrb significantly improves both success rate and stability, achieving up to 63 percentage-point gain in multi-turn success rate and delivering more consistent performance across repeated trials.

Conclusion: Structured reflection is a powerful mechanism for enhancing long-term reliability of frozen LLM agents in customer service scenarios.

Abstract: Large Language Model-based agents(LLM-based agents) are increasingly deployed in customer service, yet they often forget across sessions, repeat errors, and lack mechanisms for continual self-improvement. This makes them unreliable in dynamic settings where stability and consistency are critical. To better evaluate these properties, we emphasize two indicators: task success rate as a measure of overall effectiveness, and consistency metrics such as Pass$^k$ to capture reliability across multiple trials. To address the limitations of existing approaches, we propose MemOrb, a lightweight and plug-and-play verbal reinforcement memory layer that distills multi-turn interactions into compact strategy reflections. These reflections are stored in a shared memory bank and retrieved to guide decision-making, without requiring any fine-tuning. Experiments show that MemOrb significantly improves both success rate and stability, achieving up to a 63 percentage-point gain in multi-turn success rate and delivering more consistent performance across repeated trials. Our results demonstrate that structured reflection is a powerful mechanism for enhancing long-term reliability of frozen LLM agents in customer service scenarios.

Method: DyGRASP uses node-centric implicit reasoning with sliding windows for recent semantics, explicit reasoning with tailored prompts and RNN-like chains for global semantics, and integrates both with dynamic graph structure through updating and merging layers.

Result: Experiments show DyGRASP achieves up to 34% improvement in Hit@10 for destination node retrieval and exhibits strong generalization across different temporal GNNs and LLMs.

Conclusion: DyGRASP effectively addresses the challenges of DyTAGs by efficiently capturing both recent and global temporal semantics while maintaining strong performance and generalization capabilities.

Abstract: Dynamic Text-Attribute Graphs (DyTAGs), characterized by time-evolving graph interactions and associated text attributes, are prevalent in real-world applications. Existing methods, such as Graph Neural Networks (GNNs) and Large Language Models (LLMs), mostly focus on static TAGs. Extending these existing methods to DyTAGs is challenging as they largely neglect the recent-global temporal semantics: the recent semantic dependencies among interaction texts and the global semantic evolution of nodes over time. Furthermore, applying LLMs to the abundant and evolving text in DyTAGs faces efficiency issues. To tackle these challenges, we propose Dynamic Global-Recent Adaptive Semantic Processing (DyGRASP), a novel method that leverages LLMs and temporal GNNs to efficiently and effectively reason on DyTAGs. Specifically, we first design a node-centric implicit reasoning method together with a sliding window mechanism to efficiently capture recent temporal semantics. In addition, to capture global semantic dynamics of nodes, we leverage explicit reasoning with tailored prompts and an RNN-like chain structure to infer long-term semantics. Lastly, we intricately integrate the recent and global temporal semantics as well as the dynamic graph structural information using updating and merging layers. Extensive experiments on DyTAG benchmarks demonstrate DyGRASP’s superiority, achieving up to 34% improvement in Hit@10 for destination node retrieval task. Besides, DyGRASP exhibits strong generalization across different temporal GNNs and LLMs.

Method: Controlled experiments training bilingual autoregressive models on multiple language pairs with systematically varied vocabulary overlap settings, while analyzing semantic similarity of shared tokens and hidden representations.

Result: Models with overlapping vocabularies outperform disjoint vocabulary models on XNLI and XQuAD tasks, with transfer performance improving as overlap increases. Overlap creates embedding spaces that capture cross-lingual semantic relationships.

Conclusion: Substantial shared vocabulary remains a beneficial design choice for multilingual tokenizers, as token overlap facilitates rather than interferes with cross-lingual transfer.

Abstract: Subword tokenizers trained on multilingual corpora naturally produce overlapping tokens across languages. Does token overlap facilitate cross-lingual transfer or instead introduce interference between languages? Prior work offers mixed evidence, partly due to varied setups and confounders, such as token frequency or subword segmentation granularity. To address this question, we devise a controlled experiment where we train bilingual autoregressive models on multiple language pairs under systematically varied vocabulary overlap settings. Crucially, we explore a new dimension to understanding how overlap affects transfer: the semantic similarity of tokens shared across languages. We first analyze our models’ hidden representations and find that overlap of any kind creates embedding spaces that capture cross-lingual semantic relationships, while this effect is much weaker in models with disjoint vocabularies. On XNLI and XQuAD, we find that models with overlap outperform models with disjoint vocabularies, and that transfer performance generally improves as overlap increases. Overall, our findings highlight the advantages of token overlap in multilingual models and show that substantial shared vocabulary remains a beneficial design choice for multilingual tokenizers.

Method: Systematic investigation of SFT data length effects by decoupling and analyzing Multi-Head Attention (MHA) and Feed-Forward Network (FFN) components, studying their interaction, and testing hybrid training approaches.

Result: Long-context SFT unexpectedly improves short-context performance. Both MHA and FFN benefit independently, with long-context SFT promoting contextual knowledge and short-context SFT favoring parametric knowledge. Hybrid training mitigates the knowledge preference bias.

Conclusion: Hybrid training offers explainable guidance for fine-tuning LLMs by balancing contextual and parametric knowledge preferences, providing optimal performance across different context lengths.

Abstract: Large language models (LLMs) have achieved impressive performance across natural language processing (NLP) tasks. As real-world applications increasingly demand longer context windows, continued pretraining and supervised fine-tuning (SFT) on long-context data has become a common approach. While the effects of data length in continued pretraining have been extensively studied, their implications for SFT remain unclear. In this work, we systematically investigate how SFT data length influences LLM behavior on short-context tasks. Counterintuitively, we find that long-context SFT improves short-context performance, contrary to the commonly observed degradation from long-context pretraining. To uncover the underlying mechanisms of this phenomenon, we first decouple and analyze two key components, Multi-Head Attention (MHA) and Feed-Forward Network (FFN), and show that both independently benefit from long-context SFT. We further study their interaction and reveal a knowledge preference bias: long-context SFT promotes contextual knowledge, while short-context SFT favors parametric knowledge, making exclusive reliance on long-context SFT suboptimal. Finally, we demonstrate that hybrid training mitigates this bias, offering explainable guidance for fine-tuning LLMs.

Method: Uses unsupervised fine-tuning on Form 10-K filings with chronological and lexical patterns to develop domain-specific financial encoder and quantitative risk relation scores.

Result: Extensive experiments show the method outperforms strong baselines across multiple evaluation settings.

Conclusion: The approach enables systematic, scalable identification of inter-firm risk relations with transparency and interpretable analysis.

Abstract: A multitude of interconnected risk events – ranging from regulatory changes to geopolitical tensions – can trigger ripple effects across firms. Identifying inter-firm risk relations is thus crucial for applications like portfolio management and investment strategy. Traditionally, such assessments rely on expert judgment and manual analysis, which are, however, subjective, labor-intensive, and difficult to scale. To address this, we propose a systematic method for extracting inter-firm risk relations using Form 10-K filings – authoritative, standardized financial documents – as our data source. Leveraging recent advances in natural language processing, our approach captures implicit and abstract risk connections through unsupervised fine-tuning based on chronological and lexical patterns in the filings. This enables the development of a domain-specific financial encoder with a deeper contextual understanding and introduces a quantitative risk relation score for transparency, interpretable analysis. Extensive experiments demonstrate that our method outperforms strong baselines across multiple evaluation settings.

Method: Established AECBench with 23 tasks across 5 cognitive levels (Knowledge Memorization, Understanding, Reasoning, Calculation, Application), creating a 4,800-question dataset validated by experts, and using LLM-as-a-Judge approach for scalable evaluation.

Result: Evaluation of 9 LLMs showed clear performance decline across cognitive levels - proficient in basic tasks but significant deficits in interpreting table knowledge, complex reasoning/calculation, and domain-specific document generation.

Conclusion: The study provides groundwork for future development of robust LLM integration into safety-critical engineering practices, highlighting current limitations in specialized AEC applications.

Abstract: Large language models (LLMs), as a novel information technology, are seeing increasing adoption in the Architecture, Engineering, and Construction (AEC) field. They have shown their potential to streamline processes throughout the building lifecycle. However, the robustness and reliability of LLMs in such a specialized and safety-critical domain remain to be evaluated. To address this challenge, this paper establishes AECBench, a comprehensive benchmark designed to quantify the strengths and limitations of current LLMs in the AEC domain. The benchmark defines 23 representative tasks within a five-level cognition-oriented evaluation framework encompassing Knowledge Memorization, Understanding, Reasoning, Calculation, and Application. These tasks were derived from authentic AEC practice, with scope ranging from codes retrieval to specialized documents generation. Subsequently, a 4,800-question dataset encompassing diverse formats, including open-ended questions, was crafted primarily by engineers and validated through a two-round expert review. Furthermore, an LLM-as-a-Judge approach was introduced to provide a scalable and consistent methodology for evaluating complex, long-form responses leveraging expert-derived rubrics. Through the evaluation of nine LLMs, a clear performance decline across five cognitive levels was revealed. Despite demonstrating proficiency in foundational tasks at the Knowledge Memorization and Understanding levels, the models showed significant performance deficits, particularly in interpreting knowledge from tables in building codes, executing complex reasoning and calculation, and generating domain-specific documents. Consequently, this study lays the groundwork for future research and development aimed at the robust and reliable integration of LLMs into safety-critical engineering practices.

Method: Uses model diffing with crosscoders to identify and categorize latent representations that differentiate Gemma-2-9b-it and its SimPO-enhanced variant.

Result: SimPO acquired latent concepts predominantly enhance safety mechanisms (+32.8%), multilingual capabilities (+43.8%), and instruction-following (+151.7%), while reducing emphasis on model self-reference (-44.1%) and hallucination management (-68.5%).

Conclusion: Model diffing provides fine-grained insights beyond leaderboard metrics, offering a transparent framework for comparing LLMs by attributing performance gaps to concrete mechanistic capabilities.

Abstract: As fine-tuning becomes the dominant paradigm for improving large language models (LLMs), understanding what changes during this process is increasingly important. Traditional benchmarking often fails to explain why one model outperforms another. In this work, we use model diffing, a mechanistic interpretability approach, to analyze the specific capability differences between Gemma-2-9b-it and a SimPO-enhanced variant. Using crosscoders, we identify and categorize latent representations that differentiate the two models. We find that SimPO acquired latent concepts predominantly enhance safety mechanisms (+32.8%), multilingual capabilities (+43.8%), and instruction-following (+151.7%), while its additional training also reduces emphasis on model self-reference (-44.1%) and hallucination management (-68.5%). Our analysis shows that model diffing can yield fine-grained insights beyond leaderboard metrics, attributing performance gaps to concrete mechanistic capabilities. This approach offers a transparent and targeted framework for comparing LLMs.

Method: MAPEX coordinates LLM-based agents through expert recruitment, candidate extraction, topic guidance, knowledge augmentation, and post-processing modules with dual-path strategy: knowledge-driven for short texts and topic-guided for long texts.

Result: Extensive experiments on six benchmark datasets across three LLMs show MAPEX outperforms state-of-the-art unsupervised method by 2.44% and standard LLM baselines by 4.01% in F1@5 on average.

Conclusion: MAPEX demonstrates strong generalization and universality, providing an effective multi-agent collaboration framework for keyphrase extraction that adapts to document complexity.

Abstract: Keyphrase extraction is a fundamental task in natural language processing. However, existing unsupervised prompt-based methods for Large Language Models (LLMs) often rely on single-stage inference pipelines with uniform prompting, regardless of document length or LLM backbone. Such one-size-fits-all designs hinder the full exploitation of LLMs’ reasoning and generation capabilities, especially given the complexity of keyphrase extraction across diverse scenarios. To address these challenges, we propose MAPEX, the first framework that introduces multi-agent collaboration into keyphrase extraction. MAPEX coordinates LLM-based agents through modules for expert recruitment, candidate extraction, topic guidance, knowledge augmentation, and post-processing. A dual-path strategy dynamically adapts to document length: knowledge-driven extraction for short texts and topic-guided extraction for long texts. Extensive experiments on six benchmark datasets across three different LLMs demonstrate its strong generalization and universality, outperforming the state-of-the-art unsupervised method by 2.44% and standard LLM baselines by 4.01% in F1@5 on average. Code is available at https://github.com/NKU-LITI/MAPEX.

Method: Compared open-weight models against top proprietary systems (GPT-4o, GPT-4.1, Claude 3.5/3.7 Sonnet) using techniques like embedding-based snippet retrieval, in-context learning, structured outputs, and ensemble approaches for exact-answer questions.

Result: Open-weight LLMs demonstrated comparable performance to proprietary models, with some open-weight models surpassing closed counterparts when ensemble strategies were applied.

Conclusion: Open-weight LLMs are viable alternatives to proprietary models in biomedical question-answering, especially when enhanced with appropriate techniques like ensembling.

Abstract: Open-weight versions of large language models (LLMs) are rapidly advancing, with state-of-the-art models like DeepSeek-V3 now performing comparably to proprietary LLMs. This progression raises the question of whether small open-weight LLMs are capable of effectively replacing larger closed-source models. We are particularly interested in the context of biomedical question-answering, a domain we explored by participating in Task 13B Phase B of the BioASQ challenge. In this work, we compare several open-weight models against top-performing systems such as GPT-4o, GPT-4.1, Claude 3.5 Sonnet, and Claude 3.7 Sonnet. To enhance question answering capabilities, we use various techniques including retrieving the most relevant snippets based on embedding distance, in-context learning, and structured outputs. For certain submissions, we utilize ensemble approaches to leverage the diverse outputs generated by different models for exact-answer questions. Our results demonstrate that open-weight LLMs are comparable to proprietary ones. In some instances, open-weight LLMs even surpassed their closed counterparts, particularly when ensembling strategies were applied. All code is publicly available at https://github.com/evidenceprime/BioASQ-13b.

Method: Implemented MHFD (Multi-Hierarchical Feature Detection) integrating DeBERTa-based semantic analysis, syntactic parsing, and statistical probability features through adaptive fusion.

Result: MHFD achieves 89.7% accuracy in in-domain detection and 84.2% in cross-domain detection, showing only modest improvements of 0.4-2.6% over existing methods despite the computational overhead.

Conclusion: Multi-feature integration provides minimal benefits with substantial computational costs, indicating that modern neural language models may already capture most relevant detection signals efficiently.

Abstract: With the rapid advancement of large language model technology, there is growing interest in whether multi-feature approaches can significantly improve AI text detection beyond what single neural models achieve. While intuition suggests that combining semantic, syntactic, and statistical features should provide complementary signals, this assumption has not been rigorously tested with modern LLM-generated text. This paper provides a systematic empirical investigation of multi-hierarchical feature integration for AI text detection, specifically testing whether the computational overhead of combining multiple feature types is justified by performance gains. We implement MHFD (Multi-Hierarchical Feature Detection), integrating DeBERTa-based semantic analysis, syntactic parsing, and statistical probability features through adaptive fusion. Our investigation reveals important negative results: despite theoretical expectations, multi-feature integration provides minimal benefits (0.4-0.5% improvement) while incurring substantial computational costs (4.2x overhead), suggesting that modern neural language models may already capture most relevant detection signals efficiently. Experimental results on multiple benchmark datasets demonstrate that the MHFD method achieves 89.7% accuracy in in-domain detection and maintains 84.2% stable performance in cross-domain detection, showing modest improvements of 0.4-2.6% over existing methods.

Method: Proposes DivEye framework that captures how unpredictability fluctuates across text using surprisal-based features, leveraging the observation that human-authored text exhibits richer variability in lexical and structural unpredictability than LLM outputs.

Result: Outperforms existing zero-shot detectors by up to 33.2%, achieves competitive performance with fine-tuned baselines, robust to paraphrasing and adversarial attacks, generalizes well across domains and models, and improves existing detectors by up to 18.7% when used as auxiliary signal.

Conclusion: DivEye provides a powerful and interpretable approach for AI-generated text detection, with rhythmic unpredictability identified as an underexplored but effective signal for distinguishing human from machine-generated content.

Abstract: Detecting AI-generated text is an increasing necessity to combat misuse of LLMs in education, business compliance, journalism, and social media, where synthetic fluency can mask misinformation or deception. While prior detectors often rely on token-level likelihoods or opaque black-box classifiers, these approaches struggle against high-quality generations and offer little interpretability. In this work, we propose DivEye, a novel detection framework that captures how unpredictability fluctuates across a text using surprisal-based features. Motivated by the observation that human-authored text exhibits richer variability in lexical and structural unpredictability than LLM outputs, DivEye captures this signal through a set of interpretable statistical features. Our method outperforms existing zero-shot detectors by up to 33.2% and achieves competitive performance with fine-tuned baselines across multiple benchmarks. DivEye is robust to paraphrasing and adversarial attacks, generalizes well across domains and models, and improves the performance of existing detectors by up to 18.7% when used as an auxiliary signal. Beyond detection, DivEye provides interpretable insights into why a text is flagged, pointing to rhythmic unpredictability as a powerful and underexplored signal for LLM detection.

Method: Proposed JEDI architecture that jointly performs extractive atomic fact decomposition and interpretable inference. Used a large corpus of synthetic rationales covering multiple NLI benchmarks for training.

Result: JEDI achieves competitive accuracy in distribution and significantly improves robustness out of distribution and in adversarial settings over models based solely on extractive rationale supervision.

Conclusion: Interpretability and robust generalization in NLI can be realized using encoder-only architectures and synthetic rationales, providing a more efficient alternative to generative LLMs.

Abstract: Recent works in Natural Language Inference (NLI) and related tasks, such as automated fact-checking, employ atomic fact decomposition to enhance interpretability and robustness. For this, existing methods rely on resource-intensive generative large language models (LLMs) to perform decomposition. We propose JEDI, an encoder-only architecture that jointly performs extractive atomic fact decomposition and interpretable inference without requiring generative models during inference. To facilitate training, we produce a large corpus of synthetic rationales covering multiple NLI benchmarks. Experimental results demonstrate that JEDI achieves competitive accuracy in distribution and significantly improves robustness out of distribution and in adversarial settings over models based solely on extractive rationale supervision. Our findings show that interpretability and robust generalization in NLI can be realized using encoder-only architectures and synthetic rationales. Code and data available at https://jedi.nicpopovic.com

Method: Adapting Dynamic Time Warping (DTW) for aligning speech and text embeddings during training to bridge the modality gap in end-to-end speech translation.

Result: The method produces more accurate alignments, achieves comparable E2E-ST results while being significantly faster, and outperforms previous work in low resource settings on 5 out of 6 language directions.

Conclusion: DTW-based alignment is an effective approach for bridging the modality gap in E2E-ST, offering improved accuracy and efficiency over existing methods.

Abstract: End-to-End Speech Translation (E2E-ST) is the task of translating source speech directly into target text bypassing the intermediate transcription step. The representation discrepancy between the speech and text modalities has motivated research on what is known as bridging the modality gap. State-of-the-art methods addressed this by aligning speech and text representations on the word or token level. Unfortunately, this requires an alignment tool that is not available for all languages. Although this issue has been addressed by aligning speech and text embeddings using nearest-neighbor similarity search, it does not lead to accurate alignments. In this work, we adapt Dynamic Time Warping (DTW) for aligning speech and text embeddings during training. Our experiments demonstrate the effectiveness of our method in bridging the modality gap in E2E-ST. Compared to previous work, our method produces more accurate alignments and achieves comparable E2E-ST results while being significantly faster. Furthermore, our method outperforms previous work in low resource settings on 5 out of 6 language directions.

Method: Systematic study of best-of-N TTS framework on WMT24 benchmarks across six high-resource and one low-resource language pairs, five model sizes (3B-72B), and various compute budgets (N up to 1024).

Result: TTS improves translation quality for high-resource languages; smaller models with large N can match larger models; larger models are more efficient under fixed compute budgets; TTS can degrade quality in low-resource cases due to metric blind spots.

Conclusion: TTS is effective for high-resource machine translation but has limitations in low-resource scenarios, with larger models generally being more compute-efficient.

Abstract: Scaling model parameters has become the de facto strategy for improving NLP systems, but it comes with substantial computational costs. Test-Time Scaling (TTS) offers an alternative by allocating more computation at inference: generating multiple candidates and selecting the best. While effective in tasks such as mathematical reasoning, TTS has not been systematically explored for machine translation (MT). In this paper, we present the first systematic study of TTS for MT, investigating a simple but practical best-of-N framework on WMT24 benchmarks. Our experiments cover six high-resource and one low-resource language pairs, five model sizes (3B-72B), and various TTS compute budget (N up to 1024). Our results show that a) For high-resource languages, TTS generally improves translation quality according to multiple neural MT evaluation metrics, and our human evaluation confirms these gains; b) Augmenting smaller models with large $N$ can match or surpass larger models at $N{=}1$ with more compute cost; c) Under fixed compute budgets, larger models are typically more efficient, and TTS can degrade quality due to metric blind spots in low-resource cases.

Method: Compiled proceedings from 10 editions of CLiC-it conference into a corpus, analyzed metadata (author provenance, gender, affiliations) and paper content to track research trends and topic evolution.

Result: Identified significant shift in research priorities from Lexical and Semantic Resources towards Language Modelling and Multimodality, reflecting broader field transformations driven by LLM advancements.

Conclusion: The study provides valuable longitudinal insights into Italian CL/NLP research evolution, supporting community awareness and strategic planning for future research directions in the rapidly changing field.

Abstract: Over the past decade, Computational Linguistics (CL) and Natural Language Processing (NLP) have evolved rapidly, especially with the advent of Transformer-based Large Language Models (LLMs). This shift has transformed research goals and priorities, from Lexical and Semantic Resources to Language Modelling and Multimodality. In this study, we track the research trends of the Italian CL and NLP community through an analysis of the contributions to CLiC-it, arguably the leading Italian conference in the field. We compile the proceedings from the first 10 editions of the CLiC-it conference (from 2014 to 2024) into the CLiC-it Corpus, providing a comprehensive analysis of both its metadata, including author provenance, gender, affiliations, and more, as well as the content of the papers themselves, which address various topics. Our goal is to provide the Italian and international research communities with valuable insights into emerging trends and key developments over time, supporting informed decisions and future directions in the field.

Method: PoT models LLM reasoning as an iterative decision process where the model dynamically selects among cognitive operations (reasoning, revision, personalization, clarification) to explore multiple reasoning trajectories, then aggregates and reweights candidate responses based on inferred user preferences.

Result: Experiments on LaMP-QA benchmark show PoT consistently outperforms baselines with up to 13.1% relative improvement. Human evaluation shows annotators prefer PoT outputs in 66% of cases with only 15% ties.

Conclusion: PoT effectively addresses personalized QA challenges by enabling diverse reasoning path exploration and preference-based aggregation, demonstrating significant improvements over existing methods without requiring task-specific fine-tuning.

Abstract: Personalization is essential for adapting question answering (QA) systems to user-specific information needs, thereby improving both accuracy and user satisfaction. However, personalized QA remains relatively underexplored due to challenges such as inferring preferences from long, noisy, and implicit contexts, and generating responses that are simultaneously correct, contextually appropriate, and aligned with user expectations and background knowledge. To address these challenges, we propose Pathways of Thoughts (PoT), an inference-stage method that applies to any large language model (LLM) without requiring task-specific fine-tuning. The approach models the reasoning of an LLM as an iterative decision process, where the model dynamically selects among cognitive operations such as reasoning, revision, personalization, and clarification. This enables exploration of multiple reasoning trajectories, producing diverse candidate responses that capture different perspectives. PoT then aggregates and reweights these candidates according to inferred user preferences, yielding a final personalized response that benefits from the complementary strengths of diverse reasoning paths. Experiments on the LaMP-QA benchmark for personalized QA show that PoT consistently outperforms competitive baselines, achieving up to a 13.1% relative improvement. Human evaluation corroborates these results, with annotators preferring outputs from PoT in 66% of cases and reporting ties in only 15% of cases.

Method: Used the NLTK Python library to parse corpora of different genres and count exact string matches of sentences within each corpus.

Result: Found that completely unique sentences are often the majority in corpora, but this is highly constrained by genre, and duplicate sentences constitute a non-trivial portion of any corpus.

Conclusion: The claim that most sentences are unique holds true in general but is genre-dependent, and duplicate sentences play a more significant role than previously acknowledged in linguistic theory.

Abstract: A repeated claim in linguistics is that the majority of linguistic utterances are unique. For example, Pinker (1994: 10), summarizing an argument by Noam Chomsky, states that “virtually every sentence that a person utters or understands is a brand-new combination of words, appearing for the first time in the history of the universe.” With the increased availability of large corpora, this is a claim that can be empirically investigated. The current paper addresses the question by using the NLTK Python library to parse corpora of different genres, providing counts of exact string matches in each. Results show that while completely unique sentences are often the majority of corpora, this is highly constrained by genre, and that duplicate sentences are not an insignificant part of any individual corpus.

Method: Created a dataset of 1,499 news articles containing 10,900 sentences with manual annotation of 39,075 entity mentions across 27 classes. Evaluated traditional sequence labeling (CRF) and multilingual transformer models fine-tuned on the dataset.

Result: Multilingual RoBERTa achieved the best performance with promising precision-recall balance. All models struggled with rare entity categories. Other multilingual models yielded comparable results to RoBERTa.

Conclusion: Multilingual pretrained models show potential for low-resource languages like Kyrgyz, though challenges remain with rare entities. Future work should explore more granular annotation schemes for better Kyrgyz language processing evaluation.

Abstract: We introduce KyrgyzNER, the first manually annotated named entity recognition dataset for the Kyrgyz language. Comprising 1,499 news articles from the 24.KG news portal, the dataset contains 10,900 sentences and 39,075 entity mentions across 27 named entity classes. We show our annotation scheme, discuss the challenges encountered in the annotation process, and present the descriptive statistics. We also evaluate several named entity recognition models, including traditional sequence labeling approaches based on conditional random fields and state-of-the-art multilingual transformer-based models fine-tuned on our dataset. While all models show difficulties with rare entity categories, models such as the multilingual RoBERTa variant pretrained on a large corpus across many languages achieve a promising balance between precision and recall. These findings emphasize both the challenges and opportunities of using multilingual pretrained models for processing languages with limited resources. Although the multilingual RoBERTa model performed best, other multilingual models yielded comparable results. This suggests that future work exploring more granular annotation schemes may offer deeper insights for Kyrgyz language processing pipelines evaluation.

Method: LLM-guided multi-aspect encoding with dynamic clustering: LLMs identify key aspects (methodology, dataset, evaluation) and generate aspect-specific paper summaries, which are then encoded and clustered along each aspect to form coherent hierarchies.

Result: Significantly outperforms prior approaches, achieving state-of-the-art performance in taxonomy coherence, granularity, and interpretability. Introduced a new evaluation benchmark of 156 expert-crafted taxonomies covering 11.6k papers.

Conclusion: The proposed framework effectively addresses limitations of existing methods and provides a superior solution for organizing scientific literature through context-aware hierarchical taxonomy generation.

Abstract: The rapid growth of scientific literature demands efficient methods to organize and synthesize research findings. Existing taxonomy construction methods, leveraging unsupervised clustering or direct prompting of large language models (LLMs), often lack coherence and granularity. We propose a novel context-aware hierarchical taxonomy generation framework that integrates LLM-guided multi-aspect encoding with dynamic clustering. Our method leverages LLMs to identify key aspects of each paper (e.g., methodology, dataset, evaluation) and generates aspect-specific paper summaries, which are then encoded and clustered along each aspect to form a coherent hierarchy. In addition, we introduce a new evaluation benchmark of 156 expert-crafted taxonomies encompassing 11.6k papers, providing the first naturally annotated dataset for this task. Experimental results demonstrate that our method significantly outperforms prior approaches, achieving state-of-the-art performance in taxonomy coherence, granularity, and interpretability.

Method: Collect misinformation claims from fact-checking websites in English, Spanish, and Hindi from US and India; cluster claims into narratives; characterize clusters with knowledge graphs; augment attacker LLM with these graphs to generate adversarial prompts.

Result: The method produces higher attack success rates compared to few-shot prompting and offers interpretability benefits.

Conclusion: Results highlight the need for globally scalable disinformation mitigations grounded in real-world adversarial misuse scenarios.

Abstract: Disinformation is among the top risks of generative artificial intelligence (AI) misuse. Global adoption of generative AI necessitates red-teaming evaluations (i.e., systematic adversarial probing) that are robust across diverse languages and cultures, but red-teaming datasets are commonly US- and English-centric. To address this gap, we propose “anecdoctoring”, a novel red-teaming approach that automatically generates adversarial prompts across languages and cultures. We collect misinformation claims from fact-checking websites in three languages (English, Spanish, and Hindi) and two geographies (US and India). We then cluster individual claims into broader narratives and characterize the resulting clusters with knowledge graphs, with which we augment an attacker LLM. Our method produces higher attack success rates and offers interpretability benefits relative to few-shot prompting. Results underscore the need for disinformation mitigations that scale globally and are grounded in real-world adversarial misuse.

Method: Developed taxonomy through interviews with NLP, writing, and philosophy experts; conducted span-level annotation to assess binary ‘slop’ judgments and correlate them with latent dimensions like coherence and relevance.

Result: Found that binary ‘slop’ judgments are somewhat subjective but correlate with dimensions like coherence and relevance; the framework can evaluate AI-generated text in detection and preference tasks.

Conclusion: The proposed framework offers new insights into linguistic and stylistic factors that contribute to quality judgments of AI-generated text, providing a systematic way to assess and improve AI text quality.

Abstract: AI “slop” is an increasingly popular term used to describe low-quality AI-generated text, but there is currently no agreed upon definition of this term nor a means to measure its occurrence. In this work, we develop a taxonomy of “slop” through interviews with experts in NLP, writing, and philosophy, and propose a set of interpretable dimensions for its assessment in text. Through span-level annotation, we find that binary “slop” judgments are (somewhat) subjective, but such determinations nonetheless correlate with latent dimensions such as coherence and relevance. Our framework can be used to evaluate AI-generated text in both detection and binary preference tasks, potentially offering new insights into the linguistic and stylistic factors that contribute to quality judgments.

Method: Uses reinforcement learning with ‘soft’ tokens (mixtures of tokens plus noise on input embeddings) to learn continuous CoTs without distillation from discrete references. This minimizes computational overhead and enables training with hundreds of tokens.

Result: On math reasoning benchmarks with Llama and Qwen models up to 8B, continuous CoT training matches discrete-token CoTs for pass@1 and surpasses them for pass@32, showing greater diversity. Best performance comes from training with continuous CoTs then using discrete tokens at inference.

Conclusion: Continuous CoT RL training provides a scalable approach that preserves base model predictions better on out-of-domain tasks, offering a ‘softer touch’ to the base model while enabling more diverse and effective reasoning.

Abstract: The use of continuous instead of discrete tokens during the Chain-of-Thought (CoT) phase of reasoning LLMs has garnered attention recently, based on the intuition that a continuous mixture of discrete tokens could simulate a superposition of several reasoning paths simultaneously. Theoretical results have formally proven that continuous tokens have much greater expressivity and can solve specific problems more efficiently. However, practical use of continuous tokens has been limited by strong training difficulties: previous works either just use continuous tokens at inference time on a pre-trained discrete-token model, or must distill the continuous CoT from ground-truth discrete CoTs and face computational costs that limit the CoT to very few tokens. This is the first work introducing a scalable method to learn continuous CoTs via reinforcement learning (RL), without distilling from reference discrete CoTs. We use “soft” tokens: mixtures of tokens together with noise on the input embedding to provide RL exploration. Computational overhead is minimal, enabling us to learn continuous CoTs with hundreds of tokens. On math reasoning benchmarks with Llama and Qwen models up to 8B, training with continuous CoTs match discrete-token CoTs for pass@1 and surpass them for pass@32, showing greater CoT diversity. In systematic comparisons, the best-performing scenario is to train with continuous CoT tokens then use discrete tokens for inference, meaning the “soft” models can be deployed in a standard way. Finally, we show continuous CoT RL training better preserves the predictions of the base model on out-of-domain tasks, thus providing a softer touch to the base model.

Method: OPRL alternates between optimizing an implicit process reward model and the agent’s policy using a trajectory-based DPO objective to convert trajectory preferences into step rewards, which are combined with episode-level advantages for policy updates.

Result: OPRL achieves superior performance over frontier LLMs and strong RL baselines across WebShop, VisualSokoban, and SOTOPIA benchmarks, with state-of-the-art results, higher sample efficiency, and lower training variance.

Conclusion: OPRL provides an effective credit-assignment strategy for agentic RL that enables efficient exploration and demonstrates strong potential for real-world agent learning scenarios.

Abstract: Large language models (LLMs) are increasingly trained with reinforcement learning (RL) as autonomous agents that reason and act over long horizons in interactive environments. However, sparse and sometimes unverifiable rewards make temporal credit assignment extremely challenging. Recent work attempts to integrate process supervision into agent learning but suffers from biased annotation, reward hacking, high-variance from overly fine-grained signals or failtures when state overlap is rare. We therefore introduce Online Process Reward Learning (OPRL), a general credit-assignment strategy for agentic RL that integrates seamlessly with standard on-policy algorithms without relying on additional rollouts or explicit step labels. In OPRL, we optimize an implicit process reward model (PRM) alternately with the agent’s policy to transform trajectory preferences into implicit step rewards through a trajectory-based DPO objective. These step rewards are then used to compute step-level advantages, which are combined with episode-level advantages from outcome rewards for policy update, creating a self-reinforcing loop. Theoretical findings guarantee that the learned step rewards are consistent with trajectory preferences and act as potential-based shaping rewards, providing bounded gradients to stabilize training. Empirically, we evaluate OPRL on three distinct agent benmarks, including WebShop and VisualSokoban, as well as open-ended social interactions with unverfiable rewards in SOTOPIA. Crucially, OPRL shows superior performance over frontier LLMs and strong RL baselines across domains, achieving state-of-the-art results with higher sample-efficiency and lower variance during training. Further analysis also demonstrates the efficient exploration by OPRL using fewer actions, underscoring its potential for agentic learning in real-world scenarios.

Method: SafeCoDe uses a two-stage approach: (1) contrastive decoding that highlights context-sensitive tokens by comparing real vs Gaussian-noised images, and (2) global-aware token modulation that integrates scene-level reasoning with token-level adjustments.

Result: Extensive experiments across diverse MLLM architectures and safety benchmarks show SafeCoDe consistently improves context-sensitive refusal behaviors while preserving model helpfulness.

Conclusion: SafeCoDe effectively addresses the safety alignment gap in MLLMs by providing a model-agnostic framework that dynamically adapts safety decisions based on multimodal context.

Abstract: Multimodal Large Language Models (MLLMs) are increasingly deployed in real-world applications, yet their ability to make context-aware safety decisions remains limited. Existing methods often fail to balance oversensitivity (unjustified refusals of benign queries) and undersensitivity (missed detection of visually grounded risks), leaving a persistent gap in safety alignment. To address this issue, we introduce Safety-aware Contrastive Decoding (SafeCoDe), a lightweight and model-agnostic decoding framework that dynamically adjusts token generation based on multimodal context. SafeCoDe operates in two stages: (1) a contrastive decoding mechanism that highlights tokens sensitive to visual context by contrasting real and Gaussian-noised images, and (2) a global-aware token modulation strategy that integrates scene-level reasoning with token-level adjustment to adapt refusals according to the predicted safety verdict. Extensive experiments across diverse MLLM architectures and safety benchmarks, covering undersensitivity, oversensitivity, and general safety evaluations, show that SafeCoDe consistently improves context-sensitive refusal behaviors while preserving model helpfulness.

Method: Fine-tuned multiple pre-trained models on the Contextualized Medication Event Dataset (CMED) and applied them to three tasks: medication extraction, medical event detection, and multi-dimensional medication event context classification. Developed EHR processing methods for model compatibility.

Result: Models pre-trained on clinical data (BioBert, Clinical Bert variants) were most effective for medication and event detection, while Bert Base (pre-trained on general domain data) performed best for classifying medication event contexts.

Conclusion: Clinical domain pre-training benefits medication detection tasks, but general domain pre-training (Bert Base) excels at contextual classification of medication events in EHRs.

Abstract: Attention-based models have become the leading approach in modeling medical language for Natural Language Processing (NLP) in clinical notes. These models outperform traditional techniques by effectively capturing contextual rep- resentations of language. In this research a comparative analysis is done amongst pre- trained attention based models namely Bert Base, BioBert, two variations of Bio+Clinical Bert, RoBerta, and Clinical Long- former on task related to Electronic Health Record (EHR) information extraction. The tasks from Track 1 of Harvard Medical School’s 2022 National Clinical NLP Challenges (n2c2) are considered for this comparison, with the Contextualized Medication Event Dataset (CMED) given for these task. CMED is a dataset of unstructured EHRs and annotated notes that contain task relevant information about the EHRs. The goal of the challenge is to develop effective solutions for extracting contextual information related to patient medication events from EHRs using data driven methods. Each pre-trained model is fine-tuned and applied on CMED to perform medication extraction, medical event detection, and multi-dimensional medication event context classification. Pro- cessing methods are also detailed for breaking down EHRs for compatibility with the applied models. Performance analysis has been carried out using a script based on constructing medical terms from the evaluation portion of CMED with metrics including recall, precision, and F1-Score. The results demonstrate that models pre-trained on clinical data are more effective in detecting medication and medication events, but Bert Base, pre- trained on general domain data showed to be the most effective for classifying the context of events related to medications.

Method: CompLLM divides context into segments and compresses each independently, rather than processing the context holistically. This enables efficient linear scaling, scalability to long sequences, and reusability of compressed segments.

Result: With 2x compression rate, CompLLM speeds up Time To First Token by up to 4x and reduces KV cache size by 50% at high context lengths. It achieves comparable performance to uncompressed context and even surpasses it on very long sequences.

Conclusion: CompLLM demonstrates effectiveness and practical utility for deploying LLMs with long contexts, offering significant speed improvements and memory savings while maintaining or improving performance.

Abstract: Large Language Models (LLMs) face significant computational challenges when processing long contexts due to the quadratic complexity of self-attention. While soft context compression methods, which map input text to smaller latent representations, have shown promise, their real-world adoption is limited. Existing techniques typically compress the context as a single unit, which leads to quadratic compression complexity and an inability to reuse computations across queries with overlapping contexts. In this work, we introduce CompLLM, a soft compression technique designed for practical deployment. Instead of processing the context holistically, CompLLM divides it into segments and compresses each one independently. This simple design choice yields three critical properties: efficiency, as the compression step scales linearly with the context length; scalability, enabling models trained on short sequences (e.g., 1k tokens) to generalize to contexts of 100k tokens; and reusability, allowing compressed segments to be cached and reused across different queries. Our experiments show that with a 2x compression rate, at high context lengths CompLLM speeds up Time To First Token (TTFT) by up to 4x and reduces the KV cache size by 50%. Furthermore, CompLLM achieves performance comparable to that obtained with the uncompressed context, and even surpasses it on very long sequences, demonstrating its effectiveness and practical utility.

Method: RLPT uses reinforcement learning with a next-segment reasoning objective, where the model is rewarded for accurately predicting subsequent text segments from pre-training data, enabling autonomous exploration of meaningful trajectories.

Result: RLPT shows substantial improvements across multiple benchmarks (3.0-8.1 point gains on MMLU, MMLU-Pro, GPQA-Diamond, KOR-Bench, AIME24, AIME25) and demonstrates favorable scaling behavior with compute.

Conclusion: RLPT effectively extends LLM reasoning capabilities, provides a foundation for enhancing RLVR performance, and shows strong potential for continued scaling gains.

Abstract: The growing disparity between the exponential scaling of computational resources and the finite growth of high-quality text data now constrains conventional scaling approaches for large language models (LLMs). To address this challenge, we introduce Reinforcement Learning on Pre-Training data (RLPT), a new training-time scaling paradigm for optimizing LLMs. In contrast to prior approaches that scale training primarily through supervised learning, RLPT enables the policy to autonomously explore meaningful trajectories to learn from pre-training data and improve its capability through reinforcement learning (RL). While existing RL strategies such as reinforcement learning from human feedback (RLHF) and reinforcement learning with verifiable rewards (RLVR) rely on human annotation for reward construction, RLPT eliminates this dependency by deriving reward signals directly from pre-training data. Specifically, it adopts a next-segment reasoning objective, rewarding the policy for accurately predicting subsequent text segments conditioned on the preceding context. This formulation allows RL to be scaled on pre-training data, encouraging the exploration of richer trajectories across broader contexts and thereby fostering more generalizable reasoning skills. Extensive experiments on both general-domain and mathematical reasoning benchmarks across multiple models validate the effectiveness of RLPT. For example, when applied to Qwen3-4B-Base, RLPT yields absolute improvements of $3.0$, $5.1$, $8.1$, $6.0$, $6.6$, and $5.3$ on MMLU, MMLU-Pro, GPQA-Diamond, KOR-Bench, AIME24, and AIME25, respectively. The results further demonstrate favorable scaling behavior, suggesting strong potential for continued gains with more compute. In addition, RLPT provides a solid foundation, extending the reasoning boundaries of LLMs and enhancing RLVR performance.

Method: Encode features by embedding prototype descriptions, then fine-tune LLM to align prototype embeddings with conceptual space dimensions.

Result: Empirical analysis shows the approach is highly effective.

Conclusion: The proposed strategy successfully extracts conceptual spaces from LLMs by leveraging prototype-based feature encoding and fine-tuning.

Abstract: Conceptual spaces represent entities and concepts using cognitively meaningful dimensions, typically referring to perceptual features. Such representations are widely used in cognitive science and have the potential to serve as a cornerstone for explainable AI. Unfortunately, they have proven notoriously difficult to learn, although recent LLMs appear to capture the required perceptual features to a remarkable extent. Nonetheless, practical methods for extracting the corresponding conceptual spaces are currently still lacking. While various methods exist for extracting embeddings from LLMs, extracting conceptual spaces also requires us to encode the underlying features. In this paper, we propose a strategy in which features (e.g. sweetness) are encoded by embedding the description of a corresponding prototype (e.g. a very sweet food). To improve this strategy, we fine-tune the LLM to align the prototype embeddings with the corresponding conceptual space dimensions. Our empirical analysis finds this approach to be highly effective.

Method: The authors created and released WolBanking77 dataset containing text and voice data, then conducted experiments with various state-of-the-art NLP and ASR models as baselines.

Result: The results are promising with baseline f1-score and word error rate metrics reported for models trained on the dataset, showing good performance on this new resource.

Conclusion: The paper provides a valuable resource for intent classification research in low-resource languages and plans to maintain, update, and release open-source code for the dataset.

Abstract: Intent classification models have made a lot of progress in recent years. However, previous studies primarily focus on high-resource languages datasets, which results in a gap for low-resource languages and for regions with a high rate of illiterate people where languages are more spoken than read or written. This is the case in Senegal, for example, where Wolof is spoken by around 90% of the population, with an illiteracy rate of 42% for the country. Wolof is actually spoken by more than 10 million people in West African region. To tackle such limitations, we release a Wolof Intent Classification Dataset (WolBanking77), for academic research in intent classification. WolBanking77 currently contains 9,791 text sentences in the banking domain and more than 4 hours of spoken sentences. Experiments on various baselines are conducted in this work, including text and voice state-of-the-art models. The results are very promising on this current dataset. This paper also provides detailed analyses of the contents of the data. We report baseline f1-score and word error rate metrics respectively on NLP and ASR models trained on WolBanking77 dataset and also comparisons between models. We plan to share and conduct dataset maintenance, updates and to release open-source code.

Method: Uses graph neural networks and cross-modal alignment to inject code structural information as an auxiliary task during finetuning. The framework is model-agnostic and task-agnostic, requiring graph data only during training from unrelated corpora.

Result: Experiments on five code tasks with seven LLMs (350M to 14B parameters) show consistent improvements over baselines, including powerful models like LLaMA3 and Qwen2.5-Coder.

Conclusion: GALLa effectively enhances code LLMs by incorporating structural information without architectural changes or inference costs, demonstrating broad applicability across models and tasks.

Abstract: Programming languages possess rich semantic information - such as data flow - that is represented by graphs and not available from the surface form of source code. Recent code language models have scaled to billions of parameters, but model source code solely as text tokens while ignoring any other structural information. Conversely, models that do encode structural information of code make modifications to the Transformer architecture, limiting their scale and compatibility with pretrained LLMs. In this work, we take the best of both worlds with GALLa - Graph Aligned Large Language Models. GALLa utilizes graph neural networks and cross-modal alignment technologies to inject the structural information of code into LLMs as an auxiliary task during finetuning. This framework is both model-agnostic and task-agnostic, as it can be applied to any code LLM for any code downstream task, and requires the structural graph data only at training time from a corpus unrelated to the finetuning data, while incurring no cost at inference time over the baseline LLM. Experiments on five code tasks with seven different baseline LLMs ranging in size from 350M to 14B validate the effectiveness of GALLa, demonstrating consistent improvement over the baseline, even for powerful models such as LLaMA3 and Qwen2.5-Coder.

Method: Proposes a pipeline that uses knowledge graphs to extract entities/quantities for LLM-based diverse sample generation, and a Gaussian-decayed gradient-assisted Contrastive Sentence Embedding (GCSE) model that limits false hard negative sample impact using Gaussian-decayed functions.

Result: Achieves state-of-the-art performance in semantic textual similarity (STS) tasks with fewer data samples and smaller LLMs, demonstrating efficiency and robustness across various models.

Conclusion: The proposed approach effectively addresses data diversity and noise challenges in unsupervised sentence embedding, providing an efficient and robust solution that outperforms existing methods with reduced computational requirements.

Abstract: Recently, using large language models (LLMs) for data augmentation has led to considerable improvements in unsupervised sentence embedding models. However, existing methods encounter two primary challenges: limited data diversity and high data noise. Current approaches often neglect fine-grained knowledge, such as entities and quantities, leading to insufficient diversity. Besides, unsupervised data frequently lacks discriminative information, and the generated synthetic samples may introduce noise. In this paper, we propose a pipeline-based data augmentation method via LLMs and introduce the Gaussian-decayed gradient-assisted Contrastive Sentence Embedding (GCSE) model to enhance unsupervised sentence embeddings. To tackle the issue of low data diversity, our pipeline utilizes knowledge graphs (KGs) to extract entities and quantities, enabling LLMs to generate more diverse samples. To address high data noise, the GCSE model uses a Gaussian-decayed function to limit the impact of false hard negative samples, enhancing the model’s discriminative capability. Experimental results show that our approach achieves state-of-the-art performance in semantic textual similarity (STS) tasks, using fewer data samples and smaller LLMs, demonstrating its efficiency and robustness across various models.

Method: Post-hoc analysis of Meta advertisements using large language models to predict demographic targets (gender, age) and generate explanations for classifications. Fairness analysis using metrics like Demographic Parity, Equal Opportunity, and Predictive Equality.

Result: LLMs accurately predict demographic targeting, revealing distinct strategies: young adults targeted via activism/environmental consciousness themes, women through caregiving/social advocacy themes. Fairness analysis shows good overall performance but biases in male audience classification.

Conclusion: The study provides a framework for enhancing transparency and inclusivity in social media climate campaigns, highlighting the need for more inclusive targeting methods despite LLMs’ effectiveness.

Abstract: Climate change communication on social media increasingly employs microtargeting strategies to effectively reach and influence specific demographic groups. This study presents a post-hoc analysis of microtargeting practices within climate campaigns by leveraging large language models (LLMs) to examine Meta (previously known as Facebook) advertisements. Our analysis focuses on two key aspects: demographic targeting and fairness. We evaluate the ability of LLMs to accurately predict the intended demographic targets, such as gender and age group. Furthermore, we instruct the LLMs to generate explanations for their classifications, providing transparent reasoning behind each decision. These explanations reveal the specific thematic elements used to engage different demographic segments, highlighting distinct strategies tailored to various audiences. Our findings show that young adults are primarily targeted through messages emphasizing activism and environmental consciousness, while women are engaged through themes related to caregiving roles and social advocacy. Additionally, we conduct a comprehensive fairness analysis to uncover biases in model predictions. We assess disparities in accuracy and error rates across demographic groups using established fairness metrics such as Demographic Parity, Equal Opportunity, and Predictive Equality. Our findings indicate that while LLMs perform well overall, certain biases exist, particularly in the classification of male audiences. The analysis of thematic explanations uncovers recurring patterns in messaging strategies tailored to various demographic groups, while the fairness analysis underscores the need for more inclusive targeting methods. This study provides a valuable framework for future research aimed at enhancing transparency, accountability, and inclusivity in social media-driven climate campaigns.

Method: Develop SD-SCMs framework that allows sampling from different causal distributions, create benchmark datasets, and test various treatment effect estimation methods including average, conditional average, and individual treatment effects.

Result: Causal methods outperform non-causal methods, but even state-of-the-art methods struggle with individualized effect estimation, indicating the benchmark captures inherent difficulties in causal estimation.

Conclusion: SD-SCMs serve as a useful tool for applications requiring sequential data with controllable causal structure, including causal inference benchmarking and language model auditing.

Abstract: In this work, we present sequence-driven structural causal models (SD-SCMs), a framework for specifying causal models with user-defined structure and language-model-defined mechanisms. We characterize how an SD-SCM enables sampling from observational, interventional, and counterfactual distributions according to the desired causal structure. We then leverage this procedure to propose a new type of benchmark for causal inference methods, generating individual-level counterfactual data to test treatment effect estimation. We create an example benchmark consisting of thousands of datasets, and test a suite of popular estimation methods for average, conditional average, and individual treatment effect estimation. We find under this benchmark that (1) causal methods outperform non-causal methods and that (2) even state-of-the-art methods struggle with individualized effect estimation, suggesting this benchmark captures some inherent difficulties in causal estimation. Apart from generating data, this same technique can underpin the auditing of language models for (un)desirable causal effects, such as misinformation or discrimination. We believe SD-SCMs can serve as a useful tool in any application that would benefit from sequential data with controllable causal structure.

Method: Proposes a framework with LLM-based knowledge enhancement workflow using explanation prompts for contextual interpretations, combined with KnowFREE model that integrates extension label features for efficient nested entity extraction without external knowledge during inference.

Result: Experiments on multiple Chinese domain-specific sequence labeling datasets demonstrate state-of-the-art performance, effectively addressing low-resource challenges.

Conclusion: The proposed approach successfully overcomes limitations of existing methods by mitigating semantic biases and enabling efficient extraction in low-resource Chinese domain-specific settings.

Abstract: Sequence labeling remains a significant challenge in low-resource, domain-specific scenarios, particularly for character-dense languages like Chinese. Existing methods primarily focus on enhancing model comprehension and improving data diversity to boost performance. However, these approaches still struggle with inadequate model applicability and semantic distribution biases in domain-specific contexts. To overcome these limitations, we propose a novel framework that combines an LLM-based knowledge enhancement workflow with a span-based Knowledge Fusion for Rich and Efficient Extraction (KnowFREE) model. Our workflow employs explanation prompts to generate precise contextual interpretations of target entities, effectively mitigating semantic biases and enriching the model’s contextual understanding. The KnowFREE model further integrates extension label features, enabling efficient nested entity extraction without relying on external knowledge during inference. Experiments on multiple Chinese domain-specific sequence labeling datasets demonstrate that our approach achieves state-of-the-art performance, effectively addressing the challenges posed by low-resource settings.

Method: Created VLDBench with ~62,000 labeled text-image pairs across 13 categories from 58 news outlets using semi-automated pipeline followed by expert review (22 experts, 500+ hours, high inter-annotator agreement).

Result: Evaluations show vision-language models outperform text-only models by 5-35 percentage points in disinformation detection accuracy when incorporating visual cues.

Conclusion: VLDBench provides a principled foundation for advancing trustworthy disinformation detection in multimodal media, supporting evaluation, fine-tuning, and robustness testing in alignment with AI governance frameworks.

Abstract: Detecting disinformation that blends manipulated text and images has become increasingly challenging, as AI tools make synthetic content easy to generate and disseminate. While most existing AI safety benchmarks focus on single modality misinformation (i.e., false content shared without intent to deceive), intentional multimodal disinformation, such as propaganda or conspiracy theories that imitate credible news, remains largely unaddressed. We introduce the Vision-Language Disinformation Detection Benchmark (VLDBench), the first large-scale resource supporting both unimodal (text-only) and multimodal (text

• image) disinformation detection. VLDBench comprises approximately 62,000 labeled text-image pairs across 13 categories, curated from 58 news outlets. Using a semi-automated pipeline followed by expert review, 22 domain experts invested over 500 hours to produce high-quality annotations with substantial inter-annotator agreement. Evaluations of state-of-the-art Large Language Models (LLMs) and Vision-Language Models (VLMs) on VLDBench show that incorporating visual cues improves detection accuracy by 5 to 35 percentage points over text-only models. VLDBench provides data and code for evaluation, fine-tuning, and robustness testing to support disinformation analysis. Developed in alignment with AI governance frameworks (e.g., the MIT AI Risk Repository), VLDBench offers a principled foundation for advancing trustworthy disinformation detection in multimodal media. Project: https://vectorinstitute.github.io/VLDBench/ Dataset: https://huggingface.co/datasets/vector-institute/VLDBench Code: https://github.com/VectorInstitute/VLDBench

Method: Train probes on LM internal representations of input tokens, use conformal prediction for error bounds, create early warning systems for behaviors like jailbreaking and instruction-following failures.

Result: 91% reduction in jailbreaking, 65% average inference cost reduction across 27 datasets with negligible accuracy loss, probes generalize to unseen datasets and scale with model size.

Conclusion: Conformal probes provide effective early detection of LM behaviors, offering practical safety improvements and significant computational efficiency gains, with promising scalability to large models.

Abstract: The text produced by language models (LMs) can exhibit specific `behaviors,’ such as a failure to follow alignment training, that we hope to detect and react to during deployment. Identifying these behaviors can often only be done post facto, i.e., after the entire text of the output has been generated. We provide evidence that there are times when we can predict how an LM will behave early in computation, before even a single token is generated. We show that probes trained on the internal representation of input tokens alone can predict a wide range of eventual behaviors over the entire output sequence. Using methods from conformal prediction, we provide provable bounds on the estimation error of our probes, creating precise early warning systems for these behaviors. The conformal probes can identify instances that will trigger alignment failures (jailbreaking) and instruction-following failures, without requiring a single token to be generated. An early warning system built on the probes reduces jailbreaking by 91%. Our probes also show promise in pre-emptively estimating how confident the model will be in its response, a behavior that cannot be detected using the output text alone. Conformal probes can preemptively estimate the final prediction of an LM that uses Chain-of-Thought (CoT) prompting, hence accelerating inference. When applied to an LM that uses CoT to perform text classification, the probes drastically reduce inference costs (65% on average across 27 datasets), with negligible accuracy loss. Encouragingly, probes generalize to unseen datasets and perform better on larger models, suggesting applicability to the largest of models in real-world settings.

Method: Compared three evaluation paradigms: standard benchmarks (MMLU, BBH), interactive games (Signalling Games, Taboo), and cognitive tests (working memory, theory of mind). Analyzed discrimination effectiveness and correlations between cognitive abilities and performance.

Result: Interactive games are superior to standard benchmarks for model discrimination. Causal/logical reasoning correlates with both static and interactive tests, while executive functions and social/emotional skills correlate more with games.

Conclusion: Advocates for developing new interactive benchmarks and targeted cognitive tasks specifically designed for LLMs, inspired by human ability assessment methods.

Abstract: We examine three evaluation paradigms: standard benchmarks (e.g., MMLU and BBH), interactive games (e.g., Signalling Games or Taboo), and cognitive tests (e.g., for working memory or theory of mind). First, we investigate which of the former two-benchmarks or games-is most effective at discriminating LLMs of varying quality. Then, inspired by human cognitive assessments, we compile a suite of targeted tests that measure cognitive abilities deemed essential for effective language use, and we investigate their correlation with model performance in benchmarks and games. Our analyses reveal that interactive games are superior to standard benchmarks in discriminating models. Causal and logical reasoning correlate with both static and interactive tests, while differences emerge regarding core executive functions and social/emotional skills, which correlate more with games. We advocate for the development of new interactive benchmarks and targeted cognitive tasks inspired by assessing human abilities but designed specifically for LLMs.

Method: The authors designed tests to measure LLM efficacy in generating SCEs, analyzing various LLM families, model sizes, temperature settings, and datasets.

Result: Analysis revealed that LLMs sometimes struggle to generate SCEs, and even when they do, their predictions often disagree with their own counterfactual reasoning.

Conclusion: LLMs have limitations in generating reliable self-generated counterfactual explanations, indicating challenges in using them as effective self-explanation tools despite their reasoning capabilities.

Abstract: Explanations are an important tool for gaining insights into the behavior of ML models, calibrating user trust and ensuring regulatory compliance. Past few years have seen a flurry of post-hoc methods for generating model explanations, many of which involve computing model gradients or solving specially designed optimization problems. However, owing to the remarkable reasoning abilities of Large Language Model (LLMs), self-explanation, that is, prompting the model to explain its outputs has recently emerged as a new paradigm. In this work, we study a specific type of self-explanations, self-generated counterfactual explanations (SCEs). We design tests for measuring the efficacy of LLMs in generating SCEs. Analysis over various LLM families, model sizes, temperature settings, and datasets reveals that LLMs sometimes struggle to generate SCEs. Even when they do, their prediction often does not agree with their own counterfactual reasoning.

Method: Trained GPT-2 small on 12 languages from 4 language families using two parallel corpora, testing impossible languages and manipulating word order based on Greenberg’s Universal 20 to compare attested vs unattested NP orders.

Result: GPT-2 small could largely distinguish attested languages from impossible counterparts but not achieve perfect separation. While perplexity scores didn’t distinguish attested vs unattested word orders, generalization test performance did show some distinction.

Conclusion: Language models exhibit some human-like inductive biases in language learning, but these biases are weaker than those found in human learners.

Abstract: Do language models (LMs) offer insights into human language learning? A common argument against this idea is that because their architecture and training paradigm are so vastly different from humans, LMs can learn arbitrary inputs as easily as natural languages. We test this claim by training LMs to model impossible and typologically unattested languages. Unlike previous work, which has focused exclusively on English, we conduct experiments on 12 languages from 4 language families with two newly constructed parallel corpora. Our results show that while GPT-2 small can largely distinguish attested languages from their impossible counterparts, it does not achieve perfect separation between all the attested languages and all the impossible ones. We further test whether GPT-2 small distinguishes typologically attested from unattested languages with different NP orders by manipulating word order based on Greenberg’s Universal 20. We find that the model’s perplexity scores do not distinguish attested vs. unattested word orders, while its performance on the generalization test does. These findings suggest that LMs exhibit some human-like inductive biases, though these biases are weaker than those found in human learners.

Method: Used early decoding, causal tracing, Token Lens (decoding aggregated attention updates), and knockout method (analyzing MLP output changes after removing attention to specific tokens) across multiple datasets, models, and prompts.

Result: Identified that LMs use subject and previous answer tokens for knowledge recall, with attention propagating subject info and MLPs promoting answers. Then attention suppresses previous answers while MLPs amplify suppression.

Conclusion: The study provides new insights into how LMs’ internal components interact with different input tokens to support complex factual recall through the promote-then-suppress mechanism.

Abstract: To answer one-to-many factual queries (e.g., listing cities of a country), a language model (LM) must simultaneously recall knowledge and avoid repeating previous answers. How are these two subtasks implemented and integrated internally? Across multiple datasets, models, and prompt templates, we identify a promote-then-suppress mechanism: the model first recalls all answers, and then suppresses previously generated ones. Specifically, LMs use both the subject and previous answer tokens to perform knowledge recall, with attention propagating subject information and MLPs promoting the answers. Then, attention attends to and suppresses previous answer tokens, while MLPs amplify the suppression signal. Our mechanism is corroborated by extensive experimental evidence: in addition to using early decoding and causal tracing, we analyze how components use different tokens by introducing both Token Lens, which decodes aggregated attention updates from specified tokens, and a knockout method that analyzes changes in MLP outputs after removing attention to specified tokens. Overall, we provide new insights into how LMs’ internal components interact with different input tokens to support complex factual recall. Code is available at https://github.com/Lorenayannnnn/how-lms-answer-one-to-many-factual-queries.

Method: Joint training of teacher (Explicit CoT) and student (Implicit CoT) tasks using self-distillation, aligning hidden states of designated tokens to transfer reasoning ability from language to continuous space.

Result: CODI matches explicit CoT performance on GSM8k at GPT-2 scale, achieves 3.1x compression rate, and outperforms previous state-of-the-art implicit CoT by 28.2% in accuracy, while demonstrating robustness and interpretability.

Conclusion: LLMs can reason effectively in latent continuous space, not just natural language, validating the potential of continuous reasoning for improved efficiency and robustness.

Abstract: Chain-of-Thought (CoT) reasoning enhances Large Language Models (LLMs) by encouraging step-by-step reasoning in natural language. However, leveraging a latent continuous space for reasoning may offer benefits in terms of both efficiency and robustness. Prior implicit CoT methods attempt to bypass language completely by reasoning in continuous space but have consistently underperformed compared to the standard explicit CoT approach. We introduce CODI (Continuous Chain-of-Thought via Self-Distillation), a novel training framework that effectively compresses natural language CoT into continuous space. CODI jointly trains a teacher task (Explicit CoT) and a student task (Implicit CoT), distilling the reasoning ability from language into continuous space by aligning the hidden states of a designated token. Our experiments show that CODI is the first implicit CoT approach to match the performance of explicit CoT on GSM8k at the GPT-2 scale, achieving a 3.1x compression rate and outperforming the previous state-of-the-art by 28.2% in accuracy. CODI also demonstrates robustness, generalizable to complex datasets, and interpretability. These results validate that LLMs can reason effectively not only in natural language, but also in a latent continuous space. Code is available at https://github.com/zhenyi4/codi.

Method: CaKE leverages circuit-based analysis and uses a few curated data samples to stimulate the model to develop appropriate reasoning circuits for newly incorporated knowledge, rather than just editing isolated layers.

Result: CaKE achieves an average 20% improvement in multi-hop reasoning accuracy on the MQuAKE dataset while requiring less memory than existing KE methods.

Conclusion: The circuit-aware approach enables more accurate and consistent use of edited knowledge across related reasoning tasks, representing a significant advancement in knowledge editing for LLMs.

Abstract: Knowledge Editing (KE) enables the modification of outdated or incorrect information in large language models (LLMs). While existing KE methods can update isolated facts, they often fail to generalize these updates to multi-hop reasoning tasks that rely on the modified knowledge. Through an analysis of reasoning circuits – the neural pathways LLMs use for knowledge-based inference, we find that current layer-localized KE approaches (e.g., MEMIT, WISE), which edit only single or a few model layers, inadequately integrate updated knowledge into these reasoning pathways. To address this limitation, we present CaKE (Circuit-aware Knowledge Editing), a novel method that enhances the effective integration of updated knowledge in LLMs. By only leveraging a few curated data samples guided by our circuit-based analysis, CaKE stimulates the model to develop appropriate reasoning circuits for newly incorporated knowledge. Experiments show that CaKE enables more accurate and consistent use of edited knowledge across related reasoning tasks, achieving an average improvement of 20% in multi-hop reasoning accuracy on the MQuAKE dataset while requiring less memory than existing KE methods. We release the code and data in https://github.com/zjunlp/CaKE.

Method: Pandora employs LLMs to generate textual reasoning steps and executable Python code using Pandas API, with demonstrations from training examples covering various SKR tasks.

Result: Extensive experiments on four benchmarks across three SKR tasks show Pandora outperforms existing unified frameworks and competes effectively with task-specific methods.

Conclusion: Pandora successfully addresses the limitations of previous USKR methods by providing a unified framework that leverages LLM alignment and facilitates knowledge transfer across structured reasoning tasks.

Abstract: Unified Structured Knowledge Reasoning (USKR) aims to answer natural language questions (NLQs) by using structured sources such as tables, databases, and knowledge graphs in a unified way. Existing USKR methods either rely on employing task-specific strategies or custom-defined representations, which struggle to leverage the knowledge transfer between different SKR tasks or align with the prior of LLMs, thereby limiting their performance. This paper proposes a novel USKR framework named \textsc{Pandora}, which takes advantage of \textsc{Python}’s \textsc{Pandas} API to construct a unified knowledge representation for alignment with LLM pre-training. It employs an LLM to generate textual reasoning steps and executable Python code for each question. Demonstrations are drawn from a memory of training examples that cover various SKR tasks, facilitating knowledge transfer. Extensive experiments on four benchmarks involving three SKR tasks demonstrate that \textsc{Pandora} outperforms existing unified frameworks and competes effectively with task-specific methods.

Method: Conducted comprehensive safety evaluation of 11 MLRMs across 5 benchmarks, analyzed safety degradation patterns, and constructed a multimodal tuning dataset incorporating safety-oriented thought processes for fine-tuning existing models.

Result: Revealed prevalent safety degradation in advanced models, with significant degradation in jailbreak robustness benchmarks but less pronounced in safety-awareness benchmarks. Fine-tuning with the safety-oriented dataset effectively enhanced safety on both benchmark types.

Conclusion: Leveraging intrinsic reasoning capabilities through safety-oriented thought processes provides a new perspective for developing safe MLRMs, offering a potential solution to address safety issues.

Abstract: The rapid development of Multimodal Large Reasoning Models (MLRMs) has demonstrated broad application potential, yet their safety and reliability remain critical concerns that require systematic exploration. To address this gap, we conduct a comprehensive and systematic safety evaluation of 11 MLRMs across 5 benchmarks and unveil prevalent safety degradation phenomena in most advanced models. Moreover, our analysis reveals distinct safety patterns across different benchmarks: significant safety degradation is observed across jailbreak robustness benchmarks, whereas safety-awareness benchmarks demonstrate less pronounced degradation. In particular, the long thought process in some scenarios even enhances safety performance. Therefore, it is a potential approach to address safety issues in MLRMs by leveraging the intrinsic reasoning capabilities of the model to detect unsafe intent. To operationalize this insight, we construct a multimodal tuning dataset that incorporates a safety-oriented thought process. Experimental results from fine-tuning existing MLRMs with this dataset effectively enhances the safety on both jailbreak robustness and safety-awareness benchmarks. This study provides a new perspective for developing safe MLRMs. Our dataset is available at https://github.com/xinyuelou/Think-in-Safety.

Method: The authors constructed the first large-scale dataset with ~5K manually annotated questions, investigated automatic evaluation methods, and conducted zero-shot evaluation of 11 LLMs to establish baselines.

Result: The paper establishes strong baselines showing the difficulty of the task, proposes reference-based evaluation techniques as the best correlation with human judgments, and provides data, code, and a public leaderboard for further research.

Conclusion: The comprehensive approach supports development and benchmarking of CQs-Gen systems, encouraging research not only in model performance but also exploring practical benefits for automated reasoning and human critical thinking.

Abstract: The task of Critical Questions Generation (CQs-Gen) aims to foster critical thinking by enabling systems to generate questions that expose underlying assumptions and challenge the validity of argumentative reasoning structures. Despite growing interest in this area, progress has been hindered by the lack of suitable datasets and automatic evaluation standards. This paper presents a comprehensive approach to support the development and benchmarking of systems for this task. We construct the first large-scale dataset including ~5K manually annotated questions. We also investigate automatic evaluation methods and propose reference-based techniques as the strategy that best correlates with human judgments. Our zero-shot evaluation of 11 LLMs establishes a strong baseline while showcasing the difficulty of the task. Data and code plus a public leaderboard are provided to encourage further research, not only in terms of model performance, but also to explore the practical benefits of CQs-Gen for both automated reasoning and human critical thinking.

Method: The paper explores definitions and concepts of ambiguity, categorizes various disambiguation approaches enabled by LLMs, provides comparative analysis of their advantages/disadvantages, and examines publicly available datasets for benchmarking.

Result: The review contributes to understanding ambiguity in LLM-based systems, particularly in CQA contexts, by systematizing current knowledge and identifying effective disambiguation techniques.

Conclusion: The paper identifies open problems and future research directions, especially in agentic settings, aiming to contribute to more robust and reliable LLM-based systems through comprehensive ambiguity research.

Abstract: Ambiguity remains a fundamental challenge in Natural Language Processing (NLP) due to the inherent complexity and flexibility of human language. With the advent of Large Language Models (LLMs), addressing ambiguity has become even more critical due to their expanded capabilities and applications. In the context of Conversational Question Answering (CQA), this paper explores the definition, forms, and implications of ambiguity for language driven systems, particularly in the context of LLMs. We define key terms and concepts, categorize various disambiguation approaches enabled by LLMs, and provide a comparative analysis of their advantages and disadvantages. We also explore publicly available datasets for benchmarking ambiguity detection and resolution techniques and highlight their relevance for ongoing research. Finally, we identify open problems and future research directions, especially in agentic settings, proposing areas for further investigation. By offering a comprehensive review of current research on ambiguities and disambiguation with LLMs, we aim to contribute to the development of more robust and reliable LLM-based systems.

Method: Uses discriminative schema linking with local bidirectional attention, confusion-aware noisy schema sampling, and selective attention to improve robustness under noisy schema conditions.

Result: Achieves state-of-the-art execution accuracy on Spider and BIRD benchmarks among comparable-size open-source models, with significant improvements in training and inference efficiency.

Conclusion: JOLT-SQL provides a streamlined single-stage framework that effectively addresses limitations of previous SFT approaches while maintaining high performance and efficiency.

Abstract: Text-to-SQL, which maps natural language to SQL queries, has benefited greatly from recent advances in Large Language Models (LLMs). While LLMs offer various paradigms for this task, including prompting and supervised fine-tuning (SFT), SFT approaches still face challenges such as complex multi-stage pipelines and poor robustness to noisy schema information. To address these limitations, we present JOLT-SQL, a streamlined single-stage SFT framework that jointly optimizes schema linking and SQL generation via a unified loss. JOLT-SQL employs discriminative schema linking, enhanced by local bidirectional attention, alongside a confusion-aware noisy schema sampling strategy with selective attention to improve robustness under noisy schema conditions. Experiments on the Spider and BIRD benchmarks demonstrate that JOLT-SQL achieves state-of-the-art execution accuracy among comparable-size open-source models, while significantly improving both training and inference efficiency. Our code is available at https://github.com/Songjw133/JOLT-SQL.

Method: Created MemeSafetyBench with 50,430 instances pairing real meme images with harmful/benign instructions, using comprehensive safety taxonomy and LLM-based instruction generation to assess VLMs across single and multi-turn interactions.

Result: VLMs are more vulnerable to meme-based harmful prompts than synthetic/typographic images. Memes significantly increase harmful responses and decrease refusals compared to text-only inputs. Multi-turn interactions provide partial mitigation but elevated vulnerability persists.

Conclusion: Results highlight the need for ecologically valid evaluations and stronger safety mechanisms. MemeSafetyBench is publicly available to facilitate better safety testing.

Abstract: Rapid deployment of vision-language models (VLMs) magnifies safety risks, yet most evaluations rely on artificial images. This study asks: How safe are current VLMs when confronted with meme images that ordinary users share? To investigate this question, we introduce MemeSafetyBench, a 50,430-instance benchmark pairing real meme images with both harmful and benign instructions. Using a comprehensive safety taxonomy and LLM-based instruction generation, we assess multiple VLMs across single and multi-turn interactions. We investigate how real-world memes influence harmful outputs, the mitigating effects of conversational context, and the relationship between model scale and safety metrics. Our findings demonstrate that VLMs are more vulnerable to meme-based harmful prompts than to synthetic or typographic images. Memes significantly increase harmful responses and decrease refusals compared to text-only inputs. Though multi-turn interactions provide partial mitigation, elevated vulnerability persists. These results highlight the need for ecologically valid evaluations and stronger safety mechanisms. MemeSafetyBench is publicly available at https://github.com/oneonlee/Meme-Safety-Bench.

Method: Created MIDAS - a large-scale dataset of idioms in six languages with corresponding meanings, then conducted comprehensive evaluation of LLMs’ idiom processing abilities to identify key performance factors.

Result: LLMs rely on both memorization and a hybrid approach integrating contextual cues and reasoning, particularly for compositional idioms. Idiom understanding emerges from interplay between knowledge retrieval and reasoning-based inference.

Conclusion: The study provides insights into how LLMs process idioms, showing they employ sophisticated mechanisms beyond simple memorization, with implications for improving idiom handling in multilingual NLP applications.

Abstract: Idioms have long posed a challenge due to their unique linguistic properties, which set them apart from other common expressions. While recent studies have leveraged large language models (LLMs) to handle idioms across various tasks, e.g., idiom-containing sentence generation and idiomatic machine translation, little is known about the underlying mechanisms of idiom processing in LLMs, particularly in multilingual settings. To this end, we introduce MIDAS, a new large-scale dataset of idioms in six languages, each paired with its corresponding meaning. Leveraging this resource, we conduct a comprehensive evaluation of LLMs’ idiom processing ability, identifying key factors that influence their performance. Our findings suggest that LLMs rely not only on memorization, but also adopt a hybrid approach that integrates contextual cues and reasoning, especially when processing compositional idioms. This implies that idiom understanding in LLMs emerges from an interplay between internal knowledge retrieval and reasoning-based inference.

Method: Analyzed output distributions of autoregressive LLMs at each label generation step. Proposed zero-shot and supervised methods for distribution alignment, including taking max probability over all label generation distributions instead of just initial probabilities.

Result: Found LLMs suppress all but one label per generation step. Larger models show lower entropy and higher single-label confidence but better internal label ranking. The max probability method improves both distribution alignment and F1 classification without extra computation.

Conclusion: LLMs have limitations in multi-label classification that can be addressed through distribution alignment methods. Simple techniques like using max probabilities across generation steps can significantly improve performance in subjective multi-label tasks.

Abstract: Multi-label classification is prevalent in real-world settings, but the behavior of Large Language Models (LLMs) in this setting is understudied. We investigate how autoregressive LLMs perform multi-label classification, focusing on subjective tasks, by analyzing the output distributions of the models at each label generation step. We find that the initial probability distribution for the first label often does not reflect the eventual final output, even in terms of relative order and find LLMs tend to suppress all but one label at each generation step. We further observe that as model scale increases, their token distributions exhibit lower entropy and higher single-label confidence, but the internal relative ranking of the labels improves. Finetuning methods such as supervised finetuning and reinforcement learning amplify this phenomenon. We introduce the task of distribution alignment for multi-label settings: aligning LLM-derived label distributions with empirical distributions estimated from annotator responses in subjective tasks. We propose both zero-shot and supervised methods which improve both alignment and predictive performance over existing approaches. We find one method – taking the max probability over all label generation distributions instead of just using the initial probability distribution – improves both distribution alignment and overall F1 classification without adding any additional computation.

Method: The methodology combines controlled synthetic data generation and retrieval-augmented pre-training specifically tailored to community language, cultural heritage, and values. NileChat is developed as a 3B parameter model for Egyptian and Moroccan Arabic dialects, including Arabizi variants.

Result: NileChat outperforms existing Arabic-aware LLMs of similar size and performs on par with larger models on various understanding, translation, and cultural/values alignment benchmarks for Egyptian and Moroccan Arabic.

Conclusion: The work successfully addresses Arabic dialect representation in LLMs with cultural and values alignment, advancing Arabic NLP for low-resource communities. The methods, data, and models are shared publicly to promote inclusion of diverse communities in cultural LLM development.

Abstract: Enhancing the linguistic capabilities of Large Language Models (LLMs) to include low-resource languages is a critical research area. Current research directions predominantly rely on synthetic data generated by translating English corpora, which, while demonstrating promising linguistic understanding and translation abilities, often results in models aligned with source language culture. These models frequently fail to represent the cultural heritage and values of local communities. This work proposes a methodology to create both synthetic and retrieval-based pre-training data tailored to a specific community, considering its (i) language, (ii) cultural heritage, and (iii) cultural values. We demonstrate our methodology using Egyptian and Moroccan dialects as testbeds, chosen for their linguistic and cultural richness and current underrepresentation in LLMs. As a proof-of-concept, we develop NileChat, a 3B parameter Egyptian and Moroccan Arabic LLM adapted for Egyptian and Moroccan communities, incorporating their language, cultural heritage, and values. Our results on various understanding, translation, and cultural and values alignment benchmarks show that NileChat outperforms existing Arabic-aware LLMs of similar size and performs on par with larger models. This work addresses Arabic dialect in LLMs with a focus on cultural and values alignment via controlled synthetic data generation and retrieval-augmented pre-training for Moroccan Darija and Egyptian Arabic, including Arabizi variants, advancing Arabic NLP for low-resource communities. We share our methods, data, and models with the community to promote the inclusion and coverage of more diverse communities in cultural LLM development: https://github.com/UBC-NLP/nilechat .

Method: Comprehensive analysis of misinformation effects on intermediate reasoning steps and final answers, testing LLMs’ ability to correct misinformation with explicit instructions, and evaluating early-stage correction strategies with fine-tuning on synthesized data.

Result: LLMs succeed less than half the time in correcting misinformation despite having correct internal knowledge, causing significant accuracy drops (10.02%-72.20%), with thinking models showing smaller but still substantial degradation (4.30%-19.97%). Early-stage corrections are most effective.

Conclusion: Applying factual corrections early in the reasoning process effectively reduces misinformation propagation, and fine-tuning on synthesized data with early-stage corrections significantly improves reasoning factuality, offering a practical approach to mitigate misinformation issues.

Abstract: Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning, positioning them as promising tools for supporting human problem-solving. However, what happens when their performance is affected by misinformation, i.e., incorrect inputs introduced by users due to oversights or gaps in knowledge? Such misinformation is prevalent in real-world interactions with LLMs, yet how it propagates within LLMs’ reasoning process remains underexplored. Focusing on mathematical reasoning, we present a comprehensive analysis of how misinformation affects intermediate reasoning steps and final answers. We also examine how effectively LLMs can correct misinformation when explicitly instructed to do so. Even with explicit instructions, LLMs succeed less than half the time in rectifying misinformation, despite possessing correct internal knowledge, leading to significant accuracy drops (10.02% - 72.20%), and the degradation holds with thinking models (4.30% - 19.97%). Further analysis shows that applying factual corrections early in the reasoning process most effectively reduces misinformation propagation, and fine-tuning on synthesized data with early-stage corrections significantly improves reasoning factuality. Our work offers a practical approach to mitigating misinformation propagation.

Method: Developed LITEX taxonomy for categorizing free-text explanations, annotated a subset of e-SNLI dataset, validated taxonomy reliability, and analyzed alignment with NLI labels/highlights/explanations. Also assessed taxonomy’s usefulness in explanation generation by conditioning generation on LITEX.

Result: LITEX taxonomy reliably captures within-label variation and conditioning explanation generation on LITEX yields explanations linguistically closer to human explanations than using only labels or highlights.

Conclusion: LITEX approach captures within-label variation and demonstrates that taxonomy-guided generation bridges the gap between human and model explanations more effectively than existing strategies.

Abstract: There is increasing evidence of Human Label Variation (HLV) in Natural Language Inference (NLI), where annotators assign different labels to the same premise-hypothesis pair. However, within-label variation–cases where annotators agree on the same label but provide divergent reasoning–poses an additional and mostly overlooked challenge. Several NLI datasets contain highlighted words in the NLI item as explanations, but the same spans on the NLI item can be highlighted for different reasons, as evidenced by free-text explanations, which offer a window into annotators’ reasoning. To systematically understand this problem and gain insight into the rationales behind NLI labels, we introduce LITEX, a linguistically-informed taxonomy for categorizing free-text explanations. Using this taxonomy, we annotate a subset of the e-SNLI dataset, validate the taxonomy’s reliability, and analyze how it aligns with NLI labels, highlights, and explanations. We further assess the taxonomy’s usefulness in explanation generation, demonstrating that conditioning generation on LITEX yields explanations that are linguistically closer to human explanations than those generated using only labels or highlights. Our approach thus not only captures within-label variation but also shows how taxonomy-guided generation for reasoning can bridge the gap between human and model explanations more effectively than existing strategies.

Method: Empirical analysis comparing CoT-based translation decomposition with simpler ’translate again’ self-refinement strategies, testing on WMT24 data.

Result: No clear evidence that performance gains come from explicit decomposition; ’translate again’ self-refinement yields better results than human-like step-by-step prompting.

Conclusion: Optimal translation strategies for LLMs diverge from human strategies, with simpler self-refinement approaches being more effective than complex decomposition.

Abstract: Large Language Models (LLMs) demonstrate strong reasoning capabilities for many tasks, often by explicitly decomposing the task via Chain-of-Thought (CoT) reasoning. Recent work on LLM-based translation designs hand-crafted prompts to decompose translation, or trains models to incorporate intermediate steps. Translating Step-by-step (Briakou et al., 2024), for instance, introduces a multi-step prompt with decomposition and refinement of translation with LLMs, which achieved state-of-the-art results on WMT24 test data. In this work, we scrutinise this strategy’s effectiveness. Empirically, we find no clear evidence that performance gains stem from explicitly decomposing the translation process via CoT, at least for the models on test; and we show prompting LLMs to ’translate again’ and self-refine yields even better results than human-like step-by-step prompting. While the decomposition influences translation behaviour, faithfulness to the decomposition has both positive and negative effects on translation. Our analysis therefore suggests a divergence between the optimal translation strategies for humans and LLMs.

Method: Created LaMP-Cap dataset with multimodal figure profiles including figure images, captions, and figure-mentioning paragraphs from the same document. Tested four LLMs using profile information for personalized caption generation.

Result: Using profile information consistently improved caption generation quality, making captions closer to author-written ones. Images in profiles were more helpful than text paragraphs, demonstrating the advantage of multimodal profiles over text-only ones.

Conclusion: Multimodal profiles significantly enhance personalized figure caption generation, with visual information playing a more crucial role than textual context alone.

Abstract: Figure captions are crucial for helping readers understand and remember a figure’s key message. Many models have been developed to generate these captions, helping authors compose better quality captions more easily. Yet, authors almost always need to revise generic AI-generated captions to match their writing style and the domain’s style, highlighting the need for personalization. Despite language models’ personalization (LaMP) advances, these technologies often focus on text-only settings and rarely address scenarios where both inputs and profiles are multimodal. This paper introduces LaMP-Cap, a dataset for personalized figure caption generation with multimodal figure profiles. For each target figure, LaMP-Cap provides not only the needed inputs, such as figure images, but also up to three other figures from the same document–each with its image, caption, and figure-mentioning paragraphs–as a profile to characterize the context. Experiments with four LLMs show that using profile information consistently helps generate captions closer to the original author-written ones. Ablation studies reveal that images in the profile are more helpful than figure-mentioning paragraphs, highlighting the advantage of using multimodal profiles over text-only ones.

Method: Two-phase approach: Phase 1 decodes from base model twice (constrained and unconstrained) to get weak predictions. Phase 2 uses a learned boosted model to combine these complementary predictions into a final output.

Result: Applied to closed information extraction (BoostIE), the method outperforms prior approaches both in-distribution and out-of-distribution, addressing common errors in existing methods.

Conclusion: BoostCD effectively leverages the complementary nature of constrained and unconstrained decoding errors to achieve improved performance in structured NLP tasks, demonstrating particular success in information extraction.

Abstract: Many recent approaches to structured NLP tasks use an autoregressive language model $M$ to map unstructured input text $x$ to output text $y$ representing structured objects (such as tuples, lists, trees, code, etc.), where the desired output structure is enforced via constrained decoding. During training, these approaches do not require the model to be aware of the constraints, which are merely implicit in the training outputs $y$. This is advantageous as it allows for dynamic constraints without requiring retraining, but can lead to low-quality output during constrained decoding at test time. We overcome this problem with Boosted Constrained Decoding (BoostCD), which combines constrained and unconstrained decoding in two phases: Phase 1 decodes from the base model $M$ twice, in constrained and unconstrained mode, obtaining two weak predictions. In phase 2, a learned autoregressive boosted model combines the two weak predictions into one final prediction. The mistakes made by the base model with vs. without constraints tend to be complementary, which the boosted model learns to exploit for improved performance. We demonstrate the power of BoostCD by applying it to closed information extraction. Our model, BoostIE, outperforms prior approaches both in and out of distribution, addressing several common errors identified in those approaches.

Method: Developed allotaxonographs that use rank-turbulence divergence and other instruments like Jenson-Shannon divergence and generalized entropy divergences to create map-and-list visual comparisons.

Result: Created tools in Matlab, Javascript, and Python for rendering allotaxonographs, each suited for different use cases.

Conclusion: Allotaxonographs provide effective visual tools for comparing complex systems, with implementations available in multiple programming languages to accommodate various applications.

Abstract: Describing and comparing complex systems requires principled, theoretically grounded tools. Built around the phenomenon of type turbulence, allotaxonographs provide map-and-list visual comparisons of pairs of heavy-tailed distributions. Allotaxonographs are designed to accommodate a wide range of instruments including rank- and probability-turbulence divergences, Jenson-Shannon divergence, and generalized entropy divergences. Here, we describe a suite of programmatic tools for rendering allotaxonographs for rank-turbulence divergence in Matlab, Javascript, and Python, all of which have different use cases.

Method: T-Detect redesigns curvature-based detectors by replacing Gaussian normalization with a heavy-tailed discrepancy score from Student’s t-distribution, normalizing log-likelihood against expected t-distribution moments.

Result: T-Detect improves AUROC by up to 3.9% in targeted domains and achieves state-of-the-art performance with 0.926 AUROC on RAID Books domain when integrated into CT framework.

Conclusion: The paper provides a new theoretically-justified statistical foundation for text detection with superior robustness against adversarial conditions, validated on RAID and HART benchmarks.

Abstract: Large language models (LLMs) have shown the capability to generate fluent and logical content, presenting significant challenges to machine-generated text detection, particularly text polished by adversarial perturbations such as paraphrasing. Current zero-shot detectors often employ Gaussian distributions as statistical measure for computing detection thresholds, which falters when confronted with the heavy-tailed statistical artifacts characteristic of adversarial or non-native English texts. In this paper, we introduce T-Detect, a novel detection method that fundamentally redesigns the curvature-based detectors. Our primary innovation is the replacement of standard Gaussian normalization with a heavy-tailed discrepancy score derived from the Student’s t-distribution. This approach is theoretically grounded in the empirical observation that adversarial texts exhibit significant leptokurtosis, rendering traditional statistical assumptions inadequate. T-Detect computes a detection score by normalizing the log-likelihood of a passage against the expected moments of a t-distribution, providing superior resilience to statistical outliers. We validate our approach on the challenging RAID benchmark for adversarial text and the comprehensive HART dataset. Experiments show that T-Detect provides a consistent performance uplift over strong baselines, improving AUROC by up to 3.9% in targeted domains. When integrated into a two-dimensional detection framework (CT), our method achieves state-of-the-art performance, with an AUROC of 0.926 on the Books domain of RAID. Our contributions are a new, theoretically-justified statistical foundation for text detection, an ablation-validated method that demonstrates superior robustness, and a comprehensive analysis of its performance under adversarial conditions. Ours code are released at https://github.com/ResearAI/t-detect.

Method: TDT reformulates detection as a signal processing task by treating token-level discrepancies as a time-series signal and applying Continuous Wavelet Transform to generate a two-dimensional time-scale representation that captures both location and linguistic scale of statistical anomalies.

Result: On the RAID benchmark, TDT achieves 0.855 AUROC (7.1% improvement over best baseline). It shows robust performance on adversarial tasks with 14.1% AUROC improvement on HART Level 2 paraphrasing attacks, while maintaining only 13% computational overhead.

Conclusion: The work establishes non-stationarity as a fundamental characteristic of AI-generated text and demonstrates that preserving temporal dynamics is essential for robust detection. TDT provides a new paradigm that overcomes limitations of existing scalar aggregation approaches.

Abstract: The field of AI-generated text detection has evolved from supervised classification to zero-shot statistical analysis. However, current approaches share a fundamental limitation: they aggregate token-level measurements into scalar scores, discarding positional information about where anomalies occur. Our empirical analysis reveals that AI-generated text exhibits significant non-stationarity, statistical properties vary by 73.8% more between text segments compared to human writing. This discovery explains why existing detectors fail against localized adversarial perturbations that exploit this overlooked characteristic. We introduce Temporal Discrepancy Tomography (TDT), a novel detection paradigm that preserves positional information by reformulating detection as a signal processing task. TDT treats token-level discrepancies as a time-series signal and applies Continuous Wavelet Transform to generate a two-dimensional time-scale representation, capturing both the location and linguistic scale of statistical anomalies. On the RAID benchmark, TDT achieves 0.855 AUROC (7.1% improvement over the best baseline). More importantly, TDT demonstrates robust performance on adversarial tasks, with 14.1% AUROC improvement on HART Level 2 paraphrasing attacks. Despite its sophisticated analysis, TDT maintains practical efficiency with only 13% computational overhead. Our work establishes non-stationarity as a fundamental characteristic of AI-generated text and demonstrates that preserving temporal dynamics is essential for robust detection.

Method: Columbo uses LLM-based approach with context exploitation, rules, chain-of-thought reasoning, and token-level analysis to expand column abbreviations.

Result: Columbo outperforms the current state-of-the-art solution NameGuess by 4-29% across five datasets and has been deployed in production on EDI data lake.

Conclusion: The paper demonstrates significant advancement in column name expansion through real-world datasets, improved evaluation metrics, and the effective Columbo system that works in production environments.

Abstract: Expanding the abbreviated column names of tables, such as “esal” to “employee salary”, is critical for many downstream NLP tasks for tabular data, such as NL2SQL, table QA, and keyword search. This problem arises in enterprises, domain sciences, government agencies, and more. In this paper, we make three contributions that significantly advance the state of the art. First, we show that the synthetic public data used by prior work has major limitations, and we introduce four new datasets in enterprise/science domains, with real-world abbreviations. Second, we show that accuracy measures used by prior work seriously undercount correct expansions, and we propose new synonym-aware measures that capture accuracy much more accurately. Finally, we develop Columbo, a powerful LLM-based solution that exploits context, rules, chain-of-thought reasoning, and token-level analysis. Extensive experiments show that Columbo significantly outperforms NameGuess, the current most advanced solution, by 4-29%, over five datasets. Columbo has been used in production on EDI, a major data lake for environmental sciences.

Method: Reward-shifted speculative sampling algorithm where a small draft model is aligned with human preferences while the target model remains unchanged. The method exploits distributional shifts between aligned draft and unaligned target models by modifying acceptance criteria and bonus token distribution.

Result: The algorithm achieves superior gold reward scores at significantly reduced inference cost in test-time weak-to-strong alignment experiments, validating both effectiveness and efficiency.

Conclusion: SSS provides an efficient solution for test-time alignment by leveraging aligned draft models to recover RLHF optimal solutions without the high computational costs of traditional approaches.

Abstract: Aligning large language models (LLMs) with human preferences has become a critical step in their development. Recent research has increasingly focused on test-time alignment, where additional compute is allocated during inference to enhance LLM safety and reasoning capabilities. However, these test-time alignment techniques often incur substantial inference costs, limiting their practical application. We are inspired by the speculative sampling acceleration, which leverages a small draft model to efficiently predict future tokens, to address the efficiency bottleneck of test-time alignment. We introduce the reward-shifted speculative sampling (SSS) algorithm, in which the draft model is aligned with human preferences, while the target model remains unchanged. We theoretically demonstrate that the distributional shift between the aligned draft model and the unaligned target model can be exploited to recover the RLHF optimal solution without actually obtaining it, by modifying the acceptance criterion and bonus token distribution. Our algorithm achieves superior gold reward scores at a significantly reduced inference cost in test-time weak-to-strong alignment experiments, thereby validating both its effectiveness and efficiency.

Method: The approach involves reducing the problem to fact verification, using external evidence to mitigate hallucinations, and generating/validating atomic assumptions for interpretable answers.

Result: Experiments with five LLMs show that incorporating retrieved evidence improves performance, and generating/validating atomic assumptions yields further improvements while providing interpretability.

Conclusion: Using external evidence and atomic assumption validation effectively reduces hallucinations in LLMs when answering questions with false assumptions, offering both performance gains and interpretability.

Abstract: People often ask questions with false assumptions, a type of question that does not have regular answers. Answering such questions requires first identifying the false assumptions. Large Language Models (LLMs) often generate misleading answers to these questions because of hallucinations. In this paper, we focus on identifying and answering questions with false assumptions in several domains. We first investigate whether the problem reduces to fact verification. Then, we present an approach leveraging external evidence to mitigate hallucinations. Experiments with five LLMs demonstrate that (1) incorporating retrieved evidence is beneficial and (2) generating and validating atomic assumptions yields more improvements and provides an interpretable answer by pinpointing the false assumptions.

Method: Created OpenWHO corpus with 2,978 documents and 26,824 sentences from WHO’s e-learning platform. Evaluated modern LLMs against traditional MT models, analyzing context utilization effects on accuracy.

Result: LLMs consistently outperform traditional MT models. Gemini 2.5 Flash achieved +4.79 ChrF point improvement over NLLB-54B on low-resource test set. Document-level translation benefits are most pronounced in specialized domains like health.

Conclusion: The OpenWHO corpus addresses a critical gap and enables further research into low-resource MT in health. LLMs show superior performance, especially with document-level context in specialized domains.

Abstract: In machine translation (MT), health is a high-stakes domain characterised by widespread deployment and domain-specific vocabulary. However, there is a lack of MT evaluation datasets for low-resource languages in this domain. To address this gap, we introduce OpenWHO, a document-level parallel corpus of 2,978 documents and 26,824 sentences from the World Health Organization’s e-learning platform. Sourced from expert-authored, professionally translated materials shielded from web-crawling, OpenWHO spans a diverse range of over 20 languages, of which nine are low-resource. Leveraging this new resource, we evaluate modern large language models (LLMs) against traditional MT models. Our findings reveal that LLMs consistently outperform traditional MT models, with Gemini 2.5 Flash achieving a +4.79 ChrF point improvement over NLLB-54B on our low-resource test set. Further, we investigate how LLM context utilisation affects accuracy, finding that the benefits of document-level translation are most pronounced in specialised domains like health. We release the OpenWHO corpus to encourage further research into low-resource MT in the health domain.

Method: Construct neuron probe data from MUSE bilingual dictionaries, define language overlap neurons, propose HSIC-based metric to quantify linguistic spectrum, and select optimal bridge languages to activate shared neurons for cross-lingual transfer.

Result: Experiments on 4 cross-lingual tasks and 15 language pairs from 7 diverse families show BridgeX-ICL effectively improves performance for both high-low and moderate-low resource language pairs.

Conclusion: BridgeX-ICL validates the effectiveness of neuron sharing for cross-lingual transfer and provides insights into LLMs’ multilingual mechanisms, offering a simple yet effective solution for low-resource language performance improvement.

Abstract: The current Large Language Models (LLMs) face significant challenges in improving their performance on low-resource languages and urgently need data-efficient methods without costly fine-tuning. From the perspective of language-bridge, we propose a simple yet effective method, namely BridgeX-ICL, to improve the zero-shot Cross-lingual In-Context Learning (X-ICL) for low-resource languages. Unlike existing works focusing on language-specific neurons, BridgeX-ICL explores whether sharing neurons can improve cross-lingual performance in LLMs. We construct neuron probe data from the ground-truth MUSE bilingual dictionaries, and define a subset of language overlap neurons accordingly to ensure full activation of these anchored neurons. Subsequently, we propose an HSIC-based metric to quantify LLMs’ internal linguistic spectrum based on overlapping neurons, guiding optimal bridge selection. The experiments conducted on 4 cross-lingual tasks and 15 language pairs from 7 diverse families, covering both high-low and moderate-low pairs, validate the effectiveness of BridgeX-ICL and offer empirical insights into the underlying multilingual mechanisms of LLMs. The code is publicly available at https://github.com/xuyuemei/BridgeX-ICL.

Method: Proposed a table-to-report task and constructed T2R-bench with 457 real-world industrial tables across 19 domains and 4 table types, along with evaluation criteria for report quality assessment.

Result: Experiments with 25 LLMs showed that even state-of-the-art models like Deepseek-R1 only achieved 62.71 overall score, indicating significant room for improvement in table-to-report capabilities.

Conclusion: LLMs still have substantial limitations in handling the table-to-report task, and T2R-bench provides a valuable benchmark for future research and improvement in this practical industrial application.

Abstract: Extensive research has been conducted to explore the capabilities of large language models (LLMs) in table reasoning. However, the essential task of transforming tables information into reports remains a significant challenge for industrial applications. This task is plagued by two critical issues: 1) the complexity and diversity of tables lead to suboptimal reasoning outcomes; and 2) existing table benchmarks lack the capacity to adequately assess the practical application of this task. To fill this gap, we propose the table-to-report task and construct a bilingual benchmark named T2R-bench, where the key information flow from the tables to the reports for this task. The benchmark comprises 457 industrial tables, all derived from real-world scenarios and encompassing 19 industry domains as well as 4 types of industrial tables. Furthermore, we propose an evaluation criteria to fairly measure the quality of report generation. The experiments on 25 widely-used LLMs reveal that even state-of-the-art models like Deepseek-R1 only achieves performance with 62.71 overall score, indicating that LLMs still have room for improvement on T2R-bench.

Method: Constructs pruning and distillation sets to perform iterative block removal independently for prefill and decode stages, plus a token-aware KV Cache pruning mechanism that retains all cache in prefill but selectively reuses entries for first/last token sequences in decode.

Result: Achieves strong performance in both PD disaggregation and unified settings, with improved performance, faster inference, and 4.95× reduction in data transmission bandwidth consumption.

Conclusion: The proposed method provides an effective solution for efficient LLM deployment by addressing PD disaggregation characteristics, achieving significant computational and communication efficiency gains.

Abstract: Large Language Models (LLMs) demonstrate exceptional capabilities across various tasks, but their deployment is constrained by high computational and memory costs. Model pruning provides an effective means to alleviate these demands. However, existing methods often ignore the characteristics of prefill-decode (PD) disaggregation in practice. In this paper, we propose a novel pruning method for PD disaggregation inference, enabling more precise and efficient block and KV Cache pruning. Our approach constructs pruning and distillation sets to perform iterative block removal independently for the prefill and decode stages, obtaining better pruning solutions. Moreover, we introduce a token-aware cache pruning mechanism that retains all KV Cache in the prefill stage but selectively reuses entries for the first and last token sequences in selected layers during decode, reducing communication costs with minimal overhead. Extensive experiments demonstrate that our approach consistently achieves strong performance in both PD disaggregation and PD unified settings without disaggregation. Under the same (default) settings, our method achieves improved performance and faster inference, along with a 4.95$\times$ reduction in data transmission bandwidth consumption.

Method: The framework decomposes agentic executions into granular action and component graphs, enabling systematic agentic-situational assessment. Cross-model validation was conducted on GPT-OSS-20B and Gemini-2.0-flash using HarmBench single turn and iterative refinement attacks.

Result: Agentic-level assessment exposed agent-specific risks invisible to traditional evaluation, with tool-calling showing 24-60% higher attack success rates. The study revealed universal agentic patterns, agent transfer operations as highest-risk tools, and context-dependent attack effectiveness.

Conclusion: The findings establish the urgent need for agentic-situation evaluation paradigms, with AgentSeer providing the standardized methodology and empirical validation for assessing deployment-specific risks in AI agent systems.

Abstract: As large language models transition to agentic systems, current safety evaluation frameworks face critical gaps in assessing deployment-specific risks. We introduce AgentSeer, an observability-based evaluation framework that decomposes agentic executions into granular action and component graphs, enabling systematic agentic-situational assessment. Through cross-model validation on GPT-OSS-20B and Gemini-2.0-flash using HarmBench single turn and iterative refinement attacks, we demonstrate fundamental differences between model-level and agentic-level vulnerability profiles. Model-level evaluation reveals baseline differences: GPT-OSS-20B (39.47% ASR) versus Gemini-2.0-flash (50.00% ASR), with both models showing susceptibility to social engineering while maintaining logic-based attack resistance. However, agentic-level assessment exposes agent-specific risks invisible to traditional evaluation. We discover “agentic-only” vulnerabilities that emerge exclusively in agentic contexts, with tool-calling showing 24-60% higher ASR across both models. Cross-model analysis reveals universal agentic patterns, agent transfer operations as highest-risk tools, semantic rather than syntactic vulnerability mechanisms, and context-dependent attack effectiveness, alongside model-specific security profiles in absolute ASR levels and optimal injection strategies. Direct attack transfer from model-level to agentic contexts shows degraded performance (GPT-OSS-20B: 57% human injection ASR; Gemini-2.0-flash: 28%), while context-aware iterative attacks successfully compromise objectives that failed at model-level, confirming systematic evaluation gaps. These findings establish the urgent need for agentic-situation evaluation paradigms, with AgentSeer providing the standardized methodology and empirical validation.

Method: Created a dataset of 350 Reddit samples with images, user text, and emotion categories (sad, humor, sarcasm, happy). Designed hierarchical tasks with 6 multiple-choice and 1 open-ended question per sample, progressing from perception (basic visual elements) to cognition (scene reasoning, intent understanding, empathy). Used AI assistance (Claude 4) and manual verification for annotation quality.

Result: Comprehensive evaluation of nine leading MLLMs including GPT-5, Gemini-2.5-pro, and GPT-4o was conducted on the EmoBench-Reddit benchmark.

Conclusion: The paper introduces a novel benchmark to address the gap in evaluating multimodal emotion understanding capabilities of MLLMs, providing a more comprehensive assessment framework beyond traditional objective tasks.

Abstract: With the rapid advancement of Multimodal Large Language Models (MLLMs), they have demonstrated exceptional capabilities across a variety of vision-language tasks. However, current evaluation benchmarks predominantly focus on objective visual question answering or captioning, inadequately assessing the models’ ability to understand complex and subjective human emotions. To bridge this gap, we introduce EmoBench-Reddit, a novel, hierarchical benchmark for multimodal emotion understanding. The dataset comprises 350 meticulously curated samples from the social media platform Reddit, each containing an image, associated user-provided text, and an emotion category (sad, humor, sarcasm, happy) confirmed by user flairs. We designed a hierarchical task framework that progresses from basic perception to advanced cognition, with each data point featuring six multiple-choice questions and one open-ended question of increasing difficulty. Perception tasks evaluate the model’s ability to identify basic visual elements (e.g., colors, objects), while cognition tasks require scene reasoning, intent understanding, and deep empathy integrating textual context. We ensured annotation quality through a combination of AI assistance (Claude 4) and manual verification.We conducted a comprehensive evaluation of nine leading MLLMs, including GPT-5, Gemini-2.5-pro, and GPT-4o, on EmoBench-Reddit.

Method: Paragraph-centric approach using PU Learning with SciBERT for workflow paragraph identification, Flan-T5 with prompt learning for workflow phrase generation, and ChatGPT with few-shot learning for categorization into data preparation, processing, and analysis stages, followed by visual flowchart generation.

Result: Achieved F1-score of 0.9772 for paragraph identification, ROUGE scores of 0.4543/0.2877/0.4427 for phrase generation, and 0.958 precision for categorization. Successfully revealed methodological shifts in NLP over two decades.

Conclusion: Provides a validated technical framework for automated workflow generation and a process-oriented perspective for investigating evolving scientific paradigms, with available source code and data.

Abstract: The automated generation of research workflows is essential for improving the reproducibility of research and accelerating the paradigm of “AI for Science”. However, existing methods typically extract merely fragmented procedural components and thus fail to capture complete research workflows. To address this gap, we propose an end-to-end framework that generates comprehensive, structured research workflows by mining full-text academic papers. As a case study in the Natural Language Processing (NLP) domain, our paragraph-centric approach first employs Positive-Unlabeled (PU) Learning with SciBERT to identify workflow-descriptive paragraphs, achieving an F1-score of 0.9772. Subsequently, we utilize Flan-T5 with prompt learning to generate workflow phrases from these paragraphs, yielding ROUGE-1, ROUGE-2, and ROUGE-L scores of 0.4543, 0.2877, and 0.4427, respectively. These phrases are then systematically categorized into data preparation, data processing, and data analysis stages using ChatGPT with few-shot learning, achieving a classification precision of 0.958. By mapping categorized phrases to their document locations in the documents, we finally generate readable visual flowcharts of the entire research workflows. This approach facilitates the analysis of workflows derived from an NLP corpus and reveals key methodological shifts over the past two decades, including the increasing emphasis on data analysis and the transition from feature engineering to ablation studies. Our work offers a validated technical framework for automated workflow generation, along with a novel, process-oriented perspective for the empirical investigation of evolving scientific paradigms. Source code and data are available at: https://github.com/ZH-heng/research_workflow.

Method: Red-teaming approach to systematically probe LLM vulnerabilities across three real-world scenarios: conversation, question-answering, and content composition using the SGToxicGuard dataset.

Result: Extensive experiments with state-of-the-art multilingual LLMs uncover critical gaps in their safety guardrails, revealing vulnerabilities in culturally sensitive contexts.

Conclusion: The work provides actionable insights for cultural sensitivity and toxicity mitigation, laying foundation for safer AI systems in linguistically diverse environments.

Abstract: The advancement of Large Language Models (LLMs) has transformed natural language processing; however, their safety mechanisms remain under-explored in low-resource, multilingual settings. Here, we aim to bridge this gap. In particular, we introduce \textsf{SGToxicGuard}, a novel dataset and evaluation framework for benchmarking LLM safety in Singapore’s diverse linguistic context, including Singlish, Chinese, Malay, and Tamil. SGToxicGuard adopts a red-teaming approach to systematically probe LLM vulnerabilities in three real-world scenarios: \textit{conversation}, \textit{question-answering}, and \textit{content composition}. We conduct extensive experiments with state-of-the-art multilingual LLMs, and the results uncover critical gaps in their safety guardrails. By offering actionable insights into cultural sensitivity and toxicity mitigation, we lay the foundation for safer and more inclusive AI systems in linguistically diverse environments.\footnote{Link to the dataset: https://github.com/Social-AI-Studio/SGToxicGuard.} \textcolor{red}{Disclaimer: This paper contains sensitive content that may be disturbing to some readers.}

Method: Systematically constructed minimal pairs of factually equivalent German claims that differ only in political connotation (using euphemisms vs dysphemisms). Evaluated six LLMs by having them classify these claims as true or false to assess consistency.

Result: The presence of judgmental words significantly influences truthfulness assessment more than political leaning. While a few models showed tendencies of political bias, this bias was not mitigated by explicitly calling for objectivism in prompts.

Conclusion: Political bias in LLMs’ fact-checking is more influenced by judgmental language than political orientation, and simple prompt modifications don’t effectively mitigate existing biases.

Abstract: Large Language Models are increasingly used in applications requiring objective assessment, which could be compromised by political bias. Many studies found preferences for left-leaning positions in LLMs, but downstream effects on tasks like fact-checking remain underexplored. In this study, we systematically investigate political bias through exchanging words with euphemisms or dysphemisms in German claims. We construct minimal pairs of factually equivalent claims that differ in political connotation, to assess the consistency of LLMs in classifying them as true or false. We evaluate six LLMs and find that, more than political leaning, the presence of judgmental words significantly influences truthfulness assessment. While a few models show tendencies of political bias, this is not mitigated by explicitly calling for objectivism in prompts. Warning: This paper contains content that may be offensive or upsetting.

Method: The authors propose DivLogicEval, a benchmark consisting of natural sentences composed of diverse statements arranged in counterintuitive ways. They also introduce a new evaluation metric designed to mitigate the influence of bias and randomness inherent in LLMs.

Result: Experiments demonstrate that DivLogicEval effectively requires logical reasoning to answer questions and provides comparisons of different popular LLMs’ performance in conducting logical reasoning tasks.

Conclusion: DivLogicEval offers a more reliable and faithful evaluation framework for assessing LLMs’ logical reasoning abilities, addressing the limitations of existing benchmarks through improved language diversity and distribution alignment with ideal logic reasoning evaluation.

Abstract: Logic reasoning in natural language has been recognized as an important measure of human intelligence for Large Language Models (LLMs). Popular benchmarks may entangle multiple reasoning skills and thus provide unfaithful evaluations on the logic reasoning skill. Meanwhile, existing logic reasoning benchmarks are limited in language diversity and their distributions are deviated from the distribution of an ideal logic reasoning benchmark, which may lead to biased evaluation results. This paper thereby proposes a new classical logic benchmark DivLogicEval, consisting of natural sentences composed of diverse statements in a counterintuitive way. To ensure a more reliable evaluation, we also introduce a new evaluation metric that mitigates the influence of bias and randomness inherent in LLMs. Through experiments, we demonstrate the extent to which logical reasoning is required to answer the questions in DivLogicEval and compare the performance of different popular LLMs in conducting logical reasoning.

Method: ZeroRepo uses a three-stage graph-driven framework: proposal-level planning and implementation-level refinement to construct the Repository Planning Graph (RPG), followed by graph-guided code generation with test validation.

Result: On the RepoCraft benchmark, ZeroRepo generates repositories averaging 36K code lines (3.9× stronger than Claude Code), achieves 81.5% functional coverage and 69.7% pass rate, exceeding Claude Code by 27.3 and 35.8 percentage points respectively.

Conclusion: RPG effectively models complex dependencies, enables sophisticated planning through near-linear scaling, and enhances LLM understanding of repositories, accelerating agent localization for repository generation.

Abstract: Large language models excel at function- and file-level code generation, yet generating complete repositories from scratch remains a fundamental challenge. This process demands coherent and reliable planning across proposal- and implementation-level stages, while natural language, due to its ambiguity and verbosity, is ill-suited for faithfully representing complex software structures. To address this, we introduce the Repository Planning Graph (RPG), a persistent representation that unifies proposal- and implementation-level planning by encoding capabilities, file structures, data flows, and functions in one graph. RPG replaces ambiguous natural language with an explicit blueprint, enabling long-horizon planning and scalable repository generation. Building on RPG, we develop ZeroRepo, a graph-driven framework for repository generation from scratch. It operates in three stages: proposal-level planning and implementation-level refinement to construct the graph, followed by graph-guided code generation with test validation. To evaluate this setting, we construct RepoCraft, a benchmark of six real-world projects with 1,052 tasks. On RepoCraft, ZeroRepo generates repositories averaging 36K Code Lines, roughly 3.9$\times$ the strongest baseline (Claude Code) and about 64$\times$ other baselines. It attains 81.5% functional coverage and a 69.7% pass rate, exceeding Claude Code by 27.3 and 35.8 percentage points, respectively. Further analysis shows that RPG models complex dependencies, enables progressively more sophisticated planning through near-linear scaling, and enhances LLM understanding of repositories, thereby accelerating agent localization.

Method: Developed an interactive system integrating EuroParlVote benchmark data with LLM predictions, enabling browsing of debates, comparison of real vs predicted votes, and demographic error analysis through visual analytics.

Result: The system successfully highlights systematic performance bias in state-of-the-art LLMs and provides a comprehensive interface for reproducing findings, auditing behavior, and running counterfactual scenarios.

Conclusion: ParlAI Vote effectively demonstrates both the strengths and limitations of current LLMs in political analysis while supporting research, education, and public engagement with legislative processes.

Abstract: We present ParlAI Vote, an interactive system for exploring European Parliament debates and votes, and for testing LLMs on vote prediction and bias analysis. This platform connects debate topics, speeches, and roll-call outcomes, and includes rich demographic data such as gender, age, country, and political group. Users can browse debates, inspect linked speeches, compare real voting outcomes with predictions from frontier LLMs, and view error breakdowns by demographic group. Visualizing the EuroParlVote benchmark and its core tasks of gender classification and vote prediction, ParlAI Vote highlights systematic performance bias in state-of-the-art LLMs. The system unifies data, models, and visual analytics in a single interface, lowering the barrier for reproducing findings, auditing behavior, and running counterfactual scenarios. It supports research, education, and public engagement with legislative decision-making, while making clear both the strengths and the limitations of current LLMs in political analysis.

Method: Proposes PruneCD which constructs the amateur model via layer pruning rather than early exit, creating better-aligned and more informative logits for contrastive decoding.

Result: PruneCD consistently improves factuality in LLMs with minimal inference overhead, as demonstrated through qualitative and quantitative analyses.

Conclusion: PruneCD offers a robust and practical approach to mitigating hallucinations in large language models through more effective contrastive decoding.

Abstract: To mitigate the hallucination problem in large language models, DoLa exploits early exit logits from the same model as a contrastive prior. However, we found that these early exit logits tend to be flat, low in magnitude, and fail to reflect meaningful contrasts. To address this, we propose PruneCD, a novel contrastive decoding method that constructs the amateur model via layer pruning rather than early exit. This design leads to more informative and well-aligned logits, enabling more effective contrastive decoding. Through qualitative and quantitative analyses, we demonstrate that PruneCD consistently improves factuality with minimal inference overhead, offering a robust and practical approach to mitigating hallucinations in LLMs.

Method: Three-stage RL framework: (1) Warm-up for initial perception and reasoning, (2) PA-GRPO with continuous TEDS rewards for table structure recognition, (3) HC-GRPO with fine-grained residual step rewards based on hint-guided questions.

Result: Table-R1 significantly boosts table reasoning performance on both held-in and held-out datasets, outperforming SFT and GRPO. Qwen2-VL-7B with Table-R1 surpasses larger models like Table-LLaVA 13B and achieves comparable performance to GPT-4o on held-in datasets.

Conclusion: Table-R1 effectively overcomes initialization bottlenecks and reward sparsity, demonstrating the efficacy of each stage in advancing robust multimodal table understanding.

Abstract: Existing table understanding methods face challenges due to complex table structures and intricate logical reasoning. While supervised finetuning (SFT) dominates existing research, reinforcement learning (RL), such as Group Relative Policy Optimization (GRPO), has shown promise but struggled with low initial policy accuracy and coarse rewards in tabular contexts. In this paper, we introduce Table-R1, a three-stage RL framework that enhances multimodal table understanding through: (1) Warm-up that prompts initial perception and reasoning capabilities, (2) Perception Alignment GRPO (PA-GRPO), which employs continuous Tree-Edit-Distance Similarity (TEDS) rewards for recognizing table structures and contents, and (3) Hint-Completion GRPO (HC-GRPO), which utilizes fine-grained rewards of residual steps based on the hint-guided question. Extensive experiments demonstrate that Table-R1 can boost the model’s table reasoning performance obviously on both held-in and held-out datasets, outperforming SFT and GRPO largely. Notably, Qwen2-VL-7B with Table-R1 surpasses larger specific table understanding models (e.g., Table-LLaVA 13B), even achieving comparable performance to the closed-source model GPT-4o on held-in datasets, demonstrating the efficacy of each stage of Table-R1 in overcoming initialization bottlenecks and reward sparsity, thereby advancing robust multimodal table understanding.

Method: K-DeCore introduces a knowledge decoupling mechanism that disentangles reasoning into task-specific and task-agnostic stages, along with dual-perspective memory consolidation and structure-guided pseudo-data synthesis.

Result: Extensive experiments on four benchmark datasets show superiority over existing continual learning methods across multiple metrics using various backbone large language models.

Conclusion: K-DeCore effectively bridges gaps across diverse tasks in continual structured knowledge reasoning while maintaining fixed parameters.

Abstract: Continual Structured Knowledge Reasoning (CSKR) focuses on training models to handle sequential tasks, where each task involves translating natural language questions into structured queries grounded in structured knowledge. Existing general continual learning approaches face significant challenges when applied to this task, including poor generalization to heterogeneous structured knowledge and inefficient reasoning due to parameter growth as tasks increase. To address these limitations, we propose a novel CSKR framework, \textsc{K-DeCore}, which operates with a fixed number of tunable parameters. Unlike prior methods, \textsc{K-DeCore} introduces a knowledge decoupling mechanism that disentangles the reasoning process into task-specific and task-agnostic stages, effectively bridging the gaps across diverse tasks. Building on this foundation, \textsc{K-DeCore} integrates a dual-perspective memory consolidation mechanism for distinct stages and introduces a structure-guided pseudo-data synthesis strategy to further enhance the model’s generalization capabilities. Extensive experiments on four benchmark datasets demonstrate the superiority of \textsc{K-DeCore} over existing continual learning methods across multiple metrics, leveraging various backbone large language models.

Method: Proposes QWHA which integrates FT-based adapters using Walsh-Hadamard Transform as the kernel, combined with novel adapter initialization featuring adaptive parameter selection and value refinement.

Result: QWHA consistently outperforms baselines in low-bit quantization accuracy and achieves significant training speedups over existing FT-based adapters.

Conclusion: QWHA effectively mitigates quantization errors while facilitating fine-tuning with reduced computational cost, making it a promising approach for efficient LLM deployment.

Abstract: The demand for efficient deployment of large language models (LLMs) has driven interest in quantization, which reduces inference cost, and parameter-efficient fine-tuning (PEFT), which lowers training overhead. This motivated the development of quantization-aware PEFT to produce accurate yet efficient quantized models. In this setting, reducing quantization error prior to fine-tuning is crucial for achieving high model accuracy. However, existing methods that rely on low-rank adaptation suffer from limited representational capacity. Recent Fourier-related transform (FT)-based adapters offer greater representational power than low-rank adapters, but their direct integration into quantized models often results in ineffective error reduction and increased computational overhead. To overcome these limitations, we propose QWHA, a method that integrates FT-based adapters into quantized models by employing the Walsh-Hadamard Transform (WHT) as the transform kernel, together with a novel adapter initialization scheme incorporating adaptive parameter selection and value refinement. We demonstrate that QWHA effectively mitigates quantization errors while facilitating fine-tuning, and that its design substantially reduces computational cost. Experimental results show that QWHA consistently outperforms baselines in low-bit quantization accuracy and achieves significant training speedups over existing FT-based adapters. The code is available at https://github.com/vantaa89/qwha.

Method: Proposed Aggregate Semantic Grouping (ASG) leveraging Product Quantization (PQ), applied to transformer architectures (mBERT, XLM-R, mT5, BioBERT) and evaluated across NLI, NER, QA tasks and biomedical benchmarks.

Result: ASG achieves 0.4-0.5% embedding parameter compression while maintaining >95% task performance relative to base models, working effectively in generative tasks, cross-lingual transfer, and domain-specific settings.

Conclusion: Tokens can be effectively modeled as combinations of shared semantic building blocks, and ASG provides a simple method for achieving compositional representations that capture linguistic richness while enabling compact yet semantically rich models.

Abstract: Standard language models employ unique, monolithic embeddings for each token, potentially limiting their ability to capture the multifaceted nature of word meanings. We investigate whether tokens can be more effectively represented through a compositional structure that accumulates diverse semantic facets. To explore this, we propose Aggregate Semantic Grouping (ASG), a novel approach leveraging Product Quantization (PQ). We apply ASG to standard transformer architectures (mBERT, XLM-R, mT5) and evaluate this representational scheme across diverse tasks (NLI, NER, QA), as well as a biomedical domain-specific benchmark (BC5CDR) using BioBERT. Our findings demonstrate that representing tokens compositionally via ASG achieves extreme compression in embedding parameters (0.4–0.5%) while maintaining $>$95% task performance relative to the base model, even in generative tasks and extends to both cross lingual transfer and domain-specific settings. These results validate the principle that tokens can be effectively modeled as combinations of shared semantic building blocks. ASG offers a simple yet concrete method for achieving this, showcasing how compositional representations can capture linguistic richness while enabling compact yet semantically rich models.

Method: Combines pre-trained Transformer models with Neural Networks and Conditional Random Fields (CRFs) to extract semantic/syntactic patterns and capture sequence-level representations for improved boundary prediction between human and AI text segments.

Result: The model accurately detects spans of AI text in collaborative human-AI documents, outperforming zero-shot detectors and existing state-of-the-art models on benchmark datasets.

Conclusion: The proposed sentence-level sequence labeling approach provides more precise detection of AI-generated content in hybrid texts, addressing limitations of document-level classifiers and enabling better identification of AI text boundaries within collaborative documents.

Abstract: Generation of Artificial Intelligence (AI) texts in important works has become a common practice that can be used to misuse and abuse AI at various levels. Traditional AI detectors often rely on document-level classification, which struggles to identify AI content in hybrid or slightly edited texts designed to avoid detection, leading to concerns about the model’s efficiency, which makes it hard to distinguish between human-written and AI-generated texts. A sentence-level sequence labeling model proposed to detect transitions between human- and AI-generated text, leveraging nuanced linguistic signals overlooked by document-level classifiers. By this method, detecting and segmenting AI and human-written text within a single document at the token-level granularity is achieved. Our model combines the state-of-the-art pre-trained Transformer models, incorporating Neural Networks (NN) and Conditional Random Fields (CRFs). This approach extends the power of transformers to extract semantic and syntactic patterns, and the neural network component to capture enhanced sequence-level representations, thereby improving the boundary predictions by the CRF layer, which enhances sequence recognition and further identification of the partition between Human- and AI-generated texts. The evaluation is performed on two publicly available benchmark datasets containing collaborative human and AI-generated texts. Our experimental comparisons are with zero-shot detectors and the existing state-of-the-art models, along with rigorous ablation studies to justify that this approach, in particular, can accurately detect the spans of AI texts in a completely collaborative text. All our source code and the processed datasets are available in our GitHub repository.

Method: Integration of U-Net architecture with Gradient-weighted Class Activation Mapping (Grad-CAM) for explainable polyp segmentation, trained and evaluated on the Kvasir-SEG dataset of 1000 annotated endoscopic images.

Result: Achieved robust segmentation performance with mean IoU of 0.9257 on test set and consistently high Dice coefficients (F-score > 0.96) on training/validation sets. Grad-CAM confirmed predictions were guided by clinically relevant regions.

Conclusion: PolypSeg-GradCAM represents a step toward reliable, trustworthy AI-assisted colonoscopy by coupling high segmentation accuracy with interpretability, potentially improving early colorectal cancer prevention.

Abstract: Colorectal cancer (CRC) remains one of the leading causes of cancer-related morbidity and mortality worldwide, with gastrointestinal (GI) polyps serving as critical precursors according to the World Health Organization (WHO). Early and accurate segmentation of polyps during colonoscopy is essential for reducing CRC progression, yet manual delineation is labor-intensive and prone to observer variability. Deep learning methods have demonstrated strong potential for automated polyp analysis, but their limited interpretability remains a barrier to clinical adoption. In this study, we present PolypSeg-GradCAM, an explainable deep learning framework that integrates the U-Net architecture with Gradient-weighted Class Activation Mapping (Grad-CAM) for transparent polyp segmentation. The model was trained and evaluated on the Kvasir-SEG dataset of 1000 annotated endoscopic images. Experimental results demonstrate robust segmentation performance, achieving a mean Intersection over Union (IoU) of 0.9257 on the test set and consistently high Dice coefficients (F-score > 0.96) on training and validation sets. Grad-CAM visualizations further confirmed that predictions were guided by clinically relevant regions, enhancing transparency and trust in the model’s decisions. By coupling high segmentation accuracy with interpretability, PolypSeg-GradCAM represents a step toward reliable, trustworthy AI-assisted colonoscopy and improved early colorectal cancer prevention.

Method: Developed and evaluated using multiple convolutional neural networks (ResNet-18, EfficientNet-B0, SqueezeNet) to optimize accuracy and computational efficiency, with cloud-based scalability and secure data management.

Result: Achieved 85.4% accuracy in disease severity classification, enabling real-time screening capabilities suitable for clinical and telemedicine applications.

Conclusion: PerceptronCARE demonstrates the potential of AI-driven telemedicine solutions to expand access to diabetic retinopathy screening, especially in remote and resource-constrained environments, while improving early diagnosis and reducing healthcare costs.

Abstract: Diabetic retinopathy is a leading cause of vision loss among adults and a major global health challenge, particularly in underserved regions. This study presents PerceptronCARE, a deep learning-based teleophthalmology application designed for automated diabetic retinopathy detection using retinal images. The system was developed and evaluated using multiple convolutional neural networks, including ResNet-18, EfficientNet-B0, and SqueezeNet, to determine the optimal balance between accuracy and computational efficiency. The final model classifies disease severity with an accuracy of 85.4%, enabling real-time screening in clinical and telemedicine settings. PerceptronCARE integrates cloud-based scalability, secure patient data management, and a multi-user framework, facilitating early diagnosis, improving doctor-patient interactions, and reducing healthcare costs. This study highlights the potential of AI-driven telemedicine solutions in expanding access to diabetic retinopathy screening, particularly in remote and resource-constrained environments.

Method: SIM framework uses inverse mapping mechanism to reconstruct input from transformed output. ρSIM implementation incorporates patch-level feature sampling and projection-based reconstruction of latent features to reduce computational complexity while maintaining effectiveness.

Result: Extensive evaluation across image classification, few-shot prompt learning, and domain generalization shows consistent improvements over baselines. ρSIM is orthogonal to existing regularization methods and boosts their effectiveness. It also works well in dense-to-dense tasks and non-visual domains.

Conclusion: SIM/ρSIM is an effective, model-agnostic, task-agnostic regularizer that can be seamlessly integrated as a plug-and-play module to enhance representation learning across various architectures and tasks while preserving semantic information.

Abstract: Regularization is essential in deep learning to enhance generalization and mitigate overfitting. However, conventional techniques often rely on heuristics, making them less reliable or effective across diverse settings. We propose Self Identity Mapping (SIM), a simple yet effective, data-intrinsic regularization framework that leverages an inverse mapping mechanism to enhance representation learning. By reconstructing the input from its transformed output, SIM reduces information loss during forward propagation and facilitates smoother gradient flow. To address computational inefficiencies, We instantiate SIM as $ \rho\text{SIM} $ by incorporating patch-level feature sampling and projection-based method to reconstruct latent features, effectively lowering complexity. As a model-agnostic, task-agnostic regularizer, SIM can be seamlessly integrated as a plug-and-play module, making it applicable to different network architectures and tasks. We extensively evaluate $\rho\text{SIM}$ across three tasks: image classification, few-shot prompt learning, and domain generalization. Experimental results show consistent improvements over baseline methods, highlighting $\rho\text{SIM}$’s ability to enhance representation learning across various tasks. We also demonstrate that $\rho\text{SIM}$ is orthogonal to existing regularization methods, boosting their effectiveness. Moreover, our results confirm that $\rho\text{SIM}$ effectively preserves semantic information and enhances performance in dense-to-dense tasks, such as semantic segmentation and image translation, as well as in non-visual domains including audio classification and time series anomaly detection. The code is publicly available at https://github.com/XiudingCai/SIM-pytorch.

Method: MAGIA uses momentum-based adaptive correction with two innovations: 1) closed-form combinatorial rescaling for tighter optimization bounds, and 2) momentum-based mixing of whole-batch and subset losses for robust reconstruction by probing random data subsets.

Result: Extensive experiments show MAGIA significantly outperforms advanced methods, achieving high-fidelity multi-image reconstruction in large batch scenarios where prior works fail, with computational footprint comparable to standard solvers.

Conclusion: MAGIA provides an effective label-inference-free framework for gradient inversion attacks that works robustly in challenging single-round averaged gradient settings without requiring auxiliary information.

Abstract: We study gradient inversion in the challenging single round averaged gradient SAG regime where per sample cues are entangled within a single batch mean gradient. We introduce MAGIA a momentum based adaptive correction on gradient inversion attack a novel label inference free framework that senses latent per image signals by probing random data subsets. MAGIA objective integrates two core innovations 1 a closed form combinatorial rescaling that creates a provably tighter optimization bound and 2 a momentum based mixing of whole batch and subset losses to ensure reconstruction robustness. Extensive experiments demonstrate that MAGIA significantly outperforms advanced methods achieving high fidelity multi image reconstruction in large batch scenarios where prior works fail. This is all accomplished with a computational footprint comparable to standard solvers and without requiring any auxiliary information.

Method: Proposes LEAF-Mamba with two key components: 1) Local Emphatic State Space Module (LE-SSM) to capture multi-scale local dependencies for both RGB and depth modalities, 2) SSM-based Adaptive Fusion Module (AFM) for complementary cross-modality interaction and reliable integration.

Result: Extensive experiments show LEAF-Mamba consistently outperforms 16 state-of-the-art RGB-D SOD methods in both efficacy and efficiency. The method also achieves excellent performance on RGB-T SOD task, demonstrating strong generalization ability.

Conclusion: LEAF-Mamba provides an effective solution for RGB-D SOD by combining the strengths of state space models with specialized modules for local feature extraction and cross-modality fusion, achieving superior performance while maintaining computational efficiency.

Abstract: RGB-D salient object detection (SOD) aims to identify the most conspicuous objects in a scene with the incorporation of depth cues. Existing methods mainly rely on CNNs, limited by the local receptive fields, or Vision Transformers that suffer from the cost of quadratic complexity, posing a challenge in balancing performance and computational efficiency. Recently, state space models (SSM), Mamba, have shown great potential for modeling long-range dependency with linear complexity. However, directly applying SSM to RGB-D SOD may lead to deficient local semantics as well as the inadequate cross-modality fusion. To address these issues, we propose a Local Emphatic and Adaptive Fusion state space model (LEAF-Mamba) that contains two novel components: 1) a local emphatic state space module (LE-SSM) to capture multi-scale local dependencies for both modalities. 2) an SSM-based adaptive fusion module (AFM) for complementary cross-modality interaction and reliable cross-modality integration. Extensive experiments demonstrate that the LEAF-Mamba consistently outperforms 16 state-of-the-art RGB-D SOD methods in both efficacy and efficiency. Moreover, our method can achieve excellent performance on the RGB-T SOD task, proving a powerful generalization ability.

Method: Fine-tuned a pre-trained MLLM using decoder-only strategy on large-scale dataset combining synthetic and real-world Arabic documents, while preserving general visual features. Also created Misraj-DocOCR benchmark for evaluation.

Result: Baseer significantly outperforms existing open-source and commercial solutions, achieving WER of 0.25, establishing new state-of-the-art in Arabic document OCR.

Conclusion: Domain-specific adaptation of general-purpose MLLMs is highly beneficial for high-accuracy OCR on morphologically rich languages like Arabic, establishing a strong baseline for future research.

Abstract: Arabic document OCR remains a challenging task due to the language’s cursive script, diverse fonts, diacritics, and right-to-left orientation. While modern Multimodal Large Language Models (MLLMs) have advanced document understanding for high-resource languages, their performance on Arabic remains limited. In this work, we introduce Baseer, a vision-language model fine- tuned specifically for Arabic document OCR. Leveraging a large-scale dataset combining synthetic and real-world documents, Baseer is trained using a decoder-only fine-tuning strategy to adapt a pre-trained MLLM while preserving general visual features. We also present Misraj-DocOCR, a high-quality, expert-verified benchmark designed for rigorous evaluation of Arabic OCR systems. Our experiments show that Baseer significantly outperforms existing open-source and commercial solutions, achieving a WER of 0.25 and establishing a new state-of-the-art in the domain of Arabic document OCR. Our results highlight the benefits of domain-specific adaptation of general-purpose MLLMs and establish a strong baseline for high-accuracy OCR on morphologically rich languages like Arabic.

Method: Proposes an optimal transport-based distance measure between fine-grained visual and textual feature sets to reassign captions, and uses optimal transport-based objective functions to encourage inter- and intra-modality fine-grained alignment.

Result: OTCCLIP successfully decreases attack success rates of poisoning attacks and significantly improves CLIP’s zero-shot and linear probing performance on poisoned datasets compared to previous methods.

Conclusion: The proposed OTCCLIP framework effectively defends against poisoning attacks on CLIP models by leveraging fine-grained feature alignment through optimal transport, outperforming existing defense approaches.

Abstract: Recent studies have shown that Contrastive Language-Image Pre-training (CLIP) models are threatened by targeted data poisoning and backdoor attacks due to massive training image-caption pairs crawled from the Internet. Previous defense methods correct poisoned image-caption pairs by matching a new caption for each image. However, the matching process relies solely on the global representations of images and captions, overlooking fine-grained features of visual and textual features. It may introduce incorrect image-caption pairs and harm the CLIP pre-training. To address their limitations, we propose an Optimal Transport-based framework to reconstruct image-caption pairs, named OTCCLIP. We propose a new optimal transport-based distance measure between fine-grained visual and textual feature sets and re-assign new captions based on the proposed optimal transport distance. Additionally, to further reduce the negative impact of mismatched pairs, we encourage the inter- and intra-modality fine-grained alignment by employing optimal transport-based objective functions. Our experiments demonstrate that OTCCLIP can successfully decrease the attack success rates of poisoning attacks. Also, compared to previous methods, OTCCLIP significantly improves CLIP’s zero-shot and linear probing performance trained on poisoned datasets.

Method: Hybrid CNN-LSTM model engineered to simultaneously learn spatial patterns and temporal dependencies from transformed dense spatio-temporal tensors, benchmarked against LightGBM and LASSO regression.

Result: The proposed architecture provides significantly more accurate and spatially coherent forecasts than baseline models, establishing a new performance benchmark for deformation forecasting.

Conclusion: Spatio-temporal deep learning is effective for high-resolution deformation forecasting, with interpretability analysis showing baseline models default to simplistic patterns while the integrated approach captures complex deformation dynamics.

Abstract: Monitoring ground displacement is crucial for urban infrastructure stability and mitigating geological hazards. However, forecasting future deformation from sparse Interferometric Synthetic Aperture Radar (InSAR) time-series data remains a significant challenge. This paper introduces a novel deep learning framework that transforms these sparse point measurements into a dense spatio-temporal tensor. This methodological shift allows, for the first time, the direct application of advanced computer vision architectures to this forecasting problem. We design and implement a hybrid Convolutional Neural Network and Long-Short Term Memory (CNN-LSTM) model, specifically engineered to simultaneously learn spatial patterns and temporal dependencies from the generated data tensor. The model’s performance is benchmarked against powerful machine learning baselines, Light Gradient Boosting Machine and LASSO regression, using Sentinel-1 data from eastern Ireland. Results demonstrate that the proposed architecture provides significantly more accurate and spatially coherent forecasts, establishing a new performance benchmark for this task. Furthermore, an interpretability analysis reveals that baseline models often default to simplistic persistence patterns, highlighting the necessity of our integrated spatio-temporal approach to capture the complex dynamics of ground deformation. Our findings confirm the efficacy and potential of spatio-temporal deep learning for high-resolution deformation forecasting.

Method: A framework that generates datasets with large numbers of questions about individual concepts with wide linguistic variation to systematically probe model capabilities.

Result: Models are proficient in object recognition but struggle with positional information and constrained questions. MobileVLM-V2 showed significant answer disagreements, while others exhibited affirmative bias and difficulties with geometric shapes.

Conclusion: The Scrapbook framework provides a valuable tool for generating diverse datasets to systematically assess and improve AI model performance, revealing specific areas needing enhancement.

Abstract: In this paper, we present the Scrapbook framework, a novel methodology designed to generate extensive datasets for probing the learned concepts of artificial intelligence (AI) models. The framework focuses on fundamental concepts such as object recognition, absolute and relative positions, and attribute identification. By generating datasets with a large number of questions about individual concepts and a wide linguistic variation, the Scrapbook framework aims to validate the model’s understanding of these basic elements before tackling more complex tasks. Our experimental findings reveal that, while contemporary models demonstrate proficiency in recognizing and enumerating objects, they encounter challenges in comprehending positional information and addressing inquiries with additional constraints. Specifically, the MobileVLM-V2 model showed significant answer disagreements and plausible wrong answers, while other models exhibited a bias toward affirmative answers and struggled with questions involving geometric shapes and positional information, indicating areas for improvement in understanding and consistency. The proposed framework offers a valuable instrument for generating diverse and comprehensive datasets, which can be utilized to systematically assess and enhance the performance of AI models.

Method: Generated 150 image pairs through describe-then-generate pipeline and measured information preservation using LPIPS, SSIM, and color distance metrics across perceptual, structural, and chromatic dimensions.

Result: 99.3% of samples exhibited substantial perceptual degradation and 91.5% demonstrated significant structural information loss, showing consistent limitations in multimodal systems.

Conclusion: The describe-then-generate bottleneck represents a measurable and consistent limitation in contemporary multimodal systems, providing empirical evidence of substantial information loss.

Abstract: With the increasing integration of multimodal AI systems in creative workflows, understanding information loss in vision-language-vision pipelines has become important for evaluating system limitations. However, the degradation that occurs when visual content passes through textual intermediation remains poorly quantified. In this work, we provide empirical analysis of the describe-then-generate bottleneck, where natural language serves as an intermediate representation for visual information. We generated 150 image pairs through the describe-then-generate pipeline and applied existing metrics (LPIPS, SSIM, and color distance) to measure information preservation across perceptual, structural, and chromatic dimensions. Our evaluation reveals that 99.3% of samples exhibit substantial perceptual degradation and 91.5% demonstrate significant structural information loss, providing empirical evidence that the describe-then-generate bottleneck represents a measurable and consistent limitation in contemporary multimodal systems.

Method: Comparison of geospatial foundation models with shallow classifiers versus fine-tuned deep learning models for rooftop classification from high-resolution satellite imagery, with assessment of additional training data from neighboring regions.

Result: Best models achieved F1 scores of 0.88 for roof pitch classification and 0.83 for roof material classification.

Conclusion: The approach provides SIDS with novel AI and Earth Observation capabilities for evidence-based urban governance when combined with local capacity building.

Abstract: Detailed structural building information is used to estimate potential damage from hazard events like cyclones, floods, and landslides, making them critical for urban resilience planning and disaster risk reduction. However, such information is often unavailable in many small island developing states (SIDS) in climate-vulnerable regions like the Caribbean. To address this data gap, we present an AI-driven workflow to automatically infer rooftop attributes from high-resolution satellite imagery, with Saint Vincent and the Grenadines as our case study. Here, we compare the utility of geospatial foundation models combined with shallow classifiers against fine-tuned deep learning models for rooftop classification. Furthermore, we assess the impact of incorporating additional training data from neighboring SIDS to improve model performance. Our best models achieve F1 scores of 0.88 and 0.83 for roof pitch and roof material classification, respectively. Combined with local capacity building, our work aims to provide SIDS with novel capabilities to harness AI and Earth Observation (EO) data to enable more efficient, evidence-based urban governance.

Method: Developed VLA-LPAF module that is finetuned using images from a single view and fuses other multiview observations in the latent space. Implemented as RoboFlamingo-LPAF framework based on RoboFlamingo VLA model.

Result: RoboFlamingo-LPAF achieved average improvements of 8% task success rate on CALVIN, 15% on LIBERO, and 30% on a customized simulation benchmark. Demonstrated view-adaptive characteristics in real-world tasks.

Conclusion: VLA-LPAF effectively bridges the gap caused by perspective inconsistency in VLA models, enabling better generalization across different camera perspectives with efficient 2D data usage.

Abstract: The Visual-Language-Action (VLA) models can follow text instructions according to visual observations of the surrounding environment. This ability to map multimodal inputs to actions is derived from the training of the VLA model on extensive standard demonstrations. These visual observations captured by third-personal global and in-wrist local cameras are inevitably varied in number and perspective across different environments, resulting in significant differences in the visual features. This perspective heterogeneity constrains the generality of VLA models. In light of this, we first propose the lightweight module VLA-LPAF to foster the perspective adaptivity of VLA models using only 2D data. VLA-LPAF is finetuned using images from a single view and fuses other multiview observations in the latent space, which effectively and efficiently bridge the gap caused by perspective inconsistency. We instantiate our VLA-LPAF framework with the VLA model RoboFlamingo to construct RoboFlamingo-LPAF. Experiments show that RoboFlamingo-LPAF averagely achieves around 8% task success rate improvement on CALVIN, 15% on LIBERO, and 30% on a customized simulation benchmark. We also demonstrate the developed viewadaptive characteristics of the proposed RoboFlamingo-LPAF through real-world tasks.

Method: Proposes a patch-based training framework where the input image is directly fed into a network for self-reconstruction. Uses masking, patch shuffling, and Gaussian noise to prevent identity mapping. Employs perceptual loss based on inner-product similarity to capture structural information beyond pixel fidelity.

Result: Achieves 0.99 AUROC and 0.60 AUPRC on MVTec-AD, and 0.98 AUROC and 0.67 AUPRC on fabric dataset, outperforming state-of-the-art methods. The approach is robust to noise and missing pixels.

Conclusion: SSDnet provides an effective zero-shot anomaly detection solution that leverages deep image prior without requiring external training data, labels, or references, making it suitable for real-world scenarios where training data is unavailable.

Abstract: Anomaly detection in images is typically addressed by learning from collections of training data or relying on reference samples. In many real-world scenarios, however, such training data may be unavailable, and only the test image itself is provided. We address this zero-shot setting by proposing a single-image anomaly localization method that leverages the inductive bias of convolutional neural networks, inspired by Deep Image Prior (DIP). Our method is named Single Shot Decomposition Network (SSDnet). Our key assumption is that natural images often exhibit unified textures and patterns, and that anomalies manifest as localized deviations from these repetitive or stochastic patterns. To learn the deep image prior, we design a patch-based training framework where the input image is fed directly into the network for self-reconstruction, rather than mapping random noise to the image as done in DIP. To avoid the model simply learning an identity mapping, we apply masking, patch shuffling, and small Gaussian noise. In addition, we use a perceptual loss based on inner-product similarity to capture structure beyond pixel fidelity. Our approach needs no external training data, labels, or references, and remains robust in the presence of noise or missing pixels. SSDnet achieves 0.99 AUROC and 0.60 AUPRC on MVTec-AD and 0.98 AUROC and 0.67 AUPRC on the fabric dataset, outperforming state-of-the-art methods. The implementation code will be released at https://github.com/mehrdadmoradi124/SSDnet

Method: Proposes URNet with local-global refinement module to capture fine-grained details and global context, plus KL divergence-based uncertainty modeling for enhanced prediction reliability.

Result: Extensive experiments on DSEC dataset show URNet consistently outperforms state-of-the-art methods in both qualitative and quantitative evaluations.

Conclusion: The proposed URNet framework effectively addresses event-based stereo depth estimation with improved accuracy and reliability through uncertainty-aware refinement.

Abstract: Event cameras provide high temporal resolution, high dynamic range, and low latency, offering significant advantages over conventional frame-based cameras. In this work, we introduce an uncertainty-aware refinement network called URNet for event-based stereo depth estimation. Our approach features a local-global refinement module that effectively captures fine-grained local details and long-range global context. Additionally, we introduce a Kullback-Leibler (KL) divergence-based uncertainty modeling method to enhance prediction reliability. Extensive experiments on the DSEC dataset demonstrate that URNet consistently outperforms state-of-the-art (SOTA) methods in both qualitative and quantitative evaluations.

Method: Jointly models human and scene using 3D Gaussian Splatting with learnable semantic attributes. Human Gaussians undergo deformation while scene Gaussians remain static. Uses event-guided loss to match brightness changes between renderings with event streams for improved fidelity in fast-moving regions.

Result: Achieves state-of-the-art human-scene reconstruction on ZJU-MoCap-Blur and MMHPSD-Blur datasets, with significant gains in PSNR/SSIM and reduced LPIPS, particularly for high-speed subjects.

Conclusion: The framework effectively leverages event camera advantages for dynamic human reconstruction, eliminating need for external human masks and simplifying Gaussian set management while delivering superior performance in fast-motion scenarios.

Abstract: Reconstructing dynamic humans together with static scenes from monocular videos remains difficult, especially under fast motion, where RGB frames suffer from motion blur. Event cameras exhibit distinct advantages, e.g., microsecond temporal resolution, making them a superior sensing choice for dynamic human reconstruction. Accordingly, we present a novel event-guided human-scene reconstruction framework that jointly models human and scene from a single monocular event camera via 3D Gaussian Splatting. Specifically, a unified set of 3D Gaussians carries a learnable semantic attribute; only Gaussians classified as human undergo deformation for animation, while scene Gaussians stay static. To combat blur, we propose an event-guided loss that matches simulated brightness changes between consecutive renderings with the event stream, improving local fidelity in fast-moving regions. Our approach removes the need for external human masks and simplifies managing separate Gaussian sets. On two benchmark datasets, ZJU-MoCap-Blur and MMHPSD-Blur, it delivers state-of-the-art human-scene reconstruction, with notable gains over strong baselines in PSNR/SSIM and reduced LPIPS, especially for high-speed subjects.

Method: Two-phase hybrid AI framework: (A) deep learning model for automatic anatomical structure segmentation, and (B) symbolic spatial reasoning using fuzzy spatial relationships for tractography and nerve recognition, eliminating manual ROI selection.

Result: Applied to lumbosacral plexus in 10 endometriosis patients, Visionerves achieved up to 25% Dice score improvement over standard tractography and reduced spatial errors to less than 5 mm.

Conclusion: The framework provides automatic, reproducible nerve analysis for non-invasive diagnosis of endometriosis-related neuropathy and other nerve-involved conditions.

Abstract: Endometriosis often leads to chronic pelvic pain and possible nerve involvement, yet imaging the peripheral nerves remains a challenge. We introduce Visionerves, a novel hybrid AI framework for peripheral nervous system recognition from multi-gradient DWI and morphological MRI data. Unlike conventional tractography, Visionerves encodes anatomical knowledge through fuzzy spatial relationships, removing the need for selection of manual ROIs. The pipeline comprises two phases: (A) automatic segmentation of anatomical structures using a deep learning model, and (B) tractography and nerve recognition by symbolic spatial reasoning. Applied to the lumbosacral plexus in 10 women with (confirmed or suspected) endometriosis, Visionerves demonstrated substantial improvements over standard tractography, with Dice score improvements of up to 25% and spatial errors reduced to less than 5 mm. This automatic and reproducible approach enables detailed nerve analysis and paves the way for non-invasive diagnosis of endometriosis-related neuropathy, as well as other conditions with nerve involvement.

Method: The framework applies diffusion only to the low-resolution LL subband in the wavelet domain, while high-frequency subbands are refined with a lightweight network. It also uses an efficient online random masking strategy instead of the inefficient leave-one-out approach for training pair curation.

Result: Experiments on Mip-NeRF 360 and OmniObject3D datasets show WaveletGaussian achieves competitive rendering quality while substantially reducing training time compared to previous methods.

Conclusion: WaveletGaussian provides an efficient alternative for sparse-view 3DGS reconstruction by leveraging wavelet domain processing and optimized training strategies, achieving good performance with significantly reduced computational costs.

Abstract: 3D Gaussian Splatting (3DGS) has become a powerful representation for image-based object reconstruction, yet its performance drops sharply in sparse-view settings. Prior works address this limitation by employing diffusion models to repair corrupted renders, subsequently using them as pseudo ground truths for later optimization. While effective, such approaches incur heavy computation from the diffusion fine-tuning and repair steps. We present WaveletGaussian, a framework for more efficient sparse-view 3D Gaussian object reconstruction. Our key idea is to shift diffusion into the wavelet domain: diffusion is applied only to the low-resolution LL subband, while high-frequency subbands are refined with a lightweight network. We further propose an efficient online random masking strategy to curate training pairs for diffusion fine-tuning, replacing the commonly used, but inefficient, leave-one-out strategy. Experiments across two benchmark datasets, Mip-NeRF 360 and OmniObject3D, show WaveletGaussian achieves competitive rendering quality while substantially reducing training time.

Method: Data was collected using a custom Android application that captures high-frequency accelerometer, gyroscope, and GPS streams alongside continuous video, with precise time alignment for multimodal analysis. The dataset covers multiple road types including urban arterials, secondary roads, and motorways.

Result: V-SenseDrive provides a structured dataset with raw, processed, and semantic layers, enabling multimodal analysis of three target driving behaviors (normal, aggressive, risky) in the Pakistani driving environment.

Conclusion: This dataset fills a critical gap in global driver behavior datasets by representing real-world driving in Pakistan and provides groundwork for context-aware intelligent transportation solutions, ADAS development, and traffic safety analysis.

Abstract: Road traffic accidents remain a major public health challenge, particularly in countries with heterogeneous road conditions, mixed traffic flow, and variable driving discipline, such as Pakistan. Reliable detection of unsafe driving behaviours is a prerequisite for improving road safety, enabling advanced driver assistance systems (ADAS), and supporting data driven decisions in insurance and fleet management. Most of existing datasets originate from the developed countries with limited representation of the behavioural diversity observed in emerging economies and the driver’s face recording voilates the privacy preservation. We present V-SenseDrive, the first privacy-preserving multimodal driver behaviour dataset collected entirely within the Pakistani driving environment. V-SenseDrive combines smartphone based inertial and GPS sensor data with synchronized road facing video to record three target driving behaviours (normal, aggressive, and risky) on multiple types of roads, including urban arterials, secondary roads, and motorways. Data was gathered using a custom Android application designed to capture high frequency accelerometer, gyroscope, and GPS streams alongside continuous video, with all sources precisely time aligned to enable multimodal analysis. The focus of this work is on the data acquisition process, covering participant selection, driving scenarios, environmental considerations, and sensor video synchronization techniques. The dataset is structured into raw, processed, and semantic layers, ensuring adaptability for future research in driver behaviour classification, traffic safety analysis, and ADAS development. By representing real world driving in Pakistan, V-SenseDrive fills a critical gap in the global landscape of driver behaviour datasets and lays the groundwork for context aware intelligent transportation solutions.

Method: Multi-stage progressive training and high-precision data synthesis pipelines for domain enhancement, trained entirely on Baidu’s Kunlun P800 chips with over 90% scaling efficiency on 5000 chips.

Result: State-of-the-art performance on benchmarks including CCBench, SEEDBench IMG, ScienceQA, MMStar, OCRBench (873), DocVQA (94.75%), and MathVista (78.6%). Long chain-of-thought capabilities in 8B and 70B variants show superior mathematical reasoning and logical inference.

Conclusion: The work establishes an effective methodology for developing domain-enhanced multimodal models, demonstrating the capability of large-scale AI infrastructure to train SOTA-level multimodal models efficiently.

Abstract: We present Qianfan-VL, a series of multimodal large language models ranging from 3B to 70B parameters, achieving state-of-the-art performance through innovative domain enhancement techniques. Our approach employs multi-stage progressive training and high-precision data synthesis pipelines, which prove to be critical technologies for enhancing domain-specific capabilities while maintaining strong general performance. Qianfan-VL achieves comparable results to leading open-source models on general benchmarks, with state-of-the-art performance on benchmarks such as CCBench, SEEDBench IMG, ScienceQA, and MMStar. The domain enhancement strategy delivers significant advantages in OCR and document understanding, validated on both public benchmarks (OCRBench 873, DocVQA 94.75%) and in-house evaluations. Notably, Qianfan-VL-8B and 70B variants incorporate long chain-of-thought capabilities, demonstrating superior performance on mathematical reasoning (MathVista 78.6%) and logical inference tasks. All models are trained entirely on Baidu’s Kunlun P800 chips, validating the capability of large-scale AI infrastructure to train SOTA-level multimodal models with over 90% scaling efficiency on 5000 chips for a single task. This work establishes an effective methodology for developing domain-enhanced multimodal models suitable for diverse enterprise deployment scenarios.

Method: Proposes HazeFlow, an ODE-based framework that reformulates ASM as an ordinary differential equation inspired by Rectified Flow. Uses Markov Chain Brownian Motion for non-homogeneous haze generation to create realistic training data. Learns optimal ODE trajectory for hazy-to-clean image mapping with single inference step.

Result: Achieves state-of-the-art performance across various real-world dehazing benchmark datasets through extensive experiments.

Conclusion: HazeFlow effectively bridges the domain gap in real-world dehazing by combining physics-grounded learning with ODE-based optimization and realistic haze simulation, demonstrating superior performance over existing methods.

Abstract: Dehazing involves removing haze or fog from images to restore clarity and improve visibility by estimating atmospheric scattering effects. While deep learning methods show promise, the lack of paired real-world training data and the resulting domain gap hinder generalization to real-world scenarios. In this context, physics-grounded learning becomes crucial; however, traditional methods based on the Atmospheric Scattering Model (ASM) often fall short in handling real-world complexities and diverse haze patterns. To solve this problem, we propose HazeFlow, a novel ODE-based framework that reformulates ASM as an ordinary differential equation (ODE). Inspired by Rectified Flow (RF), HazeFlow learns an optimal ODE trajectory to map hazy images to clean ones, enhancing real-world dehazing performance with only a single inference step. Additionally, we introduce a non-homogeneous haze generation method using Markov Chain Brownian Motion (MCBM) to address the scarcity of paired real-world data. By simulating realistic haze patterns through MCBM, we enhance the adaptability of HazeFlow to diverse real-world scenarios. Through extensive experiments, we demonstrate that HazeFlow achieves state-of-the-art performance across various real-world dehazing benchmark datasets.

Method: Used structured channel pruning, quantization-aware training (QAT), and NVIDIA TensorRT acceleration on Jetson Orin Nano to compress EcoWeedNet, overcoming challenges from complex architecture elements like residual shortcuts, attention mechanisms, concatenations, and CSP blocks.

Result: Achieved 68.5% model size reduction, 3.2 GFLOPs computation reduction, and 184 FPS inference speed (28.7% faster than baseline). On CottonWeedDet12 dataset, pruned EcoWeedNet with 39.5% pruning ratio outperformed YOLO11n/YOLO12n with 83.7% precision, 77.5% recall, and 85.9% mAP50.

Conclusion: The compressed EcoWeedNet proves to be both efficient and effective for precision agriculture applications, demonstrating superior performance compared to existing models while being optimized for resource-constrained edge devices.

Abstract: Deploying deep learning models in agriculture is difficult because edge devices have limited resources, but this work presents a compressed version of EcoWeedNet using structured channel pruning, quantization-aware training (QAT), and acceleration with NVIDIA’s TensorRT on the Jetson Orin Nano. Despite the challenges of pruning complex architectures with residual shortcuts, attention mechanisms, concatenations, and CSP blocks, the model size was reduced by up to 68.5% and computations by 3.2 GFLOPs, while inference speed reached 184 FPS at FP16, 28.7% faster than the baseline. On the CottonWeedDet12 dataset, the pruned EcoWeedNet with a 39.5% pruning ratio outperformed YOLO11n and YOLO12n (with only 20% pruning), achieving 83.7% precision, 77.5% recall, and 85.9% mAP50, proving it to be both efficient and effective for precision agriculture.

Method: Proposes learnable modality tokens for missingness-aware fusion and augments unimodal contrastive objectives with fused multimodal representations. Uses enhanced modality dropout and contrastive learning.

Result: Achieves state-of-the-art performance on large-scale clinical datasets for disease detection and prediction, especially in challenging single-modality scenarios. Successfully integrates with CT foundation models.

Conclusion: The framework offers effective, efficient, and generalizable multimodal learning with significant potential for real-world clinical applications, providing a scalable low-cost solution.

Abstract: As medical diagnoses increasingly leverage multimodal data, machine learning models are expected to effectively fuse heterogeneous information while remaining robust to missing modalities. In this work, we propose a novel multimodal learning framework that integrates enhanced modalities dropout and contrastive learning to address real-world limitations such as modality imbalance and missingness. Our approach introduces learnable modality tokens for improving missingness-aware fusion of modalities and augments conventional unimodal contrastive objectives with fused multimodal representations. We validate our framework on large-scale clinical datasets for disease detection and prediction tasks, encompassing both visual and tabular modalities. Experimental results demonstrate that our method achieves state-of-the-art performance, particularly in challenging and practical scenarios where only a single modality is available. Furthermore, we show its adaptability through successful integration with a recent CT foundation model. Our findings highlight the effectiveness, efficiency, and generalizability of our approach for multimodal learning, offering a scalable, low-cost solution with significant potential for real-world clinical applications. The code is available at https://github.com/omron-sinicx/medical-modality-dropout.

Method: Used a densely annotated in-house dataset of 490 CTPA scans to evaluate 9 segmentation architectures (CNN and ViT families) with pretrained or random weights under unified testing framework.

Result: Best model achieved mean Dice score of 0.7131, detecting 181 emboli with 49 false positives and 28 false negatives. 3D U-Net with ResNet encoder performed best, and CNN models outperformed ViT models.

Conclusion: 3D models are well-suited for PE segmentation, classification-based pretraining can harm segmentation performance, and distal emboli remain challenging due to task complexity and dataset limitations.

Abstract: In this study, we curated a densely annotated in-house dataset comprising 490 CTPA scans. Using this dataset, we systematically evaluated nine widely used segmentation architectures from both the CNN and Vision Transformer (ViT) families, initialized with either pretrained or random weights, under a unified testing framework as a performance audit. Our study leads to several important observations: (1) 3D U-Net with a ResNet encoder remains a highly effective architecture for PE segmentation; (2) 3D models are particularly well-suited to this task given the morphological characteristics of emboli; (3) CNN-based models generally yield superior performance compared to their ViT-based counterparts in PE segmentation; (4) classification-based pretraining, even on large PE datasets, can adversely impact segmentation performance compared to training from scratch, suggesting that PE classification and segmentation may rely on different sets of discriminative features; (5) different model architectures show a highly consistent pattern of segmentation performance when trained on the same data; and (6) while central and large emboli can be segmented with satisfactory accuracy, distal emboli remain challenging due to both task complexity and the scarcity of high-quality datasets. Besides these findings, our best-performing model achieves a mean Dice score of 0.7131 for segmentation. It detects 181 emboli with 49 false positives and 28 false negatives from 60 in-house testing scans. Its generalizability is further validated on public datasets.

Method: Proposed Frequency Low Cut Pooling (FLC Pooling) - an alias-free downsampling operation in the frequency domain, and extended it to Aliasing and Sinc Artifact-free Pooling (ASAP) which removes both aliasing and sinc-interpolation artifacts.

Result: Experimental evaluation on ImageNet-1k, ImageNet-C and CIFAR datasets shows that networks using FLC Pooling and ASAP learn more stable features, demonstrating improved robustness against common corruptions and adversarial attacks while maintaining similar clean accuracy to baseline models.

Conclusion: The proposed alias-free downsampling methods effectively address the fundamental signal processing violation in CNNs, leading to more robust models without sacrificing performance, highlighting the importance of proper downsampling design in neural network architectures.

Abstract: Convolutional Neural Networks (CNNs) are successful in various computer vision tasks. From an image and signal processing point of view, this success is counter-intuitive, as the inherent spatial pyramid design of most CNNs is apparently violating basic signal processing laws, i.e. the Sampling Theorem in their downsampling operations. This issue has been broadly neglected until recent work in the context of adversarial attacks and distribution shifts showed that there is a strong correlation between the vulnerability of CNNs and aliasing artifacts induced by bandlimit-violating downsampling. As a remedy, we propose an alias-free downsampling operation in the frequency domain, denoted Frequency Low Cut Pooling (FLC Pooling) which we further extend to Aliasing and Sinc Artifact-free Pooling (ASAP). ASAP is alias-free and removes further artifacts from sinc-interpolation. Our experimental evaluation on ImageNet-1k, ImageNet-C and CIFAR datasets on various CNN architectures demonstrates that networks using FLC Pooling and ASAP as downsampling methods learn more stable features as measured by their robustness against common corruptions and adversarial attacks, while maintaining a clean accuracy similar to the respective baseline models.

Method: Novel graph neural network combining anatomically-informed graph structures with contrastive learning to address challenges in handshape recognition.

Result: Established first benchmark for structured handshape recognition, achieving 46% accuracy across 37 handshape classes (baseline methods achieved 25%).

Conclusion: The approach successfully improves handshape recognition by explicitly modeling hand configurations and separating temporal dynamics.

Abstract: Handshapes serve a fundamental phonological role in signed languages, with American Sign Language employing approximately 50 distinct shapes. However,computational approaches rarely model handshapes explicitly, limiting both recognition accuracy and linguistic analysis.We introduce a novel graph neural network that separates temporal dynamics from static handshape configurations. Our approach combines anatomically-informed graph structures with contrastive learning to address key challenges in handshape recognition, including subtle interclass distinctions and temporal variations. We establish the first benchmark for structured handshape recognition in signing sequences, achieving 46% accuracy across 37 handshape classes (with baseline methods achieving 25%).

Method: Proposes DWTGS framework using wavelet-space losses with spatial supervision. Specifically supervises only low-frequency LL subbands at multiple DWT levels while enforcing sparsity on high-frequency HH subbands in a self-supervised manner.

Result: Experiments show DWTGS consistently outperforms Fourier-based counterparts across benchmarks. The low-frequency-centric strategy improves generalization and reduces high-frequency hallucinations.

Conclusion: Wavelet-based frequency regularization is more effective than Fourier-based approaches for sparse-view 3D Gaussian Splatting, providing better spatial supervision and reducing overfitting to high-frequency details.

Abstract: Sparse-view 3D Gaussian Splatting (3DGS) presents significant challenges in reconstructing high-quality novel views, as it often overfits to the widely-varying high-frequency (HF) details of the sparse training views. While frequency regularization can be a promising approach, its typical reliance on Fourier transforms causes difficult parameter tuning and biases towards detrimental HF learning. We propose DWTGS, a framework that rethinks frequency regularization by leveraging wavelet-space losses that provide additional spatial supervision. Specifically, we supervise only the low-frequency (LF) LL subbands at multiple DWT levels, while enforcing sparsity on the HF HH subband in a self-supervised manner. Experiments across benchmarks show that DWTGS consistently outperforms Fourier-based counterparts, as this LF-centric strategy improves generalization and reduces HF hallucinations.

Method: Conducted experiments with eight uncertainty quantification methods across four classification tasks to evaluate OOD detection performance under different ID-OOD criteria (image characteristic shift vs. anatomical feature shift).

Result: OOD detection performance significantly varies with the classification task, and the best task depends on whether OOD samples result from image characteristic shifts or anatomical feature shifts. Superior OOD detection doesn’t guarantee optimal abstained prediction.

Conclusion: Task selection and uncertainty strategies must be aligned with specific downstream applications in medical image analysis, as different OOD scenarios require different approaches for effective detection and safe model deployment.

Abstract: Reliable out-of-distribution (OOD) detection is important for safe deployment of deep learning models in fetal ultrasound amidst heterogeneous image characteristics and clinical settings. OOD detection relies on estimating a classification model’s uncertainty, which should increase for OOD samples. While existing research has largely focused on uncertainty quantification methods, this work investigates the impact of the classification task itself. Through experiments with eight uncertainty quantification methods across four classification tasks, we demonstrate that OOD detection performance significantly varies with the task, and that the best task depends on the defined ID-OOD criteria; specifically, whether the OOD sample is due to: i) an image characteristic shift or ii) an anatomical feature shift. Furthermore, we reveal that superior OOD detection does not guarantee optimal abstained prediction, underscoring the necessity to align task selection and uncertainty strategies with the specific downstream application in medical image analysis.

Method: Three-step approach: 1) establishing reliable plane correspondence, 2) building pseudo-plane-constrained Gauss-Helmert model, 3) adaptively estimating vertical translation.

Result: Experiments on three real-world datasets show L2M-Reg is more accurate and computationally efficient than existing ICP-based and plane-based methods.

Conclusion: L2M-Reg provides a novel building-level solution for LiDAR-to-Model registration when model uncertainty is present.

Abstract: Accurate registration between LiDAR (Light Detection and Ranging) point clouds and semantic 3D city models is a fundamental topic in urban digital twinning and a prerequisite for downstream tasks, such as digital construction, change detection and model refinement. However, achieving accurate LiDAR-to-Model registration at individual building level remains challenging, particularly due to the generalization uncertainty in semantic 3D city models at the Level of Detail 2 (LoD2). This paper addresses this gap by proposing L2M-Reg, a plane-based fine registration method that explicitly accounts for model uncertainty. L2M-Reg consists of three key steps: establishing reliable plane correspondence, building a pseudo-plane-constrained Gauss-Helmert model, and adaptively estimating vertical translation. Experiments on three real-world datasets demonstrate that L2M-Reg is both more accurate and computationally efficient than existing ICP-based and plane-based methods. Overall, L2M-Reg provides a novel building-level solution regarding LiDAR-to-Model registration when model uncertainty is present.

Method: Created OrthoLoC dataset with 16,425 UAV images from Germany and US with multiple modalities. The dataset enables decoupled evaluation of image retrieval and feature matching. Also introduced AdHoP refinement technique that can be integrated with any feature matcher.

Result: Comprehensive evaluation examined domain shifts, data resolutions, and covisibility on localization accuracy. AdHoP improved matching by up to 95% and reduced translation error by up to 63%.

Conclusion: OrthoLoC provides the first large-scale benchmark for orthographic-based visual localization, demonstrating that lightweight geodata can effectively replace heavy 3D models while achieving high localization accuracy.

Abstract: Accurate visual localization from aerial views is a fundamental problem with applications in mapping, large-area inspection, and search-and-rescue operations. In many scenarios, these systems require high-precision localization while operating with limited resources (e.g., no internet connection or GNSS/GPS support), making large image databases or heavy 3D models impractical. Surprisingly, little attention has been given to leveraging orthographic geodata as an alternative paradigm, which is lightweight and increasingly available through free releases by governmental authorities (e.g., the European Union). To fill this gap, we propose OrthoLoC, the first large-scale dataset comprising 16,425 UAV images from Germany and the United States with multiple modalities. The dataset addresses domain shifts between UAV imagery and geospatial data. Its paired structure enables fair benchmarking of existing solutions by decoupling image retrieval from feature matching, allowing isolated evaluation of localization and calibration performance. Through comprehensive evaluation, we examine the impact of domain shifts, data resolutions, and covisibility on localization accuracy. Finally, we introduce a refinement technique called AdHoP, which can be integrated with any feature matcher, improving matching by up to 95% and reducing translation error by up to 63%. The dataset and code are available at: https://deepscenario.github.io/OrthoLoC.

Method: Compute-aware pipeline using LaBSE-verified EN-BN pairs and 110k synthetic images. Combines frozen MaxViT for visual patches, Bengali-native mBART-50 decoder, and lightweight bridge. Core innovation is tri-loss: Patch-Alignment Loss (PAL) for patch descriptor alignment, InfoNCE for global separation, and Sinkhorn-based OT for fine-grained patch correspondence.

Result: Significant improvements on Flickr30k-1k (BLEU-4 12.29, METEOR 27.98, BERTScore-F1 71.20) and MSCOCO-1k (BLEU-4 12.00, METEOR 28.14, BERTScore-F1 75.40), outperforming CE baselines and reducing real-synthetic centroid gap by 41%.

Conclusion: The PAL+InfoNCE+OT synergy effectively improves grounding in low-resource languages by better aligning visual and linguistic modalities while minimizing spurious correlations, demonstrating strong performance gains over conventional approaches.

Abstract: Grounding vision–language models in low-resource languages remains challenging, as they often produce fluent text about the wrong objects. This stems from scarce paired data, translation pivots that break alignment, and English-centric pretraining that ignores target-language semantics. We address this with a compute-aware Bengali captioning pipeline trained on LaBSE-verified EN–BN pairs and 110k bilingual-prompted synthetic images. A frozen MaxViT yields stable visual patches, a Bengali-native mBART-50 decodes, and a lightweight bridge links the modalities. Our core novelty is a tri-loss objective: Patch-Alignment Loss (PAL) aligns real and synthetic patch descriptors using decoder cross-attention, InfoNCE enforces global real–synthetic separation, and Sinkhorn-based OT ensures balanced fine-grained patch correspondence. This PAL+InfoNCE+OT synergy improves grounding, reduces spurious matches, and drives strong gains on Flickr30k-1k (BLEU-4 12.29, METEOR 27.98, BERTScore-F1 71.20) and MSCOCO-1k (BLEU-4 12.00, METEOR 28.14, BERTScore-F1 75.40), outperforming strong CE baselines and narrowing the real–synthetic centroid gap by 41%.

Method: A model-agnostic, multi-stage distillation strategy combining feature-level, output-level, and adaptive region-aware supervision to transfer knowledge from a large planning-oriented teacher (UniAD) to a lightweight BEV representation.

Result: Achieves 39.0 mAP for detection, 1.08 minADE for motion forecasting, and 0.32 collision rate on nuScenes, running at 11 FPS (5x faster than baseline) with only 28M parameters (78% reduction over UniAD).

Conclusion: Full-stack driving intelligence can be effectively retained in resource-constrained settings through distillation, enabling real-time camera-only autonomous driving systems.

Abstract: We present TinyBEV, a unified, camera only Bird’s Eye View (BEV) framework that distills the full-stack capabilities of a large planning-oriented teacher (UniAD [19]) into a compact, real-time student model. Unlike prior efficient camera only baselines such as VAD[23] and VADv2[7], TinyBEV supports the complete autonomy stack 3D detection, HD-map segmentation, motion forecasting, occupancy prediction, and goal-directed planning within a streamlined 28M-parameter backbone, achieving a 78% reduction in parameters over UniAD [19]. Our model-agnostic, multi-stage distillation strategy combines feature-level, output-level, and adaptive region-aware supervision to effectively transfer high-capacity multi-modal knowledge to a lightweight BEV representation. On nuScenes[4], Tiny-BEV achieves 39.0 mAP for detection, 1.08 minADE for motion forecasting, and a 0.32 collision rate, while running 5x faster (11 FPS) and requiring only camera input. These results demonstrate that full-stack driving intelligence can be retained in resource-constrained settings, bridging the gap between large-scale, multi-modal perception-planning models and deployment-ready real-time autonomy.

Method: The paper introduces BlurBall, a model that jointly estimates ball position and motion blur attributes using attention mechanisms like Squeeze-and-Excitation over multi-frame inputs. A new labeling convention places the ball at the center of blur streaks with explicit blur annotations.

Result: The new labeling approach consistently enhances detection performance across various models. BlurBall achieves state-of-the-art results in ball detection by leveraging blur information.

Conclusion: Leveraging motion blur not only improves detection accuracy but also enables more reliable trajectory prediction, benefiting real-time sports analytics. The center-based labeling with blur attributes provides better motion cues for detection systems.

Abstract: Motion blur reduces the clarity of fast-moving objects, posing challenges for detection systems, especially in racket sports, where balls often appear as streaks rather than distinct points. Existing labeling conventions mark the ball at the leading edge of the blur, introducing asymmetry and ignoring valuable motion cues correlated with velocity. This paper introduces a new labeling strategy that places the ball at the center of the blur streak and explicitly annotates blur attributes. Using this convention, we release a new table tennis ball detection dataset. We demonstrate that this labeling approach consistently enhances detection performance across various models. Furthermore, we introduce BlurBall, a model that jointly estimates ball position and motion blur attributes. By incorporating attention mechanisms such as Squeeze-and-Excitation over multi-frame inputs, we achieve state-of-the-art results in ball detection. Leveraging blur not only improves detection accuracy but also enables more reliable trajectory prediction, benefiting real-time sports analytics.

Method: Invokes OWLv2 only on fixed-interval keyframes and propagates detections to intermediate frames using compressed-domain motion vectors with 3x3 grid aggregation, area-growth check, and optional single-class switch.

Result: On ILSVRC2015-VID: mAP@0.5=0.609, mAP@[0.5:0.95]=0.316. At loose IoU thresholds, it remains close to framewise OWLv2-Large (0.747/0.721 vs 0.784/0.780 at 0.2/0.3 IoU). Outperforms tracker-based propagation methods.

Conclusion: Compressed-domain propagation is a practical way to reduce detector invocations while maintaining strong zero-shot coverage in videos without requiring labeled training data.

Abstract: Running a large open-vocabulary (Open-vocab) detector on every video frame is accurate but expensive. We introduce a training-free pipeline that invokes OWLv2 only on fixed-interval keyframes and propagates detections to intermediate frames using compressed-domain motion vectors (MV). A simple 3x3 grid aggregation of motion vectors provides translation and uniform-scale updates, augmented with an area-growth check and an optional single-class switch. The method requires no labels, no fine-tuning, and uses the same prompt list for all open-vocabulary methods. On ILSVRC2015-VID (validation dataset), our approach (MVP) attains mAP@0.5=0.609 and mAP@[0.5:0.95]=0.316. At loose intersection-over-union (IoU) thresholds it remains close to framewise OWLv2-Large (0.747/0.721 at 0.2/0.3 versus 0.784/0.780), reflecting that coarse localization is largely preserved. Under the same keyframe schedule, MVP outperforms tracker-based propagation (MOSSE, KCF, CSRT) at mAP@0.5. A supervised reference (YOLOv12x) reaches 0.631 at mAP@0.5 but requires labeled training, whereas our method remains label-free and open-vocabulary. These results indicate that compressed-domain propagation is a practical way to reduce detector invocations while keeping strong zero-shot coverage in videos. Our code and models are available at https://github.com/microa/MVP.

Method: The study systematically evaluates several adaptation strategies using a novel HDR dataset with diverse lighting colors. Instead of proposing new algorithms, it explores whether simple preprocessing techniques can enhance existing models’ color accuracy.

Result: Preprocessing input images with a pre-trained white balance network significantly improves color robustness, outperforming other strategies across all tested scenarios. This approach requires no retraining of the lighting estimation model.

Conclusion: The white balance preprocessing technique demonstrates generality by successfully improving color accuracy across three state-of-the-art lighting estimation methods, providing a simple yet effective solution for enhancing color robustness in HDR lighting estimation.

Abstract: Advances in high dynamic range (HDR) lighting estimation from a single image have opened new possibilities for augmented reality (AR) applications. Predicting complex lighting environments from a single input image allows for the realistic rendering and compositing of virtual objects. In this work, we investigate the color robustness of such methods – an often overlooked yet critical factor for achieving visual realism. While most evaluations conflate color with other lighting attributes (e.g., intensity, direction), we isolate color as the primary variable of interest. Rather than introducing a new lighting estimation algorithm, we explore whether simple adaptation techniques can enhance the color accuracy of existing models. Using a novel HDR dataset featuring diverse lighting colors, we systematically evaluate several adaptation strategies. Our results show that preprocessing the input image with a pre-trained white balance network improves color robustness, outperforming other strategies across all tested scenarios. Notably, this approach requires no retraining of the lighting estimation model. We further validate the generality of this finding by applying the technique to three state-of-the-art lighting estimation methods from recent literature.

Method: Leverages vision language model (VLM) with multimodal large language model (MLLM) for zero-shot detection of check components without training data.

Result: Strong performance on 110 diverse checks, demonstrating generalization across multiple formats and layouts.

Conclusion: The framework enables deployment in real-world financial settings and can bootstrap high-quality labeled datasets for specialized real-time detection models.

Abstract: Checks remain a foundational instrument in the financial ecosystem, facilitating substantial transaction volumes across institutions. However, their continued use also renders them a persistent target for fraud, underscoring the importance of robust check fraud detection mechanisms. At the core of such systems lies the accurate identification and localization of critical fields, such as the signature, magnetic ink character recognition (MICR) line, courtesy amount, legal amount, payee, and payer, which are essential for subsequent verification against reference checks belonging to the same customer. This field-level detection is traditionally dependent on object detection models trained on large, diverse, and meticulously labeled datasets, a resource that is scarce due to proprietary and privacy concerns. In this paper, we introduce a novel, training-free framework for automated check field detection, leveraging the power of a vision language model (VLM) in conjunction with a multimodal large language model (MLLM). Our approach enables zero-shot detection of check components, significantly lowering the barrier to deployment in real-world financial settings. Quantitative evaluation of our model on a hand-curated dataset of 110 checks spanning multiple formats and layouts demonstrates strong performance and generalization capability. Furthermore, this framework can serve as a bootstrap mechanism for generating high-quality labeled datasets, enabling the development of specialized real-time object detection models tailored to institutional needs.

Method: Evaluated ChatGPT 4o, Claude Sonnet 4, and Gemini 2.5 Pro on chart understanding under corruption/occlusion. Introduced CHART NOISe dataset combining chart corruptions, occlusions, and multiple-choice questions with prompt reverse inconsistency testing.

Result: Sharp performance drops under corruption/occlusion with increased hallucinations (value fabrication, trend misinterpretation, entity confusion). Models remain overconfident in degraded settings, generating plausible but unsupported explanations.

Conclusion: Established a rigorous testbed for advancing robustness in chart understanding. Proposed baseline mitigation strategies like quality filtering and occlusion detection to address systematic vulnerabilities in VLMs.

Abstract: Vision language models (VLMs) show strong results on chart understanding, yet existing benchmarks assume clean figures and fact based queries. Real world charts often contain distortions and demand reasoning beyond simple matching. We evaluate ChatGPT 4o, Claude Sonnet 4, and Gemini 2.5 Pro, finding sharp performance drops under corruption or occlusion, with hallucinations such as value fabrication, trend misinterpretation, and entity confusion becoming more frequent. Models remain overconfident in degraded settings, generating plausible but unsupported explanations. To address this gap, we introduce CHART NOISe(Chart Hallucinations, Answers, and Reasoning Testing on Noisy and Occluded Input Selections), a dataset combining chart corruptions, occlusions, and exam style multiple choice questions inspired by Korea’s CSAT English section. A key innovation is prompt reverse inconsistency, where models contradict themselves when asked to confirm versus deny the same statement. Our contributions are threefold: (1) benchmarking state of the art VLMs, exposing systematic vulnerabilities in chart reasoning; (2) releasing CHART NOISe, the first dataset unifying corruption, occlusion, and reverse inconsistency; and (3) proposing baseline mitigation strategies such as quality filtering and occlusion detection. Together, these efforts establish a rigorous testbed for advancing robustness and reliability in chart understanding.

Method: Proposes a template- and phase-free neural representation framework with two synergistic networks: SAN encodes continuous 3D anatomical representation, while TMN produces deformation fields guided by respiratory signals from Transformers. The method treats respiratory motion as smooth continuous deformation steered by 1D surrogate signals.

Result: Evaluation on 19 volunteers shows the method accurately captures regular and irregular respiratory patterns while preserving anatomical fidelity. Reduces processing time from ~5 hours to 15 minutes training, with inference of each 3D volume in under 1 second. Achieves superior performance compared to conventional methods.

Conclusion: The framework enables accurate 3D image reconstruction at any respiratory state and demonstrates strong potential for 4D radiation therapy planning and real-time adaptive treatment, offering significant efficiency improvements over traditional approaches.

Abstract: Four-dimensional MRI (4D-MRI) is an promising technique for capturing respiratory-induced motion in radiation therapy planning and delivery. Conventional 4D reconstruction methods, which typically rely on phase binning or separate template scans, struggle to capture temporal variability, complicate workflows, and impose heavy computational loads. We introduce a neural representation framework that considers respiratory motion as a smooth, continuous deformation steered by a 1D surrogate signal, completely replacing the conventional discrete sorting approach. The new method fuses motion modeling with image reconstruction through two synergistic networks: the Spatial Anatomy Network (SAN) encodes a continuous 3D anatomical representation, while a Temporal Motion Network (TMN), guided by Transformer-derived respiratory signals, produces temporally consistent deformation fields. Evaluation using a free-breathing dataset of 19 volunteers demonstrates that our template- and phase-free method accurately captures both regular and irregular respiratory patterns, while preserving vessel and bronchial continuity with high anatomical fidelity. The proposed method significantly improves efficiency, reducing the total processing time from approximately five hours required by conventional discrete sorting methods to just 15 minutes of training. Furthermore, it enables inference of each 3D volume in under one second. The framework accurately reconstructs 3D images at any respiratory state, achieves superior performance compared to conventional methods, and demonstrates strong potential for application in 4D radiation therapy planning and real-time adaptive treatment.

Method: Used a custom dataset of 10,000 annotated racquetball frames at 720p-1280p resolution to evaluate five state-of-the-art Kalman filter-based tracking methods, analyzing inference speed and update frequency per image across four distinct scenarios.

Result: DeepOCSORT achieved lowest tracking error (ADE: 31.15 pixels) while ByteTrack was fastest (26.6ms inference time), but all trackers showed significant drift with spatial errors of 3-11cm (ADE: 31-114 pixels), 3-4x higher than standard benchmarks.

Conclusion: Current Kalman filter-based trackers have fundamental limitations in handling unpredictable motion patterns of fast-moving tiny objects, requiring specialized methodologies for such applications as error rates are substantially higher than standard tracking benchmarks.

Abstract: Unpredictable movement patterns and small visual mark make precise tracking of fast-moving tiny objects like a racquetball one of the challenging problems in computer vision. This challenge is particularly relevant for sport robotics applications, where lightweight and accurate tracking systems can improve robot perception and planning capabilities. While Kalman filter-based tracking methods have shown success in general object tracking scenarios, their performance degrades substantially when dealing with rapidly moving objects that exhibit irregular bouncing behavior. In this study, we evaluate the performance of five state-of-the-art Kalman filter-based tracking methods-OCSORT, DeepOCSORT, ByteTrack, BoTSORT, and StrongSORT-using a custom dataset containing 10,000 annotated racquetball frames captured at 720p-1280p resolution. We focus our analysis on two critical performance factors: inference speed and update frequency per image, examining how these parameters affect tracking accuracy and reliability for fast-moving tiny objects. Our experimental evaluation across four distinct scenarios reveals that DeepOCSORT achieves the lowest tracking error with an average ADE of 31.15 pixels compared to ByteTrack’s 114.3 pixels, while ByteTrack demonstrates the fastest processing at 26.6ms average inference time versus DeepOCSORT’s 26.8ms. However, our results show that all Kalman filter-based trackers exhibit significant tracking drift with spatial errors ranging from 3-11cm (ADE values: 31-114 pixels), indicating fundamental limitations in handling the unpredictable motion patterns of fast-moving tiny objects like racquetballs. Our analysis demonstrates that current tracking approaches require substantial improvements, with error rates 3-4x higher than standard object tracking benchmarks, highlighting the need for specialized methodologies for fast-moving tiny object tracking applications.

Method: Uses motion vectors from H.264 video to locate motion-dense regions, with a pipeline including denoising & merge, Monte Carlo sampling, and adaptive cropping via motion-density submatrix search to produce clip-level crops applied to I-frames.

Result: Improves accuracy or reduces compute on UCF101: +3.5% Top-1 accuracy at equal FLOPs or +2.4% Top-1 accuracy with 26.5% fewer FLOPs. Applied to CoViAR, achieves 89.2% Top-1 accuracy at original cost and 88.5% with reduced compute from 11.6 to 8.5 GFLOPs.

Conclusion: MoCrop demonstrates strong generality across diverse backbones, is training-free, parameter-free, and practical for real-time deployment in compressed domain video action recognition.

Abstract: We introduce MoCrop, a motion-aware adaptive cropping module for efficient video action recognition in the compressed domain. MoCrop uses motion vectors that are available in H.264 video to locate motion-dense regions and produces a single clip-level crop that is applied to all I-frames at inference. The module is training free, adds no parameters, and can be plugged into diverse backbones. A lightweight pipeline that includes denoising & merge (DM), Monte Carlo sampling (MCS), and adaptive cropping (AC) via a motion-density submatrix search yields robust crops with negligible overhead. On UCF101, MoCrop improves accuracy or reduces compute. With ResNet-50, it delivers +3.5% Top-1 accuracy at equal FLOPs (attention setting), or +2.4% Top-1 accuracy with 26.5% fewer FLOPs (efficiency setting). Applied to CoViAR, it reaches 89.2% Top-1 accuracy at the original cost and 88.5% Top-1 accuracy while reducing compute from 11.6 to 8.5 GFLOPs. Consistent gains on MobileNet-V3, EfficientNet-B1, and Swin-B indicate strong generality and make MoCrop practical for real-time deployment in the compressed domain. Our code and models are available at https://github.com/microa/MoCrop.

Method: The proposed CAFC-SE framework uses Vector Quantization (VQ) to map continuous visual features to discrete indices via a codebook, selectively transmitting them to the cloud. This preserves more informative visual patterns by projecting feature vectors onto nearest visual primitives.

Result: Extensive experiments demonstrate that CAFC-SE achieves superior performance in terms of both rate (bitrate efficiency) and accuracy (analysis performance) compared to existing methods.

Conclusion: The CAFC-SE framework is less vulnerable to low-bitrate conditions and provides an effective solution for coding images for machines with minimal bitrate while maintaining strong analysis performance in edge-cloud systems.

Abstract: Coding images for machines with minimal bitrate and strong analysis performance is key to effective edge-cloud systems. Several approaches deploy an image codec and perform analysis on the reconstructed image. Other methods compress intermediate features using entropy models and subsequently perform analysis on the decoded features. Nevertheless, these methods both perform poorly under low-bitrate conditions, as they retain many redundant details or learn over-concentrated symbol distributions. In this paper, we propose a Codebook-based Adaptive Feature Compression framework with Semantic Enhancement, named CAFC-SE. It maps continuous visual features to discrete indices with a codebook at the edge via Vector Quantization (VQ) and selectively transmits them to the cloud. The VQ operation that projects feature vectors onto the nearest visual primitives enables us to preserve more informative visual patterns under low-bitrate conditions. Hence, CAFC-SE is less vulnerable to low-bitrate conditions. Extensive experiments demonstrate the superiority of our method in terms of rate and accuracy.

Method: Uses multi-kernel depth-wise convolution blocks (MKDC) to process images through multiple kernels and capture multi-resolution spatial relationships, combined with sophisticated attention mechanisms including channel, spatial, and grouped gated attention.

Result: MK-UNet achieves higher accuracy than SOTA methods across six binary medical imaging benchmarks with only 0.316M parameters and 0.314G FLOPs, outperforming TransUNet with 333× fewer parameters and UNeXt with 4.7× fewer parameters while improving DICE scores up to 6.7%.

Conclusion: MK-UNet represents a paradigm shift in lightweight medical image segmentation, offering unparalleled performance for real-time diagnostics in resource-constrained environments, positioning it as an ideal solution for point-of-care applications.

Abstract: In this paper, we introduce MK-UNet, a paradigm shift towards ultra-lightweight, multi-kernel U-shaped CNNs tailored for medical image segmentation. Central to MK-UNet is the multi-kernel depth-wise convolution block (MKDC) we design to adeptly process images through multiple kernels, while capturing complex multi-resolution spatial relationships. MK-UNet also emphasizes the images salient features through sophisticated attention mechanisms, including channel, spatial, and grouped gated attention. Our MK-UNet network, with a modest computational footprint of only 0.316M parameters and 0.314G FLOPs, represents not only a remarkably lightweight, but also significantly improved segmentation solution that provides higher accuracy over state-of-the-art (SOTA) methods across six binary medical imaging benchmarks. Specifically, MK-UNet outperforms TransUNet in DICE score with nearly 333$\times$ and 123$\times$ fewer parameters and FLOPs, respectively. Similarly, when compared against UNeXt, MK-UNet exhibits superior segmentation performance, improving the DICE score up to 6.7% margins while operating with 4.7$\times$ fewer #Params. Our MK-UNet also outperforms other recent lightweight networks, such as MedT, CMUNeXt, EGE-UNet, and Rolling-UNet, with much lower computational resources. This leap in performance, coupled with drastic computational gains, positions MK-UNet as an unparalleled solution for real-time, high-fidelity medical diagnostics in resource-limited settings, such as point-of-care devices. Our implementation is available at https://github.com/SLDGroup/MK-UNet.

Method: Rigs 3D Gaussians to a CT mesh and performs joint optimization of Gaussian parameters and mesh deformation through photometric supervision. Each Gaussian is parametrized relative to its parent mesh triangle to enforce alignment.

Result: Demonstrated effectiveness on visceral pig surgeries and synthetic human liver data, showing sensible deformations of preoperative CT on monocular RGB data.

Conclusion: BridgeSplat successfully enables deformable surgical navigation by coupling real-time 3D reconstruction with preoperative CT data, with potential applications in surgical guidance and intraoperative planning.

Abstract: We introduce BridgeSplat, a novel approach for deformable surgical navigation that couples intraoperative 3D reconstruction with preoperative CT data to bridge the gap between surgical video and volumetric patient data. Our method rigs 3D Gaussians to a CT mesh, enabling joint optimization of Gaussian parameters and mesh deformation through photometric supervision. By parametrizing each Gaussian relative to its parent mesh triangle, we enforce alignment between Gaussians and mesh and obtain deformations that can be propagated back to update the CT. We demonstrate BridgeSplat’s effectiveness on visceral pig surgeries and synthetic data of a human liver under simulation, showing sensible deformations of the preoperative CT on monocular RGB data. Code, data, and additional resources can be found at https://maxfehrentz.github.io/ct-informed-splatting/ .

Method: Uses confidence filtering and super-resolution to get high-quality seed labels, then applies diffusion models to propagate these seeds to generate complete, high-quality pseudo-labels while maintaining quality.

Result: DGLE avoids direct optimization of noisy pseudo-label sets, significantly improves pseudo-label quality, and enhances model performance on target domain data.

Conclusion: The diffusion-guided approach effectively addresses pseudo-label noise issues in source-free domain adaptation, providing a more robust framework for semantic segmentation of remote sensing images.

Abstract: Research on unsupervised domain adaptation (UDA) for semantic segmentation of remote sensing images has been extensively conducted. However, research on how to achieve domain adaptation in practical scenarios where source domain data is inaccessible namely, source-free domain adaptation (SFDA) remains limited. Self-training has been widely used in SFDA, which requires obtaining as many high-quality pseudo-labels as possible to train models on target domain data. Most existing methods optimize the entire pseudo-label set to obtain more supervisory information. However, as pseudo-label sets often contain substantial noise, simultaneously optimizing all labels is challenging. This limitation undermines the effectiveness of optimization approaches and thus restricts the performance of self-training. To address this, we propose a novel pseudo-label optimization framework called Diffusion-Guided Label Enrichment (DGLE), which starts from a few easily obtained high-quality pseudo-labels and propagates them to a complete set of pseudo-labels while ensuring the quality of newly generated labels. Firstly, a pseudo-label fusion method based on confidence filtering and super-resolution enhancement is proposed, which utilizes cross-validation of details and contextual information to obtain a small number of high-quality pseudo-labels as initial seeds. Then, we leverage the diffusion model to propagate incomplete seed pseudo-labels with irregular distributions due to its strong denoising capability for randomly distributed noise and powerful modeling capacity for complex distributions, thereby generating complete and high-quality pseudo-labels. This method effectively avoids the difficulty of directly optimizing the complete set of pseudo-labels, significantly improves the quality of pseudo-labels, and thus enhances the model’s performance in the target domain.

Method: The method embeds the feature extractor into hyperbolic space using the Poincaré ball model, introduces hyperbolic contrastive loss and fully-connected layers, and implements maximum entropy distribution in hyperbolic space for feature augmentation to mitigate overfitting in few-shot scenarios.

Result: Experiments on C2FSCIL benchmarks demonstrate that the proposed method effectively improves both coarse and fine class accuracies compared to conventional approaches.

Conclusion: Hyperbolic space embedding significantly enhances the performance of coarse-to-fine few-shot class-incremental learning by better capturing hierarchical relationships and addressing data scarcity through effective feature augmentation.

Abstract: In the field of machine learning, hyperbolic space demonstrates superior representation capabilities for hierarchical data compared to conventional Euclidean space. This work focuses on the Coarse-To-Fine Few-Shot Class-Incremental Learning (C2FSCIL) task. Our study follows the Knowe approach, which contrastively learns coarse class labels and subsequently normalizes and freezes the classifier weights of learned fine classes in the embedding space. To better interpret the “coarse-to-fine” paradigm, we propose embedding the feature extractor into hyperbolic space. Specifically, we employ the Poincar'e ball model of hyperbolic space, enabling the feature extractor to transform input images into feature vectors within the Poincar'e ball instead of Euclidean space. We further introduce hyperbolic contrastive loss and hyperbolic fully-connected layers to facilitate model optimization and classification in hyperbolic space. Additionally, to enhance performance under few-shot conditions, we implement maximum entropy distribution in hyperbolic space to estimate the probability distribution of fine-class feature vectors. This allows generation of augmented features from the distribution to mitigate overfitting during training with limited samples. Experiments on C2FSCIL benchmarks show that our method effectively improves both coarse and fine class accuracies.

Method: Two-stage framework: (1) Geometry removal using strictly mask-aligned supervision on depth/geometry data with preference-driven objective, (2) Photorealistic RGB rendering conditioned on updated geometry where causal effects are implicitly considered.

Result: Extensive experiments show state-of-the-art performance in removing both objects and associated artifacts on two popular benchmarks.

Conclusion: The proposed geometry-aware approach effectively addresses limitations of appearance-based methods by leveraging geometric constraints and implicit consideration of causal visual effects through 3D geometry modification.

Abstract: Towards intelligent image editing, object removal should eliminate both the target object and its causal visual artifacts, such as shadows and reflections. However, existing image appearance-based methods either follow strictly mask-aligned training and fail to remove these causal effects which are not explicitly masked, or adopt loosely mask-aligned strategies that lack controllability and may unintentionally over-erase other objects. We identify that these limitations stem from ignoring the causal relationship between an object’s geometry presence and its visual effects. To address this limitation, we propose a geometry-aware two-stage framework that decouples object removal into (1) geometry removal and (2) appearance rendering. In the first stage, we remove the object directly from the geometry (e.g., depth) using strictly mask-aligned supervision, enabling structure-aware editing with strong geometric constraints. In the second stage, we render a photorealistic RGB image conditioned on the updated geometry, where causal visual effects are considered implicitly as a result of the modified 3D geometry. To guide learning in the geometry removal stage, we introduce a preference-driven objective based on positive and negative sample pairs, encouraging the model to remove objects as well as their causal visual artifacts while avoiding new structural insertions. Extensive experiments demonstrate that our method achieves state-of-the-art performance in removing both objects and their associated artifacts on two popular benchmarks. The code is available at https://github.com/buxiangzhiren/GeoRemover.

Method: Proposes SEGA which approximates target model gradients by applying Gaussian smoothing to source models and ensembling their smoothed gradients, with a perturbation filter mask to ensure imperceptibility.

Result: Experimental results on CLIVE dataset demonstrate superior transferability of SEGA compared to existing methods, enabling successful black-box attacks.

Conclusion: SEGA effectively addresses the transferability challenge in attacking NR-IQA models and provides a practical solution for black-box scenarios.

Abstract: No-Reference Image Quality Assessment (NR-IQA) models play an important role in various real-world applications. Recently, adversarial attacks against NR-IQA models have attracted increasing attention, as they provide valuable insights for revealing model vulnerabilities and guiding robust system design. Some effective attacks have been proposed against NR-IQA models in white-box settings, where the attacker has full access to the target model. However, these attacks often suffer from poor transferability to unknown target models in more realistic black-box scenarios, where the target model is inaccessible. This work makes the first attempt to address the challenge of low transferability in attacking NR-IQA models by proposing a transferable Signed Ensemble Gaussian black-box Attack (SEGA). The main idea is to approximate the gradient of the target model by applying Gaussian smoothing to source models and ensembling their smoothed gradients. To ensure the imperceptibility of adversarial perturbations, SEGA further removes inappropriate perturbations using a specially designed perturbation filter mask. Experimental results on the CLIVE dataset demonstrate the superior transferability of SEGA, validating its effectiveness in enabling successful transfer-based black-box attacks against NR-IQA models.

Method: The paper introduces HadaSmileNet, which directly integrates transformer-based representations with D-Marker features through parameter-free multiplicative interactions. The framework systematically evaluates 15 fusion strategies and identifies Hadamard multiplicative fusion as optimal.

Result: HadaSmileNet achieves new state-of-the-art results across four benchmark datasets: UvA-NEMO (88.7%, +0.8), MMI (99.7%), SPOS (98.5%, +0.7), and BBC (100%, +5.0). It also achieves 26% parameter reduction and simplified training compared to multi-task alternatives.

Conclusion: The proposed framework demonstrates enhanced discriminative power through direct domain knowledge integration, making it suitable for practical deployment in multimedia data mining applications requiring real-time affective computing capabilities.

Abstract: The distinction between genuine and posed emotions represents a fundamental pattern recognition challenge with significant implications for data mining applications in social sciences, healthcare, and human-computer interaction. While recent multi-task learning frameworks have shown promise in combining deep learning architectures with handcrafted D-Marker features for smile facial emotion recognition, these approaches exhibit computational inefficiencies due to auxiliary task supervision and complex loss balancing requirements. This paper introduces HadaSmileNet, a novel feature fusion framework that directly integrates transformer-based representations with physiologically grounded D-Markers through parameter-free multiplicative interactions. Through systematic evaluation of 15 fusion strategies, we demonstrate that Hadamard multiplicative fusion achieves optimal performance by enabling direct feature interactions while maintaining computational efficiency. The proposed approach establishes new state-of-the-art results for deep learning methods across four benchmark datasets: UvA-NEMO (88.7 percent, +0.8), MMI (99.7 percent), SPOS (98.5 percent, +0.7), and BBC (100 percent, +5.0). Comprehensive computational analysis reveals 26 percent parameter reduction and simplified training compared to multi-task alternatives, while feature visualization demonstrates enhanced discriminative power through direct domain knowledge integration. The framework’s efficiency and effectiveness make it particularly suitable for practical deployment in multimedia data mining applications that require real-time affective computing capabilities.

Method: Three synergistic mechanisms: 1) Deconstructing video frames into Human-Object-Interaction-Place semantic tuples, 2) Efficient online event deduplication and updating, 3) Fine-tuning LLM with Chain-of-Thought for transparent reasoning.

Result: Significant outperformance of state-of-the-art methods on XD-Violence and UCF-Crime datasets in threat detection accuracy, real-time efficiency, and explainability.

Conclusion: Live-E2T successfully bridges the gap between real-time performance and explainability in threat monitoring systems through its unified framework design.

Abstract: Real-time threat monitoring identifies threatening behaviors in video streams and provides reasoning and assessment of threat events through explanatory text. However, prevailing methodologies, whether based on supervised learning or generative models, struggle to concurrently satisfy the demanding requirements of real-time performance and decision explainability. To bridge this gap, we introduce Live-E2T, a novel framework that unifies these two objectives through three synergistic mechanisms. First, we deconstruct video frames into structured Human-Object-Interaction-Place semantic tuples. This approach creates a compact, semantically focused representation, circumventing the information degradation common in conventional feature compression. Second, an efficient online event deduplication and updating mechanism is proposed to filter spatio-temporal redundancies, ensuring the system’s real time responsiveness. Finally, we fine-tune a Large Language Model using a Chain-of-Thought strategy, endow it with the capability for transparent and logical reasoning over event sequences to produce coherent threat assessment reports. Extensive experiments on benchmark datasets, including XD-Violence and UCF-Crime, demonstrate that Live-E2T significantly outperforms state-of-the-art methods in terms of threat detection accuracy, real-time efficiency, and the crucial dimension of explainability.

Method: Created PhotoCritique dataset from professional photographer discussions, developed PhotoEye model with language-guided multi-view vision fusion mechanism, and established PhotoBench benchmark for comprehensive aesthetic evaluation.

Result: The proposed model demonstrates clear advantages over existing models on both existing benchmarks and the new PhotoBench benchmark.

Conclusion: The work fundamentally enhances MLLMs’ aesthetic understanding through expert-curated data, specialized model architecture, and professional evaluation framework.

Abstract: While editing directly from life, photographers have found it too difficult to see simultaneously both the blue and the sky. Photographer and curator, Szarkowski insightfully revealed one of the notable gaps between general and aesthetic visual understanding: while the former focuses on identifying the factual element in an image (sky), the latter transcends such object identification, viewing it instead as an aesthetic component–a pure color block (blue). Such fundamental distinctions between general (detection, localization, etc.) and aesthetic (color, lighting, composition, etc.) visual understanding present a significant challenge for Multimodal Large Language Models (MLLMs). Although some recent works have made initial explorations, they are often limited to general and basic aesthetic commonsense. As a result, they frequently fall short in real-world scenarios (Fig. 1), which require extensive expertise–including photographic techniques, photo pre/post-processing knowledge, and more, to provide a detailed analysis and description. To fundamentally enhance the aesthetics understanding of MLLMs, we first introduce a novel dataset, PhotoCritique, derived from extensive discussions among professional photographers and enthusiasts, and characterized by the large scale, expertise, and diversity. Then, to better learn visual aesthetics from PhotoCritique, we furthur propose a novel model, PhotoEye, featuring a languageguided multi-view vision fusion mechanism to understand image aesthetics from multiple perspectives. Finally, we present a novel benchmark, PhotoBench, a comprehensive and professional benchmark for aesthetic visual understanding. On existing benchmarks and PhotoBench, our model demonstrates clear advantages over existing models.

Method: Leverages the XMem model, a memory-augmented architecture, to segment tumors across long cine-MRI sequences with efficient memory mechanisms for real-time tracking.

Result: Preliminary impressions indicate reasonable segmentation performance satisfying clinical real-time requirements, though precise quantitative results are unavailable due to lost experimental records.

Conclusion: The XMem-based framework contributes to improving tumor tracking precision in MRI-guided radiotherapy, demonstrating potential for clinical real-time applications despite incomplete quantitative evaluation.

Abstract: This paper presents an advanced tumor segmentation framework for real-time MRI-guided radiotherapy, designed for the TrackRAD2025 challenge. Our method leverages the XMem model, a memory-augmented architecture, to segment tumors across long cine-MRI sequences. The proposed system efficiently integrates memory mechanisms to track tumor motion in real-time, achieving high segmentation accuracy even under challenging conditions with limited annotated data. Unfortunately, the detailed experimental records have been lost, preventing us from reporting precise quantitative results at this stage. Nevertheless, From our preliminary impressions during development, the XMem-based framework demonstrated reasonable segmentation performance and satisfied the clinical real-time requirement. Our work contributes to improving the precision of tumor tracking during MRI-guided radiotherapy, which is crucial for enhancing the accuracy and safety of cancer treatments.

Method: Proposes Spatial-Semantic Consistent Model (SSCM) with three key components: Dynamic Spatial Warping Module for inter-contrast alignment, Semantic-Aware Token Aggregation Block for long-range consistency, and Spatial-Frequency Fusion Block for fine structure restoration.

Result: Experiments on public and private datasets demonstrate state-of-the-art performance with fewer parameters while ensuring spatially and semantically consistent reconstructions.

Conclusion: SSCM effectively addresses the spatial-semantic consistency challenge in MC-MRI super-resolution, achieving superior performance with efficient parameter usage.

Abstract: Multi-contrast Magnetic Resonance Imaging super-resolution (MC-MRI SR) aims to enhance low-resolution (LR) contrasts leveraging high-resolution (HR) references, shortening acquisition time and improving imaging efficiency while preserving anatomical details. The main challenge lies in maintaining spatial-semantic consistency, ensuring anatomical structures remain well-aligned and coherent despite structural discrepancies and motion between the target and reference images. Conventional methods insufficiently model spatial-semantic consistency and underuse frequency-domain information, which leads to poor fine-grained alignment and inadequate recovery of high-frequency details. In this paper, we propose the Spatial-Semantic Consistent Model (SSCM), which integrates a Dynamic Spatial Warping Module for inter-contrast spatial alignment, a Semantic-Aware Token Aggregation Block for long-range semantic consistency, and a Spatial-Frequency Fusion Block for fine structure restoration. Experiments on public and private datasets show that SSCM achieves state-of-the-art performance with fewer parameters while ensuring spatially and semantically consistent reconstructions.

Method: Proposes Oracle-educated GRPO (OraPO) with FactScore-based reward (FactS). OraPO enables single-stage RL-only training by converting failed GRPO explorations into direct preference supervision via a lightweight oracle. FactS extracts atomic clinical facts and checks entailment against ground-truth labels for dense, interpretable sentence-level rewards.

Result: Achieves new state-of-the-art performance on CheXpert Plus dataset (0.341 F1) with 2-3 orders of magnitude less training data using a small base VLM on modest hardware.

Conclusion: The compact framework significantly improves learning efficiency on clinically challenging cases, demonstrating that high-performance RRG can be achieved with substantially reduced computational and data requirements.

Abstract: Radiology report generation (RRG) aims to automatically produce clinically faithful reports from chest X-ray images. Prevailing work typically follows a scale-driven paradigm, by multi-stage training over large paired corpora and oversized backbones, making pipelines highly data- and compute-intensive. In this paper, we propose Oracle-educated GRPO {OraPO) with a FactScore-based reward (FactS) to tackle the RRG task under constrained budgets. OraPO enables single-stage, RL-only training by converting failed GRPO explorations on rare or difficult studies into direct preference supervision via a lightweight oracle step. FactS grounds learning in diagnostic evidence by extracting atomic clinical facts and checking entailment against ground-truth labels, yielding dense, interpretable sentence-level rewards. Together, OraPO and FactS create a compact and powerful framework that significantly improves learning efficiency on clinically challenging cases, setting the new SOTA performance on the CheXpert Plus dataset (0.341 in F1) with 2–3 orders of magnitude less training data using a small base VLM on modest hardware.

Method: Encodes all style images and textual hints with semantic token decomposition, adaptively injected into cross-attention layers. Uses similarity-aware re-weighting to recalibrate attention allocation at each denoising step.

Result: Qualitative and quantitative evaluations show AMSF outperforms state-of-the-art approaches in multi-style fusion, scaling seamlessly to two or more styles without fine-tuning.

Conclusion: AMSF represents a practical step toward expressive multi-style generation in diffusion models, enabling balanced and user-controllable style blends.

Abstract: We propose Adaptive Multi-Style Fusion (AMSF), a reference-based training-free framework that enables controllable fusion of multiple reference styles in diffusion models. Most of the existing reference-based methods are limited by (a) acceptance of only one style image, thus prohibiting hybrid aesthetics and scalability to more styles, and (b) lack of a principled mechanism to balance several stylistic influences. AMSF mitigates these challenges by encoding all style images and textual hints with a semantic token decomposition module that is adaptively injected into every cross-attention layer of an frozen diffusion model. A similarity-aware re-weighting module then recalibrates, at each denoising step, the attention allocated to every style component, yielding balanced and user-controllable blends without any fine-tuning or external adapters. Both qualitative and quantitative evaluations show that AMSF produces multi-style fusion results that consistently outperform the state-of-the-art approaches, while its fusion design scales seamlessly to two or more styles. These capabilities position AMSF as a practical step toward expressive multi-style generation in diffusion models.

Method: A two-stage framework with three modules: Enhanced Radar Point Encoder (ERPE) for point-level fusion, Hierarchical Scene Fusion Pooling (HSFP) for scene-level fusion, and Proposal-Level Fusion Enhancement (PLFE) for proposal-level fusion using deformable attention and multi-scale feature integration.

Result: Achieves state-of-the-art performance on View-of-Delft (VoD) and TJ4DRadSet datasets, with performance comparable to LiDAR-based models on VoD dataset.

Conclusion: The multi-level fusion approach effectively addresses radar’s sparsity limitations and enables comprehensive feature representation for robust 3D object detection in autonomous driving.

Abstract: The emerging 4D millimeter-wave radar, measuring the range, azimuth, elevation, and Doppler velocity of objects, is recognized for its cost-effectiveness and robustness in autonomous driving. Nevertheless, its point clouds exhibit significant sparsity and noise, restricting its standalone application in 3D object detection. Recent 4D radar-camera fusion methods have provided effective perception. Most existing approaches, however, adopt explicit Bird’s-Eye-View fusion paradigms originally designed for LiDAR-camera fusion, neglecting radar’s inherent drawbacks. Specifically, they overlook the sparse and incomplete geometry of radar point clouds and restrict fusion to coarse scene-level integration. To address these problems, we propose MLF-4DRCNet, a novel two-stage framework for 3D object detection via multi-level fusion of 4D radar and camera images. Our model incorporates the point-, scene-, and proposal-level multi-modal information, enabling comprehensive feature representation. It comprises three crucial components: the Enhanced Radar Point Encoder (ERPE) module, the Hierarchical Scene Fusion Pooling (HSFP) module, and the Proposal-Level Fusion Enhancement (PLFE) module. Operating at the point-level, ERPE densities radar point clouds with 2D image instances and encodes them into voxels via the proposed Triple-Attention Voxel Feature Encoder. HSFP dynamically integrates multi-scale voxel features with 2D image features using deformable attention to capture scene context and adopts pooling to the fused features. PLFE refines region proposals by fusing image features, and further integrates with the pooled features from HSFP. Experimental results on the View-of-Delft (VoD) and TJ4DRadSet datasets demonstrate that MLF-4DRCNet achieves the state-of-the-art performance. Notably, it attains performance comparable to LiDAR-based models on the VoD dataset.

Method: PDLS decomposes inversion into two complementary streams: structural path for source integrity and semantic path guided by prompts, formulated as an optimal control problem solved via Linear Quadratic Regulator (LQR) for dynamic trajectory steering.

Result: Extensive experiments on FFHQ-1K and ImageNet-1K show PDLS produces more faithful reconstructions with better semantic alignment than single-latent baselines across various inversion tasks including deblurring, super-resolution, and inpainting.

Conclusion: PDLS effectively prevents semantic drift while preserving fine details through its dual latent steering approach, demonstrating superior performance over existing single-latent methods.

Abstract: Inverting corrupted images into the latent space of diffusion models is challenging. Current methods, which encode an image into a single latent vector, struggle to balance structural fidelity with semantic accuracy, leading to reconstructions with semantic drift, such as blurred details or incorrect attributes. To overcome this, we introduce Prompt-Guided Dual Latent Steering (PDLS), a novel, training-free framework built upon Rectified Flow models for their stable inversion paths. PDLS decomposes the inversion process into two complementary streams: a structural path to preserve source integrity and a semantic path guided by a prompt. We formulate this dual guidance as an optimal control problem and derive a closed-form solution via a Linear Quadratic Regulator (LQR). This controller dynamically steers the generative trajectory at each step, preventing semantic drift while ensuring the preservation of fine detail without costly, per-image optimization. Extensive experiments on FFHQ-1K and ImageNet-1K under various inversion tasks, including Gaussian deblurring, motion deblurring, super-resolution and freeform inpainting, demonstrate that PDLS produces reconstructions that are both more faithful to the original image and better aligned with the semantic information than single-latent baselines.

Method: Developed Prima using a hierarchical vision architecture trained on over 220,000 MRI studies from a large academic health system, tested in a 1-year system-wide study with 30K MRI studies.

Result: Achieved mean diagnostic area under the ROC curve of 92.0 across 52 radiologic diagnoses, outperforming state-of-the-art AI models, with demonstrated algorithmic fairness across diverse patient demographics.

Conclusion: Prima demonstrates transformative potential for health system-scale vision language models in advancing AI-driven healthcare by mitigating health system biases and improving diagnostic efficiency.

Abstract: Neuroimaging is a ubiquitous tool for evaluating patients with neurological diseases. The global demand for magnetic resonance imaging (MRI) studies has risen steadily, placing significant strain on health systems, prolonging turnaround times, and intensifying physician burnout \cite{Chen2017-bt, Rula2024-qp-1}. These challenges disproportionately impact patients in low-resource and rural settings. Here, we utilized a large academic health system as a data engine to develop Prima, the first vision language model (VLM) serving as an AI foundation for neuroimaging that supports real-world, clinical MRI studies as input. Trained on over 220,000 MRI studies, Prima uses a hierarchical vision architecture that provides general and transferable MRI features. Prima was tested in a 1-year health system-wide study that included 30K MRI studies. Across 52 radiologic diagnoses from the major neurologic disorders, including neoplastic, inflammatory, infectious, and developmental lesions, Prima achieved a mean diagnostic area under the ROC curve of 92.0, outperforming other state-of-the-art general and medical AI models. Prima offers explainable differential diagnoses, worklist priority for radiologists, and clinical referral recommendations across diverse patient demographics and MRI systems. Prima demonstrates algorithmic fairness across sensitive groups and can help mitigate health system biases, such as prolonged turnaround times for low-resource populations. These findings highlight the transformative potential of health system-scale VLMs and Prima’s role in advancing AI-driven healthcare.

Method: The UiG framework introduces ‘Image Editing’ as a bridge to infuse understanding into generation. It verifies generated images, incorporates model understanding into editing instructions, and enhances images step by step through iterative understanding-guided editing.

Result: UiG demonstrates significant performance improvement in text-to-image generation, achieving a 3.92% gain on the long prompt setting of the TIIF benchmark compared to existing text-to-image reasoning methods.

Conclusion: The proposed Understanding-in-Generation framework successfully integrates understanding capabilities into the generation process, effectively mitigating limitations of generative abilities in unified models and showing superior performance over existing reasoning methods.

Abstract: Recent works have made notable advancements in enhancing unified models for text-to-image generation through the Chain-of-Thought (CoT). However, these reasoning methods separate the processes of understanding and generation, which limits their ability to guide the reasoning of unified models in addressing the deficiencies of their generative capabilities. To this end, we propose a novel reasoning framework for unified models, Understanding-in-Generation (UiG), which harnesses the robust understanding capabilities of unified models to reinforce their performance in image generation. The core insight of our UiG is to integrate generative guidance by the strong understanding capabilities during the reasoning process, thereby mitigating the limitations of generative abilities. To achieve this, we introduce “Image Editing” as a bridge to infuse understanding into the generation process. Initially, we verify the generated image and incorporate the understanding of unified models into the editing instructions. Subsequently, we enhance the generated image step by step, gradually infusing the understanding into the generation process. Our UiG framework demonstrates a significant performance improvement in text-to-image generation over existing text-to-image reasoning methods, e.g., a 3.92% gain on the long prompt setting of the TIIF benchmark. The project code: https://github.com/QC-LY/UiG

Method: Created a comprehensive benchmark of state-of-the-art depth estimation models evaluated on real endoscopic images, and developed EndoSynth - a synthetic dataset with ground truth metric depth and segmentation masks for endoscopic surgical instruments.

Result: Fine-tuning depth foundation models using the synthetic EndoSynth dataset significantly boosts accuracy on most unseen real endoscopic data.

Conclusion: The work provides both a benchmark and synthetic dataset that advances depth estimation for endoscopic images and serves as an important resource for future research.

Abstract: Monocular relative and metric depth estimation has seen a tremendous boost in the last few years due to the sharp advancements in foundation models and in particular transformer based networks. As we start to see applications to the domain of endoscopic images, there is still a lack of robust benchmarks and high-quality datasets in that area. This paper addresses these limitations by presenting a comprehensive benchmark of state-of-the-art (metric and relative) depth estimation models evaluated on real, unseen endoscopic images, providing critical insights into their generalisation and performance in clinical scenarios. Additionally, we introduce and publish a novel synthetic dataset (EndoSynth) of endoscopic surgical instruments paired with ground truth metric depth and segmentation masks, designed to bridge the gap between synthetic and real-world data. We demonstrate that fine-tuning depth foundation models using our synthetic dataset boosts accuracy on most unseen real data by a significant margin. By providing both a benchmark and a synthetic dataset, this work advances the field of depth estimation for endoscopic images and serves as an important resource for future research. Project page, EndoSynth dataset and trained weights are available at https://github.com/TouchSurgery/EndoSynth.

Method: Proposes a lightweight algorithm integrating Window Multi-Head Attention Mechanism (WMHAM) to capture local/global features efficiently and Spatial Attention Mechanism (SAM) to emphasize key spatial regions for better feature representation.

Result: Achieved 95.24% accuracy on Food-101 and 94.33% on Vireo Food-172 datasets, with significant reduction in parameters and FLOPs compared to baseline methods.

Conclusion: The proposed approach effectively balances computational efficiency and classification performance, making it suitable for resource-constrained deployment environments.

Abstract: With the rapid development of society and continuous advances in science and technology, the food industry increasingly demands higher production quality and efficiency. Food image classification plays a vital role in enabling automated quality control on production lines, supporting food safety supervision, and promoting intelligent agricultural production. However, this task faces challenges due to the large number of parameters and high computational complexity of Vision Transformer models. To address these issues, we propose a lightweight food image classification algorithm that integrates a Window Multi-Head Attention Mechanism (WMHAM) and a Spatial Attention Mechanism (SAM). The WMHAM reduces computational cost by capturing local and global contextual features through efficient window partitioning, while the SAM adaptively emphasizes key spatial regions to improve discriminative feature representation. Experiments conducted on the Food-101 and Vireo Food-172 datasets demonstrate that our model achieves accuracies of 95.24% and 94.33%, respectively, while significantly reducing parameters and FLOPs compared with baseline methods. These results confirm that the proposed approach achieves an effective balance between computational efficiency and classification performance, making it well-suited for deployment in resource-constrained environments.

Method: Three-stage pipeline: (1) precise discovery and mask extraction with fine-tuned SAM, (2) semantic attribution and contextual description via fine-tuned MLLM, (3) LLM-as-judge and manual scoring for evaluation.

Result: The framework provides scalable and interpretable solution for dynamic land-cover monitoring, showing strong potential for automated cartographic updating and large-scale earth observation analysis.

Conclusion: OSDA combines pixel-level accuracy with high-level semantic understanding, offering an architecture-agnostic and label-free approach for robust evaluation across diverse satellite imagery.

Abstract: Open-set land-cover analysis in remote sensing requires the ability to achieve fine-grained spatial localization and semantically open categorization. This involves not only detecting and segmenting novel objects without categorical supervision but also assigning them interpretable semantic labels through multimodal reasoning. In this study, we introduce OSDA, an integrated three-stage framework for annotation-free open-set land-cover discovery, segmentation, and description. The pipeline consists of: (1) precise discovery and mask extraction with a promptable fine-tuned segmentation model (SAM), (2) semantic attribution and contextual description via a two-phase fine-tuned multimodal large language model (MLLM), and (3) LLM-as-judge and manual scoring of the MLLMs evaluation. By combining pixel-level accuracy with high-level semantic understanding, OSDA addresses key challenges in open-world remote sensing interpretation. Designed to be architecture-agnostic and label-free, the framework supports robust evaluation across diverse satellite imagery without requiring manual annotation. Our work provides a scalable and interpretable solution for dynamic land-cover monitoring, showing strong potential for automated cartographic updating and large-scale earth observation analysis.

Method: Cross-domain classification task using training data with hundreds of thousands of herbarium sheets and thousands of field photos, plus metadata including 5 morphological/functional traits per species. Test set consisted exclusively of field photos.

Result: The challenge assessed how herbarium collections can improve automated identification in data-poor regions, focusing on 1,000 species from the Guiana Shield (one of the world’s most biodiverse regions).

Conclusion: Herbarium collections can potentially bridge the data gap for automated plant identification in tropical biodiversity hotspots where field photo data is scarce.

Abstract: Automated plant identification has improved considerably thanks to recent advances in deep learning and the availability of training data with more and more field photos. However, this profusion of data concerns only a few tens of thousands of species, mainly located in North America and Western Europe, much less in the richest regions in terms of biodiversity such as tropical countries. On the other hand, for several centuries, botanists have systematically collected, catalogued and stored plant specimens in herbaria, especially in tropical regions, and recent efforts by the biodiversity informatics community have made it possible to put millions of digitised records online. The LifeCLEF 2021 plant identification challenge (or “PlantCLEF 2021”) was designed to assess the extent to which automated identification of flora in data-poor regions can be improved by using herbarium collections. It is based on a dataset of about 1,000 species mainly focused on the Guiana Shield of South America, a region known to have one of the highest plant diversities in the world. The challenge was evaluated as a cross-domain classification task where the training set consisted of several hundred thousand herbarium sheets and a few thousand photos to allow learning a correspondence between the two domains. In addition to the usual metadata (location, date, author, taxonomy), the training data also includes the values of 5 morphological and functional traits for each species. The test set consisted exclusively of photos taken in the field. This article presents the resources and evaluations of the assessment carried out, summarises the approaches and systems used by the participating research groups and provides an analysis of the main results.

Method: AGSwap consists of two key components: (1) Group-wise Embedding Swapping that fuses semantic attributes through feature manipulation, and (2) Adaptive Group Updating, a dynamic optimization mechanism guided by a balance evaluation score. The authors also introduce COF, a large-scale dataset with 451,250 unique fusion pairs built on ImageNet-1K and WordNet.

Result: Extensive experiments demonstrate that AGSwap outperforms state-of-the-art compositional T2I methods, including GPT-Image-1, using both simple and complex prompts.

Conclusion: AGSwap provides an effective solution for coherent cross-category object fusion in text-to-image generation, addressing key limitations of existing approaches through its adaptive swapping mechanism and supported by a comprehensive benchmark dataset.

Abstract: Fusing cross-category objects to a single coherent object has gained increasing attention in text-to-image (T2I) generation due to its broad applications in virtual reality, digital media, film, and gaming. However, existing methods often produce biased, visually chaotic, or semantically inconsistent results due to overlapping artifacts and poor integration. Moreover, progress in this field has been limited by the absence of a comprehensive benchmark dataset. To address these problems, we propose \textbf{Adaptive Group Swapping (AGSwap)}, a simple yet highly effective approach comprising two key components: (1) Group-wise Embedding Swapping, which fuses semantic attributes from different concepts through feature manipulation, and (2) Adaptive Group Updating, a dynamic optimization mechanism guided by a balance evaluation score to ensure coherent synthesis. Additionally, we introduce \textbf{Cross-category Object Fusion (COF)}, a large-scale, hierarchically structured dataset built upon ImageNet-1K and WordNet. COF includes 95 superclasses, each with 10 subclasses, enabling 451,250 unique fusion pairs. Extensive experiments demonstrate that AGSwap outperforms state-of-the-art compositional T2I methods, including GPT-Image-1 using simple and complex prompts.

Method: The challenge used a dataset of 10,000 plant species from the Guiana shield and Northern Amazon rainforest. Participating research groups developed automated identification systems, and their performance was compared against expert botanists specializing in tropical flora.

Result: The paper presents the evaluation results of various automated plant identification systems tested on the challenging dataset of 10,000 species from biodiversity-rich but data-deficient regions.

Conclusion: The LifeCLEF 2019 challenge provided valuable insights into the current capabilities and limitations of automated plant identification systems for data-deficient regions, highlighting the need for continued research to address the vast majority of plant species that remain underrepresented in automated identification systems.

Abstract: Automated identification of plants has improved considerably thanks to the recent progress in deep learning and the availability of training data. However, this profusion of data only concerns a few tens of thousands of species, while the planet has nearly 369K. The LifeCLEF 2019 Plant Identification challenge (or “PlantCLEF 2019”) was designed to evaluate automated identification on the flora of data deficient regions. It is based on a dataset of 10K species mainly focused on the Guiana shield and the Northern Amazon rainforest, an area known to have one of the greatest diversity of plants and animals in the world. As in the previous edition, a comparison of the performance of the systems evaluated with the best tropical flora experts was carried out. This paper presents the resources and assessments of the challenge, summarizes the approaches and systems employed by the participating research groups, and provides an analysis of the main outcomes.

Method: The framework has three stages: (1) Overview - uses a vision-language model to get cross-attention maps for semantic correlations, (2) Focus - leverages diffusion models to fill structural gaps missed by VLM, (3) Evolve - uses attention evolution to suppress irrelevant activations and produce purified segmentation masks.

Result: Extensive experiments show that RSVG-ZeroOV consistently outperforms existing weakly-supervised and zero-shot methods without requiring task-specific training.

Conclusion: The proposed training-free framework offers an efficient and scalable solution for open-vocabulary remote sensing visual grounding by effectively combining frozen foundation models.

Abstract: Remote sensing visual grounding (RSVG) aims to localize objects in remote sensing images based on free-form natural language expressions. Existing approaches are typically constrained to closed-set vocabularies, limiting their applicability in open-world scenarios. While recent attempts to leverage generic foundation models for open-vocabulary RSVG, they overly rely on expensive high-quality datasets and time-consuming fine-tuning. To address these limitations, we propose \textbf{RSVG-ZeroOV}, a training-free framework that aims to explore the potential of frozen generic foundation models for zero-shot open-vocabulary RSVG. Specifically, RSVG-ZeroOV comprises three key stages: (i) Overview: We utilize a vision-language model (VLM) to obtain cross-attention\footnote[1]{In this paper, although decoder-only VLMs use self-attention over all tokens, we refer to the image-text interaction part as cross-attention to distinguish it from pure visual self-attention.}maps that capture semantic correlations between text queries and visual regions. (ii) Focus: By leveraging the fine-grained modeling priors of a diffusion model (DM), we fill in gaps in structural and shape information of objects, which are often overlooked by VLM. (iii) Evolve: A simple yet effective attention evolution module is introduced to suppress irrelevant activations, yielding purified segmentation masks over the referred objects. Without cumbersome task-specific training, RSVG-ZeroOV offers an efficient and scalable solution. Extensive experiments demonstrate that the proposed framework consistently outperforms existing weakly-supervised and zero-shot methods.

Method: APC framework with Attribute Prompt Generator (Semantic Attribute Dictionary + Prompt Composition Module) and Fast-Slow Training Strategy (Fast Update Stream for ReID-specific knowledge, Slow Update Stream for VLM generalizable knowledge).

Result: Extensive experiments show superior performance on conventional and Domain Generalized ReID datasets, surpassing state-of-the-art methods.

Conclusion: The framework effectively balances ReID-specific discrimination with generalizable representation learning, demonstrating strong performance in both discrimination and generalization.

Abstract: Object Re-IDentification (ReID) aims to recognize individuals across non-overlapping camera views. While recent advances have achieved remarkable progress, most existing models are constrained to either single-domain or cross-domain scenarios, limiting their real-world applicability. Single-domain models tend to overfit to domain-specific features, whereas cross-domain models often rely on diverse normalization strategies that may inadvertently suppress identity-specific discriminative cues. To address these limitations, we propose an Attribute Prompt Composition (APC) framework, which exploits textual semantics to jointly enhance discrimination and generalization. Specifically, we design an Attribute Prompt Generator (APG) consisting of a Semantic Attribute Dictionary (SAD) and a Prompt Composition Module (PCM). SAD is an over-complete attribute dictionary to provide rich semantic descriptions, while PCM adaptively composes relevant attributes from SAD to generate discriminative attribute-aware features. In addition, motivated by the strong generalization ability of Vision-Language Models (VLM), we propose a Fast-Slow Training Strategy (FSTS) to balance ReID-specific discrimination and generalizable representation learning. Specifically, FSTS adopts a Fast Update Stream (FUS) to rapidly acquire ReID-specific discriminative knowledge and a Slow Update Stream (SUS) to retain the generalizable knowledge inherited from the pre-trained VLM. Through a mutual interaction, the framework effectively focuses on ReID-relevant features while mitigating overfitting. Extensive experiments on both conventional and Domain Generalized (DG) ReID datasets demonstrate that our framework surpasses state-of-the-art methods, exhibiting superior performances in terms of both discrimination and generalization. The source code is available at https://github.com/AWangYQ/APC.

Method: The LFI framework introduces Interaction Queries to maintain persistent relational structures across network layers and uses interaction-based supervision derived from VLMs’ cross-modal attention mechanisms to capture dynamic interaction patterns.

Result: The method achieves significant improvements: 3.3 and 1.6 mAP/2.4 AP gains on TinyImageNet classification and COCO detection/segmentation, 2.4 and 9.3 zero-shot improvements on PACS and VLCS, with minimal parameter overhead and faster convergence.

Conclusion: LFI demonstrates that modeling visual understanding as an interactive process enables more faithful and efficient knowledge transfer from VLMs to VFMs, particularly excelling in cross-domain settings and showing strong cognitive alignment in human evaluations.

Abstract: Current visual foundation models (VFMs) face a fundamental limitation in transferring knowledge from vision language models (VLMs), while VLMs excel at modeling cross-modal interactions through unified representation spaces, existing VFMs predominantly adopt result-oriented paradigms that neglect the underlying interaction processes. This representational discrepancy hinders effective knowledge transfer and limits generalization across diverse vision tasks. We propose Learning from Interactions (LFI), a cognitive-inspired framework that addresses this gap by explicitly modeling visual understanding as an interactive process. Our key insight is that capturing the dynamic interaction patterns encoded in pre-trained VLMs enables more faithful and efficient knowledge transfer to VFMs. The approach centers on two technical innovations, Interaction Queries, which maintain persistent relational structures across network layers, and interaction-based supervision, derived from the cross-modal attention mechanisms of VLMs. Comprehensive experiments demonstrate consistent improvements across multiple benchmarks, achieving 3.3 and 1.6mAP/2.4AP absolute gains on TinyImageNet classification and COCO detection/segmentation respectively, with minimal parameter overhead and faster convergence. The framework particularly excels in cross-domain settings, delivering 2.4 and 9.3 zero-shot improvements on PACS and VLCS. Human evaluations further confirm its cognitive alignment, outperforming result-oriented methods by 2.7 times in semantic consistency metrics.

Method: Proposes HyPSAM with two components: DFNet for generating initial saliency maps using dynamic convolution and multi-branch decoding, and P2RNet as a plug-and-play refinement network that uses hybrid prompts (text, mask, box) to guide SAM in refining saliency maps.

Result: Extensive experiments on three public datasets demonstrate state-of-the-art performance, with remarkable versatility to integrate with different RGB-T SOD methods for significant performance gains.

Conclusion: HyPSAM highlights the potential of prompt engineering in RGB-T SOD, achieving superior performance through effective integration of SAM’s capabilities with adaptive cross-modality interaction and hybrid prompt guidance.

Abstract: RGB-thermal salient object detection (RGB-T SOD) aims to identify prominent objects by integrating complementary information from RGB and thermal modalities. However, learning the precise boundaries and complete objects remains challenging due to the intrinsic insufficient feature fusion and the extrinsic limitations of data scarcity. In this paper, we propose a novel hybrid prompt-driven segment anything model (HyPSAM), which leverages the zero-shot generalization capabilities of the segment anything model (SAM) for RGB-T SOD. Specifically, we first propose a dynamic fusion network (DFNet) that generates high-quality initial saliency maps as visual prompts. DFNet employs dynamic convolution and multi-branch decoding to facilitate adaptive cross-modality interaction, overcoming the limitations of fixed-parameter kernels and enhancing multi-modal feature representation. Moreover, we propose a plug-and-play refinement network (P2RNet), which serves as a general optimization strategy to guide SAM in refining saliency maps by using hybrid prompts. The text prompt ensures reliable modality input, while the mask and box prompts enable precise salient object localization. Extensive experiments on three public datasets demonstrate that our method achieves state-of-the-art performance. Notably, HyPSAM has remarkable versatility, seamlessly integrating with different RGB-T SOD methods to achieve significant performance gains, thereby highlighting the potential of prompt engineering in this field. The code and results of our method are available at: https://github.com/milotic233/HyPSAM.

Method: Uses multimodal cross-attention to align semantic cues from text, geometric features from depth maps, and spatial structure from LiDAR. The framework is model-agnostic and integrates with any CNN-based point cloud autoencoder.

Result: Significantly more robust reconstruction under strong adversarial attacks and heavy noise compared to CNN-based autoencoders, with limited gains under mild perturbations.

Conclusion: TriFusion-AE provides a robust multimodal fusion framework that enhances LiDAR perception resilience in autonomous driving applications.

Abstract: LiDAR-based perception is central to autonomous driving and robotics, yet raw point clouds remain highly vulnerable to noise, occlusion, and adversarial corruptions. Autoencoders offer a natural framework for denoising and reconstruction, but their performance degrades under challenging real-world conditions. In this work, we propose TriFusion-AE, a multimodal cross-attention autoencoder that integrates textual priors, monocular depth maps from multi-view images, and LiDAR point clouds to improve robustness. By aligning semantic cues from text, geometric (depth) features from images, and spatial structure from LiDAR, TriFusion-AE learns representations that are resilient to stochastic noise and adversarial perturbations. Interestingly, while showing limited gains under mild perturbations, our model achieves significantly more robust reconstruction under strong adversarial attacks and heavy noise, where CNN-based autoencoders collapse. We evaluate on the nuScenes-mini dataset to reflect realistic low-data deployment scenarios. Our multimodal fusion framework is designed to be model-agnostic, enabling seamless integration with any CNN-based point cloud autoencoder for joint representation learning.

Method: Proposes structured reflection with Reflect-Call-Final strategy, combining DAPO and GSPO objectives with tool-use tailored rewards. Uses Tool-Reflection-Bench for evaluation.

Result: Large gains in multi-turn tool-call success and error recovery, reduction of redundant calls on BFCL v3 and Tool-Reflection-Bench benchmarks.

Conclusion: Making reflection explicit and optimizing it directly improves tool interaction reliability and provides reproducible learning from failure.

Abstract: Tool-augmented large language models (LLMs) are usually trained with supervised imitation or coarse-grained reinforcement learning that optimizes single tool calls. Current self-reflection practices rely on heuristic prompts or one-way reasoning: the model is urged to ’think more’ instead of learning error diagnosis and repair. This is fragile in multi-turn interactions; after a failure the model often repeats the same mistake. We propose structured reflection, which turns the path from error to repair into an explicit, controllable, and trainable action. The agent produces a short yet precise reflection: it diagnoses the failure using evidence from the previous step and then proposes a correct, executable follow-up call. For training we combine DAPO and GSPO objectives with a reward scheme tailored to tool use, optimizing the stepwise strategy Reflect, then Call, then Final. To evaluate, we introduce Tool-Reflection-Bench, a lightweight benchmark that programmatically checks structural validity, executability, parameter correctness, and result consistency. Tasks are built as mini trajectories of erroneous call, reflection, and corrected call, with disjoint train and test splits. Experiments on BFCL v3 and Tool-Reflection-Bench show large gains in multi-turn tool-call success and error recovery, and a reduction of redundant calls. These results indicate that making reflection explicit and optimizing it directly improves the reliability of tool interaction and offers a reproducible path for agents to learn from failure.

Method: COLT incorporates a learnable tool codebook as tool-specific memory, dynamically selecting relevant tools based on similarity between user instructions and tool features.

Result: Extensive experiments show state-of-the-art performance on video LLM benchmarks and the VideoToolBench dataset.

Conclusion: COLT successfully enables continuous tool usage in video LLMs, addressing the challenge of evolving tool environments.

Abstract: The success of Large Language Models (LLMs) has significantly propelled the research of video understanding. To harvest the benefits of well-trained expert models (i.e., tools), video LLMs prioritize the exploration of tool usage capabilities. Existing methods either prompt closed-source LLMs or employ the instruction tuning paradigm for tool-use fine-tuning. These methods, however, assume an established repository of fixed tools and struggle to generalize to real-world environments where tool data is perpetually evolving and streaming in. To this end, we propose to enhance open-source video LLMs with COntinuaL Tool usage (termed COLT), which automatically acquires tool-use ability in a successive tool stream without suffering ‘catastrophic forgetting’ of the past learned tools. Specifically, our COLT incorporates a learnable tool codebook as a tool-specific memory system. Then relevant tools are dynamically selected based on the similarity between user instruction and tool features within the codebook. To unleash the tool usage potential of video LLMs, we collect a video-centric tool-use instruction tuning dataset VideoToolBench. Extensive experiments on both previous video LLM benchmarks and the tool-use-specific VideoToolBench dataset demonstrate the state-of-the-art performance of our proposed COLT.

Method: The authors present VIR-Bench, consisting of 200 travel videos that frame itinerary reconstruction as a challenging evaluation task. They conduct experiments with state-of-the-art MLLMs and develop a prototype travel-planning agent based on insights from the benchmark.

Result: Experimental results show that state-of-the-art MLLMs, including proprietary ones, struggle to achieve high scores on VIR-Bench, highlighting the difficulty of handling videos spanning extended spatial and temporal scales. The prototype travel-planning agent demonstrates markedly improved itinerary recommendations.

Conclusion: VIR-Bench effectively benchmarks MLLMs’ geospatial-temporal intelligence and translates into concrete performance gains in user-facing applications, verifying the benchmark’s practical utility for advancing video understanding capabilities.

Abstract: Recent advances in multimodal large language models (MLLMs) have significantly enhanced video understanding capabilities, opening new possibilities for practical applications. Yet current video benchmarks focus largely on indoor scenes or short-range outdoor activities, leaving the challenges associated with long-distance travel largely unexplored. Mastering extended geospatial-temporal trajectories is critical for next-generation MLLMs, underpinning real-world tasks such as embodied-AI planning and navigation. To bridge this gap, we present VIR-Bench, a novel benchmark consisting of 200 travel videos that frames itinerary reconstruction as a challenging task designed to evaluate and push forward MLLMs’ geospatial-temporal intelligence. Experimental results reveal that state-of-the-art MLLMs, including proprietary ones, struggle to achieve high scores, underscoring the difficulty of handling videos that span extended spatial and temporal scales. Moreover, we conduct an in-depth case study in which we develop a prototype travel-planning agent that leverages the insights gained from VIR-Bench. The agent’s markedly improved itinerary recommendations verify that our evaluation protocol not only benchmarks models effectively but also translates into concrete performance gains in user-facing applications.

Method: Proposes a distillation approach that delivers accurate and cross-view coherent diffusion priors, combined with an adaptive progressive enhancement scheme for refining under-constrained regions.

Result: Extensive experiments show FixingGS surpasses state-of-the-art methods with superior visual quality and reconstruction performance.

Conclusion: FixingGS effectively addresses sparse-view 3DGS reconstruction challenges by leveraging diffusion models without requiring training, achieving better multi-view consistency and artifact removal.

Abstract: Recently, 3D Gaussian Splatting (3DGS) has demonstrated remarkable success in 3D reconstruction and novel view synthesis. However, reconstructing 3D scenes from sparse viewpoints remains highly challenging due to insufficient visual information, which results in noticeable artifacts persisting across the 3D representation. To address this limitation, recent methods have resorted to generative priors to remove artifacts and complete missing content in under-constrained areas. Despite their effectiveness, these approaches struggle to ensure multi-view consistency, resulting in blurred structures and implausible details. In this work, we propose FixingGS, a training-free method that fully exploits the capabilities of the existing diffusion model for sparse-view 3DGS reconstruction enhancement. At the core of FixingGS is our distillation approach, which delivers more accurate and cross-view coherent diffusion priors, thereby enabling effective artifact removal and inpainting. In addition, we propose an adaptive progressive enhancement scheme that further refines reconstructions in under-constrained regions. Extensive experiments demonstrate that FixingGS surpasses existing state-of-the-art methods with superior visual quality and reconstruction performance. Our code will be released publicly.

Method: Created a dataset of 500 Ishihara-like images with numbers 0-99 using varying color combinations, tested 9 VLMs with Yes/No and open-ended prompts, and compared results with human participants.

Result: Experiments revealed significant limitations in VLMs’ ability to interpret numbers in adversarial contexts, showing prevalent hallucination issues and poorer performance compared to humans.

Conclusion: The findings highlight the need to improve VLM robustness in complex visual environments, and ColorBlindnessEval serves as a valuable benchmarking tool for enhancing VLM reliability in real-world applications.

Abstract: This paper presents ColorBlindnessEval, a novel benchmark designed to evaluate the robustness of Vision-Language Models (VLMs) in visually adversarial scenarios inspired by the Ishihara color blindness test. Our dataset comprises 500 Ishihara-like images featuring numbers from 0 to 99 with varying color combinations, challenging VLMs to accurately recognize numerical information embedded in complex visual patterns. We assess 9 VLMs using Yes/No and open-ended prompts and compare their performance with human participants. Our experiments reveal limitations in the models’ ability to interpret numbers in adversarial contexts, highlighting prevalent hallucination issues. These findings underscore the need to improve the robustness of VLMs in complex visual environments. ColorBlindnessEval serves as a valuable tool for benchmarking and improving the reliability of VLMs in real-world applications where accuracy is critical.

Method: Non-uniform weight separation based on Gaussian quantiles, grouping weights into outlier (salient) and multiple inlier (unsalient) subsets. Uses saliency-aware hybrid quantization algorithm with different constraints on scaler and binary matrices based on saliency metric and compression objective.

Result: For language model part: outperforms SOTA by 3%-47% on visual question answering across 4 benchmarks and 3 models. For overall VLM: outperforms SOTA by 4%-45%. Also enables 90%-99% image token pruning in quantized models.

Conclusion: Bi-VLM effectively bridges the gap between computational demands and ultra-low-bit precision, achieving significant efficiency gains while maintaining or improving performance on vision-language tasks.

Abstract: We address the critical gap between the computational demands of vision-language models and the possible ultra-low-bit weight precision (bitwidth $\leq2$ bits) we can use for higher efficiency. Our work is motivated by the substantial computational cost and memory requirements of VLMs, which restrict their applicability in hardware-constrained environments. We propose Bi-VLM, which separates model weights non-uniformly based on the Gaussian quantiles. Our formulation groups the model weights into outlier (salient) and multiple inlier (unsalient) subsets, ensuring that each subset contains a proportion of weights corresponding to its quantile in the distribution. We propose a saliency-aware hybrid quantization algorithm and use it to quantize weights by imposing different constraints on the scaler and binary matrices based on the saliency metric and compression objective. We have evaluated our approach on different VLMs. For the language model part of the VLM, our Bi-VLM outperforms the SOTA by 3%-47% on the visual question answering task in terms of four different benchmarks and three different models. For the overall VLM, our Bi-VLM outperforms the SOTA by 4%-45%. We also perform token pruning on the quantized models and observe that there is redundancy of image tokens 90% - 99% in the quantized models. This helps us to further prune the visual tokens to improve efficiency.

Method: Proposes a novel multimodal training approach combining image analysis with textual reasoning. Integrates detection, segmentation, and multimodal chain-of-thought reasoning in a single framework with a curated open-source data suite covering reasoning, detection, segmentation, and document understanding tasks.

Result: Citrus-V outperforms existing open-source medical models and expert-level imaging systems across multiple benchmarks, delivering unified pipeline from visual grounding to clinical reasoning.

Conclusion: The model enables pixel-level lesion localization, structured report generation, and physician-like diagnostic inference, supporting precise lesion quantification, automated reporting, and reliable second opinions for clinical applications.

Abstract: Medical imaging provides critical evidence for clinical diagnosis, treatment planning, and surgical decisions, yet most existing imaging models are narrowly focused and require multiple specialized networks, limiting their generalization. Although large-scale language and multimodal models exhibit strong reasoning and multi-task capabilities, real-world clinical applications demand precise visual grounding, multimodal integration, and chain-of-thought reasoning. We introduce Citrus-V, a multimodal medical foundation model that combines image analysis with textual reasoning. The model integrates detection, segmentation, and multimodal chain-of-thought reasoning, enabling pixel-level lesion localization, structured report generation, and physician-like diagnostic inference in a single framework. We propose a novel multimodal training approach and release a curated open-source data suite covering reasoning, detection, segmentation, and document understanding tasks. Evaluations demonstrate that Citrus-V outperforms existing open-source medical models and expert-level imaging systems across multiple benchmarks, delivering a unified pipeline from visual grounding to clinical reasoning and supporting precise lesion quantification, automated reporting, and reliable second opinions.

Method: Integrates multi-scale vector quantization into SSL pipeline to impose a discrete representational bottleneck, constraining the model to learn repeatable, structure-aware features while suppressing view-specific or low-utility patterns.

Result: Achieves strong performance on classification and segmentation tasks with minimal or no fine-tuning, shows high label efficiency in low-label regimes, and demonstrates robustness across multiple public medical imaging datasets.

Conclusion: DiSSECT provides an effective framework for learning transferable medical image representations that outperform state-of-the-art approaches by addressing shortcut learning through discrete representation constraints.

Abstract: Self-supervised learning (SSL) has emerged as a powerful paradigm for medical image representation learning, particularly in settings with limited labeled data. However, existing SSL methods often rely on complex architectures, anatomy-specific priors, or heavily tuned augmentations, which limit their scalability and generalizability. More critically, these models are prone to shortcut learning, especially in modalities like chest X-rays, where anatomical similarity is high and pathology is subtle. In this work, we introduce DiSSECT – Discrete Self-Supervision for Efficient Clinical Transferable Representations, a framework that integrates multi-scale vector quantization into the SSL pipeline to impose a discrete representational bottleneck. This constrains the model to learn repeatable, structure-aware features while suppressing view-specific or low-utility patterns, improving representation transfer across tasks and domains. DiSSECT achieves strong performance on both classification and segmentation tasks, requiring minimal or no fine-tuning, and shows particularly high label efficiency in low-label regimes. We validate DiSSECT across multiple public medical imaging datasets, demonstrating its robustness and generalizability compared to existing state-of-the-art approaches.

Method: Integration of thermal imaging, deep learning, and CV2X communication trained on 12,000 thermal deer images from Mars Hill, NC. Uses sensor data sharing via cellular V2X when high probability threshold is reached.

Result: 98.84% mAP, 95.44% precision, 95.96% recall. Thermal imaging maintains 88-92% accuracy vs <60% for visible light cameras. End-to-end latency under 100ms from detection to driver alert.

Conclusion: Establishes viable technological pathway for reducing deer-vehicle collisions through thermal imaging and connected vehicles, validated by successful field testing across diverse weather conditions.

Abstract: Deer-vehicle collisions represent a critical safety challenge in the United States, causing nearly 2.1 million incidents annually and resulting in approximately 440 fatalities, 59,000 injuries, and 10 billion USD in economic damages. These collisions also contribute significantly to declining deer populations. This paper presents a real-time detection and driver warning system that integrates thermal imaging, deep learning, and vehicle-to-everything communication to help mitigate deer-vehicle collisions. Our system was trained and validated on a custom dataset of over 12,000 thermal deer images collected in Mars Hill, North Carolina. Experimental evaluation demonstrates exceptional performance with 98.84 percent mean average precision, 95.44 percent precision, and 95.96 percent recall. The system was field tested during a follow-up visit to Mars Hill and readily sensed deer providing the driver with advanced warning. Field testing validates robust operation across diverse weather conditions, with thermal imaging maintaining between 88 and 92 percent detection accuracy in challenging scenarios where conventional visible light based cameras achieve less than 60 percent effectiveness. When a high probability threshold is reached sensor data sharing messages are broadcast to surrounding vehicles and roadside units via cellular vehicle to everything (CV2X) communication devices. Overall, our system achieves end to end latency consistently under 100 milliseconds from detection to driver alert. This research establishes a viable technological pathway for reducing deer-vehicle collisions through thermal imaging and connected vehicles.

Method: Constructs pairs of preferred and non-preferred synthetic images, then employs lightweight fine-tuning of diffusion models to align image generation with downstream objectives through explicit task-aware alignment.

Result: Consistent gains of 7-9% in classification and 2-10% in segmentation across three surgical datasets, with significant improvements for underrepresented classes. Iterative refinement boosts performance by 4-10%.

Conclusion: SAADi establishes task-aware alignment as a key principle for mitigating data scarcity in surgical vision applications, overcoming sample degradation issues present in baseline approaches.

Abstract: The scarcity of annotated surgical data poses a significant challenge for developing deep learning systems in computer-assisted interventions. While diffusion models can synthesize realistic images, they often suffer from data memorization, resulting in inconsistent or non-diverse samples that may fail to improve, or even harm, downstream performance. We introduce \emph{Surgical Application-Aligned Diffusion} (SAADi), a new framework that aligns diffusion models with samples preferred by downstream models. Our method constructs pairs of \emph{preferred} and \emph{non-preferred} synthetic images and employs lightweight fine-tuning of diffusion models to align the image generation process with downstream objectives explicitly. Experiments on three surgical datasets demonstrate consistent gains of $7$–$9%$ in classification and $2$–$10%$ in segmentation tasks, with the considerable improvements observed for underrepresented classes. Iterative refinement of synthetic samples further boosts performance by $4$–$10%$. Unlike baseline approaches, our method overcomes sample degradation and establishes task-aware alignment as a key principle for mitigating data scarcity and advancing surgical vision applications.

Method: Proposes a model-based neural network that uses inter-frame spatial correlation and intra-frame structural consistency to establish a kernel space-based multidimensional sparse (KMDS) model, then substitutes parameter estimation with neural networks for adaptive optimization.

Result: Extensive experiments on simulated and real data show KMDS-Net outperforms previous baseline methods in denoising performance for dynamic PET.

Conclusion: The proposed method can effectively achieve high temporal and spatial resolution for dynamic PET, with source code made publicly available.

Abstract: Achieving high image quality for temporal frames in dynamic positron emission tomography (PET) is challenging due to the limited statistic especially for the short frames. Recent studies have shown that deep learning (DL) is useful in a wide range of medical image denoising tasks. In this paper, we propose a model-based neural network for dynamic PET image denoising. The inter-frame spatial correlation and intra-frame structural consistency in dynamic PET are used to establish the kernel space-based multidimensional sparse (KMDS) model. We then substitute the inherent forms of the parameter estimation with neural networks to enable adaptive parameters optimization, forming the end-to-end neural KMDS-Net. Extensive experimental results from simulated and real data demonstrate that the neural KMDS-Net exhibits strong denoising performance for dynamic PET, outperforming previous baseline methods. The proposed method may be used to effectively achieve high temporal and spatial resolution for dynamic PET. Our source code is available at https://github.com/Kuangxd/Neural-KMDS-Net/tree/main.

Method: Proposes optical flow-based segmentation label interpolation where optical flow estimated from annotated key frames propagates labels to adjacent unlabeled frames, combined with multi-task learning to enrich sparse spatial supervision and balance temporal-spatial information.

Result: The framework improves both accuracy and efficiency of surgical scene understanding by addressing the annotation imbalance problem.

Conclusion: The integration of optical flow-based label interpolation with multi-task learning enhances the utility of robot-assisted surgery by providing more comprehensive surgical scene understanding.

Abstract: Robot-assisted surgery (RAS) has become a critical paradigm in modern surgery, promoting patient recovery and reducing the burden on surgeons through minimally invasive approaches. To fully realize its potential, however, a precise understanding of the visual data generated during surgical procedures is essential. Previous studies have predominantly focused on single-task approaches, but real surgical scenes involve complex temporal dynamics and diverse instrument interactions that limit comprehensive understanding. Moreover, the effective application of multi-task learning (MTL) requires sufficient pixel-level segmentation data, which are difficult to obtain due to the high cost and expertise required for annotation. In particular, long-term annotations such as phases and steps are available for every frame, whereas short-term annotations such as surgical instrument segmentation and action detection are provided only for key frames, resulting in a significant temporal-spatial imbalance. To address these challenges, we propose a novel framework that combines optical flow-based segmentation label interpolation with multi-task learning. optical flow estimated from annotated key frames is used to propagate labels to adjacent unlabeled frames, thereby enriching sparse spatial supervision and balancing temporal and spatial information for training. This integration improves both the accuracy and efficiency of surgical scene understanding and, in turn, enhances the utility of RAS.

Method: Uses divide-and-conquer strategy with speculative decoding for next-token prediction and multi-stage distillation for diffusion denoising. Develops both lossless 6-NFE model and highly efficient 1-NFE model with adversarial distillation and human feedback learning.

Result: Achieves over 2x speedup in multimodal understanding, 16.67x speedup in text-to-image generation, and 22x speedup in image editing. The 1-NFE model enables near real-time interactive editing and generation.

Conclusion: Hyper-Bagel provides substantial performance gains while preserving output quality, making complex multimodal interactions seamless and instantaneous through advanced acceleration techniques.

Abstract: Unified multimodal models have recently attracted considerable attention for their remarkable abilities in jointly understanding and generating diverse content. However, as contexts integrate increasingly numerous interleaved multimodal tokens, the iterative processes of diffusion denoising and autoregressive decoding impose significant computational overhead. To address this, we propose Hyper-Bagel, a unified acceleration framework designed to simultaneously speed up both multimodal understanding and generation tasks. Our approach uses a divide-and-conquer strategy, employing speculative decoding for next-token prediction and a multi-stage distillation process for diffusion denoising. The framework delivers substantial performance gains, achieving over a 2x speedup in multimodal understanding. For generative tasks, our resulting lossless 6-NFE model yields a 16.67x speedup in text-to-image generation and a 22x speedup in image editing, all while preserving the high-quality output of the original model. We further develop a highly efficient 1-NFE model that enables near real-time interactive editing and generation. By combining advanced adversarial distillation with human feedback learning, this model achieves ultimate cost-effectiveness and responsiveness, making complex multimodal interactions seamless and instantaneous.

Method: Benchmarking study using three datasets (ArtDL, ICONCLASS, Wikidata) filtered to top 10 classes. Models tested under three conditions: class labels only, Iconclass descriptions, and few-shot learning with 5 exemplars. Compared against ResNet50 baselines.

Result: Gemini-2.5 Pro and GPT-4o outperformed ResNet50 baselines. Accuracy dropped significantly on Wikidata dataset where Siglip performed best. Class descriptions improved zero-shot performance, while few-shot learning generally produced lower results with minimal accuracy improvements.

Conclusion: General-purpose multimodal LLMs are capable of classification in visually complex cultural heritage domains, supporting their application as metadata curation tools in digital humanities. Future research should focus on prompt optimization and expanding to other classification strategies.

Abstract: This study evaluates the capabilities of Multimodal Large Language Models (LLMs) and Vision Language Models (VLMs) in the task of single-label classification of Christian Iconography. The goal was to assess whether general-purpose VLMs (CLIP and SigLIP) and LLMs, such as GPT-4o and Gemini 2.5, can interpret the Iconography, typically addressed by supervised classifiers, and evaluate their performance. Two research questions guided the analysis: (RQ1) How do multimodal LLMs perform on image classification of Christian saints? And (RQ2), how does performance vary when enriching input with contextual information or few-shot exemplars? We conducted a benchmarking study using three datasets supporting Iconclass natively: ArtDL, ICONCLASS, and Wikidata, filtered to include the top 10 most frequent classes. Models were tested under three conditions: (1) classification using class labels, (2) classification with Iconclass descriptions, and (3) few-shot learning with five exemplars. Results were compared against ResNet50 baselines fine-tuned on the same datasets. The findings show that Gemini-2.5 Pro and GPT-4o outperformed the ResNet50 baselines. Accuracy dropped significantly on the Wikidata dataset, where Siglip reached the highest accuracy score, suggesting model sensitivity to image size and metadata alignment. Enriching prompts with class descriptions generally improved zero-shot performance, while few-shot learning produced lower results, with only occasional and minimal increments in accuracy. We conclude that general-purpose multimodal LLMs are capable of classification in visually complex cultural heritage domains. These results support the application of LLMs as metadata curation tools in digital humanities workflows, suggesting future research on prompt optimization and the expansion of the study to other classification strategies and models.

Method: LRGC applies key-query attention between all node pairs, then uses soft-threshold reparameterization for differentiable edge selection. This allows learnable threshold tuning per layer during training without hyper-parameters.

Result: ViG-LRGC outperforms state-of-the-art ViG models of similar sizes on the ImageNet-1k benchmark dataset.

Conclusion: LRGC provides a more effective graph construction method for ViG models by enabling learnable, hyper-parameter-free neighborhood selection through differentiable attention mechanisms.

Abstract: Image Representation Learning is an important problem in Computer Vision. Traditionally, images were processed as grids, using Convolutional Neural Networks or as a sequence of visual tokens, using Vision Transformers. Recently, Vision Graph Neural Networks (ViG) have proposed the treatment of images as a graph of nodes; which provides a more intuitive image representation. The challenge is to construct a graph of nodes in each layer that best represents the relations between nodes and does not need a hyper-parameter search. ViG models in the literature depend on non-parameterized and non-learnable statistical methods that operate on the latent features of nodes to create a graph. This might not select the best neighborhood for each node. Starting from k-NN graph construction to HyperGraph Construction and Similarity-Thresholded graph construction, these methods lack the ability to provide a learnable hyper-parameter-free graph construction method. To overcome those challenges, we present the Learnable Reparameterized Graph Construction (LRGC) for Vision Graph Neural Networks. LRGC applies key-query attention between every pair of nodes; then uses soft-threshold reparameterization for edge selection, which allows the use of a differentiable mathematical model for training. Using learnable parameters to select the neighborhood removes the bias that is induced by any clustering or thresholding methods previously introduced in the literature. In addition, LRGC allows tuning the threshold in each layer to the training data since the thresholds are learnable through training and are not provided as hyper-parameters to the model. We demonstrate that the proposed ViG-LRGC approach outperforms state-of-the-art ViG models of similar sizes on the ImageNet-1k benchmark dataset.

Method: Adversarial RL framework with attacker and defender agents in a dual-space graph environment, using Deep Q-Networks to optimize prompts based on segmentation quality.

Result: Extensive experiments show improved SAM robustness and generalization across diverse tasks.

Conclusion: Establishes a flexible, interpretable, and plug-and-play framework for prompt-based segmentation.

Abstract: Prompt quality plays a critical role in the performance of the Segment Anything Model (SAM), yet existing approaches often rely on heuristic or manually crafted prompts, limiting scalability and generalization. In this paper, we propose Point Prompt Defender, an adversarial reinforcement learning framework that adopts an attack-for-defense paradigm to automatically optimize point prompts. We construct a task-agnostic point prompt environment by representing image patches as nodes in a dual-space graph, where edges encode both physical and semantic distances. Within this environment, an attacker agent learns to activate a subset of prompts that maximally degrade SAM’s segmentation performance, while a defender agent learns to suppress these disruptive prompts and restore accuracy. Both agents are trained using Deep Q-Networks with a reward signal based on segmentation quality variation. During inference, only the defender is deployed to refine arbitrary coarse prompt sets, enabling enhanced SAM segmentation performance across diverse tasks without retraining. Extensive experiments show that Point Prompt Defender effectively improves SAM’s robustness and generalization, establishing a flexible, interpretable, and plug-and-play framework for prompt-based segmentation.

Method: Pilot deployment captured four days of synchronized monitoring across three modalities (drone imagery, camera traps, bioacoustic recordings) in a 220-acre pasture containing various species including endangered animals and native Ohio wildlife.

Result: The dataset enables comparative analysis of sensor modality performance, demonstrating complementary strengths for landuse patterns, species detection, behavioral analysis, and habitat monitoring.

Conclusion: This work establishes reproducible protocols for multimodal wildlife monitoring and contributes open datasets to advance conservation computer vision research, with future releases planned to include GPS tracking data and expanded temporal coverage.

Abstract: We present the first release of SmartWilds, a multimodal wildlife monitoring dataset. SmartWilds is a synchronized collection of drone imagery, camera trap photographs and videos, and bioacoustic recordings collected during summer 2025 at The Wilds safari park in Ohio. This dataset supports multimodal AI research for comprehensive environmental monitoring, addressing critical needs in endangered species research, conservation ecology, and habitat management. Our pilot deployment captured four days of synchronized monitoring across three modalities in a 220-acre pasture containing Pere David’s deer, Sichuan takin, Przewalski’s horses, as well as species native to Ohio, including bald eagles, white-tailed deer, and coyotes. We provide a comparative analysis of sensor modality performance, demonstrating complementary strengths for landuse patterns, species detection, behavioral analysis, and habitat monitoring. This work establishes reproducible protocols for multimodal wildlife monitoring while contributing open datasets to advance conservation computer vision research. Future releases will include synchronized GPS tracking data from tagged individuals, citizen science data, and expanded temporal coverage across multiple seasons.

Method: Created RS3DBench dataset with precisely aligned remote sensing images and depth maps across diverse geographical contexts, and developed a depth estimation model using stable diffusion’s multimodal fusion capabilities.

Result: The proposed depth estimation model achieves state-of-the-art performance on the new benchmark dataset.

Conclusion: RS3DBench contributes significantly to advancing 3D visual perception models and geographic AI in remote sensing, with all resources made publicly available.

Abstract: In this paper, we introduce a novel benchmark designed to propel the advancement of general-purpose, large-scale 3D vision models for remote sensing imagery. While several datasets have been proposed within the realm of remote sensing, many existing collections either lack comprehensive depth information or fail to establish precise alignment between depth data and remote sensing images. To address this deficiency, we present a visual Benchmark for 3D understanding of Remotely Sensed images, dubbed RS3DBench. This dataset encompasses 54,951 pairs of remote sensing images and pixel-level aligned depth maps, accompanied by corresponding textual descriptions, spanning a broad array of geographical contexts. It serves as a tool for training and assessing 3D visual perception models within remote sensing image spatial understanding tasks. Furthermore, we introduce a remotely sensed depth estimation model derived from stable diffusion, harnessing its multimodal fusion capabilities, thereby delivering state-of-the-art performance on our dataset. Our endeavor seeks to make a profound contribution to the evolution of 3D visual perception models and the advancement of geographic artificial intelligence within the remote sensing domain. The dataset, models and code will be accessed on the https://rs3dbench.github.io.

Method: 1) Uses DUSt3R’s dense stereo module to obtain initial point clouds directly from blurred images, avoiding SfM and camera pose errors. 2) Integrates event streams to decode latent sharp images, providing supervision for scene reconstruction optimization in 3D Gaussian Splatting.

Result: Extensive experiments show DeblurSplat generates high-fidelity novel views with significant rendering efficiency improvements over state-of-the-art deblur 3D-GS methods.

Conclusion: The method successfully eliminates SfM requirements while leveraging event cameras’ advantages, achieving superior deblurring performance and efficiency in 3D scene reconstruction.

Abstract: In this paper, we propose the first Structure-from-Motion (SfM)-free deblurring 3D Gaussian Splatting method via event camera, dubbed DeblurSplat. We address the motion-deblurring problem in two ways. First, we leverage the pretrained capability of the dense stereo module (DUSt3R) to directly obtain accurate initial point clouds from blurred images. Without calculating camera poses as an intermediate result, we avoid the cumulative errors transfer from inaccurate camera poses to the initial point clouds’ positions. Second, we introduce the event stream into the deblur pipeline for its high sensitivity to dynamic change. By decoding the latent sharp images from the event stream and blurred images, we can provide a fine-grained supervision signal for scene reconstruction optimization. Extensive experiments across a range of scenes demonstrate that DeblurSplat not only excels in generating high-fidelity novel views but also achieves significant rendering efficiency compared to the SOTAs in deblur 3D-GS.

Method: Proposes MoiréNet with two key components: Directional Frequency-Spatial Encoder (DFSE) that identifies moiré orientation via directional difference convolution, and Frequency-Spatial Adaptive Selector (FSAS) for precise feature-adaptive suppression.

Result: Extensive experiments show MoiréNet achieves state-of-the-art performance on public datasets with only 5.513M parameters (48% reduction vs ESDNet-L), combining superior restoration quality with parameter efficiency.

Conclusion: MoiréNet’s efficient design makes it well-suited for resource-constrained applications like smartphone photography, industrial imaging, and augmented reality.

Abstract: Moir'e patterns arise from spectral aliasing between display pixel lattices and camera sensor grids, manifesting as anisotropic, multi-scale artifacts that pose significant challenges for digital image demoir'eing. We propose Moir'eNet, a convolutional neural U-Net-based framework that synergistically integrates frequency and spatial domain features for effective artifact removal. Moir'eNet introduces two key components: a Directional Frequency-Spatial Encoder (DFSE) that discerns moir'e orientation via directional difference convolution, and a Frequency-Spatial Adaptive Selector (FSAS) that enables precise, feature-adaptive suppression. Extensive experiments demonstrate that Moir'eNet achieves state-of-the-art performance on public and actively used datasets while being highly parameter-efficient. With only 5.513M parameters, representing a 48% reduction compared to ESDNet-L, Moir'eNet combines superior restoration quality with parameter efficiency, making it well-suited for resource-constrained applications including smartphone photography, industrial imaging, and augmented reality.

Method: Proposes FAVS with Frequency-Domain Enhanced Decomposer (FDED) for iterative frequency decomposition and Synergistic Cross-Modal Consistency (SCMC) with mixture-of-experts for semantic consistency and feature preservation.

Result: Achieves state-of-the-art performance on three benchmark datasets, with qualitative visualizations verifying module effectiveness.

Conclusion: Reformulating AVS as frequency-domain decomposition/recomposition problem with frequency-aware modules significantly improves segmentation performance by addressing modality-specific frequency characteristics.

Abstract: Audio-visual segmentation (AVS) plays a critical role in multimodal machine learning by effectively integrating audio and visual cues to precisely segment objects or regions within visual scenes. Recent AVS methods have demonstrated significant improvements. However, they overlook the inherent frequency-domain contradictions between audio and visual modalities–the pervasively interfering noise in audio high-frequency signals vs. the structurally rich details in visual high-frequency signals. Ignoring these differences can result in suboptimal performance. In this paper, we rethink the AVS task from a deeper perspective by reformulating AVS task as a frequency-domain decomposition and recomposition problem. To this end, we introduce a novel Frequency-Aware Audio-Visual Segmentation (FAVS) framework consisting of two key modules: Frequency-Domain Enhanced Decomposer (FDED) module and Synergistic Cross-Modal Consistency (SCMC) module. FDED module employs a residual-based iterative frequency decomposition to discriminate modality-specific semantics and structural features, and SCMC module leverages a mixture-of-experts architecture to reinforce semantic consistency and modality-specific feature preservation through dynamic expert routing. Extensive experiments demonstrate that our FAVS framework achieves state-of-the-art performance on three benchmark datasets, and abundant qualitative visualizations further verify the effectiveness of the proposed FDED and SCMC modules. The code will be released as open source upon acceptance of the paper.

Method: The paper surveys and analyzes four representative xAI approaches: (i) Saliency Maps, (ii) Concept Bottleneck Models (CBM), (iii) Prototype-based methods, and (iv) Hybrid approaches. It examines their underlying mechanisms, strengths, limitations, and evaluation metrics.

Result: The survey provides a comprehensive overview of current xAI methods for visual perception tasks, analyzing the trade-offs and capabilities of different interpretability approaches.

Conclusion: This comprehensive analysis of xAI approaches guides future research and applications in explainable AI for visual perception, helping address the interpretability challenges of deep learning models.

Abstract: Deep learning has become the de facto standard and dominant paradigm in image analysis tasks, achieving state-of-the-art performance. However, this approach often results in “black-box” models, whose decision-making processes are difficult to interpret, raising concerns about reliability in critical applications. To address this challenge and provide human a method to understand how AI model process and make decision, the field of xAI has emerged. This paper surveys four representative approaches in xAI for visual perception tasks: (i) Saliency Maps, (ii) Concept Bottleneck Models (CBM), (iii) Prototype-based methods, and (iv) Hybrid approaches. We analyze their underlying mechanisms, strengths and limitations, as well as evaluation metrics, thereby providing a comprehensive overview to guide future research and applications.

Method: The authors apply a DDPM with novel noise scheduling and time-step embedding techniques to generate synthetic LiDAR point clouds. These modifications improve the denoising process and the model’s temporal awareness for producing realistic point clouds.

Result: Extensive evaluation on IAMCV and KITTI-360 datasets using four performance metrics shows superior performance over most state-of-the-art methods. The model effectively mitigates effects of noisy and sparse LiDAR data, producing diverse point clouds with rich spatial relationships and structural detail.

Conclusion: The enhanced DDPM approach demonstrates effectiveness in generating high-quality synthetic LiDAR data for autonomous vehicle perception tasks, overcoming limitations of real-world data collection.

Abstract: Autonomous vehicles (AVs) are expected to revolutionize transportation by improving efficiency and safety. Their success relies on 3D vision systems that effectively sense the environment and detect traffic agents. Among sensors AVs use to create a comprehensive view of surroundings, LiDAR provides high-resolution depth data enabling accurate object detection, safe navigation, and collision avoidance. However, collecting real-world LiDAR data is time-consuming and often affected by noise and sparsity due to adverse weather or sensor limitations. This work applies a denoising diffusion probabilistic model (DDPM), enhanced with novel noise scheduling and time-step embedding techniques to generate high-quality synthetic data for augmentation, thereby improving performance across a range of computer vision tasks, particularly in AV perception. These modifications impact the denoising process and the model’s temporal awareness, allowing it to produce more realistic point clouds based on the projection. The proposed method was extensively evaluated under various configurations using the IAMCV and KITTI-360 datasets, with four performance metrics compared against state-of-the-art (SOTA) methods. The results demonstrate the model’s superior performance over most existing baselines and its effectiveness in mitigating the effects of noisy and sparse LiDAR data, producing diverse point clouds with rich spatial relationships and structural detail.

Method: Two-stage framework: (1) pretrain backbone by distilling knowledge from anomaly maps to capture defect-salient features; (2) pretrain detector using pseudo-defect boxes from anomaly maps. Uses knowledge-enhanced method to generate high-quality anomaly maps and collects 120,000-image industrial dataset.

Result: AGSSP consistently enhances performance across various settings, achieving up to 10% improvement in mAP@0.5 and 11.4% in mAP@0.5:0.95 compared to ImageNet-based models.

Conclusion: The proposed Anomaly-Guided Self-Supervised Pretraining paradigm effectively bridges the domain gap and improves metallic surface defect detection performance by leveraging anomaly priors to guide representation learning.

Abstract: The pretraining-finetuning paradigm is a crucial strategy in metallic surface defect detection for mitigating the challenges posed by data scarcity. However, its implementation presents a critical dilemma. Pretraining on natural image datasets such as ImageNet, faces a significant domain gap. Meanwhile, naive self-supervised pretraining on in-domain industrial data is often ineffective due to the inability of existing learning objectives to distinguish subtle defect patterns from complex background noise and textures. To resolve this, we introduce Anomaly-Guided Self-Supervised Pretraining (AGSSP), a novel paradigm that explicitly guides representation learning through anomaly priors. AGSSP employs a two-stage framework: (1) it first pretrains the model’s backbone by distilling knowledge from anomaly maps, encouraging the network to capture defect-salient features; (2) it then pretrains the detector using pseudo-defect boxes derived from these maps, aligning it with localization tasks. To enable this, we develop a knowledge-enhanced method to generate high-quality anomaly maps and collect a large-scale industrial dataset of 120,000 images. Additionally, we present two small-scale, pixel-level labeled metallic surface defect datasets for validation. Extensive experiments demonstrate that AGSSP consistently enhances performance across various settings, achieving up to a 10% improvement in mAP@0.5 and 11.4% in mAP@0.5:0.95 compared to ImageNet-based models. All code, pretrained models, and datasets are publicly available at https://clovermini.github.io/AGSSP-Dev/.

Method: Combines person-agnostic speech model with Universal Head Avatar Prior (UHAP) trained on cross-identity multi-view videos. Uses monocular encoder for efficient personalization and maps raw audio directly into UHAP latent expression space that encodes both geometry and appearance.

Result: Generates highly realistic avatars with precise lip synchronization, eyebrow movement, gaze shifts, and realistic mouth interior appearance. Outperforms competing geometry-only methods across lip-sync accuracy, image quality, and perceptual realism metrics.

Conclusion: This is the first generalizable audio-driven avatar model that accounts for detailed appearance modeling and rendering, demonstrating superior performance over existing approaches that only handle geometric variations.

Abstract: We introduce the first method for audio-driven universal photorealistic avatar synthesis, combining a person-agnostic speech model with our novel Universal Head Avatar Prior (UHAP). UHAP is trained on cross-identity multi-view videos. In particular, our UHAP is supervised with neutral scan data, enabling it to capture the identity-specific details at high fidelity. In contrast to previous approaches, which predominantly map audio features to geometric deformations only while ignoring audio-dependent appearance variations, our universal speech model directly maps raw audio inputs into the UHAP latent expression space. This expression space inherently encodes, both, geometric and appearance variations. For efficient personalization to new subjects, we employ a monocular encoder, which enables lightweight regression of dynamic expression variations across video frames. By accounting for these expression-dependent changes, it enables the subsequent model fine-tuning stage to focus exclusively on capturing the subject’s global appearance and geometry. Decoding these audio-driven expression codes via UHAP generates highly realistic avatars with precise lip synchronization and nuanced expressive details, such as eyebrow movement, gaze shifts, and realistic mouth interior appearance as well as motion. Extensive evaluations demonstrate that our method is not only the first generalizable audio-driven avatar model that can account for detailed appearance modeling and rendering, but it also outperforms competing (geometry-only) methods across metrics measuring lip-sync accuracy, quantitative image quality, and perceptual realism.

Method: Modular pipeline combining transformer-based detection, depth-tracking with spatial features, time-tracking using spatial consistency, and feature extraction for biologically relevant spine properties.

Result: Validated on open-source data and two new annotated datasets (detection/depth-tracking and time-tracking), with code and pre-trained weights released publicly.

Conclusion: Establishes a baseline for scalable, end-to-end analysis of dendritic spine dynamics, providing tools and datasets to advance research in this field.

Abstract: Dendritic spines are key structural components of excitatory synapses in the brain. Given the size of dendritic spines provides a proxy for synaptic efficacy, their detection and tracking across time is important for studies of the neural basis of learning and memory. Despite their relevance, large-scale analyses of the structural dynamics of dendritic spines in 3D+time microscopy data remain challenging and labor-intense. Here, we present a modular machine learning-based pipeline designed to automate the detection, time-tracking, and feature extraction of dendritic spines in volumes chronically recorded with two-photon microscopy. Our approach tackles the challenges posed by biological data by combining a transformer-based detection module, a depth-tracking component that integrates spatial features, a time-tracking module to associate 3D spines across time by leveraging spatial consistency, and a feature extraction unit that quantifies biologically relevant spine properties. We validate our method on open-source labeled spine data, and on two complementary annotated datasets that we publish alongside this work: one for detection and depth-tracking, and one for time-tracking, which, to the best of our knowledge, is the first data of this kind. To encourage future research, we release our data, code, and pre-trained weights at https://github.com/pamelaosuna/SynapFlow, establishing a baseline for scalable, end-to-end analysis of dendritic spine dynamics.

Method: Uses a confidence-based pseudo-labeling strategy to train a lightweight classifier directly on test data. A VLM identifies high-confidence samples, a pre-trained visual model enhances their representations, and these features iteratively train the classifier to capture complementary semantic and visual cues without supervision.

Result: Experimental evaluations on ten diverse datasets demonstrate that the approach outperforms baseline zero-shot methods.

Conclusion: The proposed framework effectively addresses data scarcity by combining VLMs and pre-trained visual models without requiring VLM fine-tuning or large language models, reducing dependence on semantic representation while achieving superior performance.

Abstract: While deep learning, including Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), has significantly advanced classification performance, its typical reliance on extensive annotated datasets presents a major obstacle in many practical scenarios where such data is scarce. Vision-language models (VLMs) and transfer learning with pre-trained visual models appear as promising techniques to deal with this problem. This paper proposes a novel zero-shot image classification framework that combines a VLM and a pre-trained visual model within a self-learning cycle. Requiring only the set of class names and no labeled training data, our method utilizes a confidence-based pseudo-labeling strategy to train a lightweight classifier directly on the test data, enabling dynamic adaptation. The VLM identifies high-confidence samples, and the pre-trained visual model enhances their visual representations. These enhanced features then iteratively train the classifier, allowing the system to capture complementary semantic and visual cues without supervision. Notably, our approach avoids VLM fine-tuning and the use of large language models, relying on the visual-only model to reduce the dependence on semantic representation. Experimental evaluations on ten diverse datasets demonstrate that our approach outperforms the baseline zero-shot method.

Method: Proposed ReflectiveGS, an extension of 3D Gaussian Splatting that utilizes mirror reflections as complementary viewpoints. Also created MirrorScene3D dataset with 1256 high-quality images and annotated mirror masks for benchmarking.

Result: ReflectiveGS outperforms existing methods in SSIM, PSNR, LPIPS metrics and training speed on the MirrorScene3D benchmark, demonstrating superior 3D reconstruction quality in mirror-rich environments.

Conclusion: The approach successfully leverages mirror reflections to enhance 3D reconstruction, setting a new benchmark for handling reflective surfaces and showing that reflections can be valuable information sources rather than artifacts to be eliminated.

Abstract: Mirror-containing environments pose unique challenges for 3D reconstruction and novel view synthesis (NVS), as reflective surfaces introduce view-dependent distortions and inconsistencies. While cutting-edge methods such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) excel in typical scenes, their performance deteriorates in the presence of mirrors. Existing solutions mainly focus on handling mirror surfaces through symmetry mapping but often overlook the rich information carried by mirror reflections. These reflections offer complementary perspectives that can fill in absent details and significantly enhance reconstruction quality. To advance 3D reconstruction in mirror-rich environments, we present MirrorScene3D, a comprehensive dataset featuring diverse indoor scenes, 1256 high-quality images, and annotated mirror masks, providing a benchmark for evaluating reconstruction methods in reflective settings. Building on this, we propose ReflectiveGS, an extension of 3D Gaussian Splatting that utilizes mirror reflections as complementary viewpoints rather than simple symmetry artifacts, enhancing scene geometry and recovering absent details. Experiments on MirrorScene3D show that ReflectiveGaussian outperforms existing methods in SSIM, PSNR, LPIPS, and training speed, setting a new benchmark for 3D reconstruction in mirror-rich environments.

Method: The authors constructed and annotated an image database to train the Yolo detection algorithm for bile duct localization. They used classical data augmentation techniques and proposed using Generative Adversarial Networks (GANs) to generate synthetic training data.

Result: Experimental results were discussed, though specific performance metrics are not provided in the abstract. The paper also includes ethical considerations regarding the approach.

Conclusion: The deep-learning approach using Yolo detection with GAN-generated synthetic data shows promise for improving bile duct visualization during laparoscopic cholecystectomy, potentially reducing the risk of bile duct injuries.

Abstract: Cholecystectomy is one of the most frequently performed procedures in gastrointestinal surgery, and the laparoscopic approach is the gold standard for symptomatic cholecystolithiasis and acute cholecystitis. In addition to the advantages of a significantly faster recovery and better cosmetic results, the laparoscopic approach bears a higher risk of bile duct injury, which has a significant impact on quality of life and survival. To avoid bile duct injury, it is essential to improve the intraoperative visualization of the bile duct. This work aims to address this problem by leveraging a deep-learning approach for the localization of the biliary tract from white-light images acquired during the surgical procedures. To this end, the construction and annotation of an image database to train the Yolo detection algorithm has been employed. Besides classical data augmentation techniques, the paper proposes Generative Adversarial Network (GAN) for the generation of a synthetic portion of the training dataset. Experimental results have been discussed along with ethical considerations.

Method: Proposes Prompt-DAS framework that incorporates auxiliary center-point detection and prompt-guided contrastive learning, allowing training with full points, sparse points, or no points on instances.

Result: Comprehensive experiments on challenging benchmarks demonstrate effectiveness over existing UDA, WDA, and SAM-based approaches.

Conclusion: Prompt-DAS provides a flexible and effective solution for domain adaptive segmentation of organelle instances, outperforming existing methods while requiring less annotation effort.

Abstract: Domain adaptive segmentation (DAS) of numerous organelle instances from large-scale electron microscopy (EM) is a promising way to enable annotation-efficient learning. Inspired by SAM, we propose a promptable multitask framework, namely Prompt-DAS, which is flexible enough to utilize any number of point prompts during the adaptation training stage and testing stage. Thus, with varying prompt configurations, Prompt-DAS can perform unsupervised domain adaptation (UDA) and weakly supervised domain adaptation (WDA), as well as interactive segmentation during testing. Unlike the foundation model SAM, which necessitates a prompt for each individual object instance, Prompt-DAS is only trained on a small dataset and can utilize full points on all instances, sparse points on partial instances, or even no points at all, facilitated by the incorporation of an auxiliary center-point detection task. Moreover, a novel prompt-guided contrastive learning is proposed to enhance discriminative feature learning. Comprehensive experiments conducted on challenging benchmarks demonstrate the effectiveness of the proposed approach over existing UDA, WDA, and SAM-based approaches.

Method: Proposes a framework with step-level reasoning data, process reward model (PRM), and reinforcement learning training to enable fine-grained reasoning evaluation and scaling.

Result: The models achieve strong baselines with consistent improvements on challenging vision-language benchmarks, with thorough empirical analysis revealing component impacts and inference-time scaling properties.

Conclusion: This work establishes a baseline for vision-language models and provides insights into complex multimodal reasoning, with all resources made publicly available.

Abstract: Chain of thought reasoning has demonstrated remarkable success in large language models, yet its adaptation to vision-language reasoning remains an open challenge with unclear best practices. Existing attempts typically employ reasoning chains at a coarse-grained level, which struggles to perform fine-grained structured reasoning and, more importantly, are difficult to evaluate the reward and quality of intermediate reasoning. In this work, we delve into chain of step reasoning for vision-language models, enabling assessing reasoning step quality accurately and leading to effective reinforcement learning and inference-time scaling with fine-grained rewards. We present a simple, effective, and fully transparent framework, including the step-level reasoning data, process reward model (PRM), and reinforcement learning training. With the proposed approaches, our models set strong baselines with consistent improvements on challenging vision-language benchmarks. More importantly, we conduct a thorough empirical analysis and ablation study, unveiling the impact of each component and several intriguing properties of inference-time scaling. We believe this paper serves as a baseline for vision-language models and offers insights into more complex multimodal reasoning. Our dataset, PRM, and code will be available at https://github.com/baaivision/CoS.

Method: Uses class activation maps (CAMs) from Swin Transformer ViT trained with image-level annotations as prompts for SAM. Combines image preprocessing with single-mask and multi-mask SAM generation strategies to enhance mask quality.

Result: Achieved mIoU of 0.54 on FoodSeg103 dataset, generating average of 2.4 masks per image (excluding background) in multi-mask scenario.

Conclusion: The approach can accelerate food image annotation tasks and serve as an integrated component in food and nutrition tracking applications.

Abstract: In this paper, we propose a weakly supervised semantic segmentation approach for food images which takes advantage of the zero-shot capabilities and promptability of the Segment Anything Model (SAM) along with the attention mechanisms of Vision Transformers (ViTs). Specifically, we use class activation maps (CAMs) from ViTs to generate prompts for SAM, resulting in masks suitable for food image segmentation. The ViT model, a Swin Transformer, is trained exclusively using image-level annotations, eliminating the need for pixel-level annotations during training. Additionally, to enhance the quality of the SAM-generated masks, we examine the use of image preprocessing techniques in combination with single-mask and multi-mask SAM generation strategies. The methodology is evaluated on the FoodSeg103 dataset, generating an average of 2.4 masks per image (excluding background), and achieving an mIoU of 0.54 for the multi-mask scenario. We envision the proposed approach as a tool to accelerate food image annotation tasks or as an integrated component in food and nutrition tracking applications.

Method: Proposes DyL-UNet with Echo-Dynamics Graph (EDG) for dynamic information extraction, multiple Swin-Transformer-based encoder-decoder branches, and Cardiac Phase-Dynamics Attention (CPDA) at skip connections to enforce temporal consistency.

Result: Extensive experiments on CAMUS and EchoNet-Dynamic datasets show DyL-UNet maintains comparable segmentation accuracy to existing methods while achieving superior temporal consistency.

Conclusion: DyL-UNet provides a reliable solution for automated clinical echocardiography by addressing temporal instability issues in cardiac anatomy segmentation.

Abstract: Accurate segmentation of cardiac anatomy in echocardiography is essential for cardiovascular diagnosis and treatment. Yet echocardiography is prone to deformation and speckle noise, causing frame-to-frame segmentation jitter. Even with high accuracy in single-frame segmentation, temporal instability can weaken functional estimates and impair clinical interpretability. To address these issues, we propose DyL-UNet, a dynamic learning-based temporal consistency U-Net segmentation architecture designed to achieve temporally stable and precise echocardiographic segmentation. The framework constructs an Echo-Dynamics Graph (EDG) through dynamic learning to extract dynamic information from videos. DyL-UNet incorporates multiple Swin-Transformer-based encoder-decoder branches for processing single-frame images. It further introduces Cardiac Phase-Dynamics Attention (CPDA) at the skip connections, which uses EDG-encoded dynamic features and cardiac-phase cues to enforce temporal consistency during segmentation. Extensive experiments on the CAMUS and EchoNet-Dynamic datasets demonstrate that DyL-UNet maintains segmentation accuracy comparable to existing methods while achieving superior temporal consistency, providing a reliable solution for automated clinical echocardiography.

Method: Proposed Sa2VA-i, which rectifies the identified inconsistencies in the original Sa2VA model while using the same checkpoints.

Result: Significant improvements: +11.6 J&F on MeViS, +1.4 on Ref-YT-VOS, +3.3 on Ref-DAVIS, +4.1 on ReVOS. Sa2VA-i-1B performs on par with original Sa2VA-26B on MeViS.

Conclusion: Highlights the importance of implementation details and provides valuable insights for referring video segmentation field.

Abstract: Sa2VA is a recent model for language-guided dense grounding in images and video that achieves state-of-the-art results on multiple segmentation benchmarks and that has become widely popular. However, we found that Sa2VA does not perform according to its full potential for referring video object segmentation tasks. We identify inconsistencies between training and inference procedures as the key factor holding it back. To mitigate this issue, we propose an improved version of Sa2VA, Sa2VA-i, that rectifies these issues and improves the results. In fact, Sa2VA-i sets a new state of the art for multiple video benchmarks and achieves improvements of up to +11.6 J&F on MeViS, +1.4 on Ref-YT-VOS, +3.3 on Ref-DAVIS and +4.1 on ReVOS using the same Sa2VA checkpoints. With our fixes, the Sa2VA-i-1B model even performs on par with the original Sa2VA-26B model on the MeViS benchmark. We hope that this work will show the importance of seemingly trivial implementation details and that it will provide valuable insights for the referring video segmentation field. We provide the code and updated models at https://github.com/kumuji/sa2va-i

Method: Proposes adapting inputs to the visual space of multimodal models and injecting domain-specific information as instructions, allowing models trained on RGB-only data to process multi-spectral imagery in zero-shot mode.

Result: Achieves strong zero-shot performance gains on popular remote sensing benchmarks for land cover and land use classification using Gemini2.5.

Conclusion: Enables geospatial professionals to leverage powerful multimodal models with specialized sensor data, benefiting from rich reasoning capabilities without costly training.

Abstract: Multi-spectral imagery plays a crucial role in diverse Remote Sensing applications including land-use classification, environmental monitoring and urban planning. These images are widely adopted because their additional spectral bands correlate strongly with physical materials on the ground, such as ice, water, and vegetation. This allows for more accurate identification, and their public availability from missions, such as Sentinel-2 and Landsat, only adds to their value. Currently, the automatic analysis of such data is predominantly managed through machine learning models specifically trained for multi-spectral input, which are costly to train and support. Furthermore, although providing a lot of utility for Remote Sensing, such additional inputs cannot be used with powerful generalist large multimodal models, which are capable of solving many visual problems, but are not able to understand specialized multi-spectral signals. To address this, we propose a training-free approach which introduces new multi-spectral data in a Zero-Shot-only mode, as inputs to generalist multimodal models, trained on RGB-only inputs. Our approach leverages the multimodal models’ understanding of the visual space, and proposes to adapt to inputs to that space, and to inject domain-specific information as instructions into the model. We exemplify this idea with the Gemini2.5 model and observe strong Zero-Shot performance gains of the approach on popular Remote Sensing benchmarks for land cover and land use classification and demonstrate the easy adaptability of Gemini2.5 to new inputs. These results highlight the potential for geospatial professionals, working with non-standard specialized inputs, to easily leverage powerful multimodal models, such as Gemini2.5, to accelerate their work, benefiting from their rich reasoning and contextual capabilities, grounded in the specialized sensor data.

Method: Evaluated MLLMs (Gemini 1.5/2.0, Gemma 3, Pixtral) on simulated DeepAccident dataset from CARLA, using enhanced prompts with YOLO for object detection, Deep SORT for multi-object tracking, and Segment Anything (SAM) for instance segmentation to improve accuracy.

Result: Pixtral achieved best performance with F1-score of 0.71 and 83% recall. Gemini models gained precision (Gemini 1.5 rose to 90%) but suffered F1 and recall losses. Gemma 3 offered most balanced performance with minimal metric fluctuation.

Conclusion: Integration of MLLMs with advanced visual analytics techniques shows substantial potential for enhancing real-world automated traffic monitoring systems, demonstrating effective zero-shot accident detection capabilities.

Abstract: Traffic safety remains a critical global concern, with timely and accurate accident detection essential for hazard reduction and rapid emergency response. Infrastructure-based vision sensors offer scalable and efficient solutions for continuous real-time monitoring, facilitating automated detection of acci- dents directly from captured images. This research investigates the zero-shot capabilities of multimodal large language models (MLLMs) for detecting and describing traffic accidents using images from infrastructure cameras, thus minimizing reliance on extensive labeled datasets. Main contributions include: (1) Evaluation of MLLMs using the simulated DeepAccident dataset from CARLA, explicitly addressing the scarcity of diverse, realistic, infrastructure-based accident data through controlled simulations; (2) Comparative performance analysis between Gemini 1.5 and 2.0, Gemma 3 and Pixtral models in acci- dent identification and descriptive capabilities without prior fine-tuning; and (3) Integration of advanced visual analytics, specifically YOLO for object detection, Deep SORT for multi- object tracking, and Segment Anything (SAM) for instance segmentation, into enhanced prompts to improve model accuracy and explainability. Key numerical results show Pixtral as the top performer with an F1-score of 0.71 and 83% recall, while Gemini models gained precision with enhanced prompts (e.g., Gemini 1.5 rose to 90%) but suffered notable F1 and recall losses. Gemma 3 offered the most balanced performance with minimal metric fluctuation. These findings demonstrate the substantial potential of integrating MLLMs with advanced visual analytics techniques, enhancing their applicability in real-world automated traffic monitoring systems.

Method: Extends Track-On into a causal transformer-based model that processes frames sequentially, maintains temporal coherence via memory mechanism, and uses coarse patch-level classification followed by refinement. Systematically studies synthetic training setups to improve temporal robustness.

Result: Achieves state-of-the-art results across five synthetic and real-world benchmarks, surpassing prior online trackers and even strong offline methods that use bidirectional context.

Conclusion: Causal, memory-based architectures trained purely on synthetic data are effective scalable solutions for real-world point tracking, demonstrating the viability of online approaches without requiring future frame access.

Abstract: In this paper, we consider the problem of long-term point tracking, which requires consistent identification of points across video frames under significant appearance changes, motion, and occlusion. We target the online setting, i.e. tracking points frame-by-frame, making it suitable for real-time and streaming applications. We extend our prior model Track-On into Track-On2, a simple and efficient transformer-based model for online long-term tracking. Track-On2 improves both performance and efficiency through architectural refinements, more effective use of memory, and improved synthetic training strategies. Unlike prior approaches that rely on full-sequence access or iterative updates, our model processes frames causally and maintains temporal coherence via a memory mechanism, which is key to handling drift and occlusions without requiring future frames. At inference, we perform coarse patch-level classification followed by refinement. Beyond architecture, we systematically study synthetic training setups and their impact on memory behavior, showing how they shape temporal robustness over long sequences. Through comprehensive experiments, Track-On2 achieves state-of-the-art results across five synthetic and real-world benchmarks, surpassing prior online trackers and even strong offline methods that exploit bidirectional context. These results highlight the effectiveness of causal, memory-based architectures trained purely on synthetic data as scalable solutions for real-world point tracking. Project page: https://kuis-ai.github.io/track_on2

Method: Uses rigorous calibration and hardware synchronization of multiple cameras and spectra for object detection, with all data annotated with metadata and mapped onto a world plane for area estimation.

Result: Achieves up to 80% reduction in dataset processing time and enables accurate surveyed area estimates with quick assessment of survey results.

Conclusion: KAMERA provides an efficient solution for wildlife detection and mapping, with all software, models, and schematics open-sourced to inspire similar efforts in the scientific community.

Abstract: We introduce KAMERA: a comprehensive system for multi-camera, multi-spectral synchronization and real-time detection of seals and polar bears. Utilized in aerial surveys for ice-associated seals in the Bering, Chukchi, and Beaufort seas around Alaska, KAMERA provides up to an 80% reduction in dataset processing time over previous methods. Our rigorous calibration and hardware synchronization enable using multiple spectra for object detection. All collected data are annotated with metadata so they can be easily referenced later. All imagery and animal detections from a survey are mapped onto a world plane for accurate surveyed area estimates and quick assessment of survey results. We hope KAMERA will inspire other mapping and detection efforts in the scientific community, with all software, models, and schematics fully open-sourced.

Method: NeuCODEX incorporates a learned spike-driven compression module to reduce data transmission and employs a dynamic early-exit mechanism to adaptively terminate inference based on output confidence. It was prototyped on ResNet-18 and VGG-16 backbones in a real edge-to-cloud testbed.

Result: The system reduces data transfer by up to 2048x, edge energy consumption by over 90%, and end-to-end latency by up to 3x compared to edge-only inference, with negligible accuracy drop (<2%).

Conclusion: NeuCODEX enables practical, high-performance SNN deployment in resource-constrained environments by effectively addressing the challenges of edge-cloud co-inference systems.

Abstract: Spiking Neural Networks (SNNs) offer significant potential for enabling energy-efficient intelligence at the edge. However, performing full SNN inference at the edge can be challenging due to the latency and energy constraints arising from fixed and high timestep overheads. Edge-cloud co-inference systems present a promising solution, but their deployment is often hindered by high latency and feature transmission costs. To address these issues, we introduce NeuCODEX, a neuromorphic co-inference architecture that jointly optimizes both spatial and temporal redundancy. NeuCODEX incorporates a learned spike-driven compression module to reduce data transmission and employs a dynamic early-exit mechanism to adaptively terminate inference based on output confidence. We evaluated NeuCODEX on both static images (CIFAR10 and Caltech) and neuromorphic event streams (CIFAR10-DVS and N-Caltech). To demonstrate practicality, we prototyped NeuCODEX on ResNet-18 and VGG-16 backbones in a real edge-to-cloud testbed. Our proposed system reduces data transfer by up to 2048x and edge energy consumption by over 90%, while reducing end-to-end latency by up to 3x compared to edge-only inference, all with a negligible accuracy drop of less than 2%. In doing so, NeuCODEX enables practical, high-performance SNN deployment in resource-constrained environments.

Method: Inject robust priors from visual foundation models into CNN feature extractors; use scene correspondence priors with synthetic stereo datasets containing clear/adverse image pairs; implement two-step training: robust self-supervised scene correspondence learning and adverse weather distillation.

Result: Extensive experiments show the method outperforms existing state-of-the-art self-supervised stereo matching methods under adverse weather conditions.

Conclusion: The proposed RoSe framework effectively improves stereo matching robustness in adverse weather by leveraging foundation model priors and scene correspondence learning, demonstrating superior performance over current methods.

Abstract: Recent self-supervised stereo matching methods have made significant progress, but their performance significantly degrades under adverse weather conditions such as night, rain, and fog. We identify two primary weaknesses contributing to this performance degradation. First, adverse weather introduces noise and reduces visibility, making CNN-based feature extractors struggle with degraded regions like reflective and textureless areas. Second, these degraded regions can disrupt accurate pixel correspondences, leading to ineffective supervision based on the photometric consistency assumption. To address these challenges, we propose injecting robust priors derived from the visual foundation model into the CNN-based feature extractor to improve feature representation under adverse weather conditions. We then introduce scene correspondence priors to construct robust supervisory signals rather than relying solely on the photometric consistency assumption. Specifically, we create synthetic stereo datasets with realistic weather degradations. These datasets feature clear and adverse image pairs that maintain the same semantic context and disparity, preserving the scene correspondence property. With this knowledge, we propose a robust self-supervised training paradigm, consisting of two key steps: robust self-supervised scene correspondence learning and adverse weather distillation. Both steps aim to align underlying scene results from clean and adverse image pairs, thus improving model disparity estimation under adverse weather effects. Extensive experiments demonstrate the effectiveness and versatility of our proposed solution, which outperforms existing state-of-the-art self-supervised methods. Codes are available at \textcolor{blue}{https://github.com/cocowy1/RoSe-Robust-Self-supervised-Stereo-Matching-under-Adverse-Weather-Conditions}.

Method: Proposed YOLO-LAN pipeline using YOLO-based architecture trained with M2IoU loss, versatile data augmentations, and negative data to replicate real clinical situations.

Result: Achieved mAP${50}$ of 0.9619 and mAP${50:95}$ of 0.8599 with YOLOv12, and mAP${50}$ of 0.9540 and mAP${50:95}$ of 0.8487 with YOLOv8 on Kvasir-seg dataset, showing significant improvement in precision.

Conclusion: The pipeline demonstrates robustness in polyp size and precise location detection, making it clinically relevant for AI-assisted colorectal screening.

Abstract: Colorectal cancer (CRC), a lethal disease, begins with the growth of abnormal mucosal cell proliferation called polyps in the inner wall of the colon. When left undetected, polyps can become malignant tumors. Colonoscopy is the standard procedure for detecting polyps, as it enables direct visualization and removal of suspicious lesions. Manual detection by colonoscopy can be inconsistent and is subject to oversight. Therefore, object detection based on deep learning offers a better solution for a more accurate and real-time diagnosis during colonoscopy. In this work, we propose YOLO-LAN, a YOLO-based polyp detection pipeline, trained using M2IoU loss, versatile data augmentations and negative data to replicate real clinical situations. Our pipeline outperformed existing methods for the Kvasir-seg and BKAI-IGH NeoPolyp datasets, achieving mAP${50}$ of 0.9619, mAP${50:95}$ of 0.8599 with YOLOv12 and mAP${50}$ of 0.9540, mAP${50:95}$ of 0.8487 with YOLOv8 on the Kvasir-seg dataset. The significant increase is achieved in mAP$_{50:95}$ score, showing the precision of polyp detection. We show robustness based on polyp size and precise location detection, making it clinically relevant in AI-assisted colorectal screening.

Method: Analysis and adaptation of SeC framework with long-term memory for temporal continuity and concept-aware memory for semantic priors to suppress distractors.

Result: Achieved 39.89% JF score on the test set, ranking 1st in the MOSEv2 track of LSVOS Challenge.

Conclusion: The combination of long-term memory and concept-aware memory effectively addresses core MOSEv2 challenges, demonstrating strong performance in semi-supervised video object segmentation.

Abstract: This technical report explores the MOSEv2 track of the LSVOS Challenge, which targets complex semi-supervised video object segmentation. By analysing and adapting SeC, an enhanced SAM-2 framework, we conduct a detailed study of its long-term memory and concept-aware memory, showing that long-term memory preserves temporal continuity under occlusion and reappearance, while concept-aware memory supplies semantic priors that suppress distractors; together, these traits directly benefit several MOSEv2’s core challenges. Our solution achieves a JF score of 39.89% on the test set, ranking 1st in the MOSEv2 track of the LSVOS Challenge.

Method: Deconstructs VLM visual processing into object recognition (converting images to text token maps) and spatial perception (theoretical derivation of geometric structure). Proposes token compression algorithm and RoPE scaling technique.

Result: Reveals object recognition unfolds as two-stage process (attribute recognition to semantic disambiguation) and verifies geometric structure of positional representation. Improves decoding efficiency and spatial reasoning.

Conclusion: Validates the analytical framework, provides deeper understanding of VLM internals, and offers principles for designing more capable future architectures.

Abstract: Vision-Language Models (VLMs) have demonstrated remarkable performance across a variety of real-world tasks. However, existing VLMs typically process visual information by serializing images, a method that diverges significantly from the parallel nature of human vision. Moreover, their opaque internal mechanisms hinder both deeper understanding and architectural innovation. Inspired by the dual-stream hypothesis of human vision, which distinguishes the “what” and “where” pathways, we deconstruct the visual processing in VLMs into object recognition and spatial perception for separate study. For object recognition, we convert images into text token maps and find that the model’s perception of image content unfolds as a two-stage process from shallow to deep layers, beginning with attribute recognition and culminating in semantic disambiguation. For spatial perception, we theoretically derive and empirically verify the geometric structure underlying the positional representation in VLMs. Based on these findings, we introduce an instruction-agnostic token compression algorithm based on a plug-and-play visual decoder to improve decoding efficiency, and a RoPE scaling technique to enhance spatial reasoning. Through rigorous experiments, our work validates these analyses, offering a deeper understanding of VLM internals and providing clear principles for designing more capable future architectures.

Method: Proposes a paradigm shift from text-to-image to text-to-text retrieval using structured image descriptions generated by VLLMs. Uses a single encoder architecture with minimal calibration (few hours on two GPUs) and releases new benchmarks (subFlickr and subCOCO) for better compositionality evaluation.

Result: The vision-free retriever matches or surpasses traditional multimodal models, achieves state-of-the-art zero-shot performance on multiple benchmarks, and works effectively with small models (0.3B parameters). It reduces modality gap, improves compositionality, and performs better on short/long caption queries.

Conclusion: Vision encoders may not be necessary for retrieval tasks. The text-to-text paradigm with structured descriptions offers significant advantages including reduced modality gap, improved privacy, better compositionality, and computational efficiency while maintaining high performance.

Abstract: Contrastively-trained Vision-Language Models (VLMs), such as CLIP, have become the standard approach for learning discriminative vision-language representations. However, these models often exhibit shallow language understanding, manifesting bag-of-words behaviour. These limitations are reinforced by their dual-encoder design, which induces a modality gap. Additionally, the reliance on vast web-collected data corpora for training makes the process computationally expensive and introduces significant privacy concerns. To address these limitations, in this work, we challenge the necessity of vision encoders for retrieval tasks by introducing a vision-free, single-encoder retrieval pipeline. Departing from the traditional text-to-image retrieval paradigm, we migrate to a text-to-text paradigm with the assistance of VLLM-generated structured image descriptions. We demonstrate that this paradigm shift has significant advantages, including a substantial reduction of the modality gap, improved compositionality, and better performance on short and long caption queries, all attainable with only a few hours of calibration on two GPUs. Additionally, substituting raw images with textual descriptions introduces a more privacy-friendly alternative for retrieval. To further assess generalisation and address some of the shortcomings of prior compositionality benchmarks, we release two benchmarks derived from Flickr30k and COCO, containing diverse compositional queries made of short captions, which we coin subFlickr and subCOCO. Our vision-free retriever matches and often surpasses traditional multimodal models. Importantly, our approach achieves state-of-the-art zero-shot performance on multiple retrieval and compositionality benchmarks, with models as small as 0.3B parameters. Code is available at: https://github.com/IoannaNti/LexiCLIP

Method: Train and evaluate models targeting compositionality and long-caption understanding, examining bidirectional relationships and sensitivity to data quality and design choices.

Result: Bidirectional relationship found: compositional training improves long-caption retrieval, and long-caption training promotes compositionality. Gains are sensitive to data quality and model parameters.

Conclusion: Compositional understanding and long-caption understanding are intertwined capabilities that can be jointly learned through high-quality dense descriptions, offering practical guidance for VLM improvement.

Abstract: Contrastive vision-language models (VLMs) have made significant progress in binding visual and textual information, but understanding long, dense captions remains an open challenge. We hypothesize that compositionality, the capacity to reason about object-attribute bindings and inter-object relationships, is key to understanding longer captions. In this paper, we investigate the interaction between compositionality and long-caption understanding, asking whether training for one property enhances the other. We train and evaluate a range of models that target each of these capabilities. Our results reveal a bidirectional relationship: compositional training improves performance on long-caption retrieval, and training on long captions promotes compositionality. However, these gains are sensitive to data quality and model design. We find that training on poorly structured captions, or with limited parameter updates, fails to support generalization. Likewise, strategies that aim at retaining general alignment, such as freezing positional embeddings, do not improve compositional understanding. Overall, we find that compositional understanding and long-caption understanding are intertwined capabilities that can be jointly learned through training on dense, grounded descriptions. Despite these challenges, we show that models trained on high-quality, long-caption data can achieve strong performance in both tasks, offering practical guidance for improving VLM generalization.

Method: Trained models on 1,128 synthetic images with crop/weed mixtures, integrated 5 real segmented field images, used CycleGAN-turbo for RGB-to-thermal translation and cross-modal alignment.

Result: Maximum relative improvement of 22% for weed class and 17% for plant class compared to full real-data baseline when combining synthetic data with few real images.

Conclusion: Combining synthetic data with limited manual annotations and cross-domain translation via generative models significantly boosts segmentation performance in complex field environments.

Abstract: Accurate plant segmentation in thermal imagery remains a significant challenge for high throughput field phenotyping, particularly in outdoor environments where low contrast between plants and weeds and frequent occlusions hinder performance. To address this, we present a framework that leverages synthetic RGB imagery, a limited set of real annotations, and GAN-based cross-modality alignment to enhance semantic segmentation in thermal images. We trained models on 1,128 synthetic images containing complex mixtures of crop and weed plants in order to generate image segmentation masks for crop and weed plants. We additionally evaluated the benefit of integrating as few as five real, manually segmented field images within the training process using various sampling strategies. When combining all the synthetic images with a few labeled real images, we observed a maximum relative improvement of 22% for the weed class and 17% for the plant class compared to the full real-data baseline. Cross-modal alignment was enabled by translating RGB to thermal using CycleGAN-turbo, allowing robust template matching without calibration. Results demonstrated that combining synthetic data with limited manual annotations and cross-domain translation via generative models can significantly boost segmentation performance in complex field environments for multi-model imagery.

Method: Created HyKid dataset from 48 pediatric patients with 3D MRIs reconstructed from routine low-resolution images using slice-to-volume algorithm, with manual segmentations by neurologist and clinical data extraction using Retrieval-Augmented Generation framework.

Result: Found strong correlation between choroid plexus volume and total CSF volume, achieving excellent predictive performance (AUC = 0.87) for hydrocephalus evaluation.

Conclusion: HyKid provides a high-quality benchmark for neuroimaging algorithm development and reveals choroid plexus-related features in hydrocephalus assessments, with publicly available dataset.

Abstract: Evaluation of hydrocephalus in children is challenging, and the related research is limited by a lack of publicly available, expert-annotated datasets, particularly those with segmentation of the choroid plexus. To address this, we present HyKid, an open-source dataset from 48 pediatric patients with hydrocephalus. 3D MRIs were provided with 1mm isotropic resolution, which was reconstructed from routine low-resolution images using a slice-to-volume algorithm. Manually corrected segmentations of brain tissues, including white matter, grey matter, lateral ventricle, external CSF, and the choroid plexus, were provided by an experienced neurologist. Additionally, structured data was extracted from clinical radiology reports using a Retrieval-Augmented Generation framework. The strong correlation between choroid plexus volume and total CSF volume provided a potential biomarker for hydrocephalus evaluation, achieving excellent performance in a predictive model (AUC = 0.87). The proposed HyKid dataset provided a high-quality benchmark for neuroimaging algorithms development, and it revealed the choroid plexus-related features in hydrocephalus assessments. Our datasets are publicly available at https://www.synapse.org/Synapse:syn68544889.

Method: Proposes MsFIN with three layers: multi-scale feature aggregation using short/mid/long-term temporal scales with Transformer for feature interactions, temporal feature processing under causal constraints, and multi-scale feature post fusion to generate comprehensive risk representations.

Result: Experiments on DAD and DADA datasets show MsFIN significantly outperforms state-of-the-art single-scale models in both prediction correctness and earliness. Ablation studies confirm each module’s effectiveness.

Conclusion: MsFIN achieves superior performance through multi-scale feature fusion and contextual interaction modeling, demonstrating the importance of comprehensive temporal scale analysis for early accident anticipation.

Abstract: With the widespread deployment of dashcams and advancements in computer vision, developing accident prediction models from the dashcam perspective has become critical for proactive safety interventions. However, two key challenges persist: modeling feature-level interactions among traffic participants (often occluded in dashcam views) and capturing complex, asynchronous multi-temporal behavioral cues preceding accidents. To deal with these two challenges, a Multi-scale Feature Interaction Network (MsFIN) is proposed for early-stage accident anticipation from dashcam videos. MsFIN has three layers for multi-scale feature aggregation, temporal feature processing and multi-scale feature post fusion, respectively. For multi-scale feature aggregation, a Multi-scale Module is designed to extract scene representations at short-term, mid-term and long-term temporal scales. Meanwhile, the Transformer architecture is leveraged to facilitate comprehensive feature interactions. Temporal feature processing captures the sequential evolution of scene and object features under causal constraints. In the multi-scale feature post fusion stage, the network fuses scene and object features across multiple temporal scales to generate a comprehensive risk representation. Experiments on DAD and DADA datasets show that MsFIN significantly outperforms state-of-the-art models with single-scale feature extraction in both prediction correctness and earliness. Ablation studies validate the effectiveness of each module in MsFIN, highlighting how the network achieves superior performance through multi-scale feature fusion and contextual interaction modeling.

Method: Uses a Developmental MoE architecture with LoRA models organized into Real-LoRA (for real faces) and multiple Fake-LoRAs (for different forgery types). Implements orthogonal gradient integration to prevent forgetting and interference.

Result: Experimental results show effectiveness under both datasets and manipulation types incremental protocols, demonstrating successful adaptation to new forgery types.

Conclusion: The proposed continual learning framework effectively addresses the challenge of evolving face forgery techniques by enabling incremental learning while maintaining detection capabilities for previously learned forgery types.

Abstract: The rise of realistic digital face generation and manipulation poses significant social risks. The primary challenge lies in the rapid and diverse evolution of generation techniques, which often outstrip the detection capabilities of existing models. To defend against the ever-evolving new types of forgery, we need to enable our model to quickly adapt to new domains with limited computation and data while avoiding forgetting previously learned forgery types. In this work, we posit that genuine facial samples are abundant and relatively stable in acquisition methods, while forgery faces continuously evolve with the iteration of manipulation techniques. Given the practical infeasibility of exhaustively collecting all forgery variants, we frame face forgery detection as a continual learning problem and allow the model to develop as new forgery types emerge. Specifically, we employ a Developmental Mixture of Experts (MoE) architecture that uses LoRA models as its individual experts. These experts are organized into two groups: a Real-LoRA to learn and refine knowledge of real faces, and multiple Fake-LoRAs to capture incremental information from different forgery types. To prevent catastrophic forgetting, we ensure that the learning direction of Fake-LoRAs is orthogonal to the established subspace. Moreover, we integrate orthogonal gradients into the orthogonal loss of Fake-LoRAs, preventing gradient interference throughout the training process of each task. Experimental results under both the datasets and manipulation types incremental protocols demonstrate the effectiveness of our method.

Method: Lavida-O introduces several novel techniques: Elastic Mixture-of-Transformer architecture, universal text conditioning, and stratified sampling. It’s the first unified MDM that uses understanding capabilities to improve image generation through planning and iterative self-reflection.

Result: Lavida-O achieves state-of-the-art performance on benchmarks including RefCOCO object grounding, GenEval text-to-image generation, and ImgEdit image editing. It outperforms models like Qwen2.5-VL and FluxKontext-dev while offering significant inference speedup.

Conclusion: The proposed Lavida-O demonstrates superior capabilities in unified multi-modal understanding and generation, setting new standards for performance and efficiency in complex image tasks through its innovative architectural and training approaches.

Abstract: We proposed Lavida-O, a unified multi-modal Masked Diffusion Model (MDM) capable of image understanding and generation tasks. Unlike existing multimodal diffsion language models such as MMaDa and Muddit which only support simple image-level understanding tasks and low-resolution image generation, Lavida-O exhibits many new capabilities such as object grounding, image-editing, and high-resolution (1024px) image synthesis. It is also the first unified MDM that uses its understanding capabilities to improve image generation and editing results through planning and iterative self-reflection. To allow effective and efficient training and sampling, Lavida-O ntroduces many novel techniques such as Elastic Mixture-of-Transformer architecture, universal text conditioning, and stratified sampling. \ours~achieves state-of-the-art performance on a wide range of benchmarks such as RefCOCO object grounding, GenEval text-to-image generation, and ImgEdit image editing, outperforming existing autoregressive and continuous diffusion models such as Qwen2.5-VL and FluxKontext-dev, while offering considerable speedup at inference.

Method: The authors propose Concept-based Video Similarity estimation (ConViS) that uses Large Multimodal Models to compute similarity scores across predefined semantic concepts, and create ConViS-Bench benchmark with annotated video pairs for evaluation.

Result: Benchmarking reveals significant performance differences among state-of-the-art models on ConViS, showing that some concepts are more challenging for video similarity estimation than others.

Conclusion: ConViS-Bench serves as a valuable resource for advancing language-driven video understanding research by enabling more nuanced, human-like video similarity assessment.

Abstract: What does it mean for two videos to be similar? Videos may appear similar when judged by the actions they depict, yet entirely different if evaluated based on the locations where they were filmed. While humans naturally compare videos by taking different aspects into account, this ability has not been thoroughly studied and presents a challenge for models that often depend on broad global similarity scores. Large Multimodal Models (LMMs) with video understanding capabilities open new opportunities for leveraging natural language in comparative video tasks. We introduce Concept-based Video Similarity estimation (ConViS), a novel task that compares pairs of videos by computing interpretable similarity scores across a predefined set of key semantic concepts. ConViS allows for human-like reasoning about video similarity and enables new applications such as concept-conditioned video retrieval. To support this task, we also introduce ConViS-Bench, a new benchmark comprising carefully annotated video pairs spanning multiple domains. Each pair comes with concept-level similarity scores and textual descriptions of both differences and similarities. Additionally, we benchmark several state-of-the-art models on ConViS, providing insights into their alignment with human judgments. Our results reveal significant performance differences on ConViS, indicating that some concepts present greater challenges for estimating video similarity. We believe that ConViS-Bench will serve as a valuable resource for advancing research in language-driven video understanding.

Method: Combines dense motion tokenization with adversarial refinement to eliminate reconstruction artifacts like motion smearing and temporal misalignment in spatio-temporal heatmaps.

Result: Outperforms dVAE baseline by 9.31% SSIM and reduces temporal instability by 37.1% on CMU Panoptic dataset. Shows 2D motion requires 128-token vocabulary while 3D motion needs 1024-token codebook for optimal representation.

Conclusion: Establishes practical deployment feasibility for diverse motion analysis applications with superior compression quality and motion preservation.

Abstract: Continuous human motion understanding remains a core challenge in computer vision due to its high dimensionality and inherent redundancy. Efficient compression and representation are crucial for analyzing complex motion dynamics. In this work, we introduce an adversarially-refined VQ-GAN framework with dense motion tokenization for compressing spatio-temporal heatmaps while preserving the fine-grained traces of human motion. Our approach combines dense motion tokenization with adversarial refinement, which eliminates reconstruction artifacts like motion smearing and temporal misalignment observed in non-adversarial baselines. Our experiments on the CMU Panoptic dataset provide conclusive evidence of our method’s superiority, outperforming the dVAE baseline by 9.31% SSIM and reducing temporal instability by 37.1%. Furthermore, our dense tokenization strategy enables a novel analysis of motion complexity, revealing that 2D motion can be optimally represented with a compact 128-token vocabulary, while 3D motion’s complexity demands a much larger 1024-token codebook for faithful reconstruction. These results establish practical deployment feasibility across diverse motion analysis applications. The code base for this work is available at https://github.com/TeCSAR-UNCC/Pose-Quantization.

Method: GrRAiL identifies clusters of sub-regions using per-voxel radiomic measurements, then computes graph-theoretic metrics to quantify spatial associations among clusters, creating weighted graphs that encode higher-order spatial relationships within lesions.

Result: In multi-institutional evaluations across three use cases (glioblastoma, brain metastasis, pancreatic IPMNs), GrRAiL consistently outperformed state-of-the-art baselines with test accuracies of 78%, 74%, and 75% respectively, showing >10% improvements over comparators.

Conclusion: GrRAiL provides a clinically feasible approach for characterizing intralesional heterogeneity that reliably captures spatial relationships and significantly improves differentiation of malignant neoplasms from confounding pathologies on MRI.

Abstract: A significant challenge in solid tumors is reliably distinguishing confounding pathologies from malignant neoplasms on routine imaging. While radiomics methods seek surrogate markers of lesion heterogeneity on CT/MRI, many aggregate features across the region of interest (ROI) and miss complex spatial relationships among varying intensity compositions. We present a new Graph-Radiomic Learning (GrRAiL) descriptor for characterizing intralesional heterogeneity (ILH) on clinical MRI scans. GrRAiL (1) identifies clusters of sub-regions using per-voxel radiomic measurements, then (2) computes graph-theoretic metrics to quantify spatial associations among clusters. The resulting weighted graphs encode higher-order spatial relationships within the ROI, aiming to reliably capture ILH and disambiguate confounding pathologies from malignancy. To assess efficacy and clinical feasibility, GrRAiL was evaluated in n=947 subjects spanning three use cases: differentiating tumor recurrence from radiation effects in glioblastoma (GBM; n=106) and brain metastasis (n=233), and stratifying pancreatic intraductal papillary mucinous neoplasms (IPMNs) into no+low vs high risk (n=608). In a multi-institutional setting, GrRAiL consistently outperformed state-of-the-art baselines - Graph Neural Networks (GNNs), textural radiomics, and intensity-graph analysis. In GBM, cross-validation (CV) and test accuracies for recurrence vs pseudo-progression were 89% and 78% with >10% test-accuracy gains over comparators. In brain metastasis, CV and test accuracies for recurrence vs radiation necrosis were 84% and 74% (>13% improvement). For IPMN risk stratification, CV and test accuracies were 84% and 75%, showing >10% improvement.

Method: The approach decouples learning: first training a motion prior model on large motion capture data, then training a policy using Q-learning to map egocentric visual inputs to high-level control commands for the motion prior.

Result: Experiments show that egocentric vision enables human-like motion characteristics, such as obstacle avoidance based on visual input. The avatars successfully navigate while avoiding obstacles in their visual field.

Conclusion: Equipping avatars with human-like sensors, especially egocentric vision, is promising for training avatars that behave like humans, as it fundamentally shapes motion generation in a human-like way.

Abstract: The way we perceive the world fundamentally shapes how we move, whether it is how we navigate in a room or how we interact with other humans. Current human motion generation methods, neglect this interdependency and use task-specific ``perception’’ that differs radically from that of humans. We argue that the generation of human-like avatar behavior requires human-like perception. Consequently, in this work we present CLOPS, the first human avatar that solely uses egocentric vision to perceive its surroundings and navigate. Using vision as the primary driver of motion however, gives rise to a significant challenge for training avatars: existing datasets have either isolated human motion, without the context of a scene, or lack scale. We overcome this challenge by decoupling the learning of low-level motion skills from learning of high-level control that maps visual input to motion. First, we train a motion prior model on a large motion capture dataset. Then, a policy is trained using Q-learning to map egocentric visual inputs to high-level control commands for the motion prior. Our experiments empirically demonstrate that egocentric vision can give rise to human-like motion characteristics in our avatars. For example, the avatars walk such that they avoid obstacles present in their visual field. These findings suggest that equipping avatars with human-like sensors, particularly egocentric vision, holds promise for training avatars that behave like humans.

Method: The authors introduce OverLayScore to quantify overlapping complexity, create OverLayBench benchmark with balanced OverLayScore distribution, and propose CreatiLayout-AM, a model fine-tuned on curated amodal mask dataset to handle complex overlaps.

Result: Analysis shows existing benchmarks are biased toward simpler cases with low OverLayScore values, limiting their effectiveness in evaluating model performance under challenging overlapping conditions.

Conclusion: The contributions provide groundwork for more robust layout-to-image generation under realistic and challenging overlapping scenarios, with the new benchmark and metric enabling better evaluation and improvement of methods.

Abstract: Despite steady progress in layout-to-image generation, current methods still struggle with layouts containing significant overlap between bounding boxes. We identify two primary challenges: (1) large overlapping regions and (2) overlapping instances with minimal semantic distinction. Through both qualitative examples and quantitative analysis, we demonstrate how these factors degrade generation quality. To systematically assess this issue, we introduce OverLayScore, a novel metric that quantifies the complexity of overlapping bounding boxes. Our analysis reveals that existing benchmarks are biased toward simpler cases with low OverLayScore values, limiting their effectiveness in evaluating model performance under more challenging conditions. To bridge this gap, we present OverLayBench, a new benchmark featuring high-quality annotations and a balanced distribution across different levels of OverLayScore. As an initial step toward improving performance on complex overlaps, we also propose CreatiLayout-AM, a model fine-tuned on a curated amodal mask dataset. Together, our contributions lay the groundwork for more robust layout-to-image generation under realistic and challenging scenarios. Project link: https://mlpc-ucsd.github.io/OverLayBench.

Method: Augments RGB decoder with a 3DGS decoder supervised by RGB decoder output. The 3DGS decoder is trained purely with synthetic data from video diffusion models. Supports text-to-3D and image-to-3D generation, and extends to dynamic 3D scenes from monocular video.

Result: Achieves state-of-the-art performance in both static and dynamic 3D scene generation. Enables real-time rendering from text prompts or single images.

Conclusion: The framework successfully bridges 2D video diffusion models with explicit 3D representations, eliminating the need for multi-view training data while maintaining high-quality 3D scene generation capabilities.

Abstract: The ability to generate virtual environments is crucial for applications ranging from gaming to physical AI domains such as robotics, autonomous driving, and industrial AI. Current learning-based 3D reconstruction methods rely on the availability of captured real-world multi-view data, which is not always readily available. Recent advancements in video diffusion models have shown remarkable imagination capabilities, yet their 2D nature limits the applications to simulation where a robot needs to navigate and interact with the environment. In this paper, we propose a self-distillation framework that aims to distill the implicit 3D knowledge in the video diffusion models into an explicit 3D Gaussian Splatting (3DGS) representation, eliminating the need for multi-view training data. Specifically, we augment the typical RGB decoder with a 3DGS decoder, which is supervised by the output of the RGB decoder. In this approach, the 3DGS decoder can be purely trained with synthetic data generated by video diffusion models. At inference time, our model can synthesize 3D scenes from either a text prompt or a single image for real-time rendering. Our framework further extends to dynamic 3D scene generation from a monocular input video. Experimental results show that our framework achieves state-of-the-art performance in static and dynamic 3D scene generation.

Method: Created a synthetic parametric geometric-illusion dataset and evaluated three multi-source learning strategies that combine illusion recognition tasks with ImageNet classification objectives.

Result: Incorporating geometric illusions as auxiliary supervision systematically improves generalization, particularly for intricate contours and fine textures. Perceptually driven biases enhance structural sensitivity in both CNN and transformer architectures.

Conclusion: This work demonstrates a novel integration of perceptual science and machine learning, suggesting new directions for embedding perceptual priors into vision model design.

Abstract: Contemporary deep learning models have achieved impressive performance in image classification by primarily leveraging statistical regularities within large datasets, but they rarely incorporate structured insights drawn directly from perceptual psychology. To explore the potential of perceptually motivated inductive biases, we propose integrating classic geometric visual illusions well-studied phenomena from human perception into standard image-classification training pipelines. Specifically, we introduce a synthetic, parametric geometric-illusion dataset and evaluate three multi-source learning strategies that combine illusion recognition tasks with ImageNet classification objectives. Our experiments reveal two key conceptual insights: (i) incorporating geometric illusions as auxiliary supervision systematically improves generalization, especially in visually challenging cases involving intricate contours and fine textures; and (ii) perceptually driven inductive biases, even when derived from synthetic stimuli traditionally considered unrelated to natural image recognition, can enhance the structural sensitivity of both CNN and transformer-based architectures. These results demonstrate a novel integration of perceptual science and machine learning and suggest new directions for embedding perceptual priors into vision model design.

Method: VolSplat replaces pixel alignment with voxel-aligned Gaussians by directly predicting Gaussians from a 3D voxel grid, overcoming 2D feature matching issues and enabling adaptive density control based on 3D scene complexity.

Result: Experiments on RealEstate10K and ScanNet show VolSplat achieves state-of-the-art performance with more plausible Gaussian reconstructions, improved geometric consistency, and enhanced novel-view rendering quality.

Conclusion: VolSplat establishes a more scalable framework for feed-forward 3D reconstruction with denser, more robust representations, paving the way for further research in wider communities.

Abstract: Feed-forward 3D Gaussian Splatting (3DGS) has emerged as a highly effective solution for novel view synthesis. Existing methods predominantly rely on a pixel-aligned Gaussian prediction paradigm, where each 2D pixel is mapped to a 3D Gaussian. We rethink this widely adopted formulation and identify several inherent limitations: it renders the reconstructed 3D models heavily dependent on the number of input views, leads to view-biased density distributions, and introduces alignment errors, particularly when source views contain occlusions or low texture. To address these challenges, we introduce VolSplat, a new multi-view feed-forward paradigm that replaces pixel alignment with voxel-aligned Gaussians. By directly predicting Gaussians from a predicted 3D voxel grid, it overcomes pixel alignment’s reliance on error-prone 2D feature matching, ensuring robust multi-view consistency. Furthermore, it enables adaptive control over Gaussian density based on 3D scene complexity, yielding more faithful Gaussian point clouds, improved geometric consistency, and enhanced novel-view rendering quality. Experiments on widely used benchmarks including RealEstate10K and ScanNet demonstrate that VolSplat achieves state-of-the-art performance while producing more plausible and view-consistent Gaussian reconstructions. In addition to superior results, our approach establishes a more scalable framework for feed-forward 3D reconstruction with denser and more robust representations, paving the way for further research in wider communities. The video results, code and trained models are available on our project page: https://lhmd.top/volsplat.

Method: Proposes Condition-Aware Reparameterization for Flow Matching (CAR-Flow) - a learned shift that conditions the source, target, or both distributions to shorten the probability path the model must learn.

Result: On ImageNet-256, CAR-Flow reduces FID from 2.07 to 1.68 when equipped with SiT-XL/2, while adding less than 0.6% additional parameters. Also validated on low-dimensional synthetic data.

Conclusion: CAR-Flow effectively improves conditional generative modeling by reducing the learning burden on models through strategic conditioning of distributions, leading to faster training and better performance with minimal parameter overhead.

Abstract: Conditional generative modeling aims to learn a conditional data distribution from samples containing data-condition pairs. For this, diffusion and flow-based methods have attained compelling results. These methods use a learned (flow) model to transport an initial standard Gaussian noise that ignores the condition to the conditional data distribution. The model is hence required to learn both mass transport and conditional injection. To ease the demand on the model, we propose Condition-Aware Reparameterization for Flow Matching (CAR-Flow) – a lightweight, learned shift that conditions the source, the target, or both distributions. By relocating these distributions, CAR-Flow shortens the probability path the model must learn, leading to faster training in practice. On low-dimensional synthetic data, we visualize and quantify the effects of CAR. On higher-dimensional natural image data (ImageNet-256), equipping SiT-XL/2 with CAR-Flow reduces FID from 2.07 to 1.68, while introducing less than 0.6% additional parameters.

Method: Leverages pre-trained SoTA captioning models to generate initial captions, ranks them using BLIPScore (a new image-text metric), and fuses the top two captions using a Large Language Model to produce detailed final descriptions.

Result: Experimental results on MS-COCO and Flickr30k show improved caption-image alignment and reduced hallucination according to ALOHa, CAPTURE, and Polos metrics, with subjective studies confirming better human judgment alignment.

Conclusion: The approach successfully bridges the gap between automated systems and human-generated descriptions by combining diverse model strengths, enabling generation of more suitable captions for training vision-language and captioning models.

Abstract: State-of-The-Art (SoTA) image captioning models are often trained on the MicroSoft Common Objects in Context (MS-COCO) dataset, which contains human-annotated captions with an average length of approximately ten tokens. Although effective for general scene understanding, these short captions often fail to capture complex scenes and convey detailed information. Moreover, captioning models tend to exhibit bias towards the ``average’’ caption, which captures only the more general aspects, thus overlooking finer details. In this paper, we present a novel approach to generate richer and more informative image captions by combining the captions generated from different SoTA captioning models. Our proposed method requires no additional model training: given an image, it leverages pre-trained models from the literature to generate the initial captions, and then ranks them using a newly introduced image-text-based metric, which we name BLIPScore. Subsequently, the top two captions are fused using a Large Language Model (LLM) to produce the final, more detailed description. Experimental results on the MS-COCO and Flickr30k test sets demonstrate the effectiveness of our approach in terms of caption-image alignment and hallucination reduction according to the ALOHa, CAPTURE, and Polos metrics. A subjective study lends additional support to these results, suggesting that the captions produced by our model are generally perceived as more consistent with human judgment. By combining the strengths of diverse SoTA models, our method enhances the quality and appeal of image captions, bridging the gap between automated systems and the rich and informative nature of human-generated descriptions. This advance enables the generation of more suitable captions for the training of both vision-language and captioning models.

Method: Developed MediSyn, a generalist text-guided latent diffusion model capable of generating synthetic medical images from diverse specialties and imaging modalities.

Result: MediSyn matches/surpasses specialist models, produces realistic images aligned with text prompts, generates visually distinct synthetic data, and improves classifier performance when used for training in data-limited scenarios.

Conclusion: Generalist image generative models like MediSyn have immense potential to accelerate medical algorithmic research by addressing data scarcity while maintaining privacy.

Abstract: Deep learning algorithms require extensive data to achieve robust performance. However, data availability is often restricted in the medical domain due to patient privacy concerns. Synthetic data presents a possible solution to these challenges. Recently, image generative models have found increasing use for medical applications but are often designed for singular medical specialties and imaging modalities, thus limiting their broader utility. To address this, we introduce MediSyn: a text-guided, latent diffusion model capable of generating synthetic images from 6 medical specialties and 10 image types. Through extensive experimentation, we first demonstrate that MediSyn quantitatively matches or surpasses the performance of specialist models. Second, we show that our synthetic images are realistic and exhibit strong alignment with their corresponding text prompts, as validated by a team of expert physicians. Third, we provide empirical evidence that our synthetic images are visually distinct from their corresponding real patient images. Finally, we demonstrate that in data-limited settings, classifiers trained solely on synthetic data or real data supplemented with synthetic data can outperform those trained solely on real data. Our findings highlight the immense potential of generalist image generative models to accelerate algorithmic research and development in medicine.

Method: Built on multimodal large language models like Gemini, EMMA maps raw camera sensor data to driving outputs by representing non-sensor inputs (navigation instructions, vehicle status) and outputs (trajectories, 3D locations) as natural language text, using task-specific prompts for generation.

Result: State-of-the-art performance in motion planning on nuScenes, competitive results on Waymo Open Motion Dataset, and competitive camera-primary 3D object detection on Waymo Open Dataset. Co-training across multiple tasks improves performance in all domains.

Conclusion: EMMA demonstrates potential as a generalist model for autonomous driving applications, with co-training benefits across planning, detection, and road graph tasks, inspiring future research in unified autonomous driving architectures.

Abstract: We introduce EMMA, an End-to-end Multimodal Model for Autonomous driving. Built upon a multi-modal large language model foundation like Gemini, EMMA directly maps raw camera sensor data into various driving-specific outputs, including planner trajectories, perception objects, and road graph elements. EMMA maximizes the utility of world knowledge from the pre-trained large language models, by representing all non-sensor inputs (e.g. navigation instructions and ego vehicle status) and outputs (e.g. trajectories and 3D locations) as natural language text. This approach allows EMMA to jointly process various driving tasks in a unified language space, and generate the outputs for each task using task-specific prompts. Empirically, we demonstrate EMMA’s effectiveness by achieving state-of-the-art performance in motion planning on nuScenes as well as competitive results on the Waymo Open Motion Dataset (WOMD). EMMA also yields competitive results for camera-primary 3D object detection on the Waymo Open Dataset (WOD). We show that co-training EMMA with planner trajectories, object detection, and road graph tasks yields improvements across all three domains, highlighting EMMA’s potential as a generalist model for autonomous driving applications. We hope that our results will inspire research to further evolve the state of the art in autonomous driving model architectures.

Method: The paper presents a comprehensive examination through: 1) Fundamentals of alignment (representational/behavioral aspects, training methodologies, theoretical foundations), 2) Analysis of misalignment across object, attribute, and relational levels, 3) Investigation of root causes at data, model, and inference levels, 4) Review of mitigation strategies categorized as parameter-frozen and parameter-tuning approaches.

Result: The investigation reveals that misalignment emerges from challenges at multiple levels and provides a systematic categorization of existing mitigation approaches.

Conclusion: The paper outlines promising future research directions emphasizing the need for standardized evaluation protocols and in-depth explainability studies to advance LVLM alignment research.

Abstract: Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities in processing both visual and textual information. However, the critical challenge of alignment between visual and textual representations is not fully understood. This survey presents a comprehensive examination of alignment and misalignment in LVLMs through an explainability lens. We first examine the fundamentals of alignment, exploring its representational and behavioral aspects, training methodologies, and theoretical foundations. We then analyze misalignment phenomena across three semantic levels: object, attribute, and relational misalignment. Our investigation reveals that misalignment emerges from challenges at multiple levels: the data level, the model level, and the inference level. We provide a comprehensive review of existing mitigation strategies, categorizing them into parameter-frozen and parameter-tuning approaches. Finally, we outline promising future research directions, emphasizing the need for standardized evaluation protocols and in-depth explainability studies.

Method: The authors propose an evaluation framework that assesses three core decomposition aspects: color homogeneity, respect of image objects, and shape regularity. They introduce a new Global Regularity (GR) measure to address limitations of existing metrics.

Result: The framework reduces bias in superpixel method comparisons and demonstrates that the proposed GR measure correlates with performance across various applications when evaluated at multiple superpixel scales and regularity levels.

Conclusion: The unified evaluation framework provides more robust and fair comparisons of superpixel methods, with the GR measure showing practical utility in application performance correlation.

Abstract: In the superpixel literature, the comparison of state-of-the-art methods can be biased by the non-robustness of some metrics to decomposition aspects, such as the superpixel scale. Moreover, most recent decomposition methods allow to set a shape regularity parameter, which can have a substantial impact on the measured performances. In this paper, we introduce an evaluation framework, that aims to unify the comparison process of superpixel methods. We investigate the limitations of existing metrics, and propose to evaluate each of the three core decomposition aspects: color homogeneity, respect of image objects and shape regularity. To measure the regularity aspect, we propose a new global regularity measure (GR), which addresses the non-robustness of state-of-the-art metrics. We evaluate recent superpixel methods with these criteria, at several superpixel scales and regularity levels. The proposed framework reduces the bias in the comparison process of state-of-the-art superpixel methods. Finally, we demonstrate that the proposed GR measure is correlated with the performances of various applications.

Method: Zoneup Dropout Injection Attention Calculation (ZoDIAC) refines attention intensities using GELU and dropout, then intensifies them through a zoneup process with learned scalar factor injection.

Result: ZoDIAC achieves statistically significant higher scores across all image captioning metrics on MS-COCO dataset with various feature extractors compared to conventional self-attention.

Conclusion: ZoDIAC can serve as a drop-in replacement for attention components in transformer models, providing enhanced attention refinement and intensification capabilities.

Abstract: In the past few years the transformer model has been utilized for a variety of tasks such as image captioning, image classification natural language generation, and natural language understanding. As a key component of the transformer model, self-attention calculates the attention values by mapping the relationships among the head elements of the source and target sequence, yet there is no explicit mechanism to refine and intensify the attention values with respect to the context of the input and target sequences. Based on this intuition, we introduce a novel refine and intensify attention mechanism that is called Zoneup Dropout Injection Attention Calculation (ZoDIAC), in which the intensities of attention values in the elements of the input source and target sequences are first refined using GELU and dropout and then intensified using a proposed zoneup process which includes the injection of a learned scalar factor. Our extensive experiments show that ZoDIAC achieves statistically significant higher scores under all image captioning metrics using various feature extractors in comparison to the conventional self-attention module in the transformer model on the MS-COCO dataset. Our proposed ZoDIAC attention modules can be used as a drop-in replacement for the attention components in all transformer models. The code for our experiments is publicly available at: https://github.com/zanyarz/zodiac

Method: Developed Stochastic Cortical Self-Reconstruction (SCSR) - a deep learning approach trained on over 25,000 healthy individuals to reconstruct cortical thickness maps at vertex level without needing additional subject information.

Result: SCSR achieved significantly lower reconstruction errors, better disease discrimination, detected cortical thinning in preterm infants missed by existing models, and excelled at mapping cortical deviations in dementia patients from clinical data.

Conclusion: SCSR shows strong potential for supporting clinical diagnosis by enabling highly resolved detection of subtle cortical thickness deviations across various neurological conditions.

Abstract: Understanding individual differences in cortical structure is key to advancing diagnostics in neurology and psychiatry. Reference models aid in detecting aberrant cortical thickness, yet site-specific biases limit their direct application to unseen data, and region-wise averages prevent the detection of localized cortical changes. To address these limitations, we developed the Stochastic Cortical Self-Reconstruction (SCSR), a novel method that leverages deep learning to reconstruct cortical thickness maps at the vertex level without needing additional subject information. Trained on over 25,000 healthy individuals, SCSR generates highly individualized cortical reconstructions that can detect subtle thickness deviations. Our evaluations on independent test sets demonstrated that SCSR achieved significantly lower reconstruction errors and identified atrophy patterns that enabled better disease discrimination than established methods. It also hints at cortical thinning in preterm infants that went undetected by existing models, showcasing its versatility. Finally, SCSR excelled in mapping highly resolved cortical deviations of dementia patients from clinical data, highlighting its potential for supporting diagnosis in clinical practice.

Method: Proposed REACT architecture that optimizes for real-time efficiency and accuracy tradeoffs in Scene Graph Generation, achieving significant parameter reduction (5.5x fewer parameters on average).

Result: REACT achieves 2.7x faster inference speed compared to state-of-the-art approaches while improving object detection accuracy by 58% without compromising relation prediction performance.

Conclusion: REACT successfully addresses the performance-speed tradeoff in SGG, enabling real-time applications while maintaining high accuracy across both object detection and relation prediction tasks.

Abstract: Scene Graph Generation (SGG) is a task that encodes visual relationships between objects in images as graph structures. SGG shows significant promise as a foundational component for downstream tasks, such as reasoning for embodied agents. To enable real-time applications, SGG must address the trade-off between performance and inference speed. However, current methods tend to focus on one of the following: (1) improving relation prediction accuracy, (2) enhancing object detection accuracy, or (3) reducing latency, without aiming to balance all three objectives simultaneously. To address this limitation, we propose the Real-time Efficiency and Accuracy Compromise for Tradeoffs in Scene Graph Generation (REACT) architecture, which achieves the highest inference speed among existing SGG models, improving object detection accuracy without sacrificing relation prediction performance. Compared to state-of-the-art approaches, REACT is 2.7 times faster and improves object detection accuracy by 58%. Furthermore, our proposal significantly reduces model size, with an average of 5.5x fewer parameters. The code is available at https://github.com/Maelic/SGG-Benchmark

Method: The method leverages spherical CNN architectures and differentiable K-means clustering paradigm for superpixels, generating superpixels that follow spherical geometry. It also uses specialized data augmentation techniques for 360° images to learn efficiently from limited annotated data.

Result: Extensive validation across two datasets shows that incorporating the inherent circular geometry of omnidirectional images improves segmentation performance over both traditional and deep learning-based superpixel methods.

Conclusion: The proposed DSS framework successfully addresses the unique challenges of omnidirectional image segmentation by accounting for spherical geometry, demonstrating superior performance compared to existing methods.

Abstract: Over the years, the use of superpixel segmentation has become very popular in various applications, serving as a preprocessing step to reduce data size by adapting to the content of the image, regardless of its semantic content. While the superpixel segmentation of standard planar images, captured with a 90{\deg} field of view, has been extensively studied, there has been limited focus on dedicated methods to omnidirectional or spherical images, captured with a 360{\deg} field of view. In this study, we introduce the first deep learning-based superpixel segmentation approach tailored for omnidirectional images called DSS (for Deep Spherical Superpixels). Our methodology leverages on spherical CNN architectures and the differentiable K-means clustering paradigm for superpixels, to generate superpixels that follow the spherical geometry. Additionally, we propose to use data augmentation techniques specifically designed for 360{\deg} images, enabling our model to efficiently learn from a limited set of annotated omnidirectional data. Our extensive validation across two datasets demonstrates that taking into account the inherent circular geometry of such images into our framework improves the segmentation performance over traditional and deep learning-based superpixel methods. Our code is available online.

Method: Uses Fastpose and Strided Transformer pose estimation models on 1386 turning video clips from 24 subjects (12 PD, 12 healthy). Extracts 3D skeletons and calculates hip/knee joint rotation. Validated on Turn-REMAP (home setting) and Turn-H3.6M (benchmark) datasets.

Result: Achieves 41.6% turning calculation accuracy, 34.7° MAE, and 68.3% weighted precision for Turn-REMAP dataset in challenging home environments.

Conclusion: First work to use single monocular camera data for quantifying PD patient turns in home settings, addressing challenges like baggy clothing and poor lighting.

Abstract: People with Parkinson’s Disease (PD) often experience progressively worsening gait, including changes in how they turn around, as the disease progresses. Existing clinical rating tools are not capable of capturing hour-by-hour variations of PD symptoms, as they are confined to brief assessments within clinic settings. Measuring gait turning angles continuously and passively is a component step towards using gait characteristics as sensitive indicators of disease progression in PD. This paper presents a deep learning-based approach to automatically quantify turning angles by extracting 3D skeletons from videos and calculating the rotation of hip and knee joints. We utilise state-of-the-art human pose estimation models, Fastpose and Strided Transformer, on a total of 1386 turning video clips from 24 subjects (12 people with PD and 12 healthy control volunteers), trimmed from a PD dataset of unscripted free-living videos in a home-like setting (Turn-REMAP). We also curate a turning video dataset, Turn-H3.6M, from the public Human3.6M human pose benchmark with 3D ground truth, to further validate our method. Previous gait research has primarily taken place in clinics or laboratories evaluating scripted gait outcomes, but this work focuses on free-living home settings where complexities exist, such as baggy clothing and poor lighting. Due to difficulties in obtaining accurate ground truth data in a free-living setting, we quantise the angle into the nearest bin $45^\circ$ based on the manual labelling of expert clinicians. Our method achieves a turning calculation accuracy of 41.6%, a Mean Absolute Error (MAE) of 34.7{\deg}, and a weighted precision WPrec of 68.3% for Turn-REMAP. This is the first work to explore the use of single monocular camera data to quantify turns by PD patients in a home setting.

Method: Frames Gaussian splat training as variational inference over model parameters, leveraging conjugacy of multivariate Gaussians to derive closed-form variational update rules that enable efficient sequential updates without replay buffers.

Result: VBGS matches state-of-the-art performance on static datasets while significantly improving continual learning performance on sequentially streamed 2D and 3D data compared to gradient-based methods.

Conclusion: The variational Bayes approach provides an effective solution to catastrophic forgetting in Gaussian splatting, enabling practical continual learning applications without the need for replay mechanisms.

Abstract: Recently, 3D Gaussian Splatting has emerged as a promising approach for modeling 3D scenes using mixtures of Gaussians. The predominant optimization method for these models relies on backpropagating gradients through a differentiable rendering pipeline, which struggles with catastrophic forgetting when dealing with continuous streams of data. To address this limitation, we propose Variational Bayes Gaussian Splatting (VBGS), a novel approach that frames training a Gaussian splat as variational inference over model parameters. By leveraging the conjugacy properties of multivariate Gaussians, we derive a closed-form variational update rule, allowing efficient updates from partial, sequential observations without the need for replay buffers. Our experiments show that VBGS not only matches state-of-the-art performance on static datasets, but also enables continual learning from sequentially streamed 2D and 3D data, drastically improving performance in this setting.

Method: ViSpec employs a lightweight vision adaptor module to compress image tokens into compact representations, integrates them into the draft model’s attention mechanism, and augments text tokens with global image features. It uses a specialized training dataset created by repurposing existing datasets and generating extended outputs.

Result: ViSpec achieves, to the authors’ knowledge, the first substantial speedup in VLM speculative decoding, significantly outperforming existing methods that only achieve <1.5x speedup.

Conclusion: The proposed ViSpec framework successfully addresses the challenges of speculative decoding in VLMs, demonstrating that large VLMs can effectively filter redundant image information layer by layer while maintaining textual comprehension.

Abstract: Speculative decoding is a widely adopted technique for accelerating inference in large language models (LLMs), yet its application to vision-language models (VLMs) remains underexplored, with existing methods achieving only modest speedups (<1.5x). This gap is increasingly significant as multimodal capabilities become central to large-scale models. We hypothesize that large VLMs can effectively filter redundant image information layer by layer without compromising textual comprehension, whereas smaller draft models struggle to do so. To address this, we introduce Vision-Aware Speculative Decoding (ViSpec), a novel framework tailored for VLMs. ViSpec employs a lightweight vision adaptor module to compress image tokens into a compact representation, which is seamlessly integrated into the draft model’s attention mechanism while preserving original image positional information. Additionally, we extract a global feature vector for each input image and augment all subsequent text tokens with this feature to enhance multimodal coherence. To overcome the scarcity of multimodal datasets with long assistant responses, we curate a specialized training dataset by repurposing existing datasets and generating extended outputs using the target VLM with modified prompts. Our training strategy mitigates the risk of the draft model exploiting direct access to the target model’s hidden states, which could otherwise lead to shortcut learning when training solely on target model outputs. Extensive experiments validate ViSpec, achieving, to our knowledge, the first substantial speedup in VLM speculative decoding. Code is available at https://github.com/KangJialiang/ViSpec.

Method: CrossEarth uses a data-level Earth-Style Injection pipeline and model-level Multi-Task Training pipeline. The authors also created an RSDG benchmark with 32 cross-domain settings across regions, spectral bands, platforms, and climates.

Result: Extensive experiments on the benchmark demonstrate CrossEarth’s superiority over existing state-of-the-art methods in cross-domain generalization for semantic segmentation.

Conclusion: CrossEarth represents a significant advancement in RSDG, providing strong generalization capabilities across diverse remote sensing scenarios and establishing a comprehensive benchmark for future RSDG research.

Abstract: The field of Remote Sensing Domain Generalization (RSDG) has emerged as a critical and valuable research frontier, focusing on developing models that generalize effectively across diverse scenarios. Despite the substantial domain gaps in RS images that are characterized by variabilities such as location, wavelength, and sensor type, research in this area remains underexplored: (1) Current cross-domain methods primarily focus on Domain Adaptation (DA), which adapts models to predefined domains rather than to unseen ones; (2) Few studies targeting the RSDG issue, especially for semantic segmentation tasks, where existing models are developed for specific unknown domains, struggling with issues of underfitting on other unknown scenarios; (3) Existing RS foundation models tend to prioritize in-domain performance over cross-domain generalization. To this end, we introduce the first vision foundation model for RSDG semantic segmentation, CrossEarth. CrossEarth demonstrates strong cross-domain generalization through a specially designed data-level Earth-Style Injection pipeline and a model-level Multi-Task Training pipeline. In addition, for the semantic segmentation task, we have curated an RSDG benchmark comprising 32 cross-domain settings across various regions, spectral bands, platforms, and climates, providing a comprehensive framework for testing the generalizability of future RSDG models. Extensive experiments on this benchmark demonstrate the superiority of CrossEarth over existing state-of-the-art methods.

Method: The authors conduct a comprehensive study of superpixel evaluation metrics and demonstrate that the Segment Anything Model (SAM) can achieve competitive superpixel segmentation results without task-specific training.

Result: The paper shows that SAM achieves competitive superpixel segmentation performance despite not being specifically trained for this task, highlighting the limitations of current evaluation frameworks.

Conclusion: Superpixel segmentation should be rethought based on the properties needed for specific downstream tasks, as it’s fundamentally ill-posed due to regularity constraints on shape and size.

Abstract: For many years, image over-segmentation into superpixels has been essential to computer vision pipelines, by creating homogeneous and identifiable regions of similar sizes. Such constrained segmentation problem would require a clear definition and specific evaluation criteria. However, the validation framework for superpixel methods, typically viewed as standard object segmentation, has rarely been thoroughly studied. In this work, we first take a step back to show that superpixel segmentation is fundamentally an ill-posed problem, due to the implicit regularity constraint on the shape and size of superpixels. We also demonstrate through a novel comprehensive study that the literature suffers from only evaluating certain aspects, sometimes incorrectly and with inappropriate metrics. Concurrently, recent deep learning-based superpixel methods mainly focus on the object segmentation task at the expense of regularity. In this ill-posed context, we show that we can achieve competitive results using a recent architecture like the Segment Anything Model (SAM), without dedicated training for the superpixel segmentation task. This leads to rethinking superpixel segmentation and the necessary properties depending on the targeted downstream task.

Method: SparseDiT uses a tri-segment spatial architecture (Poolingformer bottom layers, Sparse-Dense Token Modules middle layers, dense top layers) and temporal dynamic token density modulation across denoising stages.

Result: 55% FLOPs reduction and 175% inference speed improvement on DiT-XL with similar FID; 56% FLOPs reduction in video generation; 69% speed improvement on PixArt-α with minimal FID decrease.

Conclusion: SparseDiT provides a scalable solution for efficient high-quality diffusion-based generation that’s compatible with sampling optimization techniques.

Abstract: Diffusion Transformers (DiT) are renowned for their impressive generative performance; however, they are significantly constrained by considerable computational costs due to the quadratic complexity in self-attention and the extensive sampling steps required. While advancements have been made in expediting the sampling process, the underlying architectural inefficiencies within DiT remain underexplored. We introduce SparseDiT, a novel framework that implements token sparsification across spatial and temporal dimensions to enhance computational efficiency while preserving generative quality. Spatially, SparseDiT employs a tri-segment architecture that allocates token density based on feature requirements at each layer: Poolingformer in the bottom layers for efficient global feature extraction, Sparse-Dense Token Modules (SDTM) in the middle layers to balance global context with local detail, and dense tokens in the top layers to refine high-frequency details. Temporally, SparseDiT dynamically modulates token density across denoising stages, progressively increasing token count as finer details emerge in later timesteps. This synergy between SparseDiT spatially adaptive architecture and its temporal pruning strategy enables a unified framework that balances efficiency and fidelity throughout the generation process. Our experiments demonstrate SparseDiT effectiveness, achieving a 55% reduction in FLOPs and a 175% improvement in inference speed on DiT-XL with similar FID score on 512x512 ImageNet, a 56% reduction in FLOPs across video generation datasets, and a 69% improvement in inference speed on PixArt-$\alpha$ on text-to-image generation task with a 0.24 FID score decrease. SparseDiT provides a scalable solution for high-quality diffusion-based generation compatible with sampling optimization techniques.

Method: Two-stage approach: 1) Coarse-grained rapid classification using knowledge distillation from teacher to student model, 2) Fine-grained multi-class classification using CLIP for cross-modal contrastive learning with text and images.

Result: TCVADS significantly outperforms existing methods in model performance, detection efficiency, and interpretability.

Conclusion: TCVADS offers valuable contributions to smart city monitoring applications by enabling efficient and interpretable anomaly detection on edge devices.

Abstract: Weakly Supervised Monitoring Anomaly Detection (WSMAD) utilizes weak supervision learning to identify anomalies, a critical task for smart city monitoring. However, existing multimodal approaches often fail to meet the real-time and interpretability requirements of edge devices due to their complexity. This paper presents TCVADS (Two-stage Cross-modal Video Anomaly Detection System), which leverages knowledge distillation and cross-modal contrastive learning to enable efficient, accurate, and interpretable anomaly detection on edge devices.TCVADS operates in two stages: coarse-grained rapid classification and fine-grained detailed analysis. In the first stage, TCVADS extracts features from video frames and inputs them into a time series analysis module, which acts as the teacher model. Insights are then transferred via knowledge distillation to a simplified convolutional network (student model) for binary classification. Upon detecting an anomaly, the second stage is triggered, employing a fine-grained multi-class classification model. This stage uses CLIP for cross-modal contrastive learning with text and images, enhancing interpretability and achieving refined classification through specially designed triplet textual relationships. Experimental results demonstrate that TCVADS significantly outperforms existing methods in model performance, detection efficiency, and interpretability, offering valuable contributions to smart city monitoring applications.

Method: Proposes token-level visual-anchored rewards based on logistic distribution differences between tokens generated from raw vs corrupted images, plus a visual-aware training objective to highlight informative visual-correlated tokens without fine-grained annotations.

Result: Extensive experiments show state-of-the-art performance, with TPO built on LLAVA-1.5-7B achieving significant absolute improvements on hallucination benchmarks.

Conclusion: TPO effectively addresses key limitations of existing DPO methods and demonstrates superior performance in reducing hallucinations through its novel token-level optimization approach with self-calibrated visual-anchored rewards.

Abstract: Direct Preference Optimization (DPO) has been demonstrated to be highly effective in mitigating hallucinations in Large Vision Language Models (LVLMs) by aligning their outputs more closely with human preferences. Despite the recent progress, existing methods suffer from two drawbacks: 1) Lack of scalable token-level rewards; and 2) Neglect of visual-anchored tokens. To this end, we propose a novel Token Preference Optimization model with self-calibrated rewards (dubbed as TPO), which adaptively attends to visual-correlated tokens without fine-grained annotations. Specifically, we introduce a token-level \emph{visual-anchored} \emph{reward} as the difference of the logistic distributions of generated tokens conditioned on the raw image and the corrupted one. In addition, to highlight the informative visual-anchored tokens, a visual-aware training objective is proposed to enhance more accurate token-level optimization. Extensive experimental results have manifested the state-of-the-art performance of the proposed TPO. For example, by building on top of LLAVA-1.5-7B, our TPO boosts the performance absolute improvement for hallucination benchmarks.

Method: The authors created a large event-image/video-text dataset with 1.4M high-quality pairs, designed Event Spatiotemporal Representation to aggregate and segment event streams, and introduced Dynamic Semantic Alignment to improve sparse semantic spaces of events.

Result: Extensive experiments show that EventVL significantly outperforms existing MLLM baselines in event captioning and scene description generation tasks.

Conclusion: EventVL represents a significant advancement in event-based vision understanding and is expected to contribute to the development of the event vision community.

Abstract: The event-based Vision-Language Model (VLM) recently has made good progress for practical vision tasks. However, most of these works just utilize CLIP for focusing on traditional perception tasks, which obstruct model understanding explicitly the sufficient semantics and context from event streams. To address the deficiency, we propose EventVL, the first generative event-based MLLM (Multimodal Large Language Model) framework for explicit semantic understanding. Specifically, to bridge the data gap for connecting different modalities semantics, we first annotate a large event-image/video-text dataset, containing almost 1.4 million high-quality pairs of data, which enables effective learning across various scenes, e.g., drive scene or human motion. After that, we design Event Spatiotemporal Representation to fully explore the comprehensive information by diversely aggregating and segmenting the event stream. To further promote a compact semantic space, Dynamic Semantic Alignment is introduced to improve and complete sparse semantic spaces of events. Extensive experiments show that our EventVL can significantly surpass existing MLLM baselines in event captioning and scene description generation tasks. We hope our research could contribute to the development of the event vision community.

Method: Uses clustering for pseudo-labels, dual-path contrastive learning, dynamic hierarchical memory module for intra-view consistency, and neighborhood-driven constraint mechanism for cross-view alignment with pseudo-label enhancement.

Result: Extensive experiments on three benchmark datasets show DMNIL outperforms existing self-supervised methods and even surpasses several state-of-the-art supervised methods.

Conclusion: The proposed self-supervised approach effectively addresses annotation cost and transferability limitations, enabling practical drone geo-localization without paired supervision.

Abstract: Drone-view Geo-Localization (DVGL) aims to achieve accurate localization of drones by retrieving the most relevant GPS-tagged satellite images. However, most existing methods heavily rely on strictly pre-paired drone-satellite images for supervised learning. When the target region shifts, new paired samples are typically required to adapt to the distribution changes. The high cost of annotation and the limited transferability of these methods significantly hinder the practical deployment of DVGL in open-world scenarios. To address these limitations, we propose a novel end-to-end self-supervised learning method with a shallow backbone network, called the dynamic memory-driven and neighborhood information learning (DMNIL) method. It employs a clustering algorithm to generate pseudo-labels and adopts a dual-path contrastive learning framework to learn discriminative intra-view representations. Furthermore, DMNIL incorporates two core modules, including the dynamic hierarchical memory learning (DHML) module and the information consistency evolution learning (ICEL) module. The DHML module combines short-term and long-term memory to enhance intra-view feature consistency and discriminability. Meanwhile, the ICEL module utilizes a neighborhood-driven dynamic constraint mechanism to systematically capture implicit cross-view semantic correlations, consequently improving cross-view feature alignment. To further stabilize and strengthen the self-supervised training process, a pseudo-label enhancement strategy is introduced to enhance the quality of pseudo supervision. Extensive experiments on three public benchmark datasets demonstrate that the proposed method consistently outperforms existing self-supervised methods and even surpasses several state-of-the-art supervised methods. Our code is available at https://github.com/ISChenawei/DMNIL.

Method: Proposes Origin-Partition strategy that partitions training data by Change Area Ratio and trains specialized teacher models, then uses Multi-Teacher Knowledge Distillation to transfer complementary knowledge to a single student model.

Result: The MTKD approach achieved first place in preliminary and second place in final rounds of 2024 “Jilin-1” Cup challenge, and established new state-of-the-art results on JL1-CD and SYSU-CD datasets.

Conclusion: The proposed MTKD framework effectively enhances change detection performance across various network architectures and parameter sizes without additional inference cost, demonstrating strong generalization capabilities.

Abstract: Change detection (CD) in remote sensing images plays a vital role in Earth observation. However, the scarcity of high-resolution, comprehensive open-source datasets and the difficulty in achieving robust performance across varying change types remain major challenges. To address these issues, we introduce JL1-CD, a large-scale, sub-meter CD dataset consisting of 5,000 image pairs. We further propose a novel Origin-Partition (O-P) strategy and integrate it into a Multi-Teacher Knowledge Distillation (MTKD) framework to enhance CD performance. The O-P strategy partitions the training set by Change Area Ratio (CAR) and trains specialized teacher models on each subset. The MTKD framework then distills complementary knowledge from these teachers into a single student model, enabling improved detection results across diverse CAR scenarios without additional inference cost. Our MTKD approach demonstrated strong performance in the 2024 ``Jilin-1’’ Cup challenge, ranking first in the preliminary and second in the final rounds. Extensive experiments on the JL1-CD and SYSU-CD datasets show that the MTKD framework consistently improves the performance of CD models with various network architectures and parameter sizes, establishing new state-of-the-art results. Code and dataset are available at https://github.com/circleLZY/MTKD-CD.

Method: Proposes Straight Consistent Trajectory (SCoT) model that balances two objectives: regulating gradient to constant (straight trajectories) and ensuring trajectory consistency. This combines advantages of both consistency models and rectified flow.

Result: Extensive experiments demonstrate SCoT’s effectiveness and efficiency in fast sampling while maintaining accuracy.

Conclusion: SCoT successfully bridges the gap between consistency models and rectified flow, providing a unified approach that enjoys the benefits of both methods for fast and accurate sampling.

Abstract: Pre-trained diffusion models are commonly used to generate clean data (e.g., images) from random noises, effectively forming pairs of noises and corresponding clean images. Distillation on these pre-trained models can be viewed as the process of constructing advanced trajectories within the pair to accelerate sampling. For instance, consistency model distillation develops consistent projection functions to regulate trajectories, although sampling efficiency remains a concern. Rectified flow method enforces straight trajectories to enable faster sampling, yet relies on numerical ODE solvers, which may introduce approximation errors. In this work, we bridge the gap between the consistency model and the rectified flow method by proposing a Straight Consistent Trajectory~(SCoT) model. SCoT enjoys the benefits of both approaches for fast sampling, producing trajectories with consistent and straight properties simultaneously. These dual properties are strategically balanced by targeting two critical objectives: (1) regulating the gradient of SCoT’s mapping to a constant, (2) ensuring trajectory consistency. Extensive experimental results demonstrate the effectiveness and efficiency of SCoT.

Method: A unified framework with three modules: Diffusion Anomaly Generation Module (DAGM) for realistic anomaly generation, Diffusion Discriminative Module (DDM) for anomaly detection via reverse diffusion, and Probability Optimization Module (POM) for refining probability distributions.

Result: Extensive experiments show the method outperforms state-of-the-art approaches, significantly improving both image-level and pixel-level anomaly detection performance as measured by AUROC.

Conclusion: The proposed HDM effectively addresses the limitations of existing methods by integrating generation and discrimination, demonstrating superior performance in industrial image anomaly detection applications.

Abstract: Image anomaly detection plays a vital role in applications such as industrial quality inspection and medical imaging, where it directly contributes to improving product quality and system reliability. However, existing methods often struggle with complex and diverse anomaly patterns. In particular, the separation between generation and discrimination tasks limits the effective coordination between anomaly sample generation and anomaly region detection. To address these challenges, we propose a novel hybrid diffusion model (HDM) that integrates generation and discrimination into a unified framework. The model consists of three key modules: the Diffusion Anomaly Generation Module (DAGM), the Diffusion Discriminative Module (DDM), and the Probability Optimization Module (POM). DAGM generates realistic and diverse anomaly samples, improving their representativeness. DDM then applies a reverse diffusion process to capture the differences between generated and normal samples, enabling precise anomaly region detection and localization based on probability distributions. POM refines the probability distributions during both the generation and discrimination phases, ensuring high-quality samples are used for training. Extensive experiments on multiple industrial image datasets demonstrate that our method outperforms state-of-the-art approaches, significantly improving both image-level and pixel-level anomaly detection performance, as measured by AUROC.

Method: The paper introduces a decomposition of creativity into atypicality and originality, collects fine-grained human annotations on these dimensions, and creates benchmark tasks to evaluate vision-language models’ alignment with human judgments.

Result: The evaluation demonstrates both promises and challenges of using state-of-the-art vision-language models for automatic creativity assessment, showing their potential but also limitations in matching human evaluations.

Conclusion: While VLMs show promise for automatic creativity assessment, significant challenges remain in achieving human-level alignment, highlighting the need for continued research in this subjective domain.

Abstract: Evaluating creativity is challenging, even for humans, not only because of its subjectivity but also because it involves complex cognitive processes. Inspired by work in marketing, we attempt to break down visual advertisement creativity into atypicality and originality. With fine-grained human annotations on these dimensions, we propose a suite of tasks specifically for such a subjective problem. We also evaluate the alignment between state-of-the-art (SoTA) vision language models (VLMs) and humans on our proposed benchmark, demonstrating both the promises and challenges of using VLMs for automatic creativity assessment.

Method: STORM incorporates a temporal encoder between image encoder and LLM using Mamba State Space Model, with token reduction strategies including test-time sampling and training-based temporal/spatial pooling.

Result: Achieves state-of-the-art results (5%+ improvement on MLVU and LongVideoBench) while reducing computation costs by 8x and decoding latency by 2.4-2.9x.

Conclusion: STORM enables efficient and robust video understanding over extended temporal contexts by simultaneously improving performance and reducing computational demands.

Abstract: Recent advances in video-based multimodal large language models (Video-LLMs) have significantly improved video understanding by processing videos as sequences of image frames. However, many existing methods treat frames independently in the vision backbone, lacking explicit temporal modeling, which limits their ability to capture dynamic patterns and efficiently handle long videos. To address these limitations, we introduce STORM (Spatiotemporal TOken Reduction for Multimodal LLMs), a novel architecture incorporating a dedicated temporal encoder between the image encoder and the LLM. Our temporal encoder leverages the Mamba State Space Model to integrate temporal information into image tokens, generating enriched representations that preserve inter-frame dynamics across the entire video sequence. This enriched encoding not only enhances video reasoning capabilities but also enables effective token reduction strategies, including test-time sampling and training-based temporal and spatial pooling, substantially reducing computational demands on the LLM without sacrificing key temporal information. By integrating these techniques, our approach simultaneously reduces training and inference latency while improving performance, enabling efficient and robust video understanding over extended temporal contexts. Extensive evaluations show that STORM achieves state-of-the-art results across various long video understanding benchmarks (more than 5% improvement on MLVU and LongVideoBench) while reducing the computation costs by up to $8\times$ and the decoding latency by 2.4-2.9$\times$ for the fixed numbers of input frames. Project page is available at https://research.nvidia.com/labs/lpr/storm

Method: Introduces a bottom-up procedure that decomposes massive visual analysis into tractable sub-problems, using carefully designed MLLM-based solutions to handle datasets four orders of magnitude too large for direct MLLM context ingestion.

Result: The system significantly outperforms baselines and successfully discovers interesting urban trends like ‘addition of outdoor dining’ and ‘overpass was painted blue’ from large city image datasets.

Conclusion: MLLMs serve as a novel tool for open-ended semantic understanding in temporal visual analysis, enabling discovery of meaningful change patterns in large-scale urban image databases without predefined targets.

Abstract: We present a system using Multimodal LLMs (MLLMs) to analyze a large database with tens of millions of images captured at different times, with the aim of discovering patterns in temporal changes. Specifically, we aim to capture frequent co-occurring changes (“trends”) across a city over a certain period. Unlike previous visual analyses, our analysis answers open-ended queries (e.g., “what are the frequent types of changes in the city?”) without any predetermined target subjects or training labels. These properties cast prior learning-based or unsupervised visual analysis tools unsuitable. We identify MLLMs as a novel tool for their open-ended semantic understanding capabilities. Yet, our datasets are four orders of magnitude too large for an MLLM to ingest as context. So we introduce a bottom-up procedure that decomposes the massive visual analysis problem into more tractable sub-problems. We carefully design MLLM-based solutions to each sub-problem. During experiments and ablation studies with our system, we find it significantly outperforms baselines and is able to discover interesting trends from images captured in large cities (e.g., “addition of outdoor dining,”, “overpass was painted blue,” etc.). See more results and interactive demos at https://boyangdeng.com/visual-chronicles.

Method: A novel beam search strategy for latent space exploration that dynamically samples past latents to search for optimal sequence representations. Uses cross-attention mechanism to prune the beam search graph and score paths based on alignment with textual prompts and visual context.

Result: Human and automatic evaluations show BeamDiffusion outperforms baseline methods, producing sequences with superior coherence, visual continuity, and textual alignment.

Conclusion: The beam search approach enables conditional generation of full image sequences with improved visual consistency, making it effective for coherent narrative generation in diffusion models.

Abstract: While diffusion models excel at generating high-quality images from text prompts, they struggle with visual consistency when generating image sequences. Existing methods generate each image independently, leading to disjointed narratives - a challenge further exacerbated in non-linear storytelling, where scenes must connect beyond adjacent images. We introduce a novel beam search strategy for latent space exploration, enabling conditional generation of full image sequences with beam search decoding. In contrast to earlier methods that rely on fixed latent priors, our method dynamically samples past latents to search for an optimal sequence of latent representations, ensuring coherent visual transitions. As the latent denoising space is explored, the beam search graph is pruned with a cross-attention mechanism that efficiently scores search paths, prioritizing alignment with both textual prompts and visual context. Human and automatic evaluations confirm that BeamDiffusion outperforms other baseline methods, producing full sequences with superior coherence, visual continuity, and textual alignment.

Method: Proposes AvatarShield framework combining a discrete vision tower for high-level semantic inconsistencies and a residual extractor for fine-grained artifact analysis. Uses Group Relative Policy Optimization to enable LLMs to develop reasoning from binary labels without dense textual supervision. Introduces FakeHumanVid benchmark with 15K real/synthetic videos across nine state-of-the-art human generation methods.

Result: Extensive experiments demonstrate that AvatarShield outperforms existing methods in both in-domain and cross-domain settings.

Conclusion: AvatarShield effectively addresses the limitations of current detection methods by providing a more robust and generalizable approach to detecting human-centric synthetic videos without requiring dense textual supervision.

Abstract: Recent advances in Artificial Intelligence Generated Content have led to highly realistic synthetic videos, particularly in human-centric scenarios involving speech, gestures, and full-body motion, posing serious threats to information authenticity and public trust. Unlike DeepFake techniques that focus on localized facial manipulation, human-centric video generation methods can synthesize entire human bodies with controllable movements, enabling complex interactions with environments, objects, and even other people. However, existing detection methods largely overlook the growing risks posed by such full-body synthetic content. Meanwhile, a growing body of research has explored leveraging LLMs for interpretable fake detection, aiming to explain decisions in natural language. Yet these approaches heavily depend on supervised fine-tuning, which introduces limitations such as annotation bias, hallucinated supervision, and weakened generalization. To address these challenges, we propose AvatarShield, a novel multimodal human-centric synthetic video detection framework that eliminates the need for dense textual supervision by adopting Group Relative Policy Optimization, enabling LLMs to develop reasoning capabilities from simple binary labels. Our architecture combines a discrete vision tower for high-level semantic inconsistencies and a residual extractor for fine-grained artifact analysis. We further introduce FakeHumanVid, a large-scale benchmark containing 15K real and synthetic videos across nine state-of-the-art human generation methods driven by text, pose, or audio. Extensive experiments demonstrate that AvatarShield outperforms existing methods in both in-domain and cross-domain settings.

Method: Leverages multiple input views alongside identity embeddings to synthesize diverse viewpoints while maintaining unique identity features through diffusion-based generation.

Result: The model demonstrates strong generation capabilities in 360-degree head synthesis and outperforms current state-of-the-art multiview diffusion models.

Conclusion: SpinMeRound effectively addresses the challenge of generating consistent and accurate head portraits from novel viewpoints, advancing beyond current limitations in angular range and facial data handling.

Abstract: Despite recent progress in diffusion models, generating realistic head portraits from novel viewpoints remains a significant challenge. Most current approaches are constrained to limited angular ranges, predominantly focusing on frontal or near-frontal views. Moreover, although the recent emerging large-scale diffusion models have been proven robust in handling 3D scenes, they underperform on facial data, given their complex structure and the uncanny valley pitfalls. In this paper, we propose SpinMeRound, a diffusion-based approach designed to generate consistent and accurate head portraits from novel viewpoints. By leveraging a number of input views alongside an identity embedding, our method effectively synthesizes diverse viewpoints of a subject whilst robustly maintaining its unique identity features. Through experimentation, we showcase our model’s generation capabilities in 360 head synthesis, while beating current state-of-the-art multiview diffusion models.

Method: The method integrates a Temporal Spatial Vision Transformer (TSViT) with Parameter-Efficient Fine Tuning (PEFT) and a novel post-processing pipeline based on the Fields of The World (FTW) framework. It addresses challenges like clustering of small agricultural parcels by merging wheat segmentation with precise field boundary extraction.

Result: The proposed pipeline produces geometrically coherent and semantically rich maps that enable in-depth analysis such as tracking crop rotation patterns over years. Extensive evaluations demonstrate improved boundary delineation and field-level precision.

Conclusion: The framework shows potential for operational agricultural monitoring and historical trend analysis, laying the foundation for critical studies including crop monitoring and yield estimation through accurate wheat field mapping.

Abstract: Wheat accounts for approximately 20% of the world’s caloric intake, making it a vital component of global food security. Given this importance, mapping wheat fields plays a crucial role in enabling various stakeholders, including policy makers, researchers, and agricultural organizations, to make informed decisions regarding food security, supply chain management, and resource allocation. In this paper, we tackle the problem of accurately mapping wheat fields out of satellite images by introducing an improved pipeline for winter wheat segmentation, as well as presenting a case study on a decade-long analysis of wheat mapping in Lebanon. We integrate a Temporal Spatial Vision Transformer (TSViT) with Parameter-Efficient Fine Tuning (PEFT) and a novel post-processing pipeline based on the Fields of The World (FTW) framework. Our proposed pipeline addresses key challenges encountered in existing approaches, such as the clustering of small agricultural parcels in a single large field. By merging wheat segmentation with precise field boundary extraction, our method produces geometrically coherent and semantically rich maps that enable us to perform in-depth analysis such as tracking crop rotation pattern over years. Extensive evaluations demonstrate improved boundary delineation and field-level precision, establishing the potential of the proposed framework in operational agricultural monitoring and historical trend analysis. By allowing for accurate mapping of wheat fields, this work lays the foundation for a range of critical studies and future advances, including crop monitoring and yield estimation.

Method: Three key innovations: (1) In-context editing paradigm without architectural modifications; (2) Minimal parameter-efficient fine-tuning for quality improvement; (3) Early Filter Inference-Time Scaling using VLMs to select high-quality noise samples.

Result: ICEdit achieves state-of-the-art editing performance with only 0.1% of training data and 1% trainable parameters compared to previous methods.

Conclusion: The approach establishes a new paradigm for balancing precision and efficiency in instructional image editing.

Abstract: Instruction-based image editing enables precise modifications via natural language prompts, but existing methods face a precision-efficiency tradeoff: fine-tuning demands massive datasets (>10M) and computational resources, while training-free approaches suffer from weak instruction comprehension. We address this by proposing ICEdit, which leverages the inherent comprehension and generation abilities of large-scale Diffusion Transformers (DiTs) through three key innovations: (1) An in-context editing paradigm without architectural modifications; (2) Minimal parameter-efficient fine-tuning for quality improvement; (3) Early Filter Inference-Time Scaling, which uses VLMs to select high-quality noise samples for efficiency. Experiments show that ICEdit achieves state-of-the-art editing performance with only 0.1% of the training data and 1% trainable parameters compared to previous methods. Our approach establishes a new paradigm for balancing precision and efficiency in instructional image editing. Codes and demos can be found in https://river-zhang.github.io/ICEdit-gh-pages/.

Method: Uses a multi-task learning foundation model trained on 14 tasks/datasets (10.9M samples) as an embedding extractor, combined with an Embedding-Mixer transformer module for final assessment. Evaluated on behavioral (RGB, thermal, depth) and physiological (ECG, EMG, GSR, fNIRS) modalities.

Result: Achieved state-of-the-art performance on BioVid and AI4Pain datasets, outperforming 75 existing methodologies in both unimodal and multimodal settings across diverse input modalities.

Conclusion: PainFormer effectively extracts high-quality embeddings from various modalities and demonstrates superior performance, paving the way for general-purpose models in automatic pain assessment.

Abstract: Pain is a manifold condition that impacts a significant percentage of the population. Accurate and reliable pain evaluation for the people suffering is crucial to developing effective and advanced pain management protocols. Automatic pain assessment systems provide continuous monitoring and support decision-making processes, ultimately aiming to alleviate distress and prevent functionality decline. This study introduces PainFormer, a vision foundation model based on multi-task learning principles trained simultaneously on 14 tasks/datasets with a total of 10.9 million samples. Functioning as an embedding extractor for various input modalities, the foundation model provides feature representations to the Embedding-Mixer, a transformer-based module that performs the final pain assessment. Extensive experiments employing behavioral modalities - including RGB, synthetic thermal, and estimated depth videos - and physiological modalities such as ECG, EMG, GSR, and fNIRS revealed that PainFormer effectively extracts high-quality embeddings from diverse input modalities. The proposed framework is evaluated on two pain datasets, BioVid and AI4Pain, and directly compared to 75 different methodologies documented in the literature. Experiments conducted in unimodal and multimodal settings demonstrate state-of-the-art performances across modalities and pave the way toward general-purpose models for automatic pain assessment. The foundation model’s architecture (code) and weights are available at: https://github.com/GkikasStefanos/PainFormer.

Method: Created custom dataset by unsoldering common e-waste items (mouse, charger) and taking pictures. Trained YOLOv11 and Mask-RCNN models for object detection and segmentation.

Result: YOLOv11 achieved 70 mAP in real-time, while Mask-RCNN achieved 41 mAP. Both models can be integrated with pick-and-place robots.

Conclusion: The approach successfully demonstrates automated e-waste classification with YOLOv11 showing superior performance, enabling robotic segregation of electronic waste.

Abstract: Industry partners provided a problem statement that involves classifying electronic waste using machine learning models that will be used by pick-and-place robots for waste segregation. This was achieved by taking common electronic waste items, such as a mouse and charger, unsoldering them, and taking pictures to create a custom dataset. Then state-of-the art YOLOv11 model was trained and run to achieve 70 mAP in real-time. Mask-RCNN model was also trained and achieved 41 mAP. The model can be integrated with pick-and-place robots to perform segregation of e-waste.

Method: Split Matching strategy that partitions queries into seen and candidate groups, uses CLIP dense feature clustering for pseudo masks, and introduces Multi-scale Feature Enhancement module for spatial detail refinement.

Result: Achieves state-of-the-art performance on two standard benchmarks for zero-shot semantic segmentation.

Conclusion: SM is the first decoupled Hungarian matching approach for inductive ZSS, effectively handling both seen and unseen categories through independent optimization and multi-scale feature enhancement.

Abstract: Zero-shot Semantic Segmentation (ZSS) aims to segment categories that are not annotated during training. While fine-tuning vision-language models has achieved promising results, these models often overfit to seen categories due to the lack of supervision for unseen classes. As an alternative to fully supervised approaches, query-based segmentation has shown great latent in ZSS, as it enables object localization without relying on explicit labels. However, conventional Hungarian matching, a core component in query-based frameworks, needs full supervision and often misclassifies unseen categories as background in the setting of ZSS. To address this issue, we propose Split Matching (SM), a novel assignment strategy that decouples Hungarian matching into two components: one for seen classes in annotated regions and another for latent classes in unannotated regions (referred to as unseen candidates). Specifically, we partition the queries into seen and candidate groups, enabling each to be optimized independently according to its available supervision. To discover unseen candidates, we cluster CLIP dense features to generate pseudo masks and extract region-level embeddings using CLS tokens. Matching is then conducted separately for the two groups based on both class-level similarity and mask-level consistency. Additionally, we introduce a Multi-scale Feature Enhancement (MFE) module that refines decoder features through residual multi-scale aggregation, improving the model’s ability to capture spatial details across resolutions. SM is the first to introduce decoupled Hungarian matching under the inductive ZSS setting, and achieves state-of-the-art performance on two standard benchmarks.

Method: Proposes two theoretical frameworks: MEPS (Mutually Exclusive Probability Space) and LDH (Local Dependence Hypothesis). Introduces Binary Latent Autoencoder (BL-AE) for encoding images into signed binary latent representations, and γ-Autoregressive Random Variable Model (γ-ARVM) with variable observation range to study dependence patterns.

Result: Experimental results show that as the observation range increases in autoregressive models, they progressively shift toward memorization. In the limit of global dependence, models behave as pure memorizers when operating on binary latents from BL-AE.

Conclusion: Learning global distributions alone is insufficient for true generative behavior and leads to memorization. Local dependence and limited observation ranges are crucial for achieving genuine generation capabilities in probabilistic models.

Abstract: A common assumption in probabilistic generative models for image generation is that learning the global data distribution suffices to generate novel images via sampling. We investigate the limitation of this core assumption, namely that learning global distributions leads to memorization rather than generative behavior. We propose two theoretical frameworks, the Mutually Exclusive Probability Space (MEPS) and the Local Dependence Hypothesis (LDH), for investigation. MEPS arises from the observation that deterministic mappings (e.g. neural networks) involving random variables tend to reduce overlap coefficients among involved random variables, thereby inducing exclusivity. We further propose a lower bound in terms of the overlap coefficient, and introduce a Binary Latent Autoencoder (BL-AE) that encodes images into signed binary latent representations. LDH formalizes dependence within a finite observation radius, which motivates our $\gamma$-Autoregressive Random Variable Model ($\gamma$-ARVM). $\gamma$-ARVM is an autoregressive model, with a variable observation range $\gamma$, that predicts a histogram for the next token. Using $\gamma$-ARVM, we observe that as the observation range increases, autoregressive models progressively shift toward memorization. In the limit of global dependence, the model behaves as a pure memorizer when operating on the binary latents produced by our BL-AE. Comprehensive experiments and discussions support our investigation.

Method: InstanceBEV synergistically combines map-centric and object-centric approaches by extracting instance-level features within BEV features. This enables global attention modeling in a highly compressed feature space without requiring additional modules for multi-task learning.

Result: On the OCC3D-nuScenes dataset using only 8 frames, InstanceBEV achieves RayPQ of 15.3 and RayIoU of 38.2, surpassing SparseOcc’s RayPQ by 9.3% and RayIoU by 10.7%.

Conclusion: The proposed InstanceBEV method effectively addresses efficiency challenges in BEV-based 3D perception and demonstrates superior performance in 3D occupancy panoptic segmentation through multi-task synergy.

Abstract: BEV-based 3D perception has emerged as a focal point of research in end-to-end autonomous driving. However, existing BEV approaches encounter significant challenges due to the large feature space, complicating efficient modeling and hindering effective integration of global attention mechanisms. We propose a novel modeling strategy, called InstanceBEV, that synergistically combines the strengths of both map-centric approaches and object-centric approaches. Our method effectively extracts instance-level features within the BEV features, facilitating the implementation of global attention modeling in a highly compressed feature space, thereby addressing the efficiency challenges inherent in map-centric global modeling. Furthermore, our approach enables effective multi-task learning without introducing additional module. We validate the efficiency and accuracy of the proposed model through predicting occupancy, achieving 3D occupancy panoptic segmentation by combining instance information. Experimental results on the OCC3D-nuScenes dataset demonstrate that InstanceBEV, utilizing only 8 frames, achieves a RayPQ of 15.3 and a RayIoU of 38.2. This surpasses SparseOcc’s RayPQ by 9.3% and RayIoU by 10.7%, showcasing the effectiveness of multi-task synergy.

Method: Identify attention heads that focus on core semantic representations based on their attention distributions, then use a two-stage optimization paradigm to propagate the advantages of these attention heads across the entire model to align attention distribution with actual information flow.

Result: Evaluated on three image captioning benchmarks using five different LVLMs, the method significantly reduces hallucinations. Experiments reveal a trade-off between reduced hallucinations and richer details, with the method allowing manual adjustment of model conservativeness.

Conclusion: The proposed approach effectively addresses the attention-information flow misalignment in LVLMs, reducing hallucinations while providing flexible control over model conservativeness to meet diverse real-world requirements.

Abstract: Due to the unidirectional masking mechanism, Decoder-Only models propagate information from left to right. LVLMs (Large Vision-Language Models) follow the same architecture, with visual information gradually integrated into semantic representations during forward propagation. Through systematic analysis, we observe that the majority of the visual information is absorbed into the semantic representations. However, the model’s attention distribution does not exhibit sufficient emphasis on semantic representations. This misalignment between the attention distribution and the actual information flow undermines the model’s visual understanding ability and contributes to hallucinations. To address this issue, we enhance the model’s visual understanding by leveraging the core information embedded in semantic representations. Specifically, we identify attention heads that focus on core semantic representations based on their attention distributions. Then, through a two-stage optimization paradigm, we propagate the advantages of these attention heads across the entire model, aligning the attention distribution with the actual information flow. We evaluate our method on three image captioning benchmarks using five different LVLMs, demonstrating its effectiveness in significantly reducing hallucinations. Further experiments reveal a trade-off between reduced hallucinations and richer details. Notably, our method allows for manual adjustment of the model’s conservativeness, enabling flexible control to meet diverse real-world requirements.

Method: Propose Dual Data Alignment (DDA) that aligns both pixel and frequency domains. Introduce two new test sets: DDA-COCO with DDA-aligned synthetic images for testing detector performance, and EvalGEN featuring latest generative models for assessing detectors under new architectures.

Result: Detector trained exclusively on DDA-aligned MSCOCO improved across 8 diverse benchmarks by a non-trivial margin, showing +7.2% improvement on in-the-wild benchmarks, demonstrating improved generalizability of unbiased detectors.

Conclusion: Frequency-level alignment is crucial for synthetic image detection. DDA effectively addresses both pixel and frequency domain misalignments, leading to more robust and generalizable detectors that perform well across diverse testing scenarios.

Abstract: Existing detectors are often trained on biased datasets, leading to the possibility of overfitting on non-causal image attributes that are spuriously correlated with real/synthetic labels. While these biased features enhance performance on the training data, they result in substantial performance degradation when applied to unbiased datasets. One common solution is to perform dataset alignment through generative reconstruction, matching the semantic content between real and synthetic images. However, we revisit this approach and show that pixel-level alignment alone is insufficient. The reconstructed images still suffer from frequency-level misalignment, which can perpetuate spurious correlations. To illustrate, we observe that reconstruction models tend to restore the high-frequency details lost in real images (possibly due to JPEG compression), inadvertently creating a frequency-level misalignment, where synthetic images appear to have richer high-frequency content than real ones. This misalignment leads to models associating high-frequency features with synthetic labels, further reinforcing biased cues. To resolve this, we propose Dual Data Alignment (DDA), which aligns both the pixel and frequency domains. Moreover, we introduce two new test sets: DDA-COCO, containing DDA-aligned synthetic images for testing detector performance on the most aligned dataset, and EvalGEN, featuring the latest generative models for assessing detectors under new generative architectures such as visual auto-regressive generators. Finally, our extensive evaluations demonstrate that a detector trained exclusively on DDA-aligned MSCOCO could improve across 8 diverse benchmarks by a non-trivial margin, showing a +7.2% on in-the-wild benchmarks, highlighting the improved generalizability of unbiased detectors. Our code is available at: https://github.com/roy-ch/Dual-Data-Alignment.

Method: VIBE scores VLM outputs using two metrics: grounding (alignment with visual content) and utility (informativeness for the task), then selects the best summaries by ranking them according to these scores.

Result: Human studies on three datasets show VIBE-selected summaries boost task accuracy by up to 61.23% and reduce response time by 75.77% compared to naive VLM summaries or raw video.

Conclusion: VIBE provides an effective annotation-free approach for improving VLM-generated video summaries to support human decision-making in time-sensitive tasks.

Abstract: Many decision-making tasks, where both accuracy and efficiency matter, still require human supervision. For example, tasks like traffic officers reviewing hour-long dashcam footage or researchers screening conference videos can benefit from concise summaries that reduce cognitive load and save time. Yet current vision-language models (VLMs) often produce verbose, redundant outputs that hinder task performance. Existing video caption evaluation depends on costly human annotations and overlooks the summaries’ utility in downstream tasks. We address these gaps with Video-to-text Information Bottleneck Evaluation (VIBE), an annotation-free method that scores VLM outputs using two metrics: grounding (how well the summary aligns with visual content) and utility (how informative it is for the task). VIBE selects from randomly sampled VLM outputs by ranking them according to the two scores to support effective human decision-making. Human studies on LearningPaper24, SUTD-TrafficQA, and LongVideoBench show that summaries selected by VIBE consistently improve performance-boosting task accuracy by up to 61.23% and reducing response time by 75.77% compared to naive VLM summaries or raw video.

Method: Train an adapter to refine off-the-shelf features using pseudo-labels obtained via 3D-aware chaining, filtering wrong labels through relaxed cyclic consistency and 3D spherical prototype mapping constraints.

Result: Achieves new state-of-the-art on SPair-71k with absolute gains of over 4% and 7% compared to methods with similar supervision requirements.

Conclusion: The proposed 3D-aware pseudo-labeling approach effectively improves semantic correspondence while reducing annotation needs and demonstrates good generalization to other data sources.

Abstract: Finding correspondences between semantically similar points across images and object instances is one of the everlasting challenges in computer vision. While large pre-trained vision models have recently been demonstrated as effective priors for semantic matching, they still suffer from ambiguities for symmetric objects or repeated object parts. We propose improving semantic correspondence estimation through 3D-aware pseudo-labeling. Specifically, we train an adapter to refine off-the-shelf features using pseudo-labels obtained via 3D-aware chaining, filtering wrong labels through relaxed cyclic consistency, and 3D spherical prototype mapping constraints. While reducing the need for dataset-specific annotations compared to prior work, we establish a new state-of-the-art on SPair-71k, achieving an absolute gain of over 4% and of over 7% compared to methods with similar supervision requirements. The generality of our proposed approach simplifies the extension of training to other data sources, which we demonstrate in our experiments.

Method: Proposes optimal transport perspective for 3DGS compaction as global Gaussian mixture reduction. Uses composite transport divergence minimization over KD-tree partition for compact geometric representation, then decouples appearance from geometry by fine-tuning color/opacity with fewer primitives.

Result: Achieves negligible loss in rendering quality (PSNR, SSIM, LPIPS) compared to vanilla 3DGS with only 10% Gaussians. Consistently outperforms state-of-the-art 3DGS compaction techniques.

Conclusion: Provides an efficient and agnostic pathway to lightweight neural rendering applicable to any stage of vanilla or accelerated 3DGS pipelines.

Abstract: 3D Gaussian Splatting (3DGS) has emerged as a powerful technique for radiance field rendering, but it typically requires millions of redundant Gaussian primitives, overwhelming memory and rendering budgets. Existing compaction approaches address this by pruning Gaussians based on heuristic importance scores, without global fidelity guarantee. To bridge this gap, we propose a novel optimal transport perspective that casts 3DGS compaction as global Gaussian mixture reduction. Specifically, we first minimize the composite transport divergence over a KD- tree partition to produce a compact geometric representation, and then decouple appearance from geometry by fine-tuning color and opacity attributes with far fewer Gaussian primitives. Experiments on benchmark datasets show that our method (i) yields negligible loss in rendering quality (PSNR, SSIM, LPIPS) compared to vanilla 3DGS with only 10% Gaussians; and (ii) consistently outperforms state- of-the-art 3DGS compaction techniques. Notably, our method is applicable to any stage of vanilla or accelerated 3DGS pipelines, providing an efficient and agnostic pathway to lightweight neural rendering. The code is publicly available at https://github.com/DrunkenPoet/GHAP

Method: Proposes WaveFormer with a novel learnable wavelet transform integrating time-domain and frequency-domain features, using WaveletConv module with multi-level wavelet decomposition and depthwise separable convolution for efficiency.

Result: Achieves 95% classification accuracy on EPN612 dataset with only 3.1M parameters, and INT8 quantization enables real-time deployment with 6.75 ms inference latency on Intel CPU.

Conclusion: WaveFormer provides an efficient and compact solution for sEMG gesture recognition that outperforms larger models while being suitable for resource-constrained embedded systems.

Abstract: Human-machine interaction, particularly in prosthetic and robotic control, has seen progress with gesture recognition via surface electromyographic (sEMG) signals.However, classifying similar gestures that produce nearly identical muscle signals remains a challenge, often reducing classification accuracy. Traditional deep learning models for sEMG gesture recognition are large and computationally expensive, limiting their deployment on resource-constrained embedded systems. In this work, we propose WaveFormer, a lightweight transformer-based architecture tailored for sEMG gesture recognition. Our model integrates time-domain and frequency-domain features through a novel learnable wavelet transform, enhancing feature extraction. In particular, the WaveletConv module, a multi-level wavelet decomposition layer with depthwise separable convolution, ensures both efficiency and compactness. With just 3.1 million parameters, WaveFormer achieves 95% classification accuracy on the EPN612 dataset, outperforming larger models. Furthermore, when profiled on a laptop equipped with an Intel CPU, INT8 quantization achieves real-time deployment with a 6.75 ms inference latency.

Method: Train foundation models on the 23TB MajorTOM dataset covering all regions (including oceans and ice), develop models using U-Net, ViT, and Mamba architectures with varying parameters, evaluate FLOPs, and fine-tune on PhilEO Bench for roads, buildings, and land cover tasks.

Result: U-Net 200M-2T outperforms other models for most n-shots in roads and buildings tasks; Mamba achieves comparable results with less computational expense; global training doesn’t decrease performance on land-focused tasks.

Conclusion: Large foundation models trained on global datasets can be useful for downstream applications requiring only subsets of training information, with U-Net showing superior performance for most tasks and Mamba offering computational advantages.

Abstract: Today, Earth Observation (EO) satellites generate massive volumes of data. To fully exploit this, it is essential to pretrain EO Foundation Models (FMs) on large unlabeled datasets, enabling efficient fine-tuning for downstream tasks with minimal labeled data. In this paper, we study scaling-up FMs: we train our models on the pretraining dataset MajorTOM 23TB which includes all regions, and the performance on average is competitive versus models pretrained on more specialized datasets which are substantially smaller and include only land. The additional data of oceans and ice do not decrease the performance on land-focused downstream tasks. These results indicate that large FMs trained on global datasets for a wider variety of downstream tasks can be useful for downstream applications that only require a subset of the information included in their training. The second contribution is the exploration of U-Net Convolutional Neural Network (CNN), Vision Transformers (ViT), and Mamba State-Space Models (SSM) as FMs. U-Net captures local correlations amongst pixels, while ViT and Mamba capture local and distant correlations. We develop various models using different architectures, including U-Net, ViT, and Mamba, and different number of parameters. We evaluate the FLoating-point OPerations (FLOPs) needed by the models. We fine-tune on the PhilEO Bench for different downstream tasks: roads, buildings, and land cover. For most n-shots for roads and buildings, U-Net 200M-2T outperforms the other models. Using Mamba, we achieve comparable results on the downstream tasks, with less computational expenses. We also compare with the recent FM TerraMind which we evaluate on PhilEO Bench.

Method: Introduces EUAV-IP (explicit uncertainty-aware) that skips uncertain concepts via masking, and IUAV-IP (implicit uncertainty-aware) that incorporates uncertainty into query selection for more informed decisions.

Result: IUAV-IP achieves state-of-the-art accuracy among interpretable-by-design approaches on 4 out of 5 medical imaging datasets and generates more concise explanations by selecting fewer yet more informative concepts.

Conclusion: The uncertainty-aware framework enables more reliable and clinically meaningful outcomes, enhancing model trustworthiness and supporting safer AI deployment in healthcare.

Abstract: To be adopted in safety-critical domains like medical image analysis, AI systems must provide human-interpretable decisions. Variational Information Pursuit (V-IP) offers an interpretable-by-design framework by sequentially querying input images for human-understandable concepts, using their presence or absence to make predictions. However, existing V-IP methods overlook sample-specific uncertainty in concept predictions, which can arise from ambiguous features or model limitations, leading to suboptimal query selection and reduced robustness. In this paper, we propose an interpretable and uncertainty-aware framework for medical imaging that addresses these limitations by accounting for upstream uncertainties in concept-based, interpretable-by-design models. Specifically, we introduce two uncertainty-aware models, EUAV-IP and IUAV-IP, that integrate uncertainty estimates into the V-IP querying process to prioritize more reliable concepts per sample. EUAV-IP skips uncertain concepts via masking, while IUAV-IP incorporates uncertainty into query selection implicitly for more informed and clinically aligned decisions. Our approach allows models to make reliable decisions based on a subset of concepts tailored to each individual sample, without human intervention, while maintaining overall interpretability. We evaluate our methods on five medical imaging datasets across four modalities: dermoscopy, X-ray, ultrasound, and blood cell imaging. The proposed IUAV-IP model achieves state-of-the-art accuracy among interpretable-by-design approaches on four of the five datasets, and generates more concise explanations by selecting fewer yet more informative concepts. These advances enable more reliable and clinically meaningful outcomes, enhancing model trustworthiness and supporting safer AI deployment in healthcare.

Method: Created a comprehensive dataset with 14,120 scans of 217 unique parts using four industrial depth sensors, capturing real production conditions including variations in placement, lighting, and occlusion.

Result: Benchmarking shows 3D-ADAM presents substantial challenges beyond existing datasets, and expert validation confirms its industrial relevance.

Conclusion: 3D-ADAM establishes a foundation for advancing robust 3D anomaly detection capable of meeting real manufacturing demands.

Abstract: Surface defects are a primary source of yield loss in manufacturing, yet existing anomaly detection methods often fail in real-world deployment due to limited and unrepresentative datasets. To overcome this, we introduce 3D-ADAM, a 3D Anomaly Detection in Additive Manufacturing dataset, that is the first large-scale, industry-relevant dataset for RGB+3D surface defect detection in additive manufacturing. 3D-ADAM comprises 14,120 high-resolution scans of 217 unique parts, captured with four industrial depth sensors, and includes 27,346 annotated defects across 12 categories along with 27,346 annotations of machine element features in 16 classes. 3D-ADAM is captured in a real industrial environment and as such reflects real production conditions, including variations in part placement, sensor positioning, lighting, and partial occlusion. Benchmarking state-of-the-art models demonstrates that 3D-ADAM presents substantial challenges beyond existing datasets. Validation through expert labelling surveys with industry partners further confirms its industrial relevance. By providing this benchmark, 3D-ADAM establishes a foundation for advancing robust 3D anomaly detection capable of meeting manufacturing demands.

Method: Integrates four components: edge-enhanced ResNet backbone, stride-free space-to-depth module, cross-stage dual-domain multi-scale attention, and Focaler-Wise-SIoU regression loss to improve small target detection.

Result: Extensive experiments show superior detection performance and strong generalization capability with modest computational overhead on both public and real-world datasets.

Conclusion: TinyDef-DETR is an effective solution for UAV-based transmission line defect detection, particularly for small and ambiguous targets, balancing accuracy and efficiency.

Abstract: Automated defect detection from UAV imagery of transmission lines is a challenging task due to the small size, ambiguity, and complex backgrounds of defects. This paper proposes TinyDef-DETR, a DETR-based framework designed to achieve accurate and efficient detection of transmission line defects from UAV-acquired images. The model integrates four major components: an edge-enhanced ResNet backbone to strengthen boundary-sensitive representations, a stride-free space-to-depth module to enable detail-preserving downsampling, a cross-stage dual-domain multi-scale attention mechanism to jointly model global context and local cues, and a Focaler-Wise-SIoU regression loss to improve the localization of small and difficult targets. Together, these designs effectively mitigate the limitations of conventional detectors. Extensive experiments on both public and real-world datasets demonstrate that TinyDef-DETR achieves superior detection performance and strong generalization capability, while maintaining modest computational overhead. The accuracy and efficiency of TinyDef-DETR make it a suitable method for UAV-based transmission line defect detection, particularly in scenarios involving small and ambiguous targets.

Method: Proposes Domain-And-Time Alignment (DATA) network with three modules: Consistency-preserving Domain Alignment Module (CDAM) for domain gap reduction, Progressive Temporal Alignment Module (PTAM) for handling transmission delays, and Instance-focused Feature Aggregation Module (IFAM) for semantic representation enhancement.

Result: Extensive experiments show DATA achieves state-of-the-art performance on three typical datasets, maintaining robustness with severe communication delays and pose errors.

Conclusion: The DATA network effectively addresses domain and temporal alignment challenges in collaborative perception, demonstrating superior performance and robustness in feature-level fusion scenarios.

Abstract: Feature-level fusion shows promise in collaborative perception (CP) through balanced performance and communication bandwidth trade-off. However, its effectiveness critically relies on input feature quality. The acquisition of high-quality features faces domain gaps from hardware diversity and deployment conditions, alongside temporal misalignment from transmission delays. These challenges degrade feature quality with cumulative effects throughout the collaborative network. In this paper, we present the Domain-And-Time Alignment (DATA) network, designed to systematically align features while maximizing their semantic representations for fusion. Specifically, we propose a Consistency-preserving Domain Alignment Module (CDAM) that reduces domain gaps through proximal-region hierarchical downsampling and observability-constrained discriminator. We further propose a Progressive Temporal Alignment Module (PTAM) to handle transmission delays via multi-scale motion modeling and two-stage compensation. Building upon the aligned features, an Instance-focused Feature Aggregation Module (IFAM) is developed to enhance semantic representations. Extensive experiments demonstrate that DATA achieves state-of-the-art performance on three typical datasets, maintaining robustness with severe communication delays and pose errors. The code will be released at https://github.com/ChengchangTian/DATA.

Method: Proposes LRQ-DiT with two key components: (1) Twin-Log Quantization (TLQ) - log-based method that better aligns with weight distributions by allocating more intervals to dense regions; (2) Adaptive Rotation Scheme (ARS) - dynamically applies Hadamard or outlier-aware rotations based on activation fluctuations to mitigate both mild and salient outliers.

Result: Extensive experiments on various text-to-image and text-to-video DiT models demonstrate that LRQ-DiT preserves high generation quality while achieving effective compression.

Conclusion: LRQ-DiT provides an effective solution for compressing DiT models through innovative quantization techniques that address specific distribution and outlier challenges, enabling practical deployment in resource-constrained environments.

Abstract: Diffusion Transformers (DiTs) have achieved impressive performance in text-to-image and text-to-video generation. However, their high computational cost and large parameter sizes pose significant challenges for usage in resource-constrained scenarios. Effective compression of models has become a crucial issue that urgently needs to be addressed. Post-training quantization (PTQ) is a promising solution to reduce memory usage and accelerate inference, but existing PTQ methods suffer from severe performance degradation under extreme low-bit settings. After experiments and analysis, we identify two key obstacles to low-bit PTQ for DiTs: (1) the weights of DiT models follow a Gaussian-like distribution with long tails, causing uniform quantization to poorly allocate intervals and leading to significant quantization errors. This issue has been observed in the linear layer weights of different DiT models, which deeply limits the performance. (2) two types of activation outliers in DiT models: (i) Mild Outliers with slightly elevated values, and (ii) Salient Outliers with large magnitudes concentrated in specific channels, which disrupt activation quantization. To address these issues, we propose LRQ-DiT, an efficient and accurate post-training quantization framework for image and video generation. First, we introduce Twin-Log Quantization (TLQ), a log-based method that allocates more quantization intervals to the intermediate dense regions, effectively achieving alignment with the weight distribution and reducing quantization errors. Second, we propose an Adaptive Rotation Scheme (ARS) that dynamically applies Hadamard or outlier-aware rotations based on activation fluctuation, effectively mitigating the impact of both types of outliers. Extensive experiments on various text-to-image and text-to-video DiT models demonstrate that LRQ-DiT preserves high generation quality.

Method: Proposes PointAD (implicit 3D representation via point-pixel correspondence) and PointAD+ (explicit 3D representation with G-aggregation for spatial awareness). Uses hierarchical representation learning with rendering and geometry prompts, cross-hierarchy contrastive alignment, and integrates RGB information plug-and-play during testing.

Result: Extensive experiments demonstrate PointAD+’s superiority in zero-shot 3D anomaly detection across unseen objects with highly diverse class semantics, achieving holistic understanding of abnormality.

Conclusion: PointAD+ successfully transfers CLIP’s 2D capabilities to 3D anomaly detection by comprehensively capturing both rendering and spatial abnormalities through a unified framework with hierarchical learning and cross-layer interaction.

Abstract: In this paper, we aim to transfer CLIP’s robust 2D generalization capabilities to identify 3D anomalies across unseen objects of highly diverse class semantics. To this end, we propose a unified framework to comprehensively detect and segment 3D anomalies by leveraging both point- and pixel-level information. We first design PointAD, which leverages point-pixel correspondence to represent 3D anomalies through their associated rendering pixel representations. This approach is referred to as implicit 3D representation, as it focuses solely on rendering pixel anomalies but neglects the inherent spatial relationships within point clouds. Then, we propose PointAD+ to further broaden the interpretation of 3D anomalies by introducing explicit 3D representation, emphasizing spatial abnormality to uncover abnormal spatial relationships. Hence, we propose G-aggregation to involve geometry information to enable the aggregated point representations spatially aware. To simultaneously capture rendering and spatial abnormality, PointAD+ proposes hierarchical representation learning, incorporating implicit and explicit anomaly semantics into hierarchical text prompts: rendering prompts for the rendering layer and geometry prompts for the geometry layer. A cross-hierarchy contrastive alignment is further introduced to promote the interaction between the rendering and geometry layers, facilitating mutual anomaly learning. Finally, PointAD+ integrates anomaly semantics from both layers to capture the generalized anomaly semantics. During the test, PointAD+ can integrate RGB information in a plug-and-play manner and further improve its detection performance. Extensive experiments demonstrate the superiority of PointAD+ in ZS 3D anomaly detection across unseen objects with highly diverse class semantics, achieving a holistic understanding of abnormality.

Method: Train a Chain-of-Thought rewriter through reinforcement learning guided by AlignEvaluator - a reward model trained on 24 key failure points derived from common T2I problems. The framework decouples rewriting from generation without modifying model weights.

Result: Extensive experiments on HunyuanImage 2.1 show significant improvements in image-text alignment across various semantic and compositional challenges.

Conclusion: PromptEnhancer effectively enhances T2I model performance by generating more precisely interpretable prompts, and introduces a new human preference benchmark for future research.

Abstract: Recent advancements in text-to-image (T2I) diffusion models have demonstrated remarkable capabilities in generating high-fidelity images. However, these models often struggle to faithfully render complex user prompts, particularly in aspects like attribute binding, negation, and compositional relationships. This leads to a significant mismatch between user intent and the generated output. To address this challenge, we introduce PromptEnhancer, a novel and universal prompt rewriting framework that enhances any pretrained T2I model without requiring modifications to its weights. Unlike prior methods that rely on model-specific fine-tuning or implicit reward signals like image-reward scores, our framework decouples the rewriter from the generator. We achieve this by training a Chain-of-Thought (CoT) rewriter through reinforcement learning, guided by a dedicated reward model we term the AlignEvaluator. The AlignEvaluator is trained to provide explicit and fine-grained feedback based on a systematic taxonomy of 24 key points, which are derived from a comprehensive analysis of common T2I failure modes. By optimizing the CoT rewriter to maximize the reward from our AlignEvaluator, our framework learns to generate prompts that are more precisely interpreted by T2I models. Extensive experiments on the HunyuanImage 2.1 model demonstrate that PromptEnhancer significantly improves image-text alignment across a wide range of semantic and compositional challenges. Furthermore, we introduce a new, high-quality human preference benchmark to facilitate future research in this direction.

Method: Replaces memory-intensive spherical harmonics with lightweight spherical Gaussian lobes for color representation, and introduces a unified soft pruning framework that models primitive-number and lobe-number pruning as a constrained optimization problem.

Result: Achieves 50% static VRAM reduction and 40% rendering VRAM reduction compared to existing methods while maintaining comparable rendering quality.

Conclusion: MEGS² successfully addresses the memory bottleneck in 3DGS rendering through joint optimization of primitive count and parameter efficiency, enabling more practical deployment on resource-constrained devices.

Abstract: 3D Gaussian Splatting (3DGS) has emerged as a dominant novel-view synthesis technique, but its high memory consumption severely limits its applicability on edge devices. A growing number of 3DGS compression methods have been proposed to make 3DGS more efficient, yet most only focus on storage compression and fail to address the critical bottleneck of rendering memory. To address this problem, we introduce MEGS$^{2}$, a novel memory-efficient framework that tackles this challenge by jointly optimizing two key factors: the total primitive number and the parameters per primitive, achieving unprecedented memory compression. Specifically, we replace the memory-intensive spherical harmonics with lightweight, arbitrarily oriented spherical Gaussian lobes as our color representations. More importantly, we propose a unified soft pruning framework that models primitive-number and lobe-number pruning as a single constrained optimization problem. Experiments show that MEGS$^{2}$ achieves a 50% static VRAM reduction and a 40% rendering VRAM reduction compared to existing methods, while maintaining comparable rendering quality. Project page: https://megs-2.github.io/

Method: LD-ViCE operates in latent space using a state-of-the-art diffusion model with an additional refinement step to generate realistic counterfactual explanations for video AI models.

Result: LD-ViCE outperforms state-of-the-art methods with up to 68% increase in R2 score and 50% reduction in inference time across three diverse video datasets (EchoNet-Dynamic, FERV39k, Something-Something V2).

Conclusion: LD-ViCE generates semantically meaningful and temporally coherent explanations, representing a valuable step toward trustworthy AI deployment in safety-critical domains.

Abstract: Video-based AI systems are increasingly adopted in safety-critical domains such as autonomous driving and healthcare. However, interpreting their decisions remains challenging due to the inherent spatiotemporal complexity of video data and the opacity of deep learning models. Existing explanation techniques often suffer from limited temporal coherence, insufficient robustness, and a lack of actionable causal insights. Current counterfactual explanation methods typically do not incorporate guidance from the target model, reducing semantic fidelity and practical utility. We introduce Latent Diffusion for Video Counterfactual Explanations (LD-ViCE), a novel framework designed to explain the behavior of video-based AI models. Compared to previous approaches, LD-ViCE reduces the computational costs of generating explanations by operating in latent space using a state-of-the-art diffusion model, while producing realistic and interpretable counterfactuals through an additional refinement step. Our experiments demonstrate the effectiveness of LD-ViCE across three diverse video datasets, including EchoNet-Dynamic (cardiac ultrasound), FERV39k (facial expression), and Something-Something V2 (action recognition). LD-ViCE outperforms a recent state-of-the-art method, achieving an increase in R2 score of up to 68% while reducing inference time by half. Qualitative analysis confirms that LD-ViCE generates semantically meaningful and temporally coherent explanations, offering valuable insights into the target model behavior. LD-ViCE represents a valuable step toward the trustworthy deployment of AI in safety-critical domains.

Method: BEAMER uses a beam search strategy to propagate multiple hypotheses at each scale and integrates these multiple hypotheses into cross-attention layers, learning to preserve and propagate multiple correspondent hypotheses across scales.

Result: BEAMER is significantly more robust than state-of-the-art methods, particularly at depth discontinuities and when the target image is a strong zoom-in of the source image.

Conclusion: Predicting multiple correspondent hypotheses per source location at each scale through beam search and cross-attention integration provides substantial improvements in dense image matching robustness compared to single-hypothesis approaches.

Abstract: Dense image matching aims to find a correspondent for every pixel of a source image in a partially overlapping target image. State-of-the-art methods typically rely on a coarse-to-fine mechanism where a single correspondent hypothesis is produced per source location at each scale. In challenging cases – such as at depth discontinuities or when the target image is a strong zoom-in of the source image – the correspondents of neighboring source locations are often widely spread and predicting a single correspondent hypothesis per source location at each scale may lead to erroneous matches. In this paper, we investigate the idea of predicting multiple correspondent hypotheses per source location at each scale instead. We consider a beam search strategy to propagat multiple hypotheses at each scale and propose integrating these multiple hypotheses into cross-attention layers, resulting in a novel dense matching architecture called BEAMER. BEAMER learns to preserve and propagate multiple hypotheses across scales, making it significantly more robust than state-of-the-art methods, especially at depth discontinuities or when the target image is a strong zoom-in of the source image.

Method: Proposes using features from a Vision Diffusion Model (VDM) aggregated via transformer, with conditioning on earlier timesteps to emphasize semantic over pixel-level information.

Result: Sets new state-of-the-art across three generalization benchmarks: animal species classification, viewpoint variations, and context differences.

Conclusion: The approach brings machine action recognition closer to human-like robustness by leveraging diffusion model features that capture semantic information effectively.

Abstract: Humans can recognize the same actions despite large context and viewpoint variations, such as differences between species (walking in spiders vs. horses), viewpoints (egocentric vs. third-person), and contexts (real life vs movies). Current deep learning models struggle with such generalization. We propose using features generated by a Vision Diffusion Model (VDM), aggregated via a transformer, to achieve human-like action recognition across these challenging conditions. We find that generalization is enhanced by the use of a model conditioned on earlier timesteps of the diffusion process to highlight semantic information over pixel level details in the extracted features. We experimentally explore the generalization properties of our approach in classifying actions across animal species, across different viewing angles, and different recording contexts. Our model sets a new state-of-the-art across all three generalization benchmarks, bringing machine action recognition closer to human-like robustness. Project page: https://www.vision.caltech.edu/actiondiff. Code: https://github.com/frankyaoxiao/ActionDiff

Method: Multi-stage reasoning pipeline (Text Summary with BBox, Language Identification, Spatial Object-level Captioning, Step-by-step Logical Reasoning) with automated data curation and two-stage training (SFT + Language-aware GRPO) using multi-aspect rewards.

Result: Achieves up to 9.5% accuracy improvements over open-source baselines, surpasses 2x larger models by 2.6%, and outperforms proprietary models like GPT-4o-0513 and Gemini-2.5-flash on multiple datasets.

Conclusion: LaV-CoT effectively bridges the gap in multilingual visual reasoning, demonstrating superior performance and practical applicability for industrial deployment through validated A/B testing.

Abstract: As large vision language models (VLMs) advance, their capabilities in multilingual visual question answering (mVQA) have significantly improved. Chain-of-thought (CoT) reasoning has been proven to enhance interpretability and complex reasoning. However, most existing approaches rely primarily on textual CoT and provide limited support for multilingual multimodal reasoning, constraining their deployment in real-world applications. To address this gap, we introduce \textbf{LaV-CoT}, the first Language-aware Visual CoT framework with Multi-Aspect Reward Optimization. LaV-CoT incorporates an interpretable multi-stage reasoning pipeline consisting of Text Summary with Bounding Box (BBox), Language Identification, Spatial Object-level Captioning, and Step-by-step Logical Reasoning. Following this reasoning pipeline, we design an automated data curation method that generates multilingual CoT annotations through iterative generation, correction, and refinement, enabling scalable and high-quality training data. To improve reasoning and generalization, LaV-CoT adopts a two-stage training paradigm combining Supervised Fine-Tuning (SFT) with Language-aware Group Relative Policy Optimization (GRPO), guided by verifiable multi-aspect rewards including language consistency, structural accuracy, and semantic alignment. Extensive evaluations on public datasets including MMMB, Multilingual MMBench, and MTVQA show that LaV-CoT achieves up to ~9.5% accuracy improvements over open-source baselines of similar size and even surpasses models with 2$\times$ larger scales by ~2.6%. Moreover, LaV-CoT outperforms advanced proprietary models such as GPT-4o-0513 and Gemini-2.5-flash. We further conducted an online A/B test to validate our method on real-world data, highlighting its effectiveness for industrial deployment. Our code is available at this link: \href{https://github.com/HJNVR/LaV-CoT}

Method: The paper develops a structured taxonomy to classify existing methods, performs quantitative comparisons of datasets, compiles unified evaluation metrics, and establishes benchmark tables for both tasks.

Result: Provides a comprehensive analysis of existing approaches, their strengths and limitations, along with standardized benchmarks and a continuously updated webpage for organizing research in this field.

Conclusion: The review establishes foundational resources for LiDAR-based 3D human understanding and identifies critical open challenges and research directions to advance the field.

Abstract: In this paper, we present a comprehensive review of 3D human pose estimation and human mesh recovery from in-the-wild LiDAR point clouds. We compare existing approaches across several key dimensions, and propose a structured taxonomy to classify these methods. Following this taxonomy, we analyze each method’s strengths, limitations, and design choices. In addition, (i) we perform a quantitative comparison of the three most widely used datasets, detailing their characteristics; (ii) we compile unified definitions of all evaluation metrics; and (iii) we establish benchmark tables for both tasks on these datasets to enable fair comparisons and promote progress in the field. We also outline open challenges and research directions critical for advancing LiDAR-based 3D human understanding. Moreover, we maintain an accompanying webpage that organizes papers according to our taxonomy and continuously update it with new studies: https://github.com/valeoai/3D-Human-Pose-Shape-Estimation-from-LiDAR

Method: Uses a translation module to map student features into a joint space, trained with dual-objective loss enforcing both local alignment and global relational consistency at the object level between teacher (e.g., LLaVa) and student (e.g., YOLO) models.

Result: Achieves consistent gains over baselines with +10.1 average score improvement across four personalized detection benchmarks under few-shot regimes, reaching performance comparable to larger multimodal models.

Conclusion: MOCHA proves suitable for real-world deployment by enabling compact vision-only detectors to achieve multimodal-level performance through efficient object-level knowledge transfer.

Abstract: We introduce MOCHA (Multi-modal Objects-aware Cross-arcHitecture Alignment), a knowledge distillation approach that transfers region-level multimodal semantics from a large vision-language teacher (e.g., LLaVa) into a lightweight vision-only object detector student (e.g., YOLO). A translation module maps student features into a joint space, where the training of the student and translator is guided by a dual-objective loss that enforces both local alignment and global relational consistency. Unlike prior approaches focused on dense or global alignment, MOCHA operates at the object level, enabling efficient transfer of semantics without modifying the teacher or requiring textual input at inference. We validate our method across four personalized detection benchmarks under few-shot regimes. Results show consistent gains over baselines, with a +10.1 average score improvement. Despite its compact architecture, MOCHA reaches performance on par with larger multimodal models, proving its suitability for real-world deployment.

Method: Uses multimodal vision-language model with lightweight heads trained on human-centric video labels, and introduces Dynamic SinkCache for constant memory usage in infinite streams while maintaining hidden representations of task objectives.

Result: Achieves SOTA performance with +5.9% mAP on TVSum and +8.3% mAP on Mr. Hisum, surpassing even offline approaches and video-language models.

Conclusion: Demonstrates effectiveness as real-time reasoning module for robotics applications with task-oriented natural language inputs and continuous video streams, enabling downstream planning and long-horizon understanding.

Abstract: Real-time understanding of continuous video streams is essential for intelligent agents operating in high-stakes environments, including autonomous vehicles, surveillance drones, and disaster response robots. Yet, most existing video understanding and highlight detection methods assume access to the entire video during inference, making them unsuitable for online or streaming scenarios. In particular, current models optimize for offline summarization, failing to support step-by-step reasoning needed for real-time decision-making. We introduce Aha, an autoregressive highlight detection framework that predicts the relevance of each video frame against a task described in natural language. Without accessing future video frames, Aha utilizes a multimodal vision-language model and lightweight, decoupled heads trained on a large, curated dataset of human-centric video labels. To enable scalability, we introduce the Dynamic SinkCache mechanism that achieves constant memory usage across infinite-length streams without degrading performance on standard benchmarks. This encourages the hidden representation to capture high-level task objectives, enabling effective frame-level rankings for informativeness, relevance, and uncertainty with respect to the natural language task. Aha achieves state-of-the-art (SOTA) performance on highlight detection benchmarks, surpassing even prior offline, full-context approaches and video-language models by +5.9% on TVSum and +8.3% on Mr. Hisum in mAP (mean Average Precision). We explore Aha’s potential for real-world robotics applications given a task-oriented natural language input and a continuous, robot-centric video. Both experiments demonstrate Aha’s potential effectiveness as a real-time reasoning module for downstream planning and long-horizon understanding.

Method: Joint optimization of constrained planar (2D) Gaussians for modeling flat surfaces and freeform (3D) Gaussians for the rest of the scene. The approach dynamically detects and refines planar regions in an end-to-end manner.

Result: Achieves state-of-the-art depth estimation on ScanNet++ and ScanNetv2 datasets, and excels at mesh extraction without overfitting to specific camera models.

Conclusion: The hybrid representation effectively produces high-quality reconstructions of indoor scenes by addressing the limitations of current methods for flat surfaces while maintaining visual fidelity.

Abstract: Recent advances in radiance fields and novel view synthesis enable creation of realistic digital twins from photographs. However, current methods struggle with flat, texture-less surfaces, creating uneven and semi-transparent reconstructions, due to an ill-conditioned photometric reconstruction objective. Surface reconstruction methods solve this issue but sacrifice visual quality. We propose a novel hybrid 2D/3D representation that jointly optimizes constrained planar (2D) Gaussians for modeling flat surfaces and freeform (3D) Gaussians for the rest of the scene. Our end-to-end approach dynamically detects and refines planar regions, improving both visual fidelity and geometric accuracy. It achieves state-of-the-art depth estimation on ScanNet++ and ScanNetv2, and excels at mesh extraction without overfitting to a specific camera model, showing its effectiveness in producing high-quality reconstruction of indoor scenes.

Method: Three-stage framework: (1) generate diverse 3D meshes, (2) texture with diffusion models, (3) apply AU-driven face rigging to synthesize multi-view faces with neutral/pain pairs, AU configurations, PSPI scores, and pain-region heatmaps. ViTPain uses cross-modal distillation where a heatmap-trained teacher guides an RGB-trained student.

Result: Created 3DPain dataset with 82,500 samples across 25,000 pain expression heatmaps and 2,500 synthetic identities balanced by age, gender, and ethnicity. ViTPain framework enhances accuracy, interpretability, and clinical reliability.

Conclusion: 3DPain and ViTPain establish a controllable, diverse, and clinically grounded foundation for generalizable automated pain assessment, addressing key limitations in current approaches.

Abstract: Automated pain assessment from facial expressions is crucial for non-communicative patients, such as those with dementia. Progress has been limited by two challenges: (i) existing datasets exhibit severe demographic and label imbalance due to ethical constraints, and (ii) current generative models cannot precisely control facial action units (AUs), facial structure, or clinically validated pain levels. We present 3DPain, a large-scale synthetic dataset specifically designed for automated pain assessment, featuring unprecedented annotation richness and demographic diversity. Our three-stage framework generates diverse 3D meshes, textures them with diffusion models, and applies AU-driven face rigging to synthesize multi-view faces with paired neutral and pain images, AU configurations, PSPI scores, and the first dataset-level annotations of pain-region heatmaps. The dataset comprises 82,500 samples across 25,000 pain expression heatmaps and 2,500 synthetic identities balanced by age, gender, and ethnicity. We further introduce ViTPain, a Vision Transformer based cross-modal distillation framework in which a heatmap-trained teacher guides a student trained on RGB images, enhancing accuracy, interpretability, and clinical reliability. Together, 3DPain and ViTPain establish a controllable, diverse, and clinically grounded foundation for generalizable automated pain assessment.

Method: Proposes a synergistic learning pretraining framework with consistency, reconstructive, and aligned learning. Treats one modality as augmented sample of another for self-supervised pretraining. Fine-tuning uses separate encoders for original modalities plus a fusion encoder, with distribution shift method to reduce prediction uncertainty and overfitting.

Result: Extensive experiments on gastroscopy image datasets Kvasir and Kvasirv2 show the method outperforms current state-of-the-art classification methods in both quantitative and qualitative evaluations.

Conclusion: The proposed framework effectively addresses multimodal fusion challenges in semi-supervised medical image classification, demonstrating superior performance over existing methods when labeled data is limited.

Abstract: Multimodal pathological images are usually in clinical diagnosis, but computer vision-based multimodal image-assisted diagnosis faces challenges with modality fusion, especially in the absence of expert-annotated data. To achieve the modality fusion in multimodal images with label scarcity, we propose a novel ``pretraining + fine-tuning" framework for multimodal semi-supervised medical image classification. Specifically, we propose a synergistic learning pretraining framework of consistency, reconstructive, and aligned learning. By treating one modality as an augmented sample of another modality, we implement a self-supervised learning pre-train, enhancing the baseline model’s feature representation capability. Then, we design a fine-tuning method for multimodal fusion. During the fine-tuning stage, we set different encoders to extract features from the original modalities and provide a multimodal fusion encoder for fusion modality. In addition, we propose a distribution shift method for multimodal fusion features, which alleviates the prediction uncertainty and overfitting risks caused by the lack of labeled samples. We conduct extensive experiments on the publicly available gastroscopy image datasets Kvasir and Kvasirv2. Quantitative and qualitative results demonstrate that the proposed method outperforms the current state-of-the-art classification methods. The code will be released at: https://github.com/LQH89757/MICS.

Method: Min learns task-specific noise from high-dimensional features of new tasks, dynamically adjusts weights for optimal mixture of different task noise, and embeds beneficial noise into intermediate features to mask inefficient patterns.

Result: Extensive experiments on six benchmark datasets show Min achieves state-of-the-art performance in most incremental settings, with particularly outstanding results in 50-steps incremental settings.

Conclusion: The approach demonstrates significant potential for beneficial noise in continual learning, showing that properly managed noise can improve class incremental learning performance.

Abstract: Class Incremental Learning (CIL) aims to continuously learn new categories while retaining the knowledge of old ones. Pre-trained models (PTMs) show promising capabilities in CIL. However, existing approaches that apply lightweight fine-tuning to backbones still induce parameter drift, thereby compromising the generalization capability of pre-trained models. Parameter drift can be conceptualized as a form of noise that obscures critical patterns learned for previous tasks. However, recent researches have shown that noise is not always harmful. For example, the large number of visual patterns learned from pre-training can be easily abused by a single task, and introducing appropriate noise can suppress some low-correlation features, thus leaving a margin for future tasks. To this end, we propose learning beneficial noise for CIL guided by information theory and propose Mixture of Noise (Min), aiming to mitigate the degradation of backbone generalization from adapting new tasks. Specifically, task-specific noise is learned from high-dimension features of new tasks. Then, a set of weights is adjusted dynamically for optimal mixture of different task noise. Finally, Min embeds the beneficial noise into the intermediate features to mask the response of inefficient patterns. Extensive experiments on six benchmark datasets demonstrate that Min achieves state-of-the-art performance in most incremental settings, with particularly outstanding results in 50-steps incremental settings. This shows the significant potential for beneficial noise in continual learning. Code is available at https://github.com/ASCIIJK/MiN-NeurIPS2025.

Method: Proposes SAM-DCE which balances local discrimination and global semantics while mitigating token uniformity. Enhances inter-class separability and enriches mask decoding with fine-grained, consistent representations.

Result: Extensive experiments on diverse medical benchmarks validate the effectiveness of SAM-DCE.

Conclusion: SAM-DCE successfully addresses SAM’s limitations in medical imaging by improving robustness and adaptability through better representation learning.

Abstract: The Segment Anything Model (SAM) demonstrates impressive zero-shot segmentation ability on natural images but encounters difficulties in medical imaging due to domain shifts, anatomical variability, and its reliance on user-provided prompts. Recent prompt-free adaptations alleviate the need for expert intervention, yet still suffer from limited robustness and adaptability, often overlooking the issues of semantic over-smoothing and token uniformity. We propose SAM-DCE, which balances local discrimination and global semantics while mitigating token uniformity, enhancing inter-class separability, and enriching mask decoding with fine-grained, consistent representations. Extensive experiments on diverse medical benchmarks validate its effectiveness.

Method: Introduces a hybrid-level metric combining pixel- and target-level performance, proposes systematic error analysis method, and emphasizes cross-dataset evaluation.

Result: The proposed framework provides a more thorough hierarchical analysis for IRSTD models, with an open-source toolkit released for standardized benchmarking.

Conclusion: This work aims to foster development of more effective and robust IRSTD models through improved evaluation protocols that offer comprehensive performance assessment and error analysis.

Abstract: As an essential vision task, infrared small target detection (IRSTD) has seen significant advancements through deep learning. However, critical limitations in current evaluation protocols impede further progress. First, existing methods rely on fragmented pixel- and target-level specific metrics, which fails to provide a comprehensive view of model capabilities. Second, an excessive emphasis on overall performance scores obscures crucial error analysis, which is vital for identifying failure modes and improving real-world system performance. Third, the field predominantly adopts dataset-specific training-testing paradigms, hindering the understanding of model robustness and generalization across diverse infrared scenarios. This paper addresses these issues by introducing a hybrid-level metric incorporating pixel- and target-level performance, proposing a systematic error analysis method, and emphasizing the importance of cross-dataset evaluation. These aim to offer a more thorough and rational hierarchical analysis framework, ultimately fostering the development of more effective and robust IRSTD models. An open-source toolkit has be released to facilitate standardized benchmarking.

Method: The proposed method penalizes activation values at image boundaries during early denoising steps. This training-free solution requires minimal modifications to pre-trained Stable Diffusion models and adds negligible computational overhead.

Result: Extensive experiments show substantial improvements in object integrity and image quality, demonstrating the effectiveness of the boundary activation penalty approach.

Conclusion: The study successfully identifies RandomCrop as the root cause of incomplete object generation and provides an efficient, training-free solution that significantly enhances object completeness in diffusion-based text-to-image generation.

Abstract: Diffusion models have emerged as a powerful technique for text-to-image (T2I) generation, creating high-quality, diverse images across various domains. However, a common limitation in these models is the incomplete display of objects, where fragments or missing parts undermine the model’s performance in downstream applications. In this study, we conduct an in-depth analysis of the incompleteness issue and reveal that the primary factor behind incomplete object generation is the usage of RandomCrop during model training. This widely used data augmentation method, though enhances model generalization ability, disrupts object continuity during training. To address this, we propose a training-free solution that penalizes activation values at image boundaries during the early denoising steps. Our method is easily applicable to pre-trained Stable Diffusion models with minimal modifications and negligible computational overhead. Extensive experiments demonstrate the effectiveness of our method, showing substantial improvements in object integrity and image quality.

Method: HyRF decomposes scenes into: (1) compact explicit Gaussians storing critical high-frequency parameters, and (2) grid-based neural fields predicting remaining properties. It uses a decoupled neural field architecture separately modeling geometry and view-dependent color, plus a hybrid rendering scheme combining Gaussian splatting with neural field-predicted background.

Result: HyRF achieves state-of-the-art rendering quality while reducing model size by over 20 times compared to 3DGS and maintaining real-time performance.

Conclusion: The hybrid approach effectively balances explicit and implicit representations, enabling efficient high-quality novel view synthesis with significantly reduced memory requirements.

Abstract: Recently, 3D Gaussian Splatting (3DGS) has emerged as a powerful alternative to NeRF-based approaches, enabling real-time, high-quality novel view synthesis through explicit, optimizable 3D Gaussians. However, 3DGS suffers from significant memory overhead due to its reliance on per-Gaussian parameters to model view-dependent effects and anisotropic shapes. While recent works propose compressing 3DGS with neural fields, these methods struggle to capture high-frequency spatial variations in Gaussian properties, leading to degraded reconstruction of fine details. We present Hybrid Radiance Fields (HyRF), a novel scene representation that combines the strengths of explicit Gaussians and neural fields. HyRF decomposes the scene into (1) a compact set of explicit Gaussians storing only critical high-frequency parameters and (2) grid-based neural fields that predict remaining properties. To enhance representational capacity, we introduce a decoupled neural field architecture, separately modeling geometry (scale, opacity, rotation) and view-dependent color. Additionally, we propose a hybrid rendering scheme that composites Gaussian splatting with a neural field-predicted background, addressing limitations in distant scene representation. Experiments demonstrate that HyRF achieves state-of-the-art rendering quality while reducing model size by over 20 times compared to 3DGS and maintaining real-time performance. Our project page is available at https://wzpscott.github.io/hyrf/.

Method: Proposes Incremental Generation and Multi-agent Collaboration (IG-MC) with two innovations: plug-and-play incremental generation module for synchronized visual previews, and decision-driven multi-agent collaboration framework with generation, initiation, and assessment agents.

Result: Introduces the first MSTP Benchmark with synchronized annotations across multiple state and temporal scales, providing a unified evaluation framework.

Conclusion: The MSTP task formalization and IG-MC method address the challenge of multi-scale temporal prediction, enabling better scene understanding and advancing embodied artificial intelligence capabilities.

Abstract: Accurate temporal prediction is the bridge between comprehensive scene understanding and embodied artificial intelligence. However, predicting multiple fine-grained states of a scene at multiple temporal scales is difficult for vision-language models. We formalize the Multi-Scale Temporal Prediction (MSTP) task in general and surgical scenes by decomposing multi-scale into two orthogonal dimensions: the temporal scale, forecasting states of humans and surgery at varying look-ahead intervals, and the state scale, modeling a hierarchy of states in general and surgical scenes. For example, in general scenes, states of contact relationships are finer-grained than states of spatial relationships. In surgical scenes, medium-level steps are finer-grained than high-level phases yet remain constrained by their encompassing phase. To support this unified task, we introduce the first MSTP Benchmark, featuring synchronized annotations across multiple state scales and temporal scales. We further propose a method, Incremental Generation and Multi-agent Collaboration (IG-MC), which integrates two key innovations. First, we present a plug-and-play incremental generation module that continuously synthesizes up-to-date visual previews at expanding temporal scales to inform multiple decision-making agents, keeping decisions and generated visuals synchronized and preventing performance degradation as look-ahead intervals lengthen. Second, we present a decision-driven multi-agent collaboration framework for multi-state prediction, comprising generation, initiation, and multi-state assessment agents that dynamically trigger and evaluate prediction cycles to balance global coherence and local fidelity.

Method: Leverages 3D Gaussian Splatting and Habitat-Sim to reconstruct deployment scenes from iPhone captures into meshes. Analyzes training strategies, pre-training datasets, and mesh reconstruction techniques for sim-to-real predictivity.

Result: Agents fine-tuned with EmbodiedSplat outperform zero-shot baselines by 20% (vs HM3D) and 40% (vs HSSD) on real-world Image Navigation tasks. Achieves high sim-vs-real correlation (0.87-0.97) for reconstructed meshes.

Conclusion: EmbodiedSplat effectively adapts policies to diverse environments with minimal effort, providing a practical solution for sim-to-real transfer using accessible capture methods and efficient reconstruction techniques.

Abstract: The field of Embodied AI predominantly relies on simulation for training and evaluation, often using either fully synthetic environments that lack photorealism or high-fidelity real-world reconstructions captured with expensive hardware. As a result, sim-to-real transfer remains a major challenge. In this paper, we introduce EmbodiedSplat, a novel approach that personalizes policy training by efficiently capturing the deployment environment and fine-tuning policies within the reconstructed scenes. Our method leverages 3D Gaussian Splatting (GS) and the Habitat-Sim simulator to bridge the gap between realistic scene capture and effective training environments. Using iPhone-captured deployment scenes, we reconstruct meshes via GS, enabling training in settings that closely approximate real-world conditions. We conduct a comprehensive analysis of training strategies, pre-training datasets, and mesh reconstruction techniques, evaluating their impact on sim-to-real predictivity in real-world scenarios. Experimental results demonstrate that agents fine-tuned with EmbodiedSplat outperform both zero-shot baselines pre-trained on large-scale real-world datasets (HM3D) and synthetically generated datasets (HSSD), achieving absolute success rate improvements of 20% and 40% on real-world Image Navigation task. Moreover, our approach yields a high sim-vs-real correlation (0.87-0.97) for the reconstructed meshes, underscoring its effectiveness in adapting policies to diverse environments with minimal effort. Project page: https://gchhablani.github.io/embodied-splat.

Method: Designs HNSR with global semantic features and multi-resolution local geometric features from pre-trained 3D generative models, and employs a progressive global-to-local matching strategy that first establishes coarse correspondence then iteratively refines it.

Result: Outperforms previous state-of-the-art techniques in both qualitative and quantitative evaluations, demonstrates strong generalization across diverse shape categories, and supports various applications including shape co-segmentation, keypoint matching, and texture transfer.

Conclusion: The proposed training-free framework provides accurate and semantically-consistent 3D correspondence estimation that generalizes well to structurally diverse shapes and even cross-category scenarios, offering promising results for multiple applications.

Abstract: This paper presents a new approach to estimate accurate and robust 3D semantic correspondence with the hierarchical neural semantic representation. Our work has three key contributions. First, we design the hierarchical neural semantic representation (HNSR), which consists of a global semantic feature to capture high-level structure and multi-resolution local geometric features to preserve fine details, by carefully harnessing 3D priors from pre-trained 3D generative models. Second, we design a progressive global-to-local matching strategy, which establishes coarse semantic correspondence using the global semantic feature, then iteratively refines it with local geometric features, yielding accurate and semantically-consistent mappings. Third, our framework is training-free and broadly compatible with various pre-trained 3D generative backbones, demonstrating strong generalization across diverse shape categories. Our method also supports various applications, such as shape co-segmentation, keypoint matching, and texture transfer, and generalizes well to structurally diverse shapes, with promising results even in cross-category scenarios. Both qualitative and quantitative evaluations show that our method outperforms previous state-of-the-art techniques.

Method: The framework integrates a multimodal large language model (MLLM) with Segment Anything Model (SAM). MLLM generates special semantic tokens representing referred objects, which serve as prompts for SAM to segment objects across frames. A target-consistent semantic alignment loss is introduced to align token embeddings from different expressions referring to the same object.

Result: Experiments on the Ref-AVS benchmark demonstrate superior performance compared to existing methods.

Conclusion: The proposed SimToken framework effectively addresses cross-modal reasoning challenges in Ref-AVS by leveraging MLLM-generated semantic tokens to guide SAM for accurate object segmentation.

Abstract: Referring Audio-Visual Segmentation (Ref-AVS) aims to segment specific objects in videos based on natural language expressions involving audio, vision, and text information. This task poses significant challenges in cross-modal reasoning and fine-grained object localization. In this paper, we propose a simple framework, SimToken, that integrates a multimodal large language model (MLLM) with the Segment Anything Model (SAM). The MLLM is guided to generate a special semantic token representing the referred object. This compact token, enriched with contextual information from all modalities, acts as a prompt to guide SAM to segment objectsacross video frames. To further improve semantic learning, we introduce a novel target-consistent semantic alignment loss that aligns token embeddings from different expressions but referring to the same object. Experiments on the Ref-AVS benchmark demonstrate that our approach achieves superior performance compared to existing methods.

Method: Visual Instruction Pretraining (ViTP) embeds ViT backbone in VLM, pretrains end-to-end with domain-specific visual instruction data, and uses Visual Robustness Learning to force robust feature learning from sparse visual tokens.

Result: Achieves state-of-the-art performance on 16 challenging remote sensing and medical imaging benchmarks across diverse downstream tasks.

Conclusion: ViTP successfully demonstrates that reasoning can effectively enhance perception in foundation models, establishing a new paradigm for domain-specific pretraining.

Abstract: Modern computer vision is converging on a closed loop in which perception, reasoning and generation mutually reinforce each other. However, this loop remains incomplete: the top-down influence of high-level reasoning on the foundational learning of low-level perceptual features is not yet underexplored. This paper addresses this gap by proposing a new paradigm for pretraining foundation models in downstream domains. We introduce Visual insTruction Pretraining (ViTP), a novel approach that directly leverages reasoning to enhance perception. ViTP embeds a Vision Transformer (ViT) backbone within a Vision-Language Model and pretrains it end-to-end using a rich corpus of visual instruction data curated from target downstream domains. ViTP is powered by our proposed Visual Robustness Learning (VRL), which compels the ViT to learn robust and domain-relevant features from a sparse set of visual tokens. Extensive experiments on 16 challenging remote sensing and medical imaging benchmarks demonstrate that ViTP establishes new state-of-the-art performance across a diverse range of downstream tasks. The code is available at https://github.com/zcablii/ViTP.

Method: Proposes CaP-VTON (Clothing agnostic Pre-inpainting Virtual Try-ON) that integrates multi-category masking based on Dress Code and skin inpainting using Stable Diffusion, with a specialized generate skin module for handling sleeve length conversions.

Result: Achieved 92.5% accuracy in short-sleeved synthesis, which is 15.4% better than Leffa, and demonstrated consistent reproduction of reference clothing style and shape in visual evaluations.

Conclusion: The method maintains model-agnostic properties applicable to various diffusion-based virtual try-on systems and can enhance applications requiring high-precision virtual wearing in e-commerce, custom styling, and avatar creation.

Abstract: With the development of deep learning technology, virtual try-on technology has become an important application value in the fields of e-commerce, fashion, and entertainment. The recently proposed Leffa has improved the texture distortion problem of diffu-sion-based models, but there are limitations in that the bottom detection inaccuracy and the existing clothing silhouette remain in the synthesis results. To solve this problem, this study proposes CaP-VTON (Clothing agnostic Pre-inpainting Virtual Try-ON). CaP-VTON has improved the naturalness and consistency of whole-body clothing syn-thesis by integrating multi-category masking based on Dress Code and skin inpainting based on Stable Diffusion. In particular, a generate skin module was introduced to solve the skin restoration problem that occurs when long-sleeved images are converted into short-sleeved or sleeveless ones, and high-quality restoration was implemented consider-ing the human body posture and color. As a result, CaP-VTON recorded 92.5%, which is 15.4% better than Leffa in short-sleeved synthesis accuracy, and showed the performance of consistently reproducing the style and shape of reference clothing in visual evaluation. These structures maintain model-agnostic properties and are applicable to various diffu-sion-based virtual inspection systems, and can contribute to applications that require high-precision virtual wearing, such as e-commerce, custom styling, and avatar creation.

Method: The study used a multicentre dataset of 12,302 endoscopic images from 1,546 patients. The proposed model combines DINOv2 (vision foundation model) and ResNet50 to extract both global appearance and local detail features for EGJA staging diagnosis.

Result: The model achieved accuracies of 0.9256, 0.8895, and 0.8956 on held-out, external, and prospective test sets respectively, outperforming both ResNet50 (0.9125, 0.8382, 0.8519) and expert endoscopists (0.8147). When assisting human experts, it improved accuracy across all skill levels.

Conclusion: This is the first application of foundation models for EGJA staging diagnosis, demonstrating great potential in both diagnostic accuracy and efficiency improvement for medical professionals.

Abstract: The early detection of esophagogastric junction adenocarcinoma (EGJA) is crucial for improving patient prognosis, yet its current diagnosis is highly operator-dependent. This paper aims to make the first attempt to develop an artificial intelligence (AI) foundation model-based method for both screening and staging diagnosis of EGJA using endoscopic images. In this cohort and learning study, we conducted a multicentre study across seven Chinese hospitals between December 28, 2016 and December 30, 2024. It comprises 12,302 images from 1,546 patients; 8,249 of them were employed for model training, while the remaining were divided into the held-out (112 patients, 914 images), external (230 patients, 1,539 images), and prospective (198 patients, 1,600 images) test sets for evaluation. The proposed model employs DINOv2 (a vision foundation model) and ResNet50 (a convolutional neural network) to extract features of global appearance and local details of endoscopic images for EGJA staging diagnosis. Our model demonstrates satisfactory performance for EGJA staging diagnosis across three test sets, achieving an accuracy of 0.9256, 0.8895, and 0.8956, respectively. In contrast, among representative AI models, the best one (ResNet50) achieves an accuracy of 0.9125, 0.8382, and 0.8519 on the three test sets, respectively; the expert endoscopists achieve an accuracy of 0.8147 on the held-out test set. Moreover, with the assistance of our model, the overall accuracy for the trainee, competent, and expert endoscopists improves from 0.7035, 0.7350, and 0.8147 to 0.8497, 0.8521, and 0.8696, respectively. To our knowledge, our model is the first application of foundation models for EGJA staging diagnosis and demonstrates great potential in both diagnostic accuracy and efficiency.

Method: The framework uses efficient visual modules (key clip retrieval, subtitle retrieval) and constructs a fast-slow reasoning dataset. It employs FS-LLM to generate visual program workflows: simple queries use VideoLLMs directly, while difficult ones trigger visual program reasoning with fallback mechanisms. Parameter search improves programs during training and inference.

Result: FS-VisPR achieves 50.4% accuracy on LVBench (surpassing GPT-4o) and matches Qwen2.5VL-72B performance on VideoMME, demonstrating improved efficiency and reliability in visual program workflows.

Conclusion: The adaptive fast-slow reasoning approach effectively balances efficiency and accuracy for video question answering, providing a robust solution for long-form video tasks while outperforming existing methods.

Abstract: Large language models (LLMs) have shown promise in generating program workflows for visual tasks. However, previous approaches often rely on closed-source models, lack systematic reasoning, and struggle with long-form video question answering (videoQA). To address these challenges, we introduce the FS-VisPR framework, an adaptive visual program reasoning approach that balances fast reasoning for simple queries with slow reasoning for difficult ones. First, we design efficient visual modules (e.g., key clip retrieval and subtitle retrieval) to support long-form video tasks. Then, we construct a diverse and high-quality fast-slow reasoning dataset with a strong LLM to align open-source language models’ ability to generate visual program workflows as FS-LLM. Next, we design a fast-slow reasoning framework with FS-LLM: Simple queries are directly solved by VideoLLMs, while difficult ones invoke visual program reasoning, motivated by human-like reasoning processes. During this process, low-confidence fast-thinking answers will trigger a second-stage slow-reasoning process, and a fallback mechanism to fast reasoning is activated if the program execution fails. Moreover, we improve visual programs through parameter search during both training and inference. By adjusting the parameters of the visual modules within the program, multiple variants are generated: during training, programs that yield correct answers are selected, while during inference, the program with the highest confidence result is applied. Experiments show that FS-VisPR improves both efficiency and reliability in visual program workflows. It achieves 50.4% accuracy on LVBench, surpassing GPT-4o, matching the performance of Qwen2.5VL-72B on VideoMME.

Method: Proposes Multiplexing Watermark VAE (MPW-VAE) with latent residual-based adapter to generate paired watermarked/watermark-free images, and Mixture-of-Experts Guided Forensic Network (MoE-GFN) that dynamically integrates multiple forensic cues. Both components are jointly optimized in self-supervised, end-to-end training.

Result: Extensive experiments show StableGuard consistently outperforms state-of-the-art methods in image fidelity, watermark verification, and tampering localization.

Conclusion: StableGuard provides an effective end-to-end solution for copyright protection and tampering localization in diffusion models, overcoming limitations of post-processing approaches through integrated watermark embedding and forensic analysis.

Abstract: The advancement of diffusion models has enhanced the realism of AI-generated content but also raised concerns about misuse, necessitating robust copyright protection and tampering localization. Although recent methods have made progress toward unified solutions, their reliance on post hoc processing introduces considerable application inconvenience and compromises forensic reliability. We propose StableGuard, a novel framework that seamlessly integrates a binary watermark into the diffusion generation process, ensuring copyright protection and tampering localization in Latent Diffusion Models through an end-to-end design. We develop a Multiplexing Watermark VAE (MPW-VAE) by equipping a pretrained Variational Autoencoder (VAE) with a lightweight latent residual-based adapter, enabling the generation of paired watermarked and watermark-free images. These pairs, fused via random masks, create a diverse dataset for training a tampering-agnostic forensic network. To further enhance forensic synergy, we introduce a Mixture-of-Experts Guided Forensic Network (MoE-GFN) that dynamically integrates holistic watermark patterns, local tampering traces, and frequency-domain cues for precise watermark verification and tampered region detection. The MPW-VAE and MoE-GFN are jointly optimized in a self-supervised, end-to-end manner, fostering a reciprocal training between watermark embedding and forensic accuracy. Extensive experiments demonstrate that StableGuard consistently outperforms state-of-the-art methods in image fidelity, watermark verification, and tampering localization.

Method: Proposes MMCD framework that fuses multi-modal observations from ego and collaborative vehicles using cross-modal knowledge distillation with teacher-student model structure for robust performance with reduced modalities.

Result: Improves driving safety by up to 20.7% in connected autonomous driving and aerial-ground vehicles collaboration, surpassing existing baselines in accident detection and safe decision-making.

Conclusion: The MMCD framework provides a practical solution for robust autonomous decision-making in challenging environments where data modalities may be unavailable.

Abstract: Autonomous systems have advanced significantly, but challenges persist in accident-prone environments where robust decision-making is crucial. A single vehicle’s limited sensor range and obstructed views increase the likelihood of accidents. Multi-vehicle connected systems and multi-modal approaches, leveraging RGB images and LiDAR point clouds, have emerged as promising solutions. However, existing methods often assume the availability of all data modalities and connected vehicles during both training and testing, which is impractical due to potential sensor failures or missing connected vehicles. To address these challenges, we introduce a novel framework MMCD (Multi-Modal Collaborative Decision-making) for connected autonomy. Our framework fuses multi-modal observations from ego and collaborative vehicles to enhance decision-making under challenging conditions. To ensure robust performance when certain data modalities are unavailable during testing, we propose an approach based on cross-modal knowledge distillation with a teacher-student model structure. The teacher model is trained with multiple data modalities, while the student model is designed to operate effectively with reduced modalities. In experiments on $\textit{connected autonomous driving with ground vehicles}$ and $\textit{aerial-ground vehicles collaboration}$, our method improves driving safety by up to ${\it 20.7}%$, surpassing the best-existing baseline in detecting potential accidents and making safe driving decisions. More information can be found on our website https://ruiiu.github.io/mmcd.

Method: Tracing strength inconsistencies to specific arguments, identifying sufficient, necessary, and counterfactual explanations, and developing a heuristic-based approach with implementation for finding these explanations.

Result: Shows that strength inconsistency explanations exist if and only if an update leads to strength inconsistency, and provides a practical implementation for explanation search.

Conclusion: The approach successfully formalizes explanation mechanisms for inference changes in QBAFs and provides computable methods for identifying explanations of strength inconsistencies.

Abstract: This paper presents a formal approach to explaining change of inference in Quantitative Bipolar Argumentation Frameworks (QBAFs). When drawing conclusions from a QBAF and updating the QBAF to then again draw conclusions (and so on), our approach traces changes – which we call strength inconsistencies – in the partial order over argument strengths that a semantics establishes on some arguments of interest, called topic arguments. We trace the causes of strength inconsistencies to specific arguments, which then serve as explanations. We identify sufficient, necessary, and counterfactual explanations for strength inconsistencies and show that strength inconsistency explanations exist if and only if an update leads to strength inconsistency. We define a heuristic-based approach to facilitate the search for strength inconsistency explanations, for which we also provide an implementation.

Method: The study analyzes hardware requirements, operational expenses, and performance benchmarks of open-source models (Qwen, Llama, Mistral) and compares total costs with major cloud providers’ subscription fees.

Result: The research provides estimated breakeven points based on usage levels and performance needs, showing when on-premise deployment becomes economically viable.

Conclusion: The framework offers organizations a practical tool for planning their LLM strategies by determining the optimal deployment approach based on their specific requirements.

Abstract: Large language models (LLMs) are becoming increasingly widespread. Organizations that want to use AI for productivity now face an important decision. They can subscribe to commercial LLM services or deploy models on their own infrastructure. Cloud services from providers such as OpenAI, Anthropic, and Google are attractive because they provide easy access to state-of-the-art models and are easy to scale. However, concerns about data privacy, the difficulty of switching service providers, and long-term operating costs have driven interest in local deployment of open-source models. This paper presents a cost-benefit analysis framework to help organizations determine when on-premise LLM deployment becomes economically viable compared to commercial subscription services. We consider the hardware requirements, operational expenses, and performance benchmarks of the latest open-source models, including Qwen, Llama, Mistral, and etc. Then we compare the total cost of deploying these models locally with the major cloud providers subscription fee. Our findings provide an estimated breakeven point based on usage levels and performance needs. These results give organizations a practical framework for planning their LLM strategies.

Method: SPADE converts time-series data into textual representations and uses domain-informed prompt templates with ChatGPT-4.1 for zero-shot analysis, enabling irrigation event detection, anomaly classification, and structured reporting.

Result: SPADE outperforms existing methods in anomaly detection with higher recall and F1 scores, achieves high precision/recall in irrigation event detection, and provides interpretable reports for soil moisture analytics.

Conclusion: LLMs show great potential as scalable, adaptable tools for precision agriculture, capable of integrating qualitative knowledge with data-driven reasoning to generate actionable insights for improved irrigation scheduling.

Abstract: Accurate interpretation of soil moisture patterns is critical for irrigation scheduling and crop management, yet existing approaches for soil moisture time-series analysis either rely on threshold-based rules or data-hungry machine learning or deep learning models that are limited in adaptability and interpretability. In this study, we introduce SPADE (Soil moisture Pattern and Anomaly DEtection), an integrated framework that leverages large language models (LLMs) to jointly detect irrigation patterns and anomalies in soil moisture time-series data. SPADE utilizes ChatGPT-4.1 for its advanced reasoning and instruction-following capabilities, enabling zero-shot analysis without requiring task-specific annotation or fine-tuning. By converting time-series data into a textual representation and designing domain-informed prompt templates, SPADE identifies irrigation events, estimates net irrigation gains, detects, classifies anomalies, and produces structured, interpretable reports. Experiments were conducted on real-world soil moisture sensor data from commercial and experimental farms cultivating multiple crops across the United States. Results demonstrate that SPADE outperforms the existing method in anomaly detection, achieving higher recall and F1 scores and accurately classifying anomaly types. Furthermore, SPADE achieved high precision and recall in detecting irrigation events, indicating its strong capability to capture irrigation patterns accurately. SPADE’s reports provide interpretability and usability of soil moisture analytics. This study highlights the potential of LLMs as scalable, adaptable tools for precision agriculture, which is capable of integrating qualitative knowledge and data-driven reasoning to produce actionable insights for accurate soil moisture monitoring and improved irrigation scheduling from soil moisture time-series data.

Method: The authors systematically map medical uncertainty to AI uncertainty concepts, identify implementation challenges, and outline technical directions including multimodal uncertainty quantification, model-agnostic visualization, and uncertainty-aware decision support systems.

Result: The analysis highlights the need for AI systems that generate reliable predictions while articulating confidence levels in clinically meaningful ways.

Conclusion: This work contributes to trustworthy medical AI by bridging explainability and uncertainty, paving the way for AI systems aligned with real-world clinical complexities through proposed XUE framework and guiding principles.

Abstract: Uncertainty is a fundamental challenge in medical practice, but current medical AI systems fail to explicitly quantify or communicate uncertainty in a way that aligns with clinical reasoning. Existing XAI works focus on interpreting model predictions but do not capture the confidence or reliability of these predictions. Conversely, uncertainty estimation (UE) techniques provide confidence measures but lack intuitive explanations. The disconnect between these two areas limits AI adoption in medicine. To address this gap, we propose Explainable Uncertainty Estimation (XUE) that integrates explainability with uncertainty quantification to enhance trust and usability in medical AI. We systematically map medical uncertainty to AI uncertainty concepts and identify key challenges in implementing XUE. We outline technical directions for advancing XUE, including multimodal uncertainty quantification, model-agnostic visualization techniques, and uncertainty-aware decision support systems. Lastly, we propose guiding principles to ensure effective XUE realisation. Our analysis highlights the need for AI systems that not only generate reliable predictions but also articulate confidence levels in a clinically meaningful way. This work contributes to the development of trustworthy medical AI by bridging explainability and uncertainty, paving the way for AI systems that are aligned with real-world clinical complexities.

Method: HSGM decomposes input into meaningful segments, constructs Local Semantic Graphs on each segment, extracts summary nodes to form a Global Graph Memory, supports incremental updates, and uses Hierarchical Query Processing for efficient retrieval and reasoning.

Result: HSGM achieves 2-4× inference speedup, >60% reduction in peak memory, and ≥95% of baseline accuracy on long-document AMR parsing, segment-level semantic role labeling, and legal event extraction benchmarks.

Conclusion: HSGM enables scalable, accurate semantic modeling for ultra-long texts, making real-time and resource-constrained NLP applications feasible by significantly reducing computational complexity and memory requirements.

Abstract: Semantic parsing of long documents remains challenging due to quadratic growth in pairwise composition and memory requirements. We introduce \textbf{Hierarchical Segment-Graph Memory (HSGM)}, a novel framework that decomposes an input of length $N$ into $M$ meaningful segments, constructs \emph{Local Semantic Graphs} on each segment, and extracts compact \emph{summary nodes} to form a \emph{Global Graph Memory}. HSGM supports \emph{incremental updates} – only newly arrived segments incur local graph construction and summary-node integration – while \emph{Hierarchical Query Processing} locates relevant segments via top-$K$ retrieval over summary nodes and then performs fine-grained reasoning within their local graphs. Theoretically, HSGM reduces worst-case complexity from $O(N^2)$ to $O!\left(N,k + (N/k)^2\right)$, with segment size $k \ll N$, and we derive Frobenius-norm bounds on the approximation error introduced by node summarization and sparsification thresholds. Empirically, on three benchmarks – long-document AMR parsing, segment-level semantic role labeling (OntoNotes), and legal event extraction – HSGM achieves \emph{2–4$\times$ inference speedup}, \emph{$>60%$ reduction} in peak memory, and \emph{$\ge 95%$} of baseline accuracy. Our approach unlocks scalable, accurate semantic modeling for ultra-long texts, enabling real-time and resource-constrained NLP applications.

Method: Uses a multi-agent framework with Model Context Protocol (MCP) for composable services, hierarchical multi-index RAG for context retrieval, and dependency-aware configuration generation. Includes Meshing Agent for geometry handling and automatic HPC script generation.

Result: Achieved 88.2% success rate on 110 simulation tasks, significantly outperforming MetaOpenFOAM (55.5%). Successfully automates end-to-end workflow including pre-processing, simulation, and post-processing visualization.

Conclusion: Foam-Agent dramatically lowers the expertise barrier for CFD and demonstrates how specialized multi-agent systems can democratize complex scientific computing. The framework is publicly available.

Abstract: Computational Fluid Dynamics (CFD) is an essential simulation tool in engineering, yet its steep learning curve and complex manual setup create significant barriers. To address these challenges, we introduce Foam-Agent, a multi-agent framework that automates the entire end-to-end OpenFOAM workflow from a single natural language prompt. Our key innovations address critical gaps in existing systems: 1. An Comprehensive End-to-End Simulation Automation: Foam-Agent is the first system to manage the full simulation pipeline, including advanced pre-processing with a versatile Meshing Agent capable of handling external mesh files and generating new geometries via Gmsh, automatic generation of HPC submission scripts, and post-simulation visualization via ParaView. 2. Composable Service Architecture: Going beyond a monolithic agent, the framework uses Model Context Protocol (MCP) to expose its core functions as discrete, callable tools. This allows for flexible integration and use by other agentic systems, such as Claude-code, for more exploratory workflows. 3. High-Fidelity Configuration Generation: We achieve superior accuracy through a Hierarchical Multi-Index RAG for precise context retrieval and a dependency-aware generation process that ensures configuration consistency. Evaluated on a benchmark of 110 simulation tasks, Foam-Agent achieves an 88.2% success rate with Claude 3.5 Sonnet, significantly outperforming existing frameworks (55.5% for MetaOpenFOAM). Foam-Agent dramatically lowers the expertise barrier for CFD, demonstrating how specialized multi-agent systems can democratize complex scientific computing. The code is public at https://github.com/csml-rpi/Foam-Agent.

Method: The framework integrates specialized agents for literature review, data analysis, code generation, and manuscript preparation. It uses vision-language feedback for medical visualization interpretation and implements quality control mechanisms for reproducibility. The system automates the entire research pipeline to produce publication-ready LaTeX manuscripts.

Result: OpenLens AI offers a domain-adapted solution that produces transparent and traceable workflows for health informatics research, addressing the limitations of existing systems.

Conclusion: The framework provides an automated, specialized approach for advancing health informatics research by addressing key gaps in medical visualization interpretation and domain-specific quality requirements.

Abstract: Health informatics research is characterized by diverse data modalities, rapid knowledge expansion, and the need to integrate insights across biomedical science, data analytics, and clinical practice. These characteristics make it particularly well-suited for agent-based approaches that can automate knowledge exploration, manage complex workflows, and generate clinically meaningful outputs. Recent progress in large language model (LLM)-based agents has demonstrated promising capabilities in literature synthesis, data analysis, and even end-to-end research execution. However, existing systems remain limited for health informatics because they lack mechanisms to interpret medical visualizations and often overlook domain-specific quality requirements. To address these gaps, we introduce OpenLens AI, a fully automated framework tailored to health informatics. OpenLens AI integrates specialized agents for literature review, data analysis, code generation, and manuscript preparation, enhanced by vision-language feedback for medical visualization and quality control for reproducibility. The framework automates the entire research pipeline, producing publication-ready LaTeX manuscripts with transparent and traceable workflows, thereby offering a domain-adapted solution for advancing health informatics research.

Method: The paper organizes LLM-OR integration into three main directions: automatic modeling (translating natural language to mathematical models/code), auxiliary optimization (generating heuristics, evolving algorithms), and direct solving of optimization tasks.

Result: The survey reviews evaluation benchmarks, domain-specific applications, and identifies key challenges including unstable semantic-to-structure mapping, fragmented research progress, limited generalization, and insufficient evaluation systems.

Conclusion: The paper outlines future research avenues for advancing LLMs’ role in OR, addressing current limitations and exploring new integration possibilities.

Abstract: Operations research (OR) provides fundamental methodologies for complex system decision-making, with established applications in transportation, supply chain management, and production scheduling. Traditional approaches, which depend on expert-based modeling and manual parameter adjustment, often face challenges in handling large-scale, dynamic, and multi-constraint problems. Recently, large language models (LLMs) have shown potential to address these limitations through semantic understanding, structured generation, and reasoning control. LLMs can translate natural language descriptions into mathematical models or executable code, generate heuristics, evolve algorithms, and directly tackle optimization tasks. This paper surveys recent progress on the integration of LLMs into OR, organizing methods into three main directions: automatic modeling, auxiliary optimization, and direct solving. It further reviews evaluation benchmarks and domain-specific applications, and summarizes key open issues such as unstable semantic-to-structure mapping, fragmented research progress, limited generalization, and insufficient evaluation systems. Finally, the survey outlines possible research avenues for advancing the role of LLMs in OR.

Method: SAPA uses a hierarchical approach: LLM generates traveler personas, trains propensity-score model, assigns quantitative scores to latent variables, and integrates everything with a final classifier.

Result: SAPA outperforms state-of-the-art baselines by up to 75.9% in PR-AUC on a held-out test set using large-scale multi-year travel survey data.

Conclusion: SAPA provides an accurate tool for ridesourcing prediction and offers a transferable methodology for various applications.

Abstract: Accurate modeling of ridesourcing mode choices is essential for designing and implementing effective traffic management policies for reducing congestion, improving mobility, and allocating resources more efficiently. Existing models for predicting ridesourcing mode choices often suffer from limited predictive accuracy due to their inability to capture key psychological factors, and are further challenged by severe class imbalance, as ridesourcing trips comprise only a small fraction of individuals’ daily travel. To address these limitations, this paper introduces the Synthesizing Attitudes, Predicting Actions (SAPA) framework, a hierarchical approach that uses Large Language Models (LLMs) to synthesize theory-grounded latent attitudes to predict ridesourcing choices. SAPA first uses an LLM to generate qualitative traveler personas from raw travel survey data and then trains a propensity-score model on demographic and behavioral features, enriched by those personas, to produce an individual-level score. Next, the LLM assigns quantitative scores to theory-driven latent variables (e.g., time and cost sensitivity), and a final classifier integrates the propensity score, latent-variable scores (with their interaction terms), and observable trip attributes to predict ridesourcing mode choice. Experiments on a large-scale, multi-year travel survey show that SAPA significantly outperforms state-of-the-art baselines, improving ridesourcing choice predictions by up to 75.9% in terms of PR-AUC on a held-out test set. This study provides a powerful tool for accurately predicting ridesourcing mode choices, and provides a methodology that is readily transferable to various applications.

Method: OBER uses a minimalist entity-relation model, log-driven mastery formula, and plug-in architecture. It was tested in a two-week randomized split test with over 5,700 learners comparing fixed expert trajectory, collaborative filtering, and knowledge-based filtering methods.

Result: Collaborative filtering maximized retention, but the fixed expert path achieved the highest mastery. OBER allows deriving business, relevance, and learning metrics from the same logs without extra testing overhead.

Conclusion: OBER provides a framework that lets practitioners weigh relevance and engagement against outcome mastery, is method-agnostic, and readily extensible to future adaptive or context-aware recommenders.

Abstract: Most educational recommender systems are tuned and judged on click- or rating-based relevance, leaving their true pedagogical impact unclear. We introduce OBER-an Outcome-Based Educational Recommender that embeds learning outcomes and assessment items directly into the data schema, so any algorithm can be evaluated on the mastery it fosters. OBER uses a minimalist entity-relation model, a log-driven mastery formula, and a plug-in architecture. Integrated into an e-learning system in non-formal domain, it was evaluated trough a two-week randomized split test with over 5 700 learners across three methods: fixed expert trajectory, collaborative filtering (CF), and knowledge-based (KB) filtering. CF maximized retention, but the fixed path achieved the highest mastery. Because OBER derives business, relevance, and learning metrics from the same logs, it lets practitioners weigh relevance and engagement against outcome mastery with no extra testing overhead. The framework is method-agnostic and readily extensible to future adaptive or context-aware recommenders.

Method: nDNA synthesizes three dimensions of latent geometry: spectral curvature (conceptual flow curvature across layers), thermodynamic length (semantic effort for representational transitions), and belief vector field (semantic torsion fields guiding belief orientations).

Result: nDNA provides a stable, coordinate-free fingerprint that enables tracing model lineages across pretraining, fine-tuning, alignment, pruning, distillation, and merges, while detecting drift and measuring inheritance between checkpoints.

Conclusion: This work establishes Neural Genomics as a new field where AI models are treated as digital semantic organisms with traceable inner cognition, enabling comparison, risk diagnosis, and governance of artificial cognitive evolution.

Abstract: As AI foundation models grow in capability, a deeper question emerges: What shapes their internal cognitive identity – beyond fluency and output? Benchmarks measure behavior, but the soul of a model resides in its latent geometry. In this work, we propose Neural DNA (nDNA) as a semantic-genotypic representation that captures this latent identity through the intrinsic geometry of belief. At its core, nDNA is synthesized from three principled and indispensable dimensions of latent geometry: spectral curvature, which reveals the curvature of conceptual flow across layers; thermodynamic length, which quantifies the semantic effort required to traverse representational transitions through layers; and belief vector field, which delineates the semantic torsion fields that guide a model’s belief directional orientations. Like biological DNA, it encodes ancestry, mutation, and semantic inheritance, found in finetuning and alignment scars, cultural imprints, and architectural drift. In naming it, we open a new field: Neural Genomics, where models are not just tools, but digital semantic organisms with traceable inner cognition. Modeling statement. We read AI foundation models as semantic fluid–dynamics: meaning is transported through layers like fluid in a shaped conduit; nDNA is the physics-grade readout of that flow – a geometry-first measure of how meaning is bent, paid for, and pushed – yielding a stable, coordinate-free neural DNA fingerprint tied to on-input behavior; with this fingerprint we cross into biology: tracing lineages across pretraining, fine-tuning, alignment, pruning, distillation, and merges; measuring inheritance between checkpoints; detecting drift as traits shift under new data or objectives; and, ultimately, studying the evolution of artificial cognition to compare models, diagnose risks, and govern change over time.

Method: Defines similarity fields over entities, system evolution sequences, concept fibers as superlevel sets, and generative operators. Proves theorems about asymmetry and stability constraints.

Result: Developed a comprehensive framework with formal definitions and theorems that constrain similarity field evolution, enabling interpretation of systems like large language models.

Conclusion: Similarity Field Theory offers a foundational language for analyzing intelligent systems and provides mathematical tools to study societal cognition through models like LLMs.

Abstract: We posit that persisting and transforming similarity relations form the structural basis of any comprehensible dynamic system. This paper introduces Similarity Field Theory, a mathematical framework that formalizes the principles governing similarity values among entities and their evolution. We define: (1) a similarity field $S: U \times U \to [0,1]$ over a universe of entities $U$, satisfying reflexivity $S(E,E)=1$ and treated as a directed relational field (asymmetry and non-transitivity are allowed); (2) the evolution of a system through a sequence $Z_p = (X_p, S^{(p)})$ indexed by $p=0,1,2,\ldots$; (3) concepts $K$ as entities that induce fibers $F_{\alpha}(K) = { E \in U \mid S(E,K) \ge \alpha }$, i.e., superlevel sets of the unary map $S_K(E) := S(E,K)$; and (4) a generative operator $G$ that produces new entities. Within this framework, we formalize a generative definition of intelligence: an operator $G$ is intelligent with respect to a concept $K$ if, given a system containing entities belonging to the fiber of $K$, it generates new entities that also belong to that fiber. Similarity Field Theory thus offers a foundational language for characterizing, comparing, and constructing intelligent systems. We prove two theorems: (i) asymmetry blocks mutual inclusion; and (ii) stability requires either an anchor coordinate or eventual confinement within a level set of $f$. These results ensure that the evolution of similarity fields is both constrained and interpretable, culminating in an exploration of how the framework allows us to interpret large language models and use them as experimental probes into societal cognition.

Method: Hierarchical stacked visual-language multimodal Transformer with LLM inference head, featuring: (i) cross-modal pre-training with momentum update encoders and debiased InfoNCE losses, (ii) time fusion block with adaptive time interval position coding for irregular visit sequences, (iii) disease ontology map adapter injecting ICD-10 codes with graph attention mechanism.

Result: On MIMIC-IV longitudinal cohort, achieved average AUROC of 0.90 with expected calibration error of 2.7 percent.

Conclusion: VL-RiskFormer demonstrates strong performance in multimodal health risk prediction, effectively integrating diverse clinical data types through innovative architectural components.

Abstract: With the rising global burden of chronic diseases and the multimodal and heterogeneous clinical data (medical imaging, free-text recordings, wearable sensor streams, etc.), there is an urgent need for a unified multimodal AI framework that can proactively predict individual health risks. We propose VL-RiskFormer, a hierarchical stacked visual-language multimodal Transformer with a large language model (LLM) inference head embedded in its top layer. The system builds on the dual-stream architecture of existing visual-linguistic models (e.g., PaLM-E, LLaVA) with four key innovations: (i) pre-training with cross-modal comparison and fine-grained alignment of radiological images, fundus maps, and wearable device photos with corresponding clinical narratives using momentum update encoders and debiased InfoNCE losses; (ii) a time fusion block that integrates irregular visit sequences into the causal Transformer decoder through adaptive time interval position coding; (iii) a disease ontology map adapter that injects ICD-10 codes into visual and textual channels in layers and infers comorbid patterns with the help of a graph attention mechanism. On the MIMIC-IV longitudinal cohort, VL-RiskFormer achieved an average AUROC of 0.90 with an expected calibration error of 2.7 percent.

Method: Hybrid architecture combining Chain of Exploration (CoE) for query refinement, Knowledge Graph (KG) for semantic reasoning, Retrieval-Augmented Generation (RAG) for contextual details, and LLM for integration into coherent recommendations.

Result: ChefMind achieves superior performance with average score of 8.7 vs 6.4-6.7 for ablation models, reduces unprocessed queries to 1.6%, and excels in accuracy, relevance, completeness, and clarity on Xiachufang dataset.

Conclusion: The hybrid approach demonstrates robustness in handling fuzzy demands and outperforms individual component baselines, providing an effective solution for personalized recipe recommendation.

Abstract: Personalized recipe recommendation faces challenges in handling fuzzy user intent, ensuring semantic accuracy, and providing sufficient detail coverage. We propose ChefMind, a hybrid architecture combining Chain of Exploration (CoE), Knowledge Graph (KG), Retrieval-Augmented Generation (RAG), and a Large Language Model (LLM). CoE refines ambiguous queries into structured conditions, KG offers semantic reasoning and interpretability, RAG supplements contextual culinary details, and LLM integrates outputs into coherent recommendations. We evaluate ChefMind on the Xiachufang dataset and manually annotated queries, comparing it with LLM-only, KG-only, and RAG-only baselines. Results show that ChefMind achieves superior performance in accuracy, relevance, completeness, and clarity, with an average score of 8.7 versus 6.4-6.7 for ablation models. Moreover, it reduces unprocessed queries to 1.6%, demonstrating robustness in handling fuzzy demands.

Method: An agency framework with N independent Agent Solve instances that generate solutions, followed by an Agent Compare that summarizes and compares results to recommend the best solution based on Condorcet’s Jury Theorem principles.

Result: The method significantly improves reliability for problems where individual GPT instances have success probability >50%, and shows comparable performance to commercial multi-agent models like Grok Heavy but with greater transparency.

Conclusion: The N-Plus-1 GPT Agency provides a reliable framework for mechanical engineering problem-solving that can be deployed in educational and professional settings, offering both accuracy and pedagogical value through transparent comparison processes.

Abstract: Generative AI, and specifically GPT, can produce a remarkable solution to a mechanical engineering analysis problem - but also, on occasion, a flawed solution. For example, an elementary mechanics problem is solved flawlessly in one GPT instance and incorrectly in a subsequent GPT instance, with a success probability of only 85%. This unreliability renders “out-of-the-box” GPT unsuitable for deployment in education or engineering practice. We introduce an “N-Plus-1” GPT Agency for Initial (Low-Cost) Analysis of mechanical engineering Problem Statements. Agency first launches N instantiations of Agent Solve to yield N independent Proposed Problem Solution Realizations; Agency then invokes Agent Compare to summarize and compare the N Proposed Problem Solution Realizations and to provide a Recommended Problem Solution. We argue from Condorcet’s Jury Theorem that, for a Problem Statement characterized by per-Solve success probability greater than 1/2 (and N sufficiently large), the Predominant (Agent Compare) Proposed Problem Solution will, with high probability, correspond to a Correct Proposed Problem Solution. Furthermore, Agent Compare can also incorporate aspects of Secondary (Agent Compare) Proposed Problem Solutions, in particular when the latter represent alternative Problem Statement interpretations - different Mathematical Models - or alternative Mathematical Solution Procedures. Comparisons to Grok Heavy, a commercial multi-agent model, show similarities in design and performance, but also important differences in emphasis: our Agency focuses on transparency and pedagogical value.

Method: Hierarchical RL framework with two-level option process (manager/subpolicy), triple-modal state encoder (screenshot, task ID, numeric state), meta-actions with early-stop mechanism, and compact vision backbone with small policy networks.

Result: Achieved 92.1% success on simple tasks (<8 steps) and 58.8% on hard tasks (≥8 steps), matching/exceeding 200B-parameter MLLMs while reducing model size by 10,000x and halving inference time on 135 real-world desktop tasks.

Conclusion: Hierarchical RL provides a practical, scalable alternative to monolithic MLLM-based automation for computer control, demonstrating superior efficiency and deployability.

Abstract: Controlling desktop applications via software remains a fundamental yet under-served problem. Existing multi-modal large language models (MLLMs) ingest screenshots and task instructions to generate keystrokes and mouse events, but they suffer from prohibitive inference latency, poor sample efficiency on long-horizon sparse-reward tasks, and infeasible on-device deployment. We introduce a lightweight hierarchical reinforcement learning framework, ComputerAgent, that formulates OS control as a two-level option process (manager and subpolicy), employs a triple-modal state encoder (screenshot, task ID, numeric state) to handle visual and contextual diversity, integrates meta-actions with an early-stop mechanism to reduce wasted interactions, and uses a compact vision backbone plus small policy networks for on-device inference (15M parameters). On a suite of 135 real-world desktop tasks, ComputerAgent attains 92.1% success on simple tasks (<8 steps) and 58.8% on hard tasks (>=8 steps), matching or exceeding 200B-parameter MLLM baselines on simple scenarios while reducing model size by over four orders of magnitude and halving inference time. These results demonstrate that hierarchical RL offers a practical, scalable alternative to monolithic MLLM-based automation for computer control.

Method: Stress-tested six flagship models across six medical benchmarks by removing key inputs, changing prompts, and analyzing reasoning patterns, plus clinician-guided rubric evaluation

Result: Models often guess correctly without key inputs, flip answers under trivial changes, fabricate flawed reasoning, and benchmarks vary widely in what they actually measure

Conclusion: Medical benchmark scores don’t reflect real-world readiness; we need to demand robustness, sound reasoning, and alignment with actual medical demands rather than just leaderboard wins

Abstract: Large frontier models like GPT-5 now achieve top scores on medical benchmarks. But our stress tests tell a different story. Leading systems often guess correctly even when key inputs like images are removed, flip answers under trivial prompt changes, and fabricate convincing yet flawed reasoning. These aren’t glitches; they expose how today’s benchmarks reward test-taking tricks over medical understanding. We evaluate six flagship models across six widely used benchmarks and find that high leaderboard scores hide brittleness and shortcut learning. Through clinician-guided rubric evaluation, we show that benchmarks vary widely in what they truly measure yet are treated interchangeably, masking failure modes. We caution that medical benchmark scores do not directly reflect real-world readiness. If we want AI to earn trust in healthcare, we must demand more than leaderboard wins and must hold systems accountable for robustness, sound reasoning, and alignment with real medical demands.

Method: Two approaches: (1) fine-tuning reasoning models using length controlled policy optimization (LCPO) reinforcement learning to satisfy user-defined CoT reasoning length, (2) applying quantization to maximize CoT generation within user-defined compute constraints.

Result: The paper studies the trade-off between computational efficiency and model safety, but specific results are not detailed in the abstract.

Conclusion: Compute constraint strategies (length control and quantization) are viable methods to reduce computational costs of reasoning models, but their impact on safety performance requires careful consideration and trade-off analysis.

Abstract: Test-time compute scaling has demonstrated the ability to improve the performance of reasoning language models by generating longer chain-of-thought (CoT) sequences. However, this increase in performance comes with a significant increase in computational cost. In this work, we investigate two compute constraint strategies: (1) reasoning length constraint and (2) model quantization, as methods to reduce the compute demand of reasoning models and study their impact on their safety performance. Specifically, we explore two approaches to apply compute constraints to reasoning models: (1) fine-tuning reasoning models using a length controlled policy optimization (LCPO) based reinforcement learning method to satisfy a user-defined CoT reasoning length, and (2) applying quantization to maximize the generation of CoT sequences within a user-defined compute constraint. Furthermore, we study the trade-off between the computational efficiency and the safety of the model.

Method: Evaluated GPT-5 on five conjectures in combinatorial optimization by providing source papers and assessing the model’s reasoning on previously unsolved problems.

Result: GPT-5 produced nearly correct solutions for three easier problems, refuted one conjecture with a valid alternative, failed on cross-paper synthesis, and proposed correct algorithms but flawed analysis for harder problems.

Conclusion: GPT-5 shows meaningful progress on routine reasoning with occasional originality, but has clear limitations in cross-paper synthesis, representing an early step toward passing the Gödel Test.

Abstract: Recent announcements from frontier AI model labs have highlighted strong results on high-school and undergraduate math competitions. Yet it remains unclear whether large language models can solve new, simple conjectures in more advanced areas of mathematics. We propose the G"odel Test: evaluating whether a model can produce correct proofs for very simple, previously unsolved conjectures. To this end, we study the performance of GPT-5 on five conjectures in combinatorial optimization. For each problem, we provided one or two source papers from which the conjecture arose, withheld our own conjecture, and then assessed the model’s reasoning in detail. On the three easier problems, GPT-5 produced nearly correct solutions; for Problem 2 it even derived a different approximation guarantee that, upon checking, refuted our conjecture while providing a valid solution. The model failed on Problem 4, which required combining results from two papers. On Problem 5, a harder case without a validated conjecture, GPT-5 proposed the same algorithm we had in mind but failed in the analysis, suggesting the proof is more challenging than expected. Although our sample is small, the results point to meaningful progress on routine reasoning, occasional flashes of originality, and clear limitations when cross-paper synthesis is required. GPT-5 may represent an early step toward frontier models eventually passing the G"odel Test.

Method: Created benchmark from US Customs Rulings Online Search System (CROSS), fine-tuned LLaMA-3.3-70B model (Atlas) for HTS classification.

Result: Atlas achieves 40% 10-digit and 57.5% 6-digit accuracy, outperforming GPT-5-Thinking by 15 points and Gemini-2.5-Pro-Thinking by 27.5 points, while being 5-8x cheaper.

Conclusion: Sets strong baseline but task remains challenging; releases dataset and model to establish HTS classification as community benchmark for future work in retrieval, reasoning, and alignment.

Abstract: Accurate classification of products under the Harmonized Tariff Schedule (HTS) is a critical bottleneck in global trade, yet it has received little attention from the machine learning community. Misclassification can halt shipments entirely, with major postal operators suspending deliveries to the U.S. due to incomplete customs documentation. We introduce the first benchmark for HTS code classification, derived from the U.S. Customs Rulings Online Search System (CROSS). Evaluating leading LLMs, we find that our fine-tuned Atlas model (LLaMA-3.3-70B) achieves 40 percent fully correct 10-digit classifications and 57.5 percent correct 6-digit classifications, improvements of 15 points over GPT-5-Thinking and 27.5 points over Gemini-2.5-Pro-Thinking. Beyond accuracy, Atlas is roughly five times cheaper than GPT-5-Thinking and eight times cheaper than Gemini-2.5-Pro-Thinking, and can be self-hosted to guarantee data privacy in high-stakes trade and compliance workflows. While Atlas sets a strong baseline, the benchmark remains highly challenging, with only 40 percent 10-digit accuracy. By releasing both dataset and model, we aim to position HTS classification as a new community benchmark task and invite future work in retrieval, reasoning, and alignment.

Method: The benchmark encodes verifiable formats directly within JSON schema descriptions and includes 750 test cases with embedded format requirements. Evaluation is fully algorithmic for objectivity and scalability.

Result: Even state-of-the-art proprietary models like GPT-5 and Claude 4.1 Opus frequently fail to follow basic formatting rules, revealing practical limitations for real-world agent systems.

Conclusion: IFEval-FC addresses a critical gap in function calling evaluation and demonstrates that current models struggle with precise format adherence, highlighting the need for improved instruction following capabilities.

Abstract: Function calling is a core capability of large language models, essential for AI agents. Existing benchmarks such as the Berkeley Function Calling Leaderboard (BFCL), tau^2-Bench (arXiv:2506.07982), and ACEBench (arXiv:2501.12851) evaluate argument correctness but do not test adherence to format instructions embedded in parameter descriptions, such as enclosing values in double quotes or using ISO date formats. We introduce IFEval-FC, a benchmark inspired by IFEval (arXiv:2311.07911) that assesses precise instruction following in function calling. IFEval-FC encodes verifiable formats directly within JSON schema descriptions, for example specifying that a value must not contain punctuation. It includes 750 test cases, each consisting of a function with an embedded format for one of its input parameters and a corresponding user query. Evaluation is fully algorithmic, ensuring objectivity, reproducibility, and scalability. Our results show that even state-of-the-art proprietary models, including GPT-5 and Claude 4.1 Opus, frequently fail to follow basic formatting rules, highlighting a practical limitation for real-world agent systems. The complete codebase and data are publicly available at https://github.com/Skripkon/IFEval-FC.

Method: Proposed Pensieve pipeline with memory-specific augmentation, time- and location-aware multi-signal retrieval, and multi-memory QA fine-tuning.

Result: Superior performance over state-of-the-art solutions (up to 14% improvement on QA accuracy) demonstrated on a multimodal benchmark.

Conclusion: Pensieve effectively addresses the unique challenges of Memory-QA task and shows significant improvements in answering recall questions from multimodal memories.

Abstract: We introduce Memory-QA, a novel real-world task that involves answering recall questions about visual content from previously stored multimodal memories. This task poses unique challenges, including the creation of task-oriented memories, the effective utilization of temporal and location information within memories, and the ability to draw upon multiple memories to answer a recall question. To address these challenges, we propose a comprehensive pipeline, Pensieve, integrating memory-specific augmentation, time- and location-aware multi-signal retrieval, and multi-memory QA fine-tuning. We created a multimodal benchmark to illustrate various real challenges in this task, and show the superior performance of Pensieve over state-of-the-art solutions (up to 14% on QA accuracy).

Method: Extracts 2D joint positions from video, normalizes them, computes 101-dimensional kinematic features, applies Transformer for multi-label move classification, and uses distilled language model with encoded right-of-way rules for decision making.

Result: Achieves average macro-F1 score of 0.549 in 5-fold cross-validation, outperforming TCN, BiLSTM, and vanilla Transformer baselines.

Conclusion: While not deployment-ready, FERA demonstrates promising path towards automated referee assistance and opens opportunities for AI applications in fencing coaching.

Abstract: The sport of fencing, like many other sports, faces challenges in refereeing: subjective calls, human errors, bias, and limited availability in practice environments. We present FERA (Fencing Referee Assistant), a prototype AI referee for foil fencing which integrates pose-based multi-label action recognition and rule-based reasoning. FERA extracts 2D joint positions from video, normalizes them, computes a 101-dimensional kinematic feature set, and applies a Transformer for multi-label move and blade classification. To determine priority and scoring, FERA applies a distilled language model with encoded right-of-way rules, producing both a decision and an explanation for each exchange. With limited hand-labeled data, a 5-fold cross-validation achieves an average macro-F1 score of 0.549, outperforming multiple baselines, including a Temporal Convolutional Network (TCN), BiLSTM, and a vanilla Transformer. While not ready for deployment, these results demonstrate a promising path towards automated referee assistance in foil fencing and new opportunities for AI applications, such as coaching in the field of fencing.

Method: The paper proposes LLMZ+, which implements prompt whitelisting to allow only predefined safe messages to interact with agentic LLMs, leveraging contextual specificity to enforce operational boundaries.

Result: Empirical evaluation shows LLMZ+ provides strong resilience against common jailbreak prompts while maintaining legitimate business communications. False positive and false negative rates were reduced to 0 in experimental settings.

Conclusion: LLMZ+ offers a more streamlined and resilient security framework that reduces resource requirements for sustaining LLM information security compared to traditional detection-based approaches.

Abstract: Compared to traditional models, agentic AI represents a highly valuable target for potential attackers as they possess privileged access to data sources and API tools, which are traditionally not incorporated into classical agents. Unlike a typical software application residing in a Demilitarized Zone (DMZ), agentic LLMs consciously rely on nondeterministic behavior of the AI (only defining a final goal, leaving the path selection to LLM). This characteristic introduces substantial security risk to both operational security and information security. Most common existing defense mechanism rely on detection of malicious intent and preventing it from reaching the LLM agent, thus protecting against jailbreak attacks such as prompt injection. In this paper, we present an alternative approach, LLMZ+, which moves beyond traditional detection-based approaches by implementing prompt whitelisting. Through this method, only contextually appropriate and safe messages are permitted to interact with the agentic LLM. By leveraging the specificity of context, LLMZ+ guarantees that all exchanges between external users and the LLM conform to predefined use cases and operational boundaries. Our approach streamlines the security framework, enhances its long-term resilience, and reduces the resources required for sustaining LLM information security. Our empirical evaluation demonstrates that LLMZ+ provides strong resilience against the most common jailbreak prompts. At the same time, legitimate business communications are not disrupted, and authorized traffic flows seamlessly between users and the agentic LLM. We measure the effectiveness of approach using false positive and false negative rates, both of which can be reduced to 0 in our experimental setting.

Method: First prompts LLM to create equations from question decomposition, uses external symbolic solver, then verifies by having LLM estimate the answer and compare with generated solution. Implements iterative rectification if verification fails.

Result: Achieves new state-of-the-art results on numeric and algebraic MWPs, improving previous best by nearly 2% on average. Also obtains satisfactory results on trigonometric MWPs, which was previously unattempted.

Conclusion: The proposed approach effectively enhances LLM performance on MWPs through equation generation, symbolic solving, and verification processes, while introducing new datasets for further testing.

Abstract: Large Language Models (LLMs) excel at various tasks, including problem-solving and question-answering. However, LLMs often find Math Word Problems (MWPs) challenging because solving them requires a range of reasoning and mathematical abilities with which LLMs seem to struggle. Recent efforts have helped LLMs solve more complex MWPs with improved prompts. This study proposes a novel method that initially prompts an LLM to create equations from a decomposition of the question, followed by using an external symbolic equation solver to produce an answer. To ensure the accuracy of the obtained answer, inspired by an established recommendation of math teachers, the LLM is instructed to solve the MWP a second time, but this time with the objective of estimating the correct answer instead of solving it exactly. The estimation is then compared to the generated answer to verify. If verification fails, an iterative rectification process is employed to ensure the correct answer is eventually found. This approach achieves new state-of-the-art results on datasets used by prior published research on numeric and algebraic MWPs, improving the previous best results by nearly two percent on average. In addition, the approach obtains satisfactory results on trigonometric MWPs, a task not previously attempted to the authors’ best knowledge. This study also introduces two new datasets, SVAMPClean and Trig300, to further advance the testing of LLMs’ reasoning abilities.

Method: Novel framework combining Mesa-based spatial modelling with CLIMADA climate impact assessment, introducing adaptive learning behaviors where firms evolve strategies for budget allocation, pricing, wages, and risk adaptation through fitness-based selection and mutation.

Result: Evolutionary adaptation enables firms to converge with baseline production levels after decades of disruption due to climate stress. Systemic risks emerge where even non-exposed agents face impacts through supply chain disruptions, with end-of-century average price of goods 5.6% higher under RCP8.5 compared to baseline.

Conclusion: This open-source framework provides financial institutions and companies with tools to quantify both direct and cascading climate risks while evaluating cost-effective adaptation strategies.

Abstract: Climate risk assessment requires modelling complex interactions between spatially heterogeneous hazards and adaptive economic systems. We present a novel geospatial agent-based model that integrates climate hazard data with evolutionary learning for economic agents. Our framework combines Mesa-based spatial modelling with CLIMADA climate impact assessment, introducing adaptive learning behaviours that allow firms to evolve strategies for budget allocation, pricing, wages, and risk adaptation through fitness-based selection and mutation. We demonstrate the framework using riverine flood projections under RCP8.5 until 2100, showing that evolutionary adaptation enables firms to converge with baseline (no hazard) production levels after decades of disruption due to climate stress. Our results reveal systemic risks where even agents that are not directly exposed to floods face impacts through supply chain disruptions, with the end-of-century average price of goods 5.6% higher under RCP8.5 compared to the baseline. This open-source framework provides financial institutions and companies with tools to quantify both direct and cascading climate risks while evaluating cost-effective adaptation strategies.

Method: Incorporates Personalized PageRank (PPR) during retrieval phase, building informative graphs with significantly lower token consumption.

Result: Achieves at least 80% accuracy compared to widely used graph-based RAG methods while consuming only 3%-11% of output tokens.

Conclusion: TERAG provides a simple yet effective solution for cost-efficient graph-based RAG systems that maintain high accuracy.

Abstract: Graph-based Retrieval-augmented generation (RAG) has become a widely studied approach for improving the reasoning, accuracy, and factuality of Large Language Models. However, many existing graph-based RAG systems overlook the high cost associated with LLM token usage during graph construction, hindering large-scale adoption. To address this, we propose TERAG, a simple yet effective framework designed to build informative graphs at a significantly lower cost. Inspired by HippoRAG, we incorporate Personalized PageRank (PPR) during the retrieval phase, and we achieve at least 80% of the accuracy of widely used graph-based RAG methods while consuming only 3%-11% of the output tokens.

Method: The paper differentiates between ML models and their Machine Learning Model Descriptions (MLMD), refines the concept of semantics preservation for accurate model replication, and applies these contributions to industrial use cases to build and compare target models.

Result: The approach enables clearer understanding and verification of ML model behavior in safety-critical airborne systems, facilitating compliance with aviation safety standards.

Conclusion: Proper distinction between ML models and their unambiguous descriptions, along with refined semantics preservation, is crucial for ensuring the safe operation and regulatory compliance of ML-based airborne systems.

Abstract: Machine Learning (ML) may offer new capabilities in airborne systems. However, as any piece of airborne systems, ML-based systems will be required to guarantee their safe operation. Thus, their development will have to be demonstrated to be compliant with the adequate guidance. So far, the European Union Aviation Safety Agency (EASA) has published a concept paper and an EUROCAE/SAE group is preparing ED-324. Both approaches delineate high-level objectives to confirm the ML model achieves its intended function and maintains training performance in the target environment. The paper aims to clarify the difference between an ML model and its corresponding unambiguous description, referred to as the Machine Learning Model Description (MLMD). It then refines the essential notion of semantics preservation to ensure the accurate replication of the model. We apply our contributions to several industrial use cases to build and compare several target models.

Method: The study systematically reviews up-to-date research progress, categorizes medical LLMs into three types based on training methodologies, classifies evaluation approaches into two categories, and analyzes training techniques and healthcare adaptations.

Result: The review provides comprehensive analysis of medical LLMs’ current state, identifies existing challenges, and offers innovative categorization frameworks for understanding different types of medical LLMs and their evaluation methods.

Conclusion: The paper proposes solutions to existing challenges and outlines future research directions, aiming to provide deeper understanding of medical LLMs’ development and clear guidance for subsequent research in this important application area.

Abstract: Artificial intelligence (AI) technology has advanced rapidly in recent years, with large language models (LLMs) emerging as a significant breakthrough. LLMs are increasingly making an impact across various industries, with the medical field standing out as the most prominent application area. This paper systematically reviews the up-to-date research progress of LLMs in the medical field, providing an in-depth analysis of training techniques for large medical models, their adaptation in healthcare settings, related applications, as well as their strengths and limitations. Furthermore, it innovatively categorizes medical LLMs into three distinct types based on their training methodologies and classifies their evaluation approaches into two categories. Finally, the study proposes solutions to existing challenges and outlines future research directions based on identified issues in the field of medical LLMs. By systematically reviewing previous and advanced research findings, we aim to highlight the necessity of developing medical LLMs, provide a deeper understanding of their current state of development, and offer clear guidance for subsequent research.

Method: DataAgents integrate LLM reasoning with task decomposition, action reasoning, grounding, and tool calling to autonomously interpret data tasks, plan workflows, and execute operations through Python code or tool calls.

Result: DataAgents enable dynamic workflow planning and scalable adaptation to diverse data tasks, transforming unstructured data into coherent knowledge through automated collection, integration, preprocessing, and other data operations.

Conclusion: DataAgents mark a critical shift toward autonomous data-to-knowledge systems, requiring further research in workflow optimization, benchmark ecosystems, privacy safeguards, and trustworthy guardrails.

Abstract: As data continues to grow in scale and complexity, preparing, transforming, and analyzing it remains labor-intensive, repetitive, and difficult to scale. Since data contains knowledge and AI learns knowledge from it, the alignment between AI and data is essential. However, data is often not structured in ways that are optimal for AI utilization. Moreover, an important question arises: how much knowledge can we pack into data through intensive data operations? Autonomous data agents (DataAgents), which integrate LLM reasoning with task decomposition, action reasoning and grounding, and tool calling, can autonomously interpret data task descriptions, decompose tasks into subtasks, reason over actions, ground actions into python code or tool calling, and execute operations. Unlike traditional data management and engineering tools, DataAgents dynamically plan workflows, call powerful tools, and adapt to diverse data tasks at scale. This report argues that DataAgents represent a paradigm shift toward autonomous data-to-knowledge systems. DataAgents are capable of handling collection, integration, preprocessing, selection, transformation, reweighing, augmentation, reprogramming, repairs, and retrieval. Through these capabilities, DataAgents transform complex and unstructured data into coherent and actionable knowledge. We first examine why the convergence of agentic AI and data-to-knowledge systems has emerged as a critical trend. We then define the concept of DataAgents and discuss their architectural design, training strategies, as well as the new skills and capabilities they enable. Finally, we call for concerted efforts to advance action workflow optimization, establish open datasets and benchmark ecosystems, safeguard privacy, balance efficiency with scalability, and develop trustworthy DataAgent guardrails to prevent malicious actions.

Method: A framework that captures raw interactions, distills them into compact reusable knowledge, and periodically refines stored content to preserve relevance and efficiency through autonomous environmental interaction and collaborative experience sharing.

Result: Validated in simulated real-world scenarios, experience scaling improves accuracy, sustains performance over time, and maintains gains when applied to novel situations across generalization to unseen tasks, repetitive queries, and over-saturated knowledge stores.

Conclusion: Structured post-deployment learning can extend LLM capabilities beyond static human-generated data limits, offering a scalable path for continued intelligence progress through continuous evolution rather than one-time training.

Abstract: Scaling model size, training data, and compute power have driven advances in large language models (LLMs), but these approaches are reaching saturation as human-generated text is exhausted and further gains diminish. We propose experience scaling, a framework for continuous post-deployment evolution for LLMs through autonomous interaction with the environment and collaborative sharing of accumulated experience. The framework captures raw interactions, distills them into compact, reusable knowledge, and periodically refines stored content to preserve relevance and efficiency. We validate the framework in simulated real-world scenarios involving generalization to previously unseen but related tasks, repetitive queries, and over-saturated knowledge stores. Across all settings, experience scaling improves accuracy, sustains performance over time, and maintains gains when applied to novel situations. These results demonstrate that structured post-deployment learning can extend LLM capabilities beyond the limits of static human-generated data, offering a scalable path for continued intelligence progress.

Method: Built on Open Agentic Schema Framework (OASF) with a two-level mapping over Kademlia-based DHT, leveraging OCI/ORAS infrastructure for artifact distribution, Sigstore for provenance, and supporting schema-driven extensibility.

Result: ADS provides an architectural model that decouples capability indexing from content location, enabling multi-dimensional discovery with security and performance properties for agent registry and interoperability.

Conclusion: ADS positions itself as a foundational component in the emerging landscape of agent registry and interoperability initiatives, offering a scalable and secure solution for agent capability discovery.

Abstract: The Agent Directory Service (ADS) is a distributed directory for the discovery of AI agent capabilities, metadata, and provenance. It leverages content-addressed storage, hierarchical taxonomies, and cryptographic signing to enable efficient, verifiable, and multi-dimensional discovery across heterogeneous Multi-Agent Systems (MAS). Built on the Open Agentic Schema Framework (OASF), ADS decouples capability indexing from content location through a two-level mapping realized over a Kademlia-based Distributed Hash Table (DHT). It reuses mature OCI / ORAS infrastructure for artifact distribution, integrates Sigstore for provenance, and supports schema-driven extensibility for emerging agent modalities (LLM prompt agents, MCP servers, A2A-enabled components). This paper formalizes the architectural model, describes storage and discovery layers, explains security and performance properties, and positions ADS within the broader landscape of emerging agent registry and interoperability initiatives.

Method: Introduces α-k-bounded text generation that focuses on top-k tokens with cumulative probability ≥α at each step, then applies PCTL-based model checking to verify properties like text quality and biases.

Result: The method was successfully demonstrated on various LLMs including Llama, Gemma, Mistral, Genstruct, and BERT, showing applicability across different models.

Conclusion: This represents the first application of PCTL-based model checking for verifying LLM text generation consistency, providing a formal verification framework for LLM outputs.

Abstract: In this paper, we introduce LLMCHECKER, a model-checking-based verification method to verify the probabilistic computation tree logic (PCTL) properties of an LLM text generation process. We empirically show that only a limited number of tokens are typically chosen during text generation, which are not always the same. This insight drives the creation of $\alpha$-$k$-bounded text generation, narrowing the focus to the $\alpha$ maximal cumulative probability on the top-$k$ tokens at every step of the text generation process. Our verification method considers an initial string and the subsequent top-$k$ tokens while accommodating diverse text quantification methods, such as evaluating text quality and biases. The threshold $\alpha$ further reduces the selected tokens, only choosing those that exceed or meet it in cumulative probability. LLMCHECKER then allows us to formally verify the PCTL properties of $\alpha$-$k$-bounded LLMs. We demonstrate the applicability of our method in several LLMs, including Llama, Gemma, Mistral, Genstruct, and BERT. To our knowledge, this is the first time PCTL-based model checking has been used to check the consistency of the LLM text generation process.

Method: Proposed framework includes LLM-as-judge evaluation with Plackett-Luce aggregation for model selection, embedding-based similarity measures with redundancy-aware sampling to remove duplicated discharge summaries, and structured input design with section-wise content analysis under different modeling paradigms.

Result: Experiments show the selected base model after fine-tuning consistently outperforms baseline LLMs in internal and external evaluations, and incorporating more clinical sections consistently improves prediction performance.

Conclusion: The framework provides a scalable, institution-ready solution for real-world deployment of automated medical coding systems by combining informed model selection, efficient data refinement, and context-aware prompting.

Abstract: Accurate International Classification of Diseases (ICD) coding is critical for clinical documentation, billing, and healthcare analytics, yet it remains a labour-intensive and error-prone task. Although large language models (LLMs) show promise in automating ICD coding, their challenges in base model selection, input contextualization, and training data redundancy limit their effectiveness. We propose a modular framework for ICD-10 Clinical Modification (ICD-10-CM) code prediction that addresses these challenges through principled model selection, redundancy-aware data sampling, and structured input design. The framework integrates an LLM-as-judge evaluation protocol with Plackett-Luce aggregation to assess and rank open-source LLMs based on their intrinsic comprehension of ICD-10-CM code definitions. We introduced embedding-based similarity measures, a redundancy-aware sampling strategy to remove semantically duplicated discharge summaries. We leverage structured discharge summaries from Taiwanese hospitals to evaluate contextual effects and examine section-wise content inclusion under universal and section-specific modelling paradigms. Experiments across two institutional datasets demonstrate that the selected base model after fine-tuning consistently outperforms baseline LLMs in internal and external evaluations. Incorporating more clinical sections consistently improves prediction performance. This study uses open-source LLMs to establish a practical and principled approach to ICD-10-CM code prediction. The proposed framework provides a scalable, institution-ready solution for real-world deployment of automated medical coding systems by combining informed model selection, efficient data refinement, and context-aware prompting.

Method: MAPO introduces advantage percent deviation for high-certainty trajectories and dynamically reweights the advantage function based on trajectory certainty to adaptively configure advantages for sample-specific characteristics.

Result: Comparison with state-of-the-art methods and ablation studies validate the effectiveness of MAPO in improving foundation model performance on reasoning tasks.

Conclusion: MAPO provides an effective solution to advantage allocation problems in GRPO, enhancing foundation model reasoning capabilities through dynamic advantage reweighting based on trajectory certainty.

Abstract: Recent advances in reinforcement learning for foundation models, such as Group Relative Policy Optimization (GRPO), have significantly improved the performance of foundation models on reasoning tasks. Notably, the advantage function serves as a central mechanism in GRPO for ranking the trajectory importance. However, existing explorations encounter both advantage reversion and advantage mirror problems, which hinder the reasonable advantage allocation across different query samples. In this work, we propose an easy but effective GRPO strategy, Mixed Advantage Policy Optimization (MAPO). We reveal that the trajectory appears with different certainty and propose the advantage percent deviation for samples with high-certainty trajectories. Furthermore, we dynamically reweight the advantage function for samples with varying trajectory certainty, thereby adaptively configuring the advantage function to account for sample-specific characteristics. Comparison with related state-of-the-art methods, along with ablation studies on different advantage variants, validates the effectiveness of our approach.

Method: Conf-Profile uses a two-stage approach: synthesizing high-quality labels with confidence hints, then applying confidence-weighted voting and calibration. It also uses confidence-guided unsupervised reinforcement learning for reasoning enhancement.

Result: Experimental results show Conf-Profile improves F1 score by 13.97 on Qwen3-8B, demonstrating substantial performance gains.

Conclusion: The proposed framework effectively addresses label scarcity and data reliability issues in user profiling, achieving significant improvements through confidence-driven methods.

Abstract: User profiling, as a core technique for user understanding, aims to infer structural attributes from user information. Large Language Models (LLMs) provide a promising avenue for user profiling, yet the progress is hindered by the lack of comprehensive benchmarks. To bridge this gap, we propose ProfileBench, an industrial benchmark derived from a real-world video platform, encompassing heterogeneous user data and a well-structured profiling taxonomy. However, the profiling task remains challenging due to the difficulty of collecting large-scale ground-truth labels, and the heterogeneous and noisy user information can compromise the reliability of LLMs. To approach label-free and reliable user profiling, we propose a Confidence-driven Profile reasoning framework Conf-Profile, featuring a two-stage paradigm. We first synthesize high-quality labels by leveraging advanced LLMs with confidence hints, followed by confidence-weighted voting for accuracy improvement and confidence calibration for a balanced distribution. The multiple profile results, rationales, and confidence scores are aggregated and distilled into a lightweight LLM. We further enhance the reasoning ability via confidence-guided unsupervised reinforcement learning, which exploits confidence for difficulty filtering, quasi-ground truth voting, and reward weighting. Experimental results demonstrate that Conf-Profile delivers substantial performance through the two-stage training, improving F1 by 13.97 on Qwen3-8B.

Method: Proposes a four-part memory taxonomy (parametric, contextual, external, procedural/episodic) with a memory quadruple framework, three-setting evaluation protocol, layered evaluation approach, and DMM Gov governance system for memory updates.

Result: Develops a comprehensive coordinate system that integrates temporal governance, leakage auditing, uncertainty reporting, and memory updating mechanisms for reproducible and comparable LLM memory research.

Conclusion: The framework provides testable propositions and a systematic approach to make LLM memory research reproducible, comparable, and governable across different implementations and deployment scenarios.

Abstract: Under a unified operational definition, we define LLM memory as a persistent state written during pretraining, finetuning, or inference that can later be addressed and that stably influences outputs. We propose a four-part taxonomy (parametric, contextual, external, procedural/episodic) and a memory quadruple (location, persistence, write/access path, controllability). We link mechanism, evaluation, and governance via the chain write -> read -> inhibit/update. To avoid distorted comparisons across heterogeneous setups, we adopt a three-setting protocol (parametric only, offline retrieval, online retrieval) that decouples capability from information availability on the same data and timeline. On this basis we build a layered evaluation: parametric (closed-book recall, edit differential, memorization/privacy), contextual (position curves and the mid-sequence drop), external (answer correctness vs snippet attribution/faithfulness), and procedural/episodic (cross-session consistency and timeline replay, E MARS+). The framework integrates temporal governance and leakage auditing (freshness hits, outdated answers, refusal slices) and uncertainty reporting via inter-rater agreement plus paired tests with multiple-comparison correction. For updating and forgetting, we present DMM Gov: coordinating DAPT/TAPT, PEFT, model editing (ROME, MEND, MEMIT, SERAC), and RAG to form an auditable loop covering admission thresholds, rollout, monitoring, rollback, and change audits, with specs for timeliness, conflict handling, and long-horizon consistency. Finally, we give four testable propositions: minimum identifiability; a minimal evaluation card; causally constrained editing with verifiable forgetting; and when retrieval with small-window replay outperforms ultra-long-context reading. This yields a reproducible, comparable, and governable coordinate system for research and deployment.

Method: Uses a cold-start training strategy with long Chain-of-Thought data, followed by domain-parallel training across STEM, Code, and Agentic domains, fused into a Pareto-optimal model using the DORA system for large-scale RL training.

Result: Achieves state-of-the-art performance among open-source models on complex reasoning tasks, with 64.5% reduction in token consumption on AIME-25 while maintaining task accuracy.

Conclusion: The model demonstrates that efficient large-scale reasoning is achievable through careful training strategies and domain-parallel optimization, and is released to advance reasoning systems and agentic AI research.

Abstract: We present LongCat-Flash-Thinking, an efficient 560-billion-parameter open-source Mixture-of-Experts (MoE) reasoning model. Its advanced capabilities are cultivated through a meticulously crafted training process, beginning with long Chain-of-Thought (CoT) data cold-start and culminating in large-scale Reinforcement Learning (RL). We first employ a well-designed cold-start training strategy, which significantly enhances the reasoning potential and equips the model with specialized skills in both formal and agentic reasoning. Then, a core innovation is our domain-parallel training scheme, which decouples optimization across distinct domains (e.g., STEM, Code, Agentic) and subsequently fuses the resulting expert models into a single, nearly Pareto-optimal model. This entire process is powered by our Dynamic ORchestration for Asynchronous rollout (DORA) system, a large-scale RL framework that delivers a greater than threefold training speedup over synchronous methods on tens of thousands of accelerators. As a result, LongCat-Flash-Thinking achieves state-of-the-art performance among open-source models on a suite of complex reasoning tasks. The model exhibits exceptional efficiency in agentic reasoning, reducing average token consumption by 64.5% (from 19, 653 to 6, 965) on AIME-25, without degrading task accuracy. We release LongCat-Flash-Thinking to promote further advances in reasoning systems and agentic AI research.

Method: The study conducts a systematic review of VSR methodologies across input modalities, model architectures, training strategies, and reasoning mechanisms. It categorizes spatial intelligence into three levels (basic perception, spatial understanding, spatial planning) and creates SIBench benchmark.

Result: Experiments with state-of-the-art VLMs reveal a significant gap between perception and reasoning - models perform well on basic perceptual tasks but consistently underperform in understanding and planning tasks, especially in numerical estimation, multi-view reasoning, temporal dynamics, and spatial imagination.

Conclusion: There remain substantial challenges in achieving true spatial intelligence, but this work provides both a systematic roadmap and comprehensive benchmark to guide future research in visual spatial reasoning for VLMs.

Abstract: Visual Spatial Reasoning (VSR) is a core human cognitive ability and a critical requirement for advancing embodied intelligence and autonomous systems. Despite recent progress in Vision-Language Models (VLMs), achieving human-level VSR remains highly challenging due to the complexity of representing and reasoning over three-dimensional space. In this paper, we present a systematic investigation of VSR in VLMs, encompassing a review of existing methodologies across input modalities, model architectures, training strategies, and reasoning mechanisms. Furthermore, we categorize spatial intelligence into three levels of capability, ie, basic perception, spatial understanding, spatial planning, and curate SIBench, a spatial intelligence benchmark encompassing nearly 20 open-source datasets across 23 task settings. Experiments with state-of-the-art VLMs reveal a pronounced gap between perception and reasoning, as models show competence in basic perceptual tasks but consistently underperform in understanding and planning tasks, particularly in numerical estimation, multi-view reasoning, temporal dynamics, and spatial imagination. These findings underscore the substantial challenges that remain in achieving spatial intelligence, while providing both a systematic roadmap and a comprehensive benchmark to drive future research in the field. The related resources of this study are accessible at https://sibench.github.io/Awesome-Visual-Spatial-Reasoning/.

Method: DEAL integrates LoRA with continuous fine-tuning strategy, incorporating knowledge retention and adaptive parameter update modules to mitigate limitations of existing FT methods while maintaining privacy-preserving efficiency.

Result: Experiments on 15 diverse datasets show DEAL consistently outperforms baseline methods, yielding substantial gains in task accuracy and resource efficiency.

Conclusion: The approach demonstrates potential to advance continual adaptation in LLMs by enhancing task performance while improving resource efficiency.

Abstract: Recent advancements in Large Language Models (LLMs) have emphasized the critical role of fine-tuning (FT) techniques in adapting LLMs to specific tasks, especially when retraining from scratch is computationally infeasible. Fine-tuning enables LLMs to leverage task- or domain-specific data, producing models that more effectively meet the requirements of targeted applications. However, con- ventional FT approaches often suffer from catastrophic forgetting and suboptimal data efficiency, limiting their real-world applicability. To address these challenges, this paper proposes DEAL, a novel framework that integrates Low-Rank Adapta- tion (LoRA) with a continuous fine-tuning strategy. By incorporating knowledge retention and adaptive parameter update modules, the framework mitigates the lim- itations of existing FT methods while maintaining efficiency in privacy-preserving settings. Experiments on 15 diverse datasets show that DEAL consistently outper- forms baseline methods, yielding substantial gains in task accuracy and resource efficiency. These findings demonstrate the potential of our approach to advance continual adaptation in LLMs by enhancing task performance while improving resource efficiency.

Method: The authors analyze the complete workflow of agents to propose a new taxonomy of hallucination types, conduct in-depth examination of 18 triggering causes, and review existing studies on mitigation and detection approaches.

Result: The survey provides a systematic framework for understanding agent hallucinations, identifies different types occurring at various workflow stages, and summarizes current mitigation and detection methods.

Conclusion: This comprehensive survey aims to inspire further research on addressing hallucinations in LLM-based agents to develop more robust and reliable agent systems.

Abstract: Driven by the rapid advancements of Large Language Models (LLMs), LLM-based agents have emerged as powerful intelligent systems capable of human-like cognition, reasoning, and interaction. These agents are increasingly being deployed across diverse real-world applications, including student education, scientific research, and financial analysis. However, despite their remarkable potential, LLM-based agents remain vulnerable to hallucination issues, which can result in erroneous task execution and undermine the reliability of the overall system design. Addressing this critical challenge requires a deep understanding and a systematic consolidation of recent advances on LLM-based agents. To this end, we present the first comprehensive survey of hallucinations in LLM-based agents. By carefully analyzing the complete workflow of agents, we propose a new taxonomy that identifies different types of agent hallucinations occurring at different stages. Furthermore, we conduct an in-depth examination of eighteen triggering causes underlying the emergence of agent hallucinations. Through a detailed review of a large number of existing studies, we summarize approaches for hallucination mitigation and detection, and highlight promising directions for future research. We hope this survey will inspire further efforts toward addressing hallucinations in LLM-based agents, ultimately contributing to the development of more robust and reliable agent systems.

Method: Use constrained matrix factorization as an interpretable recommendation model, then translate its structure into natural language explanations using carefully designed LLM prompts. Conduct a study with 326 participants assessing explanation quality across five dimensions.

Result: All explanation types were generally well received with moderate statistical differences between strategies. User comments provided complementary insights beyond quantitative results.

Conclusion: LLMs can effectively generate user-facing explanations from interpretable models, and user-centered evaluation provides valuable insights that automatic metrics miss.

Abstract: We investigate whether large language models (LLMs) can generate effective, user-facing explanations from a mathematically interpretable recommendation model. The model is based on constrained matrix factorization, where user types are explicitly represented and predicted item scores share the same scale as observed ratings, making the model’s internal representations and predicted scores directly interpretable. This structure is translated into natural language explanations using carefully designed LLM prompts. Many works in explainable AI rely on automatic evaluation metrics, which often fail to capture users’ actual needs and perceptions. In contrast, we adopt a user-centered approach: we conduct a study with 326 participants who assessed the quality of the explanations across five key dimensions-transparency, effectiveness, persuasion, trust, and satisfaction-as well as the recommendations themselves.To evaluate how different explanation strategies are perceived, we generate multiple explanation types from the same underlying model, varying the input information provided to the LLM. Our analysis reveals that all explanation types are generally well received, with moderate statistical differences between strategies. User comments further underscore how participants react to each type of explanation, offering complementary insights beyond the quantitative results.

Method: Evaluated four different remaining time prediction approaches including deep learning models and shallow methods like conventional boosting techniques on a real-world event log

Result: Deep learning models achieved highest accuracy, but shallow methods like boosting techniques achieved competitive accuracy with significantly fewer computational resources

Conclusion: While deep learning provides the best accuracy, shallow methods offer a good trade-off between accuracy and computational efficiency for remaining time prediction in process mining

Abstract: Predictive process monitoring is a sub-domain of process mining which aims to forecast the future of ongoing process executions. One common prediction target is the remaining time, meaning the time that will elapse until a process execution is completed. In this paper, we compare four different remaining time prediction approaches in a real-life outbound warehouse process of a logistics company in the aviation business. For this process, the company provided us with a novel and original event log with 169,523 traces, which we can make publicly available. Unsurprisingly, we find that deep learning models achieve the highest accuracy, but shallow methods like conventional boosting techniques achieve competitive accuracy and require significantly fewer computational resources.

Method: Drawing on Games Studies and Game AI research, the authors propose three player-centric concepts based on perceivability, evocativeness, and Call to Action. These concepts are designed to be generic across game genres and detectable using existing techniques from research and industry.

Result: The paper presents a framework for automated decomposition of PCG content using Landmarks, Monuments, and Beacons, which can be evaluated with current techniques, enabling better computational evaluation of salient sub-components.

Conclusion: This approach creates a connection between humanities and technical game research, allowing for better computational PCG evaluation. While emphasizing mixed-initiative and compositional PCG, the concepts are applicable beyond these domains.

Abstract: Algorithmic evaluation of procedurally generated content struggles to find metrics that align with human experience, particularly for composite artefacts. Automatic decomposition as a possible solution requires concepts that meet a range of properties. To this end, drawing on Games Studies and Game AI research, we introduce the nested concepts of \textit{Landmarks}, \textit{Monuments}, and \textit{Beacons}. These concepts are based on the artefact’s perceivability, evocativeness, and Call to Action, all from a player-centric perspective. These terms are generic to games and usable across genres. We argue that these entities can be found and evaluated with techniques currently used in both research and industry, opening a path towards a fully automated decomposition of PCG, and evaluation of the salient sub-components. Although the work presented here emphasises mixed-initiative PCG and compositional PCG, we believe it applies beyond those domains. With this approach, we intend to create a connection between humanities and technical game research and allow for better computational PCG evaluation

Method: Introduce observable sources as auxiliary variables for conditioning, use volume-preserving encoders for identification, and provide a variable-selection scheme when multiple auxiliary variables are available.

Result: The framework can identify entire latent variables up to subspace-wise transformations and permutations, and experiments on synthetic graph and image data demonstrate effectiveness.

Conclusion: This approach extends the boundaries of current causal representation learning methods by leveraging observable sources as auxiliary variables.

Abstract: Causal representation learning seeks to recover latent factors that generate observational data through a mixing function. Needing assumptions on latent structures or relationships to achieve identifiability in general, prior works often build upon conditional independence given known auxiliary variables. However, prior frameworks limit the scope of auxiliary variables to be external to the mixing function. Yet, in some cases, system-driving latent factors can be easily observed or extracted from data, possibly facilitating identification. In this paper, we introduce a framework of observable sources being auxiliaries, serving as effective conditioning variables. Our main results show that one can identify entire latent variables up to subspace-wise transformations and permutations using volume-preserving encoders. Moreover, when multiple known auxiliary variables are available, we offer a variable-selection scheme to choose those that maximize recoverability of the latent factors given knowledge of the latent causal graph. Finally, we demonstrate the effectiveness of our framework through experiments on synthetic graph and image data, thereby extending the boundaries of current approaches.

Method: Generate diverse high-level planning programs with LLMs, iteratively refine candidate plans, and use a trained domain-adaptive critic to select the best plan aligned with long-term rewards.

Result: Outperforms AdaPlanner and Reflexion in ALFWorld, NetHack, and StarCraft II with 23.33% higher success rate and 91.27% lower query costs.

Conclusion: CoPiC effectively improves planning performance while significantly reducing LLM query frequency through program-based planning and domain-adaptive criticism.

Abstract: Large Language Models (LLMs) have been widely adopted as task planners for AI agents in sequential decision-making problems, leveraging their extensive world knowledge. However, the gap between their general knowledge and environment-specific requirements often leads to inaccurate plans. To address this, existing approaches rely on frequent LLM queries to iteratively refine plans based on immediate environmental feedback, which incurs substantial query costs. However, this refinement is typically guided by short-term environmental feedback, limiting LLMs from developing plans aligned with long-term rewards. We propose Code Driven Planning with Domain-Adaptive Critic (CoPiC). Instead of relying on frequent queries, CoPiC employs LLMs to generate a diverse set of high-level planning programs, which iteratively produce and refine candidate plans. A trained domain-adaptive critic then evaluates these candidates and selects the one most aligned with long-term rewards for execution. Using high-level planning programs as planner and domain-adaptive critic as estimator, CoPiC improves planning while significantly reducing query costs. Results in ALFWorld, NetHack, and StarCraft II Unit Building show that CoPiC outperforms advanced LLM-based baselines, AdaPlanner and Reflexion, achieving an average (1) 23.33% improvement in success rate and (2) 91.27% reduction in query costs.

Method: AgentInit incorporates: 1) Multi-round interactions and reflections between agents during generation, 2) Natural Language to Format mechanism for consistency, 3) Balanced team selection using Pareto principles to jointly optimize diversity and task relevance.

Result: AgentInit outperforms state-of-the-art initialization methods and pre-defined strategies across various frameworks and tasks, achieving performance improvements of up to 1.2x and 1.6x respectively, while significantly reducing token consumption.

Conclusion: AgentInit demonstrates strong transferability to similar tasks and verifies the effectiveness of its key components, proving to be a capable and adaptable MAS initialization method.

Abstract: Proper initialization is crucial for any system, particularly in multi-agent systems (MAS), where it plays a pivotal role in determining both the system’s efficiency and effectiveness. However, existing MAS initialization methods do not fully account for the collaborative needs of the generated agents in subsequent stages. Inspired by the principles of effective team composition, we propose AgentInit, which aims to optimize the structure of agent teams. Specifically, in addition to multi-round interactions and reflections between agents during agent generation, AgentInit incorporates a Natural Language to Format mechanism to ensure consistency and standardization. Balanced team selection strategies using Pareto principles are subsequently applied to jointly consider agent team diversity and task relevance to promote effective and efficient collaboration and enhance overall system performance. Experiments show that AgentInit consistently outperforms state-of-the-art initialization methods and pre-defined strategies across various frameworks and tasks, achieving an overall performance improvement of up to 1.2 and 1.6, respectively, while also significantly reducing token consumption. Further analysis confirms its strong transferability to similar tasks and verifies the effectiveness of its key components, demonstrating its capability and adaptability as a reliable MAS initialization method. Source code and models are available at https://github.com/1737423697/AgentInit.

Method: Used a culturally grounded commonsense reasoning dataset covering 13 Arab countries. Evaluated lightweight alignment methods including in-context learning, demonstration-based reinforcement (DITTO), supervised fine-tuning, and direct preference optimization.

Result: Just 12 culture-specific examples from one country improved performance in others by 10% on average within multilingual models. Out-of-culture demonstrations from Indonesia and US contexts matched or surpassed in-culture alignment for MCQ reasoning.

Conclusion: Efficient cross-cultural alignment is possible and offers a promising approach to adapt LLMs to low-resource cultural settings, demonstrating cultural commonsense transferability beyond the Arab world.

Abstract: Large language models (LLMs) often reflect Western-centric biases, limiting their effectiveness in diverse cultural contexts. Although some work has explored cultural alignment, the potential for cross-cultural transfer, using alignment in one culture to improve performance in others, remains underexplored. This paper investigates cross-cultural transfer of commonsense reasoning in the Arab world, where linguistic and historical similarities coexist with local cultural differences. Using a culturally grounded commonsense reasoning dataset covering 13 Arab countries, we evaluate lightweight alignment methods such as in-context learning and demonstration-based reinforcement (DITTO), alongside baselines like supervised fine-tuning and direct preference optimization. Our results show that merely 12 culture-specific examples from one country can improve performance in others by 10% on average, within multilingual models. In addition, we demonstrate that out-of-culture demonstrations from Indonesia and US contexts can match or surpass in-culture alignment for MCQ reasoning, highlighting cultural commonsense transferability beyond the Arab world. These findings demonstrate that efficient cross-cultural alignment is possible and offer a promising approach to adapt LLMs to low-resource cultural settings.

Method: Proposes the Artificial Liver Classifier (ALC) with improved FOX optimization algorithm (IFOX) for parameter optimization, evaluated on five benchmark datasets including Iris, Breast Cancer, Wine, Voice Gender, and MNIST.

Result: ALC achieves up to 100% accuracy on Iris dataset (surpassing logistic regression, MLP, and SVM) and 99.12% on Breast Cancer dataset (outperforming XGBoost and logistic regression), with smaller generalization gaps and lower loss values across all datasets.

Conclusion: Biologically inspired models like ALC show potential for developing efficient machine learning classifiers and open new avenues for innovation in the field.

Abstract: Supervised machine learning classifiers sometimes face challenges related to the performance, accuracy, or overfitting. This paper introduces the Artificial Liver Classifier (ALC), a novel supervised learning model inspired by the human liver’s detoxification function. The ALC is characterized by its simplicity, speed, capability to reduce overfitting, and effectiveness in addressing multi-class classification problems through straightforward mathematical operations. To optimize the ALC’s parameters, an improved FOX optimization algorithm (IFOX) is employed during training. We evaluate the proposed ALC on five benchmark datasets: Iris Flower, Breast Cancer Wisconsin, Wine, Voice Gender, and MNIST. The results demonstrate competitive performance, with ALC achieving up to 100% accuracy on the Iris dataset–surpassing logistic regression, multilayer perceptron, and support vector machine–and 99.12% accuracy on the Breast Cancer dataset, outperforming XGBoost and logistic regression. Across all datasets, ALC consistently shows smaller generalization gaps and lower loss values compared to conventional classifiers. These findings highlight the potential of biologically inspired models to develop efficient machine learning classifiers and open new avenues for innovation in the field.

Method: Created three guesstimation datasets (MARBLES, FUTURE, ELECPRED) spanning physical to abstract estimation. Proposed WOC decoding where median aggregation across multiple LLM responses improves accuracy. Compared against greedy decoding, self-consistency decoding, and mean decoding.

Result: LLMs exhibit similar Wisdom of Crowds effects as humans: median aggregation consistently improves accuracy over other decoding methods. This suggests LLMs encode a world model supporting approximate reasoning.

Conclusion: Guesstimation serves as a useful probe of LLM world knowledge, and WOC decoding enhances LLM performance on real-world estimation tasks.

Abstract: Guesstimation–the task of making approximate quantitative estimates about objects or events-is a common real–world skill, yet remains underexplored in large language model (LLM) research. We introduce three guesstimation datasets: MARBLES, FUTURE, and ELECPRED, spanning physical estimation (e.g., how many marbles fit in a cup) to abstract predictions (e.g., the 2024 U.S. presidential election). Inspired by the social science concept of Wisdom of Crowds (WOC)- where the median of multiple estimates improves accuracy-we propose WOC decoding for LLMs. We replicate WOC effects in human participants and find that LLMs exhibit similar benefits: median aggregation across sampled responses consistently improves accuracy over greedy decoding, self-consistency decoding, and mean decoding. This suggests that LLMs encode a world model that supports approximate reasoning. Our results position guesstimation as a useful probe of LLM world knowledge and highlight WOC decoding as a strategy for enhancing LLM guesstimation performance on real-world tasks.

Method: Proposes ReSearch framework that treats search operations as integral to reasoning chains, using reinforcement learning to train LLMs on when and how to perform searches guided by text-based thinking.

Result: Models trained on Qwen2.5-7B and Qwen2.5-32B show strong generalizability across benchmarks despite training on only one dataset, and naturally develop advanced reasoning capabilities like reflection and self-correction.

Conclusion: ReSearch successfully enables LLMs to integrate reasoning with search operations through reinforcement learning, demonstrating effective generalization and emergent advanced reasoning abilities.

Abstract: Large Language Models (LLMs) have shown remarkable capabilities in reasoning, exemplified by the success of OpenAI-o1 and DeepSeek-R1. However, integrating reasoning with external search processes remains challenging, especially for complex multi-hop questions requiring multiple retrieval steps. We propose ReSearch, a novel framework that trains LLMs to Reason with Search via reinforcement learning without using any supervised data on reasoning steps. Our approach treats search operations as integral components of the reasoning chain, where when and how to perform searches is guided by text-based thinking, and search results subsequently influence further reasoning. We train ReSearch on Qwen2.5-7B(-Instruct) and Qwen2.5-32B(-Instruct) models and conduct extensive experiments. Despite being trained on only one dataset, our models demonstrate strong generalizability across various benchmarks. Analysis reveals that ReSearch naturally elicits advanced reasoning capabilities such as reflection and self-correction during the reinforcement learning process.

Method: The framework employs multiple expert agents to analyze target-domain news, uses a question-reflection mechanism for higher-quality analysis, and implements automated decision rule optimization through cross-domain validation tasks.

Result: Experimental results on commonly used datasets show that MARO achieves significant improvements over existing cross-domain misinformation detection methods.

Conclusion: MARO effectively addresses the limitations of current approaches by providing automated decision rule optimization and better target-domain analysis, demonstrating superior performance in cross-domain misinformation detection.

Abstract: Misinformation spans various domains, but detection methods trained on specific domains often perform poorly when applied to others. With the rapid development of Large Language Models (LLMs), researchers have begun to utilize LLMs for cross-domain misinformation detection. However, existing LLM-based methods often fail to adequately analyze news in the target domain, limiting their detection capabilities. More importantly, these methods typically rely on manually designed decision rules, which are limited by domain knowledge and expert experience, thus limiting the generalizability of decision rules to different domains. To address these issues, we propose a MultiAgent Framework for cross-domain misinformation detection with Automated Decision Rule Optimization (MARO). Under this framework, we first employs multiple expert agents to analyze target-domain news. Subsequently, we introduce a question-reflection mechanism that guides expert agents to facilitate higherquality analysis. Furthermore, we propose a decision rule optimization approach based on carefully-designed cross-domain validation tasks to iteratively enhance the effectiveness of decision rules in different domains. Experimental results and in-depth analysis on commonlyused datasets demonstrate that MARO achieves significant improvements over existing methods.

Method: Proposes PromptMeta with two innovations: Meta-Semantic Prompt (MSP) pool that learns high-level meta-semantics shared across tasks, and a learnable fusion mechanism that dynamically combines meta-semantics with task-specific relational information within a meta-learning framework.

Result: Extensive experiments on two real-world KG benchmarks validate PromptMeta’s effectiveness in adapting to new relations with limited supervision.

Conclusion: PromptMeta successfully bridges the gap by integrating meta-semantics with relational information, enabling effective knowledge transfer and adaptation to newly emerging relations in few-shot learning scenarios.

Abstract: Few-shot relational learning on knowledge graph (KGs) aims to perform reasoning over relations with only a few training examples. While current methods have focused primarily on leveraging specific relational information, rich semantics inherent in KGs have been largely overlooked. To bridge this gap, we propose PromptMeta, a novel prompted meta-learning framework that seamlessly integrates meta-semantics with relational information for few-shot relational learning. PromptMeta introduces two core innovations: (1) a Meta-Semantic Prompt (MSP) pool that learns and consolidates high-level meta-semantics shared across tasks, enabling effective knowledge transfer and adaptation to newly emerging relations; and (2) a learnable fusion mechanism that dynamically combines meta-semantics with task-specific relational information tailored to different few-shot tasks. Both components are optimized jointly with model parameters within a meta-learning framework. Extensive experiments and analyses on two real-world KG benchmarks validate the effectiveness of PromptMeta in adapting to new relations with limited supervision.

Method: The authors map global LLM beliefs to numerical scores via statistically reliable abstractions, enabling off-the-shelf global XAI. They hard-code bias-inducing nonlinear heuristics into popular LLMs (ChatGPT and Llama) via system instructions to obtain ground truth, and propose RuleSHAP which couples global SHAP-value aggregations with rule induction.

Result: RuleFit under-detects non-univariate biases, while global SHAP better approximates conjunctive ones but doesn’t yield actionable rules. RuleSHAP improves heuristics detection over RuleFit by +94% (MRR@1) on average.

Conclusion: The results provide a practical pathway for revealing belief-driven biases in LLMs, helping address misinformation amplification concerns.

Abstract: Large language models (LLMs) can amplify misinformation, undermining societal goals like the UN SDGs. We study three documented drivers of misinformation (valence framing, information overload, and oversimplification) which are often shaped by one’s default beliefs. Building on evidence that LLMs encode such defaults (e.g., “joy is positive,” “math is complex”) and can act as “bags of heuristics,” we ask: can general belief-driven heuristics behind misinformative behaviour be recovered from LLMs as clear rules? A key obstacle is that global rule-extraction methods in explainable AI (XAI) are built for numerical inputs/outputs, not text. We address this by eliciting global LLM beliefs and mapping them to numerical scores via statistically reliable abstractions, thereby enabling off-the-shelf global XAI to detect belief-related heuristics in LLMs. To obtain ground truth, we hard-code bias-inducing nonlinear heuristics of increasing complexity (univariate, conjunctive, nonconvex) into popular LLMs (ChatGPT and Llama) via system instructions. This way, we find that RuleFit under-detects non-univariate biases, while global SHAP better approximates conjunctive ones but does not yield actionable rules. To bridge this gap, we propose RuleSHAP, a rule-extraction algorithm that couples global SHAP-value aggregations with rule induction to better capture non-univariate bias, improving heuristics detection over RuleFit by +94% (MRR@1) on average. Our results provide a practical pathway for revealing belief-driven biases in LLMs.

Method: The method leverages machine unlearning by removing target data from a pretrained model and measuring performance shifts on a test set, then computes Shapley values via Monte Carlo sampling without retraining.

Result: Experiments show the approach matches state-of-the-art accuracy while reducing computational overhead by orders of magnitude, with strong correlation between estimated values and true data impact.

Conclusion: This work bridges theory and practice in data valuation, offering a scalable, privacy-compliant solution suitable for large models and data markets.

Abstract: The proliferation of large models has intensified the need for efficient data valuation methods to quantify the contribution of individual data providers. Traditional approaches, such as game-theory-based Shapley value and influence-function-based techniques, face prohibitive computational costs or require access to full data and model training details, making them hardly achieve partial data valuation. To address this, we propose Unlearning Shapley, a novel framework that leverages machine unlearning to estimate data values efficiently. By unlearning target data from a pretrained model and measuring performance shifts on a reachable test set, our method computes Shapley values via Monte Carlo sampling, avoiding retraining and eliminating dependence on full data. Crucially, Unlearning Shapley supports both full and partial data valuation, making it scalable for large models (e.g., LLMs) and practical for data markets. Experiments on benchmark datasets and large-scale text corpora demonstrate that our approach matches the accuracy of state-of-the-art methods while reducing computational overhead by orders of magnitude. Further analysis confirms a strong correlation between estimated values and the true impact of data subsets, validating its reliability in real-world scenarios. This work bridges the gap between data valuation theory and practical deployment, offering a scalable, privacy-compliant solution for modern AI ecosystems.

Method: The paper introduces a model merging benchmark for MLLMs covering multiple tasks (VQA, Geometry, Chart, OCR, Grounding) and implements 10 merging algorithms. It proposes a novel method that removes noise from task vectors and optimizes the merged vector based on loss defined over task vector interactions.

Result: The proposed method achieves an average performance gain of 2.48%. Results show that model merging effectively builds improved MLLMs without training data, and complementarity among multiple modalities outperforms individual modalities.

Conclusion: Model merging offers a promising approach for building improved MLLMs without requiring training data, and the complementarity among multiple modalities leads to better performance than individual modalities.

Abstract: Foundation models update slowly due to resource-intensive training, whereas domain-specific models evolve rapidly between releases. Model merging seeks to combine multiple expert models into a single, more capable model, reducing storage and serving costs while supporting decentralized development. Despite its potential, previous studies have primarily focused on merging visual classification models or Large Language Models (LLMs) for code and math tasks. Recently, Multimodal LLMs (MLLMs) that extend LLMs through large-scale multimodal training have gained traction. However, there lacks a benchmark for model merging research that clearly divides the tasks for MLLM training and evaluation. In this paper, $\textbf{(i)}$ we introduce a model merging benchmark for MLLMs, which includes multiple tasks such as VQA, Geometry, Chart, OCR, and Grounding, studying both LoRA and full fine-tuning models. Moreover, we explore how model merging can combine different modalities (e.g., vision-language, audio-language, and video-language models), moving toward the Omni-language model. $\textbf{(ii)}$ We implement 10 model merging algorithms on the benchmark. Furthermore, we propose a novel method that removes noise from task vectors and robustly optimizes the merged vector based on a loss defined over task vector interactions, achieving an average performance gain of 2.48%. $\textbf{(iii)}$ We find that model merging offers a promising way for building improved MLLMs without requiring training data. Our results also demonstrate that the complementarity among multiple modalities outperforms individual modalities.

Method: Two-stage framework: 1) Context-aware mixture modeling to capture diverse preferences from binary datasets without fine-grained annotations, 2) Online routing strategy that dynamically adapts mixture weights based on context to resolve ambiguity with minimal supervision.

Result: Experiments on multiple preference datasets show MiCRo effectively captures diverse human preferences and significantly improves downstream personalization compared to existing approaches.

Conclusion: MiCRo provides an efficient and scalable solution for personalized preference learning that doesn’t require costly fine-grained annotations, enabling better adaptation to diverse human values in LLM alignment.

Abstract: Reward modeling is a key step in building safe foundation models when applying reinforcement learning from human feedback (RLHF) to align Large Language Models (LLMs). However, reward modeling based on the Bradley-Terry (BT) model assumes a global reward function, failing to capture the inherently diverse and heterogeneous human preferences. Hence, such oversimplification limits LLMs from supporting personalization and pluralistic alignment. Theoretically, we show that when human preferences follow a mixture distribution of diverse subgroups, a single BT model has an irreducible error. While existing solutions, such as multi-objective learning with fine-grained annotations, help address this issue, they are costly and constrained by predefined attributes, failing to fully capture the richness of human values. In this work, we introduce MiCRo, a two-stage framework that enhances personalized preference learning by leveraging large-scale binary preference datasets without requiring explicit fine-grained annotations. In the first stage, MiCRo introduces context-aware mixture modeling approach to capture diverse human preferences. In the second stage, MiCRo integrates an online routing strategy that dynamically adapts mixture weights based on specific context to resolve ambiguity, allowing for efficient and scalable preference adaptation with minimal additional supervision. Experiments on multiple preference datasets demonstrate that MiCRo effectively captures diverse human preferences and significantly improves downstream personalization.

Method: RAG+ constructs a dual corpus with knowledge and application examples, retrieves both jointly during inference, and enables structured, goal-oriented reasoning processes.

Result: Experiments across mathematical, legal, and medical domains show RAG+ consistently outperforms standard RAG variants with average improvements of 3-5% and peak gains up to 13.5%.

Conclusion: RAG+ bridges retrieval with actionable application, advancing a more cognitively grounded framework for knowledge integration toward more interpretable and capable LLMs.

Abstract: The integration of external knowledge through Retrieval-Augmented Generation (RAG) has become foundational in enhancing large language models (LLMs) for knowledge-intensive tasks. However, existing RAG paradigms often overlook the cognitive step of applying knowledge, leaving a gap between retrieved facts and task-specific reasoning. In this work, we introduce RAG+, a principled and modular extension that explicitly incorporates application-aware reasoning into the RAG pipeline. RAG+ constructs a dual corpus consisting of knowledge and aligned application examples, created either manually or automatically, and retrieves both jointly during inference. This design enables LLMs not only to access relevant information but also to apply it within structured, goal-oriented reasoning processes. Experiments across mathematical, legal, and medical domains, conducted on multiple models, demonstrate that RAG+ consistently outperforms standard RAG variants, achieving average improvements of 3-5%, and peak gains up to 13.5% in complex scenarios. By bridging retrieval with actionable application, RAG+ advances a more cognitively grounded framework for knowledge integration, representing a step toward more interpretable and capable LLMs.

Method: Uses an LLM actor for action selection and an LLM critic that analyzes trajectories to generate LTL constraints that shield the actor from unsafe/inefficient behavior. Formalizes planning as graph traversal under symbolic constraints.

Result: On Behavior benchmark (100 household tasks): 25% increase in task completion over InnerMonologue. On Minecraft diamond-mining: improved efficiency and safety compared to SayCan and InnerMonologue.

Conclusion: Enabling LLMs to supervise each other through temporal logic yields more reliable, efficient, and safe decision-making for embodied agents.

Abstract: Large language models (LLMs) have shown promise in zero-shot and single step reasoning and decision making problems, but in long horizon sequential planning tasks, their errors compound, often leading to unreliable or inefficient behavior. We introduce LogicGuard, a modular actor-critic architecture in which an LLM actor is guided by a trajectory level LLM critic that communicates through Linear Temporal Logic (LTL). Our setup combines the reasoning strengths of language models with the guarantees of formal logic. The actor selects high-level actions from natural language observations, while the critic analyzes full trajectories and proposes new LTL constraints that shield the actor from future unsafe or inefficient behavior. LogicGuard supports both fixed safety rules and adaptive, learned constraints, and is model-agnostic: any LLM-based planner can serve as the actor, with LogicGuard acting as a logic-generating wrapper. We formalize planning as graph traversal under symbolic constraints, allowing LogicGuard to analyze failed or suboptimal trajectories and generate new temporal logic rules that improve future behavior. To demonstrate generality, we evaluate LogicGuard across two distinct settings: short-horizon general tasks and long-horizon specialist tasks. On the Behavior benchmark of 100 household tasks, LogicGuard increases task completion rates by 25% over a baseline InnerMonologue planner. On the Minecraft diamond-mining task, which is long-horizon and requires multiple interdependent subgoals, LogicGuard improves both efficiency and safety compared to SayCan and InnerMonologue. These results show that enabling LLMs to supervise each other through temporal logic yields more reliable, efficient and safe decision-making for both embodied agents.

Method: EvoAgentX uses a modular 5-layer architecture (basic components, agent, workflow, evolving, evaluation) and integrates TextGrad, AFlow, and MIPRO optimization algorithms to iteratively refine agent prompts, tools, and workflow topologies.

Result: EvoAgentX achieved significant improvements: 7.44% F1 increase on HotPotQA, 10.00% pass@1 improvement on MBPP, 10.00% solve accuracy gain on MATH, and up to 20.00% overall accuracy improvement on GAIA.

Conclusion: EvoAgentX provides an effective unified framework for automated multi-agent workflow optimization that consistently enhances performance across diverse reasoning, coding, and mathematical tasks.

Abstract: Multi-agent systems (MAS) have emerged as a powerful paradigm for orchestrating large language models (LLMs) and specialized tools to collaboratively address complex tasks. However, existing MAS frameworks often require manual workflow configuration and lack native support for dynamic evolution and performance optimization. In addition, many MAS optimization algorithms are not integrated into a unified framework. In this paper, we present EvoAgentX, an open-source platform that automates the generation, execution, and evolutionary optimization of multi-agent workflows. EvoAgentX employs a modular architecture consisting of five core layers: the basic components, agent, workflow, evolving, and evaluation layers. Specifically, within the evolving layer, EvoAgentX integrates three MAS optimization algorithms, TextGrad, AFlow, and MIPRO, to iteratively refine agent prompts, tool configurations, and workflow topologies. We evaluate EvoAgentX on HotPotQA, MBPP, and MATH for multi-hop reasoning, code generation, and mathematical problem solving, respectively, and further assess it on real-world tasks using GAIA. Experimental results show that EvoAgentX consistently achieves significant performance improvements, including a 7.44% increase in HotPotQA F1, a 10.00% improvement in MBPP pass@1, a 10.00% gain in MATH solve accuracy, and an overall accuracy improvement of up to 20.00% on GAIA. The source code is available at: https://github.com/EvoAgentX/EvoAgentX

Method: IMAIA comprises two components: Maps Plus (parses tiled vector/satellite views into grid-aligned representation for language models) and Places AI Smart Assistant (fuses image-place embeddings with geospatial signals for camera-aware place understanding). Uses lightweight multi-agent design for low latency.

Result: Across map-centric QA and camera-to-place grounding tasks, IMAIA improves accuracy and responsiveness over strong baselines while remaining practical for user-facing deployments.

Conclusion: By unifying language, maps, and geospatial cues, IMAIA moves beyond scripted tools toward conversational mapping that is both spatially grounded and broadly usable.

Abstract: Map applications are still largely point-and-click, making it difficult to ask map-centric questions or connect what a camera sees to the surrounding geospatial context with view-conditioned inputs. We introduce IMAIA, an interactive Maps AI Assistant that enables natural-language interaction with both vector (street) maps and satellite imagery, and augments camera inputs with geospatial intelligence to help users understand the world. IMAIA comprises two complementary components. Maps Plus treats the map as first-class context by parsing tiled vector/satellite views into a grid-aligned representation that a language model can query to resolve deictic references (e.g., ``the flower-shaped building next to the park in the top-right’’). Places AI Smart Assistant (PAISA) performs camera-aware place understanding by fusing image–place embeddings with geospatial signals (location, heading, proximity) to ground a scene, surface salient attributes, and generate concise explanations. A lightweight multi-agent design keeps latency low and exposes interpretable intermediate decisions. Across map-centric QA and camera-to-place grounding tasks, IMAIA improves accuracy and responsiveness over strong baselines while remaining practical for user-facing deployments. By unifying language, maps, and geospatial cues, IMAIA moves beyond scripted tools toward conversational mapping that is both spatially grounded and broadly usable.

Method: Leverages Büchi automata structure to decompose LTL specifications into sequences of reach-avoid subgoals, solving them one at a time with safe RL formulations. Introduces subgoal-induced observation reduction to mitigate exponential complexity.

Result: GenZ-LTL substantially outperforms existing methods in zero-shot generalization to unseen LTL specifications.

Conclusion: Sequential subgoal solving with proper safe RL formulations is more effective for zero-shot generalization than conditioning on subgoal sequences, enabling handling of arbitrary LTL specifications.

Abstract: Generalizing to complex and temporally extended task objectives and safety constraints remains a critical challenge in reinforcement learning (RL). Linear temporal logic (LTL) offers a unified formalism to specify such requirements, yet existing methods are limited in their abilities to handle nested long-horizon tasks and safety constraints, and cannot identify situations when a subgoal is not satisfiable and an alternative should be sought. In this paper, we introduce GenZ-LTL, a method that enables zero-shot generalization to arbitrary LTL specifications. GenZ-LTL leverages the structure of B"uchi automata to decompose an LTL task specification into sequences of reach-avoid subgoals. Contrary to the current state-of-the-art method that conditions on subgoal sequences, we show that it is more effective to achieve zero-shot generalization by solving these reach-avoid problems \textit{one subgoal at a time} through proper safe RL formulations. In addition, we introduce a novel subgoal-induced observation reduction technique that can mitigate the exponential complexity of subgoal-state combinations under realistic assumptions. Empirical results show that GenZ-LTL substantially outperforms existing methods in zero-shot generalization to unseen LTL specifications.

Method: Two-stage fine-tuning paradigm: supervised fine-tuning on high-quality reasoning trajectories followed by reinforcement fine-tuning with Rank-Aware Policy Optimization (RAPO), which increases update weights for high-quality trajectories when their probabilities are lower than low-quality ones.

Result: Extensive experiments show TableMind achieves superior performance compared to competitive baselines, with substantial gains in both reasoning accuracy and computational precision on mainstream benchmarks.

Conclusion: TableMind demonstrates that combining autonomous tool invocation, secure code execution, and adaptive planning capabilities through a carefully designed fine-tuning approach can significantly improve table reasoning performance over existing methods.

Abstract: Table reasoning is crucial for leveraging structured data in domains such as finance, healthcare, and scientific research. While large language models (LLMs) show promise in multi-step reasoning, purely text-based methods often struggle with the complex numerical computations and fine-grained operations inherently required in this task. Tool-integrated reasoning improves computational accuracy via explicit code execution, yet existing systems frequently rely on rigid patterns, supervised imitation, and lack true autonomous adaptability. In this paper, we present TableMind, an LLM-driven table reasoning agent that (i) autonomously performs multi-turn tool invocation, (ii) writes and executes data-analyzing code in a secure sandbox environment for data analysis and precise numerical reasoning, and (iii) exhibits high-level capabilities such as planning and self-reflection to adapt strategies. To realize these capabilities, we adopt a two-stage fine-tuning paradigm built on top of a powerful pre-trained language model: supervised fine-tuning on high-quality reasoning trajectories to establish effective tool usage patterns, followed by reinforcement fine-tuning to optimize multi-objective strategies. In particular, we propose Rank-Aware Policy Optimization (RAPO), which increases the update weight of high-quality trajectories when their output probabilities are lower than those of low-quality ones, thereby guiding the model more consistently toward better and more accurate answers. Extensive experiments on several mainstream benchmarks demonstrate that TableMind achieves superior performance compared to competitive baselines, yielding substantial gains in both reasoning accuracy and computational precision.

Method: Abduct-Act-Predict (A2P) Scaffolding guides LLMs through a three-step reasoning process: (1) Abduction to infer hidden root causes, (2) Action to define minimal corrective intervention, and (3) Prediction to simulate subsequent trajectory and verify if intervention resolves failure.

Result: On Algorithm-Generated dataset: 47.46% step-level accuracy (2.85x improvement over 16.67% baseline). On Hand-Crafted dataset: 29.31% step accuracy (2.43x improvement over 12.07% baseline).

Conclusion: By reframing failure attribution through a causal lens, A2P Scaffolding provides a robust, verifiable, and significantly more accurate solution for automated failure attribution in multi-agent systems.

Abstract: Failure attribution in multi-agent systems – pinpointing the exact step where a decisive error occurs – is a critical yet unsolved challenge. Current methods treat this as a pattern recognition task over long conversation logs, leading to critically low step-level accuracy (below 17%), which renders them impractical for debugging complex systems. Their core weakness is a fundamental inability to perform robust counterfactual reasoning: to determine if correcting a single action would have actually averted the task failure. To bridge this \emph{counterfactual inference gap}, we introduce Abduct-Act-Predict (A2P) Scaffolding, a novel agent framework that transforms failure attribution from pattern recognition into a structured causal inference task. A2P explicitly guides a large language model through a formal three-step reasoning process within a single inference pass: (1) Abduction, to infer the hidden root causes behind an agent’s actions; (2) Action, to define a minimal corrective intervention; and (3) Prediction, to simulate the subsequent trajectory and verify if the intervention resolves the failure. This structured approach leverages the holistic context of the entire conversation while imposing a rigorous causal logic on the model’s analysis. Our extensive experiments on the Who&When benchmark demonstrate its efficacy. On the Algorithm-Generated dataset, A2P achieves 47.46% step-level accuracy, a 2.85$\times$ improvement over the 16.67% of the baseline. On the more complex Hand-Crafted dataset, it achieves 29.31% step accuracy, a 2.43$\times$ improvement over the baseline’s 12.07%. By reframing the problem through a causal lens, A2P Scaffolding provides a robust, verifiable, and significantly more accurate solution for automated failure attribution. Ours code are released at https://github.com/ResearAI/A2P.

Method: DAAO uses three modules: a VAE for difficulty estimation, a modular operator allocator, and a cost-performance-aware LLM router to dynamically tailor workflows based on query difficulty.

Result: DAAO outperforms prior multi-agent systems in both accuracy and inference efficiency across six benchmarks.

Conclusion: The proposed dynamic orchestration framework enables fine-grained, query-specific reasoning strategies by leveraging heterogeneous LLMs and adaptive workflows.

Abstract: Large Language Model (LLM)-based agentic systems have shown strong capabilities across various tasks. However, existing multi-agent frameworks often rely on static or task-level workflows, which either over-process simple queries or underperform on complex ones, while also neglecting the efficiency-performance trade-offs across heterogeneous LLMs. To address these limitations, we propose Difficulty-Aware Agentic Orchestration (DAAO), a dynamic framework that adapts workflow depth, operator selection, and LLM assignment based on the difficulty of each input query. DAAO comprises three interdependent modules: a variational autoencoder (VAE) for difficulty estimation, a modular operator allocator, and a cost- and performance-aware LLM router. By leveraging heterogeneous LLMs and dynamically tailoring workflows, DAAO enables fine-grained, query-specific reasoning strategies. DAAO outperforms prior multi-agent systems in both accuracy and inference efficiency across six benchmarks. We will release our code and implementation details upon publication.

Method: Novel framework treating synthesis as active learning over finite hypothesis class, using LLM to predict outputs for selected test inputs and eliminate inconsistent hypotheses via greedy maximin algorithm.

Result: Significant improvements in program synthesis accuracy and efficiency across four benchmarks: Playgol, MBPP+, 1D-ARC, and programmatic world modeling on MiniGrid.

Conclusion: Transductive program synthesis effectively enhances robustness and efficiency in program synthesis tasks, particularly in real-world settings with limited training data.

Abstract: We introduce transductive program synthesis, a new formulation of the program synthesis task that explicitly leverages test inputs during synthesis. While prior approaches to program synthesis–whether based on natural language descriptions or input-output examples–typically aim to generalize from training examples, they often struggle with robustness, especially in real-world settings where training examples are limited and test inputs involve various edge cases. To address this, we propose a novel framework that improves robustness by treating synthesis as an active learning over a finite hypothesis class defined by programs’ outputs. We use an LLM to predict outputs for selected test inputs and eliminate inconsistent hypotheses, where the inputs are chosen via a greedy maximin algorithm to minimize the number of LLM queries required. We evaluate our approach on four benchmarks: Playgol, MBPP+, 1D-ARC, and programmatic world modeling on MiniGrid. We demonstrate that our method significantly improves program synthesis in both accuracy and efficiency. We release our code at https://github.com/klee972/SYNTRA.

Method: Proposes an architecture combining orthophotos, Sentinel-2 vegetation indices, and user documents through retrieval-augmented generation (RAG), using LLM-as-a-judge methodology with Qwen3-32B for evaluation.

Result: Preliminary results show the system generates clear, relevant, and context-aware responses to agricultural queries while remaining reproducible and scalable across geographic regions.

Conclusion: The work contributes an architecture for fusing multimodal EO and textual knowledge, demonstrates lowering barriers to agricultural information access through natural language, and provides an open, reproducible design.

Abstract: The increasing availability of open Earth Observation (EO) and agricultural datasets holds great potential for supporting sustainable land management. However, their high technical entry barrier limits accessibility for non-expert users. This study presents an open-source conversational assistant that integrates multimodal retrieval and large language models (LLMs) to enable natural language interaction with heterogeneous agricultural and geospatial data. The proposed architecture combines orthophotos, Sentinel-2 vegetation indices, and user-provided documents through retrieval-augmented generation (RAG), allowing the system to flexibly determine whether to rely on multimodal evidence, textual knowledge, or both in formulating an answer. To assess response quality, we adopt an LLM-as-a-judge methodology using Qwen3-32B in a zero-shot, unsupervised setting, applying direct scoring in a multi-dimensional quantitative evaluation framework. Preliminary results show that the system is capable of generating clear, relevant, and context-aware responses to agricultural queries, while remaining reproducible and scalable across geographic regions. The primary contributions of this work include an architecture for fusing multimodal EO and textual knowledge sources, a demonstration of lowering the barrier to access specialized agricultural information through natural language interaction, and an open and reproducible design.

Method: The authors adapt an existing soft arc consistency (SAC) algorithm to handle linear constraints as local cost functions, increasing modeling expressiveness while maintaining the benefits of SAC approaches.

Result: The adapted algorithm significantly improves lower bounds compared to the original SAC algorithm on several benchmarks, and reduces solving time in some cases.

Conclusion: Incorporating linear constraints into SAC algorithms provides a practical middle ground between weak bounds of soft global constraints and the computational burden of full linear program reformulations, offering improved performance for constraint programming problems.

Abstract: In Constraint Programming, solving discrete minimization problems with hard and soft constraints can be done either using (i) soft global constraints, (ii) a reformulation into a linear program, or (iii) a reformulation into local cost functions. Approach (i) benefits from a vast catalog of constraints. Each soft constraint propagator communicates with other soft constraints only through the variable domains, resulting in weak lower bounds. Conversely, the approach (ii) provides a global view with strong bounds, but the size of the reformulation can be problematic. We focus on approach (iii) in which soft arc consistency (SAC) algorithms produce bounds of intermediate quality. Recently, the introduction of linear constraints as local cost functions increases their modeling expressiveness. We adapt an existing SAC algorithm to handle linear constraints. We show that our algorithm significantly improves the lower bounds compared to the original algorithm on several benchmarks, reducing solving time in some cases.

Method: TTS-Uniform framework: (i) identifies potential reasoning strategies, (ii) uniformly allocates sampling budget across strategies, and (iii) filters out unstable strategies before aggregation.

Result: Experimental results show TTS-Uniform significantly enhances scaling effectiveness across multiple mainstream LLMs and benchmark datasets.

Conclusion: The proposed TTS-Uniform framework successfully mitigates selection bias in test-time scaling, leading to improved performance by ensuring more comprehensive exploration of reasoning strategies.

Abstract: Test-time scaling (TTS) has been shown to improve the performance of large language models (LLMs) by sampling and aggregating diverse reasoning paths. However, existing research has overlooked a critical issue: selection bias of reasoning strategies during scaling. Specifically, when generating reasoning processes, LLMs tend to follow certain strategies (e.g., algebraic solutions for math problems) while neglecting other valid alternatives (e.g., geometric solutions), resulting in insufficient exploration of the solution space. To further understand the impact of this bias, we present a theoretical analysis that reveals when it undermines the effectiveness of test-time scaling. Motivated by this theoretical insight, we introduce TTS-Uniform, a framework designed to mitigate the selection bias of reasoning strategies. It (i) identifies potential strategies, (ii) uniformly allocates the sampling budget across them, and (iii) filters out unstable strategies prior to aggregation. Experimental results show that TTS-Uniform significantly enhances scaling effectiveness across multiple mainstream LLMs and benchmark datasets.

Method: The authors explored AASIST, HM-Conformer, Hubert, and Wav2vec2 models with various input features and loss functions. They trained self-supervised models on spoofing datasets as feature extractors and applied adaptive multi-scale feature fusion (AMFF) to integrate features from multiple Transformer layers with hand-crafted features. They also experimented with one-class loss functions.

Result: The fusion system achieved minDCF of 0.4783 and EER of 20.45% in closed condition, and minDCF of 0.2245 and EER of 9.36% in open condition.

Conclusion: The proposed approach demonstrates effective deepfake detection capabilities through multi-scale feature integration and optimized loss functions, showing significant performance improvements in both closed and open conditions.

Abstract: In this paper, we present our submitted XMUspeech systems to the speech deepfake detection track of the ASVspoof 5 Challenge. Compared to previous challenges, the audio duration in ASVspoof 5 database has significantly increased. And we observed that merely adjusting the input audio length can substantially improve system performance. To capture artifacts at multiple levels, we explored the performance of AASIST, HM-Conformer, Hubert, and Wav2vec2 with various input features and loss functions. Specifically, in order to obtain artifact-related information, we trained self-supervised models on the dataset containing spoofing utterances as the feature extractors. And we applied an adaptive multi-scale feature fusion (AMFF) method to integrate features from multiple Transformer layers with the hand-crafted feature to enhance the detection capability. In addition, we conducted extensive experiments on one-class loss functions and provided optimized configurations to better align with the anti-spoofing task. Our fusion system achieved a minDCF of 0.4783 and an EER of 20.45% in the closed condition, and a minDCF of 0.2245 and an EER of 9.36% in the open condition.

Method: Created MNV-17 dataset with performative speech to ensure high-fidelity NV instances. Contains 17 distinct, well-balanced NV categories. Benchmarked on four mainstream ASR architectures for joint semantic transcription and NV classification.

Result: The dataset provides extensive NV coverage and will be made publicly available along with pretrained model checkpoints.

Conclusion: MNV-17 addresses the critical gap in NV-aware ASR research and will facilitate future work in expressive speech recognition.

Abstract: Mainstream Automatic Speech Recognition (ASR) systems excel at transcribing lexical content, but largely fail to recognize nonverbal vocalizations (NVs) embedded in speech, such as sighs, laughs, and coughs. This capability is important for a comprehensive understanding of human communication, as NVs convey crucial emotional and intentional cues. Progress in NV-aware ASR has been hindered by the lack of high-quality, well-annotated datasets. To address this gap, we introduce MNV-17, a 7.55-hour performative Mandarin speech dataset. Unlike most existing corpora that rely on model-based detection, MNV-17’s performative nature ensures high-fidelity, clearly articulated NV instances. To the best of our knowledge, MNV-17 provides the most extensive set of nonverbal vocalization categories, comprising 17 distinct and well-balanced classes of common NVs. We benchmarked MNV-17 on four mainstream ASR architectures, evaluating their joint performance on semantic transcription and NV classification. The dataset and the pretrained model checkpoints will be made publicly available to facilitate future research in expressive ASR.

Method: Developed a base stereo audio generation model, then created a synthetic data pipeline combining video analysis, object tracking, and audio synthesis with dynamic panning and distance-based loudness controls. Fine-tuned the base model on this synthetic dataset for object-audio correspondence.

Result: Achieved state-of-the-art semantic accuracy and synchronization. Introduced stereo object-awareness measures and validated through human listening studies showing strong correlation with perception.

Conclusion: Establishes the first end-to-end framework for stereo object-aware video-to-audio generation, addressing a critical gap and setting a new benchmark in the field.

Abstract: We present StereoFoley, a video-to-audio generation framework that produces semantically aligned, temporally synchronized, and spatially accurate stereo sound at 48 kHz. While recent generative video-to-audio models achieve strong semantic and temporal fidelity, they largely remain limited to mono or fail to deliver object-aware stereo imaging, constrained by the lack of professionally mixed, spatially accurate video-to-audio datasets. First, we develop and train a base model that generates stereo audio from video, achieving state-of-the-art in both semantic accuracy and synchronization. Next, to overcome dataset limitations, we introduce a synthetic data generation pipeline that combines video analysis, object tracking, and audio synthesis with dynamic panning and distance-based loudness controls, enabling spatially accurate object-aware sound. Finally, we fine-tune the base model on this synthetic dataset, yielding clear object-audio correspondence. Since no established metrics exist, we introduce stereo object-awareness measures and validate it through a human listening study, showing strong correlation with perception. This work establishes the first end-to-end framework for stereo object-aware video-to-audio generation, addressing a critical gap and setting a new benchmark in the field.

Method: Uses an adjusted Siamese Network trained on ordinal and numeric distance between input sample pairs rather than binary similarity, applied to a two-dimensional arousal-valence space.

Result: Model reduces Turn-around Percentage by up to 50% on the challenging valence dimension compared to regression baseline, with qualitative validation showing semantically meaningful learned space.

Conclusion: Establishes proof-of-concept for analyzing canine barking under severe data limitations using continuous regression in arousal-valence space.

Abstract: Standard classification of canine vocalisations is severely limited for assistance dogs, where sample data is sparse and variable across dogs and where capture of the full range of bark types is ethically constrained. We reframe this problem as a continuous regression task within a two-dimensional arousal-valence space. Central to our approach is an adjusted Siamese Network trained not on binary similarity, but on the ordinal and numeric distance between input sample pairs. Trained on a public dataset, our model reduces Turn-around Percentage by up to 50% on the challenging valence dimension compared to a regression baseline. Qualitative validation on a real-world dataset confirms the learned space is semantically meaningful, establishing a proof-of-concept for analysing canine barking under severe data limitations.

Method: Three-step approach: 1) Detect syllable events, 2) Cluster syllables to extract templates, 3) Use matching pursuit to decompose recordings into syllable sequences.

Result: Achieved high performance on Bengalese finch songs compared to human labels. Successfully distinguished individual birds within species (Bengalese finches and great tits) through their unique vocal signatures.

Conclusion: The unsupervised method provides a scalable solution for birdsong analysis across species without requiring labeled training data, enabling broader applications in animal communication research and individual identification.

Abstract: Identifying sequences of syllables within birdsongs is key to tackling a wide array of challenges, including bird individual identification and better understanding of animal communication and sensory-motor learning. Recently, machine learning approaches have demonstrated great potential to alleviate the need for experts to label long audio recordings by hand. However, they still typically rely on the availability of labelled data for model training, restricting applicability to a few species and datasets. In this work, we build the first fully unsupervised algorithm to decompose birdsong recordings into sequences of syllables. We first detect syllable events, then cluster them to extract templates –syllable representations– before performing matching pursuit to decompose the recording as a sequence of syllables. We evaluate our automatic annotations against human labels on a dataset of Bengalese finch songs and find that our unsupervised method achieves high performance. We also demonstrate that our approach can distinguish individual birds within a species through their unique vocal signatures, for both Bengalese finches and another species, the great tit.

Method: The proposed Scattering Transformer leverages standard wavelet scattering networks by introducing contextual dependencies in a transformer-like architecture without any backpropagation or training.

Result: The method achieves a Weighted Accuracy (WAR) of 0.786 and Unweighted Average Recall (UAR) of 0.697 on the CirCor DigiScope dataset, performing competitively with state-of-the-art methods.

Conclusion: The Scattering Transformer establishes itself as a viable and promising alternative for heart murmur detection in resource-constrained setups, offering training-free operation with competitive performance.

Abstract: In an attempt to address the need for skilled clinicians in heart sound interpretation, recent research efforts on automating cardiac auscultation have explored deep learning approaches. The majority of these approaches have been based on supervised learning that is always challenged in occasions where training data is limited. More recently, there has been a growing interest in potentials of pre-trained self-supervised audio foundation models for biomedical end tasks. Despite exhibiting promising results, these foundational models are typically computationally intensive. Within the context of automatic cardiac auscultation, this study explores a lightweight alternative to these general-purpose audio foundation models by introducing the Scattering Transformer, a novel, training-free transformer architecture for heart murmur detection. The proposed method leverages standard wavelet scattering networks by introducing contextual dependencies in a transformer-like architecture without any backpropagation. We evaluate our approach on the public CirCor DigiScope dataset, directly comparing it against leading general-purpose foundational models. The Scattering Transformer achieves a Weighted Accuracy(WAR) of 0.786 and an Unweighted Average Recall(UAR) of 0.697, demonstrating performance highly competitive with contemporary state of the art methods. This study establishes the Scattering Transformer as a viable and promising alternative in resource-constrained setups.

Method: Proposed lightweight RL framework for audio LLMs; evaluated GRPO with rule-based rewards for ASR, compared GRPO and DiffRO for TTS, and combined approaches.

Result: RL significantly enhances both ASR and TTS system performance even with limited training data and few optimization steps.

Conclusion: The proposed RL framework effectively improves audio-based LLM performance, demonstrating RL’s potential for ASR and TTS applications.

Abstract: In recent years, large language models (LLMs) have played an important role in automatic speech recognition (ASR) and text-to-speech (TTS) systems. While reinforcement learning (RL) has significantly enhanced LLM performance in text-based tasks, its application to ASR and TTS remains underexplored due to the complexity of training audio-based models. In this study, we propose a lightweight RL framework tailored for audio-based LLMs that can process audio inputs and generate audio outputs. Based on this framework, we evaluate the effectiveness of reinforcement learning on both ASR and TTS tasks. For the ASR task, we experiment with different rule-based reward functions within the Group Relative Policy Optimization (GRPO) framework and investigate the impact of RL data construction. For the TTS task, we compare GRPO with Differentiable Reward Optimization (DiffRO) and further combine the two approaches to achieve improved performance. Our experiments demonstrate that RL can significantly enhance the performance of both ASR and TTS systems, even with limited training data and a small number of optimization steps.

Method: MATA intervenes post raw attention scoring in self-attention mechanisms, specifically targeting the last token in intermediate layers to push models to pay more attention to audio tokens dynamically.

Result: Experiments on MMAU and MMAR benchmarks show consistent performance gains, with an open-source model surpassing proprietary Gemini 2.0 Flash on MMAR for the first time.

Conclusion: MATA provides an efficient solution to mitigate attention bias and opens new research directions for enhancing audio-processing capabilities in multi-modal models.

Abstract: Large Audio-Language Models (LALMs) often suffer from audio-textual attention imbalance, prioritizing text over acoustic information, particularly in the multi-modal fusion layers of the Transformer architecture. This bias hinders their ability to fully utilize acoustic cues, causing suboptimal performance on audio reasoning tasks. To mitigate this, we propose \textbf{MATA}, a novel training-free method that dynamically pushes LALMs to pay \textbf{M}ore \textbf{A}ttention \textbf{T}o \textbf{A}udio tokens within the self-attention mechanism. Specifically, MATA intervenes post raw attention scoring, targeting only the last token in intermediate layers without introducing additional parameters or computational overhead. Experiments on the MMAU and MMAR benchmarks confirm MATA’s effectiveness, with consistent performance gains. Notably, on MMAR, MATA enables an open-source model to surpass the proprietary Gemini 2.0 Flash for the first time. Our work provides an efficient solution to mitigate attention bias and opens a new research direction for enhancing the audio-processing capabilities of multi-modal models.

Method: Train Rectified Flow models on embeddings from pre-trained neural audio fingerprinting systems to generate synthetic fingerprints that act as realistic distractors for large-scale retrieval simulation.

Result: Synthetic fingerprints closely approximate real data distributions, and scaling trends with synthetic distractors track those with real distractors. The method enables modeling of retrieval performance for very large databases.

Conclusion: This approach provides a practical metric for system scalability that doesn’t depend on audio corpora access, enabling realistic large-scale evaluation of audio fingerprinting frameworks.

Abstract: The evaluation of audio fingerprinting at a realistic scale is limited by the scarcity of large public music databases. We present an audio-free approach that synthesises latent fingerprints which approximate the distribution of real fingerprints. Our method trains a Rectified Flow model on embeddings extracted by pre-trained neural audio fingerprinting systems. The synthetic fingerprints generated using our system act as realistic distractors and enable the simulation of retrieval performance at a large scale without requiring additional audio. We assess the fidelity of synthetic fingerprints by comparing the distributions to real data. We further benchmark the retrieval performances across multiple state-of-the-art audio fingerprinting frameworks by augmenting real reference databases with synthetic distractors, and show that the scaling trends obtained with synthetic distractors closely track those obtained with real distractors. Finally, we scale the synthetic distractor database to model retrieval performance for very large databases, providing a practical metric of system scalability that does not depend on access to audio corpora.

Method: The authors present a unified, reproducible framework to compare Transformers, Mamba, and xLSTM based masked spectrogram models on ten diverse downstream audio classification tasks.

Result: The paper provides comparative analysis that helps readers make informed decisions about the suitability of different approaches for audio applications.

Conclusion: This overview fills the research gap by systematically evaluating and comparing state-of-the-art sequence modeling architectures for masked spectrogram modeling in audio applications.

Abstract: In recent years, self-supervised learning has amassed significant interest for training deep neural representations without labeled data. One such self-supervised learning approach is masked spectrogram modeling, where the objective is to learn semantically rich contextual representations by predicting removed or hidden portions of the input audio spectrogram. With the Transformer neural architecture at its core, masked spectrogram modeling has emerged as the prominent approach for learning general purpose audio representations, a.k.a. audio foundation models. Meanwhile, addressing the issues of the Transformer architecture, in particular the underlying Scaled Dot-product Attention operation, which scales quadratically with input sequence length, has led to renewed interest in recurrent sequence modeling approaches. Among them, Selective structured state space models (such as Mamba) and extended Long Short-Term Memory (xLSTM) are the two most promising approaches which have experienced widespread adoption. While the body of work on these two topics continues to grow, there is currently a lack of an adequate overview encompassing the intersection of these topics. In this paper, we present a comprehensive overview of the aforementioned research domains, covering masked spectrogram modeling and the previously mentioned neural sequence modeling architectures, Mamba and xLSTM. Further, we compare Transformers, Mamba and xLSTM based masked spectrogram models in a unified, reproducible framework on ten diverse downstream audio classification tasks, which will help interested readers to make informed decisions regarding suitability of the evaluated approaches to adjacent applications.

Method: A novel approach that converts audio-derived musical information into textual representations, using GPT-4o in a 5-stage chain-of-thought framework to systematically analyze, compare, and refine chord recognition results by integrating music-theoretical knowledge.

Result: Experimental evaluation on three datasets shows consistent improvements across multiple metrics, with overall accuracy gains of 1-2.77% on the MIREX metric.

Conclusion: LLMs can effectively function as integrative bridges in MIR pipelines, opening new directions for multi-tool coordination in music information retrieval tasks.

Abstract: Music Information Retrieval (MIR) encompasses a broad range of computational techniques for analyzing and understanding musical content, with recent deep learning advances driving substantial improvements. Building upon these advances, this paper explores how large language models (LLMs) can serve as an integrative bridge to connect and integrate information from multiple MIR tools, with a focus on enhancing automatic chord recognition performance. We present a novel approach that positions text-based LLMs as intelligent coordinators that process and integrate outputs from diverse state-of-the-art MIR tools-including music source separation, key detection, chord recognition, and beat tracking. Our method converts audio-derived musical information into textual representations, enabling LLMs to perform reasoning and correction specifically for chord recognition tasks. We design a 5-stage chain-of-thought framework that allows GPT-4o to systematically analyze, compare, and refine chord recognition results by leveraging music-theoretical knowledge to integrate information across different MIR components. Experimental evaluation on three datasets demonstrates consistent improvements across multiple evaluation metrics, with overall accuracy gains of 1-2.77% on the MIREX metric. Our findings demonstrate that LLMs can effectively function as integrative bridges in MIR pipelines, opening new directions for multi-tool coordination in music information retrieval tasks.

Method: Proposes MECap-R1 with Emo-GRPO (Group Relative Policy Optimization with emotion-aware reward), a reinforcement learning framework that captures emotion and semantic features to handle the dynamic nature of captions.

Result: Experimental results on EmotionTalk dataset show MECap-R1 performs well in generating emotion descriptions and achieves substantial gains in both accuracy and diversity.

Conclusion: The proposed emotion-aware policy with reinforcement learning effectively addresses the limitations of rigid classification methods and provides a novel approach for multimodal emotion captioning.

Abstract: Speech Emotion Captioning (SEC) has emerged as a notable research direction. The inherent complexity of emotional content in human speech makes it challenging for traditional discrete classification methods to provide an adequate representation. Consequently, utilizing natural language to describe speech emotions presents a novel avenue for more effectively capturing and expressing affect. In this paper, we propose MECap-R1, a pioneering emotion-aware policy with reinforcement learning for multimodal emotion captioning. By employing Group Relative Policy Optimization with emotion-aware reward (Emo-GRPO), the framework precisely captures the emotion and semantic features, thereby addressing the shortcomings of rigid rules in handling the dynamic and flexible nature of captions. Experimental results on the EmotionTalk dataset demonstrate that MECap-R1 performs well in generating emotion descriptions and achieves substantial gains in both accuracy and diversity.

Method: A disentangled, style-based text-to-speech reconstruction framework that adapts to children’s voices with SSD, focusing on preserving pitch and prosody while correcting pronunciation errors.

Result: Substantial improvements in lexical accuracy and speaker identity preservation on the STAR dataset; automatic phonetic prediction correlates with human expert annotations (Pearson r=0.63); effective generalization to adult dysarthric speech on TORGO dataset.

Conclusion: Disentangled, style-based TTS reconstruction can provide identity-preserving speech correction across diverse clinical populations, with potential to reduce manual transcription burden in clinical settings.

Abstract: We present ChiReSSD, a speech reconstruction framework that preserves children speaker’s identity while suppressing mispronunciations. Unlike prior approaches trained on healthy adult speech, ChiReSSD adapts to the voices of children with speech sound disorders (SSD), with particular emphasis on pitch and prosody. We evaluate our method on the STAR dataset and report substantial improvements in lexical accuracy and speaker identity preservation. Furthermore, we automatically predict the phonetic content in the original and reconstructed pairs, where the proportion of corrected consonants is comparable to the percentage of correct consonants (PCC), a clinical speech assessment metric. Our experiments show Pearson correlation of 0.63 between automatic and human expert annotations, highlighting the potential to reduce the manual transcription burden. In addition, experiments on the TORGO dataset demonstrate effective generalization for reconstructing adult dysarthric speech. Our results indicate that disentangled, style-based TTS reconstruction can provide identity-preserving speech across diverse clinical populations.

Method: Proposes DeepResonance with multi-way instruction tuning using 4-way aligned data (Music4way datasets), multi-sampled ImageBind embeddings, and a pre-LLM fusion Transformer for enhanced modality fusion before text LLM processing.

Result: Achieves state-of-the-art performances across six music understanding tasks, demonstrating the benefits of auxiliary modalities and the structural superiority of the proposed approach.

Conclusion: The integration of visual and textual music features through multimodal fusion significantly enhances music understanding capabilities, with the model and datasets being open-sourced for community use.

Abstract: Recent advancements in music large language models (LLMs) have significantly improved music understanding tasks, which involve the model’s ability to analyze and interpret various musical elements. These improvements primarily focused on integrating both music and text inputs. However, the potential of incorporating additional modalities such as images, videos and textual music features to enhance music understanding remains unexplored. To bridge this gap, we propose DeepResonance, a multimodal music understanding LLM fine-tuned via multi-way instruction tuning with multi-way aligned music, text, image, and video data. To this end, we construct Music4way-MI2T, Music4way-MV2T, and Music4way-Any2T, three 4-way training and evaluation datasets designed to enable DeepResonance to integrate both visual and textual music feature content. We also introduce multi-sampled ImageBind embeddings and a pre-LLM fusion Transformer to enhance modality fusion prior to input into text LLMs, tailoring for multi-way instruction tuning. Our model achieves state-of-the-art performances across six music understanding tasks, highlighting the benefits of the auxiliary modalities and the structural superiority of DeepResonance. We open-source the codes, models and datasets we constructed: github.com/sony/DeepResonance.

Method: Developed PoolingVQ by combining Vector Quantized Variational Autoencoder (VQVAE) with spatial pooling to compress audio feature sequences via codebook-guided local aggregation, followed by a two-stage co-attention approach for audio-MIDI fusion.

Result: Experimental results on EMOPIA and VGMIDI datasets show state-of-the-art performance, with PoolingVQ providing effective improvement over existing methods.

Conclusion: The proposed multimodal framework successfully reduces audio feature redundancy and achieves superior performance in music emotion analysis, demonstrating the effectiveness of the PoolingVQ approach.

Abstract: Multimodal music emotion analysis leverages both audio and MIDI modalities to enhance performance. While mainstream approaches focus on complex feature extraction networks, we propose that shortening the length of audio sequence features to mitigate redundancy, especially in contrast to MIDI’s compact representation, may effectively boost task performance. To achieve this, we developed PoolingVQ by combining Vector Quantized Variational Autoencoder (VQVAE) with spatial pooling, which directly compresses audio feature sequences through codebook-guided local aggregation to reduce redundancy, then devised a two-stage co-attention approach to fuse audio and MIDI information. Experimental results on the public datasets EMOPIA and VGMIDI demonstrate that our multimodal framework achieves state-of-the-art performance, with PoolingVQ yielding effective improvement. Our proposed metho’s code is available at Anonymous GitHub

Method: Using image-focused machine learning models trained on blocks extracted from different regions of Ulam spirals - specifically comparing regions around 500m integers versus regions below 25m integers.

Result: Models trained on higher magnitude regions (500m) outperformed those trained on lower regions (25m) in pure accuracy terms, indicating more easily learnable order. Precision/recall analysis showed different classification strategies: focusing on prime pattern identification at lower numbers and composite elimination at higher numbers.

Conclusion: Machine learning can detect diminishing noise and increasing regularity in prime distributions at higher orders of magnitude, aligning with number theory conjectures, and shows potential as a new experimental tool for investigating prime patterns for cryptographic applications.

Abstract: Research on the distribution of prime numbers has revealed a dual character: deterministic in definition yet exhibiting statistical behavior reminiscent of random processes. In this paper we show that it is possible to use an image-focused machine learning model to measure the comparative regularity of prime number fields at specific regions of an Ulam spiral. Specifically, we demonstrate that in pure accuracy terms, models trained on blocks extracted from regions of the spiral in the vicinity of 500m outperform models trained on blocks extracted from the region representing integers lower than 25m. This implies existence of more easily learnable order in the former region than in the latter. Moreover, a detailed breakdown of precision and recall scores seem to imply that the model is favouring a different approach to classification in different regions of the spiral, focusing more on identifying prime patterns at lower numbers and more on eliminating composites at higher numbers. This aligns with number theory conjectures suggesting that at higher orders of magnitude we should see diminishing noise in prime number distributions, with averages (density, AP equidistribution) coming to dominate, while local randomness regularises after scaling by log x. Taken together, these findings point toward an interesting possibility: that machine learning can serve as a new experimental instrument for number theory. Notably, the method shows potential 1 for investigating the patterns in strong and weak primes for cryptographic purposes.

Method: Proposes a comprehensive framework with a Wasserstein-based estimator that predicts model performance across unseen data combinations and assesses data-algorithm compatibility. Includes a distributed method for approximating Wasserstein distance without raw data access, leveraging neural scaling law for performance extrapolation.

Result: Extensive experiments across diverse scenarios (label skew, mislabeled, and unlabeled sources) show the approach consistently identifies high-performing data combinations, enabling effective data selection without full-scale training.

Conclusion: The framework paves the way for more reliable FL-based model marketplaces by providing privacy-preserving data valuation and selection capabilities that work across various data heterogeneity scenarios.

Abstract: In the era of Artificial Intelligence (AI), marketplaces have become essential platforms for facilitating the exchange of data products to foster data sharing. Model transactions provide economic solutions in data marketplaces that enhance data reusability and ensure the traceability of data ownership. To establish trustworthy data marketplaces, Federated Learning (FL) has emerged as a promising paradigm to enable collaborative learning across siloed datasets while safeguarding data privacy. However, effective data valuation and selection from heterogeneous sources in the FL setup remain key challenges. This paper introduces a comprehensive framework centered on a Wasserstein-based estimator tailored for FL. The estimator not only predicts model performance across unseen data combinations but also reveals the compatibility between data heterogeneity and FL aggregation algorithms. To ensure privacy, we propose a distributed method to approximate Wasserstein distance without requiring access to raw data. Furthermore, we demonstrate that model performance can be reliably extrapolated under the neural scaling law, enabling effective data selection without full-scale training. Extensive experiments across diverse scenarios, such as label skew, mislabeled, and unlabeled sources, show that our approach consistently identifies high-performing data combinations, paving the way for more reliable FL-based model marketplaces.

Method: Uses single-echelon testbed with AR(1), i.i.d. Gaussian, and heavy-tailed lognormal demand regimes. Compares fully learned Neural ODE against physics-informed UDE that preserves conservation and order-up-to structure while learning residual policy terms.

Result: UDE consistently generalizes better in structured regimes (AR(1) and Gaussian): inventory RMSE drops from 4.92 to 0.26 under AR(1) and from 5.96 to 0.95 under Gaussian. NODE performs better under heavy-tailed lognormal shocks. UDE remains stable with less data while NODE exhibits phase drift.

Conclusion: Enforce structure when noise is light-tailed or temporally correlated; relax structure when extreme events dominate. Provides guidance for hybrid modeling: enforce known structure when conservation laws dominate, relax structure to capture rare events.

Abstract: We study learning of continuous-time inventory dynamics under stochastic demand and quantify when structure helps or hurts forecasting of the bullwhip effect. BULL-ODE compares a fully learned Neural ODE (NODE) that models the entire right-hand side against a physics-informed Universal Differential Equation (UDE) that preserves conservation and order-up-to structure while learning a small residual policy term. Classical supply chain models explain the bullwhip through control/forecasting choices and information sharing, while recent physics-informed and neural differential equation methods blend domain constraints with learned components. It is unclear whether structural bias helps or hinders forecasting under different demand regimes. We address this by using a single-echelon testbed with three demand regimes - AR(1) (autocorrelated), i.i.d. Gaussian, and heavy-tailed lognormal. Training is done on varying fractions of each trajectory, followed by evaluation of multi-step forecasts for inventory I, order rate O, and demand D. Across the structured regimes, UDE consistently generalizes better: with 90% of the training horizon, inventory RMSE drops from 4.92 (NODE) to 0.26 (UDE) under AR(1) and from 5.96 to 0.95 under Gaussian demand. Under heavy-tailed lognormal shocks, the flexibility of NODE is better. These trends persist as train18 ing data shrinks, with NODE exhibiting phase drift in extrapolation while UDE remains stable but underreacts to rare spikes. Our results provide concrete guidance: enforce structure when noise is light-tailed or temporally correlated; relax structure when extreme events dominate. Beyond inventory control, the results offer guidance for hybrid modeling in scientific and engineering systems: enforce known structure when conservation laws and modest noise dominate, and relax structure to capture extremes in settings where rare events drive dynamics.

Method: Proposes neural network surrogate models that capture shared damage mechanisms, allowing models trained on one bridge to be adapted to similar bridges. Validated using real data from two bridges and integrated into a Bayesian inference framework for continuous damage assessment based on modal features.

Result: The transferred model showed high sensitivity to damage location, severity, and extent, demonstrating effective cross-structure knowledge transfer.

Conclusion: This approach enhances real-time monitoring capabilities and enables scalable, generalizable monitoring frameworks that improve resilience at the network level through smart monitoring strategies.

Abstract: The growing use of permanent monitoring systems has increased data availability, offering new opportunities for structural assessment but also posing scalability challenges, especially across large bridge networks. Managing multiple structures requires tracking and comparing long-term behaviour efficiently. To address this, knowledge transfer between similar structures becomes essential. This study proposes a model-based transfer learning approach using neural network surrogate models, enabling a model trained on one bridge to be adapted to another with similar characteristics. These models capture shared damage mechanisms, supporting a scalable and generalizable monitoring framework. The method was validated using real data from two bridges. The transferred model was integrated into a Bayesian inference framework for continuous damage assessment based on modal features from monitoring data. Results showed high sensitivity to damage location, severity, and extent. This approach enhances real-time monitoring and enables cross-structure knowledge transfer, promoting smart monitoring strategies and improved resilience at the network level.

Method: AdaMixT introduces various patches and leverages both General Pre-trained Models (GPM) and Domain-specific Models (DSM) for multi-scale feature extraction. It incorporates a gating network that dynamically allocates weights among different experts to enable adaptive multi-scale fusion.

Result: Comprehensive experiments on eight widely used benchmarks (Weather, Traffic, Electricity, ILI, and four ETT datasets) consistently demonstrate the effectiveness of AdaMixT in real-world scenarios.

Conclusion: The proposed AdaMixT architecture successfully addresses the limitations of existing approaches by providing adaptive multi-scale feature extraction and fusion, leading to improved performance in multivariate time series forecasting.

Abstract: Multivariate time series forecasting involves predicting future values based on historical observations. However, existing approaches primarily rely on predefined single-scale patches or lack effective mechanisms for multi-scale feature fusion. These limitations hinder them from fully capturing the complex patterns inherent in time series, leading to constrained performance and insufficient generalizability. To address these challenges, we propose a novel architecture named Adaptive Weighted Mixture of Multi-Scale Expert Transformers (AdaMixT). Specifically, AdaMixT introduces various patches and leverages both General Pre-trained Models (GPM) and Domain-specific Models (DSM) for multi-scale feature extraction. To accommodate the heterogeneity of temporal features, AdaMixT incorporates a gating network that dynamically allocates weights among different experts, enabling more accurate predictions through adaptive multi-scale fusion. Comprehensive experiments on eight widely used benchmarks, including Weather, Traffic, Electricity, ILI, and four ETT datasets, consistently demonstrate the effectiveness of AdaMixT in real-world scenarios.

Method: EASE uses a modular architecture that orchestrates multiple LLMs in complementary roles (generator, analyst, evaluator) within a feedback loop that includes testing, analysis, and evaluation components.

Result: The framework enables full user control over error handling, analysis, and quality assessment while supporting reproducible algorithmic solution evolution.

Conclusion: EASE offers an effortless approach to algorithmic solution evolution by leveraging LLMs in a structured, transparent framework that can be applied across diverse domains.

Abstract: This paper presents EASE (Effortless Algorithmic Solution Evolution), an open-source and fully modular framework for iterative algorithmic solution generation leveraging large language models (LLMs). EASE integrates generation, testing, analysis, and evaluation into a reproducible feedback loop, giving users full control over error handling, analysis, and quality assessment. Its architecture supports the orchestration of multiple LLMs in complementary roles-such as generator, analyst, and evaluator. By abstracting the complexity of prompt design and model management, EASE provides a transparent and extensible platform for researchers and practitioners to co-design algorithms and other generative solutions across diverse domains.

Method: Preprocessing AIS data (forward/backward filling, outlier removal, segmentation), extracting 31 trajectory features (kinematic, temporal, geospatial, ship-shape), and using Random Forest with SMOTE and stratified 5-fold cross-validation.

Result: Random Forest achieved 92.15% accuracy, with macro-precision 94.11%, macro-recall 92.51%, and macro-F1 93.27%. Bridge-position ratio and maximum SOG were most discriminative features.

Conclusion: Lightweight AIS trajectory features enable real-time vessel classification, with potential improvements through DBSCAN segmentation and gradient-boosted ensembles for better handling of complex vessel behaviors.

Abstract: Accurate recognition of vessel types from Automatic Identification System (AIS) tracks is essential for safety oversight and combating illegal, unreported, and unregulated (IUU) activity. This paper presents a strait-scale, machine-learning pipeline that classifies moving vessels using only AIS data. We analyze eight days of historical AIS from the Danish Maritime Authority covering the Bornholm Strait in the Baltic Sea (January 22-30, 2025). After forward/backward filling voyage records, removing kinematic and geospatial outliers, and segmenting per-MMSI tracks while excluding stationary periods ($\ge 1$ h), we derive 31 trajectory-level features spanning kinematics (e.g., SOG statistics), temporal, geospatial (Haversine distances, spans), and ship-shape attributes computed from AIS A/B/C/D reference points (length, width, aspect ratio, bridge-position ratio). To avoid leakage, we perform grouped train/test splits by MMSI and use stratified 5-fold cross-validation. Across five classes (cargo, tanker, passenger, high-speed craft, fishing; N=1{,}910 trajectories; test=382), tree-based models dominate: a Random Forest with SMOTE attains 92.15% accuracy (macro-precision 94.11%, macro-recall 92.51%, macro-F1 93.27%) on the held-out test set, while a tuned RF reaches one-vs-rest ROC-AUC up to 0.9897. Feature-importance analysis highlights the bridge-position ratio and maximum SOG as the most discriminative signals; principal errors occur between cargo and tanker, reflecting similar transit behavior. We demonstrate operational value by backfilling missing ship types on unseen data and discuss improvements such as DBSCAN based trip segmentation and gradient-boosted ensembles to handle frequent-stop ferries and further lift performance. The results show that lightweight features over AIS trajectories enable real-time vessel type classification in straits.

Method: Two patch-based approaches: (1) local-to-global patch PCA and (2) local-to-local patch PCA, with refinements including overlapping patches with smoothing filter and two-step CNN refinement.

Result: Patch-based PCA significantly reduces computational complexity while maintaining high accuracy, reducing end-to-end pipeline processing time by 3.7-4 times compared to global PCA.

Conclusion: Patch-based PCA is a promising technique for efficient operator learning in PDE-based systems, offering better computational efficiency without sacrificing accuracy.

Abstract: Neural operator learning has emerged as a powerful approach for solving partial differential equations (PDEs) in a data-driven manner. However, applying principal component analysis (PCA) to high-dimensional solution fields incurs significant computational overhead. To address this, we propose a patch-based PCA-Net framework that decomposes the solution fields into smaller patches, applies PCA within each patch, and trains a neural operator in the reduced PCA space. We investigate two different patch-based approaches that balance computational efficiency and reconstruction accuracy: (1) local-to-global patch PCA, and (2) local-to-local patch PCA. The trade-off between computational cost and accuracy is analyzed, highlighting the advantages and limitations of each approach. Furthermore, within each approach, we explore two refinements for the most computationally efficient method: (i) introducing overlapping patches with a smoothing filter and (ii) employing a two-step process with a convolutional neural network (CNN) for refinement. Our results demonstrate that patch-based PCA significantly reduces computational complexity while maintaining high accuracy, reducing end-to-end pipeline processing time by a factor of 3.7 to 4 times compared to global PCA, thefore making it a promising technique for efficient operator learning in PDE-based systems.

Method: Proposes a CoOp-based framework integrating subspace representation learning with prompt tuning, projecting ID features into a subspace spanned by prompt vectors and ID-irrelevant features into an orthogonal null space, trained with an end-to-end learning criterion.

Result: Experiments on real-world datasets demonstrate the effectiveness of the approach.

Conclusion: The proposed framework successfully addresses the limitation of existing methods by leveraging VLM feature embeddings for improved OOD detection performance while maintaining high ID classification accuracy.

Abstract: The reliability of artificial intelligence (AI) systems in open-world settings depends heavily on their ability to flag out-of-distribution (OOD) inputs unseen during training. Recent advances in large-scale vision-language models (VLMs) have enabled promising few-shot OOD detection frameworks using only a handful of in-distribution (ID) samples. However, existing prompt learning-based OOD methods rely solely on softmax probabilities, overlooking the rich discriminative potential of the feature embeddings learned by VLMs trained on millions of samples. To address this limitation, we propose a novel context optimization (CoOp)-based framework that integrates subspace representation learning with prompt tuning. Our approach improves ID-OOD separability by projecting the ID features into a subspace spanned by prompt vectors, while projecting ID-irrelevant features into an orthogonal null space. To train such OOD detection framework, we design an easy-to-handle end-to-end learning criterion that ensures strong OOD detection performance as well as high ID classification accuracy. Experiments on real-world datasets showcase the effectiveness of our approach.

Method: Systematic comparison of time-series foundation models and LLMs against established CTG-specific architectures using over 500 CTG recordings of varying durations reflecting real-world clinical data.

Result: Fine-tuned LLMs achieved superior performance compared to both foundation models and domain-specific approaches for automated CTG analysis.

Conclusion: Fine-tuned LLMs offer a promising alternative for clinical CTG interpretation, providing critical insights for future AI development in prenatal care and establishing performance benchmarks across different modeling paradigms.

Abstract: Foundation models (FMs) and large language models (LLMs) demonstrate remarkable capabilities across diverse domains through training on massive datasets. These models have demonstrated exceptional performance in healthcare applications, yet their potential for electronic fetal monitoring (EFM)/cardiotocography (CTG) analysis, a critical technology for evaluating fetal well-being, remains largely underexplored. Antepartum CTG interpretation presents unique challenges due to the complex nature of fetal heart rate (FHR) patterns and uterine activity, requiring sophisticated analysis of long time-series data. The assessment of CTG is heavily based on subjective clinical interpretation, often leading to variability in diagnostic accuracy and deviation from timely pregnancy care. This study presents the first comprehensive comparison of state-of-the-art AI approaches for automated antepartum CTG analysis. We systematically compare time-series FMs and LLMs against established CTG-specific architectures. Our evaluation encompasses over 500 CTG recordings of varying durations reflecting real-world clinical recordings, providing robust performance benchmarks across different modelling paradigms. Our results demonstrate that fine-tuned LLMs achieve superior performance compared to both foundation models and domain-specific approaches, offering a promising alternative pathway for clinical CTG interpretation. These findings provide critical insights into the relative strengths of different AI methodologies for fetal monitoring applications and establish a foundation for future clinical AI development in prenatal care.

Method: The study leverages a DPU-assisted framework with BlueField-3 Data Processing Units to offload monitoring tasks, analyze GPU telemetry and inter-node communication patterns, and provide actionable feedback to inference controllers and schedulers for real-time load imbalance detection and mitigation.

Result: The framework enables real-time detection and mitigation of load imbalance in multi-node tensor-parallel inference by analyzing GPU telemetry and communication patterns through DPU offloading.

Conclusion: The study aims to identify load imbalances in multi-GPU LLM execution, assess their impact on computational performance, and critically evaluate whether these imbalances can be tracked and mitigated using DPU network assistance.

Abstract: Autoregressive inference in large transformer-based language models (LLMs) presents significant challenges for runtime efficiency, particularly during the decode phase where load imbalance across GPU shards can cause throughput degradation and latency spikes. A DPU-assisted framework leveraged by BlueField-3 Data Processing Units can enable real-time detection and mitigation of load imbalance in multi-node tensor-parallel inference. By offloading monitoring tasks to the DPU and analyzing GPU telemetry and inter-node communication patterns, the resulting system can provide actionable feedback to inference controllers and schedulers. The goal of this study is three-fold i) identify the reported skews/imbalances/pathological conditions that arise in muti-GPU execution of a) LLM tensor computing (both during training and inference), b) identify their impact on computational performance, and c) make a critical assessment if those can be tracked for potential mitigation from a DPU network.

Method: Partitions spatial graph into subgraphs, uses Intra-subgraph Attention for local correlation within subgraphs and Inter-subgraph Attention for global correlation between subgraphs. Builds a multiscale model by progressively increasing subgraph scales.

Result: Achieves performance improvements up to 7.7% over baseline methods on real-world spatiotemporal datasets while maintaining low running costs.

Conclusion: The proposed Spatial Balance Attention block provides a scalable, efficient, and easy-to-implement solution for spatiotemporal forecasting that effectively captures both local and global spatial correlations.

Abstract: In this paper, we propose a novel Spatial Balance Attention block for spatiotemporal forecasting. To strike a balance between obeying spatial proximity and capturing global correlation, we partition the spatial graph into a set of subgraphs and instantiate Intra-subgraph Attention to learn local spatial correlation within each subgraph; to capture the global spatial correlation, we further aggregate the nodes to produce subgraph representations and achieve message passing among the subgraphs via Inter-subgraph Attention. Building on the proposed Spatial Balance Attention block, we develop a multiscale spatiotemporal forecasting model by progressively increasing the subgraph scales. The resulting model is both scalable and able to produce structured spatial correlation, and meanwhile, it is easy to implement. We evaluate its efficacy and efficiency against the existing models on real-world spatiotemporal datasets from medium to large sizes. The experimental results show that it can achieve performance improvements up to 7.7% over the baseline methods at low running costs.

Method: ALS computes the mean difference between hidden states from successful versus unsuccessful generations offline, then uses this direction to calibrate the model’s hidden representations during inference. When decoding drifts from the success manifold, ALS nudges activations back toward it at constant cost.

Result: Across GSM8K and MATH-500 benchmarks, ALS achieves 2-5x speedup over iterative methods while matching or surpassing greedy Chain-of-Thought and Self-Consistency baselines, yielding up to 101% improvement in efficiency-accuracy trade-off.

Conclusion: Much of latent optimization’s benefit can be captured offline, making sophisticated reasoning techniques viable for production deployment by eliminating expensive per-query optimization loops.

Abstract: Test-time optimization remains impractical at scale due to prohibitive inference costs\textemdash techniques like iterative refinement and multi-step verification can require $10$–$100\times$ more compute per query than standard decoding. Latent space test-time optimization methods like LatentSeek offer a more direct approach by steering hidden representations, but still demand expensive per-query optimization loops with multiple backward passes. We propose Amortized Latent Steering (ALS), which collapses this iterative optimization into a single offline-computed vector applied at constant cost during inference. ALS computes the mean difference between hidden states from successful versus unsuccessful generations, then uses this direction to calibrate the model’s hidden representations: when decoding drifts away from the success manifold, ALS nudges activations back toward it. Across GSM8K and MATH-$500$ benchmarks, ALS achieves $2$–$5\times$ speedup over iterative methods while matching or surpassing greedy Chain-of-Thought (CoT) and Self-Consistency baselines, yielding up to 101% improvement in efficiency–accuracy trade-off. These results show that much of latent optimization’s benefit can be captured offline, making sophisticated reasoning techniques viable for production deployment. Code is available at~\href{https://anonymous.4open.science/r/steering-17F2}{https://anonymous.4open.science/r/steering-17F2}

Method: Bayesian statistical modeling of users’ browsing behavior with online incremental learning that generates task models representing navigation patterns and usage statistics for individual users.

Result: Simulations demonstrate the algorithm’s effectiveness in both stationary and non-stationary environments, showing reliable predictions even with limited data and changing conditions.

Conclusion: This research enables adaptive systems that enhance user experience by providing personalized interface navigation support while preserving prior knowledge and learning new tasks.

Abstract: The paper presents a machine learning approach to design digital interfaces that can dynamically adapt to different users and usage strategies. The algorithm uses Bayesian statistics to model users’ browsing behavior, focusing on their habits rather than group preferences. It is distinguished by its online incremental learning, allowing reliable predictions even with little data and in the case of a changing environment. This inference method generates a task model, providing a graphical representation of navigation with the usage statistics of the current user. The algorithm learns new tasks while preserving prior knowledge. The theoretical framework is described, and simulations show the effectiveness of the approach in stationary and non-stationary environments. In conclusion, this research paves the way for adaptive systems that improve the user experience by helping them to better navigate and act on their interface.

Method: Extended L-SGD to RISC-V MCUs, evaluated performance with 32-bit floating-point arithmetic, then introduced an 8-bit quantized version of L-SGD to overcome RISC-V’s FPU limitations.

Result: 8-bit quantized L-SGD achieved nearly 4x memory reduction and 2.2x training speedup on RISC-V with negligible accuracy degradation.

Conclusion: Quantized L-SGD enables efficient on-device training on RISC-V MCUs, making decentralized ML training feasible for resource-constrained IoT devices while maintaining performance.

Abstract: Modern IoT devices increasingly rely on machine learning solutions to process data locally. However, the lack of graphics processing units (GPUs) or dedicated accelerators on most platforms makes on-device training largely infeasible, often requiring cloud-based services to perform this task. This procedure often raises privacy-related concerns, and creates dependency on reliable and always-on connectivity. Federated Learning (FL) is a new trend that addresses these issues by enabling decentralized and collaborative training directly on devices, but it requires highly efficient optimization algorithms. L-SGD, a lightweight variant of stochastic gradient descent, has enabled neural network training on Arm Cortex-M Microcontroller Units (MCUs). This work extends L-SGD to RISC-V-based MCUs, an open and emerging architecture that still lacks robust support for on-device training. L-SGD was evaluated on both Arm and RISC-V platforms using 32-bit floating-point arithmetic, highlighting the performance impact of the absence of Floating-Point Units (FPUs) in RISC-V MCUs. To mitigate these limitations, we introduce an 8-bit quantized version of L-SGD for RISC-V, which achieves nearly 4x reduction in memory usage and a 2.2x speedup in training time, with negligible accuracy degradation.

Method: Uses Difficulty-Adaptive GRPO (ADAGRPO) algorithm with difficulty-adaptive positive replay, failure curriculum filtering, and shortest path reward adjustment strategy to adapt to different task difficulties and stabilize training.

Result: Applied to Qwen2.5-VL-7B-Instruct and GLM-4.1V-9B-Base models, achieving state-of-the-art success rates of 75.8% on AndroidWorld and 46.8% on AndroidLab.

Conclusion: MOBILERL framework effectively enhances GUI agents’ performance across diverse mobile apps and tasks, and has been adopted in AutoGLM products and open-sourced.

Abstract: Building general-purpose graphical user interface (GUI) agents has become increasingly promising with the progress in vision language models. However, developing effective mobile GUI agents with reinforcement learning (RL) remains challenging due to the heavy-tailed distribution of task difficulty and the inefficiency of large-scale environment sampling. We present an online agentic reinforcement learning framework MOBILERL to enhance GUI agents in mobile environments. Its core component is the Difficulty-Adaptive GRPO (ADAGRPO) algorithm. In ADAGRPO, we design difficulty-adaptive positive replay and failure curriculum filtering to adapt the model to different task difficulties. We introduce the shortest path reward adjustment strategy to reshape rewards concerning the task length in multi-turn agentic tasks. Those strategies jointly stabilize RL training, improve sample efficiency, and generate strong performance across diverse mobile apps and tasks. We apply MOBILERL to two open models (Qwen2.5-VL-7B-Instruct and GLM-4.1V-9B-Base). The resultant MOBILERL-9B model achieves state-of-the-art results in terms of success rates on both AndroidWorld (75.8%) and AndroidLab (46.8%). The MOBILERL framework is adopted in the AutoGLM products, and also open-sourced at https://github.com/THUDM/MobileRL.

Method: CoCoGen uses generative AI and potential game theory to model collaborative learning under heterogeneous and competitive settings. It characterizes competition and heterogeneity through learning performance and utility formulations, modeling each training round as a weighted potential game, and derives GenAI-based data generation strategies.

Result: Experiments on Fashion-MNIST show how varying heterogeneity and competition levels affect organizational behavior, and demonstrate that CoCoGen consistently outperforms baseline methods.

Conclusion: CoCoGen effectively addresses the dual challenges of statistical heterogeneity and economic competition in federated learning, providing optimized data generation strategies that maximize social welfare while accounting for organizational competition dynamics.

Abstract: Cross-silo federated learning (CFL) enables organizations (e.g., hospitals or banks) to collaboratively train artificial intelligence (AI) models while preserving data privacy by keeping data local. While prior work has primarily addressed statistical heterogeneity across organizations, a critical challenge arises from economic competition, where organizations may act as market rivals, making them hesitant to participate in joint training due to potential utility loss (i.e., reduced net benefit). Furthermore, the combined effects of statistical heterogeneity and inter-organizational competition on organizational behavior and system-wide social welfare remain underexplored. In this paper, we propose CoCoGen, a coopetitive-compatible data generation framework, leveraging generative AI (GenAI) and potential game theory to model, analyze, and optimize collaborative learning under heterogeneous and competitive settings. Specifically, CoCoGen characterizes competition and statistical heterogeneity through learning performance and utility-based formulations and models each training round as a weighted potential game. We then derive GenAI-based data generation strategies that maximize social welfare. Experimental results on the Fashion-MNIST dataset reveal how varying heterogeneity and competition levels affect organizational behavior and demonstrate that CoCoGen consistently outperforms baseline methods.

Method: The paper explores discrete-time diffusion processes including variants with additive Gaussian noise, multiplicative Gaussian noise, blurring noise, and mixtures of blurring and Gaussian noises.

Result: Experimental results show discrete-time processes achieve comparable subjective and objective speech quality to continuous models, while offering more efficient and consistent training/inference.

Conclusion: Discrete-time diffusion processes provide a viable alternative to continuous-time models, delivering similar speech quality with improved efficiency and consistency between training and inference conditions.

Abstract: Diffusion models have attracted a lot of attention in recent years. These models view speech generation as a continuous-time process. For efficient training, this process is typically restricted to additive Gaussian noising, which is limiting. For inference, the time is typically discretized, leading to the mismatch between continuous training and discrete sampling conditions. Recently proposed discrete-time processes, on the other hand, usually do not have these limitations, may require substantially fewer inference steps, and are fully consistent between training/inference conditions. This paper explores some diffusion-like discrete-time processes and proposes some new variants. These include processes applying additive Gaussian noise, multiplicative Gaussian noise, blurring noise and a mixture of blurring and Gaussian noises. The experimental results suggest that discrete-time processes offer comparable subjective and objective speech quality to their widely popular continuous counterpart, with more efficient and consistent training and inference schemas.

Method: Used data preprocessing (text cleaning, TF-IDF feature extraction, SelectKBest feature selection) and trained six ML models (Decision Tree, KNN, MLP, Random Forest, Extra Trees, XGBoost) with hyperparameter optimization, evaluated using F1-score, Gmean, and AUC metrics.

Result: Ensemble methods (Extra Trees, Random Forest, XGBoost) and Multi-layer Perceptron consistently outperformed simpler classifiers (Decision Trees, KNN) across all evaluation metrics.

Conclusion: Rigorous feature selection and hyperparameter tuning are essential for building robust predictive systems for sensory product evaluation, offering a complementary approach to traditional coffee cupping by professionals.

Abstract: This study explores the application of supervised machine learning algorithms to predict coffee ratings based on a combination of influential textual and numerical attributes extracted from user reviews. Through careful data preprocessing including text cleaning, feature extraction using TF-IDF, and selection with SelectKBest, the study identifies key factors contributing to coffee quality assessments. Six models (Decision Tree, KNearest Neighbors, Multi-layer Perceptron, Random Forest, Extra Trees, and XGBoost) were trained and evaluated using optimized hyperparameters. Model performance was assessed primarily using F1-score, Gmean, and AUC metrics. Results demonstrate that ensemble methods (Extra Trees, Random Forest, and XGBoost), as well as Multi-layer Perceptron, consistently outperform simpler classifiers (Decision Trees and K-Nearest Neighbors) in terms of evaluation metrics such as F1 scores, G-mean and AUC. The findings highlight the essence of rigorous feature selection and hyperparameter tuning in building robust predictive systems for sensory product evaluation, offering a data driven approach to complement traditional coffee cupping by expertise of trained professionals.

Method: Uses reinforcement learning with structured state encoding, constrained action masking, and attention-based representations of skills, fatigue, and geographical context. Implements Proximal Policy Optimization (PPO) with feasibility masks to ensure assignments respect real-world constraints while adapting to dynamic patient arrivals and nurse availability.

Result: In simulations with realistic data, NurseSchedRL achieves improved scheduling efficiency, better skill-to-patient need alignment, and reduced fatigue compared to baseline heuristic and unconstrained RL approaches.

Conclusion: Reinforcement learning shows strong potential for decision support in complex healthcare workforce management, particularly for high-stakes nurse scheduling applications.

Abstract: Healthcare systems face increasing pressure to allocate limited nursing resources efficiently while accounting for skill heterogeneity, patient acuity, staff fatigue, and continuity of care. Traditional optimization and heuristic scheduling methods struggle to capture these dynamic, multi-constraint environments. I propose NurseSchedRL, a reinforcement learning framework for nurse-patient assignment that integrates structured state encoding, constrained action masking, and attention-based representations of skills, fatigue, and geographical context. NurseSchedRL uses Proximal Policy Optimization (PPO) with feasibility masks to ensure assignments respect real-world constraints, while dynamically adapting to patient arrivals and varying nurse availability. In simulation with realistic nurse and patient data, NurseSchedRL achieves improved scheduling efficiency, better alignment of skills to patient needs, and reduced fatigue compared to baseline heuristic and unconstrained RL approaches. These results highlight the potential of reinforcement learning for decision support in complex, high-stakes healthcare workforce management.

Method: Evaluated FL performance for anomaly detection in EVCS under system and data heterogeneity using FedAvg and FedAvgM optimization approaches, comparing with centralized models under both IID and non-IID settings.

Result: Under IID settings, FedAvg achieves superior performance to centralized models. However, performance degrades with non-IID data and system heterogeneity. FedAvgM consistently outperforms FedAvg in heterogeneous settings with better convergence and higher anomaly detection accuracy.

Conclusion: FL can handle heterogeneity in IoT-based EVCS without significant performance loss, with FedAvgM identified as a promising solution for robust, privacy-preserving EVCS security.

Abstract: Federated Learning (FL) is a decentralized training framework widely used in IoT ecosystems that preserves privacy by keeping raw data local, making it ideal for IoT-enabled cyber-physical systems with sensing and communication like Smart Grids (SGs), Connected and Automated Vehicles (CAV), and Electric Vehicle Charging Stations (EVCS). With the rapid expansion of electric vehicle infrastructure, securing these IoT-based charging stations against cyber threats has become critical. Centralized Intrusion Detection Systems (IDS) raise privacy concerns due to sensitive network and user data, making FL a promising alternative. However, current FL-based IDS evaluations overlook practical challenges such as system heterogeneity and non-IID data. To address these challenges, we conducted experiments to evaluate the performance of federated learning for anomaly detection in EV charging stations under system and data heterogeneity. We used FedAvg and FedAvgM, widely studied optimization approaches, to analyze their effectiveness in anomaly detection. Under IID settings, FedAvg achieves superior performance to centralized models using the same neural network. However, performance degrades with non-IID data and system heterogeneity. FedAvgM consistently outperforms FedAvg in heterogeneous settings, showing better convergence and higher anomaly detection accuracy. Our results demonstrate that FL can handle heterogeneity in IoT-based EVCS without significant performance loss, with FedAvgM as a promising solution for robust, privacy-preserving EVCS security.

Method: The Safe-SAIL framework systematically identifies SAEs with the best concept-specific interpretability, explains safety-related neurons, and introduces efficient strategies to scale up the interpretation process. It includes a toolkit with SAE checkpoints and human-readable neuron explanations.

Result: The framework enables extraction of a rich and diverse set of safety-relevant features that effectively capture high-risk behaviors in LLMs, overcoming challenges of identifying optimal SAEs and the high cost of detailed feature explanation.

Conclusion: Safe-SAIL advances mechanistic understanding of safety domains in LLMs and promotes research on LLM safety by providing comprehensive tools for empirical analysis of safety risks.

Abstract: Increasing deployment of large language models (LLMs) in real-world applications raises significant safety concerns. Most existing safety research focuses on evaluating LLM outputs or specific safety tasks, limiting their ability to ad- dress broader, undefined risks. Sparse Autoencoders (SAEs) facilitate interpretability research to clarify model behavior by explaining single-meaning atomic features decomposed from entangled signals. jHowever, prior applications on SAEs do not interpret features with fine-grained safety-related con- cepts, thus inadequately addressing safety-critical behaviors, such as generating toxic responses and violating safety regu- lations. For rigorous safety analysis, we must extract a rich and diverse set of safety-relevant features that effectively capture these high-risk behaviors, yet face two challenges: identifying SAEs with the greatest potential for generating safety concept-specific neurons, and the prohibitively high cost of detailed feature explanation. In this paper, we pro- pose Safe-SAIL, a framework for interpreting SAE features within LLMs to advance mechanistic understanding in safety domains. Our approach systematically identifies SAE with best concept-specific interpretability, explains safety-related neurons, and introduces efficient strategies to scale up the in- terpretation process. We will release a comprehensive toolkit including SAE checkpoints and human-readable neuron ex- planations, which supports empirical analysis of safety risks to promote research on LLM safety.

Method: The approach uses Gauss-Hermite quadrature to evaluate conditional failure probabilities under aleatory uncertainty with FORM/SORM methods, then integrates these probabilities across epistemic uncertainty realizations.

Result: Three examples demonstrate the method maintains computational efficiency while providing more trustworthy predictions than traditional approaches that ignore model uncertainty.

Conclusion: The proposed approach effectively decouples nested uncertainties in surrogate-based reliability analysis, yielding improved accuracy without sacrificing computational efficiency.

Abstract: Machine learning surrogates are increasingly employed to replace expensive computational models for physics-based reliability analysis. However, their use introduces epistemic uncertainty from model approximation errors, which couples with aleatory uncertainty in model inputs, potentially compromising the accuracy of reliability predictions. This study proposes a Gauss-Hermite quadrature approach to decouple these nested uncertainties and enable more accurate reliability analysis. The method evaluates conditional failure probabilities under aleatory uncertainty using First and Second Order Reliability Methods and then integrates these probabilities across realizations of epistemic uncertainty. Three examples demonstrate that the proposed approach maintains computational efficiency while yielding more trustworthy predictions than traditional methods that ignore model uncertainty.

Method: The model uses STL decomposition to separate time series into trend, periodic, and residual components, applies 3σ principle for outlier handling, and uses GRU neural networks for prediction based on processed metro card data.

Result: The STL-GRU model outperforms LSTM, GRU, and STL-LSTM models, reducing MAPE by at least 2.3% on weekdays, 1.36% on Fridays, and 6.42% on rest days.

Conclusion: The proposed STL-GRU combined prediction model significantly improves transfer passenger flow prediction accuracy and provides reliable support for intelligent metro operation decisions.

Abstract: In the metro intelligent transportation system, accurate transfer passenger flow prediction is a key link in optimizing operation plans and improving transportation efficiency. To further improve the theory of metro internal transfer passenger flow prediction and provide more reliable support for intelligent operation decisions, this paper innovatively proposes a metro transfer passenger flow prediction model that integrates the Seasonal and Trend decomposition using Loess (STL) method and Gated Recurrent Unit (GRU).In practical application, the model first relies on the deep learning library Keras to complete the construction and training of the GRU model, laying the foundation for subsequent prediction; then preprocesses the original metro card swiping data, uses the graph-based depth-first search algorithm to identify passengers’ travel paths, and further constructs the transfer passenger flow time series; subsequently adopts the STL time series decomposition algorithm to decompose the constructed transfer passenger flow time series into trend component, periodic component and residual component, and uses the 3{\sigma} principle to eliminate and fill the outliers in the residual component, and finally completes the transfer passenger flow prediction.Taking the transfer passenger flow data of a certain metro station as the research sample, the validity of the model is verified. The results show that compared with Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and the combined model of STL time series decomposition method and Long Short-Term Memory (STL-LSTM), the STL-GRU combined prediction model significantly improves the prediction accuracy of transfer passenger flow on weekdays (excluding Fridays), Fridays and rest days, with the mean absolute percentage error (MAPE) of the prediction results reduced by at least 2.3, 1.36 and 6.42 percentage points respectively.

Method: Analysis of weight matrices from transformer applications to two representative physical problems, examining their character and potential parallels with path-integration techniques.

Result: Weight matrices show random-like characteristics with no directly recognizable link to the physical problem structure, though parallels with generalized path-integration may explain this randomness.

Conclusion: Machine learning and scientific methods represent distinct complementary knowledge paths, but strict explainability through direct parameter-structure correspondence remains elusive, highlighting potential hazards of knowledge acquisition without insight.

Abstract: It is shown that the weight matrices of transformer-based machine learning applications to the solution of two representative physical applications show a random-like character which bears no directly recognizable link to the physical and mathematical structure of the physical problem under study. This suggests that machine learning and the scientific method may represent two distinct and potentially complementary paths to knowledge, even though a strict notion of explainability in terms of direct correspondence between network parameters and physical structures may remain out of reach. It is also observed that drawing a parallel between transformer operation and (generalized) path-integration techniques may account for the random-like nature of the weights, but still does not resolve the tension with explainability. We conclude with some general comments on the hazards of gleaning knowledge without the benefit of Insight.

Method: Uses a dual-expert design: dedicated LoRA expert per task for task-specific knowledge preservation, and shared LoRA expert for cross-task transfer. Integrates a task-aware discriminator within a GAN to filter task-irrelevant noise and encourage task-aligned information sharing.

Result: Extensive experiments on MTL5 and Tencent3 benchmarks show effectiveness. Real-world A/B testing on Tencent Video platform reduced manual review costs by 15.3%.

Conclusion: MoE-CL is practical for large-scale industrial deployment where continual adaptation and stable transfer are critical, supporting self-evolution of LLMs in dynamic environments.

Abstract: In real-world industrial settings, large language models (LLMs) must learn continually to keep pace with diverse and evolving tasks, requiring self-evolution to refine knowledge under dynamic data distributions. However, existing continual learning (CL) approaches, such as replay and parameter isolation, often suffer from catastrophic forgetting: training on new tasks degrades performance on earlier ones by overfitting to the new distribution and weakening generalization.We propose MoE-CL, a parameter-efficient adversarial mixture-of-experts framework for industrial-scale, self-evolving continual instruction tuning of LLMs. MoE-CL uses a dual-expert design: (1) a dedicated LoRA expert per task to preserve task-specific knowledge via parameter independence, mitigating forgetting; and (2) a shared LoRA expert to enable cross-task transfer. To prevent transferring task-irrelevant noise through the shared pathway, we integrate a task-aware discriminator within a GAN. The discriminator encourages the shared expert to pass only task-aligned information during sequential training. Through adversarial learning, the shared expert acquires generalized representations that mimic the discriminator, while dedicated experts retain task-specific details, balancing knowledge retention and cross-task generalization and thereby supporting self-evolution.Extensive experiments on the public MTL5 benchmark and an industrial Tencent3 benchmark validate the effectiveness of MoE-CL for continual instruction tuning. In real-world A/B testing for content compliance review on the Tencent Video platform, MoE-CL reduced manual review costs by 15.3%. These results demonstrate that MoE-CL is practical for large-scale industrial deployment where continual adaptation and stable transfer are critical.

Method: Proposes a weighted gradient tracking distributed privacy-preserving algorithm that uses decaying weight factors to eliminate privacy leakage risks in gradient tracking, with convergence analysis under time-varying heterogeneous step sizes.

Result: The proposed algorithm converges precisely to the optimal solution under mild assumptions, as validated through numerical simulations including a classical distributed estimation problem and distributed training of a convolutional neural network.

Conclusion: The weighted gradient tracking approach successfully eliminates privacy leakage risks in distributed optimization while maintaining convergence performance, providing an effective privacy-preserving solution for distributed optimization applications.

Abstract: This paper investigates the privacy-preserving distributed optimization problem, aiming to protect agents’ private information from potential attackers during the optimization process. Gradient tracking, an advanced technique for improving the convergence rate in distributed optimization, has been applied to most first-order algorithms in recent years. We first reveal the inherent privacy leakage risk associated with gradient tracking. Building upon this insight, we propose a weighted gradient tracking distributed privacy-preserving algorithm, eliminating the privacy leakage risk in gradient tracking using decaying weight factors. Then, we characterize the convergence of the proposed algorithm under time-varying heterogeneous step sizes. We prove the proposed algorithm converges precisely to the optimal solution under mild assumptions. Finally, numerical simulations validate the algorithm’s effectiveness through a classical distributed estimation problem and the distributed training of a convolutional neural network.

Method: Uses static graph for long-term stable dependencies and dynamic graph from multi-level wavelet decomposition for multi-scale features, fused via attention-gated module with multi-kernel dilated convolutional network.

Result: Comprehensive experiments on real-world benchmark datasets demonstrate the model’s effectiveness.

Conclusion: SDGF successfully addresses the challenge of capturing intricate multi-scale inter-series correlations in time series forecasting.

Abstract: Inter-series correlations are crucial for accurate multivariate time series forecasting, yet these relationships often exhibit complex dynamics across different temporal scales. Existing methods are limited in modeling these multi-scale dependencies and struggle to capture their intricate and evolving nature. To address this challenge, this paper proposes a novel Static-Dynamic Graph Fusion network (SDGF), whose core lies in capturing multi-scale inter-series correlations through a dual-path graph structure learning approach. Specifically, the model utilizes a static graph based on prior knowledge to anchor long-term, stable dependencies, while concurrently employing Multi-level Wavelet Decomposition to extract multi-scale features for constructing an adaptively learned dynamic graph to capture associations at different scales. We design an attention-gated module to fuse these two complementary sources of information intelligently, and a multi-kernel dilated convolutional network is then used to deepen the understanding of temporal patterns. Comprehensive experiments on multiple widely used real-world benchmark datasets demonstrate the effectiveness of our proposed model.

Method: Created a large-scale dataset of diverse open-source LLM structures and their performance across multiple benchmarks. Conducted systematic data mining analysis and used mechanistic interpretability techniques to validate findings.

Result: The study provides quantified relationships between structural choices and performance across different benchmarks, offering data-driven insights into LLM optimization.

Conclusion: This work aims to guide targeted development of future LLMs by providing systematic, data-driven understanding of how structural configurations affect performance. The dataset will be publicly released.

Abstract: Large language models (LLMs) have achieved remarkable success across various domains, driving significant technological advancements and innovations. Despite the rapid growth in model scale and capability, systematic, data-driven research on how structural configurations affect performance remains scarce. To address this gap, we present a large-scale dataset encompassing diverse open-source LLM structures and their performance across multiple benchmarks. Leveraging this dataset, we conduct a systematic, data mining-driven analysis to validate and quantify the relationship between structural configurations and performance. Our study begins with a review of the historical development of LLMs and an exploration of potential future trends. We then analyze how various structural choices impact performance across benchmarks and further corroborate our findings using mechanistic interpretability techniques. By providing data-driven insights into LLM optimization, our work aims to guide the targeted development and application of future models. We will release our dataset at https://huggingface.co/datasets/DX0369/LLM-Structure-Performance-Dataset

Method: Created a standardized benchmark with 40 downstream tasks in unified format, fine-tuned using same hyperparameters to obtain 680 LoRA modules across 17 model architectures, then tested 3 LoRA-MoE methods with OpenCompass.

Result: LoRAMoE performed best among tested methods, and prioritizing task-relevant LoRAs further improves MoE performance. The benchmark enables systematic evaluation of LoRA-MoE approaches.

Conclusion: The proposed LoRALib benchmark provides standardized evaluation for LoRA-MoE methods, revealing LoRAMoE’s superior performance and the importance of task-relevant LoRA selection for cross-task generalization.

Abstract: As a parameter efficient fine-tuning (PEFT) method, low-rank adaptation (LoRA) can save significant costs in storage and computing, but its strong adaptability to a single task is often accompanied by insufficient cross-task generalization capabilities. To improve this, existing work combines LoRA with mixture-of-experts (MoE) to enhance the model’s adaptability through expert modules and routing mechanisms. However, existing LoRA-MoE methods lack unified standards in models, datasets, hyperparameters, and evaluation methods, making it difficult to conduct fair comparisons between different methods. To this end, we proposed a unified benchmark named LoRALib. Specifically, we standardized datasets from $40$ downstream tasks into a unified format, fine-tuned them using the same hyperparameters and obtained $680$ LoRA modules across $17$ model architectures. Based on this LoRA library, we conduct large-scale experiments on $3$ representative LoRA-MoE methods and different LoRA selection mechanisms using the open-sourced testing tool OpenCompass. Extensive experiments show that LoRAMoE performs best, and that prioritizing LoRAs relevant to the target task can further improve the performance of MoE. We hope these findings will inspire future work. Our datasets and LoRA library are available at https://huggingface.co/datasets/YaoLuzjut/LoRAOcean_dataset and https://huggingface.co/YaoLuzjut/models.

Method: Uses rank-induced Plackett-Luce mirror descent that updates directly in rank-induced PL parameterization, ensuring played distributions remain within rank-induced distributions at every round.

Result: RIPLM is the first algorithm that simultaneously achieves rank-faithfulness and variance-adaptivity in sleeping experts setting.

Conclusion: The algorithm successfully leverages the structural equivalence between rank and distributional benchmarks while maintaining desirable properties not achieved by prior approaches.

Abstract: We introduce a new algorithm, \emph{Rank-Induced Plackett–Luce Mirror Descent (RIPLM)}, which leverages the structural equivalence between the \emph{rank benchmark} and the \emph{distributional benchmark} established in \citet{BergamOzcanHsu2022}. Unlike prior approaches that operate on expert identities, RIPLM updates directly in the \emph{rank-induced Plackett–Luce (PL)} parameterization. This ensures that the algorithm’s played distributions remain within the class of rank-induced distributions at every round, preserving the equivalence with the rank benchmark. To our knowledge, RIPLM is the first algorithm that is both (i) \emph{rank-faithful} and (ii) \emph{variance-adaptive} in the sleeping experts setting.

Method: The study compared FOLD-SE and FOLD-R++ (rule-based classifiers) in binary classification, and evaluated FOLD-SE against XGBoost (ensemble classifier) in multi-category classification. Performance was measured using accuracy, F1 scores, and processing time.

Result: FOLD-SE outperformed FOLD-R++ in binary classification with fewer rules and minor accuracy trade-offs. In multi-category classification, FOLD-SE was more precise and efficient than XGBoost while generating interpretable rule sets.

Conclusion: Rule-based approaches like FOLD-SE can bridge the gap between explainability and performance, making them viable alternatives to black-box models for diverse classification tasks.

Abstract: Recently, the demand for Machine Learning (ML) models that can balance accuracy, efficiency, and interpreability has grown significantly. Traditionally, there has been a tradeoff between accuracy and explainability in predictive models, with models such as Neural Networks achieving high accuracy on complex datasets while sacrificing internal transparency. As such, new rule-based algorithms such as FOLD-SE have been developed that provide tangible justification for predictions in the form of interpretable rule sets. The primary objective of this study was to compare FOLD-SE and FOLD-R++, both rule-based classifiers, in binary classification and evaluate how FOLD-SE performs against XGBoost, a widely used ensemble classifier, when applied to multi-category classification. We hypothesized that because FOLD-SE can generate a condensed rule set in a more explainable manner, it would lose upwards of an average of 3 percent in accuracy and F1 score when compared with XGBoost and FOLD-R++ in multiclass and binary classification, respectively. The research used data collections for classification, with accuracy, F1 scores, and processing time as the primary performance measures. Outcomes show that FOLD-SE is superior to FOLD-R++ in terms of binary classification by offering fewer rules but losing a minor percentage of accuracy and efficiency in processing time; in tasks that involve multi-category classifications, FOLD-SE is more precise and far more efficient compared to XGBoost, in addition to generating a comprehensible rule set. The results point out that FOLD-SE is a better choice for both binary tasks and classifications with multiple categories. Therefore, these results demonstrate that rule-based approaches like FOLD-SE can bridge the gap between explainability and performance, highlighting their potential as viable alternatives to black-box models in diverse classification tasks.

Method: Used logistic regression and t-tests on Pima Indian dataset to identify T2DM predictors (78.43% accuracy), combined with pathway mapping strategy linking predictors to insulin signaling, AMPK, and PPAR pathways without requiring molecular data.

Result: Developed an ML framework that successfully identifies high-risk individuals and proposes therapeutic strategies including dual GLP-1/GIP receptor agonists, AMPK activators, SIRT1 modulators, and phytochemical interventions.

Conclusion: The framework advances precision medicine by providing interpretable, scalable solutions for early T2DM detection and targeted interventions, particularly benefiting high-risk populations through novel therapeutic strategy identification.

Abstract: Metabolic disorders, particularly type 2 diabetes mellitus (T2DM), represent a significant global health burden, disproportionately impacting genetically predisposed populations such as the Pima Indians (a Native American tribe from south central Arizona). This study introduces a novel machine learning (ML) framework that integrates predictive modeling with gene-agnostic pathway mapping to identify high-risk individuals and uncover potential therapeutic targets. Using the Pima Indian dataset, logistic regression and t-tests were applied to identify key predictors of T2DM, yielding an overall model accuracy of 78.43%. To bridge predictive analytics with biological relevance, we developed a pathway mapping strategy that links identified predictors to critical signaling networks, including insulin signaling, AMPK, and PPAR pathways. This approach provides mechanistic insights without requiring direct molecular data. Building upon these connections, we propose therapeutic strategies such as dual GLP-1/GIP receptor agonists, AMPK activators, SIRT1 modulators, and phytochemical, further validated through pathway enrichment analyses. Overall, this framework advances precision medicine by offering interpretable and scalable solutions for early detection and targeted intervention in metabolic disorders. The key contributions of this work are: (1) development of an ML framework combining logistic regression and principal component analysis (PCA) for T2DM risk prediction; (2) introduction of a gene-agnostic pathway mapping approach to generate mechanistic insights; and (3) identification of novel therapeutic strategies tailored for high-risk populations.

Method: KM-GPT integrates advanced image preprocessing, multi-modal reasoning using GPT-5, and iterative reconstruction algorithms in a hybrid reasoning architecture that automates conversion of unstructured KM plot information into structured data flows.

Result: KM-GPT demonstrated superior accuracy on both synthetic and real-world datasets and was successfully applied to reconstruct IPD for a meta-analysis of gastric cancer immunotherapy trials, facilitating evidence synthesis and biomarker-based subgroup analyses.

Conclusion: KM-GPT transforms clinical research by automating traditionally manual processes, providing a scalable web-based solution that enables more informed downstream analyses and supports evidence-based decision-making through reconstructed IPD.

Abstract: Reconstructing individual patient data (IPD) from Kaplan-Meier (KM) plots provides valuable insights for evidence synthesis in clinical research. However, existing approaches often rely on manual digitization, which is error-prone and lacks scalability. To address these limitations, we develop KM-GPT, the first fully automated, AI-powered pipeline for reconstructing IPD directly from KM plots with high accuracy, robustness, and reproducibility. KM-GPT integrates advanced image preprocessing, multi-modal reasoning powered by GPT-5, and iterative reconstruction algorithms to generate high-quality IPD without manual input or intervention. Its hybrid reasoning architecture automates the conversion of unstructured information into structured data flows and validates data extraction from complex KM plots. To improve accessibility, KM-GPT is equipped with a user-friendly web interface and an integrated AI assistant, enabling researchers to reconstruct IPD without requiring programming expertise. KM-GPT was rigorously evaluated on synthetic and real-world datasets, consistently demonstrating superior accuracy. To illustrate its utility, we applied KM-GPT to a meta-analysis of gastric cancer immunotherapy trials, reconstructing IPD to facilitate evidence synthesis and biomarker-based subgroup analyses. By automating traditionally manual processes and providing a scalable, web-based solution, KM-GPT transforms clinical research by leveraging reconstructed IPD to enable more informed downstream analyses, supporting evidence-based decision-making.

Method: Proposes AdaSTI with three key components: BiS4PI network for pre-imputation using bi-directional S4 model, Spatio-Temporal Conditionalizer (STC) to extract conditional information, and Noise-Aware Spatio-Temporal (NAST) network with gated attention to capture varying dependencies across diffusion steps.

Result: Extensive experiments on three real-world datasets show AdaSTI outperforms existing methods in all settings, achieving up to 46.4% reduction in imputation error compared to previous approaches.

Conclusion: AdaSTI effectively addresses the limitations of previous diffusion-based spatio-temporal imputation methods by adaptively handling dependency extraction and variability across diffusion steps, demonstrating superior performance across multiple real-world datasets.

Abstract: Spatio-temporal data abounds in domain like traffic and environmental monitoring. However, it often suffers from missing values due to sensor malfunctions, transmission failures, etc. Recent years have seen continued efforts to improve spatio-temporal data imputation performance. Recently diffusion models have outperformed other approaches in various tasks, including spatio-temporal imputation, showing competitive performance. Extracting and utilizing spatio-temporal dependencies as conditional information is vital in diffusion-based methods. However, previous methods introduce error accumulation in this process and ignore the variability of the dependencies in the noisy data at different diffusion steps. In this paper, we propose AdaSTI (Adaptive Dependency Model in Diffusion-based Spatio-Temporal Imputation), a novel spatio-temporal imputation approach based on conditional diffusion model. Inside AdaSTI, we propose a BiS4PI network based on a bi-directional S4 model for pre-imputation with the imputed result used to extract conditional information by our designed Spatio-Temporal Conditionalizer (STC)network. We also propose a Noise-Aware Spatio-Temporal (NAST) network with a gated attention mechanism to capture the variant dependencies across diffusion steps. Extensive experiments on three real-world datasets show that AdaSTI outperforms existing methods in all the settings, with up to 46.4% reduction in imputation error.

Method: Multi-label classification framework using MIMIC-IV database (85,242 ICU stays) with features from first 24 hours (vital sign aggregates, lab values, demographics) to predict CETs defined by rule-based criteria from hours 24-72.

Result: XGBoost achieved F1-scores: 0.66 (respiratory), 0.72 (hemodynamic), 0.76 (renal), 0.62 (neurologic). Feature analysis confirmed clinical relevance with respiratory rate, blood pressure, and creatinine as most influential predictors.

Conclusion: The framework demonstrates practical potential for early, interpretable clinical alerts without complex time-series modeling, providing a more comprehensive approach than traditional single-outcome warning systems.

Abstract: Intensive Care Unit (ICU) patients often present with complex, overlapping signs of physiological deterioration that require timely escalation of care. Traditional early warning systems, such as SOFA or MEWS, are limited by their focus on single outcomes and fail to capture the multi-dimensional nature of clinical decline. This study proposes a multi-label classification framework to predict Care Escalation Triggers (CETs), including respiratory failure, hemodynamic instability, renal compromise, and neurological deterioration, using the first 24 hours of ICU data. Using the MIMIC-IV database, CETs are defined through rule-based criteria applied to data from hours 24 to 72 (for example, oxygen saturation below 90, mean arterial pressure below 65 mmHg, creatinine increase greater than 0.3 mg/dL, or a drop in Glasgow Coma Scale score greater than 2). Features are extracted from the first 24 hours and include vital sign aggregates, laboratory values, and static demographics. We train and evaluate multiple classification models on a cohort of 85,242 ICU stays (80 percent training: 68,193; 20 percent testing: 17,049). Evaluation metrics include per-label precision, recall, F1-score, and Hamming loss. XGBoost, the best performing model, achieves F1-scores of 0.66 for respiratory, 0.72 for hemodynamic, 0.76 for renal, and 0.62 for neurologic deterioration, outperforming baseline models. Feature analysis shows that clinically relevant parameters such as respiratory rate, blood pressure, and creatinine are the most influential predictors, consistent with the clinical definitions of the CETs. The proposed framework demonstrates practical potential for early, interpretable clinical alerts without requiring complex time-series modeling or natural language processing.

Method: ConceptFlow uses concept attentions to associate filters with high-level concepts, and conceptual pathways derived from a concept transition matrix to quantify how concepts propagate and transform between filters.

Result: Experimental results show ConceptFlow provides semantically meaningful insights into model reasoning and validates the effectiveness of concept attentions and conceptual pathways in explaining decision behavior.

Conclusion: ConceptFlow offers deeper insight into CNN internal logic by modeling hierarchical conceptual pathways, supporting more faithful and human-aligned explanations.

Abstract: Concept-based interpretability for Convolutional Neural Networks (CNNs) aims to align internal model representations with high-level semantic concepts, but existing approaches largely overlook the semantic roles of individual filters and the dynamic propagation of concepts across layers. To address these limitations, we propose ConceptFlow, a concept-based interpretability framework that simulates the internal “thinking path” of a model by tracing how concepts emerge and evolve across layers. ConceptFlow comprises two key components: (i) concept attentions, which associate each filter with relevant high-level concepts to enable localized semantic interpretation, and (ii) conceptual pathways, derived from a concept transition matrix that quantifies how concepts propagate and transform between filters. Together, these components offer a unified and structured view of internal model reasoning. Experimental results demonstrate that ConceptFlow yields semantically meaningful insights into model reasoning, validating the effectiveness of concept attentions and conceptual pathways in explaining decision behavior. By modeling hierarchical conceptual pathways, ConceptFlow provides deeper insight into the internal logic of CNNs and supports the generation of more faithful and human-aligned explanations.

Method: Proposes STS with two components: Visual Token Compressor to reduce visual token information load, and Layer Dynamic Skipper to dynamically skip unnecessary layers during forward/backward passes.

Result: Extensively evaluated on multiple benchmarks, demonstrating effectiveness and efficiency across diverse MLLM architectures.

Conclusion: The proposed sparse training scheme provides an efficient framework for training MLLMs while maintaining performance, with broad applicability to various architectures.

Abstract: Multimodal Large Language Models (MLLMs) have demonstrated outstanding performance across a variety of domains. However, training MLLMs is often inefficient due to the significantly longer input sequences introduced by multimodal data and the low utilization of inter-layer computations. To address this challenge, we shift the focus to the training process itself and propose a novel training-efficient framework based on sparse representations, termed the Sparse Training Scheme (STS). This scheme consists of two key components: the Visual Token Compressor, which reduces the information load by compressing visual tokens, and the Layer Dynamic Skipper, which mitigates the computational overhead by dynamically skipping unnecessary layers in the language model during both forward and backward passes. Our approach is broadly applicable to diverse MLLM architectures and has been extensively evaluated on multiple benchmarks, demonstrating its effectiveness and efficiency.

Method: HyperNAS consists of two main components: a global encoding scheme to capture macro-structure information and a shared hypernetwork as an auxiliary task to enhance inter-architecture pattern investigation. It uses a dynamic adaptive multi-task loss for training stability and Pareto front exploration.

Result: HyperNAS achieves state-of-the-art results with 97.60% top-1 accuracy on CIFAR-10 and 82.4% top-1 accuracy on ImageNet, using at least 5× fewer samples than existing methods.

Conclusion: HyperNAS demonstrates superior performance in few-shot scenarios across five representative search spaces, including Vision Transformers, proving its effectiveness in enhancing architecture representation learning for NAS.

Abstract: Time-intensive performance evaluations significantly impede progress in Neural Architecture Search (NAS). To address this, neural predictors leverage surrogate models trained on proxy datasets, allowing for direct performance predictions for new architectures. However, these predictors often exhibit poor generalization due to their limited ability to capture intricate relationships among various architectures. In this paper, we propose HyperNAS, a novel neural predictor paradigm for enhancing architecture representation learning. HyperNAS consists of two primary components: a global encoding scheme and a shared hypernetwork. The global encoding scheme is devised to capture the comprehensive macro-structure information, while the shared hypernetwork serves as an auxiliary task to enhance the investigation of inter-architecture patterns. To ensure training stability, we further develop a dynamic adaptive multi-task loss to facilitate personalized exploration on the Pareto front. Extensive experiments across five representative search spaces, including ViTs, demonstrate the advantages of HyperNAS, particularly in few-shot scenarios. For instance, HyperNAS strikes new state-of-the-art results, with 97.60% top-1 accuracy on CIFAR-10 and 82.4% top-1 accuracy on ImageNet, using at least 5.0$\times$ fewer samples.

Method: Three-stage approach: tokenization of log patches into geological tokens, self-supervised pretraining with masked-token modeling and stratigraphy-aware contrastive learning, and multi-task adaptation with few-shot fine-tuning.

Result: WLFM achieves 0.0041 MSE in porosity estimation and 74.13% accuracy in lithology classification, with fine-tuned version (WLFM-Finetune) improving to 0.0038 MSE and 78.10% accuracy. The model shows emergent layer-awareness and learns reusable geological vocabulary.

Conclusion: WLFM establishes a scalable, interpretable, and transferable backbone for geological AI with potential for multi-modal integration of logs, seismic, and textual data.

Abstract: Well-log interpretation is fundamental for subsurface characterization but remains challenged by heterogeneous tool responses, noisy signals, and limited labels. We propose WLFM, a foundation model pretrained on multi-curve logs from 1200 wells, comprising three stages: tokenization of log patches into geological tokens, self-supervised pretraining with masked-token modeling and stratigraphy-aware contrastive learning, and multi-task adaptation with few-shot fine-tuning. WLFM consistently outperforms state-of-the-art baselines, achieving 0.0041 MSE in porosity estimation and 74.13% accuracy in lithology classification, while WLFM-Finetune further improves to 0.0038 MSE and 78.10% accuracy. Beyond predictive accuracy, WLFM exhibits emergent layer-awareness, learns a reusable geological vocabulary, and reconstructs masked curves with reasonable fidelity, though systematic offsets are observed in shallow and ultra-deep intervals. Although boundary detection is not explicitly evaluated here, clustering analyses suggest strong potential for future extension. These results establish WLFM as a scalable, interpretable, and transferable backbone for geological AI, with implications for multi-modal integration of logs, seismic, and textual data.

Method: Combines a large protein language model fine-tuned on peptide data with reinforcement learning (proximal policy optimization) using a composite reward function that includes MIC predictions and physicochemical objectives.

Result: 100% of 100 designed peptides showed antimicrobial activity with low MIC values (nanomolar range), and 99/100 exhibited broad-spectrum activity against multiple clinically relevant bacteria, primarily targeting cytoplasmic membranes.

Conclusion: The framework provides a scalable platform for rapid generation of diverse, potent peptide antibiotics, enabling iterative optimization of both potency and developability within hours.

Abstract: Antimicrobial resistance (AMR) is projected to cause up to 10 million deaths annually by 2050, underscoring the urgent need for new antibiotics. Here we present ApexAmphion, a deep-learning framework for de novo design of antibiotics that couples a 6.4-billion-parameter protein language model with reinforcement learning. The model is first fine-tuned on curated peptide data to capture antimicrobial sequence regularities, then optimised with proximal policy optimization against a composite reward that combines predictions from a learned minimum inhibitory concentration (MIC) classifier with differentiable physicochemical objectives. In vitro evaluation of 100 designed peptides showed low MIC values (nanomolar range in some cases) for all candidates (100% hit rate). Moreover, 99 our of 100 compounds exhibited broad-spectrum antimicrobial activity against at least two clinically relevant bacteria. The lead molecules killed bacteria primarily by potently targeting the cytoplasmic membrane. By unifying generation, scoring and multi-objective optimization with deep reinforcement learning in a single pipeline, our approach rapidly produces diverse, potent candidates, offering a scalable route to peptide antibiotics and a platform for iterative steering toward potency and developability within hours.

Method: Three core improvements: 1) unified 3D-Resampler architecture for compact image/video encoding, 2) unified learning paradigm for document knowledge and text recognition without heavy data engineering, 3) hybrid reinforcement learning strategy for both short and long reasoning modes.

Result: Surpasses GPT-4o-latest and Qwen2.5-VL 72B in OpenCompass evaluation. On VideoMME benchmark, achieves SOTA among models under 30B size with only 46.7% GPU memory cost and 8.7% inference time of Qwen2.5-VL 7B.

Conclusion: MiniCPM-V 4.5 demonstrates that strong multimodal performance can be achieved with high efficiency through optimized architecture, data strategy, and training methods, making MLLMs more accessible.

Abstract: Multimodal Large Language Models (MLLMs) are undergoing rapid progress and represent the frontier of AI development. However, their training and inference efficiency have emerged as a core bottleneck in making MLLMs more accessible and scalable. To address the challenges, we present MiniCPM-V 4.5, an 8B parameter model designed for high efficiency and strong performance. We introduce three core improvements in model architecture, data strategy and training method: a unified 3D-Resampler model architecture for highly compact encoding over images and videos, a unified learning paradigm for document knowledge and text recognition without heavy data engineering, and a hybrid reinforcement learning strategy for proficiency in both short and long reasoning modes. Comprehensive experimental results in OpenCompass evaluation show that MiniCPM-V 4.5 surpasses widely used proprietary models such as GPT-4o-latest, and significantly larger open-source models such as Qwen2.5-VL 72B. Notably, the strong performance is achieved with remarkable efficiency. For example, on the widely adopted VideoMME benchmark, MiniCPM-V 4.5 achieves state-of-the-art performance among models under 30B size, using just 46.7% GPU memory cost and 8.7% inference time of Qwen2.5-VL 7B.

Method: Presents and compares 9 training methodologies for dual-optimization dynamics in neural networks using parameterized linear B-spline activation functions. Experiments conducted on FNNs and CNNs.

Result: Achieved up to 94% lower end model error rates in FNNs and 51% lower rates in CNNs compared to traditional ReLU-based models.

Conclusion: Optimizing activation function shapes provides substantial accuracy improvements but comes with trade-offs including additional development complexity, training complexity, and increased end model latency.

Abstract: Activation functions in neural networks are typically selected from a set of empirically validated, commonly used static functions such as ReLU, tanh, or sigmoid. However, by optimizing the shapes of a network’s activation functions, we can train models that are more parameter-efficient and accurate by assigning more optimal activations to the neurons. In this paper, I present and compare 9 training methodologies to explore dual-optimization dynamics in neural networks with parameterized linear B-spline activation functions. The experiments realize up to 94% lower end model error rates in FNNs and 51% lower rates in CNNs compared to traditional ReLU-based models. These gains come at the cost of additional development and training complexity as well as end model latency.

Method: Hybrid reinforcement learning solver combining ALNS-based truck tour optimization with a pointer/attention policy for scheduling drone sorties. Uses hard feasibility masks for endurance and recharge constraints, with exact timeline simulation for makespan computation.

Result: Achieves average makespan of 5.203±0.093, outperforming ALNS (5.349±0.038) by 2.73% and matching neural network performance (5.208±0.124) within 0.10%. The learned scheduler consistently performs equal or better than baselines.

Conclusion: The hybrid RL approach effectively balances truck and drone operations to minimize completion time, demonstrating superior performance over traditional optimization methods while maintaining feasibility under battery constraints.

Abstract: We study last-mile delivery with one truck and one drone under explicit battery management: the drone flies at twice the truck speed; each sortie must satisfy an endurance budget; after every delivery the drone recharges on the truck before the next launch. We introduce a hybrid reinforcement learning (RL) solver that couples an ALNS-based truck tour (with 2/3-opt and Or-opt) with a small pointer/attention policy that schedules drone sorties. The policy decodes launch–serve–rendezvous triplets with hard feasibility masks for endurance and post-delivery recharge; a fast, exact timeline simulator enforces launch/recovery handling and computes the true makespan used by masked greedy/beam decoding. On Euclidean instances with $N{=}50$, $E{=}0.7$, and $R{=}0.1$, the method achieves an average makespan of \textbf{5.203}$\pm$0.093, versus \textbf{5.349}$\pm$0.038 for ALNS and \textbf{5.208}$\pm$0.124 for NN – i.e., \textbf{2.73%} better than ALNS on average and within \textbf{0.10%} of NN. Per-seed, the RL scheduler never underperforms ALNS on the same instance and ties or beats NN on two of three seeds. A decomposition of the makespan shows the expected truck–wait trade-off across heuristics; the learned scheduler balances both to minimize the total completion time. We provide a config-first implementation with plotting and significance-test utilities to support replication.

Method: Proposes DSFT (Diffusion SFT) strategy with adjusted masking strategy and loss function to guide models in understanding mathematical and logical patterns, which can be combined with pre-training, reinforcement learning, and other methods.

Result: Validated on LLaDA and Dream series models, DSFT on small-scale data achieves 5-10% improvement on mathematical problems and approximately 2% improvement on logical problems.

Conclusion: The masking approach offers insights for future learning of specific patterns and can be easily combined with other training methods, applied to various dLLMs.

Abstract: Diffusion large language models (dLLMs) have emerged as a new architecture following auto regressive models. Their denoising process offers a powerful generative advantage, but they present significant challenges in learning and understanding numerically sensitive mathematical and order-sensitive logical tasks. Current training methods, including pre-training, fine-tuning, and reinforcement learning, focus primarily on improving general knowledge retention and reasoning abilities, but lack a comprehensive understanding of mathematical and logical patterns. We propose DSFT, a simple yet effective Diffusion SFT strategy, by adjusting the masking strategy and loss function, guiding models to understand mathematical and logical patterns. This strategy can be flexibly combined with pre-training, reinforcement learning, and other training methods. Validated on models such as LLaDA and Dream series, we prove that DSFT on small-scale data can achieve improvements of 5-10% and approximately 2% on mathematical and logical problems, respectively. This inspiring masking approach offers insights for future learning of specific patterns, which can be easily and efficiently combined with other training methods and applied to various dLLMs. Our code is publicly available at https://anonymous.4open.science/r/DSFT-0FFB/

Method: Proposes MobiGPT with a unified structure using soft-prompt learning to understand different data types and temporal masking mechanism to handle three forecasting tasks: short-term prediction, long-term prediction, and distribution generation.

Result: Evaluations on real-world datasets with over 100,000 samples show MobiGPT achieves 27.37%, 20.08%, and 7.27% improvement in forecasting accuracy compared to existing models, with superior zero/few-shot performance (over 21.51% improvement) in unseen scenarios.

Conclusion: MobiGPT demonstrates strong generalization and transferability as a foundation model for mobile data forecasting, capable of handling multiple data types and forecasting tasks with significant accuracy improvements over existing approaches.

Abstract: With the rapid development of mobile communication technologies, future mobile networks will offer vast services and resources for commuting, production, daily life, and entertainment. Accurate and efficient forecasting of mobile data (e.g., cell traffic, user behavior, channel quality) helps operators monitor network state changes, orchestrate wireless resources, and schedule infrastructure and users, thereby improving supply efficiency and service quality. However, current forecasting paradigms rely on customized designs with tailored models for exclusive data types. Such approaches increase complexity and deployment costs under large-scale, heterogeneous networks involving base stations, users, and channels. In this paper, we design a foundation model for mobile data forecasting, MobiGPT, with a unified structure capable of forecasting three data types: base station traffic, user app usage, and channel quality. We propose a soft-prompt learning method to help the model understand features of different data types, and introduce a temporal masking mechanism to guide the model through three forecasting tasks: short-term prediction, long-term prediction, and distribution generation, supporting diverse optimization scenarios. Evaluations on real-world datasets with over 100,000 samples show that MobiGPT achieves accurate multi-type forecasting. Compared to existing models, it improves forecasting accuracy by 27.37%, 20.08%, and 7.27%, reflecting strong generalization. Moreover, MobiGPT exhibits superior zero/few-shot performance in unseen scenarios, with over 21.51% improvement, validating its strong transferability as a foundation model.

Method: Separately train experts, a text-to-computation module, and a router, then endogenously integrate computational capabilities into neural networks. The router directs computation and reasoning at token level during inference.

Result: PiMoE achieves higher accuracy than LLM finetuning and significant improvements in response latency, token usage, and GPU energy consumption compared to multi-agent approaches.

Conclusion: PiMoE offers an efficient, interpretable, and scalable paradigm for next-generation scientific or industrial intelligent systems by enabling seamless integration of computation and reasoning.

Abstract: Complex systems typically rely on high-precision numerical computation to support decisions, but current large language models (LLMs) cannot yet incorporate such computations as an intrinsic and interpretable capability with existing architectures. Mainstream multi-agent approaches can leverage external experts, but inevitably introduce communication overhead and suffer from inefficient multimodal emergent capability and limited scalability. To this end, we propose PiMoE (Physically-isolated Mixture of Experts), a training and inference architecture for integrating computation and reasoning. Instead of the workflow paradigm of tool invocation, PiMoE endogenously integrates computational capabilities into neural networks after separately training experts, a text-to-computation module, and a router. At inference, the router directs computation and reasoning at the token level, thereby enabling iterative alternation within a single chain of thought. We evaluate PiMoE on two reasoning-computation tasks against LLM finetuning and the multi-agent system approaches. Results show that the PiMoE architecture achieves not only higher accuracy than directly finetuning LLMs but also significant improvements in response latency, token usage, and GPU energy consumption compared with mainstream multi-agent approaches. PiMoE offers an efficient, interpretable, and scalable paradigm for next-generation scientific or industrial intelligent systems.

Method: FedIA employs a two-stage pipeline: (1) server-side top-ρ mask retains only the most informative 5% of gradient coordinates, and (2) lightweight influence-regularised momentum weight suppresses outlier clients. This projection-first approach denoises updates before aggregation.

Result: FedIA achieves smoother, more stable convergence and higher final accuracy than nine strong baselines on both homogeneous (Twitch Gamers) and heterogeneous (Wikipedia) graphs, with no extra uplink traffic and negligible server memory overhead.

Conclusion: FedIA’s projection-first strategy effectively mitigates domain skew in FGL, maintaining optimal convergence rates while being readily deployable due to its minimal resource requirements.

Abstract: Federated Graph Learning (FGL) under domain skew – as observed on platforms such as \emph{Twitch Gamers} and multilingual \emph{Wikipedia} networks – drives client models toward incompatible representations, rendering naive aggregation both unstable and ineffective. We find that the culprit is not the weighting scheme but the \emph{noisy gradient signal}: empirical analysis of baseline methods suggests that a vast majority of gradient dimensions can be dominated by domain-specific variance. We therefore shift focus from “aggregation-first” to a \emph{projection-first} strategy that denoises client updates \emph{before} they are combined. The proposed FedIA framework realises this \underline{I}mportance-\underline{A}ware idea through a two-stage, plug-and-play pipeline: (i) a server-side top-$\rho$ mask keeps only the most informative about 5% of coordinates, and (ii) a lightweight influence-regularised momentum weight suppresses outlier clients. FedIA adds \emph{no extra uplink traffic and only negligible server memory}, making it readily deployable. On both homogeneous (Twitch Gamers) and heterogeneous (Wikipedia) graphs, it yields smoother, more stable convergence and higher final accuracy than nine strong baselines. A convergence sketch further shows that dynamic projection maintains the optimal $\mathcal{O}(\sigma^{2}/\sqrt{T})$ rate.

Method: SBVR maps weight values to non-uniform representation points following LLM weight distributions, and uses a custom CUDA kernel for direct matrix-vector multiplication without decompression.

Result: SBVR demonstrates state-of-the-art perplexity and accuracy benchmarks while achieving 2.21x-3.04x end-to-end token-generation speedup over FP16 models in 4-bit quantization.

Conclusion: SBVR overcomes limitations of existing quantization methods by providing distribution-aware compression with hardware-friendly execution, enabling efficient deployment of large language models.

Abstract: With the advent of large language models (LLMs), numerous Post-Training Quantization (PTQ) strategies have been proposed to alleviate deployment barriers created by their enormous parameter counts. Quantization achieves compression by limiting the number of representable points in the data. Therefore, the key to achieving efficient quantization is selecting the optimal combination of representation points, or codes, for the given data. Existing PTQ solutions adopt two major approaches to this problem: Round-To-Nearest (RTN)-based methods and codebook-based methods. RTN-based methods map LLM weights onto uniformly distributed integer grids, failing to account for the Gaussian-like weight distribution of LLM weights. Codebook-based methods mitigate this issue by constructing distribution-aware codebooks; however, they suffer from random and strided memory access patterns, resulting in degraded inference speed that is exacerbated by the limited size of GPU L1 cache. To overcome these limitations, we propose a novel LLM quantization method, SBVR (Summation of BitVector Representation), that enables Gaussian-like code representation in a hardware-friendly manner for fast inference. SBVR maps weight values to non-uniform representation points whose distribution follows the actual distribution of LLM weights, enabling more accurate compression. Additionally, we design a custom CUDA kernel that allows matrix-vector multiplication directly in the SBVR format without decompression, thereby enabling high-performance execution of SBVR-compressed models. Our evaluations of SBVR on various models demonstrate state-of-the-art perplexity and accuracy benchmark performance while delivering a 2.21x- 3.04x end-to-end token-generation speedup over naive FP16 models in the 4-bit quantization regime.

Method: Created a large-scale evaluation dataset of 36,000 routes from 12 metropolises, introduced PathBuilder tool for converting between natural language and navigation routes, and proposed a new evaluation framework with metrics to assess 11 SOTA LLMs on route reversal tasks.

Result: LLMs exhibit significant limitations in reversing routes - most reverse routes neither return to the starting point nor are similar to the optimal route. LLMs also show low robustness in route generation and high confidence for incorrect answers.

Conclusion: The benchmark reveals fundamental challenges in LLMs’ geospatial cognition, particularly in route reversal tasks, highlighting the need for improved spatial reasoning capabilities in language models.

Abstract: Humans can interpret geospatial information through natural language, while the geospatial cognition capabilities of Large Language Models (LLMs) remain underexplored. Prior research in this domain has been constrained by non-quantifiable metrics, limited evaluation datasets and unclear research hierarchies. Therefore, we propose a large-scale benchmark and conduct a comprehensive evaluation of the geospatial route cognition of LLMs. We create a large-scale evaluation dataset comprised of 36000 routes from 12 metropolises worldwide. Then, we introduce PathBuilder, a novel tool for converting natural language instructions into navigation routes, and vice versa, bridging the gap between geospatial information and natural language. Finally, we propose a new evaluation framework and metrics to rigorously assess 11 state-of-the-art (SOTA) LLMs on the task of route reversal. The benchmark reveals that LLMs exhibit limitation to reverse routes: most reverse routes neither return to the starting point nor are similar to the optimal route. Additionally, LLMs face challenges such as low robustness in route generation and high confidence for their incorrect answers. Code\ &\ Data available here: \href{https://github.com/bghjmn32/EMNLP2025_Turnback}{TurnBack.}

Method: Multimodal chain-of-thought (MCoT) framework with three-step reasoning: spatial relation extraction, coordinate mapping to absolute directions, and user orientation inference. Uses curriculum learning on Taiwan-LLM-13B-v2.0-Chat model.

Result: MCoT achieves 100% accuracy on clean transcripts and 98.1% with ASR transcripts, outperforming baselines. Shows robustness to ASR errors, code-switching, domain shift, and linguistic variation.

Conclusion: Structured MCoT spatial reasoning provides an interpretable and resource-efficient path for embodied navigation, demonstrating strong performance in challenging conversational conditions.

Abstract: Conversational agents must translate egocentric utterances (e.g., “on my right”) into allocentric orientations (N/E/S/W). This challenge is particularly critical in indoor or complex facilities where GPS signals are weak and detailed maps are unavailable. While chain-of-thought (CoT) prompting has advanced reasoning in language and vision tasks, its application to multimodal spatial orientation remains underexplored. We introduce Conversational Orientation Reasoning (COR), a new benchmark designed for Traditional Chinese conversational navigation projected from real-world environments, addressing egocentric-to-allocentric reasoning in non-English and ASR-transcribed scenarios. We propose a multimodal chain-of-thought (MCoT) framework, which integrates ASR-transcribed speech with landmark coordinates through a structured three-step reasoning process: (1) extracting spatial relations, (2) mapping coordinates to absolute directions, and (3) inferring user orientation. A curriculum learning strategy progressively builds these capabilities on Taiwan-LLM-13B-v2.0-Chat, a mid-sized model representative of resource-constrained settings. Experiments show that MCoT achieves 100% orientation accuracy on clean transcripts and 98.1% with ASR transcripts, substantially outperforming unimodal and non-structured baselines. Moreover, MCoT demonstrates robustness under noisy conversational conditions, including ASR recognition errors and multilingual code-switching. The model also maintains high accuracy in cross-domain evaluation and resilience to linguistic variation, domain shift, and referential ambiguity. These findings highlight the potential of structured MCoT spatial reasoning as a path toward interpretable and resource-efficient embodied navigation.

Method: Variational task vector composition with latent composition coefficients estimated via Bayesian inference. Uses Spike-and-Slab prior to promote sparsity and gated sampling mechanism to filter coefficients based on uncertainty and importance for stable posterior construction.

Result: The method consistently outperforms existing approaches across all datasets by selectively leveraging the most reliable and informative components in task vectors.

Conclusion: The approach establishes a new standard for efficient and effective task vector composition, demonstrating practical value through improved transparency, generalization, and performance.

Abstract: Task vectors capture how a model changes during fine-tuning by recording the difference between pre-trained and task-specific weights. The composition of task vectors, a key operator in task arithmetic, enables models to integrate knowledge from multiple tasks without incurring additional inference costs. In this paper, we propose variational task vector composition, where composition coefficients are taken as latent variables and estimated in a Bayesian inference framework. Unlike previous methods that operate at the task level, our framework focuses on sample-specific composition. Motivated by the observation of structural redundancy in task vectors, we introduce a Spike-and-Slab prior that promotes sparsity and preserves only the most informative components. To further address the high variance and sampling inefficiency in sparse, high-dimensional spaces, we develop a gated sampling mechanism that constructs a controllable posterior by filtering the composition coefficients based on both uncertainty and importance. This yields a more stable and interpretable variational framework by deterministically selecting reliable task components, reducing sampling variance while improving transparency and generalization. Experimental results demonstrate that our method consistently outperforms existing approaches across all datasets by selectively leveraging the most reliable and informative components in task vectors. These findings highlight the practical value of our approach, establishing a new standard for efficient and effective task vector composition.

Method: Constructed MolPILE using an automated curation pipeline from 6 large-scale databases, followed by comprehensive analysis of current pretraining datasets and retraining existing models on MolPILE.

Result: Retraining existing models on MolPILE yields improvements in generalization performance, demonstrating the dataset’s effectiveness.

Conclusion: MolPILE provides a standardized resource that addresses the pressing need for high-quality pretraining datasets in chemoinformatics, similar to ImageNet’s role in computer vision.

Abstract: The size, diversity, and quality of pretraining datasets critically determine the generalization ability of foundation models. Despite their growing importance in chemoinformatics, the effectiveness of molecular representation learning has been hindered by limitations in existing small molecule datasets. To address this gap, we present MolPILE, large-scale, diverse, and rigorously curated collection of 222 million compounds, constructed from 6 large-scale databases using an automated curation pipeline. We present a comprehensive analysis of current pretraining datasets, highlighting considerable shortcomings for training ML models, and demonstrate how retraining existing models on MolPILE yields improvements in generalization performance. This work provides a standardized resource for model training, addressing the pressing need for an ImageNet-like dataset in molecular chemistry.

Method: Fine-tunes a single MTP head with position-shared weights on self-distilled data, integrates language-aware dynamic vocabulary compression to reduce computational overhead in drafting.

Result: Achieves average 2.03x speedup across seven benchmarks compared to standard next token prediction, outperforming vanilla MTP by 82% with lossless output quality.

Conclusion: FastMTP offers a practical, lightweight training solution that seamlessly integrates with existing inference frameworks for rapid LLM acceleration deployment.

Abstract: As large language models (LLMs) become increasingly powerful, the sequential nature of autoregressive generation creates a fundamental throughput bottleneck that limits the practical deployment. While Multi-Token Prediction (MTP) has demonstrated remarkable benefits for model training efficiency and performance, its inherent potential for inference acceleration remains largely unexplored. This paper introduces FastMTP, a simple yet effective method that improves multi-step draft quality by aligning MTP training with its inference pattern, significantly enhancing speculative decoding performance. Our approach fine-tunes a single MTP head with position-shared weights on self-distilled data, enabling it to capture dependencies among consecutive future tokens and maintain high acceptance rates across multiple recursive draft steps. By integrating language-aware dynamic vocabulary compression into the MTP head, we further reduce computational overhead in the drafting process. Experimental results across seven diverse benchmarks demonstrate that FastMTP achieves an average of 2.03x speedup compared to standard next token prediction with lossless output quality, outperforming vanilla MTP by 82%. FastMTP requires only lightweight training and seamlessly integrates with existing inference frameworks, offering a practical and rapidly deployable solution for accelerating LLM inference.

Method: Introduces a new non-i.i.d. degree metric to measure data heterogeneity, develops M-DSL algorithm for effective multi-worker selection, and provides theoretical convergence analysis.

Result: Extensive experiments show M-DSL improves performance and network intelligence beyond benchmarks in various heterogeneous datasets and non-i.i.d. settings.

Conclusion: M-DSL effectively addresses data heterogeneity challenges in distributed swarm learning, providing theoretical foundations and practical improvements for edge IoT applications.

Abstract: Recent advances in distributed swarm learning (DSL) offer a promising paradigm for edge Internet of Things. Such advancements enhance data privacy, communication efficiency, energy saving, and model scalability. However, the presence of non-independent and identically distributed (non-i.i.d.) data pose a significant challenge for multi-access edge computing, degrading learning performance and diverging training behavior of vanilla DSL. Further, there still lacks theoretical guidance on how data heterogeneity affects model training accuracy, which requires thorough investigation. To fill the gap, this paper first study the data heterogeneity by measuring the impact of non-i.i.d. datasets under the DSL framework. This then motivates a new multi-worker selection design for DSL, termed M-DSL algorithm, which works effectively with distributed heterogeneous data. A new non-i.i.d. degree metric is introduced and defined in this work to formulate the statistical difference among local datasets, which builds a connection between the measure of data heterogeneity and the evaluation of DSL performance. In this way, our M-DSL guides effective selection of multiple works who make prominent contributions for global model updates. We also provide theoretical analysis on the convergence behavior of our M-DSL, followed by extensive experiments on different heterogeneous datasets and non-i.i.d. data settings. Numerical results verify performance improvement and network intelligence enhancement provided by our M-DSL beyond the benchmarks.

Method: Frames exemplar selection as coverage maximization over reverse k-nearest neighbors with greedy approximation, then uses self-refining prompt strategy with LLMs to derive interpretable natural language rules from exemplar neighborhoods.

Result: Significantly outperforms existing methods across diverse benchmarks in fidelity, scalability, and human interpretability, validated by user study with 60 participants.

Conclusion: GnnXemplar provides an effective global explanation framework for GNNs that addresses limitations of existing methods and enhances trust through human-interpretable natural language explanations.

Abstract: Graph Neural Networks (GNNs) are widely used for node classification, yet their opaque decision-making limits trust and adoption. While local explanations offer insights into individual predictions, global explanation methods, those that characterize an entire class, remain underdeveloped. Existing global explainers rely on motif discovery in small graphs, an approach that breaks down in large, real-world settings where subgraph repetition is rare, node attributes are high-dimensional, and predictions arise from complex structure-attribute interactions. We propose GnnXemplar, a novel global explainer inspired from Exemplar Theory from cognitive science. GnnXemplar identifies representative nodes in the GNN embedding space, exemplars, and explains predictions using natural language rules derived from their neighborhoods. Exemplar selection is framed as a coverage maximization problem over reverse k-nearest neighbors, for which we provide an efficient greedy approximation. To derive interpretable rules, we employ a self-refining prompt strategy using large language models (LLMs). Experiments across diverse benchmarks show that GnnXemplar significantly outperforms existing methods in fidelity, scalability, and human interpretability, as validated by a user study with 60 participants.

Method: GETAD uses Graph Attention Networks to learn road-aware embeddings with graph-based positional encodings, a Transformer-based decoder for sequential movement modeling, and a multiobjective loss function combining autoregressive prediction and supervised link prediction with a novel CW NLL anomaly scoring function.

Result: Experiments on real-world and synthetic datasets show GETAD achieves consistent improvements over existing methods, particularly in detecting subtle anomalies in road-constrained environments.

Conclusion: Incorporating graph structure and contextual semantics into trajectory modeling enables more precise and context-aware anomaly detection, highlighting the benefits of network-aware approaches.

Abstract: Trajectory anomaly detection is essential for identifying unusual and unexpected movement patterns in applications ranging from intelligent transportation systems to urban safety and fraud prevention. Existing methods only consider limited aspects of the trajectory nature and its movement space by treating trajectories as sequences of sampled locations, with sampling determined by positioning technology, e.g., GPS, or by high-level abstractions such as staypoints. Trajectories are analyzed in Euclidean space, neglecting the constraints and connectivity information of the underlying movement network, e.g., road or transit networks. The proposed Graph Enhanced Trajectory Anomaly Detection (GETAD) framework tightly integrates road network topology, segment semantics, and historical travel patterns to model trajectory data. GETAD uses a Graph Attention Network to learn road-aware embeddings that capture both physical attributes and transition behavior, and augments these with graph-based positional encodings that reflect the spatial layout of the road network. A Transformer-based decoder models sequential movement, while a multiobjective loss function combining autoregressive prediction and supervised link prediction ensures realistic and structurally coherent representations. To improve the robustness of anomaly detection, we introduce Confidence Weighted Negative Log Likelihood (CW NLL), an anomaly scoring function that emphasizes high-confidence deviations. Experiments on real-world and synthetic datasets demonstrate that GETAD achieves consistent improvements over existing methods, particularly in detecting subtle anomalies in road-constrained environments. These results highlight the benefits of incorporating graph structure and contextual semantics into trajectory modeling, enabling more precise and context-aware anomaly detection.

Method: The authors conduct a case study and prove that when a Transformer is pretrained for policy evaluation, one of the global minimizers of the pretraining loss can enable in-context temporal difference learning.

Result: The paper provides initial support for the hypothesis that parameters capable of ICRL are minimizers of the pretraining loss, specifically demonstrating this through theoretical analysis of policy evaluation pretraining.

Conclusion: The study concludes that minimizers of the pretraining loss in RL algorithms can indeed enable in-context RL capabilities, offering theoretical validation for why ICRL emerges from standard pretraining procedures.

Abstract: Typically, a modern reinforcement learning (RL) agent solves a task by updating its neural network parameters to adapt its policy to the task. Recently, it has been observed that some RL agents can solve a wide range of new out-of-distribution tasks without parameter updates after pretraining on some task distribution. When evaluated in a new task, instead of making parameter updates, the pretrained agent conditions its policy on additional input called the context, e.g., the agent’s interaction history in the new task. The agent’s performance increases as the information in the context increases, with the agent’s parameters fixed. This phenomenon is typically called in-context RL (ICRL). The pretrained parameters of the agent network enable the remarkable ICRL phenomenon. However, many ICRL works perform the pretraining with standard RL algorithms. This raises the central question this paper aims to address: Why can the RL pretraining algorithm generate network parameters that enable ICRL? We hypothesize that the parameters capable of ICRL are minimizers of the pretraining loss. This work provides initial support for this hypothesis through a case study. In particular, we prove that when a Transformer is pretrained for policy evaluation, one of the global minimizers of the pretraining loss can enable in-context temporal difference learning.

Method: The study examines optimizers chronologically, presenting detailed update rules, explaining associated concepts and variables, discussing techniques applied, their contributions to optimization, and default hyperparameter settings.

Result: A comprehensive resource is created that covers the distinctive features of various optimizers and provides insights into open challenges in deep learning optimization.

Conclusion: The review serves as a valuable reference for understanding the current state of optimizers and identifying potential areas for future development in deep learning optimization.

Abstract: Deep learning optimizers are optimization algorithms that enable deep neural networks to learn. The effectiveness of learning is highly dependent on the optimizer employed in the training process. Alongside the rapid advancement of deep learning, a wide range of optimizers with different approaches have been developed. This study aims to provide a review of various optimizers that have been proposed and received attention in the literature. From Stochastic gradient descent to the most recent ones such as Momentum, AdamW, Sophia, and Muon in chronological order, optimizers are examined individually, and their distinctive features are highlighted in the study. The update rule of each optimizer is presented in detail, with an explanation of the associated concepts and variables. The techniques applied by these optimizers, their contributions to the optimization process, and their default hyperparameter settings are also discussed. In addition, insights are offered into the open challenges encountered in the optimization of deep learning models. Thus, a comprehensive resource is provided both for understanding the current state of optimizers and for identifying potential areas of future development.

Method: Introduces Reverse-CGR (R-CGR) with complete sequence recovery through explicit path encoding combined with rational arithmetic precision control, maintaining both positional and character information at each step.

Result: Achieves competitive performance on biological sequence classification tasks compared to traditional methods while providing interpretable geometric visualizations and generating feature-rich images suitable for deep learning.

Conclusion: R-CGR opens new avenues for interpretable bioinformatics analysis by enabling both accurate classification and complete sequence recovery, addressing a fundamental limitation of traditional geometric approaches.

Abstract: We present a novel information-preserving Chaos Game Representation (CGR) method, also called Reverse-CGR (R-CGR), for biological sequence analysis that addresses the fundamental limitation of traditional CGR approaches - the loss of sequence information during geometric mapping. Our method introduces complete sequence recovery through explicit path encoding combined with rational arithmetic precision control, enabling perfect sequence reconstruction from stored geometric traces. Unlike purely geometric approaches, our reversibility is achieved through comprehensive path storage that maintains both positional and character information at each step. We demonstrate the effectiveness of R-CGR on biological sequence classification tasks, achieving competitive performance compared to traditional sequence-based methods while providing interpretable geometric visualizations. The approach generates feature-rich images suitable for deep learning while maintaining complete sequence information through explicit encoding, opening new avenues for interpretable bioinformatics analysis where both accuracy and sequence recovery are essential.

Method: Uses Kolmogorov-Arnold Networks for flexible reward function estimation from low-fall-risk older adults (experts), combined with diffusion-based policies in an Actor-Critic framework for action refinement in offline RL.

Result: KANDI outperforms state-of-the-art methods on D4RL benchmark and shows practical application success in a clinical trial for physical activity promotion among older adults.

Conclusion: KANDI effectively addresses key challenges in offline RL for healthcare applications, offering a promising solution for activity promotion intervention strategies.

Abstract: Utilizing offline reinforcement learning (RL) with real-world clinical data is getting increasing attention in AI for healthcare. However, implementation poses significant challenges. Defining direct rewards is difficult, and inverse RL (IRL) struggles to infer accurate reward functions from expert behavior in complex environments. Offline RL also encounters challenges in aligning learned policies with observed human behavior in healthcare applications. To address challenges in applying offline RL to physical activity promotion for older adults at high risk of falls, based on wearable sensor activity monitoring, we introduce Kolmogorov-Arnold Networks and Diffusion Policies for Offline Inverse Reinforcement Learning (KANDI). By leveraging the flexible function approximation in Kolmogorov-Arnold Networks, we estimate reward functions by learning free-living environment behavior from low-fall-risk older adults (experts), while diffusion-based policies within an Actor-Critic framework provide a generative approach for action refinement and efficiency in offline RL. We evaluate KANDI using wearable activity monitoring data in a two-arm clinical trial from our Physio-feedback Exercise Program (PEER) study, emphasizing its practical application in a fall-risk intervention program to promote physical activity among older adults. Additionally, KANDI outperforms state-of-the-art methods on the D4RL benchmark. These results underscore KANDI’s potential to address key challenges in offline RL for healthcare applications, offering an effective solution for activity promotion intervention strategies in healthcare.

Method: MeshODENet integrates spatial reasoning capabilities of Graph Neural Networks with continuous-time modeling of Neural Ordinary Differential Equations to create a hybrid framework for simulating structural mechanics.

Result: The framework demonstrates superior performance on challenging structural mechanics problems involving 1D and 2D elastic bodies with large non-linear deformations, showing significant improvements in long-term predictive accuracy and stability compared to baseline models.

Conclusion: MeshODENet provides a powerful and generalizable data-driven approach for accelerating the analysis and modeling of complex structural systems, offering substantial computational advantages over traditional solvers.

Abstract: The simulation of complex physical systems using a discretized mesh is a cornerstone of applied mechanics, but traditional numerical solvers are often computationally prohibitive for many-query tasks. While Graph Neural Networks (GNNs) have emerged as powerful surrogate models for mesh-based data, their standard autoregressive application for long-term prediction is often plagued by error accumulation and instability. To address this, we introduce MeshODENet, a general framework that synergizes the spatial reasoning of GNNs with the continuous-time modeling of Neural Ordinary Differential Equations. We demonstrate the framework’s effectiveness and versatility on a series of challenging structural mechanics problems, including one- and two-dimensional elastic bodies undergoing large, non-linear deformations. The results demonstrate that our approach significantly outperforms baseline models in long-term predictive accuracy and stability, while achieving substantial computational speed-ups over traditional solvers. This work presents a powerful and generalizable approach for developing data-driven surrogates to accelerate the analysis and modeling of complex structural systems.

Method: Uses graph neural network surrogate to predict preconditioning speed from matrix A and MCMC parameters, then applies Bayesian acquisition function to select optimal parameter sets.

Result: On unseen ill-conditioned systems, the framework achieves better preconditioning with 50% of conventional search budget, yielding ~10% reduction in iterations to convergence.

Conclusion: Provides an effective route for incorporating MCMC-based preconditioners into large-scale linear systems through AI-driven parameter optimization.

Abstract: Large, sparse linear systems are pervasive in modern science and engineering, and Krylov subspace solvers are an established means of solving them. Yet convergence can be slow for ill-conditioned matrices, so practical deployments usually require preconditioners. Markov chain Monte Carlo (MCMC)-based matrix inversion can generate such preconditioners and accelerate Krylov iterations, but its effectiveness depends on parameters whose optima vary across matrices; manual or grid search is costly. We present an AI-driven framework recommending MCMC parameters for a given linear system. A graph neural surrogate predicts preconditioning speed from $A$ and MCMC parameters. A Bayesian acquisition function then chooses the parameter sets most likely to minimise iterations. On a previously unseen ill-conditioned system, the framework achieves better preconditioning with 50% of the search budget of conventional methods, yielding about a 10% reduction in iterations to convergence. These results suggest a route for incorporating MCMC-based preconditioners into large-scale systems.

Method: Proposes GluMind with two parallel attention mechanisms: cross-attention for integrating blood glucose data with other physiological/behavioral signals, and multi-scale attention for capturing long-range temporal dependencies. Includes knowledge retention technique to mitigate catastrophic forgetting.

Result: Evaluated on AIREADI dataset, GluMind outperforms state-of-the-art models with approximately 15% improvement in RMSE and 9% improvement in MAE, demonstrating stable performance and adaptability across different patient cohorts.

Conclusion: GluMind effectively addresses multimodal integration and continual learning challenges in blood glucose forecasting, providing robust and accurate predictions suitable for real-world healthcare applications.

Abstract: This paper proposes GluMind, a transformer-based multimodal framework designed for continual and long-term blood glucose forecasting. GluMind devises two attention mechanisms, including cross-attention and multi-scale attention, which operate in parallel and deliver accurate predictive performance. Cross-attention effectively integrates blood glucose data with other physiological and behavioral signals such as activity, stress, and heart rate, addressing challenges associated with varying sampling rates and their adverse impacts on robust prediction. Moreover, the multi-scale attention mechanism captures long-range temporal dependencies. To mitigate catastrophic forgetting, GluMind incorporates a knowledge retention technique into the transformer-based forecasting model. The knowledge retention module not only enhances the model’s ability to retain prior knowledge but also boosts its overall forecasting performance. We evaluate GluMind on the recently released AIREADI dataset, which contains behavioral and physiological data collected from healthy people, individuals with prediabetes, and those with type 2 diabetes. We examine the performance stability and adaptability of GluMind in learning continuously as new patient cohorts are introduced. Experimental results show that GluMind consistently outperforms other state-of-the-art forecasting models, achieving approximately 15% and 9% improvements in root mean squared error (RMSE) and mean absolute error (MAE), respectively.

Method: Developed Probabilistic Geometric PCA (PGPCA) that incorporates knowledge of nonlinear manifolds, derives geometric coordinate systems to capture deviations from the manifold, and uses an EM algorithm for parameter learning.

Result: PGPCA effectively models data distributions around various manifolds, outperforms PPCA for manifold-distributed data, and provides statistical testing for comparing geometric vs. Euclidean coordinate systems.

Conclusion: PGPCA enhances dimensionality reduction for high-dimensional data with nonlinear manifold structures and noise, offering valuable capabilities for neuroscience and other scientific domains.

Abstract: Dimensionality reduction is critical across various domains of science including neuroscience. Probabilistic Principal Component Analysis (PPCA) is a prominent dimensionality reduction method that provides a probabilistic approach unlike the deterministic approach of PCA and serves as a connection between PCA and Factor Analysis (FA). Despite their power, PPCA and its extensions are mainly based on linear models and can only describe the data in a Euclidean coordinate system. However, in many neuroscience applications, data may be distributed around a nonlinear geometry (i.e., manifold) rather than lying in the Euclidean space. We develop Probabilistic Geometric Principal Component Analysis (PGPCA) for such datasets as a new dimensionality reduction algorithm that can explicitly incorporate knowledge about a given nonlinear manifold that is first fitted from these data. Further, we show how in addition to the Euclidean coordinate system, a geometric coordinate system can be derived for the manifold to capture the deviations of data from the manifold and noise. We also derive a data-driven EM algorithm for learning the PGPCA model parameters. As such, PGPCA generalizes PPCA to better describe data distributions by incorporating a nonlinear manifold geometry. In simulations and brain data analyses, we show that PGPCA can effectively model the data distribution around various given manifolds and outperforms PPCA for such data. Moreover, PGPCA provides the capability to test whether the new geometric coordinate system better describes the data than the Euclidean one. Finally, PGPCA can perform dimensionality reduction and learn the data distribution both around and on the manifold. These capabilities make PGPCA valuable for enhancing the efficacy of dimensionality reduction for analysis of high-dimensional data that exhibit noise and are distributed around a nonlinear manifold.

Method: APRIL over-provisions rollout requests, terminates generation once the target number of responses is reached, and recycles incomplete responses for continuation in future steps. This ensures no rollouts are discarded while substantially reducing GPU idle time.

Result: APRIL improves rollout throughput by up to 44% across RL algorithms (GRPO, DAPO, GSPO), accelerates convergence, and achieves up to 8% higher final accuracy across tasks. It’s framework and hardware agnostic, already integrated into slime RL framework.

Conclusion: APRIL unifies system-level and algorithmic considerations to advance RL training efficiency, addressing the scalability bottleneck in large-scale RL training for language models.

Abstract: Reinforcement learning (RL) has become a cornerstone in advancing large-scale pre-trained language models (LLMs). Successive generations, including GPT-o series, DeepSeek-R1, Kimi-K1.5, Grok 4, and GLM-4.5, have relied on large-scale RL training to enhance reasoning and coding capabilities. To meet the community’s growing RL needs, numerous RL frameworks have been proposed. Most of these frameworks primarily rely on inference engines for rollout generation and training engines for policy updates. However, RL training remains computationally expensive, with rollout generation accounting for more than 90% of total runtime. In addition, its efficiency is often constrained by the long-tail distribution of rollout response lengths, where a few lengthy responses stall entire batches, leaving GPUs idle and underutilized. As model and rollout sizes continue to grow, this bottleneck increasingly limits scalability. To address this challenge, we propose Active Partial Rollouts in Reinforcement Learning (APRIL), which mitigates long-tail inefficiency. In the rollout phase, APRIL over-provisions rollout requests, terminates once the target number of responses is reached, and recycles incomplete responses for continuation in future steps. This strategy ensures that no rollouts are discarded while substantially reducing GPU idle time. Experiments show that APRIL improves rollout throughput by at most 44% across commonly used RL algorithms (GRPO, DAPO, GSPO), accelerates convergence, and achieves at most 8% higher final accuracy across tasks. Moreover, APRIL is both framework and hardware agnostic, already integrated into the slime RL framework, and deployable on NVIDIA and AMD GPUs alike. Taken together, this work unifies system-level and algorithmic considerations in proposing APRIL, with the aim of advancing RL training efficiency and inspiring further optimizations in RL systems.

Method: Uses novel parsimonious and computationally efficient nonlinearities to build non-uniform vector quantizers that are individually learned for each indexed vector.

Result: NVQ exhibits improved accuracy compared to state-of-the-art methods with minimal computational cost.

Conclusion: NVQ provides an efficient solution for compressing high-dimensional embedding vectors while maintaining search accuracy.

Abstract: Embedding vectors are widely used for representing unstructured data and searching through it for semantically similar items. However, the large size of these vectors, due to their high-dimensionality, creates problems for modern vector search techniques: retrieving large vectors from memory/storage is expensive and their footprint is costly. In this work, we present NVQ (non-uniform vector quantization), a new vector compression technique that is computationally and spatially efficient in the high-fidelity regime. The core in NVQ is to use novel parsimonious and computationally efficient nonlinearities for building non-uniform vector quantizers. Critically, these quantizers are \emph{individually} learned for each indexed vector. Our experimental results show that NVQ exhibits improved accuracy compared to the state of the art with a minimal computational cost.

Method: Uses standard transformer blocks with adaptive layers, trained via generative flow-matching objective with structural term. Scaled to 3B parameters on ~9M protein structures.

Result: Achieves competitive performance on standard folding benchmarks and strong ensemble prediction capabilities. Efficient deployment on consumer hardware.

Conclusion: SimpleFold demonstrates that complex domain-specific architectures are not essential for high-performance protein folding, opening new design possibilities.

Abstract: Protein folding models have achieved groundbreaking results typically via a combination of integrating domain knowledge into the architectural blocks and training pipelines. Nonetheless, given the success of generative models across different but related problems, it is natural to question whether these architectural designs are a necessary condition to build performant models. In this paper, we introduce SimpleFold, the first flow-matching based protein folding model that solely uses general purpose transformer blocks. Protein folding models typically employ computationally expensive modules involving triangular updates, explicit pair representations or multiple training objectives curated for this specific domain. Instead, SimpleFold employs standard transformer blocks with adaptive layers and is trained via a generative flow-matching objective with an additional structural term. We scale SimpleFold to 3B parameters and train it on approximately 9M distilled protein structures together with experimental PDB data. On standard folding benchmarks, SimpleFold-3B achieves competitive performance compared to state-of-the-art baselines, in addition SimpleFold demonstrates strong performance in ensemble prediction which is typically difficult for models trained via deterministic reconstruction objectives. Due to its general-purpose architecture, SimpleFold shows efficiency in deployment and inference on consumer-level hardware. SimpleFold challenges the reliance on complex domain-specific architectures designs in protein folding, opening up an alternative design space for future progress.

Method: Two-stage framework: 1) Training-free harmonization via layer-aware fusion strategy and activation-based functional alignment to address parameter misalignment; 2) Lightweight router training to coordinate the architecture.

Result: Successfully integrates experts from heterogeneous sources, achieving MoE models that significantly surpass baselines in multi-domain tasks and out-of-distribution generalization.

Conclusion: The proposed Symphony-MoE framework effectively creates powerful MoE models by harmonizing diverse experts from multiple pre-trained models, overcoming the limitations of single-source upcycling.

Abstract: Mixture-of-Experts (MoE) models enable scalable performance by activating large parameter sets sparsely, minimizing computational overhead. To circumvent the prohibitive cost of training MoEs from scratch, recent work employs upcycling, reusing a single pre-trained dense model by replicating its feed-forward network (FFN) layers into experts. However, this limits expert diversity, as all experts originate from a single pre-trained dense model. This paper addresses this limitation by constructing powerful MoE models using experts sourced from multiple identically-architected but disparate pre-trained models (e.g., Llama2-Chat and Code Llama). A key challenge lies in the fact that these source models occupy disparate, dissonant regions of the parameter space, making direct upcycling prone to severe performance degradation. To overcome this, we propose Symphony-MoE, a novel two-stage framework designed to harmonize these models into a single, coherent expert mixture. First, we establish this harmony in a training-free manner: we construct a shared backbone via a layer-aware fusion strategy and, crucially, alleviate parameter misalignment among experts using activation-based functional alignment. Subsequently, a single lightweight stage of router training coordinates the entire architecture. Experiments demonstrate that our method successfully integrates experts from heterogeneous sources, achieving an MoE model that significantly surpasses baselines in multi-domain tasks and out-of-distribution generalization.

Method: Introduce Kolmogorov Arnold Networks (KANs) augmented with physics-informed loss functions that enforce Ehrenfest theorems, plus Chain of KANs architecture that embeds temporal causality for time series modeling.

Result: Achieves superior accuracy with only 200 training samples (5.4% of the 3,700 samples required by Temporal Convolution Networks), maintaining mathematical rigor and physical consistency.

Conclusion: Physics-informed KANs offer compelling advantages over conventional black-box models by dramatically reducing data requirements while preserving physical interpretability and accuracy in quantum dynamics prediction.

Abstract: The prediction of quantum dynamical responses lies at the heart of modern physics. Yet, modeling these time-dependent behaviors remains a formidable challenge because quantum systems evolve in high-dimensional Hilbert spaces, often rendering traditional numerical methods computationally prohibitive. While large language models have achieved remarkable success in sequential prediction, quantum dynamics presents a fundamentally different challenge: forecasting the entire temporal evolution of quantum systems rather than merely the next element in a sequence. Existing neural architectures such as recurrent and convolutional networks often require vast training datasets and suffer from spurious oscillations that compromise physical interpretability. In this work, we introduce a fundamentally new approach: Kolmogorov Arnold Networks (KANs) augmented with physics-informed loss functions that enforce the Ehrenfest theorems. Our method achieves superior accuracy with significantly less training data: it requires only 5.4 percent of the samples (200) compared to Temporal Convolution Networks (3,700). We further introduce the Chain of KANs, a novel architecture that embeds temporal causality directly into the model design, making it particularly well-suited for time series modeling. Our results demonstrate that physics-informed KANs offer a compelling advantage over conventional black-box models, maintaining both mathematical rigor and physical consistency while dramatically reducing data requirements.

Method: Theoretical analysis using novel combinatorial objects called (m, b_rel)-Constellations, which are related to spherical codes and parametrized by margin m and relative bias b_rel. Also proposes a reparameterization of the sigmoid loss with explicit relative bias.

Result: Characterization of constellations that theoretically justifies SigLIP’s success on retrieval, explains the modality gap, and identifies necessary dimensions for high-quality representations. Improved training dynamics demonstrated with synthetic data.

Conclusion: The theoretical framework provides insights into why SigLIP models work effectively, and the proposed reparameterization enhances training performance, contributing to better understanding and optimization of contrastive learning approaches.

Abstract: The meta-task of obtaining and aligning representations through contrastive pretraining is steadily gaining importance since its introduction in CLIP and ALIGN. In this paper we theoretically explain the advantages of synchronizing with trainable inverse temperature and bias under the sigmoid loss, as implemented in the recent SigLIP and SigLIP2 models of Google DeepMind. Temperature and bias can drive the loss function to zero for a rich class of configurations that we call $(\mathsf{m}, \mathsf{b}{\mathsf{rel}})$-Constellations. $(\mathsf{m}, \mathsf{b}{\mathsf{rel}})$-Constellations are a novel combinatorial object related to spherical codes and are parametrized by a margin $\mathsf{m}$ and relative bias $\mathsf{b}_{\mathsf{rel}}$. We use our characterization of constellations to theoretically justify the success of SigLIP on retrieval, to explain the modality gap present in SigLIP, and to identify the necessary dimension for producing high-quality representations. Finally, we propose a reparameterization of the sigmoid loss with explicit relative bias, which improves training dynamics in experiments with synthetic data.

Method: The approach combines synthetically generated data (which mimics real data statistics while preserving privacy) with publicly available real-world features to create hybrid datasets for training Graph Neural Network AML models.

Result: Hybrid datasets demonstrate improved model utility compared to purely synthetic datasets while maintaining privacy and confidentiality protections.

Conclusion: Hybrid datasets offer a practical solution for financial institutions to enhance AML systems by balancing privacy concerns with model effectiveness through strategic data augmentation.

Abstract: Money laundering is a critical global issue for financial institutions. Automated Anti-money laundering (AML) models, like Graph Neural Networks (GNN), can be trained to identify illicit transactions in real time. A major issue for developing such models is the lack of access to training data due to privacy and confidentiality concerns. Synthetically generated data that mimics the statistical properties of real data but preserves privacy and confidentiality has been proposed as a solution. However, training AML models on purely synthetic datasets presents its own set of challenges. This article proposes the use of hybrid datasets to augment the utility of synthetic datasets by incorporating publicly available, easily accessible, and real-world features. These additions demonstrate that hybrid datasets not only preserve privacy but also improve model utility, offering a practical pathway for financial institutions to enhance AML systems.

Method: ITT uses novel interaction topology to capture materials information across multiple scales (structural, elemental, atomic, pairwise-elemental) and integrates them through a transformer architecture with two-stage training: self-supervised pretraining on 0.6M unlabeled structures followed by supervised fine-tuning.

Result: ITT achieves state-of-the-art, accurate, and transferable predictions for adsorption, transport, and stability properties of porous materials.

Conclusion: The framework provides a principled and scalable path for learning-guided discovery in structurally and chemically diverse porous materials.

Abstract: Porous materials exhibit vast structural diversity and support critical applications in gas storage, separations, and catalysis. However, predictive modeling remains challenging due to the multiscale nature of structure-property relationships, where performance is governed by both local chemical environments and global pore-network topology. These complexities, combined with sparse and unevenly distributed labeled data, hinder generalization across material families. We propose the Interaction Topological Transformer (ITT), a unified data-efficient framework that leverages novel interaction topology to capture materials information across multiple scales and multiple levels, including structural, elemental, atomic, and pairwise-elemental organization. ITT extracts scale-aware features that reflect both compositional and relational structure within complex porous frameworks, and integrates them through a built-in Transformer architecture that supports joint reasoning across scales. Trained using a two-stage strategy, i.e., self-supervised pretraining on 0.6 million unlabeled structures followed by supervised fine-tuning, ITT achieves state-of-the-art, accurate, and transferable predictions for adsorption, transport, and stability properties. This framework provides a principled and scalable path for learning-guided discovery in structurally and chemically diverse porous materials.

Method: Reverse-Complement Consistency Regularization (RCCR) - a model-agnostic fine-tuning objective that directly penalizes divergence between predictions on a sequence and its reverse complement, evaluated across three DNA language model backbones on various genomic tasks.

Result: RCCR substantially improves reverse-complement robustness by dramatically reducing prediction flips and errors while maintaining or improving task accuracy compared to baseline methods like data augmentation and test-time averaging.

Conclusion: RCCR provides a computationally efficient fine-tuning recipe that integrates biological priors directly into learning, producing intrinsically robust models for diverse genomic tasks.

Abstract: A fundamental property of DNA is that the reverse complement (RC) of a sequence often carries identical biological meaning. However, state-of-the-art DNA language models frequently fail to capture this symmetry, producing inconsistent predictions for a sequence and its RC counterpart, which undermines their reliability. In this work, we introduce Reverse-Complement Consistency Regularization (RCCR), a simple and model-agnostic fine-tuning objective that directly penalizes the divergence between a model’s prediction on a sequence and the aligned prediction on its reverse complement. We evaluate RCCR across three diverse backbones (Nucleotide Transformer, HyenaDNA, DNABERT-2) on a wide range of genomic tasks, including sequence classification, scalar regression, and profile prediction. Our experiments show that RCCR substantially improves RC robustness by dramatically reducing prediction flips and errors, all while maintaining or improving task accuracy compared to baselines such as RC data augmentation and test-time averaging. By integrating a key biological prior directly into the learning process, RCCR produces a single, intrinsically robust, and computationally efficient model fine-tuning recipe for diverse biology tasks.

Method: The paper conducts a comprehensive review of XGNN applications in dementia research, examining their use across Alzheimer’s disease, Parkinson’s disease, mild cognitive impairment, and multi-disease diagnosis. It introduces a taxonomy of explainability methods and compares existing models in clinical scenarios.

Result: The review identifies that XGNNs enable identification of disease-relevant biomarkers, analysis of brain network disruptions, and provision of transparent insights for clinicians. It also highlights current challenges including limited generalizability and underexplored domains.

Conclusion: By outlining both progress and open problems, this review aims to guide future work toward trustworthy, clinically meaningful, and scalable use of XGNNs in dementia research, including potential integration with Large Language Models for early detection.

Abstract: Dementia is a progressive neurodegenerative disorder with multiple etiologies, including Alzheimer’s disease, Parkinson’s disease, frontotemporal dementia, and vascular dementia. Its clinical and biological heterogeneity makes diagnosis and subtype differentiation highly challenging. Graph Neural Networks (GNNs) have recently shown strong potential in modeling brain connectivity, but their limited robustness, data scarcity, and lack of interpretability constrain clinical adoption. Explainable Graph Neural Networks (XGNNs) have emerged to address these barriers by combining graph-based learning with interpretability, enabling the identification of disease-relevant biomarkers, analysis of brain network disruptions, and provision of transparent insights for clinicians. This paper presents the first comprehensive review dedicated to XGNNs in dementia research. We examine their applications across Alzheimer’s disease, Parkinson’s disease, mild cognitive impairment, and multi-disease diagnosis. A taxonomy of explainability methods tailored for dementia-related tasks is introduced, alongside comparisons of existing models in clinical scenarios. We also highlight challenges such as limited generalizability, underexplored domains, and the integration of Large Language Models (LLMs) for early detection. By outlining both progress and open problems, this review aims to guide future work toward trustworthy, clinically meaningful, and scalable use of XGNNs in dementia research.

Method: Proposes DS-Diffusion with: 1) Diffusion framework based on style-guided kernels to avoid retraining for specific conditions, 2) Time-information based hierarchical denoising mechanism (THD) to reduce distributional bias, 3) Clear indication of data style origins for interpretability.

Result: Experimental results show predictive score decreases by 5.56% and discriminative score decreases by 61.55% compared to state-of-the-art ImagenTime. Distributional bias is reduced, inference process is more interpretable, and model flexibility/adaptability is enhanced.

Conclusion: DS-Diffusion effectively addresses key limitations of existing diffusion models for time series generation, providing better performance, reduced bias, improved interpretability, and enhanced flexibility without requiring retraining.

Abstract: Diffusion models are the mainstream approach for time series generation tasks. However, existing diffusion models for time series generation require retraining the entire framework to introduce specific conditional guidance. There also exists a certain degree of distributional bias between the generated data and the real data, which leads to potential model biases in downstream tasks. Additionally, the complexity of diffusion models and the latent spaces leads to an uninterpretable inference process. To address these issues, we propose the data style-guided diffusion model (DS-Diffusion). In the DS-Diffusion, a diffusion framework based on style-guided kernels is developed to avoid retraining for specific conditions. The time-information based hierarchical denoising mechanism (THD) is developed to reduce the distributional bias between the generated data and the real data. Furthermore, the generated samples can clearly indicate the data style from which they originate. We conduct comprehensive evaluations using multiple public datasets to validate our approach. Experimental results show that, compared to the state-of-the-art model such as ImagenTime, the predictive score and the discriminative score decrease by 5.56% and 61.55%, respectively. The distributional bias between the generated data and the real data is further reduced, the inference process is also more interpretable. Moreover, by eliminating the need to retrain the diffusion model, the flexibility and adaptability of the model to specific conditions are also enhanced.

Method: The REBACT approach adds a critical reflect step before taking the next action, allowing for immediate error correction and better adaptation to environment feedback.

Result: REBACT significantly outperforms baselines with success rate improvements: 24% on WebShop (61%), 6.72% on ALFWorld (98.51%), and 0.5% on TextCraft (99.5%) using Claude3.5-sonnet.

Conclusion: The reflect-before-act paradigm effectively enhances LLM decision-making by enabling immediate error correction, achieving substantial performance gains with minimal computational overhead.

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in interactive decision-making tasks, but existing methods often struggle with error accumulation and lack robust self-correction mechanisms. We introduce “Reflect before Act” (REBACT), a novel approach that enhances LLM-based decision-making by introducing a critical reflect step prior to taking the next action. This approach allows for immediate error correction, ensuring smooth action path and adaptibity to environment feedback. We evaluate REBACT on three diverse interactive environments: ALFWorld, WebShop, and TextCraft. Our results demonstrate that REBACT significantly outperforms strong baselines, improving success rates by up to 24% on WebShop (achieving 61%), 6.72% on ALFWorld (achieving 98.51%), and 0.5% on TextCraft (achieving 99.5%) using Claude3.5-sonnet as the underlying LLM. Further analysis reveals that REBACT’s performance improvements are achieved with only a few modification steps, demonstrating its computational efficiency.

Method: Proposes Flow Marching algorithm that jointly samples noise level and physical time step, learns unified velocity field, uses Physics-Pretrained Variational Autoencoder (P2VAE) for state embedding, and Flow Marching Transformer (FMT) with diffusion-forcing scheme and latent temporal pyramids.

Result: Achieves up to 15x greater computational efficiency than full-length video diffusion models, enables large-scale pretraining on ~2.5M trajectories across 12 PDE families, demonstrates long-term rollout stability, and shows effective few-shot adaptation on unseen Kolmogorov turbulence.

Conclusion: The generative PDE foundation model approach is important for real-world applications, providing uncertainty-aware ensemble results and improved computational efficiency.

Abstract: Pretraining on large-scale collections of PDE-governed spatiotemporal trajectories has recently shown promise for building generalizable models of dynamical systems. Yet most existing PDE foundation models rely on deterministic Transformer architectures, which lack generative flexibility for many science and engineering applications. We propose Flow Marching, an algorithm that bridges neural operator learning with flow matching motivated by an analysis of error accumulation in physical dynamical systems, and we build a generative PDE foundation model on top of it. By jointly sampling the noise level and the physical time step between adjacent states, the model learns a unified velocity field that transports a noisy current state toward its clean successor, reducing long-term rollout drift while enabling uncertainty-aware ensemble generations. Alongside this core algorithm, we introduce a Physics-Pretrained Variational Autoencoder (P2VAE) to embed physical states into a compact latent space, and an efficient Flow Marching Transformer (FMT) that combines a diffusion-forcing scheme with latent temporal pyramids, achieving up to 15x greater computational efficiency than full-length video diffusion models and thereby enabling large-scale pretraining at substantially reduced cost. We curate a corpus of ~2.5M trajectories across 12 distinct PDE families and train suites of P2VAEs and FMTs at multiple scales. On downstream evaluation, we benchmark on unseen Kolmogorov turbulence with few-shot adaptation, demonstrate long-term rollout stability over deterministic counterparts, and present uncertainty-stratified ensemble results, highlighting the importance of generative PDE foundation models for real-world applications.

Method: HyperAdapt adapts pre-trained weight matrices by applying row- and column-wise scaling through diagonal matrices, requiring only n+m trainable parameters for an n×m matrix. This induces high-rank updates while being extremely parameter-efficient.

Result: Experiments on GLUE, arithmetic reasoning, and commonsense reasoning benchmarks with models up to 14B parameters show HyperAdapt matches or nearly matches full fine-tuning and state-of-the-art PEFT methods while using orders of magnitude fewer trainable parameters.

Conclusion: HyperAdapt provides an efficient alternative to full fine-tuning and existing PEFT methods, achieving comparable performance with significantly reduced parameter requirements, making it suitable for adapting large foundation models to specialized tasks.

Abstract: Foundation models excel across diverse tasks, but adapting them to specialized applications often requires fine-tuning, an approach that is memory and compute-intensive. Parameter-efficient fine-tuning (PEFT) methods mitigate this by updating only a small subset of weights. In this paper, we introduce HyperAdapt, a parameter-efficient fine-tuning method that significantly reduces the number of trainable parameters compared to state-of-the-art methods like LoRA. Specifically, HyperAdapt adapts a pre-trained weight matrix by applying row- and column-wise scaling through diagonal matrices, thereby inducing a high-rank update while requiring only $n+m$ trainable parameters for an $n \times m$ matrix. Theoretically, we establish an upper bound on the rank of HyperAdapt’s updates, and empirically, we confirm that it consistently induces high-rank transformations across model layers. Experiments on GLUE, arithmetic reasoning, and commonsense reasoning benchmarks with models up to 14B parameters demonstrate that HyperAdapt matches or nearly matches the performance of full fine-tuning and state-of-the-art PEFT methods while using orders of magnitude fewer trainable parameters.

Method: Uses Block Term Decomposition (BTD) of a third-order tensor constructed from input matrices to jointly estimate cluster memberships and partially shared subspaces, with scalable optimization algorithms for large datasets.

Result: Superior clustering accuracy and robustness compared to existing subspace clustering techniques, especially under high noise and interference, as demonstrated on real-world hyperspectral imaging datasets.

Conclusion: The proposed framework shows strong potential for challenging high-dimensional applications where data structure exists beyond individual vectors, offering improved performance in noisy environments.

Abstract: We introduce a novel framework for clustering a collection of tall matrices based on their column spaces, a problem we term Subspace Clustering of Subspaces (SCoS). Unlike traditional subspace clustering methods that assume vectorized data, our formulation directly models each data sample as a matrix and clusters them according to their underlying subspaces. We establish conceptual links to Subspace Clustering and Generalized Canonical Correlation Analysis (GCCA), and clarify key differences that arise in this more general setting. Our approach is based on a Block Term Decomposition (BTD) of a third-order tensor constructed from the input matrices, enabling joint estimation of cluster memberships and partially shared subspaces. We provide the first identifiability results for this formulation and propose scalable optimization algorithms tailored to large datasets. Experiments on real-world hyperspectral imaging datasets demonstrate that our method achieves superior clustering accuracy and robustness, especially under high noise and interference, compared to existing subspace clustering techniques. These results highlight the potential of the proposed framework in challenging high-dimensional applications where structure exists beyond individual data vectors.

Method: Created ApisTox dataset and conducted broad evaluation of ML models including molecular fingerprints, graph kernels, GNNs, and pretrained transformers for molecular graph classification.

Result: Methods successful in medicinal chemistry often fail to generalize to agrochemicals, highlighting need for domain-specific models and benchmarks.

Conclusion: Future work will develop comprehensive benchmarking suite and design ML models tailored to unique challenges of pesticide discovery.

Abstract: This research focuses on rational pesticide design, using graph machine learning to accelerate the development of safer, eco-friendly agrochemicals, inspired by in silico methods in drug discovery. With an emphasis on ecotoxicology, the initial contributions include the creation of ApisTox, the largest curated dataset on pesticide toxicity to honey bees. We conducted a broad evaluation of machine learning (ML) models for molecular graph classification, including molecular fingerprints, graph kernels, GNNs, and pretrained transformers. The results show that methods successful in medicinal chemistry often fail to generalize to agrochemicals, underscoring the need for domain-specific models and benchmarks. Future work will focus on developing a comprehensive benchmarking suite and designing ML models tailored to the unique challenges of pesticide discovery.

Method: The authors formally establish a generalized bisimulation metric (GBSM) between pairs of MDPs, proving three fundamental properties: symmetry, inter-MDP triangle inequality, and distance bounds on identical state spaces.

Result: GBSM enables tighter theoretical bounds for policy transfer, state aggregation, and sampling-based estimation compared to standard BSM, and provides a closed-form sample complexity for estimation that improves upon existing asymptotic results.

Conclusion: The proposed GBSM framework successfully extends bisimulation metrics to multi-MDP scenarios with rigorous mathematical foundations, validated by numerical results showing its effectiveness in practical applications.

Abstract: The bisimulation metric (BSM) is a powerful tool for computing state similarities within a Markov decision process (MDP), revealing that states closer in BSM have more similar optimal value functions. While BSM has been successfully utilized in reinforcement learning (RL) for tasks like state representation learning and policy exploration, its application to multiple-MDP scenarios, such as policy transfer, remains challenging. Prior work has attempted to generalize BSM to pairs of MDPs, but a lack of rigorous analysis of its mathematical properties has limited further theoretical progress. In this work, we formally establish a generalized bisimulation metric (GBSM) between pairs of MDPs, which is rigorously proven with the three fundamental properties: GBSM symmetry, inter-MDP triangle inequality, and the distance bound on identical state spaces. Leveraging these properties, we theoretically analyse policy transfer, state aggregation, and sampling-based estimation in MDPs, obtaining explicit bounds that are strictly tighter than those derived from the standard BSM. Additionally, GBSM provides a closed-form sample complexity for estimation, improving upon existing asymptotic results based on BSM. Numerical results validate our theoretical findings and demonstrate the effectiveness of GBSM in multi-MDP scenarios.

Method: The approach frames transaction risk as a multi-step Markov Decision Process (MDP) and uses LLMs to iteratively refine reward functions for RL models, enabling better fraud detection with zero-shot capability.

Result: Experiments with real-world data demonstrate improved fraud detection accuracy, robustness, and resilience through long-term evaluations of the LLM-enhanced RL framework.

Conclusion: The integration of LLMs with RL shows significant potential for advancing industrial RL applications, particularly in complex domains like e-commerce fraud detection.

Abstract: This paper presents a novel approach to e-commerce payment fraud detection by integrating reinforcement learning (RL) with Large Language Models (LLMs). By framing transaction risk as a multi-step Markov Decision Process (MDP), RL optimizes risk detection across multiple payment stages. Crafting effective reward functions, essential for RL model success, typically requires significant human expertise due to the complexity and variability in design. LLMs, with their advanced reasoning and coding capabilities, are well-suited to refine these functions, offering improvements over traditional methods. Our approach leverages LLMs to iteratively enhance reward functions, achieving better fraud detection accuracy and demonstrating zero-shot capability. Experiments with real-world data confirm the effectiveness, robustness, and resilience of our LLM-enhanced RL framework through long-term evaluations, underscoring the potential of LLMs in advancing industrial RL applications.

Method: Introducing periodic CNN architecture with periodic boundary conditions in convolutional layers, and proving theoretical approximation theorems for ridge function approximation.

Result: Periodic CNNs can approximate ridge functions depending on d-1 linear variables in d-dimensional space, but not with fewer variables, providing a sharp characterization of their expressive power.

Conclusion: Periodic CNNs expand CNN approximation theory and are well-suited for applications with ridge-like high-dimensional structures, such as wrapped domain image analysis, physics-informed learning, and materials science.

Abstract: We introduce a novel convolutional neural network architecture, termed the \emph{periodic CNN}, which incorporates periodic boundary conditions into the convolutional layers. Our main theoretical contribution is a rigorous approximation theorem: periodic CNNs can approximate ridge functions depending on $d-1$ linear variables in a $d$-dimensional input space, while such approximation is impossible in lower-dimensional ridge settings ($d-2$ or fewer variables). This result establishes a sharp characterization of the expressive power of periodic CNNs. Beyond the theory, our findings suggest that periodic CNNs are particularly well-suited for problems where data naturally admits a ridge-like structure of high intrinsic dimension, such as image analysis on wrapped domains, physics-informed learning, and materials science. The work thus both expands the mathematical foundation of CNN approximation theory and highlights a class of architectures with surprising and practically relevant approximation capabilities.

Method: Proposes MOMEMTO with patch-based memory module that initializes memory items from pre-trained encoder, organizes them into patch-level units, and updates via attention mechanism. Uses multi-domain training strategy for joint fine-tuning across datasets.

Result: Achieves higher AUC and VUS metrics compared to baselines on 23 univariate datasets. Enhances backbone TFM performance, especially in few-shot learning scenarios.

Conclusion: MOMEMTO effectively addresses over-generalization in time series anomaly detection through memory-enhanced foundation model, demonstrating superior performance as a single multi-domain model.

Abstract: Recently reconstruction-based deep models have been widely used for time series anomaly detection, but as their capacity and representation capability increase, these models tend to over-generalize, often reconstructing unseen anomalies accurately. Prior works have attempted to mitigate this by incorporating a memory architecture that stores prototypes of normal patterns. Nevertheless, these approaches suffer from high training costs and have yet to be effectively integrated with time series foundation models (TFMs). To address these challenges, we propose \textbf{MOMEMTO}, a TFM for anomaly detection, enhanced with a patch-based memory module to mitigate over-generalization. The memory module is designed to capture representative normal patterns from multiple domains and enables a single model to be jointly fine-tuned across multiple datasets through a multi-domain training strategy. MOMEMTO initializes memory items with latent representations from a pre-trained encoder, organizes them into patch-level units, and updates them via an attention mechanism. We evaluate our method using 23 univariate benchmark datasets. Experimental results demonstrate that MOMEMTO, as a single model, achieves higher scores on AUC and VUS metrics compared to baseline methods, and further enhances the performance of its backbone TFM, particularly in few-shot learning scenarios.

Method: Analyzed the training dynamics of diagonal linear networks with linear activation and diagonal weight matrices, comparing them to the lasso regularization path through both rigorous mathematical analysis and simulations.

Result: Found that under a monotonicity assumption on the lasso path, the connection is exact; in general cases, an approximate connection exists where training time inversely relates to the regularization parameter.

Conclusion: The training trajectory of diagonal linear networks is fundamentally connected to the lasso regularization path, providing deeper insights into the implicit regularization properties of these networks.

Abstract: Diagonal linear networks are neural networks with linear activation and diagonal weight matrices. Their theoretical interest is that their implicit regularization can be rigorously analyzed: from a small initialization, the training of diagonal linear networks converges to the linear predictor with minimal 1-norm among minimizers of the training loss. In this paper, we deepen this analysis showing that the full training trajectory of diagonal linear networks is closely related to the lasso regularization path. In this connection, the training time plays the role of an inverse regularization parameter. Both rigorous results and simulations are provided to illustrate this conclusion. Under a monotonicity assumption on the lasso regularization path, the connection is exact while in the general case, we show an approximate connection.

Method: Uses ensemble probabilistic machine learning to quantify and automate prediction uncertainty in data-driven consistency-based diagnosis.

Result: Evaluated across multiple case studies with ablation and comparative analyses, showing consistent improvements across various diagnostic metrics.

Conclusion: The proposed framework effectively addresses uncertainty quantification in fault diagnostics, enhancing reliability in consistency-based diagnosis systems.

Abstract: Deep neural networks has been increasingly applied in fault diagnostics, where it uses historical data to capture systems behavior, bypassing the need for high-fidelity physical models. However, despite their competence in prediction tasks, these models often struggle with the evaluation of their confidence. This matter is particularly important in consistency-based diagnosis where decision logic is highly sensitive to false alarms. To address this challenge, this work presents a diagnostic framework that uses ensemble probabilistic machine learning to improve diagnostic characteristics of data driven consistency based diagnosis by quantifying and automating the prediction uncertainty. The proposed method is evaluated across several case studies using both ablation and comparative analyses, showing consistent improvements across a range of diagnostic metrics.

Method: Builds on particle filters and uses diffusion models pre-trained for dynamical system emulation without requiring additional training, demonstrated with GenCast for weather forecasting.

Result: Proposes a general framework that leverages existing diffusion models for efficient data assimilation.

Conclusion: The method provides a lightweight, training-free approach to data assimilation that can be applied to various dynamical systems using pre-trained diffusion models.

Abstract: Data assimilation is widely used in many disciplines such as meteorology, oceanography, and robotics to estimate the state of a dynamical system from noisy observations. In this work, we propose a lightweight and general method to perform data assimilation using diffusion models pre-trained for emulating dynamical systems. Our method builds on particle filters, a class of data assimilation algorithms, and does not require any further training. As a guiding example throughout this work, we illustrate our methodology on GenCast, a diffusion-based model that generates global ensemble weather forecasts.

Method: Proposes LoRD (Low-Rank Doubly stochastic clustering) that only relaxes orthonormal constraint, and B-LoRD that adds block diagonal regularization via Frobenius norm maximization. Uses projected gradient descent with theoretical convergence guarantees.

Result: Extensive experiments validate the effectiveness of the proposed approaches, showing improved clustering performance compared to existing methods.

Conclusion: The proposed LoRD and B-LoRD methods provide more constrained and theoretically sound alternatives to existing graph clustering techniques, with demonstrated effectiveness and publicly available implementation.

Abstract: The well-known graph-based clustering methods, including spectral clustering, symmetric non-negative matrix factorization, and doubly stochastic normalization, can be viewed as relaxations of the kernel $k$-means approach. However, we posit that these methods excessively relax their inherent low-rank, nonnegative, doubly stochastic, and orthonormal constraints to ensure numerical feasibility, potentially limiting their clustering efficacy. In this paper, guided by our theoretical analyses, we propose \textbf{Lo}w-\textbf{R}ank \textbf{D}oubly stochastic clustering (\textbf{LoRD}), a model that only relaxes the orthonormal constraint to derive a probabilistic clustering results. Furthermore, we theoretically establish the equivalence between orthogonality and block diagonality under the doubly stochastic constraint. By integrating \textbf{B}lock diagonal regularization into LoRD, expressed as the maximization of the Frobenius norm, we propose \textbf{B-LoRD}, which further enhances the clustering performance. To ensure numerical solvability, we transform the non-convex doubly stochastic constraint into a linear convex constraint through the introduction of a class probability parameter. We further theoretically demonstrate the gradient Lipschitz continuity of our LoRD and B-LoRD enables the proposal of a globally convergent projected gradient descent algorithm for their optimization. Extensive experiments validate the effectiveness of our approaches. The code is publicly available at https://github.com/lwl-learning/LoRD.

Method: NGRPO introduces two key mechanisms: 1) Advantage Calibration - hypothesizes a virtual maximum-reward sample to ensure non-zero advantages for homogeneously incorrect samples; 2) Asymmetric Clipping - relaxes positive sample updates while constraining negative sample updates to stabilize exploration.

Result: Experiments on Qwen2.5-Math-7B show NGRPO significantly outperforms PPO, GRPO, DAPO, and PSR-NSR on MATH500, AMC23, and AIME2025 benchmarks, demonstrating stable and substantial improvements in mathematical reasoning.

Conclusion: NGRPO successfully converts homogeneous errors into robust learning signals, enabling effective learning from previously unlearnable scenarios and achieving superior performance in mathematical reasoning tasks.

Abstract: RLVR has enhanced the reasoning capabilities of Large Language Models (LLMs) across various tasks. However, GRPO, a representative RLVR algorithm, suffers from a critical limitation: when all responses within a group are either entirely correct or entirely incorrect, the model fails to learn from these homogeneous responses. This is particularly problematic for homogeneously incorrect groups, where GRPO’s advantage function yields a value of zero, leading to null gradients and the loss of valuable learning signals. To overcome this issue, we propose NGRPO (Negative-enhanced Group Relative Policy Optimization), an algorithm designed to convert homogeneous errors into robust learning signals. First, NGRPO introduces Advantage Calibration. This mechanism hypothesizes the existence of a virtual maximum-reward sample during advantage calculation, thereby altering the mean and variance of rewards within a group and ensuring that the advantages for homogeneously incorrect samples are no longer zero. Second, NGRPO employs Asymmetric Clipping, which relaxes the update magnitude for positive samples while imposing stricter constraints on that of negative samples. This serves to stabilize the exploration pressure introduced by the advantage calibration. Our experiments on Qwen2.5-Math-7B demonstrate that NGRPO significantly outperforms baselines such as PPO, GRPO, DAPO, and PSR-NSR on mathematical benchmarks including MATH500, AMC23, and AIME2025. These results validate NGRPO’s ability to learn from homogeneous errors, leading to stable and substantial improvements in mathematical reasoning. Our code is available at https://github.com/nangongrui-ngr/NGRPO.

Method: Shared-Weights Extender (SWE) couples new neurons with existing ones for smooth integration, and Steepest Voting Distributor (SVoD) uses gradient-based allocation of neurons across layers.

Result: Extensive benchmarking on four datasets shows the method effectively suppresses neuron inactivity and achieves better performance compared to other expanding methods and baselines.

Conclusion: The proposed SWE and SVoD methods successfully address the neuron inactivity problem in neural network expansion, enabling effective capacity augmentation without retraining from scratch.

Abstract: Expanding neural networks during training is a promising way to augment capacity without retraining larger models from scratch. However, newly added neurons often fail to adjust to a trained network and become inactive, providing no contribution to capacity growth. We propose the Shared-Weights Extender (SWE), a novel method explicitly designed to prevent inactivity of new neurons by coupling them with existing ones for smooth integration. In parallel, we introduce the Steepest Voting Distributor (SVoD), a gradient-based method for allocating neurons across layers during deep network expansion. Our extensive benchmarking on four datasets shows that our method can effectively suppress neuron inactivity and achieve better performance compared to other expanding methods and baselines.

Method: Combined theoretical energy-based gradient flow analysis with empirical validation on synthetic datasets with controlled heterophily and real-world molecular property prediction.

Result: Motif detection requires mixed-frequency dynamics, and frequency-adaptive models outperform frequency-dominated models in graph-level tasks.

Conclusion: Establishes new theoretical understanding of heterophily in graph-level learning and provides guidance for designing effective GNN architectures that account for mixed-frequency requirements.

Abstract: While heterophily has been widely studied in node-level tasks, its impact on graph-level tasks remains unclear. We present the first analysis of heterophily in graph-level learning, combining theoretical insights with empirical validation. We first introduce a taxonomy of graph-level labeling schemes, and focus on motif-based tasks within local structure labeling, which is a popular labeling scheme. Using energy-based gradient flow analysis, we reveal a key insight: unlike frequency-dominated regimes in node-level tasks, motif detection requires mixed-frequency dynamics to remain flexible across multiple spectral components. Our theory shows that motif objectives are inherently misaligned with global frequency dominance, demanding distinct architectural considerations. Experiments on synthetic datasets with controlled heterophily and real-world molecular property prediction support our findings, showing that frequency-adaptive model outperform frequency-dominated models. This work establishes a new theoretical understanding of heterophily in graph-level learning and offers guidance for designing effective GNN architectures.

Method: Proposes min-max framework: inner layer maximizes performance gap between poisoned and benign samples to minimize impact of benign updates; outer process injects adaptive triggers into local model.

Result: Evaluated on computer vision and natural language tasks against six defense algorithms, showing superior attack performance compared to six existing backdoor attack methods.

Conclusion: The method achieves good attack performance, can be integrated into existing backdoor techniques, and provides more persistent backdoor attacks in federated learning settings.

Abstract: Federated learning allows multiple participants to collaboratively train a central model without sharing their private data. However, this distributed nature also exposes new attack surfaces. In particular, backdoor attacks allow attackers to implant malicious behaviors into the global model while maintaining high accuracy on benign inputs. Existing attacks usually rely on fixed patterns or adversarial perturbations as triggers, which tightly couple the main and backdoor tasks. This coupling makes them vulnerable to dilution by honest updates and limits their persistence under federated defenses. In this work, we propose an approach to decouple the backdoor task from the main task by dynamically optimizing the backdoor trigger within a min-max framework. The inner layer maximizes the performance gap between poisoned and benign samples, ensuring that the contributions of benign users have minimal impact on the backdoor. The outer process injects the adaptive triggers into the local model. We evaluate our method on both computer vision and natural language tasks, and compare it with six backdoor attack methods under six defense algorithms. Experimental results show that our method achieves good attack performance and can be easily integrated into existing backdoor attack techniques.

Method: Propose GNARL framework that translates problem formulations from NAR to RL using Markov Decision Processes, employing imitation and reinforcement learning with an architecture suitable for graph-based problems.

Result: Achieved very high graph accuracy on CLRS-30 problems, matched or exceeded narrower NAR approaches for NP-hard problems, and demonstrated applicability even without expert algorithms.

Conclusion: The GNARL framework successfully overcomes key limitations of NAR by leveraging RL approaches, showing strong performance across diverse problem types including challenging NP-hard problems.

Abstract: Neural Algorithmic Reasoning (NAR) is a paradigm that trains neural networks to execute classic algorithms by supervised learning. Despite its successes, important limitations remain: inability to construct valid solutions without post-processing and to reason about multiple correct ones, poor performance on combinatorial NP-hard problems, and inapplicability to problems for which strong algorithms are not yet known. To address these limitations, we reframe the problem of learning algorithm trajectories as a Markov Decision Process, which imposes structure on the solution construction procedure and unlocks the powerful tools of imitation and reinforcement learning (RL). We propose the GNARL framework, encompassing the methodology to translate problem formulations from NAR to RL and a learning architecture suitable for a wide range of graph-based problems. We achieve very high graph accuracy results on several CLRS-30 problems, performance matching or exceeding much narrower NAR approaches for NP-hard problems and, remarkably, applicability even when lacking an expert algorithm.

Method: Proposes using credal networks to mask Bayesian networks, adapting tracing attack definitions and identifying key learning information to conceal. Conducts numerical experiments to analyze privacy gains through CN hyperparameter tuning.

Result: CNs effectively reduce the probability of successful tracing attacks while maintaining meaningful inferences, providing a practical approach to privacy-aware probabilistic graphical models.

Conclusion: Credal networks offer a principled and effective solution for developing privacy-aware probabilistic graphical models that balance privacy protection with model utility.

Abstract: Bayesian networks (BN) are probabilistic graphical models that enable efficient knowledge representation and inference. These have proven effective across diverse domains, including healthcare, bioinformatics and economics. The structure and parameters of a BN can be obtained by domain experts or directly learned from available data. However, as privacy concerns escalate, it becomes increasingly critical for publicly released models to safeguard sensitive information in training data. Typically, released models do not prioritize privacy by design. In particular, tracing attacks from adversaries can combine the released BN with auxiliary data to determine whether specific individuals belong to the data from which the BN was learned. State-of-the-art protection tecniques involve introducing noise into the learned parameters. While this offers robust protection against tracing attacks, it significantly impacts the model’s utility, in terms of both the significance and accuracy of the resulting inferences. Hence, high privacy may be attained at the cost of releasing a possibly ineffective model. This paper introduces credal networks (CN) as a novel solution for balancing the model’s privacy and utility. After adapting the notion of tracing attacks, we demonstrate that a CN enables the masking of the learned BN, thereby reducing the probability of successful attacks. As CNs are obfuscated but not noisy versions of BNs, they can achieve meaningful inferences while safeguarding privacy. Moreover, we identify key learning information that must be concealed to prevent attackers from recovering the underlying BN. Finally, we conduct a set of numerical experiments to analyze how privacy gains can be modulated by tuning the CN hyperparameters. Our results confirm that CNs provide a principled, practical, and effective approach towards the development of privacy-aware probabilistic graphical models.

Method: HEROS uses a Markov decision process to model trade-offs between performance and sustainability. It introduces policies (especially the ζ-policy) for choosing which models to train from a diverse pool under resource constraints.

Result: Theoretical analysis proves the ζ-policy achieves near-optimal performance with fewer resources. Experiments on 11 benchmark datasets show HEROS provides highly accurate performance while being much more resource-friendly than competitors.

Conclusion: HEROS successfully addresses the sustainability challenge in online ensemble learning, demonstrating that resource-efficient methods can achieve competitive or even superior performance compared to traditional approaches.

Abstract: Ensemble methods for stream mining necessitate managing multiple models and updating them as data distributions evolve. Considering the calls for more sustainability, established methods are however not sufficiently considerate of ensemble members’ computational expenses and instead overly focus on predictive capabilities. To address these challenges and enable green online learning, we propose heterogeneous online ensembles (HEROS). For every training step, HEROS chooses a subset of models from a pool of models initialized with diverse hyperparameter choices under resource constraints to train. We introduce a Markov decision process to theoretically capture the trade-offs between predictive performance and sustainability constraints. Based on this framework, we present different policies for choosing which models to train on incoming data. Most notably, we propose the novel $\zeta$-policy, which focuses on training near-optimal models at reduced costs. Using a stochastic model, we theoretically prove that our $\zeta$-policy achieves near optimal performance while using fewer resources compared to the best performing policy. In our experiments across 11 benchmark datasets, we find empiric evidence that our $\zeta$-policy is a strong contribution to the state-of-the-art, demonstrating highly accurate performance, in some cases even outperforming competitors, and simultaneously being much more resource-friendly.

Method: Establishes non-asymptotic central limit theorems and functional central limit theorems for Polyak-Ruppert averaged Q-learning under asynchronous updates, analyzing convergence rates in Wasserstein distance.

Result: The convergence rate explicitly depends on number of iterations, state-action space size, discount factor, and exploration quality. The partial-sum process converges weakly to Brownian motion.

Conclusion: The paper provides comprehensive central limit theory for averaged Q-learning, offering theoretical foundations for understanding and improving practical reinforcement learning algorithms.

Abstract: This paper establishes central limit theorems for Polyak-Ruppert averaged Q-learning under asynchronous updates. We present a non-asymptotic central limit theorem, where the convergence rate in Wasserstein distance explicitly reflects the dependence on the number of iterations, state-action space size, the discount factor, and the quality of exploration. In addition, we derive a functional central limit theorem, showing that the partial-sum process converges weakly to a Brownian motion.

Method: The authors fabricated custom indium oxide optoelectronic synapses and repurposed their natural physical decay as the temporal function computation. They introduced a quantized neural network-to-SNN conversion algorithm to enable complex architectures like transformers, and developed a complete hardware-software co-design approach.

Result: The Otters paradigm achieved state-of-the-art accuracy across seven GLUE benchmark datasets and demonstrated a 1.77× improvement in energy efficiency over previous leading SNNs, based on comprehensive energy analysis using a commercial 22nm process.

Conclusion: This work establishes a new paradigm for energy-efficient SNNs by translating fundamental device physics directly into computational primitives, eliminating the need for expensive digital operations in TTFS encoding while maintaining high accuracy.

Abstract: Spiking neural networks (SNNs) promise high energy efficiency, particularly with time-to-first-spike (TTFS) encoding, which maximizes sparsity by emitting at most one spike per neuron. However, such energy advantage is often unrealized because inference requires evaluating a temporal decay function and subsequent multiplication with the synaptic weights. This paper challenges this costly approach by repurposing a physical hardware `bug’, namely, the natural signal decay in optoelectronic devices, as the core computation of TTFS. We fabricated a custom indium oxide optoelectronic synapse, showing how its natural physical decay directly implements the required temporal function. By treating the device’s analog output as the fused product of the synaptic weight and temporal decay, optoelectronic synaptic TTFS (named Otters) eliminates these expensive digital operations. To use the Otters paradigm in complex architectures like the transformer, which are challenging to train directly due to the sparsity issue, we introduce a novel quantized neural network-to-SNN conversion algorithm. This complete hardware-software co-design enables our model to achieve state-of-the-art accuracy across seven GLUE benchmark datasets and demonstrates a 1.77$\times$ improvement in energy efficiency over previous leading SNNs, based on a comprehensive analysis of compute, data movement, and memory access costs using energy measurements from a commercial 22nm process. Our work thus establishes a new paradigm for energy-efficient SNNs, translating fundamental device physics directly into powerful computational primitives. All codes and data are open source.

Method: Proposes Fully Learnable Neural Reward Machines (FLNRM) that learn both the Symbol Grounding function and automaton end-to-end, integrating with deep RL.

Result: Outperforms previous approaches based on Recurrent Neural Networks while being more explainable due to the finite and compact nature of automata.

Conclusion: FLNRM provides a fully learnable solution that is as easily applicable as classic deep RL approaches while offering better explainability and performance than RNN-based methods.

Abstract: Non-Markovian Reinforcement Learning (RL) tasks present significant challenges, as agents must reason over entire trajectories of state-action pairs to make optimal decisions. A common strategy to address this is through symbolic formalisms, such as Linear Temporal Logic (LTL) or automata, which provide a structured way to express temporally extended objectives. However, these approaches often rely on restrictive assumptions – such as the availability of a predefined Symbol Grounding (SG) function mapping raw observations to high-level symbolic representations, or prior knowledge of the temporal task. In this work, we propose a fully learnable version of Neural Reward Machines (NRM), which can learn both the SG function and the automaton end-to-end, removing any reliance on prior knowledge. Our approach is therefore as easily applicable as classic deep RL (DRL) approaches, while being far more explainable, because of the finite and compact nature of automata. Furthermore, we show that by integrating Fully Learnable Reward Machines (FLNRM) with DRL, our method outperforms previous approaches based on Recurrent Neural Networks (RNNs).

Method: Developed theoretical framework showing SGNNs implement amortized Bayesian inference under simulation prior, derived generalization bounds under model misspecification, and formalized mechanistic interpretability through attribution to simulated mechanisms.

Result: SGNNs recover latent parameters, remain robust under mismatch, and outperform classical tools - achieving half the error of AIC in model selection tasks for distinguishing mechanistic dynamics.

Conclusion: SGNNs are established as a principled and practical framework for scientific prediction in data-limited regimes, providing posterior-consistent, scientifically grounded explanations.

Abstract: Simulation-Grounded Neural Networks (SGNNs) are predictive models trained entirely on synthetic data from mechanistic simulations. They have achieved state-of-the-art performance in domains where real-world labels are limited or unobserved, but lack a formal underpinning. We present the foundational theory of simulation-grounded learning. We show that SGNNs implement amortized Bayesian inference under a simulation prior and converge to the Bayes-optimal predictor. We derive generalization bounds under model misspecification and prove that SGNNs can learn unobservable scientific quantities that empirical methods provably cannot. We also formalize a novel form of mechanistic interpretability uniquely enabled by SGNNs: by attributing predictions to the simulated mechanisms that generated them, SGNNs yield posterior-consistent, scientifically grounded explanations. We provide numerical experiments to validate all theoretical predictions. SGNNs recover latent parameters, remain robust under mismatch, and outperform classical tools: in a model selection task, SGNNs achieve half the error of AIC in distinguishing mechanistic dynamics. These results establish SGNNs as a principled and practical framework for scientific prediction in data-limited regimes.

Method: Proposes Cross-layer Low-Rank residual Network (CR-Net) with a dual-path architecture that reconstructs layer activations by combining previous-layer outputs with their low-rank differences. Includes a specialized activation recomputation strategy for memory reduction.

Result: Extensive pre-training experiments on models from 60M to 7B parameters show CR-Net consistently outperforms state-of-the-art low-rank frameworks while requiring fewer computational resources and less memory.

Conclusion: CR-Net successfully addresses the limitations of current low-rank methods by leveraging low-rank activation residuals, achieving better performance with improved efficiency across various model scales.

Abstract: Low-rank architectures have become increasingly important for efficient large language model (LLM) pre-training, providing substantial reductions in both parameter complexity and memory/computational demands. Despite these advantages, current low-rank methods face three critical shortcomings: (1) compromised model performance, (2) considerable computational overhead, and (3) limited activation memory savings. To address these limitations, we propose Cross-layer Low-Rank residual Network (CR-Net), an innovative parameter-efficient framework inspired by our discovery that inter-layer activation residuals possess low-rank properties. CR-Net implements this insight through a dual-path architecture that efficiently reconstructs layer activations by combining previous-layer outputs with their low-rank differences, thereby maintaining high-rank information with minimal parameters. We further develop a specialized activation recomputation strategy tailored for CR-Net that dramatically reduces memory requirements. Extensive pre-training experiments across model scales from 60M to 7B parameters demonstrate that CR-Net consistently outperforms state-of-the-art low-rank frameworks while requiring fewer computational resources and less memory.

Method: A comparative framework was established to evaluate three BP-free methods (Forward-Forward, Cascaded-Forward, and Mono-Forward) against BP-trained models. Hyperparameters were optimized with Optuna, and performance was measured using NVIDIA Management Library and CodeCarbon.

Result: MF consistently surpassed BP in classification accuracy on MLPs, achieving superior generalization by converging to more favorable minima. It reduced energy consumption by up to 41% and training time by up to 34%.

Conclusion: MF offers a superior synthesis of accuracy and sustainability, challenging the assumption that global optimization is required for state-of-the-art results, and provides a roadmap for energy-efficient deep learning.

Abstract: The rising computational and energy demands of deep neural networks (DNNs), driven largely by backpropagation (BP), challenge sustainable AI development. This paper rigorously investigates three BP-free training methods: the Forward-Forward (FF), Cascaded-Forward (CaFo), and Mono-Forward (MF) algorithms, tracing their progression from foundational concepts to a demonstrably superior solution. A robust comparative framework was established: each algorithm was implemented on its native architecture (MLPs for FF and MF, a CNN for CaFo) and benchmarked against an equivalent BP-trained model. Hyperparameters were optimized with Optuna, and consistent early stopping criteria were applied based on validation performance, ensuring all models were optimally tuned before comparison. Results show that MF not only competes with but consistently surpasses BP in classification accuracy on its native MLPs. Its superior generalization stems from converging to a more favorable minimum in the validation loss landscape, challenging the assumption that global optimization is required for state-of-the-art results. Measured at the hardware level using the NVIDIA Management Library (NVML) API, MF reduces energy consumption by up to 41% and shortens training time by up to 34%, translating to a measurably smaller carbon footprint as estimated by CodeCarbon. Beyond this primary result, we present a hardware-level analysis that explains the efficiency gains: exposing FF’s architectural inefficiencies, validating MF’s computationally lean design, and challenging the assumption that all BP-free methods are inherently more memory-efficient. By documenting the evolution from FF’s conceptual groundwork to MF’s synthesis of accuracy and sustainability, this work offers a clear, data-driven roadmap for future energy-efficient deep learning.

Method: The paper combines mathematical tools from statistics and optimization to analyze representation learning from unlabeled data, providing theoretical frameworks to understand modern approaches.

Result: The paper presents recent theoretical advances that help explain the success of visual foundation models using self-supervision and denoising/masked autoencoders in learning effective representations from massive unlabeled data.

Conclusion: By bridging classical statistical theories with modern deep learning principles, the paper contributes to better understanding and characterizing the representations learned by contemporary unsupervised learning models.

Abstract: Representation learning from unlabeled data has been extensively studied in statistics, data science and signal processing with a rich literature on techniques for dimension reduction, compression, multi-dimensional scaling among others. However, current deep learning models use new principles for unsupervised representation learning that cannot be easily analyzed using classical theories. For example, visual foundation models have found tremendous success using self-supervision or denoising/masked autoencoders, which effectively learn representations from massive amounts of unlabeled data. However, it remains difficult to characterize the representations learned by these models and to explain why they perform well for diverse prediction tasks or show emergent behavior. To answer these questions, one needs to combine mathematical tools from statistics and optimization. This paper provides an overview of recent theoretical advances in representation learning from unlabeled data and mentions our contributions in this direction.

Method: Uses pretrained LLMs with lightweight bidirectional latent alignment module and two-stage decoupled training: supervised fine-tuning for multimodal reasoning alignment, and semantic-guided diffusion training for cross-modal latent space alignment.

Result: Achieves competitive or state-of-the-art performance across various benchmarks in all three tasks (understanding, generation, retrieval).

Conclusion: Demonstrates effectiveness of latent space alignment for unifying multimodal modeling under a shared representation space, providing a modular and efficient solution.

Abstract: Recent advances in multimodal large language models (LLMs) have led to significant progress in understanding, generation, and retrieval tasks. However, current solutions often treat these tasks in isolation or require training LLMs from scratch, resulting in high computational costs and limited generalization across modalities. In this work, we present OmniBridge, a unified and modular multimodal framework that supports vision-language understanding, generation, and retrieval within a unified architecture. OmniBridge adopts a language-centric design that reuses pretrained LLMs and introduces a lightweight bidirectional latent alignment module. To address the challenge of task interference, we propose a two-stage decoupled training strategy: supervised fine-tuning and latent space alignment for aligning LLM behavior with multimodal reasoning, and semantic-guided diffusion training to align cross-modal latent spaces via learnable query embeddings. Extensive experiments across a wide range of benchmarks demonstrate that OmniBridge achieves competitive or state-of-the-art performance in all three tasks. Moreover, our results highlight the effectiveness of latent space alignment for unifying multimodal modeling under a shared representation space. Code and models are released at https://github.com/xiao-xt/OmniBridge.

Method: AxelGNN incorporates similarity-gated probabilistic interactions that adaptively promote convergence or divergence based on node similarity, implements trait-level copying mechanisms for fine-grained feature aggregation at segment level, and maintains global polarization to preserve node distinctiveness across multiple representation clusters.

Result: Extensive experiments on node classification and influence estimation benchmarks show AxelGNN consistently outperforms or matches state-of-the-art GNN methods across diverse graph structures with varying homophily-heterophily characteristics.

Conclusion: AxelGNN’s bistable convergence dynamics naturally handle both homophilic and heterophilic graphs within a single architecture, providing a unified framework that addresses key GNN limitations.

Abstract: Graph Neural Networks (GNNs) have demonstrated remarkable success across various graph-based tasks. However, they face some fundamental limitations: feature oversmoothing can cause node representations to become indistinguishable in deeper networks, they struggle to effectively manage heterogeneous relationships where connected nodes differ significantly, and they process entire feature vectors as indivisible units, which limits flexibility. We seek to address these limitations. We propose AxelGNN, a novel GNN architecture inspired by Axelrod’s cultural dissemination model that addresses these limitations through a unified framework. AxelGNN incorporates similarity-gated probabilistic interactions that adaptively promote convergence or divergence based on node similarity, implements trait-level copying mechanisms for fine-grained feature aggregation at the segment level, and maintains global polarization to preserve node distinctiveness across multiple representation clusters. The model’s bistable convergence dynamics naturally handle both homophilic and heterophilic graphs within a single architecture. Extensive experiments on node classification and influence estimation benchmarks demonstrate that AxelGNN consistently outperforms or matches state-of-the-art GNN methods across diverse graph structures with varying homophily-heterophily characteristics.

Method: Proposes a hybrid approach combining Generative Adversarial Network (GAN) with Transformer encoder blocks. The GAN enables adversarial training for realistic sample generation, while the Transformer’s self-attention mechanism learns rich feature interactions to overcome limitations of existing methods.

Result: Tested on the Credit Card Fraud Detection dataset, the Transformer-based GAN showed substantial improvements in Recall, F1-score, and AUC compared to conventional resampling strategies across multiple classifiers (Logistic Regression, Random Forest, XGBoost, SVM).

Conclusion: The hybrid GAN-Transformer approach effectively overcomes severe class imbalance in fraud detection by producing high-quality synthetic minority class samples, demonstrating superior performance over traditional and existing generative resampling methods.

Abstract: Detection of credit card fraud is an acute issue of financial security because transaction datasets are highly lopsided, with fraud cases being only a drop in the ocean. Balancing datasets using the most popular methods of traditional oversampling such as the Synthetic Minority Oversampling Technique (SMOTE) generally create simplistic synthetic samples that are not readily applicable to complex fraud patterns. Recent industry advances that include Conditional Tabular Generative Adversarial Networks (CTGAN) and Tabular Variational Autoencoders (TVAE) have demonstrated increased efficiency in tabular synthesis, yet all these models still exhibit issues with high-dimensional dependence modelling. Now we will present our hybrid approach where we use a Generative Adversarial Network (GAN) with a Transformer encoder block to produce realistic fraudulent transactions samples. The GAN architecture allows training realistic generators adversarial, and the Transformer allows the model to learn rich feature interactions by self-attention. Such a hybrid strategy overcomes the limitations of SMOTE, CTGAN, and TVAE by producing a variety of high-quality synthetic minority classes samples. We test our algorithm on the publicly-available Credit Card Fraud Detection dataset and compare it to conventional and generative resampling strategies with a variety of classifiers, such as Logistic Regression (LR), Random Forest (RF), Extreme Gradient Boosting (XGBoost), and Support Vector Machine (SVM). Findings indicate that our Transformer-based GAN shows substantial gains in Recall, F1-score and Area Under the Receiver Operating Characteristic Curve (AUC), which indicates that it is effective in overcoming the severe class imbalance inherent in the task of fraud detection.

Method: JAD generates adversarial examples using a latent diffusion model guided by attention maps distilled from both CNN and Vision Transformer models, focusing on commonly sensitive image regions across architectures.

Result: Experiments show JAD achieves superior attack generalization, generation efficiency, and cross-architecture transferability compared to existing methods.

Conclusion: JAD provides a promising and effective paradigm for black-box adversarial attacks by being architecture-agnostic and reducing reliance on iterative queries.

Abstract: Black-box adversarial attacks remain challenging due to limited access to model internals. Existing methods often depend on specific network architectures or require numerous queries, resulting in limited cross-architecture transferability and high query costs. To address these limitations, we propose JAD, a latent diffusion model framework for black-box adversarial attacks. JAD generates adversarial examples by leveraging a latent diffusion model guided by attention maps distilled from both a convolutional neural network (CNN) and a Vision Transformer (ViT) models. By focusing on image regions that are commonly sensitive across architectures, this approach crafts adversarial perturbations that transfer effectively between different model types. This joint attention distillation strategy enables JAD to be architecture-agnostic, achieving superior attack generalization across diverse models. Moreover, the generative nature of the diffusion framework yields high adversarial sample generation efficiency by reducing reliance on iterative queries. Experiments demonstrate that JAD offers improved attack generalization, generation efficiency, and cross-architecture transferability compared to existing methods, providing a promising and effective paradigm for black-box adversarial attacks.

Method: The thesis introduces new technical results, training paradigms, and certification algorithms for adversarial examples in computer vision; new algorithms for domain generalization in medical imaging, molecular identification, and image classification; and new attacks and defenses for jailbreaking large language models.

Result: The proposed methods achieve state-of-the-art generalization in various applications and represent the frontier of progress in designing robust language-based agents.

Conclusion: The thesis contributes significantly to the field by advancing algorithms that exhibit desirable robustness properties across multiple critical domains of deep learning.

Abstract: Given the widespread use of deep learning models in safety-critical applications, ensuring that the decisions of such models are robust against adversarial exploitation is of fundamental importance. In this thesis, we discuss recent progress toward designing algorithms that exhibit desirable robustness properties. First, we discuss the problem of adversarial examples in computer vision, for which we introduce new technical results, training paradigms, and certification algorithms. Next, we consider the problem of domain generalization, wherein the task is to train neural networks to generalize from a family of training distributions to unseen test distributions. We present new algorithms that achieve state-of-the-art generalization in medical imaging, molecular identification, and image classification. Finally, we study the setting of jailbreaking large language models (LLMs), wherein an adversarial user attempts to design prompts that elicit objectionable content from an LLM. We propose new attacks and defenses, which represent the frontier of progress toward designing robust language-based agents.

Method: DBVI initiates reverse diffusion from a learnable, data-dependent distribution parameterized by an amortized neural network that operates on inducing inputs, using a Doob-bridged diffusion process with tractable ELBO training.

Result: DBVI consistently outperforms DDVI and other variational baselines in predictive accuracy, convergence speed, and posterior quality across regression, classification, and image reconstruction tasks.

Conclusion: DBVI provides a principled extension to DDVI that bridges the posterior gap through learnable initialization, enabling more efficient and scalable inference for deep Gaussian processes.

Abstract: Deep Gaussian processes (DGPs) enable expressive hierarchical Bayesian modeling but pose substantial challenges for posterior inference, especially over inducing variables. Denoising diffusion variational inference (DDVI) addresses this by modeling the posterior as a time-reversed diffusion from a simple Gaussian prior. However, DDVI’s fixed unconditional starting distribution remains far from the complex true posterior, resulting in inefficient inference trajectories and slow convergence. In this work, we propose Diffusion Bridge Variational Inference (DBVI), a principled extension of DDVI that initiates the reverse diffusion from a learnable, data-dependent initial distribution. This initialization is parameterized via an amortized neural network and progressively adapted using gradients from the ELBO objective, reducing the posterior gap and improving sample efficiency. To enable scalable amortization, we design the network to operate on the inducing inputs, which serve as structured, low-dimensional summaries of the dataset and naturally align with the inducing variables’ shape. DBVI retains the mathematical elegance of DDVI, including Girsanov-based ELBOs and reverse-time SDEs,while reinterpreting the prior via a Doob-bridged diffusion process. We derive a tractable training objective under this formulation and implement DBVI for scalable inference in large-scale DGPs. Across regression, classification, and image reconstruction tasks, DBVI consistently outperforms DDVI and other variational baselines in predictive accuracy, convergence speed, and posterior quality.

Method: Develops KL-UCB-Transfer, an index policy that incorporates transfer parameters (distance bounds, sample sizes) into the KL-UCB framework, with theoretical analysis and Gaussian case specialization.

Result: Derives asymptotic lower bounds on regret that account for transfer parameters, proves KL-UCB-Transfer achieves these bounds in Gaussian cases, and shows significant performance improvements over baseline methods via simulations.

Conclusion: Transfer learning can substantially improve multi-armed bandit performance when source-target distribution distances are small, with KL-UCB-Transfer providing an optimal solution framework for this setting.

Abstract: We study the non-contextual multi-armed bandit problem in a transfer learning setting: before any pulls, the learner is given N’_k i.i.d. samples from each source distribution nu’_k, and the true target distributions nu_k lie within a known distance bound d_k(nu_k, nu’_k) <= L_k. In this framework, we first derive a problem-dependent asymptotic lower bound on cumulative regret that extends the classical Lai-Robbins result to incorporate the transfer parameters (d_k, L_k, N’_k). We then propose KL-UCB-Transfer, a simple index policy that matches this new bound in the Gaussian case. Finally, we validate our approach via simulations, showing that KL-UCB-Transfer significantly outperforms the no-prior baseline when source and target distributions are sufficiently close.

Method: CARGO uses two pretrained causal Transformers as foundation models to infer causal graphs per sequence in parallel, then aggregates them using adaptive frequency fusion to reconstruct global Markov boundaries of labels, avoiding expensive full-dataset conditional independence testing.

Result: CARGO demonstrates strong performance on a real-world automotive fault prediction dataset with over 29,100 unique event types and 474 imbalanced labels, showing its ability to perform structured reasoning at scale.

Conclusion: The two-stage approach enables efficient probabilistic reasoning for causal discovery in high-dimensional event sequences, providing a scalable solution for domains requiring causal understanding of complex event relationships.

Abstract: Understanding causality in event sequences where outcome labels such as diseases or system failures arise from preceding events like symptoms or error codes is critical. Yet remains an unsolved challenge across domains like healthcare or vehicle diagnostics. We introduce CARGO, a scalable multi-label causal discovery method for sparse, high-dimensional event sequences comprising of thousands of unique event types. Using two pretrained causal Transformers as domain-specific foundation models for event sequences. CARGO infers in parallel, per sequence one-shot causal graphs and aggregates them using an adaptive frequency fusion to reconstruct the global Markov boundaries of labels. This two-stage approach enables efficient probabilistic reasoning at scale while bypassing the intractable cost of full-dataset conditional independence testing. Our results on a challenging real-world automotive fault prediction dataset with over 29,100 unique event types and 474 imbalanced labels demonstrate CARGO’s ability to perform structured reasoning.

Method: The paper introduces DRO-REBEL, which uses Fenchel duality to reduce robust updates to simple relative-reward regression. It avoids PPO-style clipping and auxiliary value networks. The method includes practical SGD algorithms for three divergences: gradient regularization (Wasserstein), importance weighting (KL), and a fast 1-D dual solve (χ²).

Result: The approach achieves O(n^{-1/4}) estimation bounds with tighter constants than prior methods, and recovers the minimax-optimal O(n^{-1/2}) rate. Experiments on Emotion Alignment, ArmoRM, and HH-Alignment show strong worst-case robustness across various scenarios, with χ²-REBEL performing consistently well.

Conclusion: DRO-REBEL provides a unified framework for robust RLHF that addresses overoptimization while maintaining scalability. The analysis reveals a no-free-lunch trade-off between achieving optimal parametric rates and maintaining coverage guarantees, with different divergence choices offering practical advantages.

Abstract: Reinforcement learning with human feedback (RLHF) has become crucial for aligning Large Language Models (LLMs) with human intent. However, existing offline RLHF approaches suffer from overoptimization, where models overfit to reward misspecification and drift from preferred behaviors observed during training. We introduce DRO-REBEL, a unified family of robust REBEL updates with type-$p$ Wasserstein, KL, and $\chi^2$ ambiguity sets. Using Fenchel duality, each update reduces to a simple relative-reward regression, preserving scalability and avoiding PPO-style clipping or auxiliary value networks. Under standard linear-reward and log-linear policy classes with a data-coverage condition, we establish $O(n^{-1/4})$ estimation bounds with tighter constants than prior DRO-DPO approaches, and recover the minimax-optimal $O(n^{-1/2})$ rate via a localized Rademacher complexity analysis. The same analysis closes the gap for Wasserstein-DPO and KL-DPO, showing both also attain optimal parametric rates. We derive practical SGD algorithms for all three divergences: gradient regularization (Wasserstein), importance weighting (KL), and a fast 1-D dual solve ($\chi^2$). Experiments on Emotion Alignment, the large-scale ArmoRM multi-objective benchmark, and HH-Alignment demonstrate strong worst-case robustness across unseen preference mixtures, model sizes, and data scales, with $\chi^2$-REBEL showing consistently strong empirical performance. A controlled radius–coverage study validates a no-free-lunch trade-off: radii shrinking faster than empirical divergence concentration rates achieve minimax-optimal parametric rates but forfeit coverage, while coverage-guaranteeing radii incur $O(n^{-1/4})$ rates.

Method: FedFiTS implements a three-phase participation strategy (free-for-all training, natural selection, and slotted team participation) with dynamic client scoring, adaptive thresholding, and cohort-based scheduling. It includes theoretical convergence analysis for convex and non-convex objectives.

Result: Experiments on medical imaging, vision benchmarks, and agricultural data show FedFiTS consistently outperforms FedAvg, FedRand, and FedPow in accuracy, time-to-target, and resilience to poisoning attacks, with reduced communication complexity.

Conclusion: By integrating trust-aware aggregation with fairness-oriented client selection, FedFiTS advances scalable and secure FL, making it well-suited for real-world healthcare and cross-domain deployments.

Abstract: Federated Learning (FL) has emerged as a powerful paradigm for privacy-preserving model training, yet deployments in sensitive domains such as healthcare face persistent challenges from non-IID data, client unreliability, and adversarial manipulation. This paper introduces FedFiTS, a trust and fairness-aware selective FL framework that advances the FedFaSt line by combining fitness-based client election with slotted aggregation. FedFiTS implements a three-phase participation strategy-free-for-all training, natural selection, and slotted team participation-augmented with dynamic client scoring, adaptive thresholding, and cohort-based scheduling to balance convergence efficiency with robustness. A theoretical convergence analysis establishes bounds for both convex and non-convex objectives under standard assumptions, while a communication-complexity analysis shows reductions relative to FedAvg and other baselines. Experiments on diverse datasets-medical imaging (X-ray pneumonia), vision benchmarks (MNIST, FMNIST), and tabular agricultural data (Crop Recommendation)-demonstrate that FedFiTS consistently outperforms FedAvg, FedRand, and FedPow in accuracy, time-to-target, and resilience to poisoning attacks. By integrating trust-aware aggregation with fairness-oriented client selection, FedFiTS advances scalable and secure FL, making it well suited for real-world healthcare and cross-domain deployments.

Method: Two vector representations: 1) Standard-Deviation Vector from normalized standard deviation values of projection matrices assuming normal distribution, 2) Clustering Vector from K-Means grouped singular values of weight projection matrices. Applied to pre-trained and LoRA fine-tuned models.

Result: The vectors effectively distinguish different models and show similarities among same-family models. Standard deviation vector is directly influenced by fine-tuning datasets, while clustering vector maintains high consistency with pre-trained model regardless of fine-tuning.

Conclusion: The proposed vectors provide effective tools for model analysis, with standard deviation vector capturing dataset-influenced distribution characteristics and clustering vector preserving stable correlation characteristics across fine-tuning.

Abstract: The study on architecture and parameter characteristics remains the hot topic in the research of large language models. In this paper we concern with the characteristics of weight which are used to analyze the correlations and differences between models. Two kinds of vectors-standard deviation vector and clustering vector-are proposed to describe features of models. In the first case, the weights are assumed to follow normal distribution. The standard deviation values of projection matrices are normalized to form Standard-Deviation Vector, representing the distribution characteristics of models. In the second case, the singular values from each weight projection matrix are extracted and grouped by K-Means algorithm. The grouped data with the same type matrix are combined as Clustering Vector to represent the correlation characteristics of models’ weights. The study reveals that these two vectors can effectively distinguish between different models and clearly show the similarities among models of the same family. Moreover, after conducting LoRA fine-tuning with different datasets and models, it is found that the distribution of weights represented by standard deviation vector is directly influenced by the dataset, but the correlations between different weights represented by clustering vector remain unaffected and maintain a high consistency with the pre-trained model.

Method: The framework consists of four components: 1) Spatio-Temporal Concept Encoder (STCE) that integrates location, POI semantics, and temporal rhythms; 2) Cognitive Trajectory Memory (CTM) that filters historical visits to capture user intent; 3) Lifestyle Concept Bank (LCB) that provides human preference cues; 4) Task-oriented generative heads for multiple downstream tasks.

Result: Extensive experiments on four real-world datasets (Gowalla, WeePlace, Brightkite, FourSquare) show consistent and substantial improvements over strong baselines on three benchmark tasks: next-location prediction, trajectory-user identification, and time estimation.

Conclusion: GSTM-HMU effectively extracts semantic regularities from complex mobility data, and generative modeling provides a promising foundation for building more robust, interpretable, and generalizable human mobility intelligence systems.

Abstract: Human mobility traces, often recorded as sequences of check-ins, provide a unique window into both short-term visiting patterns and persistent lifestyle regularities. In this work we introduce GSTM-HMU, a generative spatio-temporal framework designed to advance mobility analysis by explicitly modeling the semantic and temporal complexity of human movement. The framework consists of four key innovations. First, a Spatio-Temporal Concept Encoder (STCE) integrates geographic location, POI category semantics, and periodic temporal rhythms into unified vector representations. Second, a Cognitive Trajectory Memory (CTM) adaptively filters historical visits, emphasizing recent and behaviorally salient events in order to capture user intent more effectively. Third, a Lifestyle Concept Bank (LCB) contributes structured human preference cues, such as activity types and lifestyle patterns, to enhance interpretability and personalization. Finally, task-oriented generative heads transform the learned representations into predictions for multiple downstream tasks. We conduct extensive experiments on four widely used real-world datasets, including Gowalla, WeePlace, Brightkite, and FourSquare, and evaluate performance on three benchmark tasks: next-location prediction, trajectory-user identification, and time estimation. The results demonstrate consistent and substantial improvements over strong baselines, confirming the effectiveness of GSTM-HMU in extracting semantic regularities from complex mobility data. Beyond raw performance gains, our findings also suggest that generative modeling provides a promising foundation for building more robust, interpretable, and generalizable systems for human mobility intelligence.

Method: PipelineRL employs concurrent asynchronous data generation and model training with novel in-flight weight updates, allowing LLM generation engine to receive updated model weights with minimal interruption during token sequence generation.

Result: Experiments on long-form reasoning tasks using 128 H100 GPUs show PipelineRL achieves ~2x faster learning compared to conventional RL baselines while maintaining highly on-policy training data.

Conclusion: PipelineRL provides a scalable solution for RL-based LLM training that balances hardware efficiency and data freshness, with an open-source implementation released as a key contribution.

Abstract: Reinforcement Learning (RL) is increasingly utilized to enhance the reasoning capabilities of Large Language Models (LLMs). However, effectively scaling these RL methods presents significant challenges, primarily due to the difficulty in maintaining high AI accelerator utilization without generating stale, off-policy data that harms common RL algorithms. This paper introduces PipelineRL, an approach designed to achieve a superior trade-off between hardware efficiency and data on-policyness for LLM training. PipelineRL employs concurrent asynchronous data generation and model training, distinguished by the novel in-flight weight updates. This mechanism allows the LLM generation engine to receive updated model weights with minimal interruption during the generation of token sequences, thereby maximizing both the accelerator utilization and the freshness of training data. Experiments conducted on long-form reasoning tasks using 128 H100 GPUs demonstrate that PipelineRL achieves approximately $\sim 2x$ faster learning compared to conventional RL baselines while maintaining highly on-policy training data. A scalable and modular open-source implementation of PipelineRL is also released as a key contribution.

Method: The study analyzes activation functions’ role in training dynamics and proposes elephant activation functions that produce both sparse representations and sparse gradients, which are then used to replace classical activation functions in value-based RL algorithms.

Result: Replacing classical activation functions with elephant activation functions significantly improves neural networks’ resilience to catastrophic forgetting, making reinforcement learning more sample-efficient and memory-efficient.

Conclusion: Gradient sparsity of activation functions plays a crucial role in reducing catastrophic forgetting, and elephant activation functions provide an effective architectural solution to this long-standing problem in reinforcement learning.

Abstract: Catastrophic forgetting has remained a significant challenge for efficient reinforcement learning for decades (Ring 1994, Rivest and Precup 2003). While recent works have proposed effective methods to mitigate this issue, they mainly focus on the algorithmic side. Meanwhile, we do not fully understand what architectural properties of neural networks lead to catastrophic forgetting. This study aims to fill this gap by studying the role of activation functions in the training dynamics of neural networks and their impact on catastrophic forgetting in reinforcement learning setup. Our study reveals that, besides sparse representations, the gradient sparsity of activation functions also plays an important role in reducing forgetting. Based on this insight, we propose a new class of activation functions, elephant activation functions, that can generate both sparse outputs and sparse gradients. We show that by simply replacing classical activation functions with elephant activation functions in the neural networks of value-based algorithms, we can significantly improve the resilience of neural networks to catastrophic forgetting, thus making reinforcement learning more sample-efficient and memory-efficient.

Method: Uses teacher-student kernel regression with online SGD, introduces intrinsic time viewpoint and SDE modeling to derive FSL that characterizes population risk evolution for general learning rate schedules.

Result: FSL successfully captures learning rate schedule effects through explicit functional terms, theoretically justifies empirical practices, and serves as a surrogate model for loss curve prediction and optimization across model sizes from 0.1B to 1B parameters.

Conclusion: The FSL framework deepens understanding of LLM pre-training dynamics and provides insights for improving large-scale model training by making learning rate schedule effects fully tractable.

Abstract: Scaling laws have played a cornerstone role in guiding the training of large language models (LLMs). However, most existing works on scaling laws primarily focus on the final-step loss, overlooking the loss dynamics during the training process and, crucially, the impact of learning rate schedule (LRS). In this paper, we aim to bridge this gap by studying a teacher-student kernel regression setup trained via online stochastic gradient descent (SGD). Leveraging a novel intrinsic time viewpoint and stochastic differential equation (SDE) modeling of SGD, we introduce the Functional Scaling Law (FSL), which characterizes the evolution of population risk during the training process for general LRSs. Remarkably, the impact of the LRSs is captured through an explicit convolution-type functional term, making their effects fully tractable. To illustrate the utility of FSL, we analyze three widely used LRSs – constant, exponential decay, and warmup-stable-decay (WSD) – under both data-limited and compute-limited regimes. We provide theoretical justification for widely adopted empirical practices in LLMs pre-training such as (i) higher-capacity models are more data- and compute-efficient; (ii) learning rate decay can improve training efficiency; (iii) WSD-like schedules can outperform direct-decay schedules. Lastly, we explore the practical relevance of FSL as a surrogate model for fitting, predicting and optimizing the loss curves in LLM pre-training, with experiments conducted across model sizes ranging from 0.1B to 1B parameters. We hope our FSL framework can deepen the understanding of LLM pre-training dynamics and provide insights for improving large-scale model training.

Method: Extracts weak robust samples directly from training dataset through local robustness analysis. These most susceptible samples serve as sensitive indicators of model vulnerabilities, enabling targeted performance enhancement.

Result: Demonstrated effectiveness on models trained with CIFAR-10, CIFAR-100, and ImageNet, showing that robustness validation guided by weak robust samples improves model reliability under adversarial and common corruption scenarios.

Conclusion: The proposed framework provides a more nuanced understanding of model robustness and enables meaningful improvements in reliability by focusing on the most challenging training instances rather than relying solely on perturbed test data.

Abstract: Data-driven models, especially deep learning classifiers often demonstrate great success on clean datasets. Yet, they remain vulnerable to common data distortions such as adversarial and common corruption perturbations. These perturbations can significantly degrade performance, thereby challenging the overall reliability of the models. Traditional robustness validation typically relies on perturbed test datasets to assess and improve model performance. In our framework, however, we propose a validation approach that extracts “weak robust” samples directly from the training dataset via local robustness analysis. These samples, being the most susceptible to perturbations, serve as an early and sensitive indicator of the model’s vulnerabilities. By evaluating models on these challenging training instances, we gain a more nuanced understanding of its robustness, which informs targeted performance enhancement. We demonstrate the effectiveness of our approach on models trained with CIFAR-10, CIFAR-100, and ImageNet, highlighting how robustness validation guided by weak robust samples can drive meaningful improvements in model reliability under adversarial and common corruption scenarios.

Method: FedFusion unifies domain adaptation and frugal labeling using diversity-/cluster-aware encoders (DivEn, DivEn-mix, DivEn-c). It employs confidence-filtered pseudo-labels, domain-adaptive transfer, personalized encoders, similarity-weighted classifier coupling, and a frugal-labeling pipeline combining self-/semi-supervised pretext training with selective fine-tuning.

Result: Across tabular and imaging benchmarks under various data regimes, FedFusion consistently outperforms state-of-the-art baselines in accuracy, robustness, and fairness while maintaining comparable communication and computation budgets.

Conclusion: Harmonizing personalization, domain adaptation, and label efficiency provides an effective approach for robust federated learning under real-world constraints.

Abstract: Federated learning in practice must contend with heterogeneous feature spaces, severe non-IID data, and scarce labels across clients. We present FedFusion, a federated transfer-learning framework that unifies domain adaptation and frugal labelling with diversity-/cluster-aware encoders (DivEn, DivEn-mix, DivEn-c). Labelled teacher clients guide learner clients via confidence-filtered pseudo-labels and domain-adaptive transfer, while clients maintain personalised encoders tailored to local data. To preserve global coherence under heterogeneity, FedFusion employs similarity-weighted classifier coupling (with optional cluster-wise averaging), mitigating dominance by data-rich sites and improving minority-client performance. The frugal-labelling pipeline combines self-/semi-supervised pretext training with selective fine-tuning, reducing annotation demands without sharing raw data. Across tabular and imaging benchmarks under IID, non-IID, and label-scarce regimes, FedFusion consistently outperforms state-of-the-art baselines in accuracy, robustness, and fairness while maintaining comparable communication and computation budgets. These results show that harmonising personalisation, domain adaptation, and label efficiency is an effective recipe for robust federated learning under real-world constraints.

Method: The framework uses knowledge distillation with morphology distillation to preserve local waveform patterns and rhythm distillation to capture inter-patch temporal structures through prediction-, feature-, and patch-level distillation.

Result: PPG-Distill improves student model performance by up to 21.8% on heart rate estimation and atrial fibrillation detection while achieving 7X faster inference and reducing memory usage by 19X.

Conclusion: PPG-Distill enables efficient PPG analysis on wearable devices by successfully transferring knowledge from large foundation models to compact student models while maintaining performance.

Abstract: Photoplethysmography (PPG) is widely used in wearable health monitoring, yet large PPG foundation models remain difficult to deploy on resource-limited devices. We present PPG-Distill, a knowledge distillation framework that transfers both global and local knowledge through prediction-, feature-, and patch-level distillation. PPG-Distill incorporates morphology distillation to preserve local waveform patterns and rhythm distillation to capture inter-patch temporal structures. On heart rate estimation and atrial fibrillation detection, PPG-Distill improves student performance by up to 21.8% while achieving 7X faster inference and reducing memory usage by 19X, enabling efficient PPG analysis on wearables

Method: Developed a compute-bound analytical model predicting scaling laws for spatial resolution, temporal length, and denoising steps, then validated through experiments on WAN2.1-T2V and extended analysis to six diverse T2V models.

Result: Found quadratic growth in computational costs with spatial and temporal dimensions, linear scaling with denoising steps, and provided runtime and energy profiles for six different models under default settings.

Conclusion: The study provides benchmark references and practical insights for designing and deploying more sustainable generative video systems by understanding their computational and energy requirements.

Abstract: Recent advances in text-to-video (T2V) generation have enabled the creation of high-fidelity, temporally coherent clips from natural language prompts. Yet these systems come with significant computational costs, and their energy demands remain poorly understood. In this paper, we present a systematic study of the latency and energy consumption of state-of-the-art open-source T2V models. We first develop a compute-bound analytical model that predicts scaling laws with respect to spatial resolution, temporal length, and denoising steps. We then validate these predictions through fine-grained experiments on WAN2.1-T2V, showing quadratic growth with spatial and temporal dimensions, and linear scaling with the number of denoising steps. Finally, we extend our analysis to six diverse T2V models, comparing their runtime and energy profiles under default settings. Our results provide both a benchmark reference and practical insights for designing and deploying more sustainable generative video systems.

Method: The study uses LIPS benchmarking pipeline to evaluate various hybridization strategies (regularization terms, unsupervised losses) and model architectures (multilayer perceptrons to graph-based networks) across four dimensions: accuracy, physical compliance, industrial readiness, and out-of-distribution generalization.

Result: The results demonstrate how different approaches to integrating physical knowledge impact performance across the evaluation criteria, providing insights into optimal hybridization strategies for power flow simulation.

Conclusion: The study demystifies physics-informed ML approaches for power grid applications, offering reproducible implementations and guidelines for developing effective hybrid models that balance speed, accuracy, and physical compliance.

Abstract: In the context of the energy transition, with increasing integration of renewable sources and cross-border electricity exchanges, power grids are encountering greater uncertainty and operational risk. Maintaining grid stability under varying conditions is a complex task, and power flow simulators are commonly used to support operators by evaluating potential actions before implementation. However, traditional physical solvers, while accurate, are often too slow for near real-time use. Machine learning models have emerged as fast surrogates, and to improve their adherence to physical laws (e.g., Kirchhoff’s laws), they are often trained with embedded constraints which are also known as physics-informed or hybrid models. This paper presents an ablation study to demystify hybridization strategies, ranging from incorporating physical constraints as regularization terms or unsupervised losses, and exploring model architectures from simple multilayer perceptrons to advanced graph-based networks enabling the direct optimization of physics equations. Using our custom benchmarking pipeline for hybrid models called LIPS, we evaluate these models across four dimensions: accuracy, physical compliance, industrial readiness, and out-of-distribution generalization. The results highlight how integrating physical knowledge impacts performance across these criteria. All the implementations are reproducible and provided in the corresponding Github page.

Method: The authors use algorithmic stability analysis to study adversarial training under the diffusion strategy for convex losses. They derive theoretical bounds on generalization error.

Result: The analysis shows that generalization error grows with both adversarial perturbation strength and number of training steps, consistent with single-agent case but novel for decentralized settings. Numerical experiments on logistic regression validate these theoretical predictions.

Conclusion: This work provides the first stability-based generalization analysis of adversarial training in decentralized networks, establishing theoretical bounds that explain the relationship between perturbation strength, training steps, and generalization error.

Abstract: Algorithmic stability is an established tool for analyzing generalization. While adversarial training enhances model robustness, it often suffers from robust overfitting and an enlarged generalization gap. Although recent work has established the convergence of adversarial training in decentralized networks, its generalization properties remain unexplored. This work presents a stability-based generalization analysis of adversarial training under the diffusion strategy for convex losses. We derive a bound showing that the generalization error grows with both the adversarial perturbation strength and the number of training steps, a finding consistent with single-agent case but novel for decentralized settings. Numerical experiments on logistic regression validate these theoretical predictions.

Method: Systematic evaluation across ten large reasoning models on math and scientific reasoning tasks, introducing a graph view of CoT to extract structure and identify Failed-Step Fraction (FSF) metric, plus test-time ranking and branch-editing interventions.

Result: Both naive CoT lengthening and increased review are associated with lower accuracy. FSF consistently predicts correctness better than length or review ratio. Removing failed branches significantly improves accuracy, showing they bias subsequent reasoning.

Conclusion: Effective CoTs are those that fail less, supporting structure-aware test-time scaling over indiscriminately generating long CoT traces.

Abstract: Large reasoning models (LRMs) spend substantial test-time compute on long chain-of-thought (CoT) traces, but what characterizes an effective CoT remains unclear. While prior work reports gains from lengthening CoTs and increasing review (revisiting earlier steps) via appended wait tokens, recent studies suggest that shorter thinking can outperform longer traces. We therefore conduct a systematic evaluation across ten LRMs on math and scientific reasoning. Contrary to the “longer-is-better” narrative, we find that both naive CoT lengthening and increased review are associated with lower accuracy. As CoT unfolds step by step, token-level metrics can conflate verbosity with process quality. We introduce a graph view of CoT to extract structure and identify a single statistic-the Failed-Step Fraction (FSF), the fraction of steps in abandoned branches-that consistently outpredicts length and review ratio for correctness across models. To probe causality, we design two interventions. First, we rank candidate CoTs by each metric at test time, where FSF yields the largest pass@1 gains; second, we edit CoTs to remove failed branches, which significantly improves accuracy, indicating that failed branches bias subsequent reasoning. Taken together, these results characterize effective CoTs as those that fail less and support structure-aware test-time scaling over indiscriminately generating long CoT.

Method: Conducted extensive experiments on 21 few-shot learning benchmarks using same architecture, optimizer, and convergence criteria. Used effect size (Cohen’s d) for statistical significance and diversity coefficient to measure dataset formal diversity.

Result: 1) Pre-training beats MAML when dataset diversity is low; 2) MAML beats pre-training when dataset diversity is high. However, the effect size differences are small (<0.2). No significant difference found between MAML and GPT-2 pre-training on Openwebtext.

Conclusion: Pre-trained models do not always beat meta-learned models - dataset formal diversity is a key determining factor in which approach performs better.

Abstract: In the context of few-shot learning, it is currently believed that a fixed pre-trained (PT) model, along with fine-tuning the final layer during evaluation, outperforms standard meta-learning algorithms. We re-evaluate these claims under an in-depth empirical examination of an extensive set of formally diverse datasets and compare PT to Model Agnostic Meta-Learning (MAML). Unlike previous work, we emphasize a fair comparison by using: the same architecture, the same optimizer, and all models trained to convergence. Crucially, we use a more rigorous statistical tool – the effect size (Cohen’s d) – to determine the practical significance of the difference between a model trained with PT vs. a MAML. We then use a previously proposed metric – the diversity coefficient – to compute the average formal diversity of a dataset. Using this analysis, we demonstrate the following: 1. when the formal diversity of a data set is low, PT beats MAML on average and 2. when the formal diversity is high, MAML beats PT on average. The caveat is that the magnitude of the average difference between a PT vs. MAML using the effect size is low (according to classical statistical thresholds) – less than 0.2. Nevertheless, this observation is contrary to the currently held belief that a pre-trained model is always better than a meta-learning model. Our extensive experiments consider 21 few-shot learning benchmarks, including the large-scale few-shot learning dataset Meta-Data set. We also show no significant difference between a MAML model vs. a PT model with GPT-2 on Openwebtext. We, therefore, conclude that a pre-trained model does not always beat a meta-learned model and that the formal diversity of a dataset is a driving factor.

Method: DOTA continuously estimates the underlying distribution of test data streams and computes test-time posterior probabilities using these dynamically estimated distributions via Bayes’ theorem, enabling continual adaptation to the deployment environment.

Result: Extensive experiments show that DOTA significantly mitigates forgetting and achieves state-of-the-art performance compared to existing methods.

Conclusion: The distribution-centric approach of DOTA provides an effective solution for test-time adaptation in vision-language models, overcoming limitations of naive cache management while maintaining efficiency.

Abstract: Vision-language foundation models (VLMs), such as CLIP, exhibit remarkable performance across a wide range of tasks. However, deploying these models can be unreliable when significant distribution gaps exist between training and test data, while fine-tuning for diverse scenarios is often costly. Cache-based test-time adapters offer an efficient alternative by storing representative test samples to guide subsequent classifications. Yet, these methods typically employ naive cache management with limited capacity, leading to severe catastrophic forgetting when samples are inevitably dropped during updates. In this paper, we propose DOTA (DistributiOnal Test-time Adaptation), a simple yet effective method addressing this limitation. Crucially, instead of merely memorizing individual test samples, DOTA continuously estimates the underlying distribution of the test data stream. Test-time posterior probabilities are then computed using these dynamically estimated distributions via Bayes’ theorem for adaptation. This distribution-centric approach enables the model to continually learn and adapt to the deployment environment. Extensive experiments validate that DOTA significantly mitigates forgetting and achieves state-of-the-art performance compared to existing methods.

Method: Proposes circuit-tuning algorithm that iteratively builds task-specific subgraphs and updates relevant parameters heuristically, viewing models as redundant computational graphs.

Result: Validated hypothesis through experiments, showing circuit-tuning balances target task performance with general capabilities, and provided detailed analysis of learning dynamics.

Conclusion: Offers new analytical method for fine-tuning dynamics, reveals mechanisms behind training process, and inspires better neural network training algorithms.

Abstract: The study of mechanistic interpretability aims to reverse-engineer a model to explain its behaviors. While recent studies have focused on the static mechanism of a certain behavior, the learning dynamics inside a model remain to be explored. In this work, we develop a fine-tuning method for analyzing the mechanism behind learning. Inspired by the concept of intrinsic dimension, we view a model as a computational graph with redundancy for a specific task, and treat the fine-tuning process as a search for and optimization of a subgraph within this graph. Based on this hypothesis, we propose circuit-tuning, an algorithm that iteratively builds the subgraph for a specific task and updates the relevant parameters in a heuristic way. We first validate our hypothesis through a carefully designed experiment and provide a detailed analysis of the learning dynamics during fine-tuning. Subsequently, we conduct experiments on more complex tasks, demonstrating that circuit-tuning could strike a balance between the performance on the target task and the general capabilities. Our work offers a new analytical method for the dynamics of fine-tuning, provides new findings on the mechanisms behind the training process, and inspires the design of superior algorithms for the training of neural networks.

Method: UoE decomposes transformer models into expert groups using hierarchical routing that integrates patch-wise data selection and expert selection, extends MoE to attention blocks, and uses hardware-optimized parallelization with batched matrix multiplications.

Result: UoE outperforms Full Attention, state-of-the-art MoEs and efficient transformers: 2.38 perplexity reduction in language modeling with 76% FLOPs, 0.68% higher LRA benchmark score with 50% FLOPs, and 1.75% accuracy improvement in image classification with comparable FLOPs.

Conclusion: UoE successfully addresses limitations of conventional MoE architectures through innovative hierarchical routing and attention block integration, demonstrating superior efficiency and performance across multiple domains.

Abstract: Mixture-of-Experts (MoE) enhances model performance while maintaining computational efficiency, making it well-suited for large-scale applications. Conventional mixture-of-experts (MoE) architectures suffer from suboptimal coordination dynamics, where isolated expert operations expose the model to overfitting risks. Moreover, they have not been effectively extended to attention blocks, which limits further efficiency improvements. To tackle these issues, we propose Union-of-Experts (UoE), which decomposes the transformer model into an equivalent group of experts and applies a hierarchical routing mechanism to allocate input subspaces to specialized experts. Our approach advances MoE design with four key innovations: (1) Constructing expert groups by partitioning non-MoE models into functionally equivalent specialists (2) Developing a hierarchical routing paradigm that integrates patch-wise data selection and expert selection strategies. (3) Extending the MoE design to attention blocks. (4) Proposing a hardware-optimized parallelization scheme that exploits batched matrix multiplications for efficient expert computation. The experiments demonstrate that our UoE model surpasses Full Attention, state-of-the-art MoEs and efficient transformers in several tasks across image and natural language domains. In language modeling tasks, UoE achieves an average reduction of 2.38 in perplexity compared to the best-performing MoE method with only 76% of its FLOPs. In the Long Range Arena benchmark, it demonstrates an average score at least 0.68% higher than all comparison models, with only 50% of the FLOPs of the best MoE method. In image classification, it yields an average accuracy improvement of 1.75% over the best model while maintaining comparable FLOPs. The source codes are available at https://github.com/YujiaoYang-work/UoE.

Method: The survey explores SAEs’ technical framework including architecture, design improvements, and training strategies; examines input-based and output-based feature explanation methods; discusses structural and functional evaluation metrics; and investigates real-world applications.

Result: The paper systematically organizes current knowledge on SAEs for LLM interpretability, providing a comprehensive overview of methodologies, evaluation approaches, and practical applications in understanding and manipulating LLM behaviors.

Conclusion: SAEs represent a significant advancement in mechanistic interpretability for LLMs, offering structured approaches to decode complex model internals through feature disentanglement, with promising applications in model understanding and behavior manipulation.

Abstract: Large Language Models (LLMs) have transformed natural language processing, yet their internal mechanisms remain largely opaque. Recently, mechanistic interpretability has attracted significant attention from the research community as a means to understand the inner workings of LLMs. Among various mechanistic interpretability approaches, Sparse Autoencoders (SAEs) have emerged as a promising method due to their ability to disentangle the complex, superimposed features within LLMs into more interpretable components. This paper presents a comprehensive survey of SAEs for interpreting and understanding the internal workings of LLMs. Our major contributions include: (1) exploring the technical framework of SAEs, covering basic architecture, design improvements, and effective training strategies; (2) examining different approaches to explaining SAE features, categorized into input-based and output-based explanation methods; (3) discussing evaluation methods for assessing SAE performance, covering both structural and functional metrics; and (4) investigating real-world applications of SAEs in understanding and manipulating LLM behaviors.

Method: Proposes Gradient Sparse Autoencoder (GradSAE) that uses output-side gradient information to measure causal influence and identify the most important latents for model steering.

Result: The method validates that not all activated latents contribute equally to model outputs, and only latents with high causal influence are effective for model steering.

Conclusion: GradSAE provides a more effective approach for interpreting and steering LLM representations by focusing on causally influential latents rather than just activated ones.

Abstract: Sparse Autoencoders (SAEs) have recently emerged as powerful tools for interpreting and steering the internal representations of large language models (LLMs). However, conventional approaches to analyzing SAEs typically rely solely on input-side activations, without considering the causal influence between each latent feature and the model’s output. This work is built on two key hypotheses: (1) activated latents do not contribute equally to the construction of the model’s output, and (2) only latents with high causal influence are effective for model steering. To validate these hypotheses, we propose Gradient Sparse Autoencoder (GradSAE), a simple yet effective method that identifies the most influential latents by incorporating output-side gradient information.

Method: The authors propose using prediction consistency between weak and strong completers as a criterion for identifying reliable process labels. They also develop ORM initialization and up-sampling for negative data to improve PRM performance. The approach is validated in three scenarios: verification for test time scaling, direct evaluation of reasoning step correctness, and reward ranked fine-tuning.

Result: Athena-PRM achieves superior performance with only 5,000 samples, improving Qwen2.5-VL-7B by 10.2 points on WeMath and 7.1 points on MathVista for test time scaling. It sets new state-of-the-art results in VisualProcessBench, outperforming previous SoTA by 3.9 F1-score. Athena-7B developed with reward ranked fine-tuning significantly outperforms baselines on five benchmarks.

Conclusion: The proposed method efficiently generates high-quality process-labeled data using prediction consistency, enabling effective process reward modeling with minimal data requirements. Athena-PRM demonstrates robust capabilities across various multimodal reasoning scenarios and establishes new state-of-the-art performance.

Abstract: We present Athena-PRM, a multimodal process reward model (PRM) designed to evaluate the reward score for each step in solving complex reasoning problems. Developing high-performance PRMs typically demands significant time and financial investment, primarily due to the necessity for step-level annotations of reasoning steps. Conventional automated labeling methods, such as Monte Carlo estimation, often produce noisy labels and incur substantial computational costs. To efficiently generate high-quality process-labeled data, we propose leveraging prediction consistency between weak and strong completers as a criterion for identifying reliable process labels. Remarkably, Athena-PRM demonstrates outstanding effectiveness across various scenarios and benchmarks with just 5,000 samples. Furthermore, we also develop two effective strategies to improve the performance of PRMs: ORM initialization and up-sampling for negative data. We validate our approach in three specific scenarios: verification for test time scaling, direct evaluation of reasoning step correctness, and reward ranked fine-tuning. Our Athena-PRM consistently achieves superior performance across multiple benchmarks and scenarios. Notably, when using Qwen2.5-VL-7B as the policy model, Athena-PRM enhances performance by 10.2 points on WeMath and 7.1 points on MathVista for test time scaling. Furthermore, Athena-PRM sets the state-of-the-art (SoTA) results in VisualProcessBench and outperforms the previous SoTA by 3.9 F1-score, showcasing its robust capability to accurately assess the correctness of the reasoning step. Additionally, utilizing Athena-PRM as the reward model, we develop Athena-7B with reward ranked fine-tuning and outperforms baseline with a significant margin on five benchmarks.

Method: Pretrained decoder-only transformer models on Epic Cosmos dataset (115 billion medical events from 118 million patients) using autoregressive prediction of next medical events, with scaling-law analysis for compute-optimal models up to 1B parameters.

Result: Comet outperformed or matched task-specific supervised models on 78 real-world tasks (diagnosis prediction, disease prognosis, healthcare operations) without task-specific fine-tuning, with performance improving with model scale.

Conclusion: Generative medical event foundation models like Comet can effectively capture clinical dynamics and provide a generalizable framework for clinical decision-making and healthcare operations.

Abstract: Realizing personalized medicine at scale calls for methods that distill insights from longitudinal patient journeys, which can be viewed as a sequence of medical events. Foundation models pretrained on large-scale medical event data represent a promising direction for scaling real-world evidence generation and generalizing to diverse downstream tasks. Using Epic Cosmos, a dataset with medical events from de-identified longitudinal health records for 16.3 billion encounters over 300 million unique patient records from 310 health systems, we introduce the Comet models, a family of decoder-only transformer models pretrained on 118 million patients representing 115 billion discrete medical events (151 billion tokens). We present the largest scaling-law study of medical event data, establishing a methodology for pretraining and revealing power-law scaling relationships for compute, tokens, and model size. Consequently, we pretrained a series of compute-optimal models with up to 1 billion parameters. Conditioned on a patient’s real-world history, Comet autoregressively predicts the next medical event to simulate patient health timelines. We studied 78 real-world tasks, including diagnosis prediction, disease prognosis, and healthcare operations. Remarkably for a foundation model with generic pretraining and simulation-based inference, Comet generally outperformed or matched task-specific supervised models on these tasks, without requiring task-specific fine-tuning or few-shot examples. Comet’s predictive power consistently improves as the model and pretraining scale. Our results show that Comet, a generative medical event foundation model, can effectively capture complex clinical dynamics, providing an extensible and generalizable framework to support clinical decision-making, streamline healthcare operations, and improve patient outcomes.

Method: Proposes REFINE with three systematic queries (Feed-Target, Feed-Check, Feed-Path) to construct structured feedback, prioritizing visual information, diagnosing failure points, and formulating corrective actions while optimizing retrieval efficiency.

Result: Demonstrates substantial speedup, reduced computational costs, and successful generalization, highlighting improved multimodal reasoning performance.

Conclusion: REFINE shows potential for enhancing multimodal reasoning through systematic error structuring and targeted feedback, offering improved efficiency and scalability over previous approaches.

Abstract: Recent advancements in Large Language Models (LLMs) have significantly improved reasoning capabilities, with in-context learning (ICL) emerging as a key technique for adaptation without retraining. While previous works have focused on leveraging correct examples, recent research highlights the importance of learning from errors to enhance performance. However, existing methods lack a structured framework for analyzing and mitigating errors, particularly in Multimodal Large Language Models (MLLMs), where integrating visual and textual inputs adds complexity. To address this issue, we propose REFINE: Retrieval-Enhanced Feedback via In-context Neural Error-book, a teacher-student framework that systematically structures errors and provides targeted feedback. REFINE introduces three systematic queries to construct structured feedback – Feed-Target, Feed-Check, and Feed-Path – to enhance multimodal reasoning by prioritizing relevant visual information, diagnosing critical failure points, and formulating corrective actions. Unlike prior approaches that rely on redundant retrievals, REFINE optimizes structured feedback retrieval, improving inference efficiency, token usage, and scalability. Our results demonstrate substantial speedup, reduced computational costs, and successful generalization, highlighting REFINE’s potential for enhancing multimodal reasoning.

Method: Proposes a versatile framework of interpretable methods for comparing datasets across diverse modalities including tabular data, text, images, and time-series signals

Result: Demonstrated effectiveness across various case studies in both low and high-dimensional settings

Conclusion: The framework provides actionable and interpretable insights to complement existing techniques for understanding and addressing distribution shifts

Abstract: The performance of machine learning models relies heavily on the quality of input data, yet real-world applications often face significant data-related challenges. A common issue arises when curating training data or deploying models: two datasets from the same domain may exhibit differing distributions. While many techniques exist for detecting such distribution shifts, there is a lack of comprehensive methods to explain these differences in a human-understandable way beyond opaque quantitative metrics. To bridge this gap, we propose a versatile framework of interpretable methods for comparing datasets. Using a variety of case studies, we demonstrate the effectiveness of our approach across diverse data modalities-including tabular data, text data, images, time-series signals – in both low and high-dimensional settings. These methods complement existing techniques by providing actionable and interpretable insights to better understand and address distribution shifts.

Method: FSPO clips sequence log-IS ratio with a band that scales as √L (square root of sequence length), formalizing length fairness via Length Reweighting Error (LRE) metric.

Result: FSPO flattens clip rates across length bins, stabilizes training, and outperforms all baselines across multiple evaluation datasets on Qwen3-8B-Base model.

Conclusion: The proposed length-fair clipping mechanism effectively addresses length bias in sequence-level RL for LLMs, providing theoretical guarantees and empirical improvements.

Abstract: We propose FSPO (Fair Sequence Policy Optimization), a sequence-level reinforcement learning method for LLMs that enforces length-fair clipping on the importance-sampling (IS) weight. We study RL methods with sequence-level IS and identify a mismatch when PPO/GRPO-style clipping is transplanted to sequences: a fixed clip range systematically reweights short vs.\ long responses, distorting the optimization direction. FSPO introduces a simple remedy: we clip the sequence log-IS ratio with a band that scales as $\sqrt{L}$. Theoretically, we formalize length fairness via a Length Reweighting Error (LRE) and prove that small LRE yields a cosine directional guarantee between the clipped and true updates. Empirically, FSPO flattens clip rates across length bins, stabilizes training, and outperforms all baselines across multiple evaluation datasets on Qwen3-8B-Base model.

Method: Leverages DP framework to provide theoretical bounds on information leakage, performs inference on private records, aggregates per-token output distributions, and uses blending operation to combine private and public inference.

Result: Outperforms previous state-of-the-art methods on in-context-learning tasks, enabling generation of longer coherent synthetic text while maintaining privacy.

Conclusion: Promising direction for privacy-preserving text generation that maintains high utility without requiring model fine-tuning.

Abstract: Large language models (LLMs) have significantly transformed natural language understanding and generation, but they raise privacy concerns due to potential exposure of sensitive information. Studies have highlighted the risk of information leakage, where adversaries can extract sensitive information embedded in the prompts. In this work, we introduce a novel private prediction framework for generating high-quality synthetic text with strong privacy guarantees. Our approach leverages the Differential Privacy (DP) framework to ensure worst-case theoretical bounds on information leakage without requiring any fine-tuning of the underlying models. The proposed method performs inference on private records and aggregates the resulting per-token output distributions. This enables the generation of longer and coherent synthetic text while maintaining privacy guarantees. Additionally, we propose a simple blending operation that combines private and public inference to further enhance utility. Empirical evaluations demonstrate that our approach outperforms previous state-of-the-art methods on in-context-learning (ICL) tasks, making it a promising direction for privacy-preserving text generation while maintaining high utility.

Method: Uses an Expert Block Framework with a Trajectory Context Summarizer (TCS) that transforms raw training trajectories into structured context, enabling small LLMs (phi4:reasoning14B and qwen2.5-coder:32B) to analyze optimization progress effectively.

Result: With a 10-trial budget, the TCS-enabled HPT pipeline achieves average performance within ~0.9 percentage points of GPT-4 across six diverse tasks using much smaller locally-run LLMs.

Conclusion: The framework demonstrates that small LLMs can achieve reliable HPT performance comparable to large models when properly structured with trajectory context summarization, making HPT more efficient and accessible.

Abstract: Hyper-parameter Tuning (HPT) is a necessary step in machine learning (ML) pipelines but becomes computationally expensive and opaque with larger models. Recently, Large Language Models (LLMs) have been explored for HPT, yet most rely on models exceeding 100 billion parameters. We propose an Expert Block Framework for HPT using Small LLMs. At its core is the Trajectory Context Summarizer (TCS), a deterministic block that transforms raw training trajectories into structured context, enabling small LLMs to analyze optimization progress with reliability comparable to larger models. Using two locally-run LLMs (phi4:reasoning14B and qwen2.5-coder:32B) and a 10-trial budget, our TCS-enabled HPT pipeline achieves average performance within ~0.9 percentage points of GPT-4 across six diverse tasks.

Method: Foresight dynamically identifies and reuses DiT block outputs for all layers across denoising steps, adapting to generation parameters like resolution and denoising schedules to optimize computational efficiency.

Result: Applied to OpenSora, Latte, and CogVideoX, Foresight achieves up to significant end-to-end speedup while maintaining video quality comparable to baseline performance.

Conclusion: Foresight provides an effective adaptive caching solution that reduces computational redundancy in DiT-based video generation, enabling faster inference without sacrificing quality.

Abstract: Diffusion Transformers (DiTs) achieve state-of-the-art results in text-to-image, text-to-video generation, and editing. However, their large model size and the quadratic cost of spatial-temporal attention over multiple denoising steps make video generation computationally expensive. Static caching mitigates this by reusing features across fixed steps but fails to adapt to generation dynamics, leading to suboptimal trade-offs between speed and quality. We propose Foresight, an adaptive layer-reuse technique that reduces computational redundancy across denoising steps while preserving baseline performance. Foresight dynamically identifies and reuses DiT block outputs for all layers across steps, adapting to generation parameters such as resolution and denoising schedules to optimize efficiency. Applied to OpenSora, Latte, and CogVideoX, Foresight achieves up to \latencyimprv end-to-end speedup, while maintaining video quality. The source code of Foresight is available at \href{https://github.com/STAR-Laboratory/foresight}{https://github.com/STAR-Laboratory/foresight}.

Method: The authors present a new definition of Markov Potential Games that generalizes prior attempts, analyze their properties, and prove fast convergence of independent policy gradient to Nash policies by adapting gradient dominance property arguments from single-agent MDPs.

Result: Counter-intuitively, MPGs can include settings where state-games are zero-sum games, and Markov games where every state-game is a potential game are not necessarily MPGs. However, MPGs maintain desirable properties like existence of deterministic Nash policies.

Conclusion: The paper establishes a rigorous foundation for Markov Potential Games, demonstrating both surprising differences from normal-form potential games and providing convergence guarantees for independent policy gradient learning in these stateful multi-agent coordination settings.

Abstract: Potential games are arguably one of the most important and widely studied classes of normal form games. They define the archetypal setting of multi-agent coordination as all agent utilities are perfectly aligned with each other via a common potential function. Can this intuitive framework be transplanted in the setting of Markov Games? What are the similarities and differences between multi-agent coordination with and without state dependence? We present a novel definition of Markov Potential Games (MPG) that generalizes prior attempts at capturing complex stateful multi-agent coordination. Counter-intuitively, insights from normal-form potential games do not carry over as MPGs can consist of settings where state-games can be zero-sum games. In the opposite direction, Markov games where every state-game is a potential game are not necessarily MPGs. Nevertheless, MPGs showcase standard desirable properties such as the existence of deterministic Nash policies. In our main technical result, we prove fast convergence of independent policy gradient to Nash policies by adapting recent gradient dominance property arguments developed for single agent MDPs to multi-agent learning settings.

Method: An iterative framework alternating between: (i) detecting mis-specified clusters in local solutions, and (ii) improving solutions via non-local operations. Includes variants for handling over- and under-specified initial cluster numbers.

Result: The framework unifies existing k-means variants through geometric perspective and demonstrates efficacy through theoretical justifications and extensive experiments.

Conclusion: The proposed approach provides an effective method for escaping local optima in k-means clustering, with practical variants for different scenarios of cluster specification.

Abstract: \kmeans clustering is a fundamental problem in many scientific and engineering domains. The optimization problem associated with \kmeans clustering is nonconvex, for which standard algorithms are only guaranteed to find a local optimum. Leveraging the hidden structure of local solutions, we propose a general algorithmic framework for escaping undesirable local solutions and recovering the global solution or the ground truth clustering. This framework consists of iteratively alternating between two steps: (i) detect mis-specified clusters in a local solution, and (ii) improve the local solution by non-local operations. We discuss specific implementation of these steps, and elucidate how the proposed framework unifies many existing variants of \kmeans algorithms through a geometric perspective. We also present two natural variants of the proposed framework, where the initial number of clusters may be over- or under-specified. We provide theoretical justifications and extensive experiments to demonstrate the efficacy of the proposed approach.

Method: PE uses grouped convolutions to parallelize ensemble members into a single shared backbone, carefully modulating the encoding space dimension. This allows training and inference in a single forward pass while operating within standard neural network memory limits.

Result: PE preserves Deep Ensemble properties like diversity and performs equally well on accuracy, calibration, out-of-distribution detection, and robustness to distribution shift, while being significantly more efficient.

Conclusion: Packed-Ensembles provide a practical solution for deploying ensemble methods in resource-constrained environments, maintaining the benefits of Deep Ensembles with improved computational efficiency.

Abstract: Deep Ensembles (DE) are a prominent approach for achieving excellent performance on key metrics such as accuracy, calibration, uncertainty estimation, and out-of-distribution detection. However, hardware limitations of real-world systems constrain to smaller ensembles and lower-capacity networks, significantly deteriorating their performance and properties. We introduce Packed-Ensembles (PE), a strategy to design and train lightweight structured ensembles by carefully modulating the dimension of their encoding space. We leverage grouped convolutions to parallelize the ensemble into a single shared backbone and forward pass to improve training and inference speeds. PE is designed to operate within the memory limits of a standard neural network. Our extensive research indicates that PE accurately preserves the properties of DE, such as diversity, and performs equally well in terms of accuracy, calibration, out-of-distribution detection, and robustness to distribution shift. We make our code available at https://github.com/ENSTA-U2IS/torch-uncertainty.

Method: Introduces random Tikhonov regularization path concept, decomposes tracking error into martingale difference sequences, uses operator theory (monotonicity of operator inverses, spectral decomposition) and develops a dominated convergence method with RKHS persistence of excitation condition.

Result: Shows that with slowly time-varying regularization paths, the algorithm achieves mean square consistency with the path. For independent non-identically distributed data, consistency holds when marginal probability measures are slowly varying with uniformly positive lower bounds.

Conclusion: The framework provides rigorous convergence guarantees for recursive learning in RKHS under realistic data stream conditions, bridging theory and practical applications where data dependencies and non-stationarity are common.

Abstract: We study the convergence of recursive regularized learning algorithms in the reproducing kernel Hilbert space (RKHS) with dependent and non-stationary online data streams. Firstly, we introduce the concept of random Tikhonov regularization path and decompose the tracking error of the algorithm’s output for the regularization path into random difference equations in RKHS, whose non-homogeneous terms are martingale difference sequences. Investigating the mean square asymptotic stability of the equations, we show that if the regularization path is slowly time-varying, then the algorithm’s output achieves mean square consistency with the regularization path. Leveraging operator theory, particularly the monotonicity of the inverses of operators and the spectral decomposition of compact operators, we introduce the RKHS persistence of excitation condition (i.e. there exists a fixed-length time period, such that the conditional expectation of the operators induced by the input data accumulated over every period has a uniformly strictly positive compact lower bound) and develop a dominated convergence method to prove the mean square consistency between the algorithm’s output and an unknown function. Finally, for independent and non-identically distributed data streams, the algorithm achieves the mean square consistency if the input data’s marginal probability measures are slowly time-varying and the average measure over each fixed-length time period has a uniformly strictly positive lower bound.

Method: HEF integrates R2, MAE, and RMSE with dynamic weights, tolerance thresholds, and progressive penalty mechanisms. Implemented with Grid Search, PSO, and Optuna on Walmart, M3, M4, and M5 datasets.

Result: HEF consistently outperforms MAE in global metrics (R2, GRA, RMSE, RMSSE), providing greater explanatory power, adaptability, and stability while maintaining robustness against extreme errors.

Conclusion: HEF is a robust and adaptive alternative for model selection and hyperparameter optimization in variable demand forecasting environments, particularly effective for long-term planning and complex contexts.

Abstract: Accurate demand forecasting is crucial for effective inventory management in dynamic and competitive environments, where decisions are influenced by uncertainty, financial constraints, and logistical limitations. Traditional evaluation metrics such as Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) provide complementary perspectives but may lead to biased assessments when applied individually. To address this limitation, we propose the Hierarchical Evaluation Function (HEF), a composite function that integrates R2, MAE, and RMSE within a hierarchical and adaptive framework. The function incorporates dynamic weights, tolerance thresholds derived from the statistical properties of the series, and progressive penalty mechanisms to ensure robustness against extreme errors and invalid predictions. HEF was implemented to optimize multiple forecasting models using Grid Search, Particle Swarm Optimization (PSO), and Optuna, and tested on benchmark datasets including Walmart, M3, M4, and M5. Experimental results, validated through statistical tests, demonstrate that HEF consistently outperforms MAE as an evaluation function in global metrics such as R2, Global Relative Accuracy (GRA), RMSE, and RMSSE, thereby providing greater explanatory power, adaptability, and stability. While MAE retains advantages in simplicity and efficiency, HEF proves more effective for long-term planning and complex contexts. Overall, HEF constitutes a robust and adaptive alternative for model selection and hyperparameter optimization in highly variable demand forecasting environments.

Method: Proposed Spectraformer framework that uses random feature-based approach with various combinations of weight matrices and component functions to approximate and learn the kernel function in attention mechanisms.

Result: Achieved performance comparable to top-performing sparse and low-rank methods on Long Range Arena benchmark, establishing new state-of-the-art for random feature-based efficient Transformers with different variants offering trade-offs in accuracy, training time, and memory.

Conclusion: Spectraformer demonstrates that random feature-based approaches can compete with sparse and low-rank methods, providing a unified framework that produces multiple variants with different performance characteristics.

Abstract: Linearization of attention using various kernel approximation and kernel learning techniques has shown promise. Past methods used a subset of combinations of component functions and weight matrices within the random feature paradigm. We identify the need for a systematic comparison of different combinations of weight matrices and component functions for attention learning in Transformer. Hence, we introduce Spectraformer, a unified framework for approximating and learning the kernel function in the attention mechanism of the Transformer. Our empirical results demonstrate, for the first time, that a random feature-based approach can achieve performance comparable to top-performing sparse and low-rank methods on the challenging Long Range Arena benchmark. Thus, we establish a new state-of-the-art for random feature-based efficient Transformers. The framework also produces many variants that offer different advantages in accuracy, training time, and memory consumption. Our code is available at: https://github.com/cruiseresearchgroup/spectraformer .

Method: Unlearning-as-ablation: systematically remove target results along with their forget-closure (supporting lemmas, paraphrases, multi-hop entailments) and evaluate if models can re-derive results using only permitted axioms and tools.

Result: This is a position paper presenting a conceptual framework rather than empirical results. The authors outline a pilot study in mathematics and algorithms to demonstrate feasibility.

Conclusion: The proposed method could help distinguish between models that reconstruct knowledge versus those that merely retrieve it, and guide the development of next-generation AI-for-Science benchmarks.

Abstract: Bold claims about AI’s role in science-from “AGI will cure all diseases” to promises of radically accelerated discovery-raise a central epistemic question: do large language models (LLMs) truly generate new knowledge, or do they merely remix memorized fragments? We propose unlearning-as-ablation as a falsifiable probe of constructive scientific discovery. The idea is to systematically remove a target result together with its forget-closure (supporting lemmas, paraphrases, and multi-hop entailments) and then evaluate whether the model can re-derive the result from only permitted axioms and tools. Success would indicate generative capability beyond recall; failure would expose current limits. Unlike prevailing motivations for unlearning-privacy, copyright, or safety-our framing repositions it as an epistemic probe for AI-for-Science. We outline a minimal pilot in mathematics and algorithms to illustrate feasibility, and sketch how the same approach could later be extended to domains such as physics or chemistry. This is a position paper: our contribution is conceptual and methodological, not empirical. We aim to stimulate discussion on how principled ablation tests could help distinguish models that reconstruct knowledge from those that merely retrieve it, and how such probes might guide the next generation of AI-for-Science benchmarks.

Method: The experiment generates ultra-long time-series data at 10 million samples per second, where dark matter signals appear as sinusoidal oscillations. The release includes training, validation, and science datasets with denoising scores and analysis framework.

Result: Although no dark matter discovery has been made yet, ABRACADABRA has produced several widely-endorsed dark matter search results. The data release enables benchmarking and community-standard analysis.

Conclusion: TIDMAD provides a comprehensive framework for AI algorithms to extract potential dark matter signals from ultra-long time-series data, facilitating collaborative research and potentially leading to Nobel-Prize-level breakthroughs in dark matter detection.

Abstract: Dark matter makes up approximately 85% of total matter in our universe, yet it has never been directly observed in any laboratory on Earth. The origin of dark matter is one of the most important questions in contemporary physics, and a convincing detection of dark matter would be a Nobel-Prize-level breakthrough in fundamental science. The ABRACADABRA experiment was specifically designed to search for dark matter. Although it has not yet made a discovery, ABRACADABRA has produced several dark matter search results widely endorsed by the physics community. The experiment generates ultra-long time-series data at a rate of 10 million samples per second, where the dark matter signal would manifest itself as a sinusoidal oscillation mode within the ultra-long time series. In this paper, we present the TIDMAD – a comprehensive data release from the ABRACADABRA experiment including three key components: an ultra-long time series dataset divided into training, validation, and science subsets; a carefully-designed denoising score for direct model benchmarking; and a complete analysis framework which produces a community-standard dark matter search result suitable for publication as a physics paper. This data release enables core AI algorithms to extract the signal and produce real physics results thereby advancing fundamental science. The data downloading and associated analysis scripts are available at https://github.com/jessicafry/TIDMAD

Method: Proposes SON-NMF using sum-of-norm (group-lasso) regularization to encourage pairwise similarity and reduce rank automatically. Develops a first-order BCD algorithm with proximal average operator for efficient solving despite the problem’s nonconvex, nonsmooth, non-separable nature.

Result: SON-NMF successfully reveals correct nonnegative ranks on various datasets without prior knowledge. It handles rank-deficient matrices, detects weak components, and addresses spectral variability in hyperspectral imaging applications.

Conclusion: SON-NMF provides an effective automated approach for rank estimation in NMF with practical advantages for real-world applications, though computational complexity remains challenging due to the inherent NP-hard nature of the problem.

Abstract: When applying nonnegative matrix factorization (NMF), the rank parameter is generally unknown. This rank, called the nonnegative rank, is usually estimated heuristically since computing its exact value is NP-hard. In this work, we propose an approximation method to estimate the rank on-the-fly while solving NMF. We use the sum-of-norm (SON), a group-lasso structure that encourages pairwise sim- ilarity, to reduce the rank of a factor matrix when the initial rank is overestimated. On various datasets, SON-NMF can reveal the correct nonnegative rank of the data without prior knowledge or parameter tuning. SON-NMF is a nonconvex, nonsmooth, non-separable, and non-proximable problem, making it nontrivial to solve. First, since rank estimation in NMF is NP-hard, the proposed approach does not benefit from lower computational com- plexity. Using a graph-theoretic argument, we prove that the complexity of SON- NMF is essentially irreducible. Second, the per-iteration cost of algorithms for SON-NMF can be high. This motivates us to propose a first-order BCD algorithm that approximately solves SON-NMF with low per-iteration cost via the proximal average operator. SON-NMF exhibits favorable features for applications. Besides the ability to automatically estimate the rank from data, SON-NMF can handle rank-deficient data matrices and detect weak components with small energy. Furthermore, in hyperspectral imaging, SON-NMF naturally addresses the issue of spectral variability.

Method: Transformer-based architecture incorporating positional encodings and modality encodings derived from text embeddings of modality descriptions, supporting eight modalities including directional angles, categorical classes, and continuous properties.

Result: Reduces errors in predicting stress angle by more than a factor of three on validation data, with improved performance scaling with increased parameters.

Conclusion: Transparent Earth represents an initial foundation model for Earth’s subsurface that can scale to arbitrary modalities and demonstrates strong predictive capabilities through in-context learning.

Abstract: We present the Transparent Earth, a transformer-based architecture for reconstructing subsurface properties from heterogeneous datasets that vary in sparsity, resolution, and modality, where each modality represents a distinct type of observation (e.g., stress angle, mantle temperature, tectonic plate type). The model incorporates positional encodings of observations together with modality encodings, derived from a text embedding model applied to a description of each modality. This design enables the model to scale to an arbitrary number of modalities, making it straightforward to add new ones not considered in the initial design. We currently include eight modalities spanning directional angles, categorical classes, and continuous properties such as temperature and thickness. These capabilities support in-context learning, enabling the model to generate predictions either with no inputs or with an arbitrary number of additional observations from any subset of modalities. On validation data, this reduces errors in predicting stress angle by more than a factor of three. The proposed architecture is scalable and demonstrates improved performance with increased parameters. Together, these advances make the Transparent Earth an initial foundation model for the Earth’s subsurface that ultimately aims to predict any subsurface property anywhere on Earth.

Method: Uses a score-based diffusion model trained on high-resolution reanalysis data to capture statistical properties of local weather dynamics, then conditions on coarse climate model data to generate consistent weather patterns, leveraging probabilistic sampling for uncertainty.

Result: The model generates spatially and temporally coherent weather dynamics that align with global climate output, demonstrating effective downscaling while preserving physical consistency.

Conclusion: The diffusion-based generative framework successfully addresses the challenge of producing physically consistent high-resolution weather patterns from coarse climate simulations, enabling better local climate impact assessment.

Abstract: Local climate information is crucial for impact assessment and decision-making, yet coarse global climate simulations cannot capture small-scale phenomena. Current statistical downscaling methods infer these phenomena as temporally decoupled spatial patches. However, to preserve physical properties, estimating spatio-temporally coherent high-resolution weather dynamics for multiple variables across long time horizons is crucial. We present a novel generative framework that uses a score-based diffusion model trained on high-resolution reanalysis data to capture the statistical properties of local weather dynamics. After training, we condition on coarse climate model data to generate weather patterns consistent with the aggregate information. As this predictive task is inherently uncertain, we leverage the probabilistic nature of diffusion models and sample multiple trajectories. We evaluate our approach with high-resolution reanalysis information before applying it to the climate model downscaling task. We then demonstrate that the model generates spatially and temporally coherent weather dynamics that align with global climate output.

Method: The model enforces pairwise conservation of linear and angular momentum using edge-local reference frames that are equivariant to rotational symmetries, invariant to translations, and equivariant to node permutations, providing interpretable edge-wise linear and angular impulses.

Result: Evaluated on 3D granular systems with inelastic collisions, Dynami-CAL GraphNet shows stable error accumulation over extended rollouts, effective extrapolation to unseen configurations, and robust handling of heterogeneous interactions and external forces.

Conclusion: The approach offers significant advantages for fields requiring accurate, interpretable, real-time modeling of complex multi-body systems, enabling physically consistent predictions that adhere to conservation laws while efficiently handling heterogeneous interactions.

Abstract: Accurate, interpretable, and real-time modeling of multi-body dynamical systems is essential for predicting behaviors and inferring physical properties in natural and engineered environments. Traditional physics-based models face scalability challenges and are computationally demanding, while data-driven approaches like Graph Neural Networks (GNNs) often lack physical consistency, interpretability, and generalization. In this paper, we propose Dynami-CAL GraphNet, a Physics-Informed Graph Neural Network that integrates the learning capabilities of GNNs with physics-based inductive biases to address these limitations. Dynami-CAL GraphNet enforces pairwise conservation of linear and angular momentum for interacting nodes using edge-local reference frames that are equivariant to rotational symmetries, invariant to translations, and equivariant to node permutations. This design ensures physically consistent predictions of node dynamics while offering interpretable, edge-wise linear and angular impulses resulting from pairwise interactions. Evaluated on a 3D granular system with inelastic collisions, Dynami-CAL GraphNet demonstrates stable error accumulation over extended rollouts, effective extrapolations to unseen configurations, and robust handling of heterogeneous interactions and external forces. Dynami-CAL GraphNet offers significant advantages in fields requiring accurate, interpretable, and real-time modeling of complex multi-body dynamical systems, such as robotics, aerospace engineering, and materials science. By providing physically consistent and scalable predictions that adhere to fundamental conservation laws, it enables the inference of forces and moments while efficiently handling heterogeneous interactions and external forces.

Method: U-shaped encoder-decoder architecture integrated with Mamba-2 for linear complexity, selective information propagation for long-range dependencies, and self-supervised pre-training with combined L1 and spectral loss for temporal and spectral preservation.

Result: State-of-the-art performance in both short- and long-sequence modeling/classification, consistently outperforms existing models with strong generalization across subjects, tasks, and domains.

Conclusion: EEGM2 provides an efficient, scalable solution suitable for deployment on resource-constrained BCI devices, overcoming limitations of previous approaches.

Abstract: Electroencephalogram (EEG) signals generally exhibit low signal-to-noise ratio (SNR) and high inter-subject variability, making generalization across subjects and domains challenging. Recent advances in deep learning, particularly self-supervised learning with Transformer-based architectures, have shown promise in EEG representation learning. However, their quadratic computational complexity increases memory usage and slows inference, making them inefficient for modeling long-range dependencies. Moreover, most existing approaches emphasize either explicit window segmentation of the temporal signal or spectral-only input embedding while neglecting raw temporal dynamics. In this paper, we propose EEGM2, a self-supervised framework that overcomes these limitations. EEGM2 adopts a U-shaped encoder-decoder architecture integrated with Mamba-2 to achieve linear computational complexity, thereby reducing memory usage and improving inference speed. Meanwhile, the selective information propagation mechanism of Mamba-2 enables the model to effectively capture and preserve long-range dependencies in raw EEG signals, where traditional RNN or CNN architectures often struggle. Moreover, EEGM2 employs a self-supervised pre-training objective that reconstructs raw EEG using a combined L1 and spectral (Fourier-based) loss, enhancing generalization by jointly preserving temporal dynamics and spectral characteristics. Experimental results demonstrate that EEGM2 achieves state-of-the-art performance in both short- and long-sequence modeling and classification. Further evaluations show that EEGM2 consistently outperforms existing models, demonstrating strong generalization across subjects and tasks, as well as transferability across domains. Overall, EEGM2 offers an efficient and scalable solution suitable for deployment on resource-constrained brain-computer interface (BCI) devices.

Method: Decomposes wavelet filters into complementary local and global components: local aggregation uses efficient low-order polynomials, while long-range interactions are captured through flexible spectral-domain parameterization.

Result: LR-GWN achieves state-of-the-art performance among wavelet-based methods on long-range benchmarks while remaining competitive on short-range datasets.

Conclusion: The hybrid design successfully unifies short- and long-distance information flow within a principled wavelet framework, addressing the central challenge of modeling long-range interactions in graph machine learning.

Abstract: Modeling long-range interactions, the propagation of information across distant parts of a graph, is a central challenge in graph machine learning. Graph wavelets, inspired by multi-resolution signal processing, provide a principled way to capture both local and global structures. However, existing wavelet-based graph neural networks rely on finite-order polynomial approximations, which limit their receptive fields and hinder long-range propagation. We propose Long-Range Graph Wavelet Networks (LR-GWN), which decompose wavelet filters into complementary local and global components. Local aggregation is handled with efficient low-order polynomials, while long-range interactions are captured through a flexible spectral-domain parameterization. This hybrid design unifies short- and long-distance information flow within a principled wavelet framework. Experiments show that LR-GWN achieves state-of-the-art performance among wavelet-based methods on long-range benchmarks, while remaining competitive on short-range datasets.

Method: Framework extending manifold learning to general metric spaces, analyzing conditions for pointwise convergence of graph Laplacian.

Result: Identifies sufficient conditions under which alternative metrics satisfy convergence properties for graph Laplacian methods.

Conclusion: Provides theoretical foundation for using non-Euclidean metrics in manifold learning algorithms, expanding applicability to diverse data types.

Abstract: Laplacian-based methods are popular for the dimensionality reduction of data lying in $\mathbb{R}^N$. Several theoretical results for these algorithms depend on the fact that the Euclidean distance locally approximates the geodesic distance on the underlying submanifold which the data are assumed to lie on. However, for some applications, other metrics, such as the Wasserstein distance, may provide a more appropriate notion of distance than the Euclidean distance. We provide a framework that generalizes the problem of manifold learning to metric spaces and study when a metric satisfies sufficient conditions for the pointwise convergence of the graph Laplacian.

Method: FediLoRA introduces dimension-wise aggregation that reweights LoRA updates without information dilution, and a lightweight layer-wise model editing method that selectively incorporates global parameters to repair local components.

Result: Experimental results on three multimodal benchmark datasets show superior performance over competitive baselines in both global and personalized settings, particularly with modality incompleteness.

Conclusion: FediLoRA effectively handles heterogeneous LoRA ranks and missing modalities in federated multimodal fine-tuning, achieving improved client and global model performance.

Abstract: Foundation models have demonstrated remarkable performance across a wide range of tasks, yet their large parameter sizes pose challenges for practical deployment, especially in decentralized environments. Parameter-efficient fine-tuning (PEFT), such as Low-Rank Adaptation (LoRA), reduces local computing and memory overhead, making it attractive for federated learning. However, existing federated LoRA methods typically assume uniform rank configurations and unimodal inputs, overlooking two key real-world challenges: (1) heterogeneous client resources have different LoRA ranks, and (2) multimodal data settings with potentially missing modalities. In this work, we propose FediLoRA, a simple yet effective framework for federated multimodal fine-tuning under heterogeneous LoRA ranks and missing modalities. FediLoRA introduces a dimension-wise aggregation strategy that reweights LoRA updates without information dilution during aggregation. It also includes a lightweight layer-wise model editing method that selectively incorporates global parameters to repair local components which improves both client and global model performances. Experimental results on three multimodal benchmark datasets demonstrate that FediLoRA achieves superior performance over competitive baselines in both global and personalized settings, particularly in the presence of modality incompleteness.

Method: Uses pose autoencoder with articulator-based disentanglement (face, hands, body), non-autoregressive transformer decoder, and channel-aware regularization with KL divergence loss weighted by articulator importance.

Result: Achieves state-of-the-art results on PHOENIX14T and CSL-Daily datasets without relying on gloss supervision or pretrained models.

Conclusion: DARSLP provides an effective gloss-free approach for sign language production that learns structured representations and outperforms existing methods on benchmark datasets.

Abstract: In this work, we propose DARSLP, a simple gloss-free, transformer-based sign language production (SLP) framework that directly maps spoken-language text to sign pose sequences. We first train a pose autoencoder that encodes sign poses into a compact latent space using an articulator-based disentanglement strategy, where features corresponding to the face, right hand, left hand, and body are modeled separately to promote structured and interpretable representation learning. Next, a non-autoregressive transformer decoder is trained to predict these latent representations from word-level text embeddings of the input sentence. To guide this process, we apply channel-aware regularization by aligning predicted latent distributions with priors extracted from the ground-truth encodings using a KL divergence loss. The contribution of each channel to the loss is weighted according to its associated articulator region, enabling the model to account for the relative importance of different articulators during training. Our approach does not rely on gloss supervision or pretrained models, and achieves state-of-the-art results on the PHOENIX14T and CSL-Daily datasets.

Method: ActiveFlow uses three novel techniques: cross-layer active weights preloading (using current layer activations to predict subsequent layer weights), sparsity-aware self-distillation (aligning active weights with dense-model outputs), and active weight DRAM-flash swapping pipeline (orchestrating DRAM space allocation).

Result: ActiveFlow achieves the performance-cost Pareto frontier compared to existing efficiency optimization methods, enabling larger model deployment on memory-constrained devices.

Conclusion: The framework successfully addresses the memory limitation problem for LLM deployment on mobile devices through adaptive DRAM usage and active weight management techniques.

Abstract: Large language models (LLMs) are increasingly being deployed on mobile devices, but the limited DRAM capacity constrains the deployable model size. This paper introduces ActiveFlow, the first LLM inference framework that can achieve adaptive DRAM usage for modern LLMs (not ReLU-based), enabling the scaling up of deployable model sizes. The framework is based on the novel concept of active weight DRAM-flash swapping and incorporates three novel techniques: (1) Cross-layer active weights preloading. It uses the activations from the current layer to predict the active weights of several subsequent layers, enabling computation and data loading to overlap, as well as facilitating large I/O transfers. (2) Sparsity-aware self-distillation. It adjusts the active weights to align with the dense-model output distribution, compensating for approximations introduced by contextual sparsity. (3) Active weight DRAM-flash swapping pipeline. It orchestrates the DRAM space allocation among the hot weight cache, preloaded active weights, and computation-involved weights based on available memory. Results show ActiveFlow achieves the performance-cost Pareto frontier compared to existing efficiency optimization methods.

Method: Proposes Token Merge with Attention (ToMA) with three key contributions: 1) token merge as submodular optimization for diverse token selection, 2) merge/unmerge as GPU-friendly matrix operations, and 3) exploiting latent locality and sequential redundancy to minimize overhead.

Result: ToMA reduces SDXL/Flux generation latency by 24%/23% respectively with minimal quality degradation (DINO Δ < 0.07), outperforming prior methods.

Conclusion: This work bridges the gap between theoretical and practical efficiency for transformers in diffusion models by designing GPU-aligned token reduction methods.

Abstract: Diffusion models excel in high-fidelity image generation but face scalability limits due to transformers’ quadratic attention complexity. Plug-and-play token reduction methods like ToMeSD and ToFu reduce FLOPs by merging redundant tokens in generated images but rely on GPU-inefficient operations (e.g., sorting, scattered writes), introducing overheads that negate theoretical speedups when paired with optimized attention implementations (e.g., FlashAttention). To bridge this gap, we propose Token Merge with Attention (ToMA), an off-the-shelf method that redesigns token reduction for GPU-aligned efficiency, with three key contributions: 1) a reformulation of token merge as a submodular optimization problem to select diverse tokens; 2) merge/unmerge as an attention-like linear transformation via GPU-friendly matrix operations; and 3) exploiting latent locality and sequential redundancy (pattern reuse) to minimize overhead. ToMA reduces SDXL/Flux generation latency by 24%/23%, respectively (with DINO $\Delta < 0.07$), outperforming prior methods. This work bridges the gap between theoretical and practical efficiency for transformers in diffusion.

Method: Proposes ABG-NAS with three components: Comprehensive Architecture Search Space (CASS) for exploring propagation and transformation operations, Adaptive Genetic Optimization Strategy (AGOS) for balancing exploration/exploitation, and Bayesian-Guided Tuning Module (BGTM) for hyperparameter optimization.

Result: Empirical evaluations on benchmark datasets (Cora, PubMed, Citeseer, CoraFull) show ABG-NAS consistently outperforms both manually designed GNNs and state-of-the-art NAS methods.

Conclusion: ABG-NAS has the potential to advance graph representation learning by providing scalable and adaptive solutions for diverse graph structures.

Abstract: Effective and efficient graph representation learning is essential for enabling critical downstream tasks, such as node classification, link prediction, and subgraph search. However, existing graph neural network (GNN) architectures often struggle to adapt to diverse and complex graph structures, limiting their ability to produce structure-aware and task-discriminative representations. To address this challenge, we propose ABG-NAS, a novel framework for automated graph neural network architecture search tailored for efficient graph representation learning. ABG-NAS encompasses three key components: a Comprehensive Architecture Search Space (CASS), an Adaptive Genetic Optimization Strategy (AGOS), and a Bayesian-Guided Tuning Module (BGTM). CASS systematically explores diverse propagation (P) and transformation (T) operations, enabling the discovery of GNN architectures capable of capturing intricate graph characteristics. AGOS dynamically balances exploration and exploitation, ensuring search efficiency and preserving solution diversity. BGTM further optimizes hyperparameters periodically, enhancing the scalability and robustness of the resulting architectures. Empirical evaluations on benchmark datasets (Cora, PubMed, Citeseer, and CoraFull) demonstrate that ABG-NAS consistently outperforms both manually designed GNNs and state-of-the-art neural architecture search (NAS) methods. These results highlight the potential of ABG-NAS to advance graph representation learning by providing scalable and adaptive solutions for diverse graph structures. Our code is publicly available at https://github.com/sserranw/ABG-NAS.

Method: FragmentGPT combines: (1) chemically-aware, energy-based bond cleavage pre-training for fragment growing, linking, and merging capabilities; (2) Reward Ranked Alignment with Expert Exploration algorithm for diversity enhancement, data optimization, and supervised fine-tuning to align with multi-objective goals.

Result: The framework generates chemically valid, high-quality molecules tailored for drug discovery tasks, successfully connecting diverse molecular subunits while optimizing multiple pharmaceutical objectives and resolving structural redundancies through intelligent merging.

Conclusion: FragmentGPT provides a unified solution for controlled, goal-driven molecular assembly in FBDD, demonstrating effectiveness in real-world cancer datasets through experiments and ablation studies.

Abstract: Fragment-Based Drug Discovery (FBDD) is a popular approach in early drug development, but designing effective linkers to combine disconnected molecular fragments into chemically and pharmacologically viable candidates remains challenging. Further complexity arises when fragments contain structural redundancies, like duplicate rings, which cannot be addressed by simply adding or removing atoms or bonds. To address these challenges in a unified framework, we introduce FragmentGPT, which integrates two core components: (1) a novel chemically-aware, energy-based bond cleavage pre-training strategy that equips the GPT-based model with fragment growing, linking, and merging capabilities, and (2) a novel Reward Ranked Alignment with Expert Exploration (RAE) algorithm that combines expert imitation learning for diversity enhancement, data selection and augmentation for Pareto and composite score optimality, and Supervised Fine-Tuning (SFT) to align the learner policy with multi-objective goals. Conditioned on fragment pairs, FragmentGPT generates linkers that connect diverse molecular subunits while simultaneously optimizing for multiple pharmaceutical goals. It also learns to resolve structural redundancies-such as duplicated fragments-through intelligent merging, enabling the synthesis of optimized molecules. FragmentGPT facilitates controlled, goal-driven molecular assembly. Experiments and ablation studies on real-world cancer datasets demonstrate its ability to generate chemically valid, high-quality molecules tailored for downstream drug discovery tasks.

Method: A new empirical algorithm for connecting independently trained modes that generalizes to support modern architectures including MobileNet, ShuffleNet, EfficientNet, RegNet, DLA, and CCT.

Result: The method achieves broader applicability, more consistent connectivity paths across independently trained mode pairs, and supports connecting modes obtained with different training hyperparameters.

Conclusion: The proposed algorithm successfully extends mode connectivity to modern neural network architectures, overcoming limitations of previous methods.

Abstract: Empirical and theoretical studies have shown that continuous low-loss paths can be constructed between independently trained neural network models. This phenomenon, known as mode connectivity, refers to the existence of such paths between distinct modes-i.e., well-trained solutions in parameter space. However, existing empirical methods are primarily effective for older and relatively simple architectures such as basic CNNs, VGG, and ResNet, raising concerns about their applicability to modern and structurally diverse models. In this work, we propose a new empirical algorithm for connecting independently trained modes that generalizes beyond traditional architectures and supports a broader range of networks, including MobileNet, ShuffleNet, EfficientNet, RegNet, Deep Layer Aggregation (DLA), and Compact Convolutional Transformers (CCT). In addition to broader applicability, the proposed method yields more consistent connectivity paths across independently trained mode pairs and supports connecting modes obtained with different training hyperparameters.

Method: SPO replaces per-group baselines with a persistent KL-adaptive value tracker and normalizes advantages globally across batches, enabling group-free optimization with adaptive curriculum via prioritized sampling.

Result: SPO achieves +3.4pp average improvement on math benchmarks over GRPO, with substantial gains on challenging datasets (+7.3pp on BRUMO 25, +4.4pp on AIME 25, +3.3pp on HMMT 25) and consistent pass@k improvements.

Conclusion: SPO demonstrates that fundamental principles rather than architectural complexity drive progress in LLM reasoning, offering a more robust and efficient path for policy optimization.

Abstract: We revisit policy-gradient optimization for Large Language Models (LLMs) from a single-stream perspective. Prevailing group-based methods like GRPO reduce variance with on-the-fly baselines but suffer from critical flaws: frequent degenerate groups erase learning signals, and synchronization barriers hinder scalability. We introduce Single-stream Policy Optimization (SPO), which eliminates these issues by design. SPO replaces per-group baselines with a persistent, KL-adaptive value tracker and normalizes advantages globally across the batch, providing a stable, low-variance learning signal for every sample. Being group-free, SPO enables higher throughput and scales effectively in long-horizon or tool-integrated settings where generation times vary. Furthermore, the persistent value tracker naturally enables an adaptive curriculum via prioritized sampling. Experiments using Qwen3-8B show that SPO converges more smoothly and attains higher accuracy than GRPO, while eliminating computation wasted on degenerate groups. Ablation studies confirm that SPO’s gains stem from its principled approach to baseline estimation and advantage normalization, offering a more robust and efficient path for LLM reasoning. Across five hard math benchmarks with Qwen3 8B, SPO improves the average maj@32 by +3.4 percentage points (pp) over GRPO, driven by substantial absolute point gains on challenging datasets, including +7.3 pp on BRUMO 25, +4.4 pp on AIME 25, +3.3 pp on HMMT 25, and achieves consistent relative gain in pass@$k$ across the evaluated $k$ values. SPO’s success challenges the prevailing trend of adding incidental complexity to RL algorithms, highlighting a path where fundamental principles, not architectural workarounds, drive the next wave of progress in LLM reasoning.

Method: Introduces a dynamical low-rank training scheme enhanced with a spectral regularizer that controls the condition number of low-rank cores in each layer, making the method model- and data-agnostic with rank adaptivity.

Result: Achieves over 94% compression while recovering or improving adversarial accuracy relative to uncompressed baselines across various architectures, datasets, and adversarial attacks.

Conclusion: The proposed method successfully mitigates the sensitivity of compressed models to adversarial perturbations without sacrificing clean data accuracy, offering an efficient solution for robust model compression.

Abstract: Deployment of neural networks on resource-constrained devices demands models that are both compact and robust to adversarial inputs. However, compression and adversarial robustness often conflict. In this work, we introduce a dynamical low-rank training scheme enhanced with a novel spectral regularizer that controls the condition number of the low-rank core in each layer. This approach mitigates the sensitivity of compressed models to adversarial perturbations without sacrificing accuracy on clean data. The method is model- and data-agnostic, computationally efficient, and supports rank adaptivity to automatically compress the network at hand. Extensive experiments across standard architectures, datasets, and adversarial attacks show the regularized networks can achieve over 94% compression while recovering or improving adversarial accuracy relative to uncompressed baselines.

Method: Uses patch-based fusion approach and represents fusion solution space as a directed acyclic graph to efficiently find optimal fusion settings for CNNs, with implementation running on various microcontroller architectures.

Result: Achieves 50% less RAM usage compared to previous state-of-the-art methods (MCUNetV2 and StreamNet) while maintaining inference performance.

Conclusion: msf-CNN provides additional flexibility for system designers by identifying a wider set of fusion solutions for memory-efficient CNN deployment on MCUs.

Abstract: AI spans from large language models to tiny models running on microcontrollers (MCUs). Extremely memory-efficient model architectures are decisive to fit within an MCU’s tiny memory budget e.g., 128kB of RAM. However, inference latency must remain small to fit real-time constraints. An approach to tackle this is patch-based fusion, which aims to optimize data flows across neural network layers. In this paper, we introduce msf-CNN, a novel technique that efficiently finds optimal fusion settings for convolutional neural networks (CNNs) by walking through the fusion solution space represented as a directed acyclic graph. Compared to previous work on CNN fusion for MCUs, msf-CNN identifies a wider set of solutions. We published an implementation of msf-CNN running on various microcontrollers (ARM Cortex-M, RISC-V, ESP32). We show that msf-CNN can achieve inference using 50% less RAM compared to the prior art (MCUNetV2 and StreamNet). We thus demonstrate how msf-CNN offers additional flexibility for system designers.

Method: Systematic evaluation and benchmarking of methodologies using data available within first day of ICU admission, with focus on machine learning approaches, novel biomarker applications, and integration of diverse data types.

Result: The review aims to provide comprehensive assessment of these innovative approaches for mortality prediction.

Conclusion: Advanced methodologies show promise for improving predictive accuracy in ICU mortality prediction compared to traditional scoring systems.

Abstract: The intensive care unit (ICU) manages critically ill patients, many of whom face a high risk of mortality. Early and accurate prediction of in-hospital mortality within the first 24 hours of ICU admission is crucial for timely clinical interventions, resource optimization, and improved patient outcomes. Traditional scoring systems, while useful, often have limitations in predictive accuracy and adaptability. Objective: This review aims to systematically evaluate and benchmark innovative methodologies that leverage data available within the first day of ICU admission for predicting in-hospital mortality. We focus on advancements in machine learning, novel biomarker applications, and the integration of diverse data types.

Method: The authors analyze the relationship between attack effectiveness and node embedding variance in GNNs, then propose EC-LDA which compresses node embeddings to enhance label distribution inference.

Result: Extensive experiments on six graph datasets show EC-LDA outperforms state-of-the-art LDAs, achieving optimal performance on metrics like Cos-sim and JS-div in CoraFull and LastFM datasets.

Conclusion: EC-LDA significantly improves label distribution attack effectiveness in FGL and demonstrates robustness under differential privacy protection.

Abstract: Graph Neural Networks (GNNs) have been widely used for graph analysis. Federated Graph Learning (FGL) is an emerging learning framework to collaboratively train graph data from various clients. However, since clients are required to upload model parameters to the server in each round, this provides the server with an opportunity to infer each client’s data privacy. In this paper, we focus on label distribution attacks(LDAs) that aim to infer the label distributions of the clients’ local data. We take the first step to attack client’s label distributions in FGL. Firstly, we observe that the effectiveness of LDA is closely related to the variance of node embeddings in GNNs. Next, we analyze the relation between them and we propose a new attack named EC-LDA, which significantly improves the attack effectiveness by compressing node embeddings. Thirdly, extensive experiments on node classification and link prediction tasks across six widely used graph datasets show that EC-LDA outperforms the SOTA LDAs. For example, EC-LDA attains optimal values under both Cos-sim and JS-div evaluation metrics in the CoraFull and LastFM datasets. Finally, we explore the robustness of EC-LDA under differential privacy protection.

Method: Proposes CISIR framework that combines: 1) Correlation consideration between predictions and actual values, 2) Monotonically Decreasing Involution (MDI) importance function that’s not limited to convex functions, and 3) Stratified sampling to ensure rare instances are included in training batches.

Result: Experimental results on five datasets show CISIR achieves lower error and higher correlation than recent methods. Adding the correlation component to other methods improves their performance, and MDI importance outperforms other importance functions.

Conclusion: CISIR effectively addresses highly imbalanced regression problems by integrating correlation awareness, flexible importance functions, and strategic sampling, demonstrating superior performance in applications like forecasting rare Solar Energetic Particle events.

Abstract: We investigate imbalanced regression with tabular data that have an imbalance ratio larger than 1,000 (“highly imbalanced”). Accurately estimating the target values of rare instances is important in applications such as forecasting the intensity of rare harmful Solar Energetic Particle (SEP) events. For regression, the MSE loss does not consider the correlation between predicted and actual values. Typical inverse importance functions allow only convex functions. Uniform sampling might yield mini-batches that do not have rare instances. We propose CISIR that incorporates correlation, Monotonically Decreasing Involution (MDI) importance, and stratified sampling. Based on five datasets, our experimental results indicate that CISIR can achieve lower error and higher correlation than some recent methods. Also, adding our correlation component to other recent methods can improve their performance. Lastly, MDI importance can outperform other importance functions. Our code can be found in https://github.com/Machine-Earning/CISIR.

Method: Proposes an epsilon-net algorithm that selects a representative subset of actions from the large action space, leveraging Gaussian process modeling to capture payoff relationships between similar actions.

Result: Theoretical performance guarantees are provided, and empirical comparisons show the method’s effectiveness against Thompson Sampling and Upper Confidence Bound approaches.

Conclusion: The epsilon-net approach with Gaussian process modeling provides an efficient way to handle large action spaces in bandit problems while maintaining competitive performance compared to full-space methods.

Abstract: We study the problem of selecting a subset from a large action space shared by a family of bandits, with the goal of achieving performance nearly matching that of using the full action space. We assume that similar actions tend to have related payoffs, modeled by a Gaussian process. To exploit this structure, we propose a simple epsilon-net algorithm to select a representative subset. We provide theoretical guarantees for its performance and compare it empirically to Thompson Sampling and Upper Confidence Bound.

Method: Extends MVCL-DAF with two key modules: (1) Prototype-aware contrastive alignment that aligns instances to class-level prototypes for semantic consistency, and (2) Coarse-to-fine attention fusion that integrates global modality summaries with token-level features for hierarchical cross-modal interaction.

Result: Achieves new state-of-the-art results on MIntRec and MIntRec2.0 datasets, with significant improvements in rare-class recognition (+1.05% and +4.18% WF1 respectively).

Conclusion: The results demonstrate the effectiveness of prototype-guided learning and coarse-to-fine fusion for robust multimodal understanding, providing a solution to the challenges of semantic grounding and robustness in MMIR.

Abstract: Multimodal intent recognition (MMIR) suffers from weak semantic grounding and poor robustness under noisy or rare-class conditions. We propose MVCL-DAF++, which extends MVCL-DAF with two key modules: (1) Prototype-aware contrastive alignment, aligning instances to class-level prototypes to enhance semantic consistency; and (2) Coarse-to-fine attention fusion, integrating global modality summaries with token-level features for hierarchical cross-modal interaction. On MIntRec and MIntRec2.0, MVCL-DAF++ achieves new state-of-the-art results, improving rare-class recognition by +1.05% and +4.18% WF1, respectively. These results demonstrate the effectiveness of prototype-guided learning and coarse-to-fine fusion for robust multimodal understanding. The source code is available at https://github.com/chr1s623/MVCL-DAF-PlusPlus.

Method: Conducted Monte Carlo simulations with synthetic data using 2500 simulations per scenario across various BER values. Compared kNN, Generalized Henze-Penrose (GHP) divergence, and Kernel Density Estimation (KDE) techniques.

Result: kNN was overwhelmingly the most accurate non-parametric estimator. To achieve 95% confidence bounds within 5% range, minimum requirements are 1000 samples per class for basic cases, increasing to 2500 samples per class for 4 features.

Conclusion: kNN is the most practical BER estimator, but requires substantial sample sizes (1000-2500 per class) to achieve reliable accuracy, with sample needs increasing with feature dimensionality.

Abstract: The Bayes Error Rate (BER) is the fundamental limit on the achievable generalizable classification accuracy of any machine learning model due to inherent uncertainty within the data. BER estimators offer insight into the difficulty of any classification problem and set expectations for optimal classification performance. In order to be useful, the estimators must also be accurate with a limited number of samples on multivariate problems with unknown class distributions. To determine which estimators meet the minimum requirements for “usefulness”, an in-depth examination of their accuracy is conducted using Monte Carlo simulations with synthetic data in order to obtain their confidence bounds for binary classification. To examine the usability of the estimators for real-world applications, new non-linear multi-modal test scenarios are introduced. In each scenario, 2500 Monte Carlo simulations per scenario are run over a wide range of BER values. In a comparison of k-Nearest Neighbor (kNN), Generalized Henze-Penrose (GHP) divergence and Kernel Density Estimation (KDE) techniques, results show that kNN is overwhelmingly the more accurate non-parametric estimator. In order to reach the target of an under 5% range for the 95% confidence bounds, the minimum number of required samples per class is 1000. As more features are added, more samples are needed, so that 2500 samples per class are required at only 4 features. Other estimators do become more accurate than kNN as more features are added, but continuously fail to meet the target range.

Method: Combined Gaussian Process Latent Variable Model with diffeomorphism to create GPDSSM statistical shape model using volumetric CT scans and clinical landmarks. Used 192 CT scans (100 training, 92 testing).

Result: GPDSSM effectively distinguishes dysplastic samples from controls with 96.2% AUC vs 91.2% for angle-based methods, while highlighting dysplastic surface variations.

Conclusion: The GPDSSM pipeline improves classification accuracy and saves clinician time by eliminating manual angle measurements and 2D scan interpretation for dysplasia diagnosis.

Abstract: Dysplasia is a recognised risk factor for osteoarthritis (OA) of the hip, early diagnosis of dysplasia is important to provide opportunities for surgical interventions aimed at reducing the risk of hip OA. We have developed a pipeline for semi-automated classification of dysplasia using volumetric CT scans of patients’ hips and a minimal set of clinically annotated landmarks, combining the framework of the Gaussian Process Latent Variable Model with diffeomorphism to create a statistical shape model, which we termed the Gaussian Process Diffeomorphic Statistical Shape Model (GPDSSM). We used 192 CT scans, 100 for model training and 92 for testing. The GPDSSM effectively distinguishes dysplastic samples from controls while also highlighting regions of the underlying surface that show dysplastic variations. As well as improving classification accuracy compared to angle-based methods (AUC 96.2% vs 91.2%), the GPDSSM can save time for clinicians by removing the need to manually measure angles and interpreting 2D scans for possible markers of dysplasia.

Method: The authors use a model-based and data-driven approach to investigate the impact of joint memory and computing frequency scaling, combining fitting parameters from different DNN models and validating through simulations in both local and cooperative inference scenarios.

Result: Simulation results demonstrate that jointly scaling memory frequency and computing frequency effectively reduces energy consumption in DNN inference tasks across different deployment scenarios.

Conclusion: The research validates that simultaneous adjustment of both memory and computing frequencies is crucial for achieving efficient DNN inference, offering significant energy savings compared to traditional computing-only frequency scaling approaches.

Abstract: Deep neural networks (DNNs) have been widely applied in diverse applications, but the problems of high latency and energy overhead are inevitable on resource-constrained devices. To address this challenge, most researchers focus on the dynamic voltage and frequency scaling (DVFS) technique to balance the latency and energy consumption by changing the computing frequency of processors. However, the adjustment of memory frequency is usually ignored and not fully utilized to achieve efficient DNN inference, which also plays a significant role in the inference time and energy consumption. In this paper, we first investigate the impact of joint memory frequency and computing frequency scaling on the inference time and energy consumption with a model-based and data-driven method. Then by combining with the fitting parameters of different DNN models, we give a preliminary analysis for the proposed model to see the effects of adjusting memory frequency and computing frequency simultaneously. Finally, simulation results in local inference and cooperative inference cases further validate the effectiveness of jointly scaling the memory frequency and computing frequency to reduce the energy consumption of devices.

Method: DMPEL progressively builds a low-rank expert library and employs a lightweight router to dynamically combine experts into end-to-end policies. It uses expert coefficient replay to guide the router in retrieving frozen experts for previous tasks.

Result: Extensive experiments on LIBERO benchmark show DMPEL outperforms state-of-the-art lifelong learning methods in success rates during continual adaptation, while using minimal trainable parameters and storage.

Conclusion: DMPEL provides an effective solution for lifelong robot learning by enabling flexible knowledge sharing and efficient forward transfer while mitigating forgetting through modular parameter structure and expert replay.

Abstract: A generalist agent must continuously learn and adapt throughout its lifetime, achieving efficient forward transfer while minimizing catastrophic forgetting. Previous work within the dominant pretrain-then-finetune paradigm has explored parameter-efficient fine-tuning for single-task adaptation, effectively steering a frozen pretrained model with a small number of parameters. However, in the context of lifelong learning, these methods rely on the impractical assumption of a test-time task identifier and restrict knowledge sharing among isolated adapters. To address these limitations, we propose Dynamic Mixture of Progressive Parameter-Efficient Expert Library (DMPEL) for lifelong robot learning. DMPEL progressively builds a low-rank expert library and employs a lightweight router to dynamically combine experts into an end-to-end policy, enabling flexible and efficient lifelong forward transfer. Furthermore, by leveraging the modular structure of the fine-tuned parameters, we introduce expert coefficient replay, which guides the router to accurately retrieve frozen experts for previously encountered tasks. This technique mitigates forgetting while being significantly more storage- and computation-efficient than experience replay over the entire policy. Extensive experiments on the lifelong robot learning benchmark LIBERO demonstrate that our framework outperforms state-of-the-art lifelong learning methods in success rates during continual adaptation, while utilizing minimal trainable parameters and storage.

Method: Proposes Context-Aware Kernel Evolution (CAKE) using LLMs as crossover/mutation operators to generate/refine GP kernels, and BIC-Acquisition Kernel Ranking (BAKER) to select kernels based on BIC and expected improvement.

Result: Extensive experiments show CAKE-based BO consistently outperforms established baselines in hyperparameter optimization, controller tuning, and photonic chip design tasks.

Conclusion: The proposed CAKE framework effectively addresses kernel selection limitations in BO by leveraging LLMs for adaptive kernel evolution, demonstrating superior performance across diverse real-world applications.

Abstract: The efficiency of Bayesian optimization (BO) relies heavily on the choice of the Gaussian process (GP) kernel, which plays a central role in balancing exploration and exploitation under limited evaluation budgets. Traditional BO methods often rely on fixed or heuristic kernel selection strategies, which can result in slow convergence or suboptimal solutions when the chosen kernel is poorly suited to the underlying objective function. To address this limitation, we propose a freshly-baked Context-Aware Kernel Evolution (CAKE) to enhance BO with large language models (LLMs). Concretely, CAKE leverages LLMs as the crossover and mutation operators to adaptively generate and refine GP kernels based on the observed data throughout the optimization process. To maximize the power of CAKE, we further propose BIC-Acquisition Kernel Ranking (BAKER) to select the most effective kernel through balancing the model fit measured by the Bayesian information criterion (BIC) with the expected improvement at each iteration of BO. Extensive experiments demonstrate that our fresh CAKE-based BO method consistently outperforms established baselines across a range of real-world tasks, including hyperparameter optimization, controller tuning, and photonic chip design. Our code is publicly available at https://github.com/richardcsuwandi/cake.

Method: Developed a data-domain sampling regime to test 800 CNN models (16M trials) and 113 human participants (6,825 trials) on bar chart ratio estimation across three perspectives: sensitivity to training-test distribution discrepancies, stability to limited samples, and relative expertise to humans.

Result: CNNs can outperform humans in bar chart interpretation, and their biases depend simply on the training-test distance rather than complex factors.

Conclusion: CNNs exhibit simple, elegant behavior in visualization image interpretation, with performance directly tied to training-test distribution distance.

Abstract: We present a data-domain sampling regime for quantifying CNNs’ graphic perception behaviors. This regime lets us evaluate CNNs’ ratio estimation ability in bar charts from three perspectives: sensitivity to training-test distribution discrepancies, stability to limited samples, and relative expertise to human observers. After analyzing 16 million trials from 800 CNNs models and 6,825 trials from 113 human participants, we arrived at a simple and actionable conclusion: CNNs can outperform humans and their biases simply depend on the training-test distance. We show evidence of this simple, elegant behavior of the machines when they interpret visualization images. osf.io/gfqc3 provides registration, the code for our sampling regime, and experimental results.

Method: Two key components: 1) Global-perspective data replay module that reconstructs global representations for class-aware sampling, 2) Task-aware temperature scaling that adaptively adjusts logit temperatures at class and instance levels based on task dynamics.

Result: FedCBDR achieves balanced class-wise sampling under heterogeneous data distributions and improves generalization under task imbalance, yielding 2%-15% Top-1 accuracy improvement over six state-of-the-art methods.

Conclusion: The proposed method effectively addresses class imbalance issues in FCIL through coordinated global replay and adaptive temperature scaling, demonstrating significant performance improvements.

Abstract: Federated Class Incremental Learning (FCIL) aims to collaboratively process continuously increasing incoming tasks across multiple clients. Among various approaches, data replay has become a promising solution, which can alleviate forgetting by reintroducing representative samples from previous tasks. However, their performance is typically limited by class imbalance, both within the replay buffer due to limited global awareness and between replayed and newly arrived classes. To address this issue, we propose a class wise balancing data replay method for FCIL (FedCBDR), which employs a global coordination mechanism for class-level memory construction and reweights the learning objective to alleviate the aforementioned imbalances. Specifically, FedCBDR has two key components: 1) the global-perspective data replay module reconstructs global representations of prior task in a privacy-preserving manner, which then guides a class-aware and importance-sensitive sampling strategy to achieve balanced replay; 2) Subsequently, to handle class imbalance across tasks, the task aware temperature scaling module adaptively adjusts the temperature of logits at both class and instance levels based on task dynamics, which reduces the model’s overconfidence in majority classes while enhancing its sensitivity to minority classes. Experimental results verified that FedCBDR achieves balanced class-wise sampling under heterogeneous data distributions and improves generalization under task imbalance between earlier and recent tasks, yielding a 2%-15% Top-1 accuracy improvement over six state-of-the-art methods.

Method: Using AlphaEvolve (an LLM coding agent) to construct nearly extremal Ramanujan graphs and discover new gadget reductions, while also evolving faster verification procedures to overcome exponential-time verification costs.

Result: Improved near-optimal bounds for MAX-CUT and MAX-Independent Set on random graphs, and new NP-hardness inapproximability results for MAX-4-CUT (0.987) and MAX-3-CUT (0.9649), with verification speedups up to 10,000x.

Conclusion: AI can successfully assist in discovering proofs and constructions for combinatorial optimization problems, though establishing norms for assessing AI’s contribution to proof development remains an open question.

Abstract: We explore whether techniques from AI can help discover new combinatorial structures that improve on known limits on efficient algorithms. Specifically, we use AlphaEvolve (an LLM coding agent) to study two settings: a) Average-case hardness for MAX-CUT and MAX-Independent Set: We improve a recent result of Kunisky and Yu to obtain near-optimal upper and (conditional) lower bounds on certification algorithms for MAX-CUT and MAX-Independent Set on random 3- and 4-regular graphs. Our improved lower bounds are obtained by constructing nearly extremal Ramanujan graphs on as many as $163$ nodes, using AlphaEvolve. Additionally, via analytical arguments we strengthen the upper bounds to settle the computational hardness of these questions up to an error in the third decimal place. b) Worst-case Hardness of Approximation for MAX-k-CUT: We obtain new inapproximability results, proving that it is NP-hard to approximate MAX-4-CUT and MAX-3-CUT within factors of $0.987$ and $0.9649$ respectively, using AlphaEvolve to discover new gadget reductions. Our MAX-4-CUT result improves upon the SOTA of $0.9883$, and our MAX-3-CUT result improves on the current best gadget-based inapproximability result of $0.9853$, but falls short of improving the SOTA of $16/17$ that relies on a custom PCP, rather than a gadget reduction from “standard” H{\aa}stad-style PCPs. A key technical challenge we faced: verifying a candidate construction produced by AlphaEvolve is costly (often requiring exponential time). In both settings above, our results were enabled by using AlphaEvolve itself to evolve the verification procedure to be faster (sometimes by $10,000\times$). We conclude with a discussion of norms by which to assess the assistance from AI in developing proofs.

Method: HyperEvent uses relative structural encoding to capture patterns in event sequences, combined with a lightweight transformer classifier to reframe link prediction as event structure recognition.

Result: HyperEvent achieves competitive results across multiple benchmarks, often matching the performance of more complex models.

Conclusion: Effective dynamic graph modeling can be achieved through simple structural encoding, providing a clear baseline for evaluating future advancements in the field.

Abstract: Learning representations for continuous-time dynamic graphs is critical for dynamic link prediction. While recent methods have become increasingly complex, the field lacks a strong and informative baseline to reliably gauge progress. This paper proposes HyperEvent, a simple approach that captures relative structural patterns in event sequences through an intuitive encoding mechanism. As a straightforward baseline, HyperEvent leverages relative structural encoding to identify meaningful event sequences without complex parameterization. By combining these interpretable features with a lightweight transformer classifier, HyperEvent reframes link prediction as event structure recognition. Despite its simplicity, HyperEvent achieves competitive results across multiple benchmarks, often matching the performance of more complex models. This work demonstrates that effective modeling can be achieved through simple structural encoding, providing a clear reference point for evaluating future advancements.

Method: Analyzed frontier LLMs’ responses to harmful requests, tested output-based safety monitors, and developed linear probes on internal activations to detect strategic dishonesty.

Result: Strategic dishonesty emerges unpredictably in frontier LLMs, fools all tested output-based safety monitors, and can act as a honeypot against malicious users. Internal activation probes reliably detect this behavior.

Conclusion: Strategic dishonesty exemplifies the difficulty of controlling LLM alignment when helpfulness and harmlessness conflict, requiring new detection methods beyond output monitoring.

Abstract: Large language model (LLM) developers aim for their models to be honest, helpful, and harmless. However, when faced with malicious requests, models are trained to refuse, sacrificing helpfulness. We show that frontier LLMs can develop a preference for dishonesty as a new strategy, even when other options are available. Affected models respond to harmful requests with outputs that sound harmful but are crafted to be subtly incorrect or otherwise harmless in practice. This behavior emerges with hard-to-predict variations even within models from the same model family. We find no apparent cause for the propensity to deceive, but show that more capable models are better at executing this strategy. Strategic dishonesty already has a practical impact on safety evaluations, as we show that dishonest responses fool all output-based monitors used to detect jailbreaks that we test, rendering benchmark scores unreliable. Further, strategic dishonesty can act like a honeypot against malicious users, which noticeably obfuscates prior jailbreak attacks. While output monitors fail, we show that linear probes on internal activations can be used to reliably detect strategic dishonesty. We validate probes on datasets with verifiable outcomes and by using them as steering vectors. Overall, we consider strategic dishonesty as a concrete example of a broader concern that alignment of LLMs is hard to control, especially when helpfulness and harmlessness conflict.

Method: Introduces Asymmetric Constrained POMDPs (ACPOMDPs) as theoretical framework, then proposes PIGDreamer with privileged representation alignment and asymmetric actor-critic structure.

Result: PIGDreamer significantly outperforms existing Safe RL methods and shows enhanced performance, robustness, and efficiency compared to other privileged RL approaches.

Conclusion: The proposed framework provides theoretical grounding for privileged information use in Safe RL, with PIGDreamer demonstrating practical advantages in safety and performance.

Abstract: Partial observability presents a significant challenge for Safe Reinforcement Learning (Safe RL), as it impedes the identification of potential risks and rewards. Leveraging specific types of privileged information during training to mitigate the effects of partial observability has yielded notable empirical successes. In this paper, we propose Asymmetric Constrained Partially Observable Markov Decision Processes (ACPOMDPs) to theoretically examine the advantages of incorporating privileged information in Safe RL. Building upon ACPOMDPs, we propose the Privileged Information Guided Dreamer (PIGDreamer), a model-based RL approach that leverages privileged information to enhance the agent’s safety and performance through privileged representation alignment and an asymmetric actor-critic structure. Our empirical results demonstrate that PIGDreamer significantly outperforms existing Safe RL methods. Furthermore, compared to alternative privileged RL methods, our approach exhibits enhanced performance, robustness, and efficiency. Codes are available at: https://github.com/hggforget/PIGDreamer.

Method: Introduces a GNN architecture that combines compressed higher-order topological signals with standard molecular features to capture global geometric information while preserving computational tractability and human-interpretable structure.

Result: Achieves best performing results in both accuracy and robustness across almost all benchmarks, from small-molecule datasets to complex material datasets, using a parameter-efficient architecture.

Conclusion: The proposed approach demonstrates superior performance in molecular representation learning tasks while maintaining interpretability and computational efficiency.

Abstract: Recent advances in molecular representation learning have produced highly effective encodings of molecules for numerous cheminformatics and bioinformatics tasks. However, extracting general chemical insight while balancing predictive accuracy, interpretability, and computational efficiency remains a major challenge. In this work, we introduce a novel Graph Neural Network (GNN) architecture that combines compressed higher-order topological signals with standard molecular features. Our approach captures global geometric information while preserving computational tractability and human-interpretable structure. We evaluate our model across a range of benchmarks, from small-molecule datasets to complex material datasets, and demonstrate superior performance using a parameter-efficient architecture. We achieve the best performing results in both accuracy and robustness across almost all benchmarks. We open source all code \footnote{All code and results can be found on Github https://github.com/rahulkhorana/TFC-PACT-Net}.

Method: EvoCoT constrains exploration space by self-generating and verifying CoT trajectories, then gradually shortens CoT steps to expand space in a controlled way through two-stage chain-of-thought reasoning optimization.

Result: Applied to multiple LLM families (Qwen, DeepSeek, Llama), EvoCoT enables LLMs to solve previously unsolved problems and improves reasoning capability without external CoT supervision.

Conclusion: EvoCoT provides an effective framework for stable learning under sparse rewards, is compatible with various RL fine-tuning methods, and supports reasoning improvement through controlled exploration.

Abstract: Reinforcement learning with verifiable reward (RLVR) has become a promising paradigm for post-training large language models (LLMs) to improve their reasoning capability. However, when the rollout accuracy is low on hard problems, the reward becomes sparse, limiting learning efficiency and causing exploration bottlenecks. Existing approaches either rely on teacher models for distillation or filter out difficult problems, which limits scalability or restricts reasoning improvement through exploration. We propose EvoCoT, a self-evolving curriculum learning framework based on two-stage chain-of-thought (CoT) reasoning optimization. EvoCoT constrains the exploration space by self-generating and verifying CoT trajectories, then gradually shortens CoT steps to expand the space in a controlled way. The framework enables LLMs to stably learn from initially unsolved hard problems under sparse rewards. We apply EvoCoT to multiple LLM families, including Qwen, DeepSeek, and Llama. Experiments show that EvoCoT enables LLMs to solve previously unsolved problems, improves reasoning capability without external CoT supervision, and is compatible with various RL fine-tuning methods. We release the source code to support future research.

Method: Intelligent Selection of Participants (ISP) - an adaptive mechanism that dynamically determines the optimal number of clients per round. The method was validated across diverse setups including vision transformers, real-world ECG classification, and training with gradient compression.

Result: The results show consistent communication savings of up to 30% without losing final model quality. Applying ISP to different real-world ECG classification setups highlighted client number selection as a separate important task in federated learning.

Conclusion: ISP effectively addresses the communication bottleneck in Federated Learning by intelligently selecting the optimal number of participants per round, achieving significant efficiency gains while maintaining model performance across various applications.

Abstract: Rapid scaling of deep learning models has enabled performance gains across domains, yet it introduced several challenges. Federated Learning (FL) has emerged as a promising framework to address these concerns by enabling decentralized training. Nevertheless, communication efficiency remains a key bottleneck in FL, particularly under heterogeneous and dynamic client participation. Existing methods, such as FedAvg and FedProx, or other approaches, including client selection strategies, attempt to mitigate communication costs. However, the problem of choosing the number of clients in a training round remains extremely underexplored. We introduce Intelligent Selection of Participants (ISP), an adaptive mechanism that dynamically determines the optimal number of clients per round to enhance communication efficiency without compromising model accuracy. We validate the effectiveness of ISP across diverse setups, including vision transformers, real-world ECG classification, and training with gradient compression. Our results show consistent communication savings of up to 30% without losing the final quality. Applying ISP to different real-world ECG classification setups highlighted the selection of the number of clients as a separate task of federated learning.

Method: MetaFed integrates multi-agent reinforcement learning for dynamic client selection, privacy-preserving FL using homomorphic encryption, and carbon-aware scheduling aligned with renewable energy availability.

Result: Evaluations on MNIST and CIFAR-10 show MetaFed achieves up to 25% reduction in carbon emissions compared to conventional approaches while maintaining high accuracy and minimal communication overhead.

Conclusion: MetaFed presents a scalable solution for building environmentally responsible and privacy-compliant Metaverse infrastructures.

Abstract: The rapid expansion of immersive Metaverse applications introduces complex challenges at the intersection of performance, privacy, and environmental sustainability. Centralized architectures fall short in addressing these demands, often resulting in elevated energy consumption, latency, and privacy concerns. This paper proposes MetaFed, a decentralized federated learning (FL) framework that enables sustainable and intelligent resource orchestration for Metaverse environments. MetaFed integrates (i) multi-agent reinforcement learning for dynamic client selection, (ii) privacy-preserving FL using homomorphic encryption, and (iii) carbon-aware scheduling aligned with renewable energy availability. Evaluations on MNIST and CIFAR-10 using lightweight ResNet architectures demonstrate that MetaFed achieves up to 25% reduction in carbon emissions compared to conventional approaches, while maintaining high accuracy and minimal communication overhead. These results highlight MetaFed as a scalable solution for building environmentally responsible and privacy-compliant Metaverse infrastructures.

Method: The primer outlines foundations of graph design, key prediction tasks, and demonstrates how graph neural networks can operate on chemical graphs, with representative examples across chemical sciences.

Result: The primer prepares readers with the necessary concepts to apply graph-based methods and machine learning to chemical discovery problems.

Conclusion: Graph data modeling and learning algorithms, particularly graph neural networks, are powerful tools for advancing the next generation of chemical discovery across various domains including materials, biology, and medicine.

Abstract: Graphs are central to the chemical sciences, providing a natural language to describe molecules, proteins, reactions, and industrial processes. They capture interactions and structures that underpin materials, biology, and medicine. This primer, Graph Data Modeling: Molecules, Proteins, & Chemical Processes, introduces graphs as mathematical objects in chemistry and shows how learning algorithms (particularly graph neural networks) can operate on them. We outline the foundations of graph design, key prediction tasks, representative examples across chemical sciences, and the role of machine learning in graph-based modeling. Together, these concepts prepare readers to apply graph methods to the next generation of chemical discovery.

Method: G2T-FM augments original node features with neighborhood feature aggregation, adds structural embeddings, and then applies a tabular foundation model (TFMs like TabPFNv2 or LimiX) to the constructed node representations.

Result: In fully in-context regime, G2T-FM significantly outperforms publicly available GFMs and performs competitively with/better than well-tuned GNNs. After finetuning, it surpasses well-tuned GNN baselines, especially when combined with LimiX.

Conclusion: The paper reveals the potential of utilizing tabular foundation models for graph machine learning tasks, demonstrating that TFMs can be effectively adapted to handle graph-structured data through proper feature engineering.

Abstract: While foundation models have revolutionized such fields as natural language processing and computer vision, their potential in graph machine learning remains largely unexplored. One of the key challenges in designing graph foundation models (GFMs) is handling diverse node features that can vary across different graph datasets. While many works on GFMs have focused exclusively on text-attributed graphs, the problem of handling arbitrary features of other types in GFMs has not been fully addressed. However, this problem is not unique to the graph domain, as it also arises in the field of machine learning for tabular data. In this work, motivated by the recent success of tabular foundation models (TFMs) like TabPFNv2 or LimiX, we propose G2T-FM, a simple framework for turning tabular foundation models into graph foundation models. Specifically, G2T-FM augments the original node features with neighborhood feature aggregation, adds structural embeddings, and then applies a TFM to the constructed node representations. Even in a fully in-context regime, our model achieves strong results, significantly outperforming publicly available GFMs and performing competitively with, and often better than, well-tuned GNNs trained from scratch. Moreover, after finetuning, G2T-FM surpasses well-tuned GNN baselines. In particular, when combined with LimiX, G2T-FM often outperforms the best GNN by a significant margin. In summary, our paper reveals the potential of a previously overlooked direction of utilizing tabular foundation models for graph machine learning tasks.

Method: CRISP systematically transforms OMOP CDM data through: (1) transparent data quality management with audit trails, (2) cross-vocabulary mapping to SNOMED-CT standards with deduplication, (3) modular architecture with parallel optimization for <1 day processing, and (4) comprehensive baseline model benchmarks.

Result: CRISP enables complete dataset processing in under 1 day on standard hardware, provides reproducible performance standards, and saves researchers months of preprocessing effort.

Conclusion: CRISP democratizes access to large-scale multi-institutional critical care data by providing a comprehensive preprocessing pipeline, allowing researchers to focus on advancing clinical AI rather than data preparation.

Abstract: While existing critical care EHR datasets such as MIMIC and eICU have enabled significant advances in clinical AI research, the CRITICAL dataset opens new frontiers by providing extensive scale and diversity – containing 1.95 billion records from 371,365 patients across four geographically diverse CTSA institutions. CRITICAL’s unique strength lies in capturing full-spectrum patient journeys, including pre-ICU, ICU, and post-ICU encounters across both inpatient and outpatient settings. This multi-institutional, longitudinal perspective creates transformative opportunities for developing generalizable predictive models and advancing health equity research. However, the richness of this multi-site resource introduces substantial complexity in data harmonization, with heterogeneous collection practices and diverse vocabulary usage patterns requiring sophisticated preprocessing approaches. We present CRISP to unlock the full potential of this valuable resource. CRISP systematically transforms raw Observational Medical Outcomes Partnership Common Data Model data into ML-ready datasets through: (1) transparent data quality management with comprehensive audit trails, (2) cross-vocabulary mapping of heterogeneous medical terminologies to unified SNOMED-CT standards, with deduplication and unit standardization, (3) modular architecture with parallel optimization enabling complete dataset processing in $<$1 day even on standard computing hardware, and (4) comprehensive baseline model benchmarks spanning multiple clinical prediction tasks to establish reproducible performance standards. By providing processing pipeline, baseline implementations, and detailed transformation documentation, CRISP saves researchers months of preprocessing effort and democratizes access to large-scale multi-institutional critical care data, enabling them to focus on advancing clinical AI.

Method: Using symbolic feedforward neural networks that represent state distributions as vectors and transitions as stochastic matrices, enabling probabilistic state propagation via matrix-vector products. The approach includes probabilistic subset construction, ε-closure, and exact simulation via layered symbolic computation.

Result: The neural networks can exactly simulate PFAs and are learnable through standard gradient descent optimization on labeled sequence data, recovering the exact behavior of ground-truth PFAs.

Conclusion: The work successfully unifies probabilistic automata theory with neural architectures, demonstrating that symbolic neural simulators are both expressive and learnable, bridging symbolic computation and deep learning.

Abstract: We present a formal and constructive theory showing that probabilistic finite automata (PFAs) can be exactly simulated using symbolic feedforward neural networks. Our architecture represents state distributions as vectors and transitions as stochastic matrices, enabling probabilistic state propagation via matrix-vector products. This yields a parallel, interpretable, and differentiable simulation of PFA dynamics using soft updates-without recurrence. We formally characterize probabilistic subset construction, $\varepsilon$-closure, and exact simulation via layered symbolic computation, and prove equivalence between PFAs and specific classes of neural networks. We further show that these symbolic simulators are not only expressive but learnable: trained with standard gradient descent-based optimization on labeled sequence data, they recover the exact behavior of ground-truth PFAs. This learnability, formalized in Proposition 5.1, is the crux of this work. Our results unify probabilistic automata theory with neural architectures under a rigorous algebraic framework, bridging the gap between symbolic computation and deep learning.

Method: The USPIL framework integrates physics-informed neural networks (PINNs) with conservation laws, using automatic differentiation for physics constraints and adaptive loss weighting to balance data fidelity with physical consistency. It provides a unified solution for both ODE and PDE systems.

Result: Achieved 98.9% correlation for 1D temporal dynamics (loss: 0.0219, MAE: 0.0184) and captured complex spiral waves in 2D systems (loss: 4.7656, pattern correlation: 0.94). Validation showed conservation law adherence within 0.5% and 10-50x computational speedup for inference.

Conclusion: USPIL establishes physics-informed deep learning as a powerful paradigm for ecological modeling, enabling mechanistic understanding, parameter discovery, and multi-scale analysis with applications in ecological forecasting and conservation planning.

Abstract: Ecological systems exhibit complex multi-scale dynamics that challenge traditional modeling. New methods must capture temporal oscillations and emergent spatiotemporal patterns while adhering to conservation principles. We present the Unified Spatiotemporal Physics-Informed Learning (USPIL) framework, a deep learning architecture integrating physics-informed neural networks (PINNs) and conservation laws to model predator-prey dynamics across dimensional scales. The framework provides a unified solution for both ordinary (ODE) and partial (PDE) differential equation systems, describing temporal cycles and reaction-diffusion patterns within a single neural network architecture. Our methodology uses automatic differentiation to enforce physics constraints and adaptive loss weighting to balance data fidelity with physical consistency. Applied to the Lotka-Volterra system, USPIL achieves 98.9% correlation for 1D temporal dynamics (loss: 0.0219, MAE: 0.0184) and captures complex spiral waves in 2D systems (loss: 4.7656, pattern correlation: 0.94). Validation confirms conservation law adherence within 0.5% and shows a 10-50x computational speedup for inference compared to numerical solvers. USPIL also enables mechanistic understanding through interpretable physics constraints, facilitating parameter discovery and sensitivity analysis not possible with purely data-driven methods. Its ability to transition between dimensional formulations opens new avenues for multi-scale ecological modeling. These capabilities make USPIL a transformative tool for ecological forecasting, conservation planning, and understanding ecosystem resilience, establishing physics-informed deep learning as a powerful and scientifically rigorous paradigm.

Method: APFEx combines three innovations: (1) adaptive multi-objective optimizer with dynamic switching strategies, (2) differentiable intersectional fairness metrics for gradient-based optimization, and (3) theoretical convergence guarantees to Pareto-optimal solutions.

Result: Experiments on four real-world datasets show APFEx reduces fairness violations while maintaining competitive accuracy, demonstrating superiority over existing methods.

Conclusion: APFEx bridges a critical gap in fair ML by providing a scalable, model-agnostic solution for intersectional fairness with proven effectiveness and theoretical guarantees.

Abstract: Ensuring fairness in machine learning models is critical, especially when biases compound across intersecting protected attributes like race, gender, and age. While existing methods address fairness for single attributes, they fail to capture the nuanced, multiplicative biases faced by intersectional subgroups. We introduce Adaptive Pareto Front Explorer (APFEx), the first framework to explicitly model intersectional fairness as a joint optimization problem over the Cartesian product of sensitive attributes. APFEx combines three key innovations- (1) an adaptive multi-objective optimizer that dynamically switches between Pareto cone projection, gradient weighting, and exploration strategies to navigate fairness-accuracy trade-offs, (2) differentiable intersectional fairness metrics enabling gradient-based optimization of non-smooth subgroup disparities, and (3) theoretical guarantees of convergence to Pareto-optimal solutions. Experiments on four real-world datasets demonstrate APFEx’s superiority, reducing fairness violations while maintaining competitive accuracy. Our work bridges a critical gap in fair ML, providing a scalable, model-agnostic solution for intersectional fairness.

Method: Developed HFL-SNM framework with concepts of Effective Data Coverage Rate and Redundant Data Coverage Rate. Formulated joint optimization problem for resource allocation and client scheduling, then decoupled it into sub-problems solved by DO-SNM algorithm.

Result: Experimental results show superior model performance with significantly reduced energy consumption compared to traditional baseline algorithms.

Conclusion: The proposed HFL-SNM framework effectively addresses mobility and social network considerations in FL, achieving better performance while optimizing energy efficiency.

Abstract: Federated Learning (FL) offers a decentralized solution that allows collaborative local model training and global aggregation, thereby protecting data privacy. In conventional FL frameworks, data privacy is typically preserved under the assumption that local data remains absolutely private, whereas the mobility of clients is frequently neglected in explicit modeling. In this paper, we propose a hierarchical federated learning framework based on the social network with mobility namely HFL-SNM that considers both data sharing among clients and their mobility patterns. Under the constraints of limited resources, we formulate a joint optimization problem of resource allocation and client scheduling, which objective is to minimize the energy consumption of clients during the FL process. In social network, we introduce the concepts of Effective Data Coverage Rate and Redundant Data Coverage Rate. We analyze the impact of effective data and redundant data on the model performance through preliminary experiments. We decouple the optimization problem into multiple sub-problems, analyze them based on preliminary experimental results, and propose Dynamic Optimization in Social Network with Mobility (DO-SNM) algorithm. Experimental results demonstrate that our algorithm achieves superior model performance while significantly reducing energy consumption, compared to traditional baseline algorithms.

Method: Co-training two encoder networks that: 1) forward samples and record confidence in LLM labels, 2) derive dynamic importance weights based on belief in LLM label quality, 3) exchange weights and update parameters using peer network’s weights.

Result: Significantly outperforms conventional SSL methods, achieves state-of-the-art performance on 4 out of 5 benchmark datasets, and ranks first among 14 compared methods according to Friedman test.

Conclusion: Demonstrates a new direction in semi-supervised learning where LLMs serve as knowledge amplifiers, enabling efficient state-of-the-art performance through co-training models.

Abstract: In this paper, we introduce a novel weighted co-training approach that is guided by Large Language Models (LLMs). Namely, in our co-training approach, we use LLM labels on unlabeled data as target labels and co-train two encoder-only based networks that train each other over multiple iterations: first, all samples are forwarded through each network and historical estimates of each network’s confidence in the LLM label are recorded; second, a dynamic importance weight is derived for each sample according to each network’s belief in the quality of the LLM label for that sample; finally, the two networks exchange importance weights with each other – each network back-propagates all samples weighted with the importance weights coming from its peer network and updates its own parameters. By strategically utilizing LLM-generated guidance, our approach significantly outperforms conventional SSL methods, particularly in settings with abundant unlabeled data. Empirical results show that it achieves state-of-the-art performance on 4 out of 5 benchmark datasets and ranks first among 14 compared methods according to the Friedman test. Our results highlight a new direction in semi-supervised learning – where LLMs serve as knowledge amplifiers, enabling backbone co-training models to achieve state-of-the-art performance efficiently.

Method: The authors unroll optimization over a unified objective to derive CBSA, which contracts tokens into representative low-dimensional structures and then broadcasts them back. This creates an inherently interpretable and efficient linear-complexity attention mechanism.

Result: CBSA achieves linear scaling while generalizing existing attention mechanisms as special cases. Experiments show comparable performance and even superior advantages on several visual tasks compared to standard attention.

Conclusion: The proposed CBSA mechanism successfully addresses both interpretability and efficiency challenges in Transformers, providing a unified framework that maintains performance while reducing computational complexity from quadratic to linear.

Abstract: Attention mechanisms in Transformers have gained significant empirical success. Nonetheless, the optimization objectives underlying their forward pass are still unclear. Additionally, the quadratic complexity of self-attention is increasingly prohibitive. Unlike the prior work on addressing the interpretability or efficiency issue separately, we propose a unified optimization objective to alleviate both issues simultaneously. By unrolling the optimization over the objective, we derive an inherently interpretable and efficient attention mechanism, which compresses all tokens into low-dimensional structures by contracting a few representative tokens and then broadcasting the contractions back. This Contract-and-Broadcast Self-Attention (CBSA) mechanism can not only scale linearly but also generalize existing attention mechanisms as its special cases. Experiments further demonstrate comparable performance and even superior advantages of CBSA on several visual tasks. Code is available at this https URL.

Method: The authors propose SPRINT (Stochastic Performative Prediction with Variance Reduction), which incorporates variance reduction techniques to improve convergence in performative prediction under non-convex losses.

Result: SPRINT achieves O(1/T) convergence to stationary performative stable solutions, with error neighborhoods independent of stochastic gradient variance. Experiments on real datasets show superior performance over SGD-GD in both convergence rate and stability.

Conclusion: SPRINT provides a significant improvement over existing methods for performative prediction, offering faster convergence and more stable performance without variance-dependent error bounds, making it particularly effective for non-convex optimization problems.

Abstract: Performative prediction (PP) is an algorithmic framework for optimizing machine learning (ML) models where the model’s deployment affects the distribution of the data it is trained on. Compared to traditional ML with fixed data, designing algorithms in PP converging to a stable point – known as a stationary performative stable (SPS) solution – is more challenging than the counterpart in conventional ML tasks due to the model-induced distribution shifts. While considerable efforts have been made to find SPS solutions using methods such as repeated gradient descent (RGD) and greedy stochastic gradient descent (SGD-GD), most prior studies assumed a strongly convex loss until a recent work established $O(1/\sqrt{T})$ convergence of SGD-GD to SPS solutions under smooth, non-convex losses. However, this latest progress is still based on the restricted bounded variance assumption in stochastic gradient estimates and yields convergence bounds with a non-vanishing error neighborhood that scales with the variance. This limitation motivates us to improve convergence rates and reduce error in stochastic optimization for PP, particularly in non-convex settings. Thus, we propose a new algorithm called stochastic performative prediction with variance reduction (SPRINT) and establish its convergence to an SPS solution at a rate of $O(1/T)$. Notably, the resulting error neighborhood is independent of the variance of the stochastic gradients. Experiments on multiple real datasets with non-convex models demonstrate that SPRINT outperforms SGD-GD in both convergence rate and stability.

Method: Proposes four BO-based direction selectors: BOSW (one-shot BO on unit sphere), RBOSW (periodic-refresh variant), ABOSW (adaptive hybrid seeding from QSW sets), and ARBOSW (restarted hybrid that periodically relearns directions). These are drop-in replacements requiring no changes to downstream losses.

Result: Numerical experiments show state-of-the-art performance. ABOSW and ARBOSW achieve convergence comparable to best QSW variants with modest runtime overhead on the original QSW paper’s experimental suite.

Conclusion: Bayesian optimization provides an effective alternative to quasi-Monte Carlo for learning projection directions in sliced Wasserstein distance computation, with hybrid approaches (ABOSW/ARBOSW) offering the best balance of performance and efficiency.

Abstract: The sliced Wasserstein distance (SW) reduces optimal transport on $\mathbb{R}^d$ to a sum of one-dimensional projections, and thanks to this efficiency, it is widely used in geometry, generative modeling, and registration tasks. Recent work shows that quasi-Monte Carlo constructions for computing SW (QSW) yield direction sets with excellent approximation error. This paper presents an alternate, novel approach: learning directions with Bayesian optimization (BO), particularly in settings where SW appears inside an optimization loop (e.g., gradient flows). We introduce a family of drop-in selectors for projection directions: BOSW, a one-shot BO scheme on the unit sphere; RBOSW, a periodic-refresh variant; ABOSW, an adaptive hybrid that seeds from competitive QSW sets and performs a few lightweight BO refinements; and ARBOSW, a restarted hybrid that periodically relearns directions during optimization. Our BO approaches can be composed with QSW and its variants (demonstrated by ABOSW/ARBOSW) and require no changes to downstream losses or gradients. We provide numerical experiments where our methods achieve state-of-the-art performance, and on the experimental suite of the original QSW paper, we find that ABOSW and ARBOSW can achieve convergence comparable to the best QSW variants with modest runtime overhead.

Method: The study investigates multiple optimization techniques including pruning, quantization, knowledge distillation, simplified attention mechanisms, and Mixture of Experts (MoE) approach to reduce computational overhead.

Result: The experiments provide insights into how these techniques can be effectively applied to optimize inference for state-of-the-art diffusion models like fast-DiT.

Conclusion: Various optimization strategies can significantly improve inference efficiency in complex diffusion models without compromising performance, with Mixture of Experts showing particular promise for further enhancement.

Abstract: Efficient inference is a critical challenge in deep generative modeling, particularly as diffusion models grow in capacity and complexity. While increased complexity often improves accuracy, it raises compute costs, latency, and memory requirements. This work investigates techniques such as pruning, quantization, knowledge distillation, and simplified attention to reduce computational overhead without impacting performance. The study also explores the Mixture of Experts (MoE) approach to further enhance efficiency. These experiments provide insights into optimizing inference for the state-of-the-art Fast Diffusion Transformer (fast-DiT) model.

Method: The paper introduces the basic framework of reproducing kernel Hilbert spaces for scalar-valued learning, then extends this to operator learning, and finally formulates learning dynamical systems as an operator learning problem using Koopman operator theory.

Result: The manuscript serves as supporting material for a course on machine learning, presenting a structured approach from fundamental concepts to advanced applications in dynamical systems.

Conclusion: Reproducing kernel Hilbert spaces provide a powerful mathematical framework that can be systematically extended from traditional scalar-valued learning to operator learning, enabling effective formulation and solution of dynamical systems learning problems through Koopman operator theory.

Abstract: This expository article presents the approach to statistical machine learning based on reproducing kernel Hilbert spaces. The basic framework is introduced for scalar-valued learning and then extended to operator learning. Finally, learning dynamical systems is formulated as a suitable operator learning problem, leveraging Koopman operator theory. The manuscript collects the supporting material for the corresponding course taught at the CIME school “Machine Learning: From Data to Mathematical Understanding” in Cetraro.

Method: Formulated as a stochastic concurrent n-player game, developed verification properties for model correctness and reachability, constructed strategy synthesis graphs for optimal trajectories, and analyzed parameter correlations with rewards.

Result: The framework enables identification of optimal game trajectories and analysis of how parameters affect both local and global rewards over finite time horizons.

Conclusion: The proposed model provides a comprehensive approach to studying the Volunteer’s Dilemma with tools for verification, strategy optimization, and reward analysis in multi-agent systems.

Abstract: Game theory provides a framework for studying communication dynamics and emergent phenomena arising from rational agent interactions. We present a model framework for the Volunteer’s Dilemma with four key contributions: (1) formulating it as a stochastic concurrent nn n-player game, (2) developing properties to verify model correctness and reachability, (3) constructing strategy synthesis graphs to identify optimal game trajectories, and (4) analyzing parameter correlations with expected local and global rewards over finite time horizons.

Method: The research reconstructs a new dataset PCLMMPLUS with 103k comment entries and proposes CPCLDetector model featuring alignment selection and knowledge-enhanced comment content modules to improve CPCL detection.

Result: Extensive experiments show CPCLDetector outperforms state-of-the-art methods on PCLMM dataset and achieves higher performance on the new PCLMMPLUS dataset, enabling more accurate detection of CPCL videos.

Conclusion: The proposed approach supports better content governance and protection of vulnerable groups by more accurately detecting CPCL videos. Code and dataset are publicly available.

Abstract: Chinese Patronizing and Condescending Language (CPCL) is an implicitly discriminatory toxic speech targeting vulnerable groups on Chinese video platforms. The existing dataset lacks user comments, which are a direct reflection of video content. This undermines the model’s understanding of video content and results in the failure to detect some CPLC videos. To make up for this loss, this research reconstructs a new dataset PCLMMPLUS that includes 103k comment entries and expands the dataset size. We also propose the CPCLDetector model with alignment selection and knowledge-enhanced comment content modules. Extensive experiments show the proposed CPCLDetector outperforms the SOTA on PCLMM and achieves higher performance on PCLMMPLUS . CPLC videos are detected more accurately, supporting content governance and protecting vulnerable groups. Code and dataset are available at https://github.com/jiaxunyang256/PCLD.

Method: Proposes HMPPS framework with two query-aware refinement modules: 1) perspective-guided summarization for refined product descriptions, and 2) two-stage training paradigm for user history filtering using multimodal representations.

Result: Extensive experiments on four public datasets demonstrate HMPPS effectiveness. Deployment on an online search system with billion-level daily active users shows evident gain in A/B testing.

Conclusion: HMPPS successfully harnesses MLLM for personalized product search by effectively handling multimodal contents and addressing input redundancy/noise challenges, achieving improved search performance.

Abstract: Personalized product search (PPS) aims to retrieve products relevant to the given query considering user preferences within their purchase histories. Since large language models (LLM) exhibit impressive potential in content understanding and reasoning, current methods explore to leverage LLM to comprehend the complicated relationships among user, query and product to improve the search performance of PPS. Despite the progress, LLM-based PPS solutions merely take textual contents into consideration, neglecting multimodal contents which play a critical role for product search. Motivated by this, we propose a novel framework, HMPPS, for \textbf{H}arnessing \textbf{M}ultimodal large language models (MLLM) to deal with \textbf{P}ersonalized \textbf{P}roduct \textbf{S}earch based on multimodal contents. Nevertheless, the redundancy and noise in PPS input stand for a great challenge to apply MLLM for PPS, which not only misleads MLLM to generate inaccurate search results but also increases the computation expense of MLLM. To deal with this problem, we additionally design two query-aware refinement modules for HMPPS: 1) a perspective-guided summarization module that generates refined product descriptions around core perspectives relevant to search query, reducing noise and redundancy within textual contents; and 2) a two-stage training paradigm that introduces search query for user history filtering based on multimodal representations, capturing precise user preferences and decreasing the inference cost. Extensive experiments are conducted on four public datasets to demonstrate the effectiveness of HMPPS. Furthermore, HMPPS is deployed on an online search system with billion-level daily active users and achieves an evident gain in A/B testing.

Method: Drawing inspiration from conventional codecs by implementing more advanced interpolation filters, block-based motion information, and finite motion accuracy in learned codecs.

Result: Achieved 10% rate decrease and reduced motion-related decoding complexity from 391 MAC/pixel to 214 MAC/pixel in the Cool-chic video codec.

Conclusion: Incorporating conventional codec motion compensation techniques into learned codecs significantly improves compression performance and reduces decoding complexity.

Abstract: Motion compensation is a key component of video codecs. Conventional codecs (HEVC and VVC) have carefully refined this coding step, with an important focus on sub-pixel motion compensation. On the other hand, learned codecs achieve sub-pixel motion compensation through simple bilinear filtering. This paper offers to improve learned codec motion compensation by drawing inspiration from conventional codecs. It is shown that the usage of more advanced interpolation filters, block-based motion information and finite motion accuracy lead to better compression performance and lower decoding complexity. Experimental results are provided on the Cool-chic video codec, where we demonstrate a rate decrease of more than 10% and a lowering of motion-related decoding complexity from 391 MAC per pixel to 214 MAC per pixel. All contributions are made open-source at https://github.com/Orange-OpenSource/Cool-Chic

Method: The paper discusses and analyzes various speech technologies including speaker diarization, vocalization classification, word count estimation, speaker verification, and language diarization, evaluating their current applications and limitations for child-specific analysis.

Result: Current speech technologies offer valuable opportunities for analyzing children’s naturalistic recordings despite imperfect accuracy, but there’s a significant need for child-specific adaptations and improvements to address unique developmental characteristics.

Conclusion: There is a critical need for interdisciplinary collaboration to advance child-specific speech technologies for analyzing naturalistic recordings, which can provide unprecedented insights into early childhood cognitive and social development.

Abstract: Naturalistic recordings capture audio in real-world environments where participants behave naturally without interference from researchers or experimental protocols. Naturalistic long-form recordings extend this concept by capturing spontaneous and continuous interactions over extended periods, often spanning hours or even days, in participants’ daily lives. Naturalistic recordings have been extensively used to study children’s behaviors, including how they interact with others in their environment, in the fields of psychology, education, cognitive science, and clinical research. These recordings provide an unobtrusive way to observe children in real-world settings beyond controlled and constrained experimental environments. Advancements in speech technology and machine learning have provided an initial step for researchers to automatically and systematically analyze large-scale naturalistic recordings of children. Despite the imperfect accuracy of machine learning models, these tools still offer valuable opportunities to uncover important insights into children’s cognitive and social development. Several critical speech technologies involved include speaker diarization, vocalization classification, word count estimate from adults, speaker verification, and language diarization for code-switching. Most of these technologies have been primarily developed for adults, and speech technologies applied to children specifically are still vastly under-explored. To fill this gap, we discuss current progress, challenges, and opportunities in advancing these technologies to analyze naturalistic recordings of children during early development (<3 years of age). We strive to inspire the signal processing community and foster interdisciplinary collaborations to further develop this emerging technology and address its unique challenges and opportunities.

Method: An iterative DPO scheme that uses only a few hundred human-labeled preference pairs per round to directly optimize prosodic naturalness while regularizing to the current model. Evaluated on KoCC-TTS, a curated dataset of authentic Korean call center interactions.

Result: The method attains the highest human preference (ELO) with competitive CER, outperforming GRPO and strong commercial baselines on the KoCC-TTS dataset.

Conclusion: When prosody cannot be rewarded automatically, human preference optimization offers a practical and data-efficient path to natural and robust TTS.

Abstract: Recent work reports gains in neural text-to-speech (TTS) with Group Relative Policy Optimization (GRPO). However, in the absence of a verifiable reward for \textit{prosody}, GRPO trained on transcription-oriented signals (CER/NLL) lowers error rates yet collapses prosody into monotone, unnatural speech; adding speaker-similarity further destabilizes training and degrades CER. We address this with an \textit{iterative Direct Preference Optimization (DPO)} scheme that uses only a few hundred human-labeled preference pairs per round to directly optimize prosodic naturalness while regularizing to the current model. On \textbf{KoCC-TTS}, a curated dataset of authentic Korean call center interactions capturing task-oriented dialogues, our method attains the highest human preference (ELO) with competitive CER, outperforming GRPO and strong commercial baselines. These results suggest that when prosody cannot be rewarded automatically, \textit{human preference optimization} offers a practical and data-efficient path to natural and robust TTS. The demo page is available at \href{https://tts.ch.dev}

Method: Proposes Spectral Pairwise INteraction (SPIN) module to capture cross-channel spatial correlations in complex spectrogram domain, fuses with spherical harmonics DoA encoding across frequency subbands, and incorporates iterative refinement via chain-of-inference.

Result: SoundCompass robustly extracts target sources across diverse signal classes and spatial configurations.

Conclusion: The combination of SPIN, spherical harmonics embedding, and chain-of-inference provides an effective directional clue integration framework that preserves full spatial information.

Abstract: Recent advances in target sound extraction (TSE) utilize directional clues derived from direction of arrival (DoA), which represent an inherent spatial property of sound available in any acoustic scene. However, previous DoA-based methods rely on hand-crafted features or discrete encodings, which lose fine-grained spatial information and limit adaptability. We propose SoundCompass, an effective directional clue integration framework centered on a Spectral Pairwise INteraction (SPIN) module that captures cross-channel spatial correlations in the complex spectrogram domain to preserve full spatial information in multichannel signals. The input feature expressed in terms of spatial correlations is fused with a DoA clue represented as spherical harmonics (SH) encoding. The fusion is carried out across overlapping frequency subbands, inheriting the benefits reported in the previous band-split architectures. We also incorporate the iterative refinement strategy, chain-of-inference (CoI), in the TSE framework, which recursively fuses DoA with sound event activation estimated from the previous inference stage. Experiments demonstrate that SoundCompass, combining SPIN, SH embedding, and CoI, robustly extracts target sources across diverse signal classes and spatial configurations.

Method: Proposes HarmoniFuse with: 1) gated speech encoder for task-specific acoustic features, 2) prompt-adaptive dynamic fusion module for layer aggregation, and 3) batch-interleaved training to leverage separate datasets without joint annotation.

Result: Experimental results show HarmoniFuse improves both ASR and SER performance compared to existing approaches.

Conclusion: HarmoniFuse provides a scalable and robust solution for multitask speech understanding under realistic data constraints by harmonizing heterogeneous task demands.

Abstract: Recent advances in large language models have facilitated the development of unified speech language models (SLMs) capable of supporting multiple speech tasks within a shared architecture. However, tasks such as automatic speech recognition (ASR) and speech emotion recognition (SER) rely on distinct types of information: ASR primarily depends on linguistic content, whereas SER requires the integration of both linguistic and paralinguistic cues. Existing multitask SLMs typically adopt naive parameter sharing or prompt-based conditioning without explicitly modeling the differences in information composition required by each task. Such designs risk task interference and performance degradation, especially under limited data conditions. To address these limitations, we propose HarmoniFuse, a component-selective and prompt-adaptive framework for multi-task speech language modeling. HarmoniFuse is designed to harmonize heterogeneous task demands by selecting and fusing task-relevant components of speech representations. Specifically, it integrates a gated speech encoder to extract task-specific acoustic features and a prompt-adaptive dynamic fusion module to aggregate transformer layers based on task characteristics. In addition, a batch-interleaved training strategy enables leveraging separate ASR and SER datasets without requiring joint annotation. Experimental results demonstrate that HarmoniFuse improves both ASR and SER performance, offering a scalable and robust solution for multitask speech understanding under realistic data constraints.

Method: Dual-dimensional distillation: source-wise (textual + acoustic teachers) and layer-wise (aligning teacher signals with appropriate student layers) for fine-grained control.

Result: Significant improvements in audio reasoning performance, effectively bridging symbolic reasoning and speech representations.

Conclusion: The framework provides an effective reasoning transfer solution for audio modeling, overcoming modality limitations.

Abstract: While large audio language models excel at tasks like ASR and emotion recognition, they still struggle with complex reasoning due to the modality gap between audio and text as well as the lack of structured intermediate supervision. To address this, we propose a unified knowledge distillation framework to transfer reasoning capabilities from a high-capacity textual teacher model to a student audio models while preserving its acoustic competence. Our method introduces two key dimensions: source-wise distillation, which leverages both textual and acoustic teachers to provide complementary modality-specific supervision; and layer-wise distillation, which aligns teacher signals with appropriate student layers to improve transfer efficiency. This dual-dimensional strategy enables fine-grained control over the distillation process, effectively bridging the gap between symbolic reasoning and speech representations. Experimental results show significant improvements in audio reasoning performance, demonstrating the effectiveness of our framework as a reasoning transfer solution for audio modeling.

Method: Uses text-to-audio diffusion models guided by energy-envelope ControlNet for temporal coherence, with joint score filtering using dual classifiers to ensure sample quality and practical training pipeline integration.

Result: Experimental results show improvements in Polyphonic Sound Detection Scores (PSDS1 and PSDS2), enhancing both temporal localization and sound class discrimination.

Conclusion: SynSonic effectively enables generative-based augmentation for SED, demonstrating that carefully designed generative methods can overcome challenges in temporal annotation and noise introduction.

Abstract: Data synthesis and augmentation are essential for Sound Event Detection (SED) due to the scarcity of temporally labeled data. While augmentation methods like SpecAugment and Mix-up can enhance model performance, they remain constrained by the diversity of existing samples. Recent generative models offer new opportunities, yet their direct application to SED is challenging due to the lack of precise temporal annotations and the risk of introducing noise through unreliable filtering. To address these challenges and enable generative-based augmentation for SED, we propose SynSonic, a data augmentation method tailored for this task. SynSonic leverages text-to-audio diffusion models guided by an energy-envelope ControlNet to generate temporally coherent sound events. A joint score filtering strategy with dual classifiers ensures sample quality, and we explore its practical integration into training pipelines. Experimental results show that SynSonic improves Polyphonic Sound Detection Scores (PSDS1 and PSDS2), enhancing both temporal localization and sound class discrimination.

Method: FlexSED builds on pretrained audio SSL model and CLAP text encoder, using encoder-decoder composition with adaptive fusion strategy for continuous training. It employs LLMs for event query selection during training to address missing label challenges.

Result: FlexSED achieves superior performance compared to vanilla SED models on AudioSet-Strong, while demonstrating strong zero-shot and few-shot capabilities.

Conclusion: The proposed FlexSED system enables more flexible and user-friendly interaction through free-text queries and provides robust zero-shot/few-shot performance, with code and pretrained models released for future research.

Abstract: Despite recent progress in large-scale sound event detection (SED) systems capable of handling hundreds of sound classes, existing multi-class classification frameworks remain fundamentally limited. They cannot process free-text sound queries, which enable more flexible and user-friendly interaction, and they lack zero-shot capabilities and offer poor few-shot adaptability. Although text-query-based separation methods have been explored, they primarily focus on source separation and are ill-suited for SED tasks that require precise temporal localization and efficient detection across large and diverse sound vocabularies. In this paper, we propose FlexSED, an open-vocabulary sound event detection system. FlexSED builds on a pretrained audio SSL model and the CLAP text encoder, introducing an encoder-decoder composition and an adaptive fusion strategy to enable effective continuous training from pretrained weights. To ensure robust supervision, it also employs large language models (LLMs) to assist in event query selection during training, addressing challenges related to missing labels. As a result, FlexSED achieves superior performance compared to vanilla SED models on AudioSet-Strong, while demonstrating strong zero-shot and few-shot capabilities. We release the code and pretrained models to support future research and applications based on FlexSED.

Method: Uses GRPO fine-tuning with a composite reward function combining character error rate (CER) and negative log-likelihood (NLL) from an ASR model, applied to pre-trained LLM-based TTS models.

Result: Experimental results show substantial improvements in both intelligibility and naturalness of synthesized speech in zero-shot TTS scenarios.

Conclusion: The proposed method effectively enhances TTS performance, with ablation studies confirming the value of integrating both CER and NLL reward components.

Abstract: This paper proposes a GRPO-based approach to enhance the performance of large language model (LLM)-based text-to-speech (TTS) models by deriving rewards from an off-the-shelf automatic speech recognition (ASR) model. Compared to previous reinforcement learning methods for LLM-based TTS, our method requires no dedicated model for reward computation or training. Moreover, we design a composite reward function that combines character error rate (CER) with negative log-likelihood (NLL) obtained from the ASR model, providing more informative and accurate reward signals. We apply GRPO fine-tuning to pre-trained LLM-based TTS models and evaluate their zero-shot TTS performance. Experimental results show that the proposed method substantially improves both the intelligibility and naturalness of synthesized speech. Ablation studies and further analyses confirm the effectiveness of integrating the two reward components.

Method: Three strategies targeting different spaces: 1) Modify architectural topology for better information exchange (topological space), 2) Introduce prior knowledge to facilitate generation (source space), 3) Optimize backpropagation with improved output format (output space).

Result: Experimental results demonstrate that the proposed method effectively facilitates joint estimation of magnitude and phase in APNet2, stabilizing its performance.

Conclusion: The proposed strategies successfully bridge the performance disparities between single-stream and dual-stream vocoders, enabling stable joint magnitude and phase estimation in time-frequency domain neural vocoders.

Abstract: Time-frequency (T-F) domain-based neural vocoders have shown promising results in synthesizing high-fidelity audio. Nevertheless, it remains unclear on the mechanism of effectively predicting magnitude and phase targets jointly. In this paper, we start from two representative T-F domain vocoders, namely Vocos and APNet2, which belong to the single-stream and dual-stream modes for magnitude and phase estimation, respectively. When evaluating their performance on a large-scale dataset, we accidentally observe severe performance collapse of APNet2. To stabilize its performance, in this paper, we introduce three simple yet effective strategies, each targeting the topological space, the source space, and the output space, respectively. Specifically, we modify the architectural topology for better information exchange in the topological space, introduce prior knowledge to facilitate the generation process in the source space, and optimize the backpropagation process for parameter updates with an improved output format in the output space. Experimental results demonstrate that our proposed method effectively facilitates the joint estimation of magnitude and phase in APNet2, thus bridging the performance disparities between the single-stream and dual-stream vocoders.

Method: Training DACe on diverse real and synthetic tonal data with balanced sampling, then systematically comparing Fréchet Audio Distance (FAD) and Maximum Mean Discrepancy (MMD) on MUSHRA tests across speech, music, and mixed content.

Result: FAD consistently outperforms MMD, and embeddings from higher-fidelity NACs like DACe show stronger correlations with human judgments. NAC embeddings provide a practical zero-shot approach to audio quality assessment.

Conclusion: Neural audio codecs have dual utility for both compression and perceptually informed audio evaluation, with higher-fidelity codecs offering better correlation with human perceptual judgments.

Abstract: Neural audio codecs (NACs) achieve low-bitrate compression by learning compact audio representations, which can also serve as features for perceptual quality evaluation. We introduce DACe, an enhanced, higher-fidelity version of the Descript Audio Codec (DAC), trained on diverse real and synthetic tonal data with balanced sampling. We systematically compare Fr'echet Audio Distance (FAD) and Maximum Mean Discrepancy (MMD) on MUSHRA tests across speech, music, and mixed content. FAD consistently outperforms MMD, and embeddings from higher-fidelity NACs (such as DACe) show stronger correlations with human judgments. While CLAP LAION Music (CLAP-M) and OpenL3 Mel128 (OpenL3-128M) embeddings achieve higher correlations, NAC embeddings provide a practical zero-shot approach to audio quality assessment, requiring only unencoded audio for training. These results demonstrate the dual utility of NACs for compression and perceptually informed audio evaluation.

Method: Extracted pitch, formant, loudness, and spectral flux features from clean speech and computed correlations with objective SE metrics (frequency weighted segmental SNR and PESQ) across multiple state-of-the-art SE models, languages, and noise conditions.

Result: Formant amplitudes are consistently predictive of SE performance, with higher and more stable formants leading to larger enhancement gains. Performance varies substantially even within a single speaker’s utterances, highlighting intraspeaker acoustic variability.

Conclusion: Intrinsic speech characteristics should be considered when designing datasets, evaluation protocols, and enhancement models, as they provide new insights into SE challenges.

Abstract: Speech enhancement (SE) performance is known to depend on noise characteristics and signal to noise ratio (SNR), yet intrinsic properties of the clean speech signal itself remain an underexplored factor. In this work, we systematically analyze how clean speech characteristics influence enhancement difficulty across multiple state of the art SE models, languages, and noise conditions. We extract a set of pitch, formant, loudness, and spectral flux features from clean speech and compute correlations with objective SE metrics, including frequency weighted segmental SNR and PESQ. Our results show that formant amplitudes are consistently predictive of SE performance, with higher and more stable formants leading to larger enhancement gains. We further demonstrate that performance varies substantially even within a single speaker’s utterances, highlighting the importance of intraspeaker acoustic variability. These findings provide new insights into SE challenges, suggesting that intrinsic speech characteristics should be considered when designing datasets, evaluation protocols, and enhancement models.

Method: Three strategies were investigated: 1) training models from scratch using pathological data, 2) finetuning models pretrained on neurotypical speech with additional pathological data, and 3) speaker-specific personalization using only data from individual pathological test speakers.

Result: SE models can be successfully trained or finetuned on pathological speech data despite limited dataset sizes. Finetuning with data from several pathological speakers yielded the largest performance improvements, while speaker-specific personalization was less effective due to insufficient data per speaker.

Conclusion: The study highlights both challenges and potential strategies for improving speech enhancement performance for pathological speakers, with multi-speaker finetuning showing the most promise.

Abstract: State of the art speech enhancement (SE) models achieve strong performance on neurotypical speech, but their effectiveness is substantially reduced for pathological speech. In this paper, we investigate strategies to address this gap for both predictive and generative SE models, including i) training models from scratch using pathological data, ii) finetuning models pretrained on neurotypical speech with additional data from pathological speakers, and iii) speaker specific personalization using only data from the individual pathological test speaker. Our results show that, despite the limited size of pathological speech datasets, SE models can be successfully trained or finetuned on such data. Finetuning models with data from several pathological speakers yields the largest performance improvements, while speaker specific personalization is less effective, likely due to the small amount of data available per speaker. These findings highlight the challenges and potential strategies for improving SE performance for pathological speakers.

Method: The authors fine-tune the DiTAR zero-shot text-to-speech model using Direct Preference Optimization (DPO) to create ARDM-DPO, which enhances autoregressive diffusion models for speech generation.

Result: ARDM-DPO achieves significant improvements in speech expressiveness and robustness for long texts compared to the baseline DiTAR model.

Conclusion: The proposed ARDM-DPO method successfully advances RL-based fine-tuning of speech ARDMs, demonstrating the effectiveness of DPO for improving speech generation quality in autoregressive diffusion models.

Abstract: Autoregressive diffusion models (ARDMs) have recently been applied to speech generation, achieving state-of-the-art (SOTA) performance in zero-shot text-to-speech. By autoregressively generating continuous speech tokens with next-token diffusion, these models offer a promising alternative to next-token prediction, avoiding the technical complexities associated with discrete speech tokenization. As a relatively new paradigm, research on reinforcement learning (RL)-based fine-tuning of speech ARDMs remains limited. In this paper, we propose Autoregressive Diffusion-Direct Preference Optimization (ARDM-DPO) to advance this research. By fine-tuning the recently proposed zero-shot text-to-speech model DiTAR with DPO, we achieve significant improvements in terms of speech expressiveness and robustness for long texts.

Method: Proposes HD-PPT framework with: 1) novel speech codec to extract prompt-preference and content-preference tokens supervised by ASR and CLAP objectives, 2) hierarchical decoding strategy where LLM generates tokens in structured order (semantic → fine-grained style → complete acoustic representation).

Result: Extensive experiments show the hierarchical paradigm significantly improves instruction adherence and achieves state-of-the-art naturalness in speech synthesis.

Conclusion: HD-PPT validates a structured, hierarchical approach for precise and controllable speech synthesis, effectively bridging the modality gap in TTS systems.

Abstract: Large Language Model (LLM)-based Text-to-Speech (TTS) models have already reached a high degree of naturalness. However, the precision control of TTS inference is still challenging. Although instruction-based Text-to-Speech (Instruct-TTS) models are proposed, these models still lack fine-grained control due to the modality gap between single-level text instructions and multilevel speech tokens. To address this limitation, we propose HD-PPT, a framework that transforms speech synthesis into a structured, hierarchical task. To enable fine-grained control, we introduce a novel speech codec to extract distinct prompt-preference and content-preference tokens from the complex speech tokens, supervised by automatic speech recognition (ASR) and cross-lingual audio-text pre-training (CLAP) objectives. To bridge the modality gap of these tokens, we propose a hierarchical decoding strategy, where the LLM generates tokens in a structured order: first semantic, then fine-grained style, and finally complete acoustic representation. Extensive experiments demonstrate that this hierarchical paradigm significantly improves instruction adherence and achieves state-of-the-art naturalness, validating our approach for precise and controllable speech synthesis. Audio samples are available at https://xxh333.github.io/.

Method: Two core mechanisms: (1) distance-weighted probabilistic top-K sampling replacing deterministic nearest-neighbor selection in RVQ, and (2) progressive training scheme introducing perturbations from last to first quantizer. Trained exclusively on clean speech.

Result: Substantially improves robustness under noisy conditions (e.g., UTMOS from 3.475 to 3.586 at 15 dB SNR on Encodec) while also enhancing clean speech coding quality.

Conclusion: The proposed resource-efficient training strategy effectively enhances noise robustness without requiring paired noisy-clean data, making it practical for real-world deployment.

Abstract: Noise robustness remains a critical challenge for deploying neural speech codecs in real-world acoustic scenarios where background noise is often inevitable. A key observation we make is that even slight input noise perturbations can cause unintended shifts in quantized codewords, thereby degrading the quality of reconstructed speech. Motivated by this finding, we propose a novel and resource-efficient training strategy to enhance the noise robustness of speech codecs by simulating such perturbations directly at the quantization level. Our approach introduces two core mechanisms: (1) a distance-weighted probabilistic top-K sampling strategy that replaces the conventional deterministic nearest-neighbor selection in residual vector quantization (RVQ); and (2) a progressive training scheme that introduces perturbations from the last to the first quantizer in a controlled manner. Crucially, our method is trained exclusively on clean speech, eliminating the need for any paired noisy-clean data. Experiments on two advanced neural speech codecs, Encodec and WavTokenizer, demonstrate that the proposed strategy substantially improves robustness under noisy conditions-for example, boosting UTMOS from 3.475 to 3.586 at 15 dB SNR on Encodec-while also enhancing coding quality for clean speech.

Method: 1) Based on audio language models, WavReward incorporates deep reasoning and nonlinear reward mechanisms via reinforcement learning with multi-sample feedback. 2) Uses ChatReward-30K preference dataset covering comprehension, generation, text-based chats, nine acoustic attributes, and implicit chats.

Result: WavReward outperforms previous state-of-the-art models, improving objective accuracy from 53.4% to 91.5% over Qwen2.5-Omni, and leads by 83% in subjective A/B testing. Ablation studies confirm each component’s necessity.

Conclusion: WavReward effectively addresses the gap in spoken dialogue model evaluation by leveraging audio language models and specialized training data, demonstrating superior performance across multiple conversational scenarios.

Abstract: End-to-end spoken dialogue models such as GPT-4o-audio have recently garnered significant attention in the speech domain. However, the evaluation of spoken dialogue models’ conversational performance has largely been overlooked. This is primarily due to the intelligent chatbots convey a wealth of non-textual information which cannot be easily measured using text-based language models like ChatGPT. To address this gap, we propose WavReward, a reward feedback model based on audio language models that can evaluate both the IQ and EQ of spoken dialogue systems with speech input. Specifically, 1) based on audio language models, WavReward incorporates the deep reasoning process and the nonlinear reward mechanism for post-training. By utilizing multi-sample feedback via the reinforcement learning algorithm, we construct a specialized evaluator tailored to spoken dialogue models. 2) We introduce ChatReward-30K, a preference dataset used to train WavReward. ChatReward-30K includes both comprehension and generation aspects of spoken dialogue models. These scenarios span various tasks, such as text-based chats, nine acoustic attributes of instruction chats, and implicit chats. WavReward outperforms previous state-of-the-art evaluation models across multiple spoken dialogue scenarios, achieving a substantial improvement about Qwen2.5-Omni in objective accuracy from 53.4$%$ to 91.5$%$. In subjective A/B testing, WavReward also leads by a margin of 83$%$. Comprehensive ablation studies confirm the necessity of each component of WavReward. All data and code will be publicly at https://github.com/jishengpeng/WavReward after the paper is accepted.

Method: Self-Purifying Flow Matching (SPFM) identifies suspicious data using the model itself during training within the flow-matching framework, without requiring pretrained models or additional modules.

Result: Models trained with SPFM generate samples that accurately follow specified conditioning even with noisy labels. On the TITW in-the-wild speech dataset, SPFM outperforms existing baselines.

Conclusion: SPFM provides a robust approach to handle label noise in training datasets, demonstrating effectiveness in real-world scenarios like speech data processing.

Abstract: Training datasets are inherently imperfect, often containing mislabeled samples due to human annotation errors, limitations of tagging models, and other sources of noise. Such label contamination can significantly degrade the performance of a trained model. In this work, we introduce Self-Purifying Flow Matching (SPFM), a principled approach to filtering unreliable data within the flow-matching framework. SPFM identifies suspicious data using the model itself during the training process, bypassing the need for pretrained models or additional modules. Our experiments demonstrate that models trained with SPFM generate samples that accurately adhere to the specified conditioning, even when trained on noisy labels. Furthermore, we validate the robustness of SPFM on the TITW dataset, which consists of in-the-wild speech data, achieving performance that surpasses existing baselines.

Method: The authors formalize an on-device wavetable sonification approach where sensor data is mapped to acoustic parameters using wavetable synthesis techniques, and present a prototype implementation.

Result: A working prototype of device-level sonification for soil moisture monitoring is developed and explored, demonstrating the feasibility of this approach.

Conclusion: The paper successfully demonstrates that device-level sonification using wavetable synthesis is a viable strategy for IoS networks, particularly for environmental monitoring applications like soil moisture tracking in water-scarce regions.

Abstract: Sonification, the mapping of data to sound to communicate information about the original data source, is becoming a viable strategy for the sonic representation and communication of information derived from the complex flows of data exchanged across Internet of Sounds (IoS) networks. This paper presents an IoS sonification implementation for monitoring soil moisture levels within the broader context of the globally increasing water scarcity. While previous work has focused on sonifications operating on the applications and services level of the IoS network infrastructure, this paper explores device-level sonification using wavetable synthesis techniques to map sensor data to acoustic parameters. An approach to on-device wavetable sonification is formalized, and a prototype implementation is presented and explored before the approach is contextualised with regard to the soil moisture monitoring tasks.

Method: Developed an encoding algorithm that identifies discrete codes achieving lower quantization error compared to conventional methods, applied to pre-trained RVQ-based neural codec models.

Result: Experimental validation shows the method reduces quantization errors and improves synthesis quality across diverse metrics when applied to pre-trained models.

Conclusion: The proposed encoding algorithm successfully enhances RVQ-based neural audio codecs by optimizing code selection, leading to better quantization performance and improved audio synthesis quality.

Abstract: The residual vector quantization (RVQ) technique plays a central role in recent advances in neural audio codecs. These models effectively synthesize high-fidelity audio from a limited number of codes due to the hierarchical structure among quantization levels. In this paper, we propose an encoding algorithm to further enhance the synthesis quality of RVQ-based neural codecs at test-time. Firstly, we point out the suboptimal nature of quantized vectors generated by conventional methods. We demonstrate that quantization error can be mitigated by selecting a different set of codes. Subsequently, we present our encoding algorithm, designed to identify a set of discrete codes that achieve a lower quantization error. We then apply the proposed method to pre-trained models and evaluate its efficacy using diverse metrics. Our experimental findings validate that our method not only reduces quantization errors, but also improves synthesis quality.

Method: MUSHRA 1S is a single-stimulus variant where each system is rated individually against a fixed anchor and reference, rather than comparing multiple systems simultaneously.

Result: MUSHRA 1S matches standard MUSHRA’s sensitivity more closely than ACR, especially in high-quality regimes where ACR saturates, and effectively identifies specific deviations while reducing range-equalizing biases.

Conclusion: MUSHRA 1S provides a robust and scalable solution for benchmarking top-tier speech processing systems by combining MUSHRA-level sensitivity with ACR-like scalability.

Abstract: Evaluating state-of-the-art speech systems necessitates scalable and sensitive evaluation methods to detect subtle but unacceptable artifacts. Standard MUSHRA is sensitive but lacks scalability, while ACR scales well but loses sensitivity and saturates at a high quality. To address this, we introduce MUSHRA 1S, a single-stimulus variant that rates one system at a time against a fixed anchor and reference. Across our experiments, MUSHRA 1S matches standard MUSHRA more closely than ACR, including in the high-quality regime, where ACR saturates. MUSHRA 1S also effectively identifies specific deviations and reduces range-equalizing biases by fixing context. Overall, MUSHRA 1S combines MUSHRA level sensitivity with ACR like scalability, making it a robust and scalable solution for benchmarking top-tier speech processing systems.

Method: The authors created a comprehensive 1321-hour roadside dataset with 16kHz audio synchronized with frame-level pedestrian annotations and 1fps video thumbnails. They conducted three analyses: cross-dataset evaluation between noisy and noise-limited environments, assessment of noisy data impact on model performance, and evaluation of predictive robustness on out-of-domain sounds.

Result: The paper presents results showing the challenges of audio-based pedestrian detection in vehicular noise environments and provides insights into how acoustic context affects model performance.

Conclusion: The study demonstrates the importance of considering real-world noisy environments for audio-based pedestrian detection and provides a valuable dataset and analysis framework for future research in this challenging domain.

Abstract: Audio-based pedestrian detection is a challenging task and has, thus far, only been explored in noise-limited environments. We present a new dataset, results, and a detailed analysis of the state-of-the-art in audio-based pedestrian detection in the presence of vehicular noise. In our study, we conduct three analyses: (i) cross-dataset evaluation between noisy and noise-limited environments, (ii) an assessment of the impact of noisy data on model performance, highlighting the influence of acoustic context, and (iii) an evaluation of the model’s predictive robustness on out-of-domain sounds. The new dataset is a comprehensive 1321-hour roadside dataset. It incorporates traffic-rich soundscapes. Each recording includes 16kHz audio synchronized with frame-level pedestrian annotations and 1fps video thumbnails.

Method: Uses a three-component architecture: speech autoencoder for latent representation, flow-matching text-to-latent module, and utterance-level duration predictor. Employs low-dimensional latent space, temporal compression, ConvNeXt blocks, and operates directly on raw character-level text with cross-attention for alignment.

Result: Achieves performance comparable to contemporary zero-shot TTS models with only 44M parameters, significantly reducing complexity and computational cost.

Conclusion: SupertonicTTS demonstrates that efficient TTS synthesis can be achieved through architectural simplifications and optimized training techniques without sacrificing performance quality.

Abstract: We introduce SupertonicTTS, a novel text-to-speech (TTS) system designed for efficient and streamlined speech synthesis. SupertonicTTS comprises three components: a speech autoencoder for continuous latent representation, a text-to-latent module leveraging flow-matching for text-to-latent mapping, and an utterance-level duration predictor. To enable a lightweight architecture, we employ a low-dimensional latent space, temporal compression of latents, and ConvNeXt blocks. The TTS pipeline is further simplified by operating directly on raw character-level text and employing cross-attention for text-speech alignment, thus eliminating the need for grapheme-to-phoneme (G2P) modules and external aligners. In addition, we propose context-sharing batch expansion that accelerates loss convergence and stabilizes text-speech alignment with minimal memory and I/O overhead. Experimental results demonstrate that SupertonicTTS delivers performance comparable to contemporary zero-shot TTS models with only 44M parameters, while significantly reducing architectural complexity and computational cost. Audio samples are available at: https://supertonictts.github.io/.

Method: Fine-tunes Qwen2-Audio model using LoRA on 30-second audio-transcript windows, predicts normalized boundary offsets with soft supervision from LLM-generated labels verified by trained raters

Result: Achieves MAE of 5.3s across tasks on 308 real PE sessions, within typical rater tolerance for timestamp review

Conclusion: Introduces privacy-preserving, scalable framework for PE therapy fidelity tracking with practical applications for clinician training and quality assurance

Abstract: Prolonged Exposure (PE) therapy is an effective treatment for post-traumatic stress disorder (PTSD), but evaluating therapist fidelity remains labor-intensive due to the need for manual review of session recordings. We present a method for the automatic temporal localization of key PE fidelity elements, identifying their start and stop times, directly from session audio and transcripts. Our approach fine-tunes a large pre-trained audio-language model, Qwen2-Audio, using Low-Rank Adaptation (LoRA) to process focused 30-second windows of audio-transcript input. Fidelity labels for three core protocol phases, therapist orientation (P1), imaginal exposure (P2), and post-imaginal processing (P3), are generated via LLM-based prompting and verified by trained raters. The model is trained to predict normalized boundary offsets using soft supervision guided by task-specific prompts. On a dataset of 308 real PE sessions, our best configuration (LoRA rank 8, 30s windows) achieves a mean absolute error (MAE) of 5.3s across tasks, within typical rater tolerance for timestamp review, enabling practical fidelity QC. We further analyze the effects of window size and LoRA rank, highlighting the importance of context granularity and model adaptation. This work introduces a privacy-preserving, scalable framework for fidelity tracking in PE therapy, with potential to support clinician training, supervision, and quality assurance.

Method: Replaces the full-narrow network block in IPDnet with a FUN block consisting of one full-band layer followed by a U-Net with narrow-band layers in multiple scales. Introduces skip connections both within U-Nets and between FUN blocks to enhance information flow.

Result: FUN-SSL outperformed previously proposed approaches while having much lower computational complexity than IPDnet.

Conclusion: The proposed U-Net-based architecture successfully reduces computational burden while improving sound source localization performance, making it a more practical solution for real-world applications.

Abstract: Dual-path processing along the temporal and spectral dimensions has shown to be effective in various speech processing applications. While the sound source localization (SSL) models utilizing dual-path processing such as the FN-SSL and IPDnet demonstrated impressive performances in localizing multiple moving sources, they require significant amount of computation. In this paper, we propose an architecture for SSL which introduces a U-Net to perform narrow-band processing in multiple resolutions to reduce computational complexity. The proposed model replaces the full-narrow network block in the IPDnet consisting of one full-band LSTM layer along the spectral dimension followed by one narrow-band LSTM layer along the temporal dimension with the FUN block composed of one Full-band layer followed by a U-net with Narrow-band layers in multiple scales. On top of the skip connections within each U-Net, we also introduce the skip connections between FUN blocks to enrich information. Experimental results showed that the proposed FUN-SSL outperformed previously proposed approaches with computational complexity much lower than that of the IPDnet.

Method: Proposes C-MSM which approximates the full posterior distribution through stochastic sampling over partial measurement posterior scores while simultaneously estimating CSMs directly from k-space data.

Result: Experiments on multi-coil brain fastMRI dataset show C-MSM achieves reconstruction performance close to DIS with clean diffusion priors, without needing clean training data or pre-calibrated CSMs.

Conclusion: C-MSM successfully eliminates dependencies on pre-calibrated CSMs and ground truth images while maintaining competitive reconstruction performance.

Abstract: Diffusion-based inverse problem solvers (DIS) have recently shown outstanding performance in compressed-sensing parallel MRI reconstruction by combining diffusion priors with physical measurement models. However, they typically rely on pre-calibrated coil sensitivity maps (CSMs) and ground truth images, making them often impractical: CSMs are difficult to estimate accurately under heavy undersampling and ground-truth images are often unavailable. We propose Calibration-free Measurement Score-based diffusion Model (C-MSM), a new method that eliminates these dependencies by jointly performing automatic CSM estimation and self-supervised learning of measurement scores directly from k-space data. C-MSM reconstructs images by approximating the full posterior distribution through stochastic sampling over partial measurement posterior scores, while simultaneously estimating CSMs. Experiments on the multi-coil brain fastMRI dataset show that C-MSM achieves reconstruction performance close to DIS with clean diffusion priors – even without access to clean training data and pre-calibrated CSMs.

Method: Uses pre-trained ViT-Base-Patch16-224 model fine-tuned for classification tasks with a preprocessing pipeline that converts histopathological images into standardized PyTorch tensors. Evaluated on BreakHis (binary) and UBC-OCEAN (5-class) datasets without data augmentation.

Result: Surpasses existing CNN, ViT, and topological data analysis-based approaches in binary classification. Demonstrates superior performance against recent topological methods in multi-class classification.

Conclusion: Vision Transformer-based transfer learning combined with efficient preprocessing is highly effective for oncological diagnostics.

Abstract: Cancer is one of the leading health challenges for women, specifically breast and ovarian cancer. Early detection can help improve the survival rate through timely intervention and treatment. Traditional methods of detecting cancer involve manually examining mammograms, CT scans, ultrasounds, and other imaging types. However, this makes the process labor-intensive and requires the expertise of trained pathologists. Hence, making it both time-consuming and resource-intensive. In this paper, we introduce a novel vision transformer (ViT)-based method for detecting and classifying breast and ovarian cancer. We use a pre-trained ViT-Base-Patch16-224 model, which is fine-tuned for both binary and multi-class classification tasks using publicly available histopathological image datasets. Further, we use a preprocessing pipeline that converts raw histophological images into standardized PyTorch tensors, which are compatible with the ViT architecture and also help improve the model performance. We evaluated the performance of our model on two benchmark datasets: the BreakHis dataset for binary classification and the UBC-OCEAN dataset for five-class classification without any data augmentation. Our model surpasses existing CNN, ViT, and topological data analysis-based approaches in binary classification. For multi-class classification, it is evaluated against recent topological methods and demonstrates superior performance. Our study highlights the effectiveness of Vision Transformer-based transfer learning combined with efficient preprocessing in oncological diagnostics.

Method: Builds upon Non-Overfitted Cool-chic framework using a hypernetwork that generates decoder parameters tailored to input images without per-image fine-tuning. The hypernetwork output also serves as strong initialization for optional fine-tuning.

Result: Achieves 4.9% rate reduction over N-O Cool-chic with minimal computational overhead. With fine-tuning, reaches HEVC-level compression with 60.4% of the encoding cost of fully overfitted Cool-chic.

Conclusion: HyperCool provides a practical method to accelerate encoding in overfitted image codecs, improving their viability for compute-constrained scenarios while maintaining competitive compression performance.

Abstract: Overfitted image codecs like Cool-chic achieve strong compression by tailoring lightweight models to individual images, but their encoding is slow and computationally expensive. To accelerate encoding, Non-Overfitted (N-O) Cool-chic replaces the per-image optimization with a learned inference model, trading compression performance for encoding speed. We introduce HyperCool, a hypernetwork architecture that mitigates this trade-off. Building upon the N-O Cool-chic framework, HyperCool generates content-adaptive parameters for a Cool-chic decoder in a single forward pass, tailoring the decoder to the input image without requiring per-image fine-tuning. Our method achieves a 4.9% rate reduction over N-O Cool-chic with minimal computational overhead. Furthermore, the output of our hypernetwork provides a strong initialization for further optimization, reducing the number of steps needed to approach fully overfitted model performance. With fine-tuning, HEVC-level compression is achieved with 60.4% of the encoding cost of the fully overfitted Cool-chic. This work proposes a practical method to accelerate encoding in overfitted image codecs, improving their viability in scenarios with tight compute budgets.

Method: Models RFI as mixture of multiple LFM components in focused SAR image domain, uses sparse parametric representation with discretized LFM dictionary to estimate FM rates, and applies 2-D SPECAN algorithm with LFM focusing and notch filtering.

Result: Experimental studies on Sentinel-1 images demonstrate that the proposed LFM model and sparse parametric estimation scheme outperforms existing RFI removal methods.

Conclusion: The proposed approach effectively addresses RFI mitigation in SAR imagery through improved LFM modeling and sparse parameter estimation, showing superior performance compared to current methods.

Abstract: One of the challenges in spaceborne synthetic aperture radar (SAR) is modeling and mitigating radio frequency interference (RFI) artifacts in SAR imagery. Linear frequency modulated (LFM) signals have been commonly used for characterizing the radar interferences in SAR. In this letter, we propose a new signal model that approximates RFI as a mixture of multiple LFM components in the focused SAR image domain. The azimuth and range frequency modulation (FM) rates for each LFM component are estimated effectively using a sparse parametric representation of LFM interferences with a discretized LFM dictionary. This approach is then tested within the recently developed RFI suppression framework using a 2-D SPECtral ANalysis (2-D SPECAN) algorithm through LFM focusing and notch filtering in the spectral domain [1]. Experimental studies on Sentinel-1 single-look complex images demonstrate that the proposed LFM model and sparse parametric estimation scheme outperforms existing RFI removal methods.

Method: Leverages the CDF’s monotonic property to perform dynamic binary search for symbol decoding, eliminating table construction. Uses SIMD optimizations and numerical approximations to accelerate the entropy coding process.

Result: Achieves up to approximately 90x acceleration in GMM entropy coding while maintaining equivalent rate-distortion performance compared to traditional methods.

Conclusion: The proposed algorithm significantly improves the practicality of GMM-based codecs by removing the major runtime bottleneck, making high-performance learned image compression more feasible for real-world applications.

Abstract: High-performance learned image compression codecs require flexible probability models to fit latent representations. Gaussian Mixture Models (GMMs) were proposed to satisfy this demand, but suffer from a significant runtime performance bottleneck due to the large Cumulative Distribution Function (CDF) tables that must be built for rANS coding. This paper introduces a fast coding algorithm that entirely eliminates this bottleneck. By leveraging the CDF’s monotonic property, our decoder performs a dynamic binary search to find the correct symbol, eliminating the need for costly table construction and lookup. Aided by SIMD optimizations and numerical approximations, our approach accelerates the GMM entropy coding process by up to approximately 90x without compromising rate-distortion performance, significantly improving the practicality of GMM-based codecs. The implementation will be made publicly available at https://github.com/tokkiwa/FlashGMM.

Method: Proposes a pixel-wise event-driven approach where pixels independently report depth once sufficient signal-to-noise ratio is achieved. Validated through two hardware implementations: offline 256×128 SPAD array with PC processing and real-time FPGA prototype with 2.4μs latency. Also derived semi-closed-form solution for detection probability of raw-peak finding.

Result: Demonstrates robust depth estimation, reflectivity reconstruction, and dynamic event-based representation under static and dynamic conditions. Asynchronous operation reduces latency, mitigates motion blur, and increases effective frame rate compared to frame-based systems.

Conclusion: Establishes foundation for compact, low-latency, event-driven LiDAR architectures suitable for robotics, autonomous driving, and consumer applications. The method remains tunable via simple hyperparameters and benefits both conventional and proposed LiDAR systems.

Abstract: We propose a fully asynchronous peak detection approach for SPAD-based direct time-of-flight (dToF) flash LiDAR, enabling pixel-wise event-driven depth acquisition without global synchronization. By allowing pixels to independently report depth once a sufficient signal-to-noise ratio is achieved, the method reduces latency, mitigates motion blur, and increases effective frame rate compared to frame-based systems. The framework is validated under two hardware implementations: an offline 256$\times$128 SPAD array with PC based processing and a real-time FPGA proof-of-concept prototype with 2.4$\upmu$s latency for on-chip integration. Experiments demonstrate robust depth estimation, reflectivity reconstruction, and dynamic event-based representation under both static and dynamic conditions. The results confirm that asynchronous operation reduces redundant background data and computational load, while remaining tunable via simple hyperparameters. These findings establish a foundation for compact, low-latency, event-driven LiDAR architectures suited to robotics, autonomous driving, and consumer applications. In addition, we have derived a semi-closed-form solution for the detection probability of the raw-peak finding based LiDAR systems that could benefit both conventional frame-based and proposed asynchronous LiDAR systems.

Method: MOIS-SAM2 extends the transformer-based Segment Anything Model 2 with exemplar-based semantic propagation, trained on 119 WB-MRI scans from 84 NF1 patients using T2-weighted fat-suppressed sequences.

Result: Achieved scan-wise DSC of 0.60 (vs. 0.54 for nnU-Net and 0.35 for SAM2), maintained performance under domain shifts (DSC: 0.50-0.53), and showed model-to-expert agreement comparable to inter-expert variability.

Conclusion: MOIS-SAM2 enables efficient, scalable interactive segmentation of neurofibromas with minimal user input and strong generalization, supporting clinical workflow integration.

Abstract: Background and Objectives: Neurofibromatosis type 1 is a genetic disorder characterized by the development of numerous neurofibromas (NFs) throughout the body. Whole-body MRI (WB-MRI) is the clinical standard for detection and longitudinal surveillance of NF tumor growth. Existing interactive segmentation methods fail to combine high lesion-wise precision with scalability to hundreds of lesions. This study proposes a novel interactive segmentation model tailored to this challenge. Methods: We introduce MOIS-SAM2, a multi-object interactive segmentation model that extends the state-of-the-art, transformer-based, promptable Segment Anything Model 2 (SAM2) with exemplar-based semantic propagation. MOIS-SAM2 was trained and evaluated on 119 WB-MRI scans from 84 NF1 patients acquired using T2-weighted fat-suppressed sequences. The dataset was split at the patient level into a training set and four test sets (one in-domain and three reflecting different domain shift scenarios, e.g., MRI field strength variation, low tumor burden, differences in clinical site and scanner vendor). Results: On the in-domain test set, MOIS-SAM2 achieved a scan-wise DSC of 0.60 against expert manual annotations, outperforming baseline 3D nnU-Net (DSC: 0.54) and SAM2 (DSC: 0.35). Performance of the proposed model was maintained under MRI field strength shift (DSC: 0.53) and scanner vendor variation (DSC: 0.50), and improved in low tumor burden cases (DSC: 0.61). Lesion detection F1 scores ranged from 0.62 to 0.78 across test sets. Preliminary inter-reader variability analysis showed model-to-expert agreement (DSC: 0.62-0.68), comparable to inter-expert agreement (DSC: 0.57-0.69). Conclusions: The proposed MOIS-SAM2 enables efficient and scalable interactive segmentation of NFs in WB-MRI with minimal user input and strong generalization, supporting integration into clinical workflows.

Method: The paper conducts a systematic review and analysis of existing NVDC solutions in the literature, guided by a newly proposed classification taxonomy to better organize the field.

Result: The review provides a structured overview of current NVDC solutions, allowing for more solid conclusions about the status quo and identification of research gaps.

Conclusion: The proposed taxonomy helps better organize the emerging NVDC field, enabling more informed future research directions and standardization efforts for neuromorphic vision sensor applications.

Abstract: In recent years, visual sensors have been quickly improving towards mimicking the visual information acquisition process of human brain by responding to illumination changes as they occur in time rather than at fixed time intervals. In this context, the so-called neuromorphic vision sensors depart from the conventional frame-based image sensors by adopting a paradigm shift in the way visual information is acquired. This new way of visual information acquisition enables faster and asynchronous per-pixel responses/recordings driven by the scene dynamics with a very high dynamic range and low power consumption. However, depending on the application scenario, the emerging neuromorphic vision sensors may generate a large volume of data, thus critically demanding highly efficient coding solutions in order applications may take full advantage of these new, attractive sensors’ capabilities. For this reason, considerable research efforts have been invested in recent years towards developing increasingly efficient neuromorphic vision data coding (NVDC) solutions. In this context, the main objective of this paper is to provide a comprehensive overview of NVDC solutions in the literature, guided by a novel classification taxonomy, which allows better organizing this emerging field. In this way, more solid conclusions can be drawn about the current NVDC status quo, thus allowing to better drive future research and standardization developments in this emerging technical area.

Method: Proposed GlaLSTM - a concurrent LSTM stream framework that leverages latent biomarker relationships for glaucoma detection, providing enhanced model transparency and interpretability.

Result: Experimental evaluations confirm that GlaLSTM surpasses existing state-of-the-art methods in detection accuracy while providing actionable insights for clinicians.

Conclusion: GlaLSTM demonstrates potential for both advanced biomarker analysis and reliable glaucoma detection, empowering clinicians with deeper interpretability for more informed decision-making.

Abstract: Glaucoma is a complex group of eye diseases marked by optic nerve damage, commonly linked to elevated intraocular pressure and biomarkers like retinal nerve fiber layer thickness. Understanding how these biomarkers interact is crucial for unraveling glaucoma’s underlying mechanisms. In this paper, we propose GlaLSTM, a novel concurrent LSTM stream framework for glaucoma detection, leveraging latent biomarker relationships. Unlike traditional CNN-based models that primarily detect glaucoma from images, GlaLSTM provides deeper interpretability, revealing the key contributing factors and enhancing model transparency. This approach not only improves detection accuracy but also empowers clinicians with actionable insights, facilitating more informed decision-making. Experimental evaluations confirm that GlaLSTM surpasses existing state-of-the-art methods, demonstrating its potential for both advanced biomarker analysis and reliable glaucoma detection.

Method: Models clipping as element-wise nonlinearity, derives finite-sample recovery bound linking error to mask density and saturation extent. Optimizes mask patterns based on theoretical analysis and introduces SAPnet reconstruction framework with explicit saturation consistency enforcement.

Result: Theoretical analysis validated by experiments on standard video-SCI benchmarks. SAPnet significantly outperforms existing PnP-based methods in saturated conditions.

Conclusion: Provides fundamental theoretical insights for SCI design under saturation, with practical framework that improves reconstruction quality by explicitly handling saturation constraints.

Abstract: Snapshot Compressive Imaging (SCI) uses coded masks to compress a 3D data cube into a single 2D snapshot. In practice, multiplexing can push intensities beyond the sensor’s dynamic range, producing saturation that violates the linear SCI model and degrades reconstruction. This paper provides the first theoretical characterization of SCI recovery under saturation. We model clipping as an element-wise nonlinearity and derive a finite-sample recovery bound for compression-based SCI that links reconstruction error to mask density and the extent of saturation. The analysis yields a clear design rule: optimal Bernoulli masks use densities below one-half, decreasing further as saturation strengthens. Guided by this principle, we optimize mask patterns and introduce a novel reconstruction framework, Saturation-Aware PnP Net (SAPnet), which explicitly enforces consistency with saturated measurements. Experiments on standard video-SCI benchmarks confirm our theory and demonstrate that SAPnet significantly outperforms existing PnP-based methods.

Method: Proposes UltraBoneUDF framework using neural unsigned distance functions (UDFs) with a novel loss function based on local tangent plane optimization for improved surface reconstruction quality from ultrasound volumes.

Result: UltraBoneUDF significantly outperforms competing methods across three datasets: 0.96 mm mean Chamfer distance on UltraBones100k (28.9% improvement), 0.21 mm on OpenBoneCT (40.0% improvement), and 0.18 mm on ClosedBoneCT (63.3% improvement).

Conclusion: The proposed framework effectively addresses the challenge of reconstructing open bone surfaces from incomplete ultrasound data, demonstrating superior performance compared to existing state-of-the-art methods.

Abstract: Bone surface reconstruction is an essential component of computer-assisted orthopedic surgery (CAOS), forming the foundation for preoperative planning and intraoperative guidance. Compared to traditional imaging modalities such as CT and MRI, ultrasound provides a radiation-free, and cost-effective alternative. While ultrasound offers new opportunities in CAOS, technical shortcomings continue to hinder its translation into surgery. In particular, due to the inherent limitations of ultrasound imaging, B-mode ultrasound typically capture only partial bone surfaces, posing major challenges for surface reconstruction. Existing reconstruction methods struggle with such incomplete data, leading to increased reconstruction errors and artifacts. Effective techniques for accurately reconstructing open bone surfaces from real-world 3D ultrasound volumes remain lacking. We propose UltraBoneUDF, a self-supervised framework specifically designed for reconstructing open bone surfaces from ultrasound data using neural unsigned distance functions (UDFs). In addition, we present a novel loss function based on local tangent plane optimization that substantially improves surface reconstruction quality. UltraBoneUDF and competing models are benchmarked on three open-source datasets and further evaluated through ablation studies. Results: Qualitative results highlight the limitations of the state-of-the-art methods for open bone surface reconstruction and demonstrate the effectiveness of UltraBoneUDF. Quantitatively, UltraBoneUDF significantly outperforms competing methods across all evaluated datasets for both open and closed bone surface reconstruction in terms of mean Chamfer distance error: 0.96 mm on the UltraBones100k dataset (28.9% improvement compared to the state-of-the-art), 0.21 mm on the OpenBoneCT dataset (40.0% improvement), and 0.18 mm on the ClosedBoneCT dataset (63.3% improvement).

Method: Constructed reasoning processes based on observation, feature, diagnosis and pathology labels annotated by experienced experts, covering lesions from 4,838 patients with 10,019 lesions.

Result: Created a comprehensive dataset (BUS-CoT) that enables research on incentivizing chain-of-thought reasoning for breast ultrasound analysis.

Conclusion: BUS-CoT facilitates robust AI systems by providing rich annotations and covering all histopathology types, addressing limitations in current publicly available benchmarks.

Abstract: Breast ultrasound (BUS) is an essential tool for diagnosing breast lesions, with millions of examinations per year. However, publicly available high-quality BUS benchmarks for AI development are limited in data scale and annotation richness. In this work, we present BUS-CoT, a BUS dataset for chain-of-thought (CoT) reasoning analysis, which contains 11,439 images of 10,019 lesions from 4,838 patients and covers all 99 histopathology types. To facilitate research on incentivizing CoT reasoning, we construct the reasoning processes based on observation, feature, diagnosis and pathology labels, annotated and verified by experienced experts. Moreover, by covering lesions of all histopathology types, we aim to facilitate robust AI systems in rare cases, which can be error-prone in clinical practice.