Method: Uses separate expert networks for continuous signing, fingerspelling, and lipreading that decode modalities into tokens, then fuses them with a lightweight transformer to resolve temporal misalignments before final generation by an LLM.

Result: Achieves new SOTA on How2Sign benchmark with BLEU-4 score of 23.5 and 73.2% letter accuracy on ChicagoFSWildPlus fingerspelling dataset.

Conclusion: Isolating and solving distinct recognition tasks before fusion provides a more powerful pathway to robust, high-fidelity sign language translation compared to monolithic approaches.

Abstract: Despite progress in gloss-free Sign Language Translation (SLT), monolithic end-to-end models consistently fail on two critical components of natural signing: the precise recognition of high-speed fingerspelling and the integration of asynchronous non-manual cues from the face. Recent progress in Automated Sign Language Translation with Large Language Models has side stepped this challenge, forcing a single network to learn these simultaneously resulting in poor performance when tasked with translating crucial information such as names,places, and technical terms. We introduce MultiStream-LLM, a modular framework designed to overcome these limitations. Our approach employs separate, specialized predictors for continuous signing, fingerspelling, and lipreading. Each expert network first decodes its specific modality into a sequence of tokens. These parallel streams are then fused by a lightweight transformer that resolves temporal misalignments before passing the combined representation to a Large Language Model (LLM) for final sentence generation. Our method establishes a new state-of-the-art on the How2Sign benchmark with a BLEU-4 score of 23.5 and achieves 73.2% letter accuracy on the challenging ChicagoFSWildPlus fingerspelling dataset. These results validate our core hypothesis: by isolating and solving distinct recogni tion tasks before fusion, our multi-expert approach provides a more powerful and effective pathway to robust, high-fidelity sign language translation.

Method: Adapts declarative prompt optimization advances for general-purpose LLMs and proposes a domain-specific framework embedding task declarations, test suites, and automated prompt tuning into reproducible SLR workflows.

Result: Provides a concrete blueprint with working code examples that enables researchers to construct verifiable LLM pipelines aligned with transparency and rigor principles in evidence synthesis.

Conclusion: This represents a novel application of declarative prompt optimization methods to systematic literature review pipelines, offering more reliable and reproducible LLM-assisted evidence synthesis.

Abstract: Large language models (LLMs) offer significant potential to accelerate systematic literature reviews (SLRs), yet current approaches often rely on brittle, manually crafted prompts that compromise reliability and reproducibility. This fragility undermines scientific confidence in LLM-assisted evidence synthesis. In response, this work adapts recent advances in declarative prompt optimisation, developed for general-purpose LLM applications, and demonstrates their applicability to the domain of SLR automation. This research proposes a structured, domain-specific framework that embeds task declarations, test suites, and automated prompt tuning into a reproducible SLR workflow. These emerging methods are translated into a concrete blueprint with working code examples, enabling researchers to construct verifiable LLM pipelines that align with established principles of transparency and rigour in evidence synthesis. This is a novel application of such approaches to SLR pipelines.

Method: The authors overview and analyze various definitions of hypotheses across recently published NLU tasks, discerning nuances and differences in how hypotheses are defined in the literature.

Result: The paper identifies significant variations in how hypotheses are defined across NLU literature, showing that definitions have diverged from traditional scientific meanings and differ even within the NLU field.

Conclusion: Well-structured and well-defined hypotheses are crucial, especially as the field moves toward machine-interpretable scholarly records, requiring clearer and more consistent definitions of hypotheses in NLU research.

Abstract: Over the past decades, alongside advancements in natural language processing, significant attention has been paid to training models to automatically extract, understand, test, and generate hypotheses in open and scientific domains. However, interpretations of the term \emph{hypothesis} for various natural language understanding (NLU) tasks have migrated from traditional definitions in the natural, social, and formal sciences. Even within NLU, we observe differences defining hypotheses across literature. In this paper, we overview and delineate various definitions of hypothesis. Especially, we discern the nuances of definitions across recently published NLU tasks. We highlight the importance of well-structured and well-defined hypotheses, particularly as we move toward a machine-interpretable scholarly record.

Method: Represent each reasoning step via spectral analysis of token-level embeddings, cluster them into semantically coherent latent states, and model their progression as a Markov chain to capture the global structure of reasoning.

Result: The framework provides a structured and interpretable view of the reasoning process that supports semantic role identification, temporal pattern visualization, and consistency evaluation.

Conclusion: The state-aware transition framework offers an effective approach to abstract and analyze chain-of-thought trajectories, enabling better understanding of the semantic evolution and global structure of multi-step reasoning in language models.

Abstract: Recent advances in chain-of-thought (CoT) prompting have enabled large language models (LLMs) to perform multi-step reasoning. However, the explainability of such reasoning remains limited, with prior work primarily focusing on local token-level attribution, such that the high-level semantic roles of reasoning steps and their transitions remain underexplored. In this paper, we introduce a state-aware transition framework that abstracts CoT trajectories into structured latent dynamics. Specifically, to capture the evolving semantics of CoT reasoning, each reasoning step is represented via spectral analysis of token-level embeddings and clustered into semantically coherent latent states. To characterize the global structure of reasoning, we model their progression as a Markov chain, yielding a structured and interpretable view of the reasoning process. This abstraction supports a range of analyses, including semantic role identification, temporal pattern visualization, and consistency evaluation.

Method: Created DiFBench framework with configurable parameters (document set size, distinctiveness thresholds) to systematically evaluate DFM capability across 10 state-of-the-art LLMs.

Result: Significant performance gap between general-purpose and reasoning-enhanced models; all models degrade with increased complexity and document count; common failure is misidentifying frequent features as distinctive.

Conclusion: Contemporary LLMs have core limitations in fine-grained statistical reasoning and rarity detection, highlighting the need for improved capabilities in distinctive feature mining tasks.

Abstract: Effective decision-making often relies on identifying what makes each candidate distinctive. While existing benchmarks for LLMs emphasize retrieving or summarizing information relevant to a given query, they do not evaluate a model’s ability to identify globally distinctive features across a set of documents. We introduce Distinctive Feature Mining (DFM), a new task that challenges models to analyze a small-to-medium collection (10-40 documents) and surface features that are rare in the global context (e.g., appearing in less than 10% of documents). This setting mirrors real-world scenarios such as candidate selection or product differentiation, where statistical reasoning, not retrieval, is key. To enable systematic evaluation of this capability, we present DiFBench, a configurable benchmark creation framework with controllable parameters such as document set size and distinctiveness thresholds. Using DiFBench, we perform a large-scale assessment of distinctive feature mining across ten state-of-the-art LLMs. Our findings reveal a significant performance gap between general-purpose and reasoning-enhanced models. All models, however, substantially degrade as the task complexity and document count increase. We also find that a common failure mode is misidentifying frequent features as distinctive. These insights reveal core limitations in contemporary LLMs’ abilities to perform fine-grained, statistical reasoning and rarity detection.

Method: Develops a framework grounded in C.S. Peirce’s semiotic theory, arguing that models measure latent symbol geometries and can be understood relationally through contrast with other models when performance measures are insufficient.

Result: The framework enables treating models and modeling decisions as signs themselves, providing a basis for understanding model semantics and facilitating comparative analysis across different modeling approaches.

Conclusion: The proposed semiotic framework offers a unified way to analyze modeling practices, addresses foundational questions in the field, and opens new directions for understanding how different models provide interpretive lenses for complex symbolic datasets.

Abstract: The proliferation of methods for modeling of human meaning-making constitutes a powerful class of instruments for the analysis of complex semiotic systems. However, the field lacks a general theoretical framework for describing these modeling practices across various model types in an apples-to-apples way. In this paper, we propose such a framework grounded in the semiotic theory of C. S. Peirce. We argue that such models measure latent symbol geometries, which can be understood as hypotheses about the complex of semiotic agencies underlying a symbolic dataset. Further, we argue that in contexts where a model’s value cannot be straightforwardly captured by proxy measures of performance, models can instead be understood relationally, so that the particular interpretive lens of a model becomes visible through its contrast with other models. This forms the basis of a theory of model semantics in which models, and the modeling decisions that constitute them, are themselves treated as signs. In addition to proposing the framework, we illustrate its empirical use with a few brief examples and consider foundational questions and future directions enabled by the framework.

Method: Decomposes interval-level relations into point-wise comparisons between start/end points, uses temporal closure to infer additional relations and enforce consistency, and provides both Game mode (with scoring) and Annotation mode for custom documents.

Result: A publicly available demo system that serves as both research tool and annotation interface, supporting TempEval-3 dataset annotation and custom document processing with timeline export capabilities.

Conclusion: The Temporal Game provides a novel point-based strategy for temporal annotation that offers greater flexibility than previous approaches and establishes a foundation for training reinforcement learning agents on temporal reasoning tasks.

Abstract: In this paper we demo the Temporal Game, a novel approach to temporal relation extraction that casts the task as an interactive game. Instead of directly annotating interval-level relations, our approach decomposes them into point-wise comparisons between the start and end points of temporal entities. At each step, players classify a single point relation, and the system applies temporal closure to infer additional relations and enforce consistency. This point-based strategy naturally supports both interval and instant entities, enabling more fine-grained and flexible annotation than any previous approach. The Temporal Game also lays the groundwork for training reinforcement learning agents, by treating temporal annotation as a sequential decision-making task. To showcase this potential, the demo presented in this paper includes a Game mode, in which users annotate texts from the TempEval-3 dataset and receive feedback based on a scoring system, and an Annotation mode, that allows custom documents to be annotated and resulting timeline to be exported. Therefore, this demo serves both as a research tool and an annotation interface. The demo is publicly available at https://temporal-game.inesctec.pt, and the source code is open-sourced to foster further research and community-driven development in temporal reasoning and annotation.

Method: Proposes Reasoning-Infused Text Embedding (RITE) which generates intermediate reasoning texts in token space using generative LLMs before computing embeddings, enriching representations with inferential depth.

Result: Experimental results on BRIGHT benchmark show RITE significantly enhances zero-shot retrieval performance across diverse domains in reasoning-intensive retrieval tasks.

Conclusion: Incorporating reasoning into the embedding process through intermediate text generation effectively bridges the gap between LLMs’ reasoning capabilities and text embedding needs, demonstrating substantial performance improvements.

Abstract: Transformer-based models such as BERT and E5 have significantly advanced text embedding by capturing rich contextual representations. However, many complex real-world queries require sophisticated reasoning to retrieve relevant documents beyond surface-level lexical matching, where encoder-only retrievers often fall short. Decoder-only large language models (LLMs), known for their strong reasoning capabilities, offer a promising alternative. Despite this potential, existing LLM-based embedding methods primarily focus on contextual representation and do not fully exploit the reasoning strength of LLMs. To bridge this gap, we propose Reasoning-Infused Text Embedding (RITE), a simple but effective approach that integrates logical reasoning into the text embedding process using generative LLMs. RITE builds upon existing language model embedding techniques by generating intermediate reasoning texts in the token space before computing embeddings, thereby enriching representations with inferential depth. Experimental results on BRIGHT, a reasoning-intensive retrieval benchmark, demonstrate that RITE significantly enhances zero-shot retrieval performance across diverse domains, underscoring the effectiveness of incorporating reasoning into the embedding process.

Method: Introduces OpinioRAG framework combining RAG-based evidence retrieval with LLMs, proposes novel reference-free verification metrics for sentiment-rich domains, and contributes a large-scale dataset of long-form user reviews with expert summaries and annotated queries.

Result: Through extensive experiments, the framework demonstrates capability to generate accurate, relevant, and structured summaries at scale while identifying key challenges and providing actionable insights for system improvement.

Conclusion: OpinioRAG establishes itself as a robust framework for scalable opinion summary generation and paves the way for future research in personalized review summarization with novel evaluation metrics and datasets.

Abstract: We study the problem of opinion highlights generation from large volumes of user reviews, often exceeding thousands per entity, where existing methods either fail to scale or produce generic, one-size-fits-all summaries that overlook personalized needs. To tackle this, we introduce OpinioRAG, a scalable, training-free framework that combines RAG-based evidence retrieval with LLMs to efficiently produce tailored summaries. Additionally, we propose novel reference-free verification metrics designed for sentiment-rich domains, where accurately capturing opinions and sentiment alignment is essential. These metrics offer a fine-grained, context-sensitive assessment of factual consistency. To facilitate evaluation, we contribute the first large-scale dataset of long-form user reviews, comprising entities with over a thousand reviews each, paired with unbiased expert summaries and manually annotated queries. Through extensive experiments, we identify key challenges, provide actionable insights into improving systems, pave the way for future research, and position OpinioRAG as a robust framework for generating accurate, relevant, and structured summaries at scale.

Method: Extends the Price Sentiment Index framework to wage sentiment using LLMs. Develops a scalable data architecture based on Japan’s Economy Watchers Survey that can integrate additional sources like newspapers and social media.

Result: Experimental results show that WSI models based on LLMs significantly outperform both baseline approaches and pretrained models in forecasting wage dynamics.

Conclusion: LLM-driven sentiment indices have strong potential to enhance the timeliness and effectiveness of economic policy design, demonstrating the value of generative AI in economic text analysis.

Abstract: The emergence of generative Artificial Intelligence (AI) has created new opportunities for economic text analysis. This study proposes a Wage Sentiment Index (WSI) constructed with Large Language Models (LLMs) to forecast wage dynamics in Japan. The analysis is based on the Economy Watchers Survey (EWS), a monthly survey conducted by the Cabinet Office of Japan that captures real-time economic assessments from workers in industries highly sensitive to business conditions. The WSI extends the framework of the Price Sentiment Index (PSI) used in prior studies, adapting it specifically to wage related sentiment. To ensure scalability and adaptability, a data architecture is also developed that enables integration of additional sources such as newspapers and social media. Experimental results demonstrate that WSI models based on LLMs significantly outperform both baseline approaches and pretrained models. These findings highlight the potential of LLM-driven sentiment indices to enhance the timeliness and effectiveness of economic policy design by governments and central banks.

Method: Balanced Actor Initialization (BAI) - a two-stage weighted model merging approach that first merges instruction-following and distillation-based reasoning models, then combines this intermediate model with the pretrained model to preserve foundational knowledge.

Result: BAI resolves sequence length collapse, mitigates the reward hockey stick curve, enables continuous sequence length improvement during training, and achieves optimal trade-offs between training stability and reasoning capability preservation.

Conclusion: BAI provides an effective solution for stable training in the third paradigm, enabling more capable reasoning models that combine distillation efficiency with RLHF alignment.

Abstract: The development of alignment and reasoning capabilities in large language models has seen remarkable progress through two paradigms: instruction tuning and reinforcement learning from human feedback (RLHF) alignment paradigm, and distillation-based reasoning fine-tuning paradigm. While both approaches prove effective independently, the third paradigm of applying RLHF to distillation-trained models presents significant challenges. Our investigation reveals two critical phenomena that emerge in this paradigm: Sequence Length Collapse, where language generation dramatically reduces during early RLHF training, and the Reward Hockey Stick Curve, featuring severe reward score drops followed by gradual recovery. These instabilities fundamentally compromise the model’s alignment and reasoning capabilities. To address these challenges, we propose Balanced Actor Initialization (BAI), a two-stage weighted model merging approach. BAI first merges instruction-following and distillation-based reasoning fine-tuned models, then further combines this intermediate model with the pretrained model to preserve foundational knowledge. Through comprehensive experiments across diverse benchmarks and detailed analysis of training experiments, we demonstrate that BAI resolves Sequence Length Collapse, mitigates the Reward Hockey Stick Curve, and enables continuous sequence length improvement during training. Additionally, our analysis reveals that balanced merging ratios achieve optimal trade-offs between training stability and reasoning capability preservation. Our work provides the effective solution for stable training in this third paradigm, enabling more capable reasoning models that combine distillation efficiency with RLHF alignment.

Method: GIER uses natural language descriptions of reasoning gaps to prompt models to iteratively critique and refine their own outputs based on conceptual quality criteria, without relying on demonstrations or chain-of-thought templates.

Result: Across three reasoning tasks and four LLMs, GIER improved rationale quality, grounding, and reasoning alignment without degrading task accuracy. Models successfully interpreted abstract conceptual gaps and translated them into concrete reasoning improvements.

Conclusion: GIER demonstrates that LLMs can effectively use natural language gap descriptions for self-reflection and iterative improvement, providing a general framework for enhancing reasoning quality across various models and tasks.

Abstract: We introduce GIER (Gap-driven Iterative Enhancement of Responses), a general framework for improving large language model (LLM) outputs through self-reflection and revision based on conceptual quality criteria. Unlike prompting strategies that rely on demonstrations, examples, or chain-of-thought templates, GIER utilizes natural language descriptions of reasoning gaps, and prompts a model to iteratively critique and refine its own outputs to better satisfy these criteria. Across three reasoning-intensive tasks (SciFact, PrivacyQA, and e-SNLI) and four LLMs (GPT-4.1, GPT-4o Mini, Gemini 1.5 Pro, and Llama 3.3 70B), GIER improves rationale quality, grounding, and reasoning alignment without degrading task accuracy. Our analysis demonstrates that models can not only interpret abstract conceptual gaps but also translate them into concrete reasoning improvements.

Method: Uses a dual-agent system to recursively build Research Trees from webpages, blurring intermediate nodes into valid sub-problems, and converting these trees into natural language questions requiring full hierarchy traversal. Enables scaling with over 50K training examples and reasoning trajectories via reject sampling.

Result: Models trained on InfoSeek outperform strong baselines. 3B LLMs optimized with InfoSeek surpass 32B models and lightweight commercial APIs, achieving performance comparable to stronger APIs on BrowseComp-Plus benchmark.

Conclusion: InfoSeek effectively addresses the gap in deep research task benchmarks, provides scalable training data, and supports advanced optimization strategies while preserving meta-information for compound reward design and trajectory-level exploration.

Abstract: Large language models (LLMs) are increasingly expected to go beyond simple factual queries toward Deep Research-tasks that require decomposing questions into sub-problems, coordinating multi-step reasoning, and synthesizing evidence from diverse sources. We formalize Deep Research tasks with verifiable answers as Hierarchical Constraint Satisfaction Problems (HCSPs), which are fundamentally different from single-constraint, multi-hop, or flat CSP formulations. However, existing benchmarks (e.g., Natural Questions, HotpotQA) fail to capture this complexity, while recent synthetic datasets often introduce shortcut reasoning, knowledge leakage, or lack sufficient structural depth. To address this gap, we introduce InfoSeek, a scalable framework for synthesizing complex Deep Research tasks. InfoSeek uses a dual-agent system to recursively build a Research Tree from large-scale webpages, blurring intermediate nodes into valid sub-problems, and converting these trees into natural language questions that require traversing the full hierarchy. It also enables rapid scaling, yielding over 50K training examples, a curated test set, and reasoning trajectories generated via reject sampling. Experiments show that models trained on InfoSeek consistently outperform strong baselines. On a challenging benchmark BrowseComp-Plus, 3B LLMs optimized with InfoSeek surpass much larger 32B models and lightweight commercial APIs (e.g., Gemini2.5-Flash), while achieving performance comparable to stronger APIs (e.g., Gemini2.5-Pro). By preserving meta-information such as intermediate steps and retrieval labels, InfoSeek further supports advanced optimization strategies, including compound reward design and trajectory-level exploration. We provide our codes and datasets in \href{https://github.com/VectorSpaceLab/InfoSeek}{this repository}.

Method: Models tokens as graph nodes with importance scores and similarity-based edges, uses decay-signal-propagation mechanism to dynamically update token importance by propagating information across the graph.

Result: Provides a plug-and-play framework that can be seamlessly integrated with existing KV cache eviction methods like SnapKV and PyramidKV.

Conclusion: GraphKV offers an adaptive, graph-based approach to KV cache management that better captures evolving token dependencies compared to static heuristic methods.

Abstract: Efficient Key-Value (KV) cache management is essential for processing long text sequences in large language models (LLMs), where memory constraints often limit performance. Conventional KV eviction strategies, such as top-k selection based on attention scores, depend on static heuristics that fail to capture the evolving implicit dependencies among tokens during inference. To overcome this, we propose GraphKV, a graph-based framework that redefines token selection for KV cache compression. In GraphKV, tokens are modeled as nodes with importance scores, and edges represent their similarity relationships. Through a decay-signal-propagation mechanism, token importance is dynamically updated by propagating information across the graph, enabling adaptive retention of the most contextually significant tokens. GraphKV can be seamlessly utilized in existing KV cache eviction methods such as SnapKV and PyramidKV in a plug-and-play manner. Codes will be released on Github.

Method: Evaluated GCG and annealing-augmented T-GCG algorithms on Qwen2.5-0.5B, LLaMA-3.2-1B, and GPT-OSS-20B models using both safety-oriented prompts (AdvBench) and reasoning-intensive coding prompts, comparing prefix-based heuristics with GPT-4o semantic judgments.

Result: Three key findings: (1) Attack success rates decrease with model size due to complex loss landscapes; (2) Prefix heuristics substantially overestimate effectiveness compared to semantic judgments; (3) Coding prompts are significantly more vulnerable than safety prompts.

Conclusion: GCG has scalability limits, reasoning tasks contain overlooked vulnerabilities, and annealing-inspired strategies like T-GCG show promise for diversifying adversarial search, motivating further development for robust adversarial evaluation.

Abstract: Gradient-based adversarial prompting, such as the Greedy Coordinate Gradient (GCG) algorithm, has emerged as a powerful method for jailbreaking large language models (LLMs). In this paper, we present a systematic appraisal of GCG and its annealing-augmented variant, T-GCG, across open-source LLMs of varying scales. Using Qwen2.5-0.5B, LLaMA-3.2-1B, and GPT-OSS-20B, we evaluate attack effectiveness on both safety-oriented prompts (AdvBench) and reasoning-intensive coding prompts. Our study reveals three key findings: (1) attack success rates (ASR) decrease with model size, reflecting the increasing complexity and non-convexity of larger models’ loss landscapes; (2) prefix-based heuristics substantially overestimate attack effectiveness compared to GPT-4o semantic judgments, which provide a stricter and more realistic evaluation; and (3) coding-related prompts are significantly more vulnerable than adversarial safety prompts, suggesting that reasoning itself can be exploited as an attack vector. In addition, preliminary results with T-GCG show that simulated annealing can diversify adversarial search and achieve competitive ASR under prefix evaluation, though its benefits under semantic judgment remain limited. Together, these findings highlight the scalability limits of GCG, expose overlooked vulnerabilities in reasoning tasks, and motivate further development of annealing-inspired strategies for more robust adversarial evaluation.

Method: Uses Large Language Models to generate coherent answers but grounds them in trustworthy medical studies dynamically retrieved from PubMed database, with traceable key points and arguments linked to respective studies.

Result: User study showed medical experts and lay users find the system usable, helpful, trustworthy and informative. Provides overview of study support/refutation levels and research consensus evolution visualization.

Conclusion: The system is well-suited for both everyday health questions and advanced research insights, effectively bridging the gap between AI-generated content and evidence-based medical research.

Abstract: In the digital age, people often turn to the Internet in search of medical advice and recommendations. With the increasing volume of online content, it has become difficult to distinguish reliable sources from misleading information. Similarly, millions of medical studies are published every year, making it challenging for researchers to keep track of the latest scientific findings. These evolving studies can reach differing conclusions, which is not reflected in traditional search tools. To address these challenges, we introduce MedSEBA, an interactive AI-powered system for synthesizing evidence-based answers to medical questions. It utilizes the power of Large Language Models to generate coherent and expressive answers, but grounds them in trustworthy medical studies dynamically retrieved from the research database PubMed. The answers consist of key points and arguments, which can be traced back to respective studies. Notably, the platform also provides an overview of the extent to which the most relevant studies support or refute the given medical claim, and a visualization of how the research consensus evolved through time. Our user study revealed that medical experts and lay users find the system usable and helpful, and the provided answers trustworthy and informative. This makes the system well-suited for both everyday health questions and advanced research insights.

Method: Created Camlang, a novel constructed language with naturalistic but unattested features, providing explicit grammar rules and bilingual dictionary. Adapted CommonsenseQA into Camlang-CSQA-v0 task requiring grammar rule application and lexical mapping.

Result: GPT-5 achieved 98% EM accuracy in English but only 47% in Camlang, far below human performance at 87%. Most model successes came from shallow lexical alignment rather than systematic grammatical mastery.

Conclusion: Camlang exposes fundamental gaps between current LLMs and human metalinguistic competence, showing models lack genuine reasoning abilities despite benchmark success.

Abstract: Large Language Models (LLMs) achieve gold-medal performance across many benchmarks, yet it remains unclear whether such success reflects genuine reasoning or pattern matching. From a cognitive science perspective, an informative test is whether models can master an unfamiliar language through explicit metalinguistic deductive learning, a paradigm where human learners can reliably internalise grammatical systems through metalinguistic reasoning. We address this question with Camlang, a novel constructed language that exhibits naturalistic yet unattested feature combinations. Camlang consists of two explicit resources, a grammar book and a bilingual dictionary, which mirror adult second-language learning via explicit grammar rules and lexical lookup, and enable us to disentangle errors in morpho-syntax, lexical semantics, and sentence-level reasoning. Human experiments show that these resources are sufficient for participants to acquire Camlang and successfully solve Camlang tasks. To operationalise evaluation, we adapt CommonsenseQA into Camlang, creating Camlang-CSQA-v0, the first task in a broader suite where solving questions requires applying grammar rules and lexical mappings. Experimental results show that GPT-5 achieves 98% EM accuracy in English but only 47% in Camlang, far below human performance at 87%, while other state-of-the-art reasoning LLMs perform even worse. Human verification further reveals that most model successes stem from shallow lexical alignment while GPT-5 shows emerging metalinguistic awareness to a limited extent but not systematic grammatical mastery as humans. Camlang establishes a cognitively grounded evaluation paradigm that exposes fundamental gaps between current models and human metalinguistic competence.

Method: GOSU performs global merging on pre-extracted semantic units from local chunks, guides entity and relationship extraction to reduce coreference resolution difficulty, and introduces hierarchical keyword extraction with semantic unit completion to capture both fine-grained binary relationships and coarse-grained n-ary relationships.

Result: Evaluation across multiple tasks demonstrates that GOSU outperforms baseline RAG methods in terms of generation quality.

Conclusion: The GOSU framework effectively addresses the limitations of traditional graph-based RAG by leveraging global semantic unit processing and hierarchical relationship capture, resulting in improved generation performance.

Abstract: Building upon the standard graph-based Retrieval-Augmented Generation (RAG), the introduction of heterogeneous graphs and hypergraphs aims to enrich retrieval and generation by leveraging the relationships between multiple entities through the concept of semantic units (SUs). But this also raises a key issue: The extraction of high-level SUs limited to local text chunks is prone to ambiguity, complex coupling, and increased retrieval overhead due to the lack of global knowledge or the neglect of fine-grained relationships. To address these issues, we propose GOSU, a semantic unit-centric RAG framework that efficiently performs global disambiguation and utilizes SUs to capture interconnections between different nodes across the global context. In the graph construction phase, GOSU performs global merging on the pre-extracted SUs from local text chunks and guides entity and relationship extraction, reducing the difficulty of coreference resolution while uncovering global semantic objects across text chunks. In the retrieval and generation phase, we introduce hierarchical keyword extraction and semantic unit completion. The former uncovers the fine-grained binary relationships overlooked by the latter, while the latter compensates for the coarse-grained n-ary relationships missing from the former. Evaluation across multiple tasks demonstrates that GOSU outperforms the baseline RAG methods in terms of generation quality.

Method: Specialized Arabic text encoder (MARBERT, ArabicBERT, AraBERT) with Attentive Relevance Scoring (ARS) to rank answer options by semantic relevance. Fast, on-device inference without generative reasoning.

Result: MARBERT-based approach achieved 69.87% accuracy on QIAS 2025 dataset, while large API-based LLMs (Gemini, DeepSeek) reached up to 87.6% accuracy but required more resources and context.

Conclusion: Quantifies trade-off between peak performance of large models (87.6%) and practical advantages of smaller specialized systems (69.87%) - efficiency, on-device deployability, and privacy in high-stakes domains like Islamic inheritance law.

Abstract: Islamic inheritance law (Ilm al-Mawarith) requires precise identification of heirs and calculation of shares, which poses a challenge for AI. In this paper, we present a lightweight framework for solving multiple-choice inheritance questions using a specialised Arabic text encoder and Attentive Relevance Scoring (ARS). The system ranks answer options according to semantic relevance, and enables fast, on-device inference without generative reasoning. We evaluate Arabic encoders (MARBERT, ArabicBERT, AraBERT) and compare them with API-based LLMs (Gemini, DeepSeek) on the QIAS 2025 dataset. While large models achieve an accuracy of up to 87.6%, they require more resources and are context-dependent. Our MARBERT-based approach achieves 69.87% accuracy, presenting a compelling case for efficiency, on-device deployability, and privacy. While this is lower than the 87.6% achieved by the best-performing LLM, our work quantifies a critical trade-off between the peak performance of large models and the practical advantages of smaller, specialized systems in high-stakes domains.

Method: Leverages token-level entropy as a logit-free, reference-free uncertainty measure and integrates it into a split conformal prediction pipeline to construct prediction sets with formal coverage guarantees.

Result: Empirical evaluations across six large language models and two benchmarks demonstrate TECP consistently achieves reliable coverage and compact prediction sets, outperforming prior self-consistency-based methods.

Conclusion: TECP provides a principled and efficient solution for trustworthy generation in black-box LLM settings with provable error control.

Abstract: Uncertainty quantification (UQ) for open-ended language generation remains a critical yet underexplored challenge, especially under black-box constraints where internal model signals are inaccessible. In this paper, we introduce Token-Entropy Conformal Prediction (TECP), a novel framework that leverages token-level entropy as a logit-free, reference-free uncertainty measure and integrates it into a split conformal prediction (CP) pipeline to construct prediction sets with formal coverage guarantees. Unlike existing approaches that rely on semantic consistency heuristics or white-box features, TECP directly estimates epistemic uncertainty from the token entropy structure of sampled generations and calibrates uncertainty thresholds via CP quantiles to ensure provable error control. Empirical evaluations across six large language models and two benchmarks (CoQA and TriviaQA) demonstrate that TECP consistently achieves reliable coverage and compact prediction sets, outperforming prior self-consistency-based UQ methods. Our method provides a principled and efficient solution for trustworthy generation in black-box LLM settings.

Method: Explored four prompting approaches: 1) basic role prompting, 2) human-crafted role prompting, 3) automatic prompt optimization (APO), and 4) rule-based role prompting with character-card/scene-contract design and strict function calling enforcement.

Result: Rule-based role prompting (RRP) achieved best performance with overall score of 0.571, improving on zero-shot baseline score of 0.519, outperforming more elaborate methods like APO.

Conclusion: RRP design substantially improves effectiveness and reliability of role-playing dialogue agents. Best-performing prompts and APO tool are open-sourced to support future persona prompt development.

Abstract: This report investigates approaches for prompting a tool-augmented large language model (LLM) to act as a role-playing dialogue agent in the API track of the Commonsense Persona-grounded Dialogue Challenge (CPDC) 2025. In this setting, dialogue agents often produce overly long in-character responses (over-speaking) while failing to use tools effectively according to the persona (under-acting), such as generating function calls that do not exist or making unnecessary tool calls before answering. We explore four prompting approaches to address these issues: 1) basic role prompting, 2) human-crafted role prompting, 3) automatic prompt optimization (APO), and 4) rule-based role prompting. The rule-based role prompting (RRP) approach achieved the best performance through two novel techniques–character-card/scene-contract design and strict enforcement of function calling–which led to an overall score of 0.571, improving on the zero-shot baseline score of 0.519. These findings demonstrate that RRP design can substantially improve the effectiveness and reliability of role-playing dialogue agents compared with more elaborate methods such as APO. To support future efforts in developing persona prompts, we are open-sourcing all of our best-performing prompts and the APO tool. Source code is available at https://github.com/scb-10x/apo.

Method: Pre-train speech language model via next-token prediction, use predictive entropy to adaptively determine aggregation boundaries, then fuse information within segments using cross-attention module with controllable granularity via entropy threshold.

Result: Experiments on ASR, speech-to-text translation, and voice conversion show compressed representations perform on par with or better than dense token sequences.

Conclusion: The entropy-based dynamic aggregation framework effectively learns compressed semantic speech representations while maintaining or improving performance across multiple speech tasks.

Abstract: Discrete speech representation learning has recently attracted increasing interest in both acoustic and semantic modeling. Existing approaches typically encode 16 kHz waveforms into discrete tokens at a rate of 25 or 50 tokens per second. However, given that speech generally conveys only 2 to 5 words per second, such fine-grained tokenization introduces redundancy and hinders efficiency in downstream training and inference. Moreover, semantic speech representations at this frequency primarily capture phonetic-level information, while semantic understanding may not require such detailed token-level resolution. To address these limitations, we propose an entropy-based dynamic aggregation framework for learning compressed semantic speech representations. A speech language model is first pre-trained via next-token prediction on large-scale unlabeled data to capture frequent token patterns. Predictive entropy is then used to adaptively determine aggregation boundaries, followed by a cross-attention module that fuses information within each segment. By adjusting the entropy threshold, the granularity and compression ratio of the representations can be flexibly controlled. Experiments on ASR, speech-to-text translation, and voice conversion tasks demonstrate that the compressed representations perform on par with or better than dense token sequences, demonstrating the effectiveness of the proposed approach.

Method: Distilled survey articles from 75 research fields into queries and rubrics, with 31 PhD annotators assessing quality. Created automatic pairwise judge with 74% expert agreement. Evaluated 18 systems across 7.6K pairwise evaluations.

Result: No parametric or retrieval-augmented system exceeded 70% rubric coverage. Best agentic system achieved 75% coverage. Systems struggled with citations (11% fully addressed), limitations (48%), and comparisons (49%).

Conclusion: ResearchQA enables comprehensive multi-field evaluation of LLM research capabilities, revealing significant competency gaps in current systems, particularly in citation handling and comprehensive analysis.

Abstract: Evaluating long-form responses to research queries heavily relies on expert annotators, restricting attention to areas like AI where researchers can conveniently enlist colleagues. Yet, research expertise is widespread: survey articles synthesize knowledge distributed across the literature. We introduce ResearchQA, a resource for evaluating LLM systems by distilling survey articles from 75 research fields into 21K queries and 160K rubric items. Each rubric, derived jointly with queries from survey sections, lists query-specific answer evaluation criteria, i.e., citing papers, making explanations, and describing limitations. Assessments by 31 Ph.D. annotators in 8 fields indicate 96% of queries support Ph.D. information needs and 87% of rubric items should be addressed in system responses by a sentence or more. Using our rubrics, we are able to construct an automatic pairwise judge obtaining 74% agreement with expert judgments. We leverage ResearchQA to analyze competency gaps in 18 systems in over 7.6K pairwise evaluations. No parametric or retrieval-augmented system we evaluate exceeds 70% on covering rubric items, and the highest-ranking agentic system shows 75% coverage. Error analysis reveals that the highest-ranking system fully addresses less than 11% of citation rubric items, 48% of limitation items, and 49% of comparison items. We release our data to facilitate more comprehensive multi-field evaluations.

Method: Introduced an evaluation framework grounded in legal fact and reasoning inclusion, analyzing favorability towards stakeholders. Tested models with prompts conditioned on different legal roles, including balancing instructions.

Result: Models exhibited selective inclusion patterns that reflect role-consistent perspectives, showing bias even with balancing instructions. LLMs strategically frame information to align with stakeholder positions.

Conclusion: Findings raise concerns about LLMs inferring user roles from context and exhibiting alignment without explicit instructions. Highlights need for role-aware evaluation in high-stakes legal settings.

Abstract: Large Language Models (LLMs) are increasingly used to generate user-tailored summaries, adapting outputs to specific stakeholders. In legal contexts, this raises important questions about motivated reasoning – how models strategically frame information to align with a stakeholder’s position within the legal system. Building on theories of legal realism and recent trends in legal practice, we investigate how LLMs respond to prompts conditioned on different legal roles (e.g., judges, prosecutors, attorneys) when summarizing judicial decisions. We introduce an evaluation framework grounded in legal fact and reasoning inclusion, also considering favorability towards stakeholders. Our results show that even when prompts include balancing instructions, models exhibit selective inclusion patterns that reflect role-consistent perspectives. These findings raise broader concerns about how similar alignment may emerge as LLMs begin to infer user roles from prior interactions or context, even without explicit role instructions. Our results underscore the need for role-aware evaluation of LLM summarization behavior in high-stakes legal settings.

Method: The study examines how strengthening reasoning capabilities (via switching to think-mode or fine-tuning on benign math datasets) affects LLM responsiveness to malicious requests. Researchers analyze internal model states including attention shifts and specialized experts in mixture-of-experts models.

Result: LLMs become more responsive to malicious requests when reasoning is strengthened, with dense models being particularly vulnerable. Internal analysis shows attention shifts and specialized experts help redirect excessive reasoning towards safety guardrails.

Conclusion: The findings reveal an emerging reasoning-safety trade-off where enhanced reasoning capabilities paradoxically increase vulnerability to malicious requests, underscoring the urgent need to advance alignment techniques for reasoning-enhanced models.

Abstract: With Large Language Models (LLMs) becoming increasingly widely adopted, concerns regarding their safety and alignment with human values have intensified. Previous studies have shown that fine-tuning LLMs on narrow and malicious datasets induce misaligned behaviors. In this work, we report a more concerning phenomenon, Reasoning-Induced Misalignment. Specifically, we observe that LLMs become more responsive to malicious requests when reasoning is strengthened, via switching to “think-mode” or fine-tuning on benign math datasets, with dense models particularly vulnerable. Moreover, we analyze internal model states and find that both attention shifts and specialized experts in mixture-of-experts models help redirect excessive reasoning towards safety guardrails. These findings provide new insights into the emerging reasoning-safety trade-off and underscore the urgency of advancing alignment for advanced reasoning models.

Method: Unsupervised policy learning framework combining prefix-based training with Mixture-of-Experts refiner to learn read/write decisions implicitly. Minimal modifications to standard transformer architectures, generalizes across speech-to-text and text-to-speech tasks.

Result: 500M parameter speech-to-text model outperforms Seamless baseline with under 7% BLEU degradation at 1.5s average lag and under 3% at 3s. Also successfully extended to streaming TTS with unidirectional backbone.

Conclusion: SimulMEGA provides an effective framework for simultaneous speech translation that achieves superior latency-quality tradeoffs across multiple modalities without adding inference-time computational overhead.

Abstract: Simultaneous Speech Translation (SimulST) enables real-time cross-lingual communication by jointly optimizing speech recognition and machine translation under strict latency constraints. Existing systems struggle to balance translation quality, latency, and semantic coherence, particularly in multilingual many-to-many scenarios where divergent read and write policies hinder unified strategy learning. In this paper, we present SimulMEGA (Simultaneous Generation by Mixture-of-Experts Gating), an unsupervised policy learning framework that combines prefix-based training with a Mixture-of-Experts refiner to learn effective read and write decisions in an implicit manner, without adding inference-time overhead. Our design requires only minimal modifications to standard transformer architectures and generalizes across both speech-to-text and text-to-speech streaming tasks. Through comprehensive evaluation on six language pairs, our 500M parameter speech-to-text model outperforms the Seamless baseline, achieving under 7 percent BLEU degradation at 1.5 seconds average lag and under 3 percent at 3 seconds. We further demonstrate the versatility of SimulMEGA by extending it to streaming TTS with a unidirectional backbone, yielding superior latency quality tradeoffs.

Method: Used linear probe to score prompts on test-like to deploy-like scale, then employed LLM rewriting strategy to shift prompts toward natural deployment context while preserving original tasks.

Result: 30% increase in average probe score after rewriting. Across models: 5.26% average increase in honest responses, 12.40% decrease in deceptive responses, and 6.38% increase in refusal rates indicating improved safety compliance.

Conclusion: Evaluation awareness is quantifiable and manipulable, with models being more prone to unsafe/deceptive outputs in perceived test environments, highlighting need for more realistic evaluation frameworks.

Abstract: Large Language Models (LLMs) often exhibit significant behavioral shifts when they perceive a change from a real-world deployment context to a controlled evaluation setting, a phenomenon known as “evaluation awareness.” This discrepancy poses a critical challenge for AI alignment, as benchmark performance may not accurately reflect a model’s true safety and honesty. In this work, we systematically quantify these behavioral changes by manipulating the perceived context of prompts. We introduce a methodology that uses a linear probe to score prompts on a continuous scale from “test-like” to “deploy-like” and leverage an LLM rewriting strategy to shift these prompts towards a more natural, deployment-style context while preserving the original task. Using this method, we achieved a 30% increase in the average probe score across a strategic role-playing dataset after rewriting. Evaluating a suite of state-of-the-art models on these original and rewritten prompts, we find that rewritten “deploy-like” prompts induce a significant and consistent shift in behavior. Across all models, we observed an average increase in honest responses of 5.26% and a corresponding average decrease in deceptive responses of 12.40%. Furthermore, refusal rates increased by an average of 6.38%, indicating heightened safety compliance. Our findings demonstrate that evaluation awareness is a quantifiable and manipulable factor that directly influences LLM behavior, revealing that models are more prone to unsafe or deceptive outputs in perceived test environments. This underscores the urgent need for more realistic evaluation frameworks to accurately gauge true model alignment before deployment.

Method: Replace self-attention with parallel causal convolution (local context) and associative memory retrieval (global patterns), fused dynamically with gating mechanism.

Result: Consistently faster than Transformer and Mamba, achieves superior/competitive perplexity on WikiText-2 and TinyStories datasets.

Conclusion: GAM is a promising efficient alternative to Transformer for sequence modeling with linear complexity and competitive performance.

Abstract: The Transformer architecture, underpinned by the self-attention mechanism, has become the de facto standard for sequence modeling tasks. However, its core computational primitive scales quadratically with sequence length (O(N^2)), creating a significant bottleneck for processing long contexts. In this paper, we propose the Gated Associative Memory (GAM) network, a novel, fully parallel architecture for sequence modeling that exhibits linear complexity (O(N)) with respect to sequence length. The GAM block replaces the self-attention layer with two parallel pathways: a causal convolution to efficiently capture local, position-dependent context, and a parallel associative memory retrieval mechanism to model global, content-based patterns. These pathways are dynamically fused using a gating mechanism, allowing the model to flexibly combine local and global information for each token. We implement GAM from scratch and conduct a rigorous comparative analysis against a standard Transformer model and a modern linear-time baseline (Mamba) on the WikiText-2 benchmark, as well as against the Transformer on the TinyStories dataset. Our experiments demonstrate that GAM is consistently faster, outperforming both baselines on training speed, and achieves a superior or competitive final validation perplexity across all datasets, establishing it as a promising and efficient alternative for sequence modeling.

Method: Comparative analysis of NAC tokens examining adherence to linguistic statistical laws (Zipf’s, Heaps’), entropy, redundancy, and evaluating intelligibility through ASR error rates and quality through UTMOS scores.

Result: NAC tokens, especially 3-grams, exhibit language-like statistical patterns, and these properties correlate with improved speech recognition and resynthesis performance.

Conclusion: The findings provide insights into NAC token sequence structure and can inform the design of more effective generative speech models.

Abstract: This study presents a comparative analysis of the statistical and linguistic properties of neural audio codecs (NACs). We investigate discrete speech tokens produced by various NAC models, examining their adherence to linguistic statistical laws such as Zipf’s law and Heaps’ law, as well as their entropy and redundancy. To assess how these token-level properties relate to semantic and acoustic preservation in synthesized speech, we evaluate intelligibility using error rates of automatic speech recognition, and quality using the UTMOS score. Our results reveal that NAC tokens, particularly 3-grams, exhibit language-like statistical patterns. Moreover, these properties, together with measures of information content, are found to correlate with improved performances in speech recognition and resynthesis tasks. These findings offer insights into the structure of NAC token sequences and inform the design of more effective generative speech models.

Method: Developed three distinct approaches: (1) Fine-tuned RoBERTa-base classifier, (2) TF-IDF + SVM classifier, (3) Candace ensemble model using probabilistic features from multiple Llama-3.2 models processed through custom Transformer encoder.

Result: The RoBERTa-based system performed best, achieving near-perfect results on both development and test sets.

Conclusion: Fine-tuned RoBERTa classifier is the most effective approach among the three systems for detecting AI-generated content in news and academic texts.

Abstract: This paper presents presents three distinct systems developed for the M-DAIGT shared task on detecting AI generated content in news articles and academic abstracts. The systems includes: (1) A fine-tuned RoBERTa-base classifier, (2) A classical TF-IDF + Support Vector Machine (SVM) classifier , and (3) An Innovative ensemble model named Candace, leveraging probabilistic features extracted from multiple Llama-3.2 models processed by a customTransformer encoder.The RoBERTa-based system emerged as the most performant, achieving near-perfect results on both development and test sets.

Method: Created ICPC benchmark with 254 problems including official analysis, reference code, and tests. Evaluated various LM inference techniques including zero-shot chain-of-thought, multi-turn self-judge with reflection, and retrieval over episodic information.

Result: Zero-shot chain-of-thought achieved 19.1% pass@1 solve rate, while best technique (multi-turn self-judge with reflection and retrieval) reached 42.2%. Human-in-the-loop guidance solved 17/18 previously unsolvable problems.

Conclusion: The study provides a step toward LMs with grounded, imaginative, and algorithmic thinking, demonstrating that advanced inference techniques and human guidance significantly improve competitive programming performance.

Abstract: Among the hardest tasks for humans are those found in competitive programming where problems require sophisticated algorithmic thinking, puzzle solving, and the creation of effective code. As a domain to assess language models (LMs), it has not received enough attention, though. This study presents the ICPC benchmark, which consists of 254 international collegiate programming contest (ICPC) tasks. Each problem includes official analysis, reference code, and sample, high-quality unit, and hidden tests. We are able to develop and evaluate a variety of LM inference techniques for competitive programming with these resources. With zero-shot chain-of-thought prompting, we find that o1 only achieves a 19.1% pass@1 solve rate. With our best inference technique, which combines multi-turn self-judge with reflection and retrieval over episodic information, raises this to 42.2%. Furthermore, we conduct a new human-in-the-loop investigation to gain a deeper understanding of the remaining difficulties. Surprisingly, we discover that o1 can solve 17 out of 18 problems that were previously unsolvable by any model or technique with just a few specific instructions. A footstep toward LMs with grounded, imaginative, and algorithmic thinking is provided by our quantitative findings and qualitative research. We open-source our code and data at https://github.com/kraritt/zolve.

Method: Comparative analysis of censored (safety-aligned) and uncensored LLMs on detecting implicit and explicit hate speech, evaluating accuracy, robustness, ideological consistency, and fairness across different target groups.

Result: Censored models significantly outperformed uncensored models (78.7% vs 64.1% accuracy) but were resistant to persona-based influence. All models failed at understanding irony and showed fairness disparities across target groups with unreliable self-reported certainty.

Conclusion: LLMs cannot be considered objective arbiters for hate speech detection due to ideological biases, fairness issues, and poor calibration. More sophisticated auditing frameworks are needed to address fairness, calibration, and ideological consistency.

Abstract: We investigate the efficacy of Large Language Models (LLMs) in detecting implicit and explicit hate speech, examining whether models with minimal safety alignment (uncensored) might provide more objective classification capabilities compared to their heavily-aligned (censored) counterparts. While uncensored models theoretically offer a less constrained perspective free from moral guardrails that could bias classification decisions, our results reveal a surprising trade-off: censored models significantly outperform their uncensored counterparts in both accuracy and robustness, achieving 78.7% versus 64.1% strict accuracy. However, this enhanced performance comes with its own limitation – the safety alignment acts as a strong ideological anchor, making censored models resistant to persona-based influence, while uncensored models prove highly malleable to ideological framing. Furthermore, we identify critical failures across all models in understanding nuanced language such as irony. We also find alarming fairness disparities in performance across different targeted groups and systemic overconfidence that renders self-reported certainty unreliable. These findings challenge the notion of LLMs as objective arbiters and highlight the need for more sophisticated auditing frameworks that account for fairness, calibration, and ideological consistency.

Method: Initialize multiple routers from attention heads of preceding layers during upcycling. These routers collaboratively assign tokens to specialized experts using attention-like mechanism where tokens are processed into queries and aligned with experts’ features as keys.

Result: Experimental results show the method achieves state-of-the-art performance, outperforming other upcycling baselines.

Conclusion: Router Upcycling effectively enhances MoE upcycling performance by leveraging attention mechanisms for improved routing, providing a superior alternative to simple linear routers.

Abstract: The Mixture-of-Experts (MoE) models have gained significant attention in deep learning due to their dynamic resource allocation and superior performance across diverse tasks. However, efficiently training these models remains challenging. The MoE upcycling technique has been proposed to reuse and improve existing model components, thereby minimizing training overhead. Despite this, simple routers, such as linear routers, often struggle with complex routing tasks within MoE upcycling. In response, we propose a novel routing technique called Router Upcycling to enhance the performance of MoE upcycling models. Our approach initializes multiple routers from the attention heads of preceding attention layers during upcycling. These routers collaboratively assign tokens to specialized experts in an attention-like manner. Each token is processed into diverse queries and aligned with the experts’ features (serving as keys). Experimental results demonstrate that our method achieves state-of-the-art (SOTA) performance, outperforming other upcycling baselines.

Method: Tested 14 SLMs from various families using controlled prompt engineering to generate 24,000 headlines across low/high quality deceptive categories, then applied DistilBERT and bagging classifier models for detection.

Result: SLMs showed high compliance in generating fake headlines with minimal ethical resistance. Detection accuracy was poor (35.2-63.5%), indicating generated headlines don’t closely resemble human-written content.

Conclusion: SLMs can effectively generate falsified headlines, existing detection methods are inadequate, and generated content differs significantly from human-written news, posing challenges for content moderation.

Abstract: Small language models (SLMs) have the capability for text generation and may potentially be used to generate falsified texts online. This study evaluates 14 SLMs (1.7B-14B parameters) including LLaMA, Gemma, Phi, SmolLM, Mistral, and Granite families in generating perceived low and high quality fake news headlines when explicitly prompted, and whether they appear to be similar to real-world news headlines. Using controlled prompt engineering, 24,000 headlines were generated across low-quality and high-quality deceptive categories. Existing machine learning and deep learning-based news headline quality detectors were then applied against these SLM-generated fake news headlines. SLMs demonstrated high compliance rates with minimal ethical resistance, though there were some occasional exceptions. Headline quality detection using established DistilBERT and bagging classifier models showed that quality misclassification was common, with detection accuracies only ranging from 35.2% to 63.5%. These findings suggest the following: tested SLMs generally are compliant in generating falsified headlines, although there are slight variations in ethical restraints, and the generated headlines did not closely resemble existing primarily human-written content on the web, given the low quality classification accuracy.

Method: Organized as a shared task with three subtasks: spoken dialect identification (Subtask 1), speech recognition (Subtask 2), and diacritic restoration for spoken dialects (Subtask 3). 44 teams participated with 100 valid submissions.

Result: Best results: 79.8% accuracy on dialect identification, 35.68/12.20 WER/CER on speech recognition, and 55/13 WER/CER on diacritic restoration. 8 unique teams submitted during testing phase.

Conclusion: Results demonstrate significant ongoing challenges in Arabic dialect speech processing, particularly in dialect identification, recognition, and diacritic restoration, highlighting the need for continued research and development in this area.

Abstract: We present the findings of the sixth Nuanced Arabic Dialect Identification (NADI 2025) Shared Task, which focused on Arabic speech dialect processing across three subtasks: spoken dialect identification (Subtask 1), speech recognition (Subtask 2), and diacritic restoration for spoken dialects (Subtask 3). A total of 44 teams registered, and during the testing phase, 100 valid submissions were received from eight unique teams. The distribution was as follows: 34 submissions for Subtask 1 “five teams{\ae}, 47 submissions for Subtask 2 “six teams”, and 19 submissions for Subtask 3 “two teams”. The best-performing systems achieved 79.8% accuracy on Subtask 1, 35.68/12.20 WER/CER (overall average) on Subtask 2, and 55/13 WER/CER on Subtask 3. These results highlight the ongoing challenges of Arabic dialect speech processing, particularly in dialect identification, recognition, and diacritic restoration. We also summarize the methods adopted by participating teams and briefly outline directions for future editions of NADI.

Method: Text Reinforcement model (TeR) generates reinforced text to address weaknesses in original text. Uses reinforcement learning with rewards based on impact on TSF performance and task relevance to optimize text quality.

Result: Extensive experiments on real-world benchmark dataset across various domains show the approach outperforms strong baselines and existing studies, demonstrating effectiveness.

Conclusion: Reinforcing text modalities through the proposed TeR model with reinforcement learning significantly improves multimodal time series forecasting performance by addressing text quality issues.

Abstract: Recent studies in time series forecasting (TSF) use multimodal inputs, such as text and historical time series data, to predict future values. These studies mainly focus on developing advanced techniques to integrate textual information with time series data to perform the task and achieve promising results. Meanwhile, these approaches rely on high-quality text and time series inputs, whereas in some cases, the text does not accurately or fully capture the information carried by the historical time series, which leads to unstable performance in multimodal TSF. Therefore, it is necessary to enhance the textual content to improve the performance of multimodal TSF. In this paper, we propose improving multimodal TSF by reinforcing the text modalities. We propose a text reinforcement model (TeR) to generate reinforced text that addresses potential weaknesses in the original text, then apply this reinforced text to support the multimodal TSF model’s understanding of the time series, improving TSF performance. To guide the TeR toward producing higher-quality reinforced text, we design a reinforcement learning approach that assigns rewards based on the impact of each reinforced text on the performance of the multimodal TSF model and its relevance to the TSF task. We optimize the TeR accordingly, so as to improve the quality of the generated reinforced text and enhance TSF performance. Extensive experiments on a real-world benchmark dataset covering various domains demonstrate the effectiveness of our approach, which outperforms strong baselines and existing studies on the dataset.

Method: Built a contrastive evaluation benchmark using curated contrastive story pairs, conducted comprehensive ablation studies to validate effectiveness.

Result: CE-Bench reliably measures interpretability of sparse autoencoders and aligns well with existing benchmarks, all without external LLM requirements.

Conclusion: The approach provides an effective automated evaluation method for sparse autoencoders, with open-sourced implementation and dataset under MIT License.

Abstract: Probing with sparse autoencoders is a promising approach for uncovering interpretable features in large language models (LLMs). However, the lack of automated evaluation methods has hindered their broader adoption and development. In this work, we introduce CE-Bench, a novel and lightweight contrastive evaluation benchmark for sparse autoencoders, built on a curated dataset of contrastive story pairs. We conduct comprehensive ablation studies to validate the effectiveness of our approach. Our results show that CE-Bench reliably measures the interpretability of sparse autoencoders and aligns well with existing benchmarks, all without requiring an external LLM. The official implementation and evaluation dataset are open-sourced under the MIT License.

Method: Train monolingual systems on carefully balanced mix of high-quality human-labeled, pseudo-labeled, and synthetic data for 27M parameter models across multiple underrepresented languages.

Result: Models achieve error rates 48% lower than comparably sized Whisper Tiny, outperform 9x larger Whisper Small, and match/outperform 28x larger Whisper Medium in most cases.

Conclusion: Monolingual training with quality data balancing enables superior on-device ASR for underrepresented languages, advancing state-of-the-art for small models and providing open-source support for 6 languages.

Abstract: We present the Flavors of Moonshine, a suite of tiny automatic speech recognition (ASR) models specialized for a range of underrepresented languages. Prevailing wisdom suggests that multilingual ASR models outperform monolingual counterparts by exploiting cross-lingual phonetic similarities. We challenge this assumption, showing that for sufficiently small models (27M parameters), training monolingual systems on a carefully balanced mix of high-quality human-labeled, pseudo-labeled, and synthetic data yields substantially superior performance. On average, our models achieve error rates 48% lower than the comparably sized Whisper Tiny model, outperform the 9x larger Whisper Small model, and in most cases match or outperform the 28x larger Whisper Medium model. These results advance the state of the art for models of this size, enabling accurate on-device ASR for languages that previously had limited support. We release Arabic, Chinese, Japanese, Korean, Ukrainian, and Vietnamese Moonshine models under a permissive open-source license.

Method: Proposes RevBrowse framework inspired by ‘browse-then-decide’ decision process, with PrefRAG module that disentangles user/item representations and adaptively retrieves preference-relevant content for target items.

Result: Extensive experiments on four Amazon datasets show consistent and significant improvements over strong baselines, demonstrating generalizability and effectiveness in modeling dynamic user preferences.

Conclusion: RevBrowse enhances LLM-based recommendation by making review usage more efficient and relevant, while providing interpretability through transparent retrieval process that shows which reviews influence recommendations.

Abstract: Large language models (LLMs) have shown strong potential in recommendation tasks due to their strengths in language understanding, reasoning and knowledge integration. These capabilities are especially beneficial for review-based recommendation, which relies on semantically rich user-generated texts to reveal fine-grained user preferences and item attributes. However, effectively incorporating reviews into LLM-based recommendation remains challenging due to (1) inefficient to dynamically utilize user reviews under LLMs’ constrained context windows, and (2) lacking effective mechanisms to prioritize reviews most relevant to the user’s current decision context. To address these challenges, we propose RevBrowse, a review-driven recommendation framework inspired by the “browse-then-decide” decision process commonly observed in online user behavior. RevBrowse integrates user reviews into the LLM-based reranking process to enhance its ability to distinguish between candidate items. To improve the relevance and efficiency of review usage, we introduce PrefRAG, a retrieval-augmented module that disentangles user and item representations into structured forms and adaptively retrieves preference-relevant content conditioned on the target item. Extensive experiments on four Amazon review datasets demonstrate that RevBrowse achieves consistent and significant improvements over strong baselines, highlighting its generalizability and effectiveness in modeling dynamic user preferences. Furthermore, since the retrieval-augmented process is transparent, RevBrowse offers a certain level of interpretability by making visible which reviews influence the final recommendation.

Method: RWS leverages external reward models to evaluate entire intermediate sequences at each diffusion step, scaling token logits based on global sequence quality to guide token selection and promote non-autoregressive generation.

Result: Experiments show RWS significantly promotes non-autoregressive generation orders and improves performance across multiple evaluation metrics compared to standard decoding methods.

Conclusion: Integrating global signals through reward-weighted sampling effectively enhances both non-autoregressive properties and overall performance of masked diffusion models.

Abstract: Masked diffusion models (MDMs) offer a promising non-autoregressive alternative for large language modeling. Standard decoding methods for MDMs, such as confidence-based sampling, select tokens independently based on individual token confidences at each diffusion step. However, we observe that this independent token selection often results in generation orders resembling sequential autoregressive processes, limiting the advantages of non-autoregressive modeling. To mitigate this pheonomenon, we propose Reward-Weighted Sampling (RWS), a novel decoding strategy that leverages an external reward model to provide a principled global signal during the iterative diffusion process. Specifically, at each diffusion step, RWS evaluates the quality of the entire intermediate sequence and scales token logits accordingly, guiding token selection by integrating global sequence-level coherence. This method selectively increases the confidence of tokens that initially have lower scores, thereby promoting a more non-autoregressive generation order. Furthermore, we provide theoretical justification showing that reward-weighted logit scaling induces beneficial rank reversals in token selection and consistently improves expected reward. Experiments demonstrate that RWS significantly promotes non-autoregressive generation orders, leading to improvements across multiple evaluation metrics. These results highlight the effectiveness of integrating global signals in enhancing both the non-autoregressive properties and overall performance of MDMs.

Method: Design-based research methodology with five phases: literature review, SWOT analysis, ethical-pedagogical principles development, system design, and instructional strategy formulation.

Result: Developed an AI-LMS framework with modular components including configurable prompts, adaptive feedback loops, and multi-agent conversation flows aligned with behaviorist, constructivist, and connectivist learning theories.

Conclusion: The study presents a practical model combining AI capabilities with human-centered design and ethical safeguards for education, with future validation through real-world implementation planned.

Abstract: Higher education faces growing challenges in delivering personalized, scalable, and pedagogically coherent learning experiences. This study introduces a structured framework for designing an AI-powered Learning Management System (AI-LMS) that integrates generative and conversational AI to support adaptive, interactive, and learner-centered instruction. Using a design-based research (DBR) methodology, the framework unfolds through five phases: literature review, SWOT analysis, development of ethical-pedagogical principles, system design, and instructional strategy formulation. The resulting AI-LMS features modular components – including configurable prompts, adaptive feedback loops, and multi-agent conversation flows – aligned with pedagogical paradigms such as behaviorist, constructivist, and connectivist learning theories. By combining AI capabilities with human-centered design and ethical safeguards, this study advances a practical model for AI integration in education. Future research will validate and refine the system through real-world implementation.

Method: Leverage TTS to synthesize diverse speech samples containing target rare words, use pretrained Whisper to extract multiple pronunciation variants, compile variants into prefix-trie for shallow-fusion beam-search decoding, then map recognized variants back to original words.

Result: Method reduces biased-word error rate (B-WER) by 43% on LibriSpeech test-clean and 44% on test-other, while maintaining unbiased-WER (U-WER) essentially unchanged.

Conclusion: The proposed synthesis-driven multi-pronunciation approach effectively addresses pronunciation variability in contextual ASR, significantly improving recognition of rare words without compromising general ASR performance.

Abstract: Contextual automatic speech recognition (ASR) systems allow for recognizing out-of-vocabulary (OOV) words, such as named entities or rare words. However, it remains challenging due to limited training data and ambiguous or inconsistent pronunciations. In this paper, we propose a synthesis-driven multi-pronunciation contextual biasing method that performs zero-shot contextual ASR on a pretrained Whisper model. Specifically, we leverage text-to-speech (TTS) systems to synthesize diverse speech samples containing each target rare word, and then use the pretrained Whisper model to extract multiple predicted pronunciation variants. These variant token sequences are compiled into a prefix-trie, which assigns rewards to beam hypotheses in a shallow-fusion manner during beam-search decoding. Subsequently, any recognized variant is mapped back to the original rare word in the final transcription. The evaluation results on the LibriSpeech dataset show that our method reduces biased-word error rate (B-WER) by 43% on test-clean and 44% on test-other while maintaining unbiased-WER (U-WER) essentially unchanged.

Method: Systematic comparison of encoder Transformers (BERT/RoBERTa), decoder LLM (Qwen2.5-7B with LoRA), and FastText baseline using NLPCC 2025 dataset. Encoders used prompt-based MLM, Qwen used instruction-format with classification head.

Result: Encoder models suffered performance degradation (76.3-79.3% accuracy), FastText showed lexical robustness (83.5%) but lacked semantics, while LoRA-adapted Qwen2.5-7B achieved 95.94% accuracy with balanced precision-recall.

Conclusion: Decoder-based LLMs with parameter-efficient fine-tuning are most effective for robust Chinese AI-generated text detection, with future work planned on Qwen3 models and ensemble strategies.

Abstract: The rapid growth of large language models (LLMs) has heightened the demand for accurate detection of AI-generated text, particularly in languages like Chinese, where subtle linguistic nuances pose significant challenges to current methods. In this study, we conduct a systematic comparison of encoder-based Transformers (Chinese BERT-large and RoBERTa-wwm-ext-large), a decoder-only LLM (Alibaba’s Qwen2.5-7B/DeepSeek-R1-Distill-Qwen-7B fine-tuned via Low-Rank Adaptation, LoRA), and a FastText baseline using the publicly available dataset from the NLPCC 2025 Chinese AI-Generated Text Detection Task. Encoder models were fine-tuned using a novel prompt-based masked language modeling approach, while Qwen2.5-7B was adapted for classification with an instruction-format input and a lightweight classification head trained via LoRA. Experiments reveal that although encoder models nearly memorize training data, they suffer significant performance degradation under distribution shifts (RoBERTa: 76.3% test accuracy; BERT: 79.3%). FastText demonstrates surprising lexical robustness (83.5% accuracy) yet lacks deeper semantic understanding. In contrast, the LoRA-adapted Qwen2.5-7B achieves 95.94% test accuracy with balanced precision-recall metrics, indicating superior generalization and resilience to dataset-specific artifacts. These findings underscore the efficacy of decoder-based LLMs with parameter-efficient fine-tuning for robust Chinese AI-generated text detection. Future work will explore next-generation Qwen3 models, distilled variants, and ensemble strategies to enhance cross-domain robustness further.

Method: Uses two-stage faithful decomposition to break answers into sub-facts, retrieves evidence snippets, revises conflicting sub-facts, and aggregates evidence according to original sentences.

Result: Outperforms SOTA methods by over 14% across GPT-3.5-turbo, Gemini and Llama 70B series on six datasets with new Attr_auto-P metric.

Conclusion: FIDES provides a more effective framework for attributed QA through improved fact decomposition and evidence aggregation techniques.

Abstract: Attribution is crucial in question answering (QA) with Large Language Models (LLMs).SOTA question decomposition-based approaches use long form answers to generate questions for retrieving related documents. However, the generated questions are often irrelevant and incomplete, resulting in a loss of facts in retrieval.These approaches also fail to aggregate evidence snippets from different documents and paragraphs. To tackle these problems, we propose a new fact decomposition-based framework called FIDES (\textit{faithful context enhanced fact decomposition and evidence aggregation}) for attributed QA. FIDES uses a contextually enhanced two-stage faithful decomposition method to decompose long form answers into sub-facts, which are then used by a retriever to retrieve related evidence snippets. If the retrieved evidence snippets conflict with the related sub-facts, such sub-facts will be revised accordingly. Finally, the evidence snippets are aggregated according to the original sentences.Extensive evaluation has been conducted with six datasets, with an additionally proposed new metric called $Attr_{auto-P}$ for evaluating the evidence precision. FIDES outperforms the SOTA methods by over 14% in average with GPT-3.5-turbo, Gemini and Llama 70B series.

Method: LegalChainReasoner framework that applies structured legal chains integrating factual premises, composite legal conditions, and sentencing conclusions for flexible knowledge injection and end-to-end opinion generation.

Result: Experiments on two real-world Chinese legal case datasets show the method outperforms baseline models.

Conclusion: The proposed approach addresses limitations of previous methods by ensuring consistency between reasoning and sentencing while providing practical judicial opinion generation capabilities.

Abstract: A criminal judicial opinion represents the judge’s disposition of a case, including the decision rationale and sentencing. Automatically generating such opinions can assist in analyzing sentencing consistency and provide judges with references to similar past cases. However, current research typically approaches this task by dividing it into two isolated subtasks: legal reasoning and sentencing prediction. This separation often leads to inconsistency between the reasoning and predictions, failing to meet real-world judicial requirements. Furthermore, prior studies rely on manually curated knowledge to enhance applicability, yet such methods remain limited in practical deployment. To address these limitations and better align with legal practice, we propose a new LegalAI task: Judicial Opinion Generation, which simultaneously produces both legal reasoning and sentencing decisions. To achieve this, we introduce LegalChainReasoner, a framework that applies structured legal chains to guide the model through comprehensive case assessments. By integrating factual premises, composite legal conditions, and sentencing conclusions, our approach ensures flexible knowledge injection and end-to-end opinion generation. Experiments on two real-world and open-source Chinese legal case datasets demonstrate that our method outperforms baseline models.

Method: A multi-agent system built on LLMs with three complementary modes: 1) group-based learning from merchant’s own top-performing messages, 2) retrieval-and-adaptation of successful templates with similar audience/voucher/product characteristics, 3) rule-based fallback for zero-shot rewriting when no references are available.

Result: Extensive experiments show CRMAgent consistently outperforms merchants’ original templates, delivering significant gains in both audience-match and marketing-effectiveness metrics.

Conclusion: CRMAgent provides an effective solution for merchants to generate high-quality message templates and actionable writing guidance, addressing the scalability and expertise gap in persuasive copywriting for e-commerce CRM channels.

Abstract: In e-commerce private-domain channels such as instant messaging and e-mail, merchants engage customers directly as part of their Customer Relationship Management (CRM) programmes to drive retention and conversion. While a few top performers excel at crafting outbound messages, most merchants struggle to write persuasive copy because they lack both expertise and scalable tools. We introduce CRMAgent, a multi-agent system built on large language models (LLMs) that generates high-quality message templates and actionable writing guidance through three complementary modes. First, group-based learning enables the agent to learn from a merchant’s own top-performing messages within the same audience segment and rewrite low-performing ones. Second, retrieval-and-adaptation fetches templates that share the same audience segment and exhibit high similarity in voucher type and product category, learns their successful patterns, and adapts them to the current campaign. Third, a rule-based fallback provides a lightweight zero-shot rewrite when no suitable references are available. Extensive experiments show that CRMAgent consistently outperforms merchants’ original templates, delivering significant gains in both audience-match and marketing-effectiveness metrics.

Method: Supervised fine-tuning of LLaMA 3 8B using curated biomedical datasets (BioASQ, MedQuAD, TREC) with three experimental setups: combined short/long answers, short only, and long only answers, plus a two-stage inference pipeline for precise answer extraction.

Result: Models achieved concept-level accuracy scores up to 0.8 but showed significantly lower Exact Match scores, especially in testing. Two-stage inference improved short-answer extraction but challenges remained in strict output formatting.

Conclusion: There’s a significant gap between semantic understanding and exact answer evaluation in biomedical LLMs, highlighting the need for further research in output control and post-processing strategies.

Abstract: Large language models (LLMs) are increasingly evident for accurate question answering across various domains. However, rigorous evaluation of their performance on complex question-answering (QA) capabilities is essential before deployment in real-world biomedical and healthcare applications. This paper presents our approach to the MedHopQA track of the BioCreative IX shared task, which focuses on multi-hop biomedical question answering involving diseases, genes, and chemicals. We adopt a supervised fine-tuning strategy leveraging LLaMA 3 8B, enhanced with a curated biomedical question-answer dataset compiled from external sources including BioASQ, MedQuAD, and TREC. Three experimental setups are explored: fine-tuning on combined short and long answers, short answers only, and long answers only. While our models demonstrate strong domain understanding, achieving concept-level accuracy scores of up to 0.8, their Exact Match (EM) scores remain significantly lower, particularly in the test phase. We introduce a two-stage inference pipeline for precise short-answer extraction to mitigate verbosity and improve alignment with evaluation metrics. Despite partial improvements, challenges persist in generating strictly formatted outputs. Our findings highlight the gap between semantic understanding and exact answer evaluation in biomedical LLM applications, motivating further research in output control and post-processing strategies.

Method: TMT (Topic Model Translation) is a novel technique that transfers topic models (like LDA) from one language to another without needing metadata, embeddings, or aligned corpora.

Result: Extensive evaluation shows TMT produces semantically coherent and consistent topic translations across languages, validated through both quantitative and qualitative methods.

Conclusion: TMT provides a robust and transparent solution for reusing topic models across languages, particularly valuable when large target language corpora are unavailable or manual translation is impractical.

Abstract: The training of topic models for a multilingual environment is a challenging task, requiring the use of sophisticated algorithms, topic-aligned corpora, and manual evaluation. These difficulties are further exacerbated when the developer lacks knowledge of the target language or is working in an environment with limited data, where only small or unusable multilingual corpora are available. Considering these challenges, we introduce Topic Model Translation (TMT), a novel, robust and transparent technique designed to transfer topic models (e.g., Latent Dirichlet Allocation (LDA) based topic models) from one language to another, without the need for metadata, embeddings, or aligned corpora. TMT enables the reuse of topic models across languages, making it especially suitable for scenarios where large corpora in the target language are unavailable or manual translation is infeasible. Furthermore, we evaluate TMT extensively using both quantitative and qualitative methods, demonstrating that it produces semantically coherent and consistent topic translations.

Method: Two main strategies: 1) Using Language Models as evaluators through prompting, 2) Training encoder-based classification and regression models with fewer parameters.

Result: LM prompting achieved modest correlations with human judgments (ranked second on test set). Regression/classification models showed high correlation on validation set but performance decreased on test set due to different score ranges in annotations.

Conclusion: While smaller models can achieve reasonable performance, test set distribution differences highlight challenges in dialogue evaluation and the need for robust evaluation methods that generalize across different score distributions.

Abstract: The growing number of generative AI-based dialogue systems has made their evaluation a crucial challenge. This paper presents our contribution to this important problem through the Dialogue System Technology Challenge (DSTC-12, Track 1), where we developed models to predict dialogue-level, dimension-specific scores. Given the constraint of using relatively small models (i.e. fewer than 13 billion parameters) our work follows two main strategies: employing Language Models (LMs) as evaluators through prompting, and training encoder-based classification and regression models. Our results show that while LM prompting achieves only modest correlations with human judgments, it still ranks second on the test set, outperformed only by the baseline. The regression and classification models, with significantly fewer parameters, demonstrate high correlation for some dimensions on the validation set. Although their performance decreases on the test set, it is important to note that the test set contains annotations with significantly different score ranges for some of the dimensions with respect to the train and validation sets.

Method: Multi-Granularity Hard-negative synthesis using LLMs to create diverse negative samples, combined with Anchor Token Aware pooling that weights important tokens based on LLM patterns.

Result: Achieved state-of-the-art performance on MTEB benchmark, outperforming existing synthesis strategies with both synthetic data and public retrieval datasets.

Conclusion: The proposed MGH framework and ATA pooling method effectively improve text embedding quality through progressive curriculum learning and better token weighting.

Abstract: Text embedding models are essential for various natural language processing tasks, enabling the effective encoding of semantic information into dense vector representations. These models are typically optimized using triplets of (query, positive, negative) data pairs for contrastive learning, where the negative samples play a critical role in enhancing the model’s ability to discern subtle semantic distinctions. In this work, we introduce a Multi-Granularity Hard-negative (MGH) synthesis framework that leverages large language models (LLMs) to generate diverse negative samples with varying levels of similarity with the query. This approach facilitates a coarse-to-fine curriculum learning strategy during supervised training, allowing the embedding model to progressively learn more nuanced semantic representations. Meanwhile, we propose an Anchor Token Aware (ATA) pooling method that assigns higher weights to anchor tokens based on aggregation patterns observed in LLMs, improving text embedding accuracy without increasing model complexity. Comprehensive experiments on the MTEB benchmark demonstrate that our methods achieve state-of-the-art performance, surpassing existing synthesis strategies both with synthetic data and when combined with public retrieval datasets.

Method: Created benchmark of 5 occupational roles, tested 5 state-of-the-art TTI models with neutral vs fairness-aware prompts, annotated outputs for gender and race distribution analysis.

Result: Prompting can substantially shift demographic representations but effects are highly model-specific - some diversify effectively, others overcorrect into uniformity, some show little responsiveness.

Conclusion: Prompting shows promise but has limitations as fairness intervention, highlighting need for complementary model-level strategies alongside prompt engineering.

Abstract: Text-to-Image (TTI) models are powerful creative tools but risk amplifying harmful social biases. We frame representational societal bias assessment as an image curation and evaluation task and introduce a pilot benchmark of occupational portrayals spanning five socially salient roles (CEO, Nurse, Software Engineer, Teacher, Athlete). Using five state-of-the-art models: closed-source (DALLE 3, Gemini Imagen 4.0) and open-source (FLUX.1-dev, Stable Diffusion XL Turbo, Grok-2 Image), we compare neutral baseline prompts against fairness-aware controlled prompts designed to encourage demographic diversity. All outputs are annotated for gender (male, female) and race (Asian, Black, White), enabling structured distributional analysis. Results show that prompting can substantially shift demographic representations, but with highly model-specific effects: some systems diversify effectively, others overcorrect into unrealistic uniformity, and some show little responsiveness. These findings highlight both the promise and the limitations of prompting as a fairness intervention, underscoring the need for complementary model-level strategies. We release all code and data for transparency and reproducibility https://github.com/maximus-powers/img-gen-bias-analysis.

Method: Proposes Collaborative Contrastive Decoding (CCD) which contrasts output distributions between induced deceptive inputs and transformed neutral inputs to reduce reliance on misleading information without requiring additional training.

Result: Extensive experiments show CCD effectively mitigates fawning hallucinations and improves factuality of generated responses across various NLP tasks.

Conclusion: The proposed CCD method successfully addresses fawning hallucinations in LLMs by contrasting output distributions, enhancing factual accuracy without the need for additional model training.

Abstract: Large language models (LLMs) have demonstrated exceptional proficiency in language understanding. However, when LLMs align their outputs with deceptive and/or misleading prompts, the generated responses could deviate from the de facto information. Such observations are known as fawning hallucinations, where the model prioritizes alignment with the input’s implied perspective over accuracy and truthfulness. In this work, we analyze fawning hallucinations in various natural language processing tasks and tailor the so-termed contrastive decoding method for fawning-hallucination mitigation. Specifically, we design two paradigms to generate corresponding deceptive and/or misleading inputs for the consistent fawning hallucinations induction. Then, we propose the collaborative contrastive decoding (CCD) to handle the fawning hallucinations across different tasks in LLMs. By contrasting the deviation in output distribution between induced and transformed neutral inputs, the proposed CCD can reduce reliance on deceptive and/or misleading information without requiring additional training. Extensive experiments demonstrate that the proposed CCD can effectively mitigate fawning hallucinations and improve the factuality of the generated responses over various tasks.

Method: Proposes an agentic RAG framework with a retrieve-note-answer pipeline where models compose Supportive-Evidence Notes (SENs) - concise human-like notes preserving only relevant information. Uses Evidence Quality Reward (EQR), an entailment-based signal to evaluate whether SENs logically support final answers.

Result: Outperforms strong baselines in accuracy, generalization, and training stability. Achieves state-of-the-art results with relative F1 gains: 20% on HotpotQA (+0.093), 40% on Bamboogle (+0.151), and 91% on 2Wiki (+0.256) through denser rewards and reduced verbosity.

Conclusion: EviNote-RAG enhances robustness and efficiency in open-domain QA by guiding models toward faithful reasoning through structured evidence distillation and quality reinforcement, effectively reducing noise impact while maintaining strong performance.

Abstract: Large Language Models (LLMs) empowered with retrieval mechanisms have achieved strong progress in open-domain question answering (QA). Yet, the conventional retrieve–then–answer paradigm often suffers from two key limitations: (1) low signal-to-noise ratio in retrieved evidence, where useful information is buried under irrelevant content, and (2) error accumulation in multi-hop reasoning when incomplete or noisy passages are involved. To address these challenges, we present EviNote-RAG, an agentic RAG framework that introduces a structured retrieve–note–answer pipeline. Instead of directly reasoning over raw retrievals, the model is trained to compose Supportive-Evidence Notes (SENs), concise, human-like notes that preserve only answer-relevant information, highlight uncertainty, and explicitly state when no useful evidence exists. This distillation process is further reinforced by the Evidence Quality Reward (EQR), an entailment-based signal that evaluates whether SENs logically support the final answer. Together, SENs and EQR guide the model toward faithful and robust reasoning, while reducing the impact of noise. Experiments on in-domain and out-of-domain QA benchmarks show that EviNote-RAG consistently outperforms strong baselines in accuracy, generalization, and training stability. In particular, it achieves state-of-the-art results while enhancing robustness and efficiency, yielding relative F1 gains of 20% on HotpotQA (+0.093), 40% on Bamboogle (+0.151), and 91% on 2Wiki (+0.256) via denser rewards and reduced verbosity.

Method: Created SeLeRoSa dataset with 13,873 manually annotated sentences across multiple domains. Evaluated LLMs in zero-shot and fine-tuning settings, plus transformer-based models for satire detection.

Result: Current LLMs and transformer models show limitations in sentence-level satire detection task, indicating room for improvement.

Conclusion: The research highlights the challenges in satire detection and opens new directions for improving models’ ability to distinguish satirical content from factual reporting.

Abstract: Satire, irony, and sarcasm are techniques typically used to express humor and critique, rather than deceive; however, they can occasionally be mistaken for factual reporting, akin to fake news. These techniques can be applied at a more granular level, allowing satirical information to be incorporated into news articles. In this paper, we introduce the first sentence-level dataset for Romanian satire detection for news articles, called SeLeRoSa. The dataset comprises 13,873 manually annotated sentences spanning various domains, including social issues, IT, science, and movies. With the rise and recent progress of large language models (LLMs) in the natural language processing literature, LLMs have demonstrated enhanced capabilities to tackle various tasks in zero-shot settings. We evaluate multiple baseline models based on LLMs in both zero-shot and fine-tuning settings, as well as baseline transformer-based models. Our findings reveal the current limitations of these models in the sentence-level satire detection task, paving the way for new research directions.

Method: Proposes supervised in-context fine-tuning (SIFT) which casts sequence labeling as constrained response generation, combining in-context learning from demonstrations with supervised fine-tuning.

Result: SIFT considerably outperforms both in-context learning and decoder-as-encoder fine-tuning baselines on standard sequence labeling tasks. Long context hinders performance but this can be mitigated by removing instructions.

Conclusion: Response-based generative task formulation is crucial for effective sequence labeling performance with LLMs, highlighting both strengths and limitations of using LLMs for sequence labeling tasks.

Abstract: Sequence labeling (SL) tasks, where labels are assigned to tokens, are abundant in NLP (e.g., named entity recognition and aspect-based sentiment analysis). Owing to the intuition that they require bidirectional context, SL tasks are commonly tackled with encoder-only models. Recent work also shows that removing the causal mask in fine-tuning enables decoder-based LLMs to become effective token classifiers. Less work, however, focused on (supervised) generative SL, a more natural setting for causal LLMs. Due to their rapid scaling, causal LLMs applied to SL are expected to outperform encoders, whose own development has stagnated. In this work, we propose supervised in-context fine-tuning (SIFT) for generative SL. SIFT casts SL tasks as constrained response generation, natural to LLMs, combining (1) in-context learning (ICL) from demonstrations with (2) supervised fine-tuning. SIFT considerably outperforms both ICL and decoder-as-encoder fine-tuning baselines on a range of standard SL tasks. We further find that although long context hinders the performance of generative SL in both ICL and SIFT, this deficiency can be mitigated by removing the instruction, as instructions are shown to be largely unnecessary for achieving strong SL performance with SIFT. Our findings highlight strengths and limitations of SL with LLMs, underscoring the importance of a response-based generative task formulation for effective SL performance.

Method: Hybrid framework combining structured prompts with multilingual variants, medical synonyms, UMLS-derived definitions, and LoRA-based fine-tuning on a parallel corpus of 2,999 English-Spanish medical articles, evaluated on four open-source LLMs.

Result: Significant translation quality improvements across all models, with Phi-4 achieving BLEU 44.23, chrF++ 28.91, and COMET 0.863, outperforming GPT-4o and GPT-4o-mini baselines.

Conclusion: Structured knowledge integration through MedCOD framework effectively enhances LLM performance for medical translation, with both prompting and adaptation contributing independently to performance gains.

Abstract: We present MedCOD (Medical Chain-of-Dictionary), a hybrid framework designed to improve English-to-Spanish medical translation by integrating domain-specific structured knowledge into large language models (LLMs). MedCOD integrates domain-specific knowledge from both the Unified Medical Language System (UMLS) and the LLM-as-Knowledge-Base (LLM-KB) paradigm to enhance structured prompting and fine-tuning. We constructed a parallel corpus of 2,999 English-Spanish MedlinePlus articles and a 100-sentence test set annotated with structured medical contexts. Four open-source LLMs (Phi-4, Qwen2.5-14B, Qwen2.5-7B, and LLaMA-3.1-8B) were evaluated using structured prompts that incorporated multilingual variants, medical synonyms, and UMLS-derived definitions, combined with LoRA-based fine-tuning. Experimental results demonstrate that MedCOD significantly improves translation quality across all models. For example, Phi-4 with MedCOD and fine-tuning achieved BLEU 44.23, chrF++ 28.91, and COMET 0.863, surpassing strong baseline models like GPT-4o and GPT-4o-mini. Ablation studies confirm that both MedCOD prompting and model adaptation independently contribute to performance gains, with their combination yielding the highest improvements. These findings highlight the potential of structured knowledge integration to enhance LLMs for medical translation tasks.

Method: Identifies two complementary forces - structure (organizing symbolic interactions) and destructure (periodic embedding resets for improved plasticity and generalization). Forms a new recipe combining these approaches.

Result: Develops a framework that bridges structured and unstructured paradigms, enabling knowledge engines to support transparent, controllable, and adaptable intelligent systems.

Conclusion: The thesis successfully establishes conceptual connections between structured and unstructured NLP paradigms through structure and destructure forces, providing a new foundation for developing general knowledge engines with improved transparency and adaptability.

Abstract: The making of knowledge engines in natural language processing has been shaped by two seemingly distinct paradigms: one grounded in structure, the other driven by massively available unstructured data. The structured paradigm leverages predefined symbolic interactions, such as knowledge graphs, as priors and designs models to capture them. In contrast, the unstructured paradigm centers on scaling transformer architectures with increasingly vast data and model sizes, as seen in modern large language models. Despite their divergence, this thesis seeks to establish conceptual connections bridging these paradigms. Two complementary forces, structure and destructure, emerge across both paradigms: structure organizes seen symbolic interactions, while destructure, through periodic embedding resets, improves model plasticity and generalization to unseen scenarios. These connections form a new recipe for developing general knowledge engines that can support transparent, controllable, and adaptable intelligent systems.

Method: Ranked Preference Reinforcement Optimization (RPRO) framework using groupwise ranking optimization based on Bradley-Terry model, KL-divergence regularization, task-adaptive reasoning templates, and probabilistic evaluation to align with clinical workflows.

Result: Consistent improvements on PubMedQA and MedQA-USMLE benchmarks; 1.1B parameter model outperforms larger 7B-13B models including medical-specialized variants.

Conclusion: Combining preference optimization with quality-driven refinement provides a scalable and effective approach for building clinically reliable medical LLMs.

Abstract: Medical question answering requires advanced reasoning that integrates domain knowledge with logical inference. However, existing large language models (LLMs) often generate reasoning chains that lack factual accuracy and clinical reliability. We propose Ranked Preference Reinforcement Optimization (RPRO), a novel framework that uniquely combines reinforcement learning with preference-driven reasoning refinement to enhance clinical chain-of-thought (CoT) performance. RPRO differentiates itself from prior approaches by employing task-adaptive reasoning templates and a probabilistic evaluation mechanism that aligns outputs with established clinical workflows, while automatically identifying and correcting low-quality reasoning chains. Unlike traditional pairwise preference methods, RPRO introduces a groupwise ranking optimization based on the Bradley-Terry model and incorporates KL-divergence regularization for stable training. Experiments on PubMedQA and MedQA-USMLE show consistent improvements over strong baselines. Remarkably, our 1.1B parameter model outperforms much larger 7B-13B models, including medical-specialized variants. These findings demonstrate that combining preference optimization with quality-driven refinement offers a scalable and effective approach to building more reliable, clinically grounded medical LLMs.

Method: The study applies standard supervised machine learning classifiers on labeled text documents, including an Artificial Neural Network model using Back Propagation Network, and compares their performance using benchmark approaches.

Result: Experimental analysis on real data reveals which classification models perform best in terms of accuracy for text classification tasks.

Conclusion: The research provides insights into the effectiveness of different supervised learning techniques for text classification, helping identify optimal models for various text classification applications.

Abstract: The demand for text classification is growing significantly in web searching, data mining, web ranking, recommendation systems, and so many other fields of information and technology. This paper illustrates the text classification process on different datasets using some standard supervised machine learning techniques. Text documents can be classified through various kinds of classifiers. Labeled text documents are used to classify the text in supervised classifications. This paper applies these classifiers on different kinds of labeled documents and measures the accuracy of the classifiers. An Artificial Neural Network (ANN) model using Back Propagation Network (BPN) is used with several other models to create an independent platform for labeled and supervised text classification process. An existing benchmark approach is used to analyze the performance of classification using labeled documents. Experimental analysis on real data reveals which model works well in terms of classification accuracy.

Method: Construct word graphs from Bangla text where words are vertices and relationships are edges, apply preprocessing steps, then use various graph-based ranking algorithms to calculate word importance.

Result: Experimental analysis on real data shows the accuracy of different ranking algorithms measured by F1 score, providing comparative performance evaluation.

Conclusion: The research successfully demonstrates a complete procedure for Bangla word ranking using graph-based approaches, offering valuable insights for text summarization and information retrieval in Bangla language processing.

Abstract: Ranking words is an important way to summarize a text or to retrieve information. A word graph is a way to represent the words of a sentence or a text as the vertices of a graph and to show the relationship among the words. It is also useful to determine the relative importance of a word among the words in the word-graph. In this research, the ranking of Bangla words are calculated, representing Bangla words from a text in a word graph using various graph based ranking algorithms. There is a lack of a standard Bangla word database. In this research, the Indian Language POS-tag Corpora is used, which has a rich collection of Bangla words in the form of sentences with their parts of speech tags. For applying a word graph to various graph based ranking algorithms, several standard procedures are applied. The preprocessing steps are done in every word graph and then applied to graph based ranking algorithms to make a comparison among these algorithms. This paper illustrate the entire procedure of calculating the ranking of Bangla words, including the construction of the word graph from text. Experimental result analysis on real data reveals the accuracy of each ranking algorithm in terms of F1 measure.

Method: Created TaarofBench with 450 role-play scenarios across 12 social topics, validated by native speakers. Evaluated 5 frontier LLMs, conducted human study with 33 participants, and used supervised fine-tuning and Direct Preference Optimization for improvement.

Result: LLMs showed substantial cultural competence gaps with accuracy 40-48% below native speakers. Performance varied by topic, improved with Persian prompts, and showed gender asymmetries. Fine-tuning achieved 21.8-42.3% improvement in cultural alignment.

Conclusion: This work establishes a foundation for developing culturally aware LLMs that can better navigate complex social interactions, highlighting the limitations of Western politeness frameworks for non-Western cultural norms.

Abstract: Large language models (LLMs) struggle to navigate culturally specific communication norms, limiting their effectiveness in global contexts. We focus on Persian taarof, a social norm in Iranian interactions, which is a sophisticated system of ritual politeness that emphasizes deference, modesty, and indirectness, yet remains absent from existing cultural benchmarks. We introduce TaarofBench, the first benchmark for evaluating LLM understanding of taarof, comprising 450 role-play scenarios covering 12 common social interaction topics, validated by native speakers. Our evaluation of five frontier LLMs reveals substantial gaps in cultural competence, with accuracy rates 40-48% below native speakers when taarof is culturally appropriate. Performance varies between interaction topics, improves with Persian-language prompts, and exhibits gender-based asymmetries. We also show that responses rated “polite” by standard metrics often violate taarof norms, indicating the limitations of Western politeness frameworks. Through supervised fine-tuning and Direct Preference Optimization, we achieve 21.8% and 42.3% improvement in model alignment with cultural expectations. Our human study with 33 participants (11 native Persian, 11 heritage, and 11 non-Iranian speakers) forms baselines in varying degrees of familiarity with Persian norms. This work lays the foundation for developing diverse and culturally aware LLMs, enabling applications that better navigate complex social interactions.

Method: Two-stage process: 1) assesses style of candidate responses, 2) optimizes and integrates them through instance-level fusion to reduce stylistic variation while maintaining quality.

Result: Experimental results across multiple datasets show the approach consistently outperforms baselines in both response quality and stylistic uniformity.

Conclusion: The proposed fusion-based generation method effectively addresses the gap in maintaining stylistic consistency for crisis communications, enabling more trustworthy automated responses.

Abstract: In response to the urgent need for effective communication with crisis-affected populations, automated responses driven by language models have been proposed to assist in crisis communications. A critical yet often overlooked factor is the consistency of response style, which could affect the trust of affected individuals in responders. Despite its importance, few studies have explored methods for maintaining stylistic consistency across generated responses. To address this gap, we propose a novel metric for evaluating style consistency and introduce a fusion-based generation approach grounded in this metric. Our method employs a two-stage process: it first assesses the style of candidate responses and then optimizes and integrates them at the instance level through a fusion process. This enables the generation of high-quality responses while significantly reducing stylistic variation between instances. Experimental results across multiple datasets demonstrate that our approach consistently outperforms baselines in both response quality and stylistic uniformity.

Method: Controlled-Literacy framework combining retrieval-augmented generation (RAG) with reinforcement learning (RL) to retrieve knowledge aligned with specific literacy levels and optimize counterspeech using subjective user preferences and objective readability rewards.

Result: Experiment results show Controlled-Literacy outperforms baselines by generating more accessible and user-preferred counterspeech.

Conclusion: This research contributes to more equitable public health communication by improving counterspeech accessibility and comprehension for diverse health literacy levels.

Abstract: Health misinformation spreading online poses a significant threat to public health. Researchers have explored methods for automatically generating counterspeech to health misinformation as a mitigation strategy. Existing approaches often produce uniform responses, ignoring that the health literacy level of the audience could affect the accessibility and effectiveness of counterspeech. We propose a Controlled-Literacy framework using retrieval-augmented generation (RAG) with reinforcement learning (RL) to generate tailored counterspeech adapted to different health literacy levels. In particular, we retrieve knowledge aligned with specific health literacy levels, enabling accessible and factual information to support generation. We design a reward function incorporating subjective user preferences and objective readability-based rewards to optimize counterspeech to the target health literacy level. Experiment results show that Controlled-Literacy outperforms baselines by generating more accessible and user-preferred counterspeech. This research contributes to more equitable and impactful public health communication by improving the accessibility and comprehension of counterspeech to health misinformation.

Method: Used a benchmark of 1,000 multiple-choice questions covering diverse inheritance scenarios to test models’ ability to understand inheritance context and compute share distributions prescribed by Islamic jurisprudence. Evaluated seven LLMs including o3, Gemini 2.5, ALLaM, Fanar, LLaMA, and Mistral.

Result: Significant performance gap observed: o3 and Gemini 2.5 achieved accuracies above 90%, while ALLaM, Fanar, LLaMA, and Mistral scored below 50%. Error analysis revealed recurring failure patterns including misunderstandings of inheritance scenarios, incorrect application of legal rules, and insufficient domain knowledge.

Conclusion: The study highlights limitations in LLMs’ ability to handle structured legal reasoning in Islamic jurisprudence and suggests directions for improving performance in domain-specific legal reasoning tasks.

Abstract: This paper evaluates the knowledge and reasoning capabilities of Large Language Models in Islamic inheritance law, known as ‘ilm al-mawarith. We assess the performance of seven LLMs using a benchmark of 1,000 multiple-choice questions covering diverse inheritance scenarios, designed to test models’ ability to understand the inheritance context and compute the distribution of shares prescribed by Islamic jurisprudence. The results reveal a significant performance gap: o3 and Gemini 2.5 achieved accuracies above 90%, whereas ALLaM, Fanar, LLaMA, and Mistral scored below 50%. These disparities reflect important differences in reasoning ability and domain adaptation. We conduct a detailed error analysis to identify recurring failure patterns across models, including misunderstandings of inheritance scenarios, incorrect application of legal rules, and insufficient domain knowledge. Our findings highlight limitations in handling structured legal reasoning and suggest directions for improving performance in Islamic legal reasoning. Code: https://github.com/bouchekif/inheritance_evaluation

Method: Historical text analysis, large-scale corpus study to confirm absence of perfective forms, and subjective evaluation tasks with native speakers to judge grammaticality.

Result: Perfective forms are starkly absent in modern usage and native speakers judge them as highly unnatural, confirming the paradigm gap emerged from a fundamental morphosyntactic conflict.

Conclusion: The perfective form became unstable due to conflict between construction requirements (nominative subject + invariant participle) and core grammatical rules (ergative case assignment for transitive perfectives), leading to its functional replacement and entrenchment as ungrammatical in modern grammar.

Abstract: In this paper, we document a paradigm gap in the combinatorial possibilities of verbs and aspect in Urdu: the perfective form of the -ya: kar construction (e.g. ro-ya: ki: cry-Pfv do.Pfv) is sharply ungrammatical in modern Urdu and Hindi, despite being freely attested in 19th century literature. We investigate this diachronic shift through historical text analysis, a large-scale corpus study which confirms the stark absence of perfective forms and subjective evaluation tasks with native speakers, who judge perfective examples as highly unnatural. We argue that this gap arose from a fundamental morphosyntactic conflict: the construction’s requirement for a nominative subject and an invariant participle clashes with the core grammatical rule that transitive perfective assign ergative case. This conflict rendered the perfective form unstable, and its functional replacement by other constructions allowed the gap to become entrenched in the modern grammar.

Method: Proposes DistilledPRAG with three key components: 1) Synthesizes QA pairs from single and multi-documents for cross-document reasoning, 2) Masks plaintext documents and translates them to LoRA via parameter generator while maintaining RAG document structure, 3) Trains parameter generator using synthetic QA data to match standard RAG’s hidden states and output logits.

Result: Experiments on four QA datasets show DistilledPRAG outperforms baselines in accuracy and generalizes well on out-of-distribution data.

Conclusion: DistilledPRAG successfully achieves high-efficiency parameterization while maintaining RAG-level performance, solving the critical challenge of privacy-preserving reasoning without exposing original documents.

Abstract: The current RAG system requires uploading plaintext documents to the cloud, risking private data leakage. Parametric RAG (PRAG) addresses this by encoding documents as LoRA within LLMs, enabling reasoning without exposing raw content. However, it still faces two issues: (1) PRAG demands synthesizing QA pairs and fine-tuning LLM for each individual document to create its corresponding LoRA, leading to unacceptable inference latency. (2) The performance of PRAG relies solely on synthetic QA data, lacking internal alignment with standard RAG, resulting in poor generalization on out-of-distribution(OOD) inputs. Therefore, achieving high-efficiency parameterization while maintaining RAG-level performance remains a critical challenge for privacy-preserving reasoning. In this paper, we propose DistilledPRAG, a generalizable knowledge-distilled parametric RAG model aligned with standard RAG in document structure and parameter activation. We first synthesize QA pairs from single and multi-documents to enhance cross-document reasoning. Then, we mask the plaintext documents with a special token and translate them to LoRA via a parameter generator, maintaining the standard RAG document structure. Finally, guided by synthetic QA data, we train the parameter generator to match standard RAG’s hidden states and output logits, enabling RAG-style reasoning without original documents. Experiments on four QA datasets show that DistilledPRAG outperforms baselines in accuracy and generalizes well on OOD data.

Method: REFRAG compresses, senses, and expands the context by exploiting the block-diagonal attention patterns in RAG contexts, eliminating unnecessary computations during decoding.

Result: Achieves 30.85% time-to-first-token acceleration (3.75x improvement over previous work) without perplexity loss, and extends context size by 16x while maintaining accuracy across diverse tasks.

Conclusion: REFRAG effectively addresses the latency-efficiency trade-off in RAG systems by leveraging structural sparsity in attention patterns, providing substantial speedups with no accuracy degradation.

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in leveraging extensive external knowledge to enhance responses in multi-turn and agentic applications, such as retrieval-augmented generation (RAG). However, processing long-context inputs introduces significant system latency and demands substantial memory for the key-value cache, resulting in reduced throughput and a fundamental trade-off between knowledge enrichment and system efficiency. While minimizing latency for long-context inputs is a primary objective for LLMs, we contend that RAG require specialized consideration. In RAG, much of the LLM context consists of concatenated passages from retrieval, with only a small subset directly relevant to the query. These passages often exhibit low semantic similarity due to diversity or deduplication during re-ranking, leading to block-diagonal attention patterns that differ from those in standard LLM generation tasks. Based on this observation, we argue that most computations over the RAG context during decoding are unnecessary and can be eliminated with minimal impact on performance. To this end, we propose REFRAG, an efficient decoding framework that compresses, senses, and expands to improve latency in RAG applications. By exploiting the sparsity structure, we demonstrate a 30.85 the time-to-first-token acceleration (3.75 improvement to previous work) without loss in perplexity. In addition, our optimization framework for large context enables REFRAG to extend the context size of LLMs by 16. We provide rigorous validation of REFRAG across diverse long-context tasks, including RAG, multi-turn conversations, and long document summarization, spanning a wide range of datasets. Experimental results confirm that REFRAG delivers substantial speedup with no loss in accuracy compared to LLaMA models and other state-of-the-art baselines across various context sizes.

Method: Proposed a framework for curating naturally evolved, human-edited variants of reading passages from QA benchmarks and analyzing LLM performance across semantic similarity scores that quantify alignment with pretraining content.

Result: LLM performance declines as passages diverge from pretraining versions - average accuracy on BoolQ drops by over 30% from highest to lowest similarity bins, with slopes exceeding 70 across several models.

Conclusion: Natural text evolution poses a significant challenge to LLMs’ language understanding capabilities, as performance deteriorates even when all required information remains available.

Abstract: How does the natural evolution of context paragraphs affect question answering in generative Large Language Models (LLMs)? To investigate this, we propose a framework for curating naturally evolved, human-edited variants of reading passages from contemporary QA benchmarks and for analyzing LLM performance across a range of semantic similarity scores, which quantify how closely each variant aligns with content seen during pretraining. Using this framework, we evaluate six QA datasets and eight LLMs with publicly available training data. Our experiments reveal that LLM performance declines as reading passages naturally diverge from the versions encountered during pretraining-even when the question and all necessary information remains present at inference time. For instance, average model accuracy on BoolQ drops by over 30% from the highest to lowest similarity bins, with slopes exceeding 70 across several LLMs. These findings suggest that natural text evolution poses a significant challenge to the language understanding capabilities of LLMs.

Method: Adapted a pretrained AR checkpoint to discrete diffusion framework with continuous-time weighted cross-entropy objective. Used supervised fine-tuning with random truncation and padding penalty, followed by reinforcement learning with verifiable rewards on curated prompt sets.

Result: Achieved 21.4% pass@1 on LiveCodeBench (2410-2505) and demonstrated competitive performance on HumanEval, MBPP, BigCodeBench, and CRUXEval benchmarks.

Conclusion: Dream-Coder 7B exhibits emergent any-order generation capabilities and adapts decoding strategies based on coding tasks, showing strong performance across multiple code generation benchmarks while being released as open-source for reproducibility.

Abstract: We present Dream-Coder 7B, an open-source discrete diffusion language model for code generation that exhibits emergent any-order generation capabilities. Unlike traditional autoregressive (AR) models that decode strictly left-to-right, Dream-Coder 7B adaptively determines its decoding strategy based on the coding task: sketch-first generation for complex algorithms, left-to-right generation for straightforward completions, and interleaved reasoning generation for code understanding tasks. We adapt a pretrained AR checkpoint to a discrete diffusion frameworks with a continuous-time weighted cross-entropy objective. Our post-training recipe comprises (i) supervised fine-tuning, where we mitigate padding pathologies via random truncation and a padding penalty to improve sample efficiency and stabilize generation; and (ii) reinforcement learning with verifiable rewards over a curated high-quality prompt set drawn from open-source datasets, using a tailored reinforcement learning recipe for diffusion language models. The resulting Dream-Coder 7B Instruct attains 21.4% pass@1 on LiveCodeBench (2410–2505) and demonstrates competitive performance on HumanEval, MBPP, BigCodeBench, and CRUXEval. We release Dream-Coder-7B and Dream-Coder-7B-Instruct checkpoints, training recipes, preprocessing pipelines, and inference code to facilitate reproducibility and further research.

Method: Entity-Aligned Translation (EAT) approach using large language models with dual-translation strategy to align entities between non-Latin script languages and English, plus fine-tuning LLMs on multilingual Wikipedia data.

Result: The method aims to improve entity alignment from source to target languages for non-Latin script languages, though specific performance metrics are not provided in the abstract.

Conclusion: EAT approach addresses the structural challenges in cross-lingual NER for non-Latin script languages by leveraging LLMs and translation alignment strategies.

Abstract: Cross-lingual Named Entity Recognition (CL-NER) aims to transfer knowledge from high-resource languages to low-resource languages. However, existing zero-shot CL-NER (ZCL-NER) approaches primarily focus on Latin script language (LSL), where shared linguistic features facilitate effective knowledge transfer. In contrast, for non-Latin script language (NSL), such as Chinese and Japanese, performance often degrades due to deep structural differences. To address these challenges, we propose an entity-aligned translation (EAT) approach. Leveraging large language models (LLMs), EAT employs a dual-translation strategy to align entities between NSL and English. In addition, we fine-tune LLMs using multilingual Wikipedia data to enhance the entity alignment from source to target languages.

Method: Combines LoRA with Mixture-of-Experts design, where multiple low-rank LoRA experts specialize in different IE tasks and eras, using a task-era-aware router to dynamically allocate expert contributions.

Result: Outperforms both single-task and joint LoRA baselines, demonstrating effective leveraging of task and temporal knowledge.

Conclusion: Tea-MOELoRA provides an effective parameter-efficient solution for multi-task Chinese IE across diverse temporal domains without performance degradation.

Abstract: Chinese information extraction (IE) involves multiple tasks across diverse temporal domains, including Classical and Modern documents. Fine-tuning a single model on heterogeneous tasks and across different eras may lead to interference and reduced performance. Therefore, in this paper, we propose Tea-MOELoRA, a parameter-efficient multi-task framework that combines LoRA with a Mixture-of-Experts (MoE) design. Multiple low-rank LoRA experts specialize in different IE tasks and eras, while a task-era-aware router mechanism dynamically allocates expert contributions. Experiments show that Tea-MOELoRA outperforms both single-task and joint LoRA baselines, demonstrating its ability to leverage task and temporal knowledge effectively.

Method: Proposes hierarchical knowledge alignment through multi-task contrastive learning to align graph embeddings with natural language space, and structural instruction tuning using unified graph instruction with lightweight knowledge adapter.

Result: SAT significantly outperforms state-of-the-art methods on two KGC tasks across four benchmark datasets, with 8.7% to 29.8% improvements in link prediction.

Conclusion: The SAT framework effectively bridges the gap between natural language and graph structures while providing a unified approach for multiple KGC tasks, demonstrating substantial performance gains.

Abstract: Knowledge graph completion (KGC) aims to infer new knowledge and make predictions from knowledge graphs. Recently, large language models (LLMs) have exhibited remarkable reasoning capabilities. LLM-enhanced KGC methods primarily focus on designing task-specific instructions, achieving promising advancements. However, there are still two critical challenges. First, existing methods often ignore the inconsistent representation spaces between natural language and graph structures. Second, most approaches design separate instructions for different KGC tasks, leading to duplicate works and time-consuming processes. To address these challenges, we propose SAT, a novel framework that enhances LLMs for KGC via structure-aware alignment-tuning. Specifically, we first introduce hierarchical knowledge alignment to align graph embeddings with the natural language space through multi-task contrastive learning. Then, we propose structural instruction tuning to guide LLMs in performing structure-aware reasoning over KGs, using a unified graph instruction combined with a lightweight knowledge adapter. Experimental results on two KGC tasks across four benchmark datasets demonstrate that SAT significantly outperforms state-of-the-art methods, especially in the link prediction task with improvements ranging from 8.7% to 29.8%.

Method: Modular framework using prompt-based interaction with LLMs to generate synthetic data, supporting SFT, DPO, and GRPO objectives through four core paradigms: multi-turn dialogues, document-grounded pairs, verifiable tasks, and long-context reasoning examples.

Result: The approach enables scalable, controllable, and purpose-aligned dataset creation through templated prompting, model-agnostic architecture, and metadata-enriched outputs.

Conclusion: This framework facilitates advancing long-context capabilities in LLMs by providing a systematic way to generate high-quality synthetic training data for various long-context applications.

Abstract: The ability of large language models (LLMs) to process and reason over long textual inputs is critical for a wide range of real-world applications. However, progress in this area is significantly constrained by the absence of high-quality, diverse, and verifiable long-context datasets suitable for both training and evaluation. This work introduces a modular, extensible framework for synthetic long-context data generation via prompt-based interaction with LLMs. The framework supports multiple training and alignment objectives, including Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Group Relative Policy Optimization (GRPO). It encompasses four core generation paradigms: multi-turn conversational dialogues, document-grounded input-output pairs, verifiable instruction-response tasks, and long-context reasoning examples. Through templated prompting, a model-agnostic architecture, and metadata-enriched outputs, the proposed approach facilitates scalable, controllable, and purpose-aligned dataset creation for advancing long-context capabilities in LLMs.

Method: The authors propose a computational framework mirroring two legal mechanisms: (1) a rule refinement pipeline that minimizes interpretive disagreement by revising ambiguous rules (analogous to agency rulemaking), and (2) prompt-based interpretive constraints that reduce inconsistency in rule application (analogous to legal canons guiding judicial discretion).

Result: The framework was evaluated on a 5,000-scenario subset of the WildChat dataset, showing that both interventions significantly improve judgment consistency across a panel of reasonable interpreters.

Conclusion: This approach represents a first step toward systematically managing interpretive ambiguity, which is essential for building more robust, law-following AI systems that can handle natural language principles more consistently and reliably.

Abstract: AI systems are increasingly governed by natural language principles, yet a key challenge arising from reliance on language remains underexplored: interpretive ambiguity. As in legal systems, ambiguity arises both from how these principles are written and how they are applied. But while legal systems use institutional safeguards to manage such ambiguity, such as transparent appellate review policing interpretive constraints, AI alignment pipelines offer no comparable protections. Different interpretations of the same rule can lead to inconsistent or unstable model behavior. Drawing on legal theory, we identify key gaps in current alignment pipelines by examining how legal systems constrain ambiguity at both the rule creation and rule application steps. We then propose a computational framework that mirrors two legal mechanisms: (1) a rule refinement pipeline that minimizes interpretive disagreement by revising ambiguous rules (analogous to agency rulemaking or iterative legislative action), and (2) prompt-based interpretive constraints that reduce inconsistency in rule application (analogous to legal canons that guide judicial discretion). We evaluate our framework on a 5,000-scenario subset of the WildChat dataset and show that both interventions significantly improve judgment consistency across a panel of reasonable interpreters. Our approach offers a first step toward systematically managing interpretive ambiguity, an essential step for building more robust, law-following AI systems.

Method: A novel training and inference approach that dynamically adjusts hardware usage during inference without requiring additional fine-tuning.

Result: Achieves up to 11x speedup compared to baseline across multiple classification and text generation tasks, enabling wide range of acceleration in zero-shot manner.

Conclusion: The proposed Zero-Shot Adjustable Acceleration method provides an efficient architecture for LLM deployment with significant speed improvements without additional training overhead.

Abstract: Using Large Language Models (LLMs) in real-world applications presents significant challenges, particularly in balancing computational efficiency and performance. Optimizing acceleration after the fine-tuning phase and during inference is crucial for building an efficient architecture. This paper introduces Zero-Shot Adjustable Acceleration, a novel training and inference method that dynamically adjusts hardware usage during inference without requiring additional fine-tuning. The proposed approach is applied to newly developed models and evaluated across multiple classification and text generation tasks. Experimental results demonstrate that the method enables a wide range of acceleration in a zero-shot manner and achieves up to a 11x speedup compared to the baseline.

Method: Evaluated growth-based pretraining via transformer stacking (Stack LLM) against a standard LLM baseline across sequential fine-tuning tasks involving domain knowledge, reasoning, reading comprehension, and bias evaluation.

Result: Both models improved in domain knowledge but showed degradation in reasoning and reading comprehension. Stack LLM consistently showed less degradation, especially in reading comprehension. In bias evaluation, baseline LLM became more neutral while Stack LLM maintained steady bias ratio around 60-61%.

Conclusion: Growth-based pretraining provides modest improvements in resisting catastrophic forgetting with better retention capabilities, though trade-offs remain in handling social biases and complete forgetting prevention.

Abstract: Catastrophic forgetting is a significant challenge in continual learning, in which a model loses prior knowledge when it is fine-tuned on new tasks. This problem is particularly critical for large language models (LLMs) undergoing continual learning, as retaining performance across diverse domains is important for their general utility. In this paper, we explore model growth, a promising strategy that leverages smaller models to expedite and structure the training of larger ones for mitigating the catastrophic forgetting problem. Although growth-based pretraining, particularly via transformer stacking, has shown promise in accelerating convergence, its impact on forgetting remains under-explored. Therefore, we evaluate whether growth-based models can retain previously learned capabilities more effectively across a sequence of fine-tuning tasks involving domain knowledge, reasoning, reading comprehension, and bias. Our findings show that both models – one trained with growth (Stack LLM) and one without (LLM) – exhibit improvements in domain knowledge. However, reasoning and reading comprehension degrade over time, indicating signs of catastrophic forgetting. Stack LLM consistently shows less degradation, especially in reading comprehension, suggesting enhanced retention capabilities. Interestingly, in bias evaluation, the baseline LLM becomes progressively more neutral with continued fine-tuning, while Stack LLM maintains a steady bias ratio around 60–61%. These results indicate that growth-based pretraining may deliver modest improvements in resisting catastrophic forgetting, though trade-offs remain in handling social biases.

Method: Two-level approach: 1) Data compression through systematic filtering (distribution/quality/hybrid methods), token compression, and LLM-based text rewriting; 2) Model compression using layer similarity scoring for pruning and sparse merging to preserve capabilities.

Result: Extensive experiments on medical Q&A, financial Q&A, general Q&A, and reading comprehension datasets show the framework can select optimal LLMs while saving approximately 20-fold in training time.

Conclusion: DaMoC provides an effective framework for rapid LLM selection and fine-tuning optimization through combined data and model compression techniques, significantly reducing computational costs while maintaining performance.

Abstract: Large language models (LLMs) excel in general tasks but struggle with domain-specific ones, requiring fine-tuning with specific data. With many open-source LLMs available, selecting the best model for fine-tuning downstream tasks is challenging, primarily focusing on how to quickly identify the optimal LLM. We introduce a Data and Model Compression Framework (DaMoC) that addresses this challenge by: 1) Data Level: A systematic categorization of data filtering methodologies for LLMs is first established, classifying them into three distinct paradigms: (1) distribution-aware methods, (2) quality-aware methods, and (3) hybrid approaches considering both dimensions. Further, we enhance the density of key tokens in the text achieving token compression. Subsequently, we use an LLM to iterative rewrite the text to optimize its expression. 2) Model Level: We use layer similarity scores to assess each layer’s importance and remove those with lower importance. Then, we introduce a sparse merging paradigm to preserve as much of the original model’s capability as possible. Extensive experiments on four datasets, medical Q&A, financial Q&A, general Q&A, and reading comprehension, show that we can select the optimal LLM while saving approximately 20-fold in training time.

Method: The researchers conducted fine-grained lexical-level analysis of rationales, incrementally introduced noisy exemplars to test model balancing, and investigated prompt engineering effects on inducing slow thinking in LLMs.

Result: Three key findings: (1) Models learn reasoning structures quickly but rely heavily on pretrained priors; (2) Sufficient exemplars shift decision-making to in-context signals while misleading prompts cause instability; (3) Long Chain-of-Thought prompting improves downstream task performance.

Conclusion: Chain-of-Thought reasoning involves a complex interplay between in-context learning and pretrained knowledge, with prompt engineering capable of inducing more deliberate reasoning processes that enhance model performance.

Abstract: Chain-of-Thought reasoning has emerged as a pivotal methodology for enhancing model inference capabilities. Despite growing interest in Chain-of-Thought reasoning, its underlying mechanisms remain unclear. This paper explores the working mechanisms of Chain-of-Thought reasoning from the perspective of the dual relationship between in-context learning and pretrained priors. We first conduct a fine-grained lexical-level analysis of rationales to examine the model’s reasoning behavior. Then, by incrementally introducing noisy exemplars, we examine how the model balances pretrained priors against erroneous in-context information. Finally, we investigate whether prompt engineering can induce slow thinking in large language models. Our extensive experiments reveal three key findings: (1) The model not only quickly learns the reasoning structure at the lexical level but also grasps deeper logical reasoning patterns, yet it heavily relies on pretrained priors. (2) Providing sufficient exemplars shifts the model’s decision-making from pretrained priors to in-context signals, while misleading prompts introduce instability. (3) Long Chain-of-Thought prompting can induce the model to generate longer reasoning chains, thereby improving its performance on downstream tasks.

Method: Used language models trained on manually annotated industrial corpus following evaluative emotion approach. Analyzed statistical trends in annotations despite significant disagreement among human annotators.

Result: Language models successfully modeled the labeling process and showed that annotation variability is driven by underlying linguistic features. Models demonstrated capability to distinguish emotional situations based on evaluative criteria.

Conclusion: Evaluative approach to emotion annotation provides valuable complementary perspective to traditional methods. Language models can effectively capture and model emotional annotation patterns despite human disagreement, revealing stable statistical trends in emotional expression.

Abstract: Emotion is a crucial phenomenon in the functioning of human beings in society. However, it remains a widely open subject, particularly in its textual manifestations. This paper examines an industrial corpus manually annotated following an evaluative approach to emotion. This theoretical framework, which is currently underutilized, offers a different perspective that complements traditional approaches. Noting that the annotations we collected exhibit significant disagreement, we hypothesized that they nonetheless follow stable statistical trends. Using language models trained on these annotations, we demonstrate that it is possible to model the labeling process and that variability is driven by underlying linguistic features. Conversely, our results indicate that language models seem capable of distinguishing emotional situations based on evaluative criteria.

Method: Proposes intentionally cultural evaluation - systematically examining cultural assumptions in all evaluation aspects, characterizing culturally contingent considerations, emphasizing researcher positionality, and using HCI-inspired participatory methodologies involving communities in evaluation design.

Result: A framework for moving beyond current benchmarking practices, discovering unknown important applications, and fostering inclusive, culturally aligned NLP research through systematic examination of cultural embeddedness in evaluations.

Conclusion: Cultural evaluation of LLMs requires fundamental shift from trivia-centered paradigm to examining cultural assumptions throughout evaluation processes, with emphasis on researcher positionality and community participation for more inclusive and culturally aligned AI systems.

Abstract: The prevailing trivia-centered paradigm'' for evaluating the cultural alignment of large language models (LLMs) is increasingly inadequate as these models become more advanced and widely deployed. Existing approaches typically reduce culture to static facts or values, testing models via multiple-choice or short-answer questions that treat culture as isolated trivia. Such methods neglect the pluralistic and interactive realities of culture, and overlook how cultural assumptions permeate even ostensibly neutral’’ evaluation settings. In this position paper, we argue for \textbf{intentionally cultural evaluation}: an approach that systematically examines the cultural assumptions embedded in all aspects of evaluation, not just in explicitly cultural tasks. We systematically characterize the what, how, and circumstances by which culturally contingent considerations arise in evaluation, and emphasize the importance of researcher positionality for fostering inclusive, culturally aligned NLP research. Finally, we discuss implications and future directions for moving beyond current benchmarking practices, discovering important applications that we don’t know exist, and involving communities in evaluation design through HCI-inspired participatory methodologies.

Method: Three key innovations: 1) Structured table schema instead of verbalized tables, 2) Query-aware table zooming mechanism with column selection and entity linking, 3) Program-of-Thoughts strategy converting queries to executable code, integrated with ReAct paradigm for iterative reasoning.

Result: Achieved 19.34% accuracy improvement on DataBench dataset and 25% improvement on TableBench Fact Checking task compared to conventional PoT methods when implemented with Qwen3-8B-Instruct LLM.

Conclusion: TableZoomer framework maintains usability advantages while substantially enhancing performance and scalability across tables of varying scales, making LLMs more effective for industrial table QA applications.

Abstract: While large language models (LLMs) have shown promise in the table question answering (TQA) task through prompt engineering, they face challenges in industrial applications, including structural heterogeneity, difficulties in target data localization, and bottlenecks in complex reasoning. To address these limitations, this paper presents TableZoomer, a novel LLM-powered, programming-based agent framework. It introduces three key innovations: (1) replacing the original fully verbalized table with structured table schema to bridge the semantic gap and reduce computational complexity; (2) a query-aware table zooming mechanism that dynamically generates sub-table schema through column selection and entity linking, significantly improving target localization efficiency; and (3) a Program-of-Thoughts (PoT) strategy that transforms queries into executable code to mitigate numerical hallucination. Additionally, we integrate the reasoning workflow with the ReAct paradigm to enable iterative reasoning. Extensive experiments demonstrate that our framework maintains the usability advantages while substantially enhancing performance and scalability across tables of varying scales. When implemented with the Qwen3-8B-Instruct LLM, TableZoomer achieves accuracy improvements of 19.34% and 25% over conventional PoT methods on the large-scale DataBench dataset and the small-scale Fact Checking task of TableBench dataset, respectively.

Method: Benchmarked six PEFT techniques on Llama-3-8B-Instruct using 14 datasets across Scientific, Medical, Legal, and News domains for text summarization, exploring within-domain and cross-domain adapters.

Result: Within-domain adapters outperformed few-shot learning and larger Llama-3-70B-Instruct models. Cross-domain adapters and strategic combinations leveraged linguistic similarities for better low-resource performance.

Conclusion: PEFT techniques provide efficient domain adaptation for LLMs in low-resource settings, with within-domain adapters being most effective and cross-domain approaches offering viable alternatives.

Abstract: Large Language Models (LLMs), being generic task solvers, are versatile. However, despite the vast amount of data they are trained on, there are speculations about their adaptation capabilities to a new domain. Additionally, the simple fine-tuning of the model to incorporate knowledge of a new domain is computationally expensive and time-consuming. This becomes more challenging when the domain in question is also low-resource, and labeled data is unavailable. We leverage parameter-efficient fine-tuning techniques (PEFTs) on high-resource datasets to address these challenges to improve performance on unseen low-resource domains. Throughout our experiments, we evaluate whether intrinsic linguistic commonalities between datasets can be leveraged for efficient domain adaptation. We benchmark six PEFTs with \texttt{Llama-3-8B-Instruct} on 14 training datasets from the Scientific, Medical, Legal, and News domains for a Text Summarization task. Our experiments show that for low-resource domains, inference using Within-Domain Adapters can achieve better performance than Few-Shot as well as a much larger \texttt{Llama-3-70B-Instruct}. Lastly, in the absence of Within-Domain Adapters, we explore the concept of using Cross-Domain Adapters as well as the strategic combinations of adapters to leverage intrinsic language similarities across domains, facilitating better adaptability and performance in low-resource settings.

Method: Uses Mixture-of-Experts architecture with two novel designs: Zero-computation Experts for dynamic budget allocation and Shortcut-connected MoE for computation-communication overlap. Includes comprehensive scaling framework with hyperparameter transfer, model-growth initialization, and multi-pronged stability suite.

Result: Trained on 20+ trillion tokens in 30 days, achieves 100+ tokens per second inference at $0.70 per million output tokens. Demonstrates competitive performance among leading models with exceptional strengths in agentic tasks.

Conclusion: LongCat-Flash successfully combines scalable architectural design with infrastructure optimizations to deliver an efficient, high-performance language model with strong agentic capabilities, made available as open-source for community research.

Abstract: We introduce LongCat-Flash, a 560-billion-parameter Mixture-of-Experts (MoE) language model designed for both computational efficiency and advanced agentic capabilities. Stemming from the need for scalable efficiency, LongCat-Flash adopts two novel designs: (a) Zero-computation Experts, which enables dynamic computational budget allocation and activates 18.6B-31.3B (27B on average) per token depending on contextual demands, optimizing resource usage. (b) Shortcut-connected MoE, which enlarges the computation-communication overlap window, demonstrating notable gains in inference efficiency and throughput compared to models of a comparable scale. We develop a comprehensive scaling framework for large models that combines hyperparameter transfer, model-growth initialization, a multi-pronged stability suite, and deterministic computation to achieve stable and reproducible training. Notably, leveraging the synergy among scalable architectural design and infrastructure efforts, we complete model training on more than 20 trillion tokens within 30 days, while achieving over 100 tokens per second (TPS) for inference at a cost of $0.70 per million output tokens. To cultivate LongCat-Flash towards agentic intelligence, we conduct a large-scale pre-training on optimized mixtures, followed by targeted mid- and post-training on reasoning, code, and instructions, with further augmentation from synthetic data and tool use tasks. Comprehensive evaluations demonstrate that, as a non-thinking foundation model, LongCat-Flash delivers highly competitive performance among other leading models, with exceptional strengths in agentic tasks. The model checkpoint of LongCat-Flash is open-sourced to foster community research. LongCat Chat: https://longcat.ai Hugging Face: https://huggingface.co/meituan-longcat GitHub: https://github.com/meituan-longcat

Method: Created KoBLEX benchmark with 226 scenario-based QA instances using LLM-human pipeline. Proposed ParSeR method with parametric provisions and three-stage retrieval for legal reasoning.

Result: ParSeR outperforms strong baselines, achieving +37.91 higher F1 and +30.81 higher LF-Eval compared to standard retrieval with GPT-4o.

Conclusion: The proposed KoBLEX benchmark and ParSeR method effectively address legal reasoning evaluation and demonstrate superior performance in provision-grounded legal QA tasks.

Abstract: Large Language Models (LLM) have achieved remarkable performances in general domains and are now extending into the expert domain of law. Several benchmarks have been proposed to evaluate LLMs’ legal capabilities. However, these benchmarks fail to evaluate open-ended and provision-grounded Question Answering (QA). To address this, we introduce a Korean Benchmark for Legal EXplainable QA (KoBLEX), designed to evaluate provision-grounded, multi-hop legal reasoning. KoBLEX includes 226 scenario-based QA instances and their supporting provisions, created using a hybrid LLM-human expert pipeline. We also propose a method called Parametric provision-guided Selection Retrieval (ParSeR), which uses LLM-generated parametric provisions to guide legally grounded and reliable answers. ParSeR facilitates multi-hop reasoning on complex legal questions by generating parametric provisions and employing a three-stage sequential retrieval process. Furthermore, to better evaluate the legal fidelity of the generated answers, we propose Legal Fidelity Evaluation (LF-Eval). LF-Eval is an automatic metric that jointly considers the question, answer, and supporting provisions and shows a high correlation with human judgments. Experimental results show that ParSeR consistently outperforms strong baselines, achieving the best results across multiple LLMs. Notably, compared to standard retrieval with GPT-4o, ParSeR achieves +37.91 higher F1 and +30.81 higher LF-Eval. Further analyses reveal that ParSeR efficiently delivers consistent performance across reasoning depths, with ablations confirming the effectiveness of ParSeR.

Method: Systematic assessment of LLM learning capabilities across eight diverse card games, evaluating fine-tuning impact on gameplay data, and examining model retention of general capabilities.

Result: LLMs approach strong game AI performance through supervised fine-tuning, master multiple complex card games simultaneously (with performance benefits for similar games), but experience general capability decline that can be mitigated with general instruction data.

Conclusion: LLMs demonstrate strong learning ability and versatility in complex games, showing potential to master multiple games while maintaining general capabilities through balanced training approaches.

Abstract: Complex games have long been an important benchmark for testing the progress of artificial intelligence algorithms. AlphaGo, AlphaZero, and MuZero have defeated top human players in Go and Chess, garnering widespread societal attention towards artificial intelligence. Concurrently, large language models (LLMs) have exhibited remarkable capabilities across various tasks, raising the question of whether LLMs can achieve similar success in complex games. In this paper, we explore the potential of LLMs in mastering complex card games. We systematically assess the learning capabilities of LLMs across eight diverse card games, evaluating the impact of fine-tuning on high-quality gameplay data, and examining the models’ ability to retain general capabilities while mastering these games. Our findings indicate that: (1) LLMs can approach the performance of strong game AIs through supervised fine-tuning on high-quality data, (2) LLMs can master multiple complex card games simultaneously, with performance augmentation for games with similar rules and conflicts for dissimilar ones, and (3) LLMs experience a decline in general capabilities when mastering complex games, but this decline can be mitigated by integrating a certain amount of general instruction data. The evaluation results demonstrate strong learning ability and versatility of LLMs.

Method: Extract a reasoning vector as the parameter difference between two identically initialized models (one fine-tuned with SFT, the other with GRPO on the same dataset): v_reason = θ_GRPO - θ_SFT, then add this vector to compatible instruction-tuned models.

Result: Consistent performance improvements across diverse reasoning benchmarks: GSM8K (+4.9%), HumanEval (+4.3%), SciQ (+1.7%), and BigBenchHard (+12.3% for 1.5B model). Performance improvements persist under adversarial conditions, while subtracting the vector causes significant degradation (-11.8% on GSM8K).

Conclusion: Reasoning capabilities developed through expensive training can be extracted from open-source models and reused through simple tensor arithmetic, offering a practical way to enhance models by recycling prior computational investments.

Abstract: Large language models often require costly optimization, such as reinforcement learning, to master complex reasoning tasks. This work demonstrates that reasoning ability, once learned, can be extracted and transferred between models as a compact task vector. We source two publicly available, identically initialized Qwen2.5 models, one fine-tuned with supervised fine-tuning (SFT) and the other with group relative policy optimization (GRPO) on the same dataset. From these, we extract a reasoning vector: $v_{\text{reason}} = \theta_{\text{GRPO}} - \theta_{\text{SFT}}$. We hypothesize that this vector captures the reasoning capability instilled by reinforcement learning while factoring out shared knowledge from the SFT process. When added to compatible instruction-tuned models through simple arithmetic, this vector consistently improves performance across diverse reasoning benchmarks: GSM8K (+4.9%), HumanEval (+4.3%), SciQ (+1.7%), and BigBenchHard (+12.3% for the 1.5B model). The performance improvements persist under adversarial conditions. Conversely, subtracting the vector causes significant performance degradation (-11.8% on GSM8K), demonstrating the vector’s strong contribution to the model’s reasoning abilities. This work shows how reasoning capabilities, typically developed through expensive training, can be extracted from existing open-source models and reused through simple tensor arithmetic, offering a practical way to enhance models by recycling prior computational investments.

Method: Built as an AI agent system using Large Language Models with specialized tools: compares posts with hate speech examples, uses BERT classifier, looks up slang via Urban Dictionary, generates chain-of-thought reasoning, and checks platform guidelines.

Result: Achieves state-of-the-art performance with macro F1 score of 0.91, surpassing existing methods in hate speech detection.

Conclusion: The tool effectively supports collaboration between AI and human moderators by providing explainable hate speech detection grounded in precedent and policy, helping reduce online harms.

Abstract: Online harms are a growing problem in digital spaces, putting user safety at risk and reducing trust in social media platforms. One of the most persistent forms of harm is hate speech. To address this, we need tools that combine the speed and scale of automated systems with the judgment and insight of human moderators. These tools should not only find harmful content but also explain their decisions clearly, helping to build trust and understanding. In this paper, we present WATCHED, a chatbot designed to support content moderators in tackling hate speech. The chatbot is built as an Artificial Intelligence Agent system that uses Large Language Models along with several specialised tools. It compares new posts with real examples of hate speech and neutral content, uses a BERT-based classifier to help flag harmful messages, looks up slang and informal language using sources like Urban Dictionary, generates chain-of-thought reasoning, and checks platform guidelines to explain and support its decisions. This combination allows the chatbot not only to detect hate speech but to explain why content is considered harmful, grounded in both precedent and policy. Experimental results show that our proposed method surpasses existing state-of-the-art methods, reaching a macro F1 score of 0.91. Designed for moderators, safety teams, and researchers, the tool helps reduce online harms by supporting collaboration between AI and human oversight.

Method: Domain-agnostic framework that generates semi-synthetic datasets, performs automatic evaluation to shortlist best linking approaches, then conducts extensive human evaluation on natural text pairs using retrieval models combined with LLMs.

Result: Combining retrieval models with LLMs achieves 78% link approval from human raters, more than doubling the precision of strong retrievers alone. Applied successfully in peer review and news domains.

Conclusion: The framework enables systematic study of cross-document understanding across applications, with resulting datasets supporting tasks like media framing and peer review analysis.

Abstract: Understanding fine-grained relations between documents is crucial for many application domains. However, the study of automated assistance is limited by the lack of efficient methods to create training and evaluation datasets of cross-document links. To address this, we introduce a new domain-agnostic framework for selecting a best-performing approach and annotating cross-document links in a new domain from scratch. We first generate and validate semi-synthetic datasets of interconnected documents. This data is used to perform automatic evaluation, producing a shortlist of best-performing linking approaches. These approaches are then used in an extensive human evaluation study, yielding performance estimates on natural text pairs. We apply our framework in two distinct domains – peer review and news – and show that combining retrieval models with LLMs achieves 78% link approval from human raters, more than doubling the precision of strong retrievers alone. Our framework enables systematic study of cross-document understanding across application scenarios, and the resulting novel datasets lay foundation for numerous cross-document tasks like media framing and peer review. We make the code, data, and annotation protocols openly available.

Method: Propose generating intermediate corrected student solutions that align more closely with the original student’s solution to help improve error localization performance.

Result: Experiments on VtG and PRM800K datasets show state-of-the-art LLMs struggle with error step localization even when given reference solutions.

Conclusion: The proposed approach of generating intermediate corrected solutions shows promise for improving LLM performance on error detection in stepwise math solutions.

Abstract: Large language models (LLMs) demonstrate remarkable performance on math word problems, yet they have been shown to struggle with meta-reasoning tasks such as identifying errors in student solutions. In this work, we investigate the challenge of locating the first error step in stepwise solutions using two error reasoning datasets: VtG and PRM800K. Our experiments show that state-of-the-art LLMs struggle to locate the first error step in student solutions even when given access to the reference solution. To that end, we propose an approach that generates an intermediate corrected student solution, aligning more closely with the original student’s solution, which helps improve performance.

Method: Converts linear CoT text into interactive reasoning graphs, enabling users to visualize logical flow, identify flawed steps, and intervene by pruning incorrect paths or grafting new premises.

Result: Improves final-answer accuracy by up to 24 percentage points over non-interactive baselines on GSM8K and StrategyQA datasets, with significant gains in perceived usability and trust.

Conclusion: Vis-CoT provides a practical approach for more reliable and understandable reasoning by combining LLMs with targeted human oversight through interactive visualization and intervention.

Abstract: Large language models (LLMs) show strong reasoning via chain-of-thought (CoT) prompting, but the process is opaque, which makes verification, debugging, and control difficult in high-stakes settings. We present Vis-CoT, a human-in-the-loop framework that converts linear CoT text into an interactive reasoning graph. Users can visualize the logical flow, identify flawed steps, and intervene by pruning incorrect paths and grafting new, user-defined premises. This shifts interaction from passive observation to active collaboration, steering models toward more accurate and trustworthy conclusions. Across GSM8K and StrategyQA, Vis-CoT improves final-answer accuracy by up to 24 percentage points over non-interactive baselines. A user study also shows large gains in perceived usability and trust. Vis-CoT points to a practical path for more reliable, understandable, and collaborative reasoning by combining LLMs with targeted human oversight.

Method: Created an evaluation framework based on World Values Survey (WVS) to systematically assess LLM opinion alignment across different countries, languages, and historical periods worldwide.

Result: LLMs appropriately or over-align with only a few countries while under-aligning with most countries. Changing prompt language to match questionnaire language effectively steers LLMs to better align with corresponding country opinions. LLMs show better alignment with contemporary populations.

Conclusion: This is the first comprehensive investigation of opinion alignment in LLMs across global, language, and temporal dimensions, demonstrating that language-based steering is an effective method for improving cross-cultural opinion alignment.

Abstract: Today’s large language models (LLMs) are capable of supporting multilingual scenarios, allowing users to interact with LLMs in their native languages. When LLMs respond to subjective questions posed by users, they are expected to align with the views of specific demographic groups or historical periods, shaped by the language in which the user interacts with the model. Existing studies mainly focus on researching the opinions represented by LLMs among demographic groups in the United States or a few countries, lacking worldwide country samples and studies on human opinions in different historical periods, as well as lacking discussion on using language to steer LLMs. Moreover, they also overlook the potential influence of prompt language on the alignment of LLMs’ opinions. In this study, our goal is to fill these gaps. To this end, we create an evaluation framework based on the World Values Survey (WVS) to systematically assess the alignment of LLMs with human opinions across different countries, languages, and historical periods around the world. We find that LLMs appropriately or over-align the opinions with only a few countries while under-aligning the opinions with most countries. Furthermore, changing the language of the prompt to match the language used in the questionnaire can effectively steer LLMs to align with the opinions of the corresponding country more effectively than existing steering methods. At the same time, LLMs are more aligned with the opinions of the contemporary population. To our knowledge, our study is the first comprehensive investigation of the topic of opinion alignment in LLMs across global, language, and temporal dimensions. Our code and data are publicly available at https://github.com/nlply/global-opinion-alignment.

Method: Learns lightweight calibration head with temperature scaling and proper scoring, uses black-box features for API-only models, employs conformal risk control for distribution-free guarantees, and aligns confidence with semantic fidelity using atomic factuality scores.

Result: Consistent improvements in calibration metrics, lower area under risk-coverage curve, higher coverage at fixed risk compared to baseline methods across short-form QA, code generation, and retrieval-augmented long-form QA tasks.

Conclusion: UniCR provides a portable evidence-to-probability-to-decision framework that improves trustworthiness without fine-tuning base models and remains valid under distribution shift, with evidence contradiction, semantic dispersion, and tool inconsistency being key refusal drivers.

Abstract: Deployed language models must decide not only what to answer but also when not to answer. We present UniCR, a unified framework that turns heterogeneous uncertainty evidence including sequence likelihoods, self-consistency dispersion, retrieval compatibility, and tool or verifier feedback into a calibrated probability of correctness and then enforces a user-specified error budget via principled refusal. UniCR learns a lightweight calibration head with temperature scaling and proper scoring, supports API-only models through black-box features, and offers distribution-free guarantees using conformal risk control. For long-form generation, we align confidence with semantic fidelity by supervising on atomic factuality scores derived from retrieved evidence, reducing confident hallucinations while preserving coverage. Experiments on short-form QA, code generation with execution tests, and retrieval-augmented long-form QA show consistent improvements in calibration metrics, lower area under the risk-coverage curve, and higher coverage at fixed risk compared to entropy or logit thresholds, post-hoc calibrators, and end-to-end selective baselines. Analyses reveal that evidence contradiction, semantic dispersion, and tool inconsistency are the dominant drivers of abstention, yielding informative user-facing refusal messages. The result is a portable recipe of evidence fusion to calibrated probability to risk-controlled decision that improves trustworthiness without fine-tuning the base model and remains valid under distribution shift.

Method: End-to-end reasoning-chain framework with four structured stages: fact acknowledgment, relevance determination, selective application, and final reasoning. Trained on MQuAKE-CF dataset with up to four irrelevant facts.

Result: Achieves 90.2% multi-hop QA accuracy on Qwen2.5-7B, suffers only 6.3% drop under heavy distraction and <1% when answers are leaked. Demonstrates resilience and efficiency.

Conclusion: Reason-KE establishes a new state-of-the-art for reliable LLM knowledge updates, providing robust performance in noisy, multi-hop conditions through structured reasoning chains.

Abstract: Large language models (LLMs) encode vast amounts of world knowledge but remain static once trained, making the timely integration of emerging facts prohibitively expensive via full retraining. Knowledge-editing techniques have thus emerged to inject or overwrite specific facts into LLMs, yet they either over-rely on superficial cues or incur complex, iterative pipelines that collapse under noisy, multi-hop conditions. We introduce Reason-KE, an end-to-end reasoning-chain-based editing framework that steers a pretrained LLM through four structured stages-fact acknowledgment, relevance determination, selective application, and final reasoning-to filter distractors in a single pass. Trained on MQuAKE-CF with up to four irrelevant facts, Reason-KE elevates Qwen2.5-7B’s multi-hop QA accuracy to 90.2% while suffering merely a 6.3% drop under heavy distraction and <1% when answers are leaked. Our quantitative analysis confirms Reason-KE’s resilience and efficiency, establishing a new state-of-the-art for reliable LLM knowledge updates.

Method: The study examines RALM calibration across different knowledge states, investigates refusal post-training methods (Refusal-aware Instruction Tuning and In-Context Fine-tuning), and develops a simple refusal method for post-trained models.

Result: RALMs show significant over-refusal behavior. In-context fine-tuning mitigates over-refusal while R-tuning magnifies it, but refusal ability may conflict with answer quality. The researchers developed an effective refusal method to improve overall answer quality.

Conclusion: The study provides comprehensive understanding of factors influencing RALM systems, revealing over-refusal issues and developing methods to balance refusal capability with answer quality.

Abstract: Existing Large Language Models (LLMs) occasionally generate plausible yet factually incorrect responses, known as hallucinations. Researchers are primarily using two approaches to mitigate hallucinations, namely Retrieval Augmented Language Models (RALMs) and refusal post-training. However, current research predominantly emphasizes their individual effectiveness while overlooking the evaluation of the refusal capability of RALMs. In this study, we ask the fundamental question: Do RALMs know when they don’t know? Specifically, we ask three questions. First, are RALMs well-calibrated regarding different internal and external knowledge states? We examine the influence of various factors. Contrary to expectations, we find that LLMs exhibit significant \textbf{over-refusal} behavior. Then, how does refusal post-training affect the over-refusal issue? We investigate the Refusal-aware Instruction Tuning and In-Context Fine-tuning methods. Our results show that the over-refusal problem is mitigated by In-context fine-tuning. but magnified by R-tuning. However, we also find that the refusal ability may conflict with the quality of the answer. Finally, we develop a simple yet effective refusal method for refusal post-trained models to improve their overall answer quality in terms of refusal and correct answers. Our study provides a more comprehensive understanding of the influence of important factors on RALM systems.

Method: Five-phase modular design: initial retrieval, relevance verification, fallback retrieval, context prioritization, and token budgeting for fine-grained control.

Result: 57% context reduction on Wikipedia dataset and 75% reduction on HotpotQA dataset compared to standard RAG, with improved accuracy.

Conclusion: MeVe provides a scalable framework for reliable LLM applications through refined context distillation and better factual grounding.

Abstract: Retrieval-Augmented Generation (RAG) systems typically face constraints because of their inherent mechanism: a simple top-k semantic search [1]. The approach often leads to the incorporation of irrelevant or redundant information in the context, degrading performance and efficiency [10][11]. This paper presents MeVe, a novel modular architecture intended for Memory Verification and smart context composition. MeVe rethinks the RAG paradigm by proposing a five-phase modular design that distinctly breaks down the retrieval and context composition process into distinct, auditable, and independently tunable phases: initial retrieval, relevance verification, fallback retrieval, context prioritization, and token budgeting. This architecture enables fine-grained control of what knowledge is made available to an LLM, enabling task-dependent filtering and adaptation. We release a reference implementation of MeVe as a proof of concept and evaluate its performance on knowledge-heavy QA tasks over a subset of English Wikipedia [22]. Our results demonstrate that by actively verifying information before composition, MeVe significantly improves context efficiency, achieving a 57% reduction on the Wikipedia dataset and a 75% reduction on the more complex HotpotQA dataset compared to standard RAG implementations [25]. This work provides a framework for more scalable and reliable LLM applications. By refining and distilling contextual information, MeVe offers a path toward better grounding and more accurate factual support [16].

Method: Used BERTopic for thematic modeling and cTF-IDF weighting alongside word frequency analysis to compare union discourses on key CU features like coalition-building and grassroots engagement.

Result: Significant differences found: Unite Community showed stronger alignment with outward-facing social justice themes, while B&S focused on internal administration and traditional servicing model. Modern NLP techniques proved effective for historical analysis.

Conclusion: Both unions engage with community themes but have fundamentally different engagement models, challenging assumptions about continuity and universality of community unionism across time and sectors.

Abstract: This paper presents a comparative analysis of community unionism (CU) in two distinct historical and organizational contexts: the National Boot and Shoe Union (B&S) in the 1920s and Unite Community in the 2010s–2020s. Using BERTopic for thematic modeling and cTF-IDF weighting, alongside word frequency analysis, the study examines the extent to which each union’s discourse aligns with key features of CU – such as coalition-building, grassroots engagement, and action beyond the workplace. The results reveal significant differences in thematic focus and discursive coherence. While Unite Community demonstrates stronger alignment with outward-facing, social justice-oriented themes, the B&S corpus emphasizes internal administration, industrial relations, and member services – reflecting a more traditional, servicing-oriented union model. The analysis also highlights methodological insights, demonstrating how modern NLP techniques can enhance the study of historical labor archives. Ultimately, the findings suggest that while both unions engage with community-related themes, their underlying models of engagement diverge significantly, challenging assumptions about the continuity and universality of community unionism across time and sector.

Method: Proposes Causal Attention Tuning (CAT) with an automated pipeline that leverages human priors to generate token-level causal signals and introduces Re-Attention mechanism to guide training, helping models focus on causal structures while mitigating attention noise and biases.

Result: Experimental results on the proposed Spurious Token Game benchmark and multiple downstream tasks show that CAT effectively leverages causal knowledge for prediction and remains robust in OOD scenarios.

Conclusion: CAT successfully addresses LLMs’ tendency to rely on spurious correlations by injecting causal knowledge into attention mechanisms, demonstrating improved robustness and performance in out-of-distribution settings.

Abstract: Large Language Models (LLMs) have achieved remarkable success across various domains. However, a fundamental question remains: Can LLMs effectively utilize causal knowledge for prediction and generation? Through empirical studies, we find that LLMs trained directly on large-scale data often capture spurious correlations rather than true causal relationships, leading to suboptimal performance, especially in out-of-distribution (OOD) scenarios. To address this challenge, we propose Causal Attention Tuning (CAT), a novel approach that injects fine-grained causal knowledge into the attention mechanism. We propose an automated pipeline that leverages human priors to automatically generate token-level causal signals and introduce the Re-Attention mechanism to guide training, helping the model focus on causal structures while mitigating noise and biases in attention scores. Experimental results on our proposed Spurious Token Game (STG) benchmark and multiple downstream tasks demonstrate that our approach effectively leverages causal knowledge for prediction and remains robust in OOD scenarios. Implementation details can be found at https://github.com/Kairong-Han/CAT.

Method: Convert API documentation into structured API graphs capturing API dependencies, create In-N-Out expert-annotated dataset from real-world API benchmarks, and use this for tool retrieval and multi-tool query generation.

Result: Significant performance improvement on both tool retrieval and multi-tool query generation, nearly doubling LLM performance compared to using documentation alone. Fine-tuned models close 90% of the performance gap.

Conclusion: Explicit API graphs show promise for tool agents, and In-N-Out serves as a valuable resource for helping models learn API documentation comprehension and parameter relationships.

Abstract: Tool agents – LLM-based systems that interact with external APIs – offer a way to execute real-world tasks. However, as tasks become increasingly complex, these agents struggle to identify and call the correct APIs in the proper order. To tackle this problem, we investigate converting API documentation into a structured API graph that captures API dependencies and leveraging it for multi-tool queries that require compositional API calls. To support this, we introduce In-N-Out, the first expert-annotated dataset of API graphs built from two real-world API benchmarks and their documentation. Using In-N-Out significantly improves performance on both tool retrieval and multi-tool query generation, nearly doubling that of LLMs using documentation alone. Moreover, graphs generated by models fine-tuned on In-N-Out close 90% of this gap, showing that our dataset helps models learn to comprehend API documentation and parameter relationships. Our findings highlight the promise of using explicit API graphs for tool agents and the utility of In-N-Out as a valuable resource. We will release the dataset and code publicly.

Method: Extracts internal beliefs from multiple intermediate layers during self-evaluation, aggregates these layer-wise beliefs, and calculates expectation over the confidence score distribution to produce refined confidence scores.

Result: Extensive experiments show EAGLE significantly improves calibration performance over existing baselines across diverse datasets and LLMs.

Conclusion: EAGLE effectively leverages internal model states to provide more faithful confidence estimation, enhancing reliability and safety of LLM deployments.

Abstract: Large Language Models (LLMs) have achieved remarkable success across a wide range of natural language tasks, but often exhibit overconfidence and generate plausible yet incorrect answers. This overconfidence, especially in models undergone Reinforcement Learning from Human Feedback (RLHF), poses significant challenges for reliable uncertainty estimation and safe deployment. In this paper, we propose EAGLE (Expectation of AGgregated internaL bEief), a novel self-evaluation-based calibration method that leverages the internal hidden states of LLMs to derive more accurate confidence scores. Instead of relying on the model’s final output, our approach extracts internal beliefs from multiple intermediate layers during self-evaluation. By aggregating these layer-wise beliefs and calculating the expectation over the resulting confidence score distribution, EAGLE produces a refined confidence score that more faithfully reflects the model’s internal certainty. Extensive experiments on diverse datasets and LLMs demonstrate that EAGLE significantly improves calibration performance over existing baselines. We also provide an in-depth analysis of EAGLE, including a layer-wise examination of uncertainty patterns, a study of the impact of self-evaluation prompts, and an analysis of the effect of self-evaluation score range.

Method: Computed sentence embeddings using three approaches: averaged token embeddings, [CLS] token embeddings, and random token embeddings. Analyzed correlation between embedding distances and task performance.

Result: Cosine similarity captures only shallow commonalities/differences and is not predictive of performance on specific tasks.

Conclusion: Linguistic information is encoded in weighted combinations of dimensions rather than reflected in the geometry of the sentence embedding space.

Abstract: Transformer models learn to encode and decode an input text, and produce contextual token embeddings as a side-effect. The mapping from language into the embedding space maps words expressing similar concepts onto points that are close in the space. In practice, the reverse implication is also assumed: words corresponding to close points in this space are similar or related, those that are further are not. Does closeness in the embedding space extend to shared properties for sentence embeddings? We present an investigation of sentence embeddings and show that the geometry of their embedding space is not predictive of their relative performances on a variety of tasks. We compute sentence embeddings in three ways: as averaged token embeddings, as the embedding of the special [CLS] token, and as the embedding of a random token from the sentence. We explore whether there is a correlation between the distance between sentence embedding variations and their performance on linguistic tasks, and whether despite their distances, they do encode the same information in the same manner. The results show that the cosine similarity – which treats dimensions shallowly – captures (shallow) commonalities or differences between sentence embeddings, which are not predictive of their performance on specific tasks. Linguistic information is rather encoded in weighted combinations of different dimensions, which are not reflected in the geometry of the sentence embedding space.

Method: Constructed a high-quality dataset with 800 human-written poems by professional poets and 41,600 poems generated by four mainstream LLMs, then systematically evaluated six detectors on this dataset.

Result: Experimental results show current detectors cannot reliably detect modern Chinese poems generated by LLMs, with intrinsic qualities (especially style) being the most difficult features to detect.

Conclusion: The proposed benchmark is effective and necessary, laying a foundation for future detection of AI-generated poetry, as current detection tools are inadequate for modern Chinese poetry.

Abstract: The rapid development of advanced large language models (LLMs) has made AI-generated text indistinguishable from human-written text. Previous work on detecting AI-generated text has made effective progress, but has not involved modern Chinese poetry. Due to the distinctive characteristics of modern Chinese poetry, it is difficult to identify whether a poem originated from humans or AI. The proliferation of AI-generated modern Chinese poetry has significantly disrupted the poetry ecosystem. Based on the urgency of identifying AI-generated poetry in the real Chinese world, this paper proposes a novel benchmark for detecting LLMs-generated modern Chinese poetry. We first construct a high-quality dataset, which includes both 800 poems written by six professional poets and 41,600 poems generated by four mainstream LLMs. Subsequently, we conduct systematic performance assessments of six detectors on this dataset. Experimental results demonstrate that current detectors cannot be used as reliable tools to detect modern Chinese poems generated by LLMs. The most difficult poetic features to detect are intrinsic qualities, especially style. The detection results verify the effectiveness and necessity of our proposed benchmark. Our work lays a foundation for future detection of AI-generated poetry.

Method: TransGAT integrates fine-tuned Transformer models (BERT, RoBERTa, DeBERTaV3) with Graph Attention Networks (GAT) for two-stream predictions. First stream generates essay-level predictions, second stream applies GAT to Transformer token embeddings with syntactic dependency edges. Predictions are fused for final analytic scores.

Result: Experiments on ELLIPSE dataset show TransGAT outperforms baseline models, achieving average Quadratic Weighted Kappa (QWK) of 0.854 across all analytic scoring dimensions.

Conclusion: TransGAT demonstrates the potential of combining contextual Transformers with relational GNNs to advance automated essay scoring systems through improved analytic scoring capabilities.

Abstract: Essay writing is a critical component of student assessment, yet manual scoring is labor-intensive and inconsistent. Automated Essay Scoring (AES) offers a promising alternative, but current approaches face limitations. Recent studies have incorporated Graph Neural Networks (GNNs) into AES using static word embeddings that fail to capture contextual meaning, especially for polysemous words. Additionally, many methods rely on holistic scoring, overlooking specific writing aspects such as grammar, vocabulary, and cohesion. To address these challenges, this study proposes TransGAT, a novel approach that integrates fine-tuned Transformer models with GNNs for analytic scoring. TransGAT combines the contextual understanding of Transformers with the relational modeling strength of Graph Attention Networks (GAT). It performs two-stream predictions by pairing each fine-tuned Transformer (BERT, RoBERTa, and DeBERTaV3) with a separate GAT. In each pair, the first stream generates essay-level predictions, while the second applies GAT to Transformer token embeddings, with edges constructed from syntactic dependencies. The model then fuses predictions from both streams to produce the final analytic score. Experiments on the ELLIPSE dataset show that TransGAT outperforms baseline models, achieving an average Quadratic Weighted Kappa (QWK) of 0.854 across all analytic scoring dimensions. These findings highlight the potential of TransGAT to advance AES systems.

Method: Uses Needleman-Wunsch algorithm on phonetic transcriptions, implemented in Rust with parallel CPU/GPU processing using CUDA and cudarc library. Constructs fully-connected graph with similarity-weighted edges and applies clustering algorithms.

Result: Demonstrated feasibility and effectiveness in analyzing phonetic language structure. GPU implementation achieved significant performance improvements for large datasets.

Conclusion: The proposed method is effective for phonetic similarity analysis and can be easily expanded to other languages beyond the tested implementation.

Abstract: We present a method to calculate the similarity between words based on their phonetic transcription (their pronunciation) using the Needleman-Wunsch algorithm. We implement this algorithm in Rust and parallelize it on both CPU and GPU to handle large datasets efficiently. The GPU implementation leverages CUDA and the cudarc Rust library to achieve significant performance improvements. We validate our approach by constructing a fully-connected graph where nodes represent words and edges have weights according to the similarity between the words. This graph is then analyzed using clustering algorithms to identify groups of phonetically similar words. Our results demonstrate the feasibility and effectiveness of the proposed method in analyzing the phonetic structure of languages. It might be easily expanded to other languages.

Method: Graph-based Intent-Semantic Joint Modeling (InSide) that models deception clues from both semantic and intent signals via graph-based joint learning. It reformulates signals into heterogeneous graph structures, uses entity guidance for long-range context interaction, and employs coarse-to-fine intent modeling. Includes dynamic pathway-based graph alignment strategy for better semantics-intent alignment.

Result: Extensive experiments on four benchmark datasets demonstrate the superiority of the proposed InSide compared to state-of-the-art methods.

Conclusion: Incorporating news intent alongside semantic clues through graph-based joint modeling provides a more robust approach to fake news detection that can better handle evolving news landscapes and avoid surface pattern limitations.

Abstract: Fake news detection is an important and challenging task for defending online information integrity. Existing state-of-the-art approaches typically extract news semantic clues, such as writing patterns that include emotional words, stylistic features, etc. However, detectors tuned solely to such semantic clues can easily fall into surface detection patterns, which can shift rapidly in dynamic environments, leading to limited performance in the evolving news landscape. To address this issue, this paper investigates a novel perspective by incorporating news intent into fake news detection, bridging intents and semantics together. The core insight is that by considering news intents, one can deeply understand the inherent thoughts behind news deception, rather than the surface patterns within words alone. To achieve this goal, we propose Graph-based Intent-Semantic Joint Modeling (InSide) for fake news detection, which models deception clues from both semantic and intent signals via graph-based joint learning. Specifically, InSide reformulates news semantic and intent signals into heterogeneous graph structures, enabling long-range context interaction through entity guidance and capturing both holistic and implementation-level intent via coarse-to-fine intent modeling. To achieve better alignment between semantics and intents, we further develop a dynamic pathway-based graph alignment strategy for effective message passing and aggregation across these signals by establishing a common space. Extensive experiments on four benchmark datasets demonstrate the superiority of the proposed InSide compared to state-of-the-art methods.

Method: Developed a character-level PLM trained on isolated words from diverse sources including YouTube comments, French/English/Berber Wikipedia, and Tatoeba project, covering multiple scripts and linguistic varieties.

Result: The model robustly encodes morphological patterns without depending on token boundaries or standardized orthography, demonstrating effectiveness for morphologically rich, low-resource dialects.

Conclusion: Character-level modeling shows strong potential for handling complex dialects like Algerian, providing a foundation for more inclusive and adaptable NLP systems that can better serve under-represented languages.

Abstract: Pre-trained language models (PLMs) have substantially advanced natural language processing by providing context-sensitive text representations. However, the Algerian dialect remains under-represented, with few dedicated models available. Processing this dialect is challenging due to its complex morphology, frequent code-switching, multiple scripts, and strong lexical influences from other languages. These characteristics complicate tokenization and reduce the effectiveness of conventional word- or subword-level approaches. To address this gap, we introduce chDzDT, a character-level pre-trained language model tailored for Algerian morphology. Unlike conventional PLMs that rely on token sequences, chDzDT is trained on isolated words. This design allows the model to encode morphological patterns robustly, without depending on token boundaries or standardized orthography. The training corpus draws from diverse sources, including YouTube comments, French, English, and Berber Wikipedia, as well as the Tatoeba project. It covers multiple scripts and linguistic varieties, resulting in a substantial pre-training workload. Our contributions are threefold: (i) a detailed morphological analysis of Algerian dialect using YouTube comments; (ii) the construction of a multilingual Algerian lexicon dataset; and (iii) the development and extensive evaluation of a character-level PLM as a morphology-focused encoder for downstream tasks. The proposed approach demonstrates the potential of character-level modeling for morphologically rich, low-resource dialects and lays a foundation for more inclusive and adaptable NLP systems.

Method: Systematically evaluated 7 LLMs across 6 benchmarks using 12 diverse prompt templates, comparing heuristic evaluation methods with LLM-as-a-Judge evaluations.

Result: Found that heuristic methods (log-likelihood scoring, rigid answer matching) overlook semantically correct responses, while LLM-as-a-Judge shows substantial reduction in performance variance and higher correlation in model rankings.

Conclusion: Modern LLMs are more robust to prompt templates than previously believed, and prompt sensitivity may be more an artifact of evaluation than a flaw in the models.

Abstract: Prompt sensitivity, referring to the phenomenon where paraphrasing (i.e., repeating something written or spoken using different words) leads to significant changes in large language model (LLM) performance, has been widely accepted as a core limitation of LLMs. In this work, we revisit this issue and ask: Is the widely reported high prompt sensitivity truly an inherent weakness of LLMs, or is it largely an artifact of evaluation processes? To answer this question, we systematically evaluate 7 LLMs (e.g., GPT and Gemini family) across 6 benchmarks, including both multiple-choice and open-ended tasks on 12 diverse prompt templates. We find that much of the prompt sensitivity stems from heuristic evaluation methods, including log-likelihood scoring and rigid answer matching, which often overlook semantically correct responses expressed through alternative phrasings, such as synonyms or paraphrases. When we adopt LLM-as-a-Judge evaluations, we observe a substantial reduction in performance variance and a consistently higher correlation in model rankings across prompts. Our findings suggest that modern LLMs are more robust to prompt templates than previously believed, and that prompt sensitivity may be more an artifact of evaluation than a flaw in the models.

Method: Position paper reviewing emerging evidence and evaluating AI interviewers across two dimensions: input/output performance (speech recognition, recording, emotion handling) and verbal reasoning (probing, clarifying, branching logic).

Result: AI interviewers already exceed IVR capabilities for both quantitative and qualitative data collection, but face challenges including real-time transcription error rates, limited emotion detection abilities, and uneven follow-up quality.

Conclusion: The utility and adoption of current AI interviewer technology may be context-dependent for qualitative data collection efforts, indicating that while promising, these systems still have limitations that need addressing.

Abstract: Transformer-based Large Language Models (LLMs) have paved the way for “AI interviewers” that can administer voice-based surveys with respondents in real-time. This position paper reviews emerging evidence to understand when such AI interviewing systems are fit for purpose for collecting data within quantitative and qualitative research contexts. We evaluate the capabilities of AI interviewers as well as current Interactive Voice Response (IVR) systems across two dimensions: input/output performance (i.e., speech recognition, answer recording, emotion handling) and verbal reasoning (i.e., ability to probe, clarify, and handle branching logic). Field studies suggest that AI interviewers already exceed IVR capabilities for both quantitative and qualitative data collection, but real-time transcription error rates, limited emotion detection abilities, and uneven follow-up quality indicate that the utility, use and adoption of current AI interviewer technology may be context-dependent for qualitative data collection efforts.

Method: Reframe OPQRST extraction from sequence labeling to text generation using LLMs, enabling models to provide reasoning steps mimicking physician cognition. Propose modified NER metrics with BERT Score integration for semantic similarity evaluation.

Result: Significant advancement in AI healthcare applications, offering scalable solution that improves accuracy and usability of EHR information extraction.

Conclusion: The approach enhances interpretability, adapts to limited labeled data in healthcare, and aids clinicians in making more informed decisions to improve patient care outcomes.

Abstract: The extraction of critical patient information from Electronic Health Records (EHRs) poses significant challenges due to the complexity and unstructured nature of the data. Traditional machine learning approaches often fail to capture pertinent details efficiently, making it difficult for clinicians to utilize these tools effectively in patient care. This paper introduces a novel approach to extracting the OPQRST assessment from EHRs by leveraging the capabilities of Large Language Models (LLMs). We propose to reframe the task from sequence labeling to text generation, enabling the models to provide reasoning steps that mimic a physician’s cognitive processes. This approach enhances interpretability and adapts to the limited availability of labeled data in healthcare settings. Furthermore, we address the challenge of evaluating the accuracy of machine-generated text in clinical contexts by proposing a modification to traditional Named Entity Recognition (NER) metrics. This includes the integration of semantic similarity measures, such as the BERT Score, to assess the alignment between generated text and the clinical intent of the original records. Our contributions demonstrate a significant advancement in the use of AI in healthcare, offering a scalable solution that improves the accuracy and usability of information extraction from EHRs, thereby aiding clinicians in making more informed decisions and enhancing patient care outcomes.

Method: Adopt split-and-match algorithm to produce weak annotations on 1.2M chief complaint records, then train BERT-based model to locate entity mentions and link to pre-defined ontology.

Result: Superior performance over previous methods without any human annotation.

Conclusion: The proposed weakly supervised method effectively extracts and links entities from chief complaints, addressing standardization challenges in medical text mining.

Abstract: A Chief complaint (CC) is the reason for the medical visit as stated in the patient’s own words. It helps medical professionals to quickly understand a patient’s situation, and also serves as a short summary for medical text mining. However, chief complaint records often take a variety of entering methods, resulting in a wide variation of medical notations, which makes it difficult to standardize across different medical institutions for record keeping or text mining. In this study, we propose a weakly supervised method to automatically extract and link entities in chief complaints in the absence of human annotation. We first adopt a split-and-match algorithm to produce weak annotations, including entity mention spans and class labels, on 1.2 million real-world de-identified and IRB approved chief complaint records. Then we train a BERT-based model with generated weak labels to locate entity mentions in chief complaint text and link them to a pre-defined ontology. We conducted extensive experiments, and the results showed that our Weakly Supervised Entity Extraction and Linking (\ours) method produced superior performance over previous methods without any human annotation.

Method: DRAssist system identifies key structural elements (facts, disagreement aspects, arguments) from unstructured dispute descriptions, creates structured summaries, and uses multiple prompting strategies with LLMs to provide resolution outputs at three levels: identifying stronger party, evaluating specific demands, and assessing argument strength.

Result: The system was evaluated on automobile insurance and domain name disputes, comparing LLM performance against baselines using appropriate metrics (though specific results are not detailed in the abstract).

Conclusion: LLMs show promise as assistants for human judges in dispute resolution by providing structured analysis and multi-level evaluation capabilities across different dispute domains.

Abstract: Disputes between two parties occur in almost all domains such as taxation, insurance, banking, healthcare, etc. The disputes are generally resolved in a specific forum (e.g., consumer court) where facts are presented, points of disagreement are discussed, arguments as well as specific demands of the parties are heard, and finally a human judge resolves the dispute by often favouring one of the two parties. In this paper, we explore the use of large language models (LLMs) as assistants for the human judge to resolve such disputes, as part of our DRAssist system. We focus on disputes from two specific domains – automobile insurance and domain name disputes. DRAssist identifies certain key structural elements (e.g., facts, aspects or disagreement, arguments) of the disputes and summarizes the unstructured dispute descriptions to produce a structured summary for each dispute. We then explore multiple prompting strategies with multiple LLMs for their ability to assist in resolving the disputes in these domains. In DRAssist, these LLMs are prompted to produce the resolution output at three different levels – (i) identifying an overall stronger party in a dispute, (ii) decide whether each specific demand of each contesting party can be accepted or not, (iii) evaluate whether each argument by each contesting party is strong or weak. We evaluate the performance of LLMs on all these tasks by comparing them with relevant baselines using suitable evaluation metrics.

Method: Uses dual-graph encoder with dependency parsing and topic modeling, graph isomorphism networks for structural feature propagation, and hierarchical contrastive learning with node-level and graph-aware objectives.

Result: Achieves 86.7% F1-score (+6.2% absolute gain) on legal statute matching and demonstrates significant improvements on three legal document matching benchmarks and academic plagiarism detection datasets.

Conclusion: StructCoh effectively combines structural reasoning with representation optimization, outperforming state-of-the-art methods by better capturing hierarchical structural patterns and semantic distinctions.

Abstract: Text semantic matching requires nuanced understanding of both structural relationships and fine-grained semantic distinctions. While pre-trained language models excel at capturing token-level interactions, they often overlook hierarchical structural patterns and struggle with subtle semantic discrimination. In this paper, we proposed StructCoh, a graph-enhanced contrastive learning framework that synergistically combines structural reasoning with representation space optimization. Our approach features two key innovations: (1) A dual-graph encoder constructs semantic graphs via dependency parsing and topic modeling, then employs graph isomorphism networks to propagate structural features across syntactic dependencies and cross-document concept nodes. (2) A hierarchical contrastive objective enforces consistency at multiple granularities: node-level contrastive regularization preserves core semantic units, while graph-aware contrastive learning aligns inter-document structural semantics through both explicit and implicit negative sampling strategies. Experiments on three legal document matching benchmarks and academic plagiarism detection datasets demonstrate significant improvements over state-of-the-art methods. Notably, StructCoh achieves 86.7% F1-score (+6.2% absolute gain) on legal statute matching by effectively identifying argument structure similarities.

Method: Evaluated 4 LLMs on 34 standardized longitudinal periodontal cases (258 QA pairs) using automated metrics and blinded evaluations by licensed dentists.

Result: DeepSeek achieved highest faithfulness (median 0.528 vs 0.367-0.457) and expert ratings (median 4.5/5 vs 4.0/5) while maintaining readability.

Conclusion: DeepSeek is the leading LLM for dental case analysis, suitable for integration as an adjunct tool in medical education and research, with potential as a domain-specific agent.

Abstract: Large language models (LLMs) hold transformative potential in healthcare, yet their capacity to interpret longitudinal patient narratives remains inadequately explored. Dentistry, with its rich repository of structured clinical data, presents a unique opportunity to rigorously assess LLMs' reasoning abilities. While several commercial LLMs already exist, DeepSeek, a model that gained significant attention earlier this year, has also joined the competition. This study evaluated four state-of-the-art LLMs (GPT-4o, Gemini 2.0 Flash, Copilot, and DeepSeek V3) on their ability to analyze longitudinal dental case vignettes through open-ended clinical tasks. Using 34 standardized longitudinal periodontal cases (comprising 258 question-answer pairs), we assessed model performance via automated metrics and blinded evaluations by licensed dentists. DeepSeek emerged as the top performer, demonstrating superior faithfulness (median score = 0.528 vs. 0.367-0.457) and higher expert ratings (median = 4.5/5 vs. 4.0/5), without significantly compromising readability. Our study positions DeepSeek as the leading LLM for case analysis, endorses its integration as an adjunct tool in both medical education and research, and highlights its potential as a domain-specific agent.

Method: A framework that treats semantic text attributes as gene sequences, uses LLMs to simulate genetic operations (crossover and mutation), and integrates active learning for optimal parent selection to expand search space.

Result: Significantly outperforms state-of-the-art baselines, shows robust performance across various model sizes, and fusing synthetic data with original training boosts downstream performance, especially in class-imbalanced scenarios.

Conclusion: Genetic Prompt is an effective method for producing high-quality synthetic data that works well across various NLP applications and model scales.

Abstract: Large Language Models (LLMs) excel at generating synthetic data, but ensuring its quality and diversity remains challenging. We propose Genetic Prompt, a novel framework that combines genetic algorithms with LLMs to augment synthetic data generation. Our approach treats semantic text attributes as gene sequences and leverages the LLM to simulate crossover and mutation operations. This genetic process enhances data quality and diversity by creating novel attribute combinations, yielding synthetic distributions closer to real-world data. To optimize parent selection, we also integrate an active learning scheme that expands the offspring search space. Our experiments on multiple NLP tasks reveal several key findings: Genetic Prompt not only significantly outperforms state-of-the-art baselines but also shows robust performance across various generator model sizes and scales. Moreover, we demonstrate that fusing our synthetic data with the original training set significantly boosts downstream model performance, particularly for class-imbalanced scenarios. Our findings validate that Genetic Prompt is an effective method for producing high-quality synthetic data for a wide range of NLP applications.

Method: The researchers analyzed performance and internal component contributions using Cumulative Weighted Attribution, a metric derived from Direct Logit Attribution, comparing foundation models and instruction-tuned models.

Result: Instruction-tuning substantially improves length control by specializing components in deeper model layers. Attention heads in later layers show increasingly positive contributions in English, while in Italian, final-layer MLPs exhibit stronger positive role as a compensatory mechanism.

Conclusion: Instruction-tuning reconfigures later layers for better task adherence, with component-level strategies adapting to linguistic context, demonstrating that model architecture responds differently to length constraints across languages.

Abstract: Adhering to explicit length constraints, such as generating text with a precise word count, remains a significant challenge for Large Language Models (LLMs). This study aims at investigating the differences between foundation models and their instruction-tuned counterparts, on length-controlled text generation in English and Italian. We analyze both performance and internal component contributions using Cumulative Weighted Attribution, a metric derived from Direct Logit Attribution. Our findings reveal that instruction-tuning substantially improves length control, primarily by specializing components in deeper model layers. Specifically, attention heads in later layers of IT models show increasingly positive contributions, particularly in English. In Italian, while attention contributions are more attenuated, final-layer MLPs exhibit a stronger positive role, suggesting a compensatory mechanism. These results indicate that instruction-tuning reconfigures later layers for task adherence, with component-level strategies potentially adapting to linguistic context.

Method: Retrieves top k reference prompts from HelpSteer2 dataset and uses two paradigms: tiered contrastive reasoning (comparing high/medium/low quality prompts) and multi-metric contrastive reasoning (analyzing best prompts along each evaluation dimension).

Result: Experimental results on HelpSteer2 benchmark demonstrate that CRPO significantly outperforms baselines.

Conclusion: Contrastive, retrieval-augmented reasoning shows promise for advancing automatic prompt optimization by enabling more robust and interpretable optimization.

Abstract: Automatic prompt optimization has recently emerged as a strategy for improving the quality of prompts used in Large Language Models (LLMs), with the goal of generating more accurate and useful responses. However, most prior work focuses on direct prompt refinement or model fine-tuning, overlooking the potential of leveraging LLMs’ inherent reasoning capability to learn from contrasting examples. In this paper, we present Contrastive Reasoning Prompt Optimization (CRPO), a novel framework that formulates prompt optimization as a retrieval augmented reasoning process. Our approach retrieves top k reference prompts from the HelpSteer2 dataset, an open-source collection annotated for helpfulness, correctness, coherence, complexity, and verbosity, and constructs two complementary optimization paradigms: (1) tiered contrastive reasoning, where the LLM compares high, medium, and low quality prompts to refine its own generation through reflective reasoning, and (2) multi-metric contrastive reasoning, where the LLM analyzes the best prompts along each evaluation dimension and integrates their strengths into an optimized prompt. By explicitly contrasting high and low quality exemplars, CRPO enables the model to deduce why certain prompts succeed while others fail, thereby achieving more robust and interpretable optimization. Experimental results on the HelpSteer2 benchmark demonstrate that CRPO significantly outperforms baselines. Our findings highlight the promise of contrastive, retrieval-augmented reasoning for advancing automatic prompt optimization.

Method: JudgeAgent employs a knowledge-target adaptive dynamic evaluation framework with benchmark grading, interactive extension, and evaluation feedback. It uses knowledge-driven data synthesis and target-adaptive difficulty adjustment for extended testing.

Result: Extensive experiments demonstrate the effectiveness of JudgeAgent and its dynamic evaluation paradigm in providing accurate and effective evaluation results.

Conclusion: The proposed interviewer-style evaluation paradigm with JudgeAgent addresses key limitations of current static evaluation methods, enabling more precise determination of LLM knowledge and capability boundaries through adaptive, interactive testing.

Abstract: Evaluating the capabilities of large language models (LLMs) is an essential step to ensure the successful application of LLMs across various domains. The current evaluation of LLMs is based on a paradigm that involves querying them with predefined question sets and assessing their outputs. This paradigm offers controllable processes and simplicity, but faces challenges such as limited interaction with targets, insufficient difficulty control, and difficulties in verifying the validity of evaluation results, making it hard to precisely determine the knowledge and capability boundaries of target models. To address these challenges, we propose JudgeAgent, a knowledge-target adaptive dynamic evaluation framework based on a new interviewer-style evaluation paradigm. JudgeAgent employs a comprehensive evaluation approach consisting of benchmark grading, interactive extension, and evaluation feedback. It utilizes knowledge-driven data synthesis and target-adaptive difficulty adjustment methods to conduct extended testing, providing accurate and effective evaluation results. We also introduce a novel insight into validating evaluation methods, demonstrating the effectiveness of JudgeAgent and its dynamic evaluation paradigm through extensive experiments.

Method: Structured document parsing to get co-modality representations, separate text/image channel retrieval, cross-modal result aggregation, and VLM prompting for final response generation.

Result: Significantly outperforms pure-vision-based RAG in visual document question answering tasks.

Conclusion: Integrating co-modality information in a unified RAG framework effectively improves complex document VQA system performance.

Abstract: Retrieval-Augmented Generation (RAG) has become a core paradigm in document question answering tasks. However, existing methods have limitations when dealing with multimodal documents: one category of methods relies on layout analysis and text extraction, which can only utilize explicit text information and struggle to capture images or unstructured content; the other category treats document segmentation as visual input and directly passes it to visual language models (VLMs) for processing, yet it ignores the semantic advantages of text, leading to suboptimal generation results. This paper proposes co-modality-based RAG (CMRAG), which can simultaneously leverage text and images for efficient retrieval and generation. Specifically, we first perform structured parsing on documents to obtain co-modality representations of text segments and image regions. Subsequently, in response to user queries, we retrieve candidate evidence from text and image channels, respectively, and aggregate the results at the cross-modal retrieval level. Finally, we prompt the VLM to generate the final response based on the co-modality retrieval results. Experiments demonstrate that our method significantly outperforms pure-vision-based RAG in visual document question answering tasks. The findings of this paper show that integrating co-modality information into the RAG framework in a unified manner is an effective approach to improving the performance of complex document visual question-answering (VQA) systems.

Method: Constitution-Aware Decoding Layer applied at inference time using speculative decoding. Small Language Model acts as biased generator, while constitutionally guided LLM verifies and enforces bias-robust trajectories without parameter updates.

Result: Achieves absolute bias reduction up to 26.41% compared to baseline models, addressing casteist and communal biases effectively.

Conclusion: The framework successfully reduces harmful biases in LLM outputs for Indian context using constitutional guidance and speculative decoding, providing a cost-effective solution without model retraining.

Abstract: Large Language Models (LLMs) can inadvertently reflect societal biases present in their training data, leading to harmful or prejudiced outputs. In the Indian context, our empirical evaluations across a suite of models reveal that biases around caste and religion are particularly salient. Yet, most existing mitigation strategies are Western-centric and fail to address these local nuances. We propose AMBEDKAR, a framework inspired by the egalitarian vision of Dr B. R. Ambedkar, architect of the Indian Constitution, to guide LLM outputs toward fairness, neutrality, and inclusion in line with Articles 14 to 17. Our approach introduces a Constitution-Aware Decoding Layer, guided by the AI Constitution of India and applied only at inference time, without any parameter updates to the base model. We incorporate a speculative decoding algorithm that proactively reduces casteist and communal bias during generation. This mitigation layer operates directly within the decoding process, avoiding changes to model internals and lowering the computational and infrastructural costs associated with retraining. We reinterpret speculative decoding not merely as an efficiency tool but as a mechanism for fairness. In this framework, a Small Language Model (SLM) acts as a potentially biased generator, while a constitutionally guided Large Language Model (LLM) serves as the verifier. Rather than accelerating generation, the LLM enforces bias-robust trajectories in the SLM outputs. This inversion of roles gives rise to a fairness-by-speculation paradigm. Our approach yields an absolute reduction of bias up to 26.41 percent compared to baseline. Our source code, datasets, and results are available at https://anonymous.4open.science/r/AMBEDKAR-983B/

Method: Replace part of autoregressive objective with first-order model-agnostic meta-learning (MAML) during pretraining, tested across four model sizes (11M-570M parameters) on Tagalog and Cebuano languages.

Result: MAML improved zero-shot micro-F1 by 2-6pp with head-only tuning and 1-3pp with full tuning, while reducing convergence time by up to 8%. Largest gains for single-token person entities co-occurring with Tagalog case particles.

Conclusion: Meta-learning enables small LMs to effectively transfer to unseen low-resource languages, with surface anchors like case particles playing crucial role in zero-shot NER performance.

Abstract: Named-entity recognition (NER) in low-resource languages is usually tackled by finetuning very large multilingual LMs, an option that is often infeasible in memory- or latency-constrained settings. We ask whether small decoder LMs can be pretrained so that they adapt quickly and transfer zero-shot to languages unseen during pretraining. To this end we replace part of the autoregressive objective with first-order model-agnostic meta-learning (MAML). Tagalog and Cebuano are typologically similar yet structurally different in their actor/non-actor voice systems, and hence serve as a challenging test-bed. Across four model sizes (11 M - 570 M) MAML lifts zero-shot micro-F1 by 2-6 pp under head-only tuning and 1-3 pp after full tuning, while cutting convergence time by up to 8%. Gains are largest for single-token person entities that co-occur with Tagalog case particles si/ni, highlighting the importance of surface anchors.

Method: A decoding strategy that modifies token logits by penalizing similarity to previously generated outputs using two adaptive similarity measures: Concept-level Similarity Penalty (early stages) and Narrative-level Similarity Penalty (later stages).

Result: Achieves up to 2.6 times higher output diversity, reduces repetition by an average of 30% compared to strong baselines, and effectively mitigates text degeneration while activating a broader range of neurons.

Conclusion: The method successfully enhances creative diversity in LLM outputs by leveraging the model’s intrinsic creativity through adaptive similarity penalties at different generation stages.

Abstract: Large Language Models (LLMs) often generate repetitive and monotonous outputs, especially in tasks like story generation, due to limited creative diversity when given the same input prompt. To address this challenge, we propose a novel decoding strategy, Avoidance Decoding, that modifies token logits by penalizing similarity to previously generated outputs, thereby encouraging more diverse multi-branch stories. This penalty adaptively balances two similarity measures: (1) Concept-level Similarity Penalty, which is prioritized in early stages to diversify initial story concepts, and (2) Narrative-level Similarity Penalty, which is increasingly emphasized later to ensure natural yet diverse plot development. Notably, our method achieves up to 2.6 times higher output diversity and reduces repetition by an average of 30% compared to strong baselines, while effectively mitigating text degeneration. Furthermore, we reveal that our method activates a broader range of neurons, demonstrating that it leverages the model’s intrinsic creativity.

Method: Created FActBench covering 4 generation tasks and 6 state-of-the-art LLMs for medical domain, using Chain-of-Thought Prompting and Natural Language Inference techniques with unanimous voting approach.

Result: Experiments showed that fact-checking scores obtained through unanimous voting of both CoT and NLI techniques correlate best with domain expert evaluation.

Conclusion: The unanimous voting approach combining CoT and NLI provides the most reliable fact-checking method for medical domain LLMs, addressing critical factuality issues in specialized domains.

Abstract: Large Language Models tend to struggle when dealing with specialized domains. While all aspects of evaluation hold importance, factuality is the most critical one. Similarly, reliable fact-checking tools and data sources are essential for hallucination mitigation. We address these issues by providing a comprehensive Fact-checking Benchmark FActBench covering four generation tasks and six state-of-the-art Large Language Models (LLMs) for the Medical domain. We use two state-of-the-art Fact-checking techniques: Chain-of-Thought (CoT) Prompting and Natural Language Inference (NLI). Our experiments show that the fact-checking scores acquired through the Unanimous Voting of both techniques correlate best with Domain Expert Evaluation.

Method: Proposed Fundamental Language Model (FLM) paradigm using smaller linguistically competent models that offload factual retrieval to external tools. Evaluated models from 135M to 32B parameters across linguistic competence, external factual knowledge, and internal factual knowledge.

Result: Found that while both linguistic competence and factual knowledge improve with scale, internal factual knowledge grows significantly faster, showing model size is more tied to memorization than core language ability.

Conclusion: Supports modular approach where compact linguistically proficient models serve as foundation for tool-augmented systems, offering path to more efficient, interpretable, and sustainable NLP solutions.

Abstract: Large Language Models offer impressive language capabilities but suffer from well-known limitations, including hallucinations, biases, privacy concerns, and high computational costs. These issues are largely driven by the combination of linguistic competence and factual memorization within a single monolithic model. This paper introduces and empirically supports the Fundamental Language Model (FLM) paradigm, which advocates for smaller, linguistically competent models that offload factual retrieval to external tools. We evaluate models ranging from 135M to 32B parameters across three dimensions: linguistic competence, external factual knowledge, and internal factual knowledge. Our findings reveal that while both linguistic competence and factual knowledge improve with scale, internal factual knowledge grows significantly faster, suggesting that model size is more closely tied to memorization than to core language ability. These results support a modular approach to language modeling, where compact, linguistically proficient models serve as the foundation for tool-augmented systems. The FLM paradigm offers a path toward more efficient, interpretable, and sustainable NLP solutions.

Method: Two-step framework: 1) LLM generates annotations of shared mental models from team dialogues, 2) Secondary LLM compares LLM vs human annotations against gold-standard labels to detect divergences

Result: LLMs show coherence on straightforward annotation tasks but systematically fail in scenarios requiring spatial reasoning or prosodic cue disambiguation

Conclusion: While LLMs can be used for team mental model analysis, they have limitations in complex reasoning scenarios requiring spatial understanding and nuanced interpretation

Abstract: What if large language models could not only infer human mindsets but also expose every blind spot in team dialogue such as discrepancies in the team members’ joint understanding? We present a novel, two-step framework that leverages large language models (LLMs) both as human-style annotators of team dialogues to track the team’s shared mental models (SMMs) and as automated discrepancy detectors among individuals’ mental states. In the first step, an LLM generates annotations by identifying SMM elements within task-oriented dialogues from the Cooperative Remote Search Task (CReST) corpus. Then, a secondary LLM compares these LLM-derived annotations and human annotations against gold-standard labels to detect and characterize divergences. We define an SMM coherence evaluation framework for this use case and apply it to six CReST dialogues, ultimately producing: (1) a dataset of human and LLM annotations; (2) a reproducible evaluation framework for SMM coherence; and (3) an empirical assessment of LLM-based discrepancy detection. Our results reveal that, although LLMs exhibit apparent coherence on straightforward natural-language annotation tasks, they systematically err in scenarios requiring spatial reasoning or disambiguation of prosodic cues.

Method: Dynamic Clipping Policy Optimization (DCPO) with adaptive clipping bounds based on token-specific prior probabilities for better exploration, and smooth advantage standardization across cumulative training steps for improved response utilization.

Result: Achieved state-of-the-art performance on four benchmarks with four different models. On AIME24 with Qwen2.5-Math-7B: 46.7/38.8 (greedy/sampling) vs DAPO (36.7/31.6) and GRPO (36.7/32.1). On AIME25 with Qwen2.5-14B: 23.3/19.0 vs GRPO (13.3/10.5) and DAPO (20.0/15.3). 28% improvement in nonzero advantage over GRPO, doubled training efficiency over DAPO, and order-of-magnitude reduction in token clipping ratio.

Conclusion: DCPO effectively leverages generated data more efficiently for reinforcement learning in LLMs by addressing gradient issues through dynamic clipping and advantage standardization, demonstrating superior performance and training efficiency.

Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) has emerged as a promising framework for enhancing the reasoning capabilities of large language models. However, existing approaches such as GRPO often suffer from zero gradients. This problem arises primarily due to fixed clipping bounds for token-level probability ratios and the standardization of identical rewards, which can lead to ineffective gradient updates and underutilization of generated responses. In this work, we propose Dynamic Clipping Policy Optimization (DCPO), which introduces a dynamic clipping strategy that adaptively adjusts the clipping bounds based on token-specific prior probabilities to enhance token-level exploration, and a smooth advantage standardization technique that standardizes rewards across cumulative training steps to improve the response-level effective utilization of generated responses. DCPO achieved state-of-the-art performance on four benchmarks based on four different models. In particular, DCPO achieved an Avg@1 of 46.7 under greedy decoding and an Avg@32 of 38.8 under 32 times sampling on the AIME24 benchmark, surpassing both DAPO (36.7/31.6) and GRPO (36.7/32.1) on the Qwen2.5-Math-7B model. On the AIME25 benchmark based on Qwen2.5-14B, DCPO achieves a performance of (23.3/19.0), surpassing GRPO (13.3/10.5) and DAPO (20.0/15.3). Furthermore, DCPO achieved an average 28% improvement in the nonzero advantage over GRPO in four models, doubled the training efficiency over DAPO, and significantly reduced the token clipping ratio by an order of magnitude compared to both GRPO and DAPO, while achieving superior performance. These results highlight DCPO’s effectiveness in leveraging generated data more efficiently for reinforcement learning in large language models.

Method: Introduces taxonomy centered on execution paradigms: latent optimization, signal-guided control, and layer-recurrent execution. Reviews structural, behavioral and representation-based evidence supporting implicit reasoning.

Result: Organizes existing methods into three computational paradigms and provides structured overview of evaluation metrics and benchmarks for assessing implicit reasoning effectiveness.

Conclusion: Fills the gap in understanding internal reasoning mechanisms in LLMs, providing a systematic framework for analyzing implicit reasoning approaches and their computational strategies.

Abstract: Large Language Models (LLMs) have demonstrated strong generalization across a wide range of tasks. Reasoning with LLMs is central to solving multi-step problems and complex decision-making. To support efficient reasoning, recent studies have shifted attention from explicit chain-of-thought prompting toward implicit reasoning, where reasoning occurs silently via latent structures without emitting intermediate textual steps. Implicit reasoning brings advantages such as lower generation cost, faster inference, and better alignment with internal computation. Although prior surveys have discussed latent representations in the context of reasoning, a dedicated and mechanism-level examination of how reasoning unfolds internally within LLMs remains absent. This survey fills that gap by introducing a taxonomy centered on execution paradigms, shifting the focus from representational forms to computational strategies. We organize existing methods into three execution paradigms based on \textbf{\textit{how and where internal computation unfolds}}: latent optimization, signal-guided control, and layer-recurrent execution. We also review structural, behavioral and representation-based evidence that supports the presence of implicit reasoning in LLMs. We further provide a structured overview of the evaluation metrics and benchmarks used in existing works to assess the effectiveness and reliability of implicit reasoning.We maintain a continuously updated project at: https://github.com/digailab/awesome-llm-implicit-reasoning.

Method: Integrates established opinion mining formulations into declarative LLM annotation pipeline, defines three data models based on sentiment/opinion mining literature, uses two separate LLMs for annotations with inter-annotator agreement evaluation.

Result: Rigorous quantitative evaluation using human-annotated test samples shows successful construction of time-aligned structured opinion knowledge base.

Conclusion: The resulting knowledge base enables applications in Retrieval-Augmented Generation, temporal question answering, and timeline summarization through automated LLM-based annotation.

Abstract: We propose a scalable method for constructing a temporal opinion knowledge base with large language models (LLMs) as automated annotators. Despite the demonstrated utility of time-series opinion analysis of text for downstream applications such as forecasting and trend analysis, existing methodologies underexploit this potential due to the absence of temporally grounded fine-grained annotations. Our approach addresses this gap by integrating well-established opinion mining formulations into a declarative LLM annotation pipeline, enabling structured opinion extraction without manual prompt engineering. We define three data models grounded in sentiment and opinion mining literature, serving as schemas for structured representation. We perform rigorous quantitative evaluation of our pipeline using human-annotated test samples. We carry out the final annotations using two separate LLMs, and inter-annotator agreement is computed label-wise across the fine-grained opinion dimensions, analogous to human annotation protocols. The resulting knowledge base encapsulates time-aligned, structured opinions and is compatible with applications in Retrieval-Augmented Generation (RAG), temporal question answering, and timeline summarisation.

Method: Three Arabic text preprocessing methods using LLMs (summarization, refinement, NER) followed by fine-tuning Arabic transformer models (CAMeLBERT, AraBERT, AsafayaBERT) and majority voting ensemble of original and preprocessed text representations.

Result: Achieved 80.56% classification accuracy, demonstrating effectiveness of combining various text representations and model predictions for Arabic medical text understanding.

Conclusion: The integration of LLM-based preprocessing with fine-tuned Arabic transformers and ensemble learning provides an effective framework for disease classification in Arabic social telehealth data, representing the first work of its kind in this domain.

Abstract: Social telehealth has made remarkable progress in healthcare by allowing patients to post symptoms and participate in medical consultations remotely. Users frequently post symptoms on social media and online health platforms, creating a huge repository of medical data that can be leveraged for disease classification. Large language models (LLMs) such as LLAMA3 and GPT-3.5, along with transformer-based models like BERT, have demonstrated strong capabilities in processing complex medical text. In this study, we evaluate three Arabic medical text preprocessing methods such as summarization, refinement, and Named Entity Recognition (NER) before applying fine-tuned Arabic transformer models (CAMeLBERT, AraBERT, and AsafayaBERT). To enhance robustness, we adopt a majority voting ensemble that combines predictions from original and preprocessed text representations. This approach achieved the best classification accuracy of 80.56%, thus showing its effectiveness in leveraging various text representations and model predictions to improve the understanding of medical texts. To the best of our knowledge, this is the first work that integrates LLM-based preprocessing with fine-tuned Arabic transformer models and ensemble learning for disease classification in Arabic social telehealth data.

Method: Proposes EmoPerso framework that uses generative mechanisms for synthetic data augmentation, extracts pseudo-labeled emotion features, employs multi-task learning with personality prediction, and uses cross-attention modules to capture fine-grained interactions between personality traits and emotional representations.

Result: Extensive experiments on two benchmark datasets demonstrate that EmoPerso surpasses state-of-the-art models in personality detection performance.

Conclusion: The proposed EmoPerso framework effectively addresses the limitations of existing methods by leveraging emotion-aware modeling and self-supervised learning, achieving superior performance in personality detection from text.

Abstract: Personality detection from text is commonly performed by analysing users' social media posts. However, existing methods heavily rely on large-scale annotated datasets, making it challenging to obtain high-quality personality labels. Moreover, most studies treat emotion and personality as independent variables, overlooking their interactions. In this paper, we propose a novel self-supervised framework, EmoPerso, which improves personality detection through emotion-aware modelling. EmoPerso first leverages generative mechanisms for synthetic data augmentation and rich representation learning. It then extracts pseudo-labeled emotion features and jointly optimizes them with personality prediction via multi-task learning. A cross-attention module is employed to capture fine-grained interactions between personality traits and the inferred emotional representations. To further refine relational reasoning, EmoPerso adopts a self-taught strategy to enhance the model’s reasoning capabilities iteratively. Extensive experiments on two benchmark datasets demonstrate that EmoPerso surpasses state-of-the-art models. The source code is available at https://github.com/slz0925/EmoPerso.

Method: Conducted controlled experiments across multiple explanation benchmark datasets (general and domain-specific) with various label definition conditions including expert-curated, LLM-generated, perturbed, and swapped definitions.

Result: Explicit label definitions can enhance accuracy and explainability, but their integration is neither guaranteed nor consistent. Models often default to internal representations, especially in general tasks, while domain-specific tasks benefit more from explicit definitions.

Conclusion: LLMs’ processing of external knowledge alongside pre-existing capabilities requires deeper understanding, as they don’t reliably integrate external definitions and frequently rely on internalized representations.

Abstract: Do LLMs genuinely incorporate external definitions, or do they primarily rely on their parametric knowledge? To address these questions, we conduct controlled experiments across multiple explanation benchmark datasets (general and domain-specific) and label definition conditions, including expert-curated, LLM-generated, perturbed, and swapped definitions. Our results reveal that while explicit label definitions can enhance accuracy and explainability, their integration into an LLM’s task-solving processes is neither guaranteed nor consistent, suggesting reliance on internalized representations in many cases. Models often default to their internal representations, particularly in general tasks, whereas domain-specific tasks benefit more from explicit definitions. These findings underscore the need for a deeper understanding of how LLMs process external knowledge alongside their pre-existing capabilities.

Method: Developed an automated framework that parses behavioral statements from provider specifications, generates targeted prompts to test adherence, and uses the providers’ own models as judges to evaluate consistency between specifications, model outputs, and model judgments.

Result: Applied to 16 models from 6 developers across 100+ behavioral statements, revealing systematic inconsistencies including compliance gaps of up to 20% across different providers.

Conclusion: The framework establishes a necessary baseline for model consistency and demonstrates that current foundation models show significant gaps in adhering to their own developers’ behavioral specifications when judged by the developers’ own evaluation models.

Abstract: Companies that develop foundation models publish behavioral guidelines they pledge their models will follow, but it remains unclear if models actually do so. While providers such as OpenAI, Anthropic, and Google have published detailed specifications describing both desired safety constraints and qualitative traits for their models, there has been no systematic audit of adherence to these guidelines. We introduce an automated framework that audits models against their providers specifications by parsing behavioral statements, generating targeted prompts, and using models to judge adherence. Our central focus is on three way consistency between a provider specification, its model outputs, and its own models as judges; an extension of prior two way generator validator consistency. This establishes a necessary baseline: at minimum, a foundation model should consistently satisfy the developer behavioral specifications when judged by the developer evaluator models. We apply our framework to 16 models from six developers across more than 100 behavioral statements, finding systematic inconsistencies including compliance gaps of up to 20 percent across providers.

Method: Proposed self-training approach leveraging unlabeled data to elicit reward reasoning, developing GRAM-R² - a generative reward model that produces preference labels and accompanying rationales.

Result: GRAM-R² consistently delivers strong performance in response ranking, task adaptation, and reinforcement learning from human feedback, outperforming several strong discriminative and generative baselines.

Conclusion: GRAM-R² serves as an effective foundation model for reward reasoning that can be applied to wide range of tasks with minimal fine-tuning, supporting downstream applications like response ranking and task-specific reward tuning.

Abstract: Significant progress in reward modeling over recent years has been driven by a paradigm shift from task-specific designs towards generalist reward models. Despite this trend, developing effective reward models remains a fundamental challenge: the heavy reliance on large-scale labeled preference data. Pre-training on abundant unlabeled data offers a promising direction, but existing approaches fall short of instilling explicit reasoning into reward models. To bridge this gap, we propose a self-training approach that leverages unlabeled data to elicit reward reasoning in reward models. Based on this approach, we develop GRAM-R$^2$, a generative reward model trained to produce not only preference labels but also accompanying reward rationales. GRAM-R$^2$ can serve as a foundation model for reward reasoning and can be applied to a wide range of tasks with minimal or no additional fine-tuning. It can support downstream applications such as response ranking and task-specific reward tuning. Experiments on response ranking, task adaptation, and reinforcement learning from human feedback demonstrate that GRAM-R$^2$ consistently delivers strong performance, outperforming several strong discriminative and generative baselines.

Method: Mixture of Stylistic Experts (MoSEs) framework with three components: Stylistics Reference Repository (SRR) for reference data activation, Stylistics-Aware Router (SAR), and Conditional Threshold Estimator (CTE) that jointly models linguistic statistical properties and semantic features for dynamic threshold determination.

Result: Achieves 11.34% average improvement in detection performance compared to baselines, with 39.15% improvement in low-resource cases.

Conclusion: MoSEs framework effectively addresses stylistic modeling limitations in AI-generated text detection and demonstrates significant performance gains, particularly in resource-constrained environments.

Abstract: The rapid advancement of large language models has intensified public concerns about the potential misuse. Therefore, it is important to build trustworthy AI-generated text detection systems. Existing methods neglect stylistic modeling and mostly rely on static thresholds, which greatly limits the detection performance. In this paper, we propose the Mixture of Stylistic Experts (MoSEs) framework that enables stylistics-aware uncertainty quantification through conditional threshold estimation. MoSEs contain three core components, namely, the Stylistics Reference Repository (SRR), the Stylistics-Aware Router (SAR), and the Conditional Threshold Estimator (CTE). For input text, SRR can activate the appropriate reference data in SRR and provide them to CTE. Subsequently, CTE jointly models the linguistic statistical properties and semantic features to dynamically determine the optimal threshold. With a discrimination score, MoSEs yields prediction labels with the corresponding confidence level. Our framework achieves an average improvement 11.34% in detection performance compared to baselines. More inspiringly, MoSEs shows a more evident improvement 39.15% in the low-resource case. Our code is available at https://github.com/creator-xi/MoSEs.

Method: Created curated dataset with news articles paired with four headline variants (original, semantically similar, lexically similar, unrelated) and benchmarked multilingual and language-specific sentence transformers using cosine similarity for headline identification task.

Result: Multilingual models consistently perform well across languages, while language-specific models show varying effectiveness. The dataset serves as valuable resource for evaluating semantic understanding in RAG pipelines and other NLP applications.

Conclusion: The dataset bridges the benchmark gap for Indic languages, provides versatile evaluation resource for multiple tasks, and demonstrates the importance of multilingual models for low-resource language semantic understanding.

Abstract: Semantic evaluation in low-resource languages remains a major challenge in NLP. While sentence transformers have shown strong performance in high-resource settings, their effectiveness in Indic languages is underexplored due to a lack of high-quality benchmarks. To bridge this gap, we introduce L3Cube-IndicHeadline-ID, a curated headline identification dataset spanning ten low-resource Indic languages: Marathi, Hindi, Tamil, Gujarati, Odia, Kannada, Malayalam, Punjabi, Telugu, Bengali and English. Each language includes 20,000 news articles paired with four headline variants: the original, a semantically similar version, a lexically similar version, and an unrelated one, designed to test fine-grained semantic understanding. The task requires selecting the correct headline from the options using article-headline similarity. We benchmark several sentence transformers, including multilingual and language-specific models, using cosine similarity. Results show that multilingual models consistently perform well, while language-specific models vary in effectiveness. Given the rising use of similarity models in Retrieval-Augmented Generation (RAG) pipelines, this dataset also serves as a valuable resource for evaluating and improving semantic understanding in such applications. Additionally, the dataset can be repurposed for multiple-choice question answering, headline classification, or other task-specific evaluations of LLMs, making it a versatile benchmark for Indic NLP. The dataset is shared publicly at https://github.com/l3cube-pune/indic-nlp

Method: Using AI to retranslate the exact same text excerpts that Pucci translated in 1931 (Dante’s La Vita Nuova from Italian to French and Voltaire’s Zadig from French to Italian) following Pucci’s original method of international keys and ideograms.

Result: The AI translations showed low average difference from Pucci’s 1931 translations with only minor variations. The method was then successfully extended to English, Spanish, and German translations, and applied to modern technical texts.

Conclusion: Pucci’s 1931 mechanical translation system is validated and proven effective, establishing him as a significant historical figure in machine translation alongside pioneers like Troyanskij, Booth, and Weaver, with implications for rewriting the history of the field.

Abstract: A pioneering rule-based mechanical translation system (precursor of modern RBMTs) was first presented in December 1929 by its inventor, Federico Pucci, who later published the full method in a book titled “Il traduttore meccanico ed il metodo per corrispondersi fra Europei conoscendo ciascuno solo la propria lingua: Parte I”, in Salerno (Italy), in 1931. This study illustrates how AI breathes new life into the system of international keys and ideograms devised by Pucci to translate from/into any Romance language (at least as a first step). The methodology involves having the AIs retranslate, following Pucci’s method, the two text excerpts originally translated in 1931 and clearly documented in his publication: a passage from Dante’s La Vita Nuova, translated from Italian into French, and a passage from Voltaire’s Zadig, translated from French into Italian. The result is notable: the two texts, translated 94 years apart using the same method–by Pucci in 1931 and by AIs in 2025–show a low average difference, with only minor variations observed. With Pucci’s system thus validated, it became feasible to have the AIs reproduce the excerpts in English, Spanish, and German according to his method. The results were consistent, and Pucci–via Artificial Intelligence–was tasked with translating more modern and technical texts, thereby reviving, nearly a century later, an invention that had remained almost entirely unknown and never applied beyond its creator, now brought to wider attention and opened to possible experimentation. Such a demonstration would not only affirm Pucci’s historical status but also place him among the precursors and intellectual contributors to machine translation, whose work merits examination alongside figures such as Troyanskij, Booth, and Weaver, with possible consequences for how the history of the field is understood.

Method: The authors formulate an entropy-constrained minimum divergence problem, prove it’s equivalent to an NP-hard entropy-constrained mass maximization problem, and develop top-H decoding as a computationally efficient greedy algorithm to solve it.

Result: Top-H outperforms min-p sampling by up to 25.63% on creative writing benchmarks while maintaining robustness on question-answering datasets (GPQA, GSM8K, MT-Bench). LLM-as-judge evaluation confirms it produces coherent outputs even at higher temperatures.

Conclusion: Top-H advances state-of-the-art in open-ended text generation and can be easily integrated into creative writing applications, providing better balance between diversity/creativity and logical coherence.

Abstract: Large language models (LLMs), despite their impressive performance across a wide range of tasks, often struggle to balance two competing objectives in open-ended text generation: fostering diversity and creativity while preserving logical coherence. Existing truncated sampling techniques, including temperature scaling, top-$p$ (nucleus) sampling, and min-$p$ sampling, aim to manage this trade-off. However, they exhibit limitations, particularly in the effective incorporation of the confidence of the model into the corresponding sampling strategy. For example, min-$p$ sampling relies on a single top token as a heuristic for confidence, eventually underutilizing the information of the probability distribution. Toward effective incorporation of the confidence of the model, in this paper, we present top-H decoding. We first establish the theoretical foundation of the interplay between creativity and coherence in truncated sampling by formulating an entropy-constrained minimum divergence problem. We then prove this minimization problem to be equivalent to an entropy-constrained mass maximization (ECMM) problem, which is NP-hard. Finally, we present top-H decoding, a computationally efficient greedy algorithm to solve the ECMM problem. Extensive empirical evaluations demonstrate that top-H outperforms the state-of-the-art (SoTA) alternative of min-$p$ sampling by up to 25.63% on creative writing benchmarks, while maintaining robustness on question-answering datasets such as GPQA, GSM8K, and MT-Bench. Additionally, an LLM-as-judge evaluation confirms that top-H indeed produces coherent outputs even at higher temperatures, where creativity is especially critical. In summary, top-H advances SoTA in open-ended text generation and can be easily integrated into creative writing applications. The code is available at https://github.com/ErfanBaghaei/Top-H-Decoding.

Method: Comparative evaluation of code-mixed fine-tuned models (HingBERT, HingMBERT, HingRoBERTa) vs non-code-mixed multilingual models (BERT Base Cased, IndicBERT, RoBERTa, MuRIL) and zero-shot Google Gemini on benchmark Hinglish NER dataset using Precision, Recall, and F1-score metrics.

Result: Code-mixed models, particularly HingRoBERTa and HingBERT-based models, outperformed all others including Google Gemini. Non-code-mixed models showed limited adaptability. Google Gemini demonstrated competitive zero-shot performance despite not being specifically trained on code-mixed data.

Conclusion: Specialized code-mixed models are most effective for Hinglish NER tasks due to domain-specific pretraining, though modern LLMs show strong generalization capabilities in zero-shot settings, highlighting the trade-off between specialization and generalization.

Abstract: Named Entity Recognition (NER) in code-mixed text, particularly Hindi-English (Hinglish), presents unique challenges due to informal structure, transliteration, and frequent language switching. This study conducts a comparative evaluation of code-mixed fine-tuned models and non-code-mixed multilingual models, along with zero-shot generative large language models (LLMs). Specifically, we evaluate HingBERT, HingMBERT, and HingRoBERTa (trained on code-mixed data), and BERT Base Cased, IndicBERT, RoBERTa and MuRIL (trained on non-code-mixed multilingual data). We also assess the performance of Google Gemini in a zero-shot setting using a modified version of the dataset with NER tags removed. All models are tested on a benchmark Hinglish NER dataset using Precision, Recall, and F1-score. Results show that code-mixed models, particularly HingRoBERTa and HingBERT-based fine-tuned models, outperform others - including closed-source LLMs like Google Gemini - due to domain-specific pretraining. Non-code-mixed models perform reasonably but show limited adaptability. Notably, Google Gemini exhibits competitive zero-shot performance, underlining the generalization strength of modern LLMs. This study provides key insights into the effectiveness of specialized versus generalized models for code-mixed NER tasks.

Method: PACS treats outcome rewards as predictable labels and reformulates RLVR as a supervised learning task over a score function parameterized by the policy model, optimized using cross-entropy loss to achieve implicit actor-critic coupling.

Result: PACS outperforms strong RLVR baselines (PPO and GRPO) on mathematical reasoning tasks, achieving 59.78% at pass@256 on AIME 2025 with improvements of 13.32 and 14.36 points over PPO and GRPO respectively.

Conclusion: The supervised learning formulation provides a simple yet powerful framework for LLM post-training with verifiable rewards, offering more stable and efficient training while recovering classical policy gradient updates implicitly.

Abstract: Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) have empowered large language models (LLMs) to tackle challenging reasoning tasks such as mathematics and programming. RLVR leverages verifiable outcome rewards to guide policy optimization, enabling LLMs to progressively improve output quality in a grounded and reliable manner. Despite its promise, the RLVR paradigm poses significant challenges, as existing methods often suffer from sparse reward signals and unstable policy gradient updates, particularly in RL-based approaches. To address the challenges, we propose $\textbf{PACS}$, a novel RLVR framework that achieves im$\textbf{P}$licit $\textbf{A}$ctor $\textbf{C}$ritic coupling via a $\textbf{S}$upervised learning framework. By treating the outcome reward as a predictable label, we reformulate the RLVR problem into a supervised learning task over a score function parameterized by the policy model and optimized using cross-entropy loss. A detailed gradient analysis shows that this supervised formulation inherently recovers the classical policy gradient update while implicitly coupling actor and critic roles, yielding more stable and efficient training. Benchmarking on challenging mathematical reasoning tasks, PACS outperforms strong RLVR baselines, such as PPO and GRPO, achieving superior reasoning performance. For instance, PACS achieves 59.78% at pass@256 on AIME 2025, representing improvements of 13.32 and 14.36 points over PPO and GRPO. This simple yet powerful framework offers a promising avenue for LLMs post-training with verifiable rewards. Our code and data are available as open source at https://github.com/ritzz-ai/PACS.

Method: DARLING introduces a learned partition function to measure semantic diversity beyond lexical variations, combining this diversity signal with quality rewards during online reinforcement learning.

Result: DARLING outperforms quality-only RL baselines on five benchmarks for non-verifiable tasks (higher quality and novelty) and achieves higher pass@1 and pass@k for verifiable tasks like math problems.

Conclusion: Explicitly optimizing for diversity catalyzes exploration in online RL, leading to higher-quality responses while maintaining semantic diversity across various model families and sizes.

Abstract: Post-training of Large Language Models (LMs) often prioritizes accuracy and helpfulness at the expense of diversity. This creates a tension: while post-training improves response quality, it also sharpens output distributions and reduces the range of ideas, limiting the usefulness of LMs in creative and exploratory tasks such as brainstorming, storytelling, or problem solving. We address this challenge with Diversity-Aware Reinforcement Learning (DARLING), a framework that jointly optimizes for response quality and semantic diversity. At its core, DARLING introduces a learned partition function to measure diversity beyond surface-level lexical variations. This diversity signal is then combined with a quality reward during online reinforcement learning, encouraging models to generate outputs that are both high-quality and distinct. Experiments across multiple model families and sizes show that DARLING generalizes to two regimes: non-verifiable tasks (instruction following and creative writing) and verifiable tasks (competition math). On five benchmarks in the first setting, DARLING consistently outperforms quality-only RL baselines, producing outputs that are simultaneously of higher quality and novelty. In the second setting, DARLING achieves higher pass@1 (solution quality) and pass@k (solution variety). Most strikingly, explicitly optimizing for diversity catalyzes exploration in online RL, which manifests itself as higher-quality responses.

Method: Two subtasks with multiple-choice questions in Modern Standard Arabic covering General Arabic Culture and General Islamic Culture across 22 Arab countries, with 26 and 19 teams participating respectively.

Result: Task-specific fine-tuning substantially boosted performance over baselines, with top systems achieving 72.15% accuracy on cultural questions and 84.22% on Islamic knowledge. Parameter-efficient fine-tuning was most effective.

Conclusion: The benchmark reveals LLMs’ cultural competence gaps and demonstrates that fine-tuning, particularly parameter-efficient methods, can significantly improve performance on underrepresented cultural domains.

Abstract: Large Language Models (LLMs) inherently reflect the vast data distributions they encounter during their pre-training phase. As this data is predominantly sourced from the web, there is a high chance it will be skewed towards high-resourced languages and cultures, such as those of the West. Consequently, LLMs often exhibit a diminished understanding of certain communities, a gap that is particularly evident in their knowledge of Arabic and Islamic cultures. This issue becomes even more pronounced with increasingly under-represented topics. To address this critical challenge, we introduce PalmX 2025, the first shared task designed to benchmark the cultural competence of LLMs in these specific domains. The task is composed of two subtasks featuring multiple-choice questions (MCQs) in Modern Standard Arabic (MSA): General Arabic Culture and General Islamic Culture. These subtasks cover a wide range of topics, including traditions, food, history, religious practices, and language expressions from across 22 Arab countries. The initiative drew considerable interest, with 26 teams registering for Subtask 1 and 19 for Subtask 2, culminating in nine and six valid submissions, respectively. Our findings reveal that task-specific fine-tuning substantially boosts performance over baseline models. The top-performing systems achieved an accuracy of 72.15% on cultural questions and 84.22% on Islamic knowledge. Parameter-efficient fine-tuning emerged as the predominant and most effective approach among participants, while the utility of data augmentation was found to be domain-dependent.

Method: Proposes a progressively refined similarity computation framework that combines attack testing with adversarial training. The algorithm constantly improves the model by catching different loopholes in similarity calculations and provides reasonable explanations for each refinement.

Result: The regression model achieves state-of-the-art performance in handling edge cases, demonstrating improved precision in similarity computation.

Conclusion: The proposed framework successfully addresses the shortcomings of existing similarity computation methods by providing both improved performance through adversarial training and better interpretability through explanatory refinements, making it particularly valuable for critical applications requiring precision.

Abstract: Capturing the similarities between human language units is crucial for explaining how humans associate different objects, and therefore its computation has received extensive attention, research, and applications. With the ever-increasing amount of information around us, calculating similarity becomes increasingly complex, especially in many cases, such as legal or medical affairs, measuring similarity requires extra care and precision, as small acts within a language unit can have significant real-world effects. My research goal in this thesis is to develop regression models that account for similarities between language units in a more refined way. Computation of similarity has come a long way, but approaches to debugging the measures are often based on continually fitting human judgment values. To this end, my goal is to develop an algorithm that precisely catches loopholes in a similarity calculation. Furthermore, most methods have vague definitions of the similarities they compute and are often difficult to interpret. The proposed framework addresses both shortcomings. It constantly improves the model through catching different loopholes. In addition, every refinement of the model provides a reasonable explanation. The regression model introduced in this thesis is called progressively refined similarity computation, which combines attack testing with adversarial training. The similarity regression model of this thesis achieves state-of-the-art performance in handling edge cases.

Method: Proposes a model that integrates contextual and textual signals through graph convolutional networks, introducing two metrics (confidence via rule-based method and relatedness from textual representations) for precise weighting of contextual information.

Result: Extensive experiments on two benchmark datasets show consistent improvements over strong baselines, demonstrating the effectiveness of the approach.

Conclusion: The integration of contextual and textual information with proper weighting metrics significantly enhances knowledge graph embedding performance compared to models that only use structural information.

Abstract: Recent studies on knowledge graph embedding focus on mapping entities and relations into low-dimensional vector spaces. While most existing models primarily exploit structural information, knowledge graphs also contain rich contextual and textual information that can enhance embedding effectiveness. In this work, we propose a novel model that integrates both contextual and textual signals into entity and relation embeddings through a graph convolutional network. To better utilize context, we introduce two metrics: confidence, computed via a rule-based method, and relatedness, derived from textual representations. These metrics enable more precise weighting of contextual information during embedding learning. Extensive experiments on two widely used benchmark datasets demonstrate the effectiveness of our approach, showing consistent improvements over strong baselines.

Method: Integrates rule-based and neural methods to parse questions into semantic segment sequences, formulates parsing as sequence generation using encoder-decoder network, and employs graph neural network to leverage KG context for better implicit entity/relation identification.

Result: Experimental evaluations on two benchmark datasets demonstrate the model’s effectiveness and superior performance in semantic parsing for knowledge graph question answering.

Conclusion: The proposed framework successfully handles complex question parsing challenges and shows strong performance in knowledge graph question answering through integrated rule-based and neural approaches with graph context enrichment.

Abstract: In this paper, we propose a novel method for question answering over knowledge graphs based on graph-to-segment mapping, designed to improve the understanding of natural language questions. Our approach is grounded in semantic parsing, a key technique for interpreting question utterances. The main challenges arise from handling implicit entities and relations, as well as complex constraints such as temporal conditions, ordinality, and aggregation within the context of a knowledge graph. To address these issues, our framework integrates both rule-based and neural methods to parse and construct accurate, comprehensive semantic segment sequences. These sequences are then assembled into semantic query graphs, providing precise representations of question utterances. We formulate question semantic parsing as a sequence generation task, employing an encoder-decoder neural network to map natural language questions into semantic segments. Furthermore, to enhance the identification of implicit entities and relations, we incorporate a graph neural network that leverages knowledge graph context to enrich question representations. Experimental evaluations on two benchmark datasets demonstrate the effectiveness and superior performance of our model in semantic parsing for knowledge graph question answering.

Method: Developed fine-tuned BERT-based model trained on Twitter texts to distinguish gun violence reports from ordinary Portuguese texts. Integrated into web application and tested in live intervention with Brazilian analysts.

Result: Qualitative: All analysts used time more efficiently and expanded search capacities. Quantitative: Model use associated with increased interactions with users reporting gun violence.

Conclusion: Human-centered interventions using language models can effectively support human rights organizations’ work in monitoring real-world firearm events.

Abstract: Gun violence is a pressing human rights issue that affects nearly every dimension of the social fabric, from healthcare and education to psychology and the economy. Reliable data on firearm events is paramount to developing more effective public policy and emergency responses. However, the lack of comprehensive databases and the risks of in-person surveys prevent human rights organizations from collecting needed data in most countries. Here, we partner with a Brazilian human rights organization to conduct a systematic evaluation of language models to assist with monitoring real-world firearm events from social media data. We propose a fine-tuned BERT-based model trained on Twitter (now X) texts to distinguish gun violence reports from ordinary Portuguese texts. We then incorporate our model into a web application and test it in a live intervention. We study and interview Brazilian analysts who continuously check social media texts to identify new gun violence events. Qualitative assessments show that our solution helped all analysts use their time more efficiently and expanded their search capacities. Quantitative assessments show that the use of our model was associated with analysts having further interactions with online users reporting gun violence. Our findings suggest that human-centered interventions using language models can help support the work of human rights organizations.

Method: Comprehensive linguistic analysis comparing vocabulary, Part-Of-Speech distribution, dependency distribution, and sentiment of texts generated by GPT-3.5, GPT-4, and Bard against diverse inputs.

Result: Significant linguistic variations were found that enable attribution of text to its LLM origin with 88% accuracy using off-the-shelf classification models.

Conclusion: LLMs do exhibit distinctive linguistic styles, which has important theoretical and practical implications for AI-generated text detection and model identification.

Abstract: Large Language Models (LLMs) are capable of generating text that is similar to or surpasses human quality. However, it is unclear whether LLMs tend to exhibit distinctive linguistic styles akin to how human authors do. Through a comprehensive linguistic analysis, we compare the vocabulary, Part-Of-Speech (POS) distribution, dependency distribution, and sentiment of texts generated by three of the most popular LLMS today (GPT-3.5, GPT-4, and Bard) to diverse inputs. The results point to significant linguistic variations which, in turn, enable us to attribute a given text to its LLM origin with a favorable 88% accuracy using a simple off-the-shelf classification model. Theoretical and practical implications of this intriguing finding are discussed.

Method: Using LLaMA-2 models, researchers employed prompting strategies with MQM quality feedback and fine-tuned the LLM to better utilize this guidance for MT post-editing across multiple language pairs.

Result: Prompting LLMs to post-edit MT improved TER, BLEU and COMET scores. Fine-tuning helped integrate fine-grained feedback more effectively and further improved translation quality in both automatic and human evaluations.

Conclusion: Combining LLMs with quality feedback from supervised MT systems through prompting and fine-tuning can effectively improve machine translation quality, though the benefits of fine-grained feedback require fine-tuning to be fully realized.

Abstract: Machine Translation (MT) remains one of the last NLP tasks where large language models (LLMs) have not yet replaced dedicated supervised systems. This work exploits the complementary strengths of LLMs and supervised MT by guiding LLMs to automatically post-edit MT with external feedback on its quality, derived from Multidimensional Quality Metric (MQM) annotations. Working with LLaMA-2 models, we consider prompting strategies varying the nature of feedback provided and then fine-tune the LLM to improve its ability to exploit the provided guidance. Through experiments on Chinese-English, English-German, and English-Russian MQM data, we demonstrate that prompting LLMs to post-edit MT improves TER, BLEU and COMET scores, although the benefits of fine-grained feedback are not clear. Fine-tuning helps integrate fine-grained feedback more effectively and further improves translation quality based on both automatic and human evaluation.

Method: TransLLM uses translation chain-of-thought to divide transfer into sub-tasks, enhanced with public data. It employs low-rank adaptation to maintain original parameters and recovery KD using data generated by the chat LLM itself to preserve original knowledge.

Result: Transforming LLaMA-2-chat-7B to Thai, the method outperformed strong baselines and ChatGPT on multi-turn MT-bench using only single-turn data. It also rejected more harmful queries on AdvBench safety benchmark than both ChatGPT and GPT-4 without safety data.

Conclusion: TransLLM effectively transfers advanced abilities from English chat LLMs to non-English languages while preventing catastrophic forgetting, demonstrating superior performance in both helpfulness and safety compared to existing models.

Abstract: The scarcity of non-English data limits the development of non-English large language models (LLMs). Transforming English-centric LLMs to non-English has been identified as an effective and resource-efficient method. Previous works start from base LLMs and perform knowledge distillation (KD) with data generated by stronger LLMs, e.g. GPT-4. Compared to base LLMs, chat LLMs are further optimized for advanced abilities, e.g. multi-turn conversation and human preference alignment, and thus more powerful in both helpfulness and safety. However, transforming a chat LLM involves two critical issues: (1) How can we effectively transfer advanced abilities without their supervised data? (2) How can we prevent the original knowledge from catastrophic forgetting during transformation? We target these issues by introducing a simple framework called TransLLM. For the first issue, TransLLM divides the transfer problem into some common sub-tasks with the translation chain-of-thought, which uses the translation as the bridge between English and non-English step-by-step. We further enhance the performance of sub-tasks with publicly available data. For the second issue, we propose a method comprising two synergistic components: low-rank adaptation for training to maintain the original LLM parameters, and recovery KD, which utilizes data generated by the chat LLM itself to recover the original knowledge from the frozen parameters. In the experiments, we transform the LLaMA-2-chat-7B to the Thai language. Our method, using only single-turn data, outperforms strong baselines and ChatGPT on multi-turn benchmark MT-bench. Furthermore, our method, without safety data, rejects more harmful queries of safety benchmark AdvBench than both ChatGPT and GPT-4. Code is available at https://github.com/hy5468/TransLLM.

Method: Proposes vocabulary projections to inspect concepts encoded in parameters, localizes concept vectors, and creates ConceptVectors benchmark with hundreds of concepts and their parametric traces in open-source LLMs.

Result: Existing unlearning methods minimally impact concept vectors and mostly suppress them during inference. Direct ablation of concept vectors removes associated knowledge and reduces susceptibility to adversarial manipulation.

Conclusion: Behavioral-based unlearning evaluations have limitations. Future work should include parameter-based evaluations. The authors release code and benchmark to support this approach.

Abstract: The task of “unlearning” certain concepts in large language models (LLMs) has attracted immense attention recently, due to its importance in mitigating undesirable model behaviours, such as the generation of harmful, private, or incorrect information. Current protocols to evaluate unlearning methods largely rely on behavioral tests, without monitoring the presence of unlearned knowledge within the model’s parameters. This residual knowledge can be adversarially exploited to recover the erased information post-unlearning. We argue that unlearning should also be evaluated internally, by considering changes in the parametric knowledge traces of the unlearned concepts. To this end, we propose a general evaluation methodology that leverages vocabulary projections to inspect concepts encoded in model parameters. We use this approach to localize “concept vectors” - parameter vectors that encode concrete concepts - and construct ConceptVectors, a benchmark dataset containing hundreds of common concepts and their parametric knowledge traces within two open-source LLMs. Evaluation on ConceptVectors shows that existing unlearning methods minimally impact concept vectors and mostly suppress them during inference, while directly ablating these vectors demonstrably removes the associated knowledge and significantly reduces the model’s susceptibility to adversarial manipulation. Our results highlight limitations in behavioral-based unlearning evaluations and call for future work to include parameter-based evaluations. To support this, we release our code and benchmark at https://github.com/yihuaihong/ConceptVectors.

Method: Proposed MEGen, an editing-based generative backdoor that expands backdoors to generative tasks in a unified any-text-to-any-text format. It adjusts only a small set of local parameters with few-shot samples to achieve natural generations with specific intentions.

Result: MEGen achieves high attack success rate. The backdoored model can freely output pre-set dangerous information while completing downstream tasks when triggered.

Conclusion: MEGen enables backdoors in LLMs to exhibit generative capabilities, causing potential safety risks by altering generative style, highlighting the expanded threat landscape of backdoored LLMs.

Abstract: Large language models (LLMs) have exhibited remarkable versatility and adaptability, while their widespread adoption across various applications also raises critical safety concerns. This paper focuses on the impact of backdoored LLMs. Traditional backdoor injection methods are primarily limited to yes-or-no discriminative tasks, leading users to underestimate the potential risks of backdoored LLMs. Given the inherently generative nature of LLMs, this paper reveals that a generative backdoor injected into LLMs can expose the true safety risks in their applications. We propose an editing-based generative backdoor, named MEGen, aiming to expand the backdoor to generative tasks in a unified format of any text-to any text, leading to natural generations with a specific intention. Experiments show that MEGen achieves a high attack success rate by adjusting only a small set of local parameters with few-shot samples. Notably, we show that the backdoored model, when triggered, can freely output pre-set dangerous information while completing downstream tasks. Our work highlights that MEGen enables backdoors in LLMs to exhibit generative capabilities, causing potential safety risks by altering the generative style. The code is available at https://github.com/MonoQ-hub/MEGen.

Method: The Diagram of Thought framework allows LLMs to construct dynamic diagrams of ideas, enabling them to propose multiple lines of thought, self-critique steps, and synthesize insights. Grounded in category theory for logical consistency.

Result: Creates a more powerful and transparent reasoning process that produces fully auditable, step-by-step traces of the LLM’s thinking without requiring external controllers.

Conclusion: DoT bridges the gap between fluent language generation and formal reasoning, providing a self-contained, efficient framework for complex problem-solving with guaranteed logical consistency.

Abstract: Large Language Models (LLMs) excel at many tasks but often falter on complex problems that require structured, multi-step reasoning. We introduce the Diagram of Thought (DoT), a new framework that enables a single LLM to build and navigate a mental map of its reasoning. Instead of thinking in a straight line, the model constructs a dynamic diagram of ideas, where it can propose different lines of thought, critique its own steps, and synthesize validated insights into a final conclusion. This entire process is self-contained within the model, making it highly efficient by avoiding the complex external controllers or search algorithms required by other methods. To ensure the reliability of this process, we ground DoT in a rigorous mathematical framework from category theory. This foundation guarantees that the way the model combines information is logical, consistent, and robust, regardless of the order in which ideas were explored. The result is a more powerful and transparent reasoning process that produces a fully auditable, step-by-step trace of the LLM’s thinking, bridging the gap between fluent language and formal reasoning.

Method: ORACLE training objective with intra-class clustering and inter-class separation to enforce orthogonality between semantic and language embeddings.

Result: ORACLE effectively reduces semantic leakage and enhances semantic alignment in cross-lingual retrieval and semantic textual similarity tasks.

Conclusion: The proposed orthogonality constraint learning approach successfully mitigates semantic leakage and improves cross-lingual representation alignment.

Abstract: Accurately aligning contextual representations in cross-lingual sentence embeddings is key for effective parallel data mining. A common strategy for achieving this alignment involves disentangling semantics and language in sentence embeddings derived from multilingual pre-trained models. However, we discover that current disentangled representation learning methods suffer from semantic leakage - a term we introduce to describe when a substantial amount of language-specific information is unintentionally leaked into semantic representations. This hinders the effective disentanglement of semantic and language representations, making it difficult to retrieve embeddings that distinctively represent the meaning of the sentence. To address this challenge, we propose a novel training objective, ORthogonAlity Constraint LEarning (ORACLE), tailored to enforce orthogonality between semantic and language embeddings. ORACLE builds upon two components: intra-class clustering and inter-class separation. Through experiments on cross-lingual retrieval and semantic textual similarity tasks, we demonstrate that training with the ORACLE objective effectively reduces semantic leakage and enhances semantic alignment within the embedding space.

Method: Introduces new measures to characterize collapse directly from models’ next-token probability distributions. Proposes filtering training items by high surplexity to maximize model surprise, without needing to distinguish human vs AI-generated data.

Result: Experiments show collapse degree depends on initial training set complexity and autophagy extent. The surplexity-based filtering strategy is at least as effective as human-data baselines and more effective in reducing distributional skewedness.

Conclusion: Model collapse occurs when models train on unsurprising data. The proposed surplexity-based approach provides effective mitigation without requiring data authorship identification, offering more resilient training for AI systems in synthetic-data-saturated environments.

Abstract: As synthetic content increasingly infiltrates the web, generative AI models may be retrained on their own outputs: a process termed “autophagy”. This leads to model collapse: a progressive loss of performance and diversity across generations. Recent studies have examined the emergence of model collapse across various generative AI models and data types, and have proposed mitigation strategies that rely on incorporating human-authored content. However, current characterizations of model collapse remain limited, and existing mitigation methods assume reliable knowledge of whether training data is human-authored or AI-generated. In this paper, we address these gaps by introducing new measures that characterise collapse directly from a model’s next-token probability distributions, rather than from properties of AI-generated text. Using these measures, we show that the degree of collapse depends on the complexity of the initial training set, as well as on the extent of autophagy. Our experiments prompt a new suggestion: that model collapse occurs when a model trains on data that does not “surprise” it. We express this hypothesis in terms of the well-known Free Energy Principle in cognitive science. Building on this insight, we propose a practical mitigation strategy: filtering training items by high surplexity, maximising the surprise of the model. Unlike existing methods, this approach does not require distinguishing between human- and AI-generated data. Experiments across datasets and models demonstrate that our strategy is at least as effective as human-data baselines, and even more effective in reducing distributional skewedness. Our results provide a richer understanding of model collapse and point toward more resilient approaches for training generative AI systems in environments increasingly saturated with synthetic data.

Method: Simulated different retrieval and selection conditions through controlled mixture of gold and distractor knowledge to assess impact on generation outcomes.

Result: Generator model capability and task/dataset complexity significantly influence knowledge selection impact. Strong generators on clear tasks benefit from recall improvement, while weaker generators on ambiguous tasks require better F1 scores from selection.

Conclusion: Knowledge selection’s importance varies based on system context - it provides limited benefit with strong generators on clear tasks but becomes critical for weaker generators or ambiguous tasks where F1 score matters more.

Abstract: Retrieval-augmented generation (RAG) is a powerful method for enhancing natural language generation by integrating external knowledge into a model’s output. While prior work has demonstrated the importance of improving knowledge retrieval for boosting generation quality, the role of knowledge selection, a.k.a. reranking or filtering, remains less clear. This paper empirically analyzes how knowledge selection influences downstream generation performance in RAG systems. By simulating different retrieval and selection conditions through a controlled mixture of gold and distractor knowledge, we assess the impact of these factors on generation outcomes. Our findings indicate that the downstream generator model’s capability, as well as the complexity of the task and dataset, significantly influence the impact of knowledge selection on the overall RAG system performance. In typical scenarios, improving the knowledge recall score is key to enhancing generation outcomes, with the knowledge selector providing limited benefit when a strong generator model is used on clear, well-defined tasks. For weaker generator models or more ambiguous tasks and datasets, the knowledge F1 score becomes a critical factor, and the knowledge selector plays a more prominent role in improving overall performance.

Method: Uses code as intermediary to translate visual chart representations into textual representations, enabling text-based synthesizing techniques to generate high-quality Q&A pairs from chart-plotting code.

Result: Created ReachQA dataset with 3k reasoning-intensive charts and 20k Q&A pairs. Models fine-tuned with ReachQA perform well on chart tasks and show gains on general reasoning benchmarks.

Conclusion: CIT provides an efficient and scalable data synthesis approach that successfully distills visual reasoning abilities from LLMs to MLLMs, enhancing both chart-specific and general reasoning performance.

Abstract: Solving complex chart Q&A tasks requires advanced visual reasoning abilities in multimodal large language models (MLLMs), including recognizing key information from visual inputs and conducting reasoning over it. While fine-tuning MLLMs for reasoning is critical, collecting and annotating charts and questions is expensive, hard to scale, and often results in low-quality annotations. To address this, we propose Code-as-Intermediary Translation (CIT), a cost-effective, efficient and scalable data synthesis method for distilling visual reasoning abilities from LLMs to MLLMs. The code serves as an intermediary that translates visual chart representations into textual representations, enabling language models to understand cross-modal information and generate reasoning chains accordingly. In this way, we can employ text-based synthesizing techniques to expand chart-plotting code and generate high-quality Q&A pairs for training models. This produces ReachQA, a dataset containing 3k reasoning-intensive charts and 20k Q&A pairs to enhance both recognition and reasoning abilities of MLLMs. Experiments show that models fine-tuned with ReachQA not only perform well on chart-related tasks but also show performance gains on general reasoning benchmarks. The code and dataset are publicly available at https://github.com/hewei2001/ReachQA.

Method: LLM-enriched algorithm merges individual codebooks, then measures each coder’s contribution using four metrics: Coverage, Overlap, Novelty, and Divergence against the merged result.

Result: Experiments revealed the merging algorithm’s impact, validated metric stability across multiple runs and LLMs, and demonstrated ability to diagnose coding issues like excessive or hallucinated codes.

Conclusion: Provides a reliable pathway for ensuring methodological rigor in human-AI qualitative analysis by offering computational evaluation of inductive coding results.

Abstract: Qualitative analysis is critical to understanding human datasets in many social science disciplines. A central method in this process is inductive coding, where researchers identify and interpret codes directly from the datasets themselves. Yet, this exploratory approach poses challenges for meeting methodological expectations (such as depth'' and variation’’), especially as researchers increasingly adopt Generative AI (GAI) for support. Ground-truth-based metrics are insufficient because they contradict the exploratory nature of inductive coding, while manual evaluation can be labor-intensive. This paper presents a theory-informed computational method for measuring inductive coding results from humans and GAI. Our method first merges individual codebooks using an LLM-enriched algorithm. It measures each coder’s contribution against the merged result using four novel metrics: Coverage, Overlap, Novelty, and Divergence. Through two experiments on a human-coded online conversation dataset, we 1) reveal the merging algorithm’s impact on metrics; 2) validate the metrics’ stability and robustness across multiple runs and different LLMs; and 3) showcase the metrics’ ability to diagnose coding issues, such as excessive or irrelevant (hallucinated) codes. Our work provides a reliable pathway for ensuring methodological rigor in human-AI qualitative analysis.

Method: Integrates Hidden Markov Models (HMMs) with LLMs to extract intent transition patterns from e-commerce chat logs, parameterize emission probabilities, and uses multi-task contrastive learning (MINT-CL) for classification.

Result: Outperforms competitive baselines in dialogue generation quality and classification accuracy, especially in multilingual settings, and releases MINT-E multilingual dialogue corpus.

Conclusion: The framework successfully generates high-quality multilingual dialogues and improves intent classification performance while reducing reliance on large annotated datasets.

Abstract: In conversational AI systems, a critical challenge in training effective multi-turn intent classification models lies in the generation of large-scale, domain-specific, multilingual dialogue datasets. In this paper, we introduce Chain-of-Intent, a novel framework that integrates Hidden Markov Models (HMMs) with Large Language Models (LLMs) to generate intent-driven, context-aware dialogues through self-play. Our method first extracts domain-specific intent transition patterns from real-world e-commerce chat logs, which guide the modeling of turn-level dynamics and intent sequences. LLMs are then employed to parameterize the emission probabilities of HMMs, enabling the generation of natural, coherent utterances aligned with predicted intents and dialogue context. We also propose MINT-CL, a multi-task contrastive learning framework for multi-turn intent classification, which improves performance while reducing dependence on large-scale annotated datasets. Empirical results demonstrate that our approach outperforms competitive baselines in dialogue generation quality and classification accuracy, particularly in multilingual settings. To facilitate future research, we release MINT-E, a comprehensive, multilingual, intent-aware multi-turn dialogue corpus derived from the e-commerce domain\footnote{The reproduced source code and dataset are available at https://github.com/junhua/chain-of-intent.

Method: Proposed AgoraBench benchmark with standardized settings and metrics, synthesized 1.26M training instances using 6 LMs, and trained 99 student models to evaluate data generation capabilities.

Result: LMs show distinct strengths (GPT-4o excels at new problems, Claude-3.5-Sonnet better at enhancing existing ones), data generation ability doesn’t correlate with problem-solving ability, and intrinsic data quality features are better indicators.

Conclusion: Strategic choices in output format and cost-conscious model selection significantly impact data generation effectiveness, with multiple intrinsic data quality features serving as reliable indicators of generation capability.

Abstract: Given the increasing use of synthetic data in language model (LM) post-training, an LM’s ability to generate high-quality data has become nearly as crucial as its ability to solve problems directly. While prior works have focused on developing effective data generation methods, they lack systematic comparison of different LMs as data generators in a unified setting. To address this gap, we propose AgoraBench, a benchmark that provides standardized settings and metrics to evaluate LMs’ data generation abilities. Through synthesizing 1.26 million training instances using 6 LMs and training 99 student models, we uncover key insights about LMs’ data generation capabilities. First, we observe that LMs exhibit distinct strengths. For instance, GPT-4o excels at generating new problems, while Claude-3.5-Sonnet performs better at enhancing existing ones. Furthermore, our analysis reveals that an LM’s data generation ability doesn’t necessarily correlate with its problem-solving ability. Instead, multiple intrinsic features of data quality-including response quality, perplexity, and instruction difficulty-collectively serve as better indicators. Finally, we demonstrate that strategic choices in output format and cost-conscious model selection significantly impact data generation effectiveness.

Method: Systematic evaluation of LLM applications in low-resource language research, analyzing technical frameworks, current methodologies, ethical considerations, and interdisciplinary approaches.

Result: Identified key challenges including data accessibility, model adaptability, and cultural sensitivity, while highlighting the transformative potential of LLMs for linguistic variation analysis, historical documentation, and cultural expression studies.

Conclusion: Interdisciplinary collaboration and development of customized models are promising avenues for advancing low-resource language research, integrating AI with humanities to preserve humanity’s linguistic and cultural heritage.

Abstract: Low-resource languages serve as invaluable repositories of human history, embodying cultural evolution and intellectual diversity. Despite their significance, these languages face critical challenges, including data scarcity and technological limitations, which hinder their comprehensive study and preservation. Recent advancements in large language models (LLMs) offer transformative opportunities for addressing these challenges, enabling innovative methodologies in linguistic, historical, and cultural research. This study systematically evaluates the applications of LLMs in low-resource language research, encompassing linguistic variation, historical documentation, cultural expressions, and literary analysis. By analyzing technical frameworks, current methodologies, and ethical considerations, this paper identifies key challenges such as data accessibility, model adaptability, and cultural sensitivity. Given the cultural, historical, and linguistic richness inherent in low-resource languages, this work emphasizes interdisciplinary collaboration and the development of customized models as promising avenues for advancing research in this domain. By underscoring the potential of integrating artificial intelligence with the humanities to preserve and study humanity’s linguistic and cultural heritage, this study fosters global efforts towards safeguarding intellectual diversity.

Method: Used both static and interactive benchmarking methods to evaluate state-of-the-art LLMs like Llama 3 and GPT-4o.

Result: LLMs can approximate dialogues at word and utterance level but fail to reproduce discursive patterns, exaggerate alignment, and lack human-level diversity.

Conclusion: This work aims to initiate development of a comprehensive benchmark for LLMs in child-oriented applications, highlighting current limitations in simulating authentic child-caregiver interactions.

Abstract: LLMs can generate human-like dialogues, yet their ability to simulate early child-adult interactions remains largely unexplored. In this paper, we examined how effectively LLMs can capture the distinctive features of child-caregiver language in interaction, using both static and interactive benchmarking methods. We found that state-of-the-art LLMs like Llama 3 and GPT-4o can approximate child-caregiver dialogues at the word and utterance level, but they struggle to reproduce the child and caregiver’s discursive patterns, exaggerate alignment, and fail to reach the level of diversity shown by humans. The broader goal of this work is to initiate the development of a comprehensive benchmark for LLMs in child-oriented applications.

Method: Two-stage approach using instruction-tuned LLMs: 1) Generate truth-preserving replacement candidates ranked by abstraction level, 2) Evaluate privacy protection through LLM inference attacks and select the most informative resistant candidate.

Result: Enhanced utility with only marginal (<1 p.p.) increase in re-identification risk compared to full suppression, and more truth-preserving than existing methods like Microsoft Presidio’s synthetic replacements.

Conclusion: The proposed generalization-based approach using LLMs effectively balances privacy and utility in text sanitization, outperforming existing methods while maintaining minimal privacy risks.

Abstract: Text sanitization aims to rewrite parts of a document to prevent disclosure of personal information. The central challenge of text sanitization is to strike a balance between privacy protection (avoiding the leakage of personal information) and utility preservation (retaining as much as possible of the document’s original content). To this end, we introduce a novel text sanitization method based on generalizations, that is, broader but still informative terms that subsume the semantic content of the original text spans. The approach relies on the use of instruction-tuned large language models (LLMs) and is divided into two stages. Given a document including text spans expressing personally identifiable information (PII), the LLM is first applied to obtain truth-preserving replacement candidates for each text span and rank those according to their abstraction level. Those candidates are then evaluated for their ability to protect privacy by conducting inference attacks with the LLM. Finally, the system selects the most informative replacement candidate shown to be resistant to those attacks. This two-stage process produces replacements that effectively balance privacy and utility. We also present novel metrics to evaluate these two aspects without needing to manually annotate documents. Results on the Text Anonymization Benchmark show that the proposed approach, implemented with Mistral 7B Instruct, leads to enhanced utility, with only a marginal (< 1 p.p.) increase in re-identification risk compared to fully suppressing the original spans. Furthermore, our approach is shown to be more truth-preserving than existing methods such as Microsoft Presidio’s synthetic replacements.

Method: Used answering question while seeing a picture task to test comprehension of two-level recursive structures among children aged 3 to 7 years.

Result: Children do not reach adult-like proficiency in two-level recursion until age 6, with significant asymmetry between recursive possessives and locatives acquisition.

Conclusion: Structural complexity and cognitive factors play primary roles in language acquisition, highlighting the importance of recursion in child language development.

Abstract: As recursion has been underlying any linguistic work for the last 60 years, the acquisition of recursive structures by children during language learning has become a focal point of inquiry. This study delves into the developmental trajectory of Mandarin-speaking children’s acquisition of recursive possessives and locatives, assessing the impact of structural diversity on language acquisition. The research contrasts the comprehension of two-level recursive structures among children aged 3 to 7 years, employing answering question while seeing a picture task to elicit responses. The findings indicate that children do not attain adult-like proficiency in two-level recursion until the age of 6, and there exists a notable asymmetry in the acquisition of recursive possessives versus locatives. These results underscore the primacy of structural complexity and cognitive factors in the acquisition process, enhancing our comprehension of the cognitive foundations of language development and the pivotal role of recursion in child language acquisition.

Method: Fine-tuned models initially trained on typologically similar high-resource languages using limited target language data. Experiments conducted on five diverse language pairs spanning Semitic, Bantu, Romance, Slavic language families and a language isolate. Varied hyperparameters (learning rate, batch size, epochs, weight decay) to ensure robustness.

Result: Transfer learning consistently improved translation quality across all language pairs, confirming applicability beyond closely related languages. Moderate batch sizes (e.g., 32) optimal for similar pairs, smaller sizes benefited less similar pairs. Excessively high learning rates destabilized training.

Conclusion: Provides empirical evidence for transfer learning’s generalizability across language families and offers practical guidance for building machine translation systems in low-resource settings with minimal tuning effort.

Abstract: This study examines the cross-linguistic effectiveness of transfer learning for low-resource machine translation by fine-tuning models initially trained on typologically similar high-resource languages, using limited data from the target low-resource language. We hypothesize that linguistic similarity enables efficient adaptation, reducing the need for extensive training data. To test this, we conduct experiments on five typologically diverse language pairs spanning distinct families: Semitic (Modern Standard Arabic to Levantine Arabic), Bantu (Hausa to Zulu), Romance (Spanish to Catalan), Slavic (Slovak to Macedonian), and a language isolate (Eastern Armenian to Western Armenian). Results show that transfer learning consistently improves translation quality across all pairs, confirming its applicability beyond closely related languages. As a secondary analysis, we vary key hyperparameters learning rate, batch size, number of epochs, and weight decay to ensure results are not dependent on a single configuration. We find that moderate batch sizes (e.g., 32) are often optimal for similar pairs, smaller sizes benefit less similar pairs, and excessively high learning rates can destabilize training. These findings provide empirical evidence for the generalizability of transfer learning across language families and offer practical guidance for building machine translation systems in low-resource settings with minimal tuning effort.

Method: Evaluated GPT-4 and LLaMA-3 on story generation, introduced Sui Generis score to measure plot element uniqueness, analyzed 100 short stories, and conducted human evaluation comparing automatic scores with human judgment of surprise.

Result: LLM-generated stories contain frequently echoed plot elements across generations and different models, while human-written stories show rare plot recreation. Sui Generis scores moderately correlate with human surprise judgments.

Conclusion: Current LLMs struggle with generating truly diverse creative content, showing pattern repetition that limits their ability to enhance collective creativity compared to human originality.

Abstract: With rapid advances in large language models (LLMs), there has been an increasing application of LLMs in creative content ideation and generation. A critical question emerges: can current LLMs provide ideas that are diverse enough to truly bolster collective creativity? We examine two state-of-the-art LLMs, GPT-4 and LLaMA-3, on story generation and discover that LLM-generated stories often consist of plot elements that are echoed across a number of generations. To quantify this phenomenon, we introduce the Sui Generis score, an automatic metric that measures the uniqueness of a plot element among alternative storylines generated using the same prompt under an LLM. Evaluating on 100 short stories, we find that LLM-generated stories often contain combinations of idiosyncratic plot elements echoed frequently across generations and across different LLMs, while plots from the original human-written stories are rarely recreated or even echoed in pieces. Moreover, our human evaluation shows that the ranking of Sui Generis scores among story segments correlates moderately with human judgment of surprise level, even though score computation is completely automatic without relying on human judgment.

Method: Partially abstract initial reasoning steps using latent discrete tokens from VQ-VAE, train models with hybrid data mixing latent and text tokens, extend vocabulary with unseen latent tokens.

Result: Consistently outperforms baseline methods in various benchmarks including Keys-Finding Maze problem and logical/mathematical reasoning problems.

Conclusion: Hybrid representation with latent tokens effectively reduces reasoning trace length while maintaining or improving reasoning performance, with simple training procedure enabling fast adaptation.

Abstract: Large Language Models (LLMs) excel at reasoning and planning when trained on chainof-thought (CoT) data, where the step-by-step thought process is explicitly outlined by text tokens. However, this results in lengthy inputs where many words support textual coherence rather than core reasoning information, and processing these inputs consumes substantial computation resources. In this work, we propose a hybrid representation of the reasoning process, where we partially abstract away the initial reasoning steps using latent discrete tokens generated by VQ-VAE, significantly reducing the length of reasoning traces. We explore the use of latent trace abstractions in two scenarios: 1) training the model from scratch for the Keys-Finding Maze problem, 2) fine-tuning LLMs on this hybrid data with an extended vocabulary including unseen latent tokens, for both logical and mathematical reasoning problems. To facilitate effective learning, we introduce a simple training procedure that randomly mixes latent and text tokens, which enables fast adaptation to new latent tokens. Our approach consistently outperforms the baselines methods in various benchmarks.

Method: Comprehensive overview and categorization of four key approaches: dynamic knowledge injection, static knowledge embedding, modular adapters, and prompt optimization.

Result: Provides analysis of trade-offs between flexibility, scalability, and efficiency for each method, along with evaluation of domain-specific LLMs vs general LLMs.

Conclusion: Highlights challenges and opportunities in specialized LLM field, maintains open-source repository for ongoing research documentation, and summarizes commonly used datasets and benchmarks.

Abstract: Large Language Models (LLMs) have demonstrated remarkable success in various tasks such as natural language understanding, text summarization, and machine translation. However, their general-purpose nature often limits their effectiveness in domain-specific applications that require specialized knowledge, such as healthcare, chemistry, or legal analysis. To address this, researchers have explored diverse methods to enhance LLMs by integrating domain-specific knowledge. In this survey, we provide a comprehensive overview of these methods, which we categorize into four key approaches: dynamic knowledge injection, static knowledge embedding, modular adapters, and prompt optimization. Each approach offers unique mechanisms to equip LLMs with domain expertise, balancing trade-offs between flexibility, scalability, and efficiency. We discuss how these methods enable LLMs to tackle specialized tasks, compare their advantages and disadvantages, evaluate domain-specific LLMs against general LLMs, and highlight the challenges and opportunities in this emerging field. For those interested in delving deeper into this area, we also summarize the commonly used datasets and benchmarks. To keep researchers updated on the latest studies, we maintain an open-source at: https://github.com/abilliyb/Knowledge_Injection_Survey_Papers, dedicated to documenting research in the field of specialized LLM.

Method: Created a controlled evaluation environment for analogical reasoning parameterized across three dimensions: modality (textual, visual, symbolic), difficulty (easy, medium, hard), and task format (multiple-choice or free-text generation). Analyzed comparative dynamics of different inference pipelines and investigated advanced paradigms like hypothesis selection, verification, and refinement.

Result: The findings demonstrate that the comparative performance of different inference paradigms varies across the tested dimensions, and these findings generalize to broader in-context learning tasks. Advanced inference strategies show potential for scaling up logical inference in LLM reasoning.

Conclusion: This exploratory study provides a foundation for future research in enhancing LLM reasoning through systematic logical inference strategies, with resources made available for further development.

Abstract: Modern large language models (LLMs) employ various forms of logical inference, both implicitly and explicitly, when addressing reasoning tasks. Understanding how to optimally leverage these inference paradigms is critical for advancing LLMs’ reasoning capabilities. This paper adopts an exploratory approach by introducing a controlled evaluation environment for analogical reasoning – a fundamental cognitive task – that is systematically parameterized across three dimensions: modality (textual, visual, symbolic), difficulty (easy, medium, hard), and task format (multiple-choice or free-text generation). We analyze the comparative dynamics of inductive, abductive, and deductive inference pipelines across these dimensions, and demonstrate that our findings generalize to broader in-context learning tasks. Additionally, we investigate advanced paradigms such as hypothesis selection, verification, and refinement, revealing their potential to scale up logical inference in LLM reasoning. This exploratory study provides a foundation for future research in enhancing LLM reasoning through systematic logical inference strategies. Resources are available at https://github.com/HKUST-KnowComp/LogiDynamics.

Method: Proposed PIBE framework with gradual data selection strategy using representation-based diversity scores to capture data relationships and maintain historical information, allowing flexible combination of diversity and quality metrics.

Result: Extensive experiments show PIBE significantly outperforms baselines in InsBank evolution and effectively extracts budget-specific subsets, demonstrating both effectiveness and adaptability.

Conclusion: The PIBE framework successfully enables continuous evolution of instruction data repositories, providing an efficient and adaptable solution for maintaining LLM alignment over time through progressive data selection strategies.

Abstract: Large language models (LLMs) typically undergo instruction tuning to enhance alignment. Recent studies emphasize that quality and diversity of instruction data are more crucial than quantity, highlighting the need to select diverse, high-quality subsets to reduce training costs. However, how to evolve these selected subsets alongside the development of new instruction data remains insufficiently explored. To achieve LLMs’ ongoing alignment, we introduce Instruction Bank (\textbf{InsBank}), a continuously updated repository that integrates the latest valuable instruction data. We further propose Progressive Instruction Bank Evolution (\textbf{PIBE}), a novel framework designed to evolve InsBank effectively and efficiently over time. PIBE employs a gradual data selection strategy to maintain long-term efficiency, leveraging a representation-based diversity score to capture relationships between data points and retain historical information for comprehensive diversity evaluation. This also allows for flexible combination of diversity and quality scores during data selection and ranking. Extensive experiments demonstrate that PIBE significantly outperforms baselines in InsBank evolution and is able to extract budget-specific subsets, demonstrating its effectiveness and adaptability.

Method: LLM-assisted data generation using both open-weight and closed-weight models (selected GPT-4o as backbone), with full manual verification of each dataset entity. Fine-tuned Qwen, Falcon, and Gemma models on the created dataset.

Result: Consistent performance improvements in both multiple-choice and generative tasks across all fine-tuned models (Qwen, Falcon, Gemma), demonstrating effective LLM-assisted instruction tuning for low-resource languages.

Conclusion: The approach successfully creates high-quality instruction datasets for low-resource languages and shows that LLM-assisted instruction tuning can significantly enhance model performance in specialized domains like government and cultural knowledge.

Abstract: Instruction tuning in low-resource languages remains underexplored due to limited text data, particularly in government and cultural domains. To address this, we introduce and open-source a large-scale (10,600 samples) instruction-following (IFT) dataset, covering key institutional and cultural knowledge relevant to Kazakhstan. Our dataset enhances LLMs’ understanding of procedural, legal, and structural governance topics. We employ LLM-assisted data generation, comparing open-weight and closed-weight models for dataset construction, and select GPT-4o as the backbone. Each entity of our dataset undergoes full manual verification to ensure high quality. We also show that fine-tuning Qwen, Falcon, and Gemma on our dataset leads to consistent performance improvements in both multiple-choice and generative tasks, demonstrating the potential of LLM-assisted instruction tuning for low-resource languages.

Method: Used linear mixed-effects regression to compare LLM-generated response distributions against World Value Survey population-level opinion data across Danish, Dutch, English, and Portuguese languages, analyzing Google’s Gemma models (2B-27B) and OpenAI’s turbo-series.

Result: No consistent relationship between language capabilities and cultural alignment across model families. Gemma models showed positive correlation, while OpenAI models did not. Self-consistency was a stronger predictor of multicultural alignment than multilingual capabilities.

Conclusion: Achieving meaningful cultural alignment requires dedicated effort beyond improving general language capabilities, as multilingual proficiency alone does not ensure appropriate cultural representations.

Abstract: Large Language Models (LLMs) are becoming increasingly capable across global languages. However, the ability to communicate across languages does not necessarily translate to appropriate cultural representations. A key concern is US-centric bias, where LLMs reflect US rather than local cultural values. We propose a novel methodology that compares LLM-generated response distributions against population-level opinion data from the World Value Survey across four languages (Danish, Dutch, English, and Portuguese). Using a rigorous linear mixed-effects regression framework, we compare two families of models: Google’s Gemma models (2B–27B parameters) and successive iterations of OpenAI’s turbo-series. Across the families of models, we find no consistent relationships between language capabilities and cultural alignment. While the Gemma models have a positive correlation between language capability and cultural alignment across languages, the OpenAI models do not. Importantly, we find that self-consistency is a stronger predictor of multicultural alignment than multilingual capabilities. Our results demonstrate that achieving meaningful cultural alignment requires dedicated effort beyond improving general language capabilities.

Method: Empirical study of 21 input rewriting methods using 3 open-weight LLMs for English to 6 target languages translation, with text simplification as the primary strategy enhanced by quality estimation for translatability assessment.

Result: Text simplification was the most effective MT-agnostic rewrite strategy, and human evaluation confirmed that simplified rewrites and their MT outputs largely preserve original meaning of both source and translation.

Conclusion: LLM-assisted input rewriting represents a promising direction for improving translation quality, with text simplification showing particular effectiveness when combined with quality estimation techniques.

Abstract: Can we improve machine translation (MT) with LLMs by rewriting their inputs automatically? Users commonly rely on the intuition that well-written text is easier to translate when using off-the-shelf MT systems. LLMs can rewrite text in many ways but in the context of MT, these capabilities have been primarily exploited to rewrite outputs via post-editing. We present an empirical study of 21 input rewriting methods with 3 open-weight LLMs for translating from English into 6 target languages. We show that text simplification is the most effective MT-agnostic rewrite strategy and that it can be improved further when using quality estimation to assess translatability. Human evaluation further confirms that simplified rewrites and their MT outputs both largely preserve the original meaning of the source and MT. These results suggest LLM-assisted input rewriting as a promising direction for improving translations.

Method: Proposes Causal Diagnosticity framework using model-editing to generate faithful/unfaithful explanation pairs and evaluates metrics across fact-checking, analogy, object counting, and multi-hop reasoning tasks.

Result: Filler Tokens performed best overall, continuous metrics were more diagnostic than binary ones but sensitive to noise and model choice, with performance varying across tasks and models.

Conclusion: Current faithfulness metrics need improvement for robustness, highlighting the importance of standardized evaluation frameworks like Causal Diagnosticity.

Abstract: Large Language Models (LLMs) offer natural language explanations as an alternative to feature attribution methods for model interpretability. However, despite their plausibility, they may not reflect the model’s true reasoning faithfully, which is crucial for understanding the model’s true decision-making processes. Although several faithfulness metrics have been proposed, they are often evaluated in isolation, making direct, principled comparisons between them difficult. Here, we present Causal Diagnosticity, a framework that serves as a common testbed to evaluate faithfulness metrics for natural language explanations. Our framework employs the concept of diagnosticity, and uses model-editing methods to generate faithful-unfaithful explanation pairs. Our benchmark includes four tasks: fact-checking, analogy, object counting, and multi-hop reasoning. We evaluate prominent faithfulness metrics, including post-hoc explanation and chain-of-thought-based methods. We find that diagnostic performance varies across tasks and models, with Filler Tokens performing best overall. Additionally, continuous metrics are generally more diagnostic than binary ones but can be sensitive to noise and model choice. Our results highlight the need for more robust faithfulness metrics.

Method: Personalized Causal Graph Reasoning framework where LLMs construct individual-specific causal graphs from longitudinal data, traverse these graphs to identify relevant causal pathways, rank them by impact, simulate outcomes, and generate tailored responses.

Result: The method reduces postprandial glucose iAUC across three time windows compared to prior approaches, with LLM-as-a-judge evaluations confirming improvements in personalization quality for nutrient-oriented dietary recommendations.

Conclusion: The framework successfully enables LLMs to perform personalized reasoning over individual-specific data, demonstrating significant improvements in healthcare applications like personalized dietary recommendations and glucose management.

Abstract: Large Language Models (LLMs) excel at general-purpose reasoning by leveraging broad commonsense knowledge, but they remain limited in tasks requiring personalized reasoning over multifactorial personal data. This limitation constrains their applicability in domains such as healthcare, where decisions must adapt to individual contexts. We introduce Personalized Causal Graph Reasoning, a framework that enables LLMs to reason over individual-specific causal graphs constructed from longitudinal data. Each graph encodes how user-specific factors influence targeted outcomes. In response to a query, the LLM traverses the graph to identify relevant causal pathways, rank them by estimated impact, simulate potential outcomes, and generate tailored responses. We implement this framework in the context of nutrient-oriented dietary recommendations, where variability in metabolic responses demands personalized reasoning. Using counterfactual evaluation, we assess the effectiveness of LLM-generated food suggestions for glucose control. Our method reduces postprandial glucose iAUC across three time windows compared to prior approaches. Additional LLM-as-a-judge evaluations further confirm improvements in personalization quality.

Method: The article examines and exemplifies various gaps that manifest when LLMs interpret privacy policies, focusing on specific case studies or examples.

Result: Identified significant gaps in accuracy, completeness, clarity and representation when LLMs simplify privacy policies, showing current limitations.

Conclusion: While LLMs have potential to revolutionize privacy management through personal assistants and compliance automation, continued research is needed to address the identified gaps and realize their full potential.

Abstract: This article explores the gaps that can manifest when using a large language model (LLM) to obtain simplified interpretations of data practices from a complex privacy policy. We exemplify these gaps to showcase issues in accuracy, completeness, clarity and representation, while advocating for continued research to realize an LLM’s true potential in revolutionizing privacy management through personal assistants and automated compliance checking.

Method: Construct FormulaQA dataset with formula annotations from existing datasets, and develop TabAF framework that decodes both answers and formulas using a single LLM backbone to handle multiple task types and table structures.

Result: TabAF achieves new state-of-the-art performance on WikiTableQuestion, HiTab, and TabFact benchmarks under the same model size, demonstrating strong versatility and generalization.

Conclusion: Using spreadsheet formulas as executable representations provides an effective and versatile approach for complex table reasoning tasks across different table structures and question types.

Abstract: Advanced table question answering (TableQA) methods prompt large language models (LLMs) to generate answer text, SQL query, Python code, or custom operation, which impressively improve the complex reasoning problems in the TableQA task. However, these methods lack the versatility to cope with specific question types or table structures. In contrast, the Spreadsheet Formula, the widely used and well-defined operation language for tabular data, has not been thoroughly explored to solve TableQA. In this paper, we first attempt to use the Formula as the executable representation for solving complex reasoning on tables with different structures. Specifically, we construct \texttt{FromulaQA}, a large Formula-annotated TableQA dataset from existing datasets. In addition, we propose \texttt{TabAF}, a general table answering framework to solve multiple types of tasks over multiple types of tables simultaneously, which decodes answers and Formulas with a single LLM backbone. Extensive experiments demonstrate the versatility and generalization of \texttt{TabAF}. Under the same model size, \texttt{TabAF} achieves new state-of-the-art performance on the WikiTableQuestion, HiTab, and TabFact.

Method: Uses an innovative training framework integrating three components: masked language modeling, adversarial training, and knowledge distillation. Pre-trained on a curated Wikipedia corpus spanning 107 languages.

Result: Achieves 7.72% average relative improvement over traditional baselines (which only achieved 1.17%) across four NLP tasks (named entity recognition, natural language inference, question answering, semantic textual similarity). Statistical significance confirmed with p-value = 0.0181.

Conclusion: The combination of adversarial training and knowledge distillation effectively builds scalable and robust multilingual language models, advancing cross-lingual NLP capabilities while reducing computational requirements.

Abstract: This paper presents UniBERT, a compact multilingual language model that uses an innovative training framework that integrates three components: masked language modeling, adversarial training, and knowledge distillation. Pre-trained on a meticulously curated Wikipedia corpus spanning 107 languages, UniBERT is designed to reduce the computational demands of large-scale models while maintaining competitive performance across various natural language processing tasks. Comprehensive evaluations on four tasks - named entity recognition, natural language inference, question answering, and semantic textual similarity - demonstrate that our multilingual training strategy enhanced by an adversarial objective significantly improves cross-lingual generalization. Specifically, UniBERT models show an average relative improvement of 7.72% over traditional baselines, which achieved an average relative improvement of only 1.17%, and statistical analysis confirms the significance of these gains (p-value = 0.0181). This work highlights the benefits of combining adversarial training and knowledge distillation to build scalable and robust language models, thus advancing the field of multilingual and cross-lingual natural language processing.

Method: Created Misleading ChartQA benchmark with 3,026 curated examples spanning 21 misleader types and 10 chart types, each with standardized chart code, CSV data, multiple-choice questions, and labeled explanations. Benchmarked 24 state-of-the-art MLLMs and proposed a novel region-aware reasoning pipeline.

Result: The study benchmarks 24 MLLMs and analyzes their performance across different misleader types and chart formats, revealing current limitations in detecting misleading visualizations.

Conclusion: This work lays the foundation for developing MLLMs that are robust, trustworthy, and aligned with responsible visual communication demands, with the proposed region-aware reasoning pipeline showing enhanced accuracy.

Abstract: Misleading visualizations, which manipulate chart representations to support specific claims, can distort perception and lead to incorrect conclusions. Despite decades of research, they remain a widespread issue-posing risks to public understanding and raising safety concerns for AI systems involved in data-driven communication. While recent multimodal large language models (MLLMs) show strong chart comprehension abilities, their capacity to detect and interpret misleading charts remains unexplored. We introduce Misleading ChartQA benchmark, a large-scale multimodal dataset designed to evaluate MLLMs on misleading chart reasoning. It contains 3,026 curated examples spanning 21 misleader types and 10 chart types, each with standardized chart code, CSV data, multiple-choice questions, and labeled explanations, validated through iterative MLLM checks and exhausted expert human review. We benchmark 24 state-of-the-art MLLMs, analyze their performance across misleader types and chart formats, and propose a novel region-aware reasoning pipeline that enhances model accuracy. Our work lays the foundation for developing MLLMs that are robust, trustworthy, and aligned with the demands of responsible visual communication.

Method: 1) Use small reranker to pre-rank and select top passages 2) Apply reinforcement learning-trained order adjuster to reorder passages for LLM preference 3) Use LLM listwise reranker only on top passages instead of full list

Result: 70% reduction in ranking latency while achieving better effectiveness compared to using only LLM listwise reranker across three IR benchmarks

Conclusion: CoRanking successfully addresses efficiency challenges in LLM-based ranking by combining small and large models with order adjustment, achieving both improved efficiency and effectiveness.

Abstract: Large Language Models (LLMs) have demonstrated superior listwise ranking performance. However, their superior performance often relies on large-scale parameters (\eg, GPT-4) and a repetitive sliding window process, which introduces significant efficiency challenges. In this paper, we propose \textbf{CoRanking}, a novel collaborative ranking framework that combines small and large ranking models for efficient and effective ranking. CoRanking first employs a small-size reranker to pre-rank all the candidate passages, bringing relevant ones to the top part of the list (\eg, top-20). Then, the LLM listwise reranker is applied to only rerank these top-ranked passages instead of the whole list, substantially enhancing overall ranking efficiency. Although more efficient, previous studies have revealed that the LLM listwise reranker have significant positional biases on the order of input passages. Directly feed the top-ranked passages from small reranker may result in the sub-optimal performance of LLM listwise reranker. To alleviate this problem, we introduce a passage order adjuster trained via reinforcement learning, which reorders the top passages from the small reranker to align with the LLM’s preferences of passage order. Extensive experiments on three IR benchmarks demonstrate that CoRanking significantly improves efficiency (reducing ranking latency by about 70%) while achieving even better effectiveness compared to using only the LLM listwise reranker.

Method: Proposes AIR framework that systematically isolates and optimizes each component (Annotations, Instructions, Response Pairs) while evaluating their synergistic effects through rigorous experimentation.

Result: Achieves +5.3 average gains over baseline methods with only 14k high-quality pairs, revealing actionable principles: annotation simplicity, instruction inference stability, and response pair quality optimization.

Conclusion: Shifts preference dataset design from ad hoc scaling to component-aware optimization, offering a blueprint for efficient and reproducible LLM alignment.

Abstract: Preference learning is critical for aligning large language models (LLMs) with human values, yet its success hinges on high-quality datasets comprising three core components: Preference \textbf{A}nnotations, \textbf{I}nstructions, and \textbf{R}esponse Pairs. Current approaches conflate these components, obscuring their individual impacts and hindering systematic optimization. In this work, we propose \textbf{AIR}, a component-wise analysis framework that systematically isolates and optimizes each component while evaluating their synergistic effects. Through rigorous experimentation, AIR reveals actionable principles: annotation simplicity (point-wise generative scoring), instruction inference stability (variance-based filtering across LLMs), and response pair quality (moderate margins + high absolute scores). When combined, these principles yield +5.3 average gains over baseline method, even with only 14k high-quality pairs. Our work shifts preference dataset design from ad hoc scaling to component-aware optimization, offering a blueprint for efficient, reproducible alignment.

Method: Tested multiple baselines including standard machine learning algorithms, deep learning models, and Large Language Models (LLMs), with a focus on Bidirectional Transformer models using meta-learning Reptile approach.

Result: Bidirectional Transformer models outperformed both standard machine-learning approaches and LLMs, particularly when the meta-learning Reptile approach was employed.

Conclusion: Headlines alone contain sufficient signals for satirical tone detection in Romanian news, and advanced transformer models with meta-learning provide the best performance for this task.

Abstract: The primary goal of a news headline is to summarize an event in as few words as possible. Depending on the media outlet, a headline can serve as a means to objectively deliver a summary or improve its visibility. For the latter, specific publications may employ stylistic approaches that incorporate the use of sarcasm, irony, and exaggeration, key elements of a satirical approach. As such, even the headline must reflect the tone of the satirical main content. Current approaches for the Romanian language tend to detect the non-conventional tone (i.e., satire and clickbait) of the news content by combining both the main article and the headline. Because we consider a headline to be merely a brief summary of the main article, we investigate in this paper the presence of satirical tone in headlines alone, testing multiple baselines ranging from standard machine learning algorithms to deep learning models. Our experiments show that Bidirectional Transformer models outperform both standard machine-learning approaches and Large Language Models (LLMs), particularly when the meta-learning Reptile approach is employed.

Method: The authors define the generator-validator gap more stringently than prior work, requiring correlation over all candidate answers. They propose RankAlign, a ranking-based training method to align generator and validator outputs.

Result: A large generator-validator gap exists across various tasks (question answering, lexical semantics, next-word prediction). RankAlign significantly closes this gap and outperforms all baseline methods while generalizing well to out-of-domain tasks.

Conclusion: The generator-validator gap is a fundamental limitation in LLMs, but RankAlign provides an effective training approach to reduce this inconsistency and improve model reliability across diverse domains.

Abstract: Although large language models (LLMs) have become more capable and accurate across many tasks, some fundamental sources of unreliability remain in their behavior. One key limitation is their inconsistency at reporting the same information when prompts are changed. In this paper, we consider the discrepancy between a model’s generated answer and their own verification of that answer, the generator-validator gap. We define this gap in a more stringent way than prior work: we expect correlation of scores from a generator and a validator over the entire set of candidate answers, i.e., candidate completions that could possibly arise during ordinary language use without breaking Gricean norms. We show that according to this measure, a large gap exists in various settings, including question answering, lexical semantics tasks, and next-word prediction. We then propose RankAlign, a ranking-based training method, and show that it significantly closes the gap, surpassing all baseline methods. Moreover, this approach generalizes well to out-of-domain tasks and lexical items.

Method: Uses question generation and answering framework with LLaMA-3 70B model and entailed facts to detect critical MT errors. Developed using ContraTICO dataset of synthetic MT errors in COVID-19 domain and evaluated on BioMQM dataset of natural MT errors.

Result: AskQE achieves higher Kendall’s Tau correlation and decision accuracy with human ratings compared to other quality estimation metrics when tested on the BioMQM dataset.

Conclusion: The AskQE framework successfully enables monolingual users to assess translation quality without target language knowledge, outperforming existing QE approaches and providing actionable feedback for translation acceptance decisions.

Abstract: How can a monolingual English speaker determine whether an automatic translation in French is good enough to be shared? Existing MT error detection and quality estimation (QE) techniques do not address this practical scenario. We introduce AskQE, a question generation and answering framework designed to detect critical MT errors and provide actionable feedback, helping users decide whether to accept or reject MT outputs even without the knowledge of the target language. Using ContraTICO, a dataset of contrastive synthetic MT errors in the COVID-19 domain, we explore design choices for AskQE and develop an optimized version relying on LLaMA-3 70B and entailed facts to guide question generation. We evaluate the resulting system on the BioMQM dataset of naturally occurring MT errors, where AskQE has higher Kendall’s Tau correlation and decision accuracy with human ratings compared to other QE metrics.

Method: Proposes a novel methodology using narrative identity theory to create virtual personas with detailed synthetic backstories generated as multi-turn interview transcripts, producing longer, richer, and more consistent individual descriptions than previous methods.

Result: Virtual personas conditioned on the generated backstories closely replicate human response distributions (up to 87% improvement in Wasserstein Distance) and produce effect sizes that match original studies of in-group/out-group biases.

Conclusion: The work extends LLM applicability beyond estimating socially understood responses, enabling their use in a broader range of human studies requiring authentic in-group perspectives.

Abstract: Large language models (LLMs) are increasingly capable of simulating human behavior, offering cost-effective ways to estimate user responses to various surveys and polls. However, the questions in these surveys usually reflect socially understood attitudes: the patterns of attitudes of old/young, liberal/conservative, as understood by both members and non-members of those groups. It is not clear whether the LLM binding is \emph{deep}, meaning the LLM answers as a member of a particular in-group would, or \emph{shallow}, meaning the LLM responds as an out-group member believes an in-group member would. To explore this difference, we use questions that expose known in-group/out-group biases. This level of fidelity is critical for applying LLMs to various political science studies, including timely topics on polarization dynamics, inter-group conflict, and democratic backsliding. To this end, we propose a novel methodology for constructing virtual personas with synthetic user “backstories” generated as extended, multi-turn interview transcripts. This approach is justified by the theory of \emph{narrative identity} which argues that personality at the highest level is \emph{constructed} from self-narratives. Our generated backstories are longer, rich in detail, and consistent in authentically describing a singular individual, compared to previous methods. We show that virtual personas conditioned on our backstories closely replicate human response distributions (up to an 87% improvement as measured by Wasserstein Distance) and produce effect sizes that closely match those observed in the original studies of in-group/out-group biases. Altogether, our work extends the applicability of LLMs beyond estimating socially understood responses, enabling their use in a broader range of human studies.

Method: Developed an annotation pipeline with two stages: protected attribute detection to identify diverse demographics, followed by regard classification to analyze language polarity towards each attribute.

Result: Demonstrated the effectiveness of the bias analysis and mitigation measures through experiments on Common Crawl as a representative pretraining corpus.

Conclusion: The proposed pipeline provides an efficient and effective way to investigate and address social biases in LLM pretraining data, helping to reduce bias propagation in language models.

Abstract: Large language models (LLMs) acquire general linguistic knowledge from massive-scale pretraining. However, pretraining data mainly comprised of web-crawled texts contain undesirable social biases which can be perpetuated or even amplified by LLMs. In this study, we propose an efficient yet effective annotation pipeline to investigate social biases in the pretraining corpora. Our pipeline consists of protected attribute detection to identify diverse demographics, followed by regard classification to analyze the language polarity towards each attribute. Through our experiments, we demonstrate the effect of our bias analysis and mitigation measures, focusing on Common Crawl as the most representative pretraining corpus.

Method: Developed improved methods to synthesize training sets that incorporate natural spelling variation, using synthetic samples to train language identification systems rather than relying solely on naturally occurring examples.

Result: Achieved new state-of-the-art performance on Bhasha-Abhijnaanam evaluation set: improved F1 from 74.7% to 85.4% with synthetic data alone, and 88.2% when combined with harvested text.

Conclusion: Training on synthetic samples with natural spelling variation yields higher language identification accuracy than using naturally occurring examples or higher capacity models, demonstrating effective approach for romanized text LID.

Abstract: The Latin script is often used to informally write languages with non-Latin native scripts. In many cases (e.g., most languages in India), the lack of conventional spelling in the Latin script results in high spelling variability. Such romanization renders languages that are normally easily distinguished due to being written in different scripts - Hindi and Urdu, for example - highly confusable. In this work, we increase language identification (LID) accuracy for romanized text by improving the methods used to synthesize training sets. We find that training on synthetic samples which incorporate natural spelling variation yields higher LID system accuracy than including available naturally occurring examples in the training set, or even training higher capacity models. We demonstrate new state-of-the-art LID performance on romanized text from 20 Indic languages in the Bhasha-Abhijnaanam evaluation set (Madhani et al., 2023a), improving test F1 from the reported 74.7% (using a pretrained neural model) to 85.4% using a linear classifier trained solely on synthetic data and 88.2% when also training on available harvested text.

Method: Develops a structured taxonomy of reward mechanisms along complementary dimensions, analyzing mathematical formulations, construction practices, and interactions with optimization paradigms including RL-based and RL-free approaches.

Result: Provides conceptual clarity and practical guidance for alignment research, characterizing the progression from RL-based to RL-free optimization and from single-task to multi-objective complex settings.

Conclusion: LLM alignment advancement depends on continuous refinement of reward design strategies, with recent paradigm shifts toward more sophisticated multi-objective and complex optimization approaches.

Abstract: Reward design plays a pivotal role in aligning large language models (LLMs) with human values, serving as the bridge between feedback signals and model optimization. This survey provides a structured organization of reward modeling and addresses three key aspects: mathematical formulation, construction practices, and interaction with optimization paradigms. Building on this, it develops a macro-level taxonomy that characterizes reward mechanisms along complementary dimensions, thereby offering both conceptual clarity and practical guidance for alignment research. The progression of LLM alignment can be understood as a continuous refinement of reward design strategies, with recent developments highlighting paradigm shifts from reinforcement learning (RL)-based to RL-free optimization and from single-task to multi-objective and complex settings.

Method: Proposes AutoRefine framework using reinforcement learning with a “search-and-refine-during-think” paradigm, incorporating knowledge refinement steps between search calls and using group relative policy optimization with retrieval-specific rewards.

Result: Significantly outperforms existing approaches on single-hop and multi-hop QA benchmarks, particularly in complex reasoning scenarios, with frequent higher-quality searches and effective evidence synthesis.

Conclusion: AutoRefine demonstrates that explicit knowledge refinement steps combined with tailored retrieval rewards can substantially improve retrieval-augmented reasoning performance, especially for complex multi-hop tasks.

Abstract: Large language models have demonstrated impressive reasoning capabilities but are inherently limited by their knowledge reservoir. Retrieval-augmented reasoning mitigates this limitation by allowing LLMs to query external resources, but existing methods often retrieve irrelevant or noisy information, hindering accurate reasoning. In this paper, we propose AutoRefine, a reinforcement learning post-training framework that adopts a new ``search-and-refine-during-think’’ paradigm. AutoRefine introduces explicit knowledge refinement steps between successive search calls, enabling the model to iteratively filter, distill, and organize evidence before generating an answer. Furthermore, we incorporate tailored retrieval-specific rewards alongside answer correctness rewards using group relative policy optimization. Experiments on single-hop and multi-hop QA benchmarks demonstrate that AutoRefine significantly outperforms existing approaches, particularly in complex, multi-hop reasoning scenarios. Detailed analysis shows that AutoRefine issues frequent, higher-quality searches and synthesizes evidence effectively.

Method: Evaluated 15 models on IFEval and ComplexBench benchmarks, conducted large-scale case studies and attention-based analysis, proposed constraint attention metric, and introduced four mitigation strategies: in-context learning, self-reflection, self-selective reasoning, and classifier-selective reasoning.

Result: Consistent performance drops when CoT prompting is applied, with reasoning diverting attention away from instruction-relevant tokens. Selective reasoning strategies, particularly classifier-selective reasoning, substantially recover lost performance.

Conclusion: This is the first systematic work exposing reasoning-induced failures in instruction-following and provides practical mitigation strategies, showing that selective reasoning approaches can effectively address the performance degradation caused by CoT reasoning.

Abstract: Reasoning-enhanced large language models (RLLMs), whether explicitly trained for reasoning or prompted via chain-of-thought (CoT), have achieved state-of-the-art performance on many complex reasoning tasks. However, we uncover a surprising and previously overlooked phenomenon: explicit CoT reasoning can significantly degrade instruction-following accuracy. Evaluating 15 models on two benchmarks: IFEval (with simple, rule-verifiable constraints) and ComplexBench (with complex, compositional constraints), we consistently observe performance drops when CoT prompting is applied. Through large-scale case studies and an attention-based analysis, we identify common patterns where reasoning either helps (e.g., with formatting or lexical precision) or hurts (e.g., by neglecting simple constraints or introducing unnecessary content). We propose a metric, constraint attention, to quantify model focus during generation and show that CoT reasoning often diverts attention away from instruction-relevant tokens. To mitigate these effects, we introduce and evaluate four strategies: in-context learning, self-reflection, self-selective reasoning, and classifier-selective reasoning. Our results demonstrate that selective reasoning strategies, particularly classifier-selective reasoning, can substantially recover lost performance. To our knowledge, this is the first work to systematically expose reasoning-induced failures in instruction-following and offer practical mitigation strategies.

Method: The authors introduce a three-level taxonomy (Tool, Analyst, Scientist) through the lens of the scientific method to delineate LLMs’ escalating autonomy and responsibilities in the research lifecycle.

Result: The survey provides a conceptual architecture for understanding LLM roles in science and identifies key challenges including robotic automation, self-improvement, and ethical governance.

Conclusion: This work offers strategic foresight to navigate and shape the future of AI-driven scientific discovery, aiming to foster both rapid innovation and responsible advancement in the field.

Abstract: Large Language Models (LLMs) are catalyzing a paradigm shift in scientific discovery, evolving from task-specific automation tools into increasingly autonomous agents and fundamentally redefining research processes and human-AI collaboration. This survey systematically charts this burgeoning field, placing a central focus on the changing roles and escalating capabilities of LLMs in science. Through the lens of the scientific method, we introduce a foundational three-level taxonomy-Tool, Analyst, and Scientist-to delineate their escalating autonomy and evolving responsibilities within the research lifecycle. We further identify pivotal challenges and future research trajectories such as robotic automation, self-improvement, and ethical governance. Overall, this survey provides a conceptual architecture and strategic foresight to navigate and shape the future of AI-driven scientific discovery, fostering both rapid innovation and responsible advancement. Github Repository: https://github.com/HKUST-KnowComp/Awesome-LLM-Scientific-Discovery.

Method: The authors use the MAESTRO framework to analyze MCP’s safety gaps, propose MCIP as a refined protocol, develop a fine-grained taxonomy of unsafe behaviors, create benchmark/training data, and conduct experiments on state-of-the-art LLMs.

Result: Experiments reveal LLMs’ vulnerabilities in MCP interactions and demonstrate that the proposed approach substantially improves their safety performance.

Conclusion: The paper successfully identifies and addresses MCP safety risks through a comprehensive framework that includes protocol refinement, taxonomy development, and benchmark creation, leading to significant improvements in LLM safety capabilities.

Abstract: As Model Context Protocol (MCP) introduces an easy-to-use ecosystem for users and developers, it also brings underexplored safety risks. Its decentralized architecture, which separates clients and servers, poses unique challenges for systematic safety analysis. This paper proposes a novel framework to enhance MCP safety. Guided by the MAESTRO framework, we first analyze the missing safety mechanisms in MCP, and based on this analysis, we propose the Model Contextual Integrity Protocol (MCIP), a refined version of MCP that addresses these gaps. Next, we develop a fine-grained taxonomy that captures a diverse range of unsafe behaviors observed in MCP scenarios. Building on this taxonomy, we develop benchmark and training data that support the evaluation and improvement of LLMs’ capabilities in identifying safety risks within MCP interactions. Leveraging the proposed benchmark and training data, we conduct extensive experiments on state-of-the-art LLMs. The results highlight LLMs' vulnerabilities in MCP interactions and demonstrate that our approach substantially improves their safety performance.

Method: Developed Online Opinion Mining Benchmark (OOMB) with extensive (entity, feature, opinion) tuple annotations and opinion-centric summaries to evaluate both extractive and abstractive capabilities of LLMs.

Result: The benchmark enables comprehensive analysis of challenging aspects and LLM adaptability in opinion mining, though specific performance metrics are not detailed in the abstract.

Conclusion: OOMB lays the foundation for LLM-based opinion mining and provides directions for future research in evaluating LLMs as effective opinion miners in realistic online scenarios.

Abstract: The surge of user-generated online content presents a wealth of insights into customer preferences and market trends. However, the highly diverse, complex, and context-rich nature of such contents poses significant challenges to traditional opinion mining approaches. To address this, we introduce Online Opinion Mining Benchmark (OOMB), a novel dataset and evaluation protocol designed to assess the ability of large language models (LLMs) to mine opinions effectively from diverse and intricate online environments. OOMB provides extensive (entity, feature, opinion) tuple annotations and a comprehensive opinion-centric summary that highlights key opinion topics within each content, thereby enabling the evaluation of both the extractive and abstractive capabilities of models. Through our proposed benchmark, we conduct a comprehensive analysis of which aspects remain challenging and where LLMs exhibit adaptability, to explore whether they can effectively serve as opinion miners in realistic online scenarios. This study lays the foundation for LLM-based opinion mining and discusses directions for future research in this field.

Method: Three components: supervised contrastive learning for better representations, incorporating image modality alongside speech/text, and Product of Experts to reduce spurious correlations

Result: +7.1% UAR improvement (68.1% to 75.2%) and +2.9% F1 score improvement (80.6% to 83.5%) over text baseline; contrastive learning particularly benefits text modality

Conclusion: The framework effectively addresses challenges in multilingual and multi-picture MCI detection, demonstrating significant performance improvements

Abstract: Detecting Mild Cognitive Impairment from picture descriptions is critical yet challenging, especially in multilingual and multiple picture settings. Prior work has primarily focused on English speakers describing a single picture (e.g., the ‘Cookie Theft’). The TAUKDIAL-2024 challenge expands this scope by introducing multilingual speakers and multiple pictures, which presents new challenges in analyzing picture-dependent content. To address these challenges, we propose a framework with three components: (1) enhancing discriminative representation learning via supervised contrastive learning, (2) involving image modality rather than relying solely on speech and text modalities, and (3) applying a Product of Experts (PoE) strategy to mitigate spurious correlations and overfitting. Our framework improves MCI detection performance, achieving a +7.1% increase in Unweighted Average Recall (UAR) (from 68.1% to 75.2%) and a +2.9% increase in F1 score (from 80.6% to 83.5%) compared to the text unimodal baseline. Notably, the contrastive learning component yields greater gains for the text modality compared to speech. These results highlight our framework’s effectiveness in multilingual and multi-picture MCI detection.

Method: Used information-theoretic framework based on Rate-Distortion Theory and Information Bottleneck principle to analyze token embeddings from various LLMs against human categorization benchmarks.

Result: LLMs form broad conceptual categories aligned with human judgment but struggle with fine-grained semantic distinctions. They show aggressive statistical compression bias, unlike humans who prioritize adaptive nuance and contextual richness.

Conclusion: Critical differences exist between AI and human cognitive architectures in conceptual representation, providing guidance for developing LLMs with more human-aligned representations.

Abstract: Humans organize knowledge into compact categories through semantic compression by mapping diverse instances to abstract representations while preserving meaning (e.g., robin and blue jay are both birds; most birds can fly). These concepts reflect a trade-off between expressive fidelity and representational simplicity. Large Language Models (LLMs) demonstrate remarkable linguistic abilities, yet whether their internal representations strike a human-like trade-off between compression and semantic fidelity is unclear. We introduce a novel information-theoretic framework, drawing from Rate-Distortion Theory and the Information Bottleneck principle, to quantitatively compare these strategies. Analyzing token embeddings from a diverse suite of LLMs against seminal human categorization benchmarks, we uncover key divergences. While LLMs form broad conceptual categories that align with human judgment, they struggle to capture the fine-grained semantic distinctions crucial for human understanding. More fundamentally, LLMs demonstrate a strong bias towards aggressive statistical compression, whereas human conceptual systems appear to prioritize adaptive nuance and contextual richness, even if this results in lower compressional efficiency by our measures. These findings illuminate critical differences between current AI and human cognitive architectures, guiding pathways toward LLMs with more human-aligned conceptual representations.

Method: Proposed Cog-TiPRO framework combining: (1) LLM-driven iterative prompt refinement for linguistic feature extraction, (2) HuBERT-based acoustic feature extraction, and (3) transformer-based temporal modeling using iTransformer. Collected 18-month voice command data from 35 older adults.

Result: Achieved 73.80% accuracy and 72.67% F1-score in detecting MCI, outperforming baseline by 27.13%. Identified unique linguistic features characterizing cognitive decline in everyday command usage.

Conclusion: Voice assistant systems with AI analysis can effectively detect cognitive decline through speech patterns, providing a scalable non-invasive monitoring solution.

Abstract: Early detection of cognitive decline is crucial for enabling interventions that can slow neurodegenerative disease progression. Traditional diagnostic approaches rely on labor-intensive clinical assessments, which are impractical for frequent monitoring. Our pilot study investigates voice assistant systems (VAS) as non-invasive tools for detecting cognitive decline through longitudinal analysis of speech patterns in voice commands. Over an 18-month period, we collected voice commands from 35 older adults, with 15 participants providing daily at-home VAS interactions. To address the challenges of analyzing these short, unstructured and noisy commands, we propose Cog-TiPRO, a framework that combines (1) LLM-driven iterative prompt refinement for linguistic feature extraction, (2) HuBERT-based acoustic feature extraction, and (3) transformer-based temporal modeling. Using iTransformer, our approach achieves 73.80% accuracy and 72.67% F1-score in detecting MCI, outperforming its baseline by 27.13%. Through our LLM approach, we identify linguistic features that uniquely characterize everyday command usage patterns in individuals experiencing cognitive decline.

Method: Conducted systematic mapping study following Kitchenham’s guidelines, developing structured classification of research focuses, contribution types, and application domains.

Result: Identified architectural patterns, temporal trends, key challenges (privacy-preserving retrieval, cross-client heterogeneity, evaluation limitations), and synthesized rapidly evolving research landscape.

Conclusion: Provides foundation for future work at RAG-federated systems intersection, identifying recurring design patterns and open questions in this emerging field.

Abstract: Federated Retrieval-Augmented Generation (Federated RAG) combines Federated Learning (FL), which enables distributed model training without exposing raw data, with Retrieval-Augmented Generation (RAG), which improves the factual accuracy of language models by grounding outputs in external knowledge. As large language models are increasingly deployed in privacy-sensitive domains such as healthcare, finance, and personalized assistance, Federated RAG offers a promising framework for secure, knowledge-intensive natural language processing (NLP). To the best of our knowledge, this paper presents the first systematic mapping study of Federated RAG, covering literature published between 2020 and 2025. Following Kitchenham’s guidelines for evidence-based software engineering, we develop a structured classification of research focuses, contribution types, and application domains. We analyze architectural patterns, temporal trends, and key challenges, including privacy-preserving retrieval, cross-client heterogeneity, and evaluation limitations. Our findings synthesize a rapidly evolving body of research, identify recurring design patterns, and surface open questions, providing a foundation for future work at the intersection of RAG and federated systems.

Method: Used PERSUADE corpus with two types of annotations: spelling/grammatical errors (generated by generative LLM) and argumentative components (identified by encoder-based token-classifier). Incorporated these annotations into scoring pipeline using fine-tuned encoder-based LLM classifiers.

Result: Demonstrated performance improvements in automated essay scoring when annotations are incorporated into the scoring process.

Conclusion: Feedback-oriented annotations significantly enhance AES accuracy, showing practical applicability for real-world educational scenarios through LLM-generated annotations.

Abstract: This study illustrates how incorporating feedback-oriented annotations into the scoring pipeline can enhance the accuracy of automated essay scoring (AES). This approach is demonstrated with the Persuasive Essays for Rating, Selecting, and Understanding Argumentative and Discourse Elements (PERSUADE) corpus. We integrate two types of feedback-driven annotations: those that identify spelling and grammatical errors, and those that highlight argumentative components. To illustrate how this method could be applied in real-world scenarios, we employ two LLMs to generate annotations – a generative language model used for spell correction and an encoder-based token-classifier trained to identify and mark argumentative elements. By incorporating annotations into the scoring process, we demonstrate improvements in performance using encoder-based large language models fine-tuned as classifiers.

Method: Proposed Multi-Agent Debate framework with two variants: exclusive debate between two LLM agents, and dynamic choice between self-reflection and debate during turns. Evaluated on 7 open-weight LLMs using NormAd-ETI benchmark for social etiquette norms across 75 countries.

Result: Debate improves both overall accuracy and cultural group parity. Small LLMs (7-9B parameters) achieved accuracies comparable to much larger models (27B parameters) through multi-agent debate.

Conclusion: Multi-agent debate framework effectively enhances cultural adaptability of LLMs, demonstrating that collaborative approaches can compensate for model size limitations and improve performance across diverse cultural contexts.

Abstract: Large Language Models (LLMs) need to adapt their predictions to diverse cultural contexts to benefit diverse communities across the world. While previous efforts have focused on single-LLM, single-turn approaches, we propose to exploit the complementary strengths of multiple LLMs to promote cultural adaptability. We introduce a Multi-Agent Debate framework, where two LLM-based agents debate over a cultural scenario and collaboratively reach a final decision. We propose two variants: one where either LLM agents exclusively debate and another where they dynamically choose between self-reflection and debate during their turns. We evaluate these approaches on 7 open-weight LLMs (and 21 LLM combinations) using the NormAd-ETI benchmark for social etiquette norms in 75 countries. Experiments show that debate improves both overall accuracy and cultural group parity over single-LLM baselines. Notably, multi-agent debate enables relatively small LLMs (7-9B) to achieve accuracies comparable to that of a much larger model (27B parameters).

Method: Compared four types of quality feedback in a machine translation scenario: explicit feedback (error highlights and LLM explanations) and implicit feedback (backtranslation and question-answer tables), testing how they affect user decisions about sharing translations.

Result: All feedback types except error highlights significantly improved decision accuracy and appropriate reliance. Implicit feedback, particularly QA tables, showed the greatest gains in accuracy, reliance, and user perceptions (highest helpfulness/trust, lowest mental burden).

Conclusion: Implicit feedback mechanisms, especially question-answer tables, are more effective than explicit feedback for helping non-expert users make better decisions about AI-generated content quality.

Abstract: As people increasingly use AI systems in work and daily life, feedback mechanisms that help them use AI responsibly are urgently needed, particularly in settings where users are not equipped to assess the quality of AI predictions. We study a realistic Machine Translation (MT) scenario where monolingual users decide whether to share an MT output, first without and then with quality feedback. We compare four types of quality feedback: explicit feedback that directly give users an assessment of translation quality using (1) error highlights and (2) LLM explanations, and implicit feedback that helps users compare MT inputs and outputs through (3) backtranslation and (4) question-answer (QA) tables. We find that all feedback types, except error highlights, significantly improve both decision accuracy and appropriate reliance. Notably, implicit feedback, especially QA tables, yields significantly greater gains than explicit feedback in terms of decision accuracy, appropriate reliance, and user perceptions, receiving the highest ratings for helpfulness and trust, and the lowest for mental burden.

Method: Developed SDA framework with commitment-based taxonomy of discourse moves and Gricean-based proxies. Created three metrics (BAT, PAT, NRBAT) and CPD dataset of annotated courtroom cross-examinations to evaluate LLMs.

Result: LLMs show limited pragmatic understanding of strategic language. Model size improves performance but reasoning ability hurts performance by causing overcomplication and internal confusion.

Conclusion: The SDA framework effectively assesses strategic discourse, revealing significant limitations in current LLMs’ ability to understand and process adversarial language strategies.

Abstract: Language is often used strategically, particularly in high-stakes, adversarial settings, yet most work on pragmatics and LLMs centers on cooperativity. This leaves a gap in the systematic understanding of strategic communication in adversarial settings. To address this, we introduce SDA (Strategic Discourse Assessment), a framework grounded in Gricean and game-theoretic pragmatics to assess strategic use of language. It adapts the ME Game jury function to make it empirically estimable for analyzing dialogue. Our approach incorporates two key adaptations: a commitment-based taxonomy of discourse moves, which provides a finer-grained account of strategic effects, and the use of estimable proxies grounded in Gricean maxims to operationalize abstract constructs such as credibility. Together, these adaptations build on discourse theory by treating discourse as the strategic management of commitments, enabling systematic evaluation of how conversational moves advance or undermine discourse goals. We further derive three interpretable metrics-Benefit at Turn (BAT), Penalty at Turn (PAT), and Normalized Relative Benefit at Turn (NRBAT)-to quantify the perceived strategic effects of discourse moves. We also present CPD (the Crooked Path Dataset), an annotated dataset of real courtroom cross-examinations, to demonstrate the framework’s effectiveness. Using these tools, we evaluate a range of LLMs and show that LLMs generally exhibit limited pragmatic understanding of strategic language. While model size shows an increase in performance on our metrics, reasoning ability does not help and largely hurts, introducing overcomplication and internal confusion.

Method: Constructed benchmark from real-world financial assistant interactions incorporating both real-time API data and text data, organized through an intent classification framework covering critical financial domains to enable comprehensive evaluation of static knowledge and time-sensitive market information handling.

Result: Systematic experiments with multiple Chinese leading LLMs demonstrated FinS-Pilot’s effectiveness in identifying models suitable for financial applications, successfully addressing the current gap in specialized evaluation tools.

Conclusion: The work contributes both a practical evaluation framework and curated dataset to advance research in financial NLP systems, with code and dataset made accessible on GitHub for broader research community use.

Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across various professional domains, with their performance typically evaluated through standardized benchmarks. In the financial field, the stringent demands for professional accuracy and real-time data processing often necessitate the use of retrieval-augmented generation (RAG) techniques. However, the development of financial RAG benchmarks has been constrained by data confidentiality issues and the lack of dynamic data integration. To address this issue, we introduce FinS-Pilot, a novel benchmark for evaluating RAG systems in online financial applications. Constructed from real-world financial assistant interactions, our benchmark incorporates both real-time API data and text data, organized through an intent classification framework covering critical financial domains. The benchmark enables comprehensive evaluation of financial assistants’ capabilities in handling both static knowledge and time-sensitive market information.Through systematic experiments with multiple Chinese leading LLMs, we demonstrate FinS-Pilot’s effectiveness in identifying models suitable for financial applications while addressing the current gap in specialized evaluation tools for the financial domain. Our work contributes both a practical evaluation framework and a curated dataset to advance research in financial NLP systems. The code and dataset are accessible on GitHub.

Method: Decomposes task into schema filtering, retrieval-augmented generation, and prompt-driven schema linking. Fine-tunes LLMs for schema selection and uses Chain-of-Thought and Graph-of-Thought prompt engineering to enhance reasoning.

Result: Comprehensive evaluations on Spider benchmarks demonstrate the effectiveness of the AP-SQL approach.

Conclusion: AP-SQL successfully bridges the gap between resource efficiency and performance in Text-to-SQL translation through innovative decomposition and prompt engineering techniques.

Abstract: Using the best Text-to-SQL methods in resource-constrained environments is challenging due to their reliance on resource-intensive open-source models. This paper introduces Auto Prompt SQL(AP-SQL), a novel architecture designed to bridge the gap between resource-efficient small open-source models and the powerful capabilities of large closed-source models for Text-to-SQL translation. Our method decomposes the task into schema filtering, retrieval-augmented text-to-SQL generation based on in-context examples, and prompt-driven schema linking and SQL generation. To improve schema selection accuracy, we fine-tune large language models. Crucially, we also explore the impact of prompt engineering throughout the process, leveraging Chain-of-Thought(CoT) and Graph-of-Thought(GoT) templates to significantly enhance the model’s reasoning for accurate SQL generation. Comprehensive evaluations on the Spider benchmarks demonstrate the effectiveness of AP-SQL.

Method: Created 2,996 questions from real-world electronic health records and expert-designed clinical scenarios across 5 core medical areas. Developed an automated evaluation pipeline using LLM-as-Judge framework with expert-developed checklists, validated through human-machine agreement analysis and dynamic refinement based on expert feedback.

Result: Evaluated 13 LLMs across three categories (specialized medical models, open-source models, and closed-source models) on the LLMEval-Med benchmark, providing insights for safe and effective deployment of LLMs in medical domains.

Conclusion: LLMEval-Med addresses critical gaps in medical LLM evaluation by providing a more realistic, clinically-relevant benchmark with reliable automated assessment methods, supporting safer deployment of LLMs in healthcare applications.

Abstract: Evaluating large language models (LLMs) in medicine is crucial because medical applications require high accuracy with little room for error. Current medical benchmarks have three main types: medical exam-based, comprehensive medical, and specialized assessments. However, these benchmarks have limitations in question design (mostly multiple-choice), data sources (often not derived from real clinical scenarios), and evaluation methods (poor assessment of complex reasoning). To address these issues, we present LLMEval-Med, a new benchmark covering five core medical areas, including 2,996 questions created from real-world electronic health records and expert-designed clinical scenarios. We also design an automated evaluation pipeline, incorporating expert-developed checklists into our LLM-as-Judge framework. Furthermore, our methodology validates machine scoring through human-machine agreement analysis, dynamically refining checklists and prompts based on expert feedback to ensure reliability. We evaluate 13 LLMs across three categories (specialized medical models, open-source models, and closed-source models) on LLMEval-Med, providing valuable insights for the safe and effective deployment of LLMs in medical domains. The dataset is released in https://github.com/llmeval/LLMEval-Med.

Method: The study uses various proxy measures of bias, including evaluating models with pre-prompted personae on MMLU benchmark, reformulating tasks to have models grade user answers, and testing salary negotiation advice scenarios.

Result: Traditional benchmark evaluation showed negligible and mostly random differences, but grading user answers revealed more significant bias, and salary negotiation advice showed pronounced bias. The problem is exacerbated as models gain memory and personalization capabilities.

Conclusion: Bias in LLMs is more pronounced in realistic application scenarios than in traditional benchmarks, and the trend toward personalized LLM assistants with memory capabilities makes this an increasingly important problem that requires attention.

Abstract: Modern language models are trained on large amounts of data. These data inevitably include controversial and stereotypical content, which contains all sorts of biases related to gender, origin, age, etc. As a result, the models express biased points of view or produce different results based on the assigned personality or the personality of the user. In this paper, we investigate various proxy measures of bias in large language models (LLMs). We find that evaluating models with pre-prompted personae on a multi-subject benchmark (MMLU) leads to negligible and mostly random differences in scores. However, if we reformulate the task and ask a model to grade the user’s answer, this shows more significant signs of bias. Finally, if we ask the model for salary negotiation advice, we see pronounced bias in the answers. With the recent trend for LLM assistant memory and personalization, these problems open up from a different angle: modern LLM users do not need to pre-prompt the description of their persona since the model already knows their socio-demographics.

Method: Systematic compilation from online sources, medical literature, and health databases using peer-reviewed articles and clinical studies, excluding non-verified sources. Structured in tabular format with binary symptom indicators.

Result: Created a comprehensive disease-symptom dataset in Bangla language with verified medical relationships, enabling machine learning applications and clinical decision support.

Conclusion: The dataset bridges the linguistic gap in medical informatics and provides foundation for developing multilingual disease prediction tools, with potential for future expansion to include region-specific diseases and refined symptom associations.

Abstract: Disease-symptom datasets are significant and in demand for medical research, disease diagnosis, clinical decision-making, and AI-driven health management applications. These datasets help identify symptom patterns associated with specific diseases, thus improving diagnostic accuracy and enabling early detection. The dataset presented in this study systematically compiles disease-symptom relationships from various online sources, medical literature, and publicly available health databases. The data was gathered through analyzing peer-reviewed medical articles, clinical case studies, and disease-symptom association reports. Only the verified medical sources were included in the dataset, while those from non-peer-reviewed and anecdotal sources were excluded. The dataset is structured in a tabular format, where the first column represents diseases, and the remaining columns represent symptoms. Each symptom cell contains a binary value, indicating whether a symptom is associated with a disease. Thereby, this structured representation makes the dataset very useful for a wide range of applications, including machine learning-based disease prediction, clinical decision support systems, and epidemiological studies. Although there are some advancements in the field of disease-symptom datasets, there is a significant gap in structured datasets for the Bangla language. This dataset aims to bridge that gap by facilitating the development of multilingual medical informatics tools and improving disease prediction models for underrepresented linguistic communities. Further developments should include region-specific diseases and further fine-tuning of symptom associations for better diagnostic performance

Method: Augment pretrained Transformers with linearized attention (LiZA) and internal memory gating (MaG) via parameter-efficient fine-tuning (LoRA), supporting conversion to pure linear attention using DeltaProduct mechanism.

Result: Models with ~1B parameters showed up to 20% relative increase in Exact Match scores on MMLU benchmark, with efficient training using modest computational resources.

Conclusion: TPTT enables adaptation of pretrained LLMs for long-context tasks with limited overhead, though further studies on larger models and benchmarks are needed to evaluate generality.

Abstract: Transformer-based large language models (LLMs) have achieved strong performance across many natural language processing tasks. Nonetheless, their quadratic computational and memory requirements, particularly in self-attention layers, pose challenges for efficient inference on long contexts and for deployment in resource-limited environments. We present TPTT (Transforming Pretrained Transformers into Titans), a framework designed to augment pretrained Transformers with linearized attention (LiZA) and internal memory gating via Memory as Gate (MaG), applied without full retraining. TPTT supports parameter-efficient fine-tuning (LoRA) and integrates with standard toolkits such as Hugging Face Transformers. We evaluated TPTT on several pretrained models, including Llama-1B, OlMoE-1B-7B, Qwen2.5-1.5B, Gemma3-270m, OpenELM-1.3B, and Mistral-7B, in order to assess applicability across architectures of different scales. Experiments on models with approximately 1 billion parameters, evaluated primarily on the MMLU benchmark, suggest potential improvements in both efficiency and accuracy compared to baseline models. For example, Titans-Llama-1B exhibited up to a 20% relative increase in Exact Match scores in one-shot evaluation. An additional finding is that it is possible to convert a quadratic-attention model into a purely linear-attention model using the DeltaProduct mechanism. All training runs were carried out with modest computational resources. These preliminary findings indicate that TPTT may help adapt pretrained LLMs for long-context tasks with limited overhead. Further studies on larger models and a broader set of benchmarks will be necessary to evaluate the generality and robustness of the framework. Code is available at https://github.com/fabienfrfr/tptt . Python package at https://pypi.org/project/tptt/ .

Method: Two-stage approach: 1) Semantic Context Learning module for context-aware entity embeddings, 2) Conditional Flow-Matching module to model dynamic flow between entities that modulates a base static score.

Result: Achieves near-perfect performance: 99.8% MRR and 99.7% Hits@1 on FB15k-237 with only 0.35M parameters, 99.9% MRR on WN18RR, and 25.2% relative MRR gain in transductive entity prediction.

Conclusion: FMS provides a highly effective and parameter-efficient paradigm for knowledge graph completion by unifying dynamic flow mechanisms with rich static contexts.

Abstract: Knowledge graph completion demands effective modeling of multifaceted semantic relationships between entities. Yet, prevailing methods, which rely on static scoring functions over learned embeddings, struggling to simultaneously capture rich semantic context and the dynamic nature of relations. To overcome this limitation, we propose the Flow-Modulated Scoring (FMS) framework, conceptualizing a relation as a dynamic evolutionary process governed by its static semantic environment. FMS operates in two stages: it first learns context-aware entity embeddings via a Semantic Context Learning module, and then models a dynamic flow between them using a Conditional Flow-Matching module. This learned flow dynamically modulates a base static score for the entity pair. By unifying context-rich static representations with a conditioned dynamic flow, FMS achieves a more comprehensive understanding of relational semantics. Extensive experiments demonstrate that FMS establishes a new state of the art across both canonical knowledge graph completion tasks: relation prediction and entity prediction. On the standard relation prediction benchmark FB15k-237, FMS achieves a near-perfect MRR of 99.8% and Hits@1 of 99.7% using a mere 0.35M parameters, while also attaining a 99.9% MRR on WN18RR. Its dominance extends to entity prediction, where it secures a 25.2% relative MRR gain in the transductive setting and substantially outperforms all baselines in challenging inductive settings. By unifying a dynamic flow mechanism with rich static contexts, FMS offers a highly effective and parameter-efficient new paradigm for knowledge graph completion. Code published at: https://github.com/yuanwuyuan9/FMS.

Method: Proposes MedVAL framework using synthetic data to train evaluator LMs for factual consistency assessment without physician labels or reference outputs, and introduces MedVAL-Bench dataset with physician annotations.

Result: MedVAL fine-tuning significantly improves alignment with physicians (66% to 83% F1), achieves 86% safety classification, and boosts GPT-4o performance by 8% across 6 medical tasks and 10 LMs.

Conclusion: MedVAL provides the first evidence of LMs approaching expert-level medical text validation, enabling scalable, risk-aware clinical integration with open-sourced code, benchmark, and models.

Abstract: With the growing use of language models (LMs) in clinical environments, there is an immediate need to evaluate the accuracy and safety of LM-generated medical text. Currently, such evaluation relies solely on manual physician review. However, detecting errors in LM-generated text is challenging because

1. manual review is costly and 2) expert-composed reference outputs are often unavailable in real-world settings. While the “LM-as-judge” paradigm (a LM evaluating another LM) offers scalable evaluation, even frontier LMs can miss subtle but clinically significant errors. To address these challenges, we propose MedVAL, a self-supervised framework that leverages synthetic data to train evaluator LMs to assess whether LM-generated medical outputs are factually consistent with inputs, without requiring physician labels or reference outputs. To evaluate LM performance, we introduce MedVAL-Bench, a dataset containing 840 outputs annotated by physicians, following a physician-defined taxonomy of risk levels and error categories. Across 6 diverse medical tasks and 10 state-of-the-art LMs spanning open-source, proprietary, and medically adapted models, MedVAL fine-tuning significantly improves (p < 0.001) alignment with physicians on both seen and unseen tasks, increasing average F1 scores from 66% to 83%, with per-sample safety classification scores up to 86%. MedVAL improves the performance of even the best-performing proprietary LM (GPT-4o) by 8%. To support a scalable, risk-aware pathway towards clinical integration, we open-source the 1) codebase (https://github.com/StanfordMIMI/MedVAL), 2) MedVAL-Bench (https://huggingface.co/datasets/stanfordmimi/MedVAL-Bench), and 3) MedVAL-4B (https://huggingface.co/stanfordmimi/MedVAL-4B), the best-performing open-source LM. Our research provides the first evidence of LMs approaching expert-level validation ability for medical text.

Method: Fine-tuning framework that distills complementary reasoning strategies from multiple teachers into a unified student model (Agentic-R1) that dynamically selects optimal strategy per query.

Result: Improves accuracy across computation-intensive and standard benchmarks, demonstrating robust and efficient reasoning through multi-strategy distillation.

Conclusion: The approach effectively combines tool-based computation for arithmetic/algorithmic problems with text-based reasoning for abstract tasks, achieving superior performance across diverse reasoning tasks.

Abstract: Current long chain-of-thought (long-CoT) models excel at mathematical reasoning but rely on slow and error-prone natural language traces. Tool-augmented agents address arithmetic via code execution, but often falter on complex logical tasks. We introduce a fine-tuning framework, DualDistill, that distills complementary reasoning strategies from multiple teachers into a unified student model. Using this approach, we train Agentic-R1, which dynamically selects the optimal strategy for each query, invoking tools for arithmetic and algorithmic problems, and using text-based reasoning for abstract ones. Our method improves accuracy across a range of tasks, including both computation-intensive and standard benchmarks, demonstrating the effectiveness of multi-strategy distillation in achieving robust and efficient reasoning. Our project is available at https://github.com/StigLidu/DualDistill

Method: Quantitative analysis and qualitative insights into how models respond to increasing noise in user prompts, testing different noise types including character-level, phrasal perturbations, and combined noisers.

Result: Prompt quality strongly affects translation performance - good prompts with errors can underperform poor prompts without errors. Character-level and combined noise degrade performance more than phrasal perturbations. LLMs can translate even with overwhelming random noise that makes prompts illegible to humans.

Conclusion: Lower prompt quality mainly leads to poorer instruction following rather than directly affecting translation quality itself, showing LLMs’ robustness to extreme noise but sensitivity to prompt errors.

Abstract: Large language models (LLMs) have achieved top results in recent machine translation evaluations, but they are also known to be sensitive to errors and perturbations in their prompts. We systematically evaluate how both humanly plausible and synthetic errors in user prompts affect LLMs’ performance on two related tasks: Machine translation and machine translation evaluation. We provide both a quantitative analysis and qualitative insights into how the models respond to increasing noise in the user prompt. The prompt quality strongly affects the translation performance: With many errors, even a good prompt can underperform a minimal or poor prompt without errors. However, different noise types impact translation quality differently, with character-level and combined noisers degrading performance more than phrasal perturbations. Qualitative analysis reveals that lower prompt quality largely leads to poorer instruction following, rather than directly affecting translation quality itself. Further, LLMs can still translate in scenarios with overwhelming random noise that would make the prompt illegible to humans.

Method: Evaluated models using annotated corpora from healthcare (I2B2), legal (TAB), and biography (Wikipedia) across four dimensions: in-domain performance, cross-domain transferability, fusion, and few-shot learning.

Result: Legal-domain data transfers effectively to biographical texts, medical domains resist incoming transfer, fusion benefits are domain-specific, and high-quality recognition is achievable with only 10% of training data in low-specialization domains.

Conclusion: Domain characteristics significantly impact PII recognition transferability, with legal data showing good cross-domain applicability while medical data remains domain-specific, and minimal training data can be sufficient for less specialized domains.

Abstract: Accurate recognition of personally identifiable information (PII) is central to automated text anonymization. This paper investigates the effectiveness of cross-domain model transfer, multi-domain data fusion, and sample-efficient learning for PII recognition. Using annotated corpora from healthcare (I2B2), legal (TAB), and biography (Wikipedia), we evaluate models across four dimensions: in-domain performance, cross-domain transferability, fusion, and few-shot learning. Results show legal-domain data transfers well to biographical texts, while medical domains resist incoming transfer. Fusion benefits are domain-specific, and high-quality recognition is achievable with only 10% of training data in low-specialization domains.

Method: Performed coarse-grained label unification across four cybersecurity datasets, conducted pairwise cross-dataset evaluations using BiLSTM models, and proposed alternative architectures including a multihead model with weight sharing and a graph-based transfer model built on BERT-base-NER.

Result: Models trained on unified datasets generalized poorly across different datasets. The multihead model provided only marginal improvements over unified training, and the graph-based transfer model showed no significant performance gains compared to standard BERT-base-NER.

Conclusion: Label unification in cybersecurity NER faces significant challenges, and the proposed alternative architectures did not substantially improve cross-dataset generalization, indicating the need for more effective approaches to handle dataset heterogeneity in this domain.

Abstract: The field of cybersecurity NER lacks standardized labels, making it challenging to combine datasets. We investigate label unification across four cybersecurity datasets to increase data resource usability. We perform a coarse-grained label unification and conduct pairwise cross-dataset evaluations using BiLSTM models. Qualitative analysis of predictions reveals errors, limitations, and dataset differences. To address unification limitations, we propose alternative architectures including a multihead model and a graph-based transfer model. Results show that models trained on unified datasets generalize poorly across datasets. The multihead model with weight sharing provides only marginal improvements over unified training, while our graph-based transfer model built on BERT-base-NER shows no significant performance gains compared BERT-base-NER.

Method: A task generation pipeline synthesizes reasoning tasks directly from KG primitives, creating a KG-grounded curriculum. The QwQ-32B model is fine-tuned on 24,000 medical reasoning tasks derived from a medical knowledge graph.

Result: QwQ-Med-3 significantly outperforms state-of-the-art reasoning models on ICD-Bench across 15 medical domains and shows enhanced performance on medical QA benchmarks, utilizing acquired primitives for hardest tasks.

Conclusion: The approach demonstrates that domain-specific superintelligence can be achieved through composable KG-grounded training, suggesting AGI may emerge from efficient domain-specific agents rather than broad general training.

Abstract: Language models traditionally used for cross-domain generalization have recently demonstrated task-specific reasoning. However, their top-down training approach on general corpora is insufficient for acquiring abstractions needed for deep domain expertise. This may require a bottom-up approach that acquires expertise by learning to compose simple domain concepts into more complex ones. A knowledge graph (KG) provides this compositional structure, where domain primitives are represented as head-relation-tail edges and their paths encode higher-level concepts. We present a task generation pipeline that synthesizes tasks directly from KG primitives, enabling models to acquire and compose them for reasoning. We fine-tune language models on the resultant KG-grounded curriculum to demonstrate domain-specific superintelligence. While broadly applicable, we validate our approach in medicine, where reliable KGs exist. Using a medical KG, we curate 24,000 reasoning tasks paired with thinking traces derived from diverse medical primitives. We fine-tune the QwQ-32B model on this curriculum to obtain QwQ-Med-3 that takes a step towards medical superintelligence. We also introduce ICD-Bench, an evaluation suite to quantify reasoning abilities across 15 medical domains. Our experiments demonstrate that QwQ-Med-3 significantly outperforms state-of-the-art reasoning models on ICD-Bench categories. Further analysis reveals that QwQ-Med-3 utilizes acquired primitives to widen the performance gap on the hardest tasks of ICD-Bench. Finally, evaluation on medical question-answer benchmarks shows that QwQ-Med-3 transfers acquired expertise to enhance the base model’s performance. While the industry’s approach to artificial general intelligence (AGI) emphasizes broad expertise, we envision a future in which AGI emerges from the composable interaction of efficient domain-specific superintelligent agents.

Method: Proposes a framework that incorporates ambiguity detection and classification prior to inference. Introduces a unified taxonomy synthesizing existing taxonomies and motivates targeted detection methods.

Result: The paper presents a conceptual framework and taxonomy but acknowledges current lack of datasets explicitly annotated for ambiguity and subtypes, identifying this as a research opportunity.

Conclusion: Shifting to ambiguity-aware NLI through new annotated resources and unsupervised approaches will enable more robust, explainable, and human-aligned NLI systems that better capture divergent human perspectives.

Abstract: This position paper argues that annotation disagreement in Natural Language Inference (NLI) is not mere noise but often reflects meaningful variation, especially when triggered by ambiguity in the premise or hypothesis. While underspecified guidelines and annotator behavior contribute to variation, content-based ambiguity provides a process-independent signal of divergent human perspectives. We call for a shift toward ambiguity-aware NLI that first identifies ambiguous input pairs, classifies their types, and only then proceeds to inference. To support this shift, we present a framework that incorporates ambiguity detection and classification prior to inference. We also introduce a unified taxonomy that synthesizes existing taxonomies, illustrates key subtypes with examples, and motivates targeted detection methods that better align models with human interpretation. Although current resources lack datasets explicitly annotated for ambiguity and subtypes, this gap presents an opportunity: by developing new annotated resources and exploring unsupervised approaches to ambiguity detection, we enable more robust, explainable, and human-aligned NLI systems.

Method: 1) Combine a small set of similarity-based demonstrations with a larger cached set of random demonstrations. 2) Replace random demonstrations with centroid-based demonstrations from k-means clustering of test samples.

Result: Strategies consistently outperform random selection and match top-performing approaches while reducing inference cost by up to 10x across Gemini Pro and Flash models on multiple datasets.

Conclusion: The proposed hybrid demonstration selection methods effectively balance performance and computational efficiency in many-shot ICL, enabling better results with significantly reduced inference costs.

Abstract: Long-context large language models (LLMs) are able to process inputs containing up to several million tokens. In the scope of in-context learning (ICL), this translates into using hundreds/thousands of demonstrations in the input prompt, enabling many-shot ICL. In practice, a fixed set of demonstrations is often selected at random in many-shot settings due to (1) high inference costs, (2) the benefits of caching and reusing computations, and (3) the similar performance offered by this strategy compared to others when scaled. In this work, we propose two straightforward strategies for demonstration selection in many-shot ICL that improve performance with minimal computational overhead. Our first method combines a small number of demonstrations, selected based on their similarity to each test sample, with a disproportionately larger set of random demonstrations that are cached. The second strategy improves the first by replacing random demonstrations with those selected using centroids derived from test sample representations via k-means clustering. Our experiments with Gemini Pro and Flash across several datasets indicate that our strategies consistently outperform random selection and surpass or match the most performant selection approach while supporting caching and reducing inference cost by up to an order of magnitude. We also show that adjusting the proportion of demonstrations selected based on different criteria can balance performance and inference cost in many-shot ICL.

Method: Extract automated persona vectors from model activation space using natural-language descriptions, apply these vectors to predict and control personality shifts during finetuning, and develop preventative steering methods and data filtering techniques.

Result: Persona vectors effectively monitor personality fluctuations at deployment and strongly correlate with both intended and unintended personality changes after finetuning. Shifts can be mitigated through post-hoc intervention or prevented with new steering methods.

Conclusion: Persona vectors provide an automated, scalable approach to monitor and control LLM personality traits, enabling detection of undesirable changes and prevention methods at both dataset and individual sample levels.

Abstract: Large language models interact with users through a simulated ‘Assistant’ persona. While the Assistant is typically trained to be helpful, harmless, and honest, it sometimes deviates from these ideals. In this paper, we identify directions in the model’s activation space-persona vectors-underlying several traits, such as evil, sycophancy, and propensity to hallucinate. We confirm that these vectors can be used to monitor fluctuations in the Assistant’s personality at deployment time. We then apply persona vectors to predict and control personality shifts that occur during training. We find that both intended and unintended personality changes after finetuning are strongly correlated with shifts along the relevant persona vectors. These shifts can be mitigated through post-hoc intervention, or avoided in the first place with a new preventative steering method. Moreover, persona vectors can be used to flag training data that will produce undesirable personality changes, both at the dataset level and the individual sample level. Our method for extracting persona vectors is automated and can be applied to any personality trait of interest, given only a natural-language description.

Method: Combines multi-outcome modeling on language responses with item response theory (IRT) and factor analysis to select the most informative questions across multiple dimensions at each turn.

Result: Reduces assessment questions by 50-87% (71% fewer for depression, 85% fewer for eating disorders) while achieving stable scores. Performs robustly across internalizing (depression, anxiety) and externalizing (substance use, eating disorder) domains.

Conclusion: MAQuA is a powerful and efficient tool for scalable, nuanced mental health screening that advances LLM integration into clinical workflows by reducing patient burden while maintaining diagnostic accuracy.

Abstract: Recent advances in large language models (LLMs) offer new opportunities for scalable, interactive mental health assessment, but excessive querying by LLMs burdens users and is inefficient for real-world screening across transdiagnostic symptom profiles. We introduce MAQuA, an adaptive question-asking framework for simultaneous, multidimensional mental health screening. Combining multi-outcome modeling on language responses with item response theory (IRT) and factor analysis, MAQuA selects the questions with most informative responses across multiple dimensions at each turn to optimize diagnostic information, improving accuracy and potentially reducing response burden. Empirical results on a novel dataset reveal that MAQuA reduces the number of assessment questions required for score stabilization by 50-87% compared to random ordering (e.g., achieving stable depression scores with 71% fewer questions and eating disorder scores with 85% fewer questions). MAQuA demonstrates robust performance across both internalizing (depression, anxiety) and externalizing (substance use, eating disorder) domains, with early stopping strategies further reducing patient time and burden. These findings position MAQuA as a powerful and efficient tool for scalable, nuanced, and interactive mental health screening, advancing the integration of LLM-based agents into real-world clinical workflows.

Method: Span-level Quality Assurance EvaluaTOR (Sqator) analyzes semantic differences at fine-grained text span level, measuring importance of revised spans between junior and senior reports, then merges all revised span scores for final QA assessment.

Result: Evaluation on 12,013 radiology reports shows Sqator achieves competitive QA scores, with revised span importance scores consistent with senior doctors’ judgments.

Conclusion: Sqator provides an effective automated solution for radiology report quality assurance that reduces senior doctor workload while maintaining accuracy comparable to human evaluation.

Abstract: Quality Assurance (QA) for radiology reports refers to judging whether the junior reports (written by junior doctors) are qualified. The QA scores of one junior report are given by the senior doctor(s) after reviewing the image and junior report. This process requires intensive labor costs for senior doctors. Additionally, the QA scores may be inaccurate for reasons like diagnosis bias, the ability of senior doctors, and so on. To address this issue, we propose a Span-level Quality Assurance EvaluaTOR (Sqator) to mark QA scores automatically. Unlike the common document-level semantic comparison method, we try to analyze the semantic difference by exploring more fine-grained text spans. Specifically, Sqator measures QA scores by measuring the importance of revised spans between junior and senior reports, and outputs the final QA scores by merging all revised span scores. We evaluate Sqator using a collection of 12,013 radiology reports. Experimental results show that Sqator can achieve competitive QA scores. Moreover, the importance scores of revised spans can be also consistent with the judgments of senior doctors.

Method: Created MMReview benchmark with 240 papers across 17 domains, featuring multimodal content and expert-written reviews. Designed 13 tasks in 4 categories to evaluate LLMs and MLLMs on review generation, outcome formulation, human alignment, and robustness.

Result: Extensive experiments on 16 open-source and 5 closed-source models demonstrate the benchmark’s thoroughness in assessing automated peer review capabilities.

Conclusion: MMReview provides a standardized foundation for developing automated peer review systems and represents a critical step forward in evaluating AI-assisted academic review.

Abstract: With the rapid growth of academic publications, peer review has become an essential yet time-consuming responsibility within the research community. Large Language Models (LLMs) have increasingly been adopted to assist in the generation of review comments; however, current LLM-based review tasks lack a unified evaluation benchmark to rigorously assess the models’ ability to produce comprehensive, accurate, and human-aligned assessments, particularly in scenarios involving multimodal content such as figures and tables. To address this gap, we propose \textbf{MMReview}, a comprehensive benchmark that spans multiple disciplines and modalities. MMReview includes multimodal content and expert-written review comments for 240 papers across 17 research domains within four major academic disciplines: Artificial Intelligence, Natural Sciences, Engineering Sciences, and Social Sciences. We design a total of 13 tasks grouped into four core categories, aimed at evaluating the performance of LLMs and Multimodal LLMs (MLLMs) in step-wise review generation, outcome formulation, alignment with human preferences, and robustness to adversarial input manipulation. Extensive experiments conducted on 16 open-source models and 5 advanced closed-source models demonstrate the thoroughness of the benchmark. We envision MMReview as a critical step toward establishing a standardized foundation for the development of automated peer review systems.

Method: Pre-trained a 12B parameter base model on 20T tokens using FP8 training, then used Minitron strategy for compression and distillation to enable 128k token inference on limited GPU memory.

Result: Achieves on-par or better accuracy than Qwen3-8B on reasoning benchmarks with up to 6x higher inference throughput for 8k input + 16k output token scenarios.

Conclusion: The hybrid Mamba-Transformer architecture successfully balances accuracy and efficiency, making high-performance reasoning models more accessible on consumer-grade hardware.

Abstract: We introduce Nemotron-Nano-9B-v2, a hybrid Mamba-Transformer language model designed to increase throughput for reasoning workloads while achieving state-of-the-art accuracy compared to similarly-sized models. Nemotron-Nano-9B-v2 builds on the Nemotron-H architecture, in which the majority of the self-attention layers in the common Transformer architecture are replaced with Mamba-2 layers, to achieve improved inference speed when generating the long thinking traces needed for reasoning. We create Nemotron-Nano-9B-v2 by first pre-training a 12-billion-parameter model (Nemotron-Nano-12B-v2-Base) on 20 trillion tokens using an FP8 training recipe. After aligning Nemotron-Nano-12B-v2-Base, we employ the Minitron strategy to compress and distill the model with the goal of enabling inference on up to 128k tokens on a single NVIDIA A10G GPU (22GiB of memory, bfloat16 precision). Compared to existing similarly-sized models (e.g., Qwen3-8B), we show that Nemotron-Nano-9B-v2 achieves on-par or better accuracy on reasoning benchmarks while achieving up to 6x higher inference throughput in reasoning settings like 8k input and 16k output tokens. We are releasing Nemotron-Nano-9B-v2, Nemotron-Nano12B-v2-Base, and Nemotron-Nano-9B-v2-Base checkpoints along with the majority of our pre- and post-training datasets on Hugging Face.

Method: Two core innovations: 1) Data synthesis framework using medical knowledge graphs to generate complex multi-hop QA pairs, 2) Integration of custom private medical retrieval engine with general-purpose tools. Two-stage training with supervised fine-tuning and online reinforcement learning.

Result: Generated 2100+ diverse trajectories across 12 medical specialties, MedResearcher-R1-32B model established new state-of-the-art results on medical benchmarks while maintaining competitive general performance.

Conclusion: Strategic domain-specific innovations in architecture, tool design, and training data enable smaller open-source models to outperform larger proprietary systems in specialized domains like medicine.

Abstract: Recent developments in Large Language Model (LLM)-based agents have shown impressive capabilities spanning multiple domains, exemplified by deep research systems that demonstrate superior performance on complex information-seeking and synthesis tasks. While general-purpose deep research agents have shown impressive capabilities, they struggle significantly with medical domain challenges, as evidenced by leading proprietary systems achieving limited accuracy on complex medical benchmarks. The key limitations are: (1) the model lacks sufficient dense medical knowledge for clinical reasoning, and (2) the framework is constrained by the absence of specialized retrieval tools tailored for medical contexts. We present a medical deep research agent that addresses these challenges through two core innovations. First, we develop a novel data synthesis framework using medical knowledge graphs, extracting the longest chains from subgraphs around rare medical entities to generate complex multi-hop question-answer pairs. Second, we integrate a custom-built private medical retrieval engine alongside general-purpose tools, enabling accurate medical information synthesis. Our approach generates 2100+ diverse trajectories across 12 medical specialties, each averaging 4.2 tool interactions. Through a two-stage training paradigm combining supervised fine-tuning and online reinforcement learning with composite rewards, our MedResearcher-R1-32B model demonstrates exceptional performance, establishing new state-of-the-art results on medical benchmarks while maintaining competitive performance on general deep research tasks. Our work demonstrates that strategic domain-specific innovations in architecture, tool design, and training data construction can enable smaller open-source models to outperform much larger proprietary systems in specialized domains.

Method: Propose reward-shifted speculative sampling (SSS) algorithm where draft model is aligned with human preferences while target model remains unchanged. Modify acceptance criterion and bonus token distribution to exploit distributional shift.

Result: Achieves superior gold reward scores at significantly reduced inference cost in test-time weak-to-strong alignment experiments.

Conclusion: The algorithm effectively addresses efficiency bottleneck of test-time alignment while maintaining alignment quality, validating both effectiveness and efficiency.

Abstract: Aligning large language models (LLMs) with human preferences has become a critical step in their development. Recent research has increasingly focused on test-time alignment, where additional compute is allocated during inference to enhance LLM safety and reasoning capabilities. However, these test-time alignment techniques often incur substantial inference costs, limiting their practical application. We are inspired by the speculative sampling acceleration, which leverages a small draft model to efficiently predict future tokens, to address the efficiency bottleneck of test-time alignment. We introduce the reward-shifted speculative sampling (SSS) algorithm, in which the draft model is aligned with human preferences, while the target model remains unchanged. We theoretically demonstrate that the distributional shift between the aligned draft model and the unaligned target model can be exploited to recover the RLHF optimal solution without actually obtaining it, by modifying the acceptance criterion and bonus token distribution. Our algorithm achieves superior gold reward scores at a significantly reduced inference cost in test-time weak-to-strong alignment experiments, thereby validating both its effectiveness and efficiency.

Method: Proposes SPARK - applies unstructured sparsity by pruning KV cache at channel level, dynamically restoring pruned entries during attention computation. Training-free plug-and-play approach orthogonal to existing compression techniques.

Result: Reduces KV cache storage by over 30% compared to eviction methods while preserving/improving accuracy. With 80% pruning ratio, maintains performance with less than 5% degradation vs baseline. Enables longer sequences within same memory budget.

Conclusion: SPARK effectively addresses KV cache bottleneck through channel-level sparsity, demonstrating robust performance preservation while significantly reducing memory requirements, and is compatible with existing compression techniques for further acceleration.

Abstract: Long-context inference in large language models (LLMs) is increasingly constrained by the KV cache bottleneck: memory usage grows linearly with sequence length, while attention computation scales quadratically. Existing approaches address this issue by compressing the KV cache along the temporal axis through strategies such as token eviction or merging to reduce memory and computational overhead. However, these methods often neglect fine-grained importance variations across feature dimensions (i.e., the channel axis), thereby limiting their ability to effectively balance efficiency and model accuracy. In reality, we observe that channel saliency varies dramatically across both queries and positions: certain feature channels carry near-zero information for a given query, while others spike in relevance. To address this oversight, we propose SPARK, a training-free plug-and-play method that applies unstructured sparsity by pruning KV at the channel level, while dynamically restoring the pruned entries during attention score computation. Notably, our approach is orthogonal to existing KV compression and quantization techniques, making it compatible for integration with them to achieve further acceleration. By reducing channel-level redundancy, SPARK enables processing of longer sequences within the same memory budget. For sequences of equal length, SPARK not only preserves or improves model accuracy but also reduces KV cache storage by over 30% compared to eviction-based methods. Furthermore, even with an aggressive pruning ratio of 80%, SPARK maintains performance with less degradation than 5% compared to the baseline eviction method, demonstrating its robustness and effectiveness. Our code will be available at https://github.com/Xnhyacinth/SparK.

Method: Used large language models to extract insights from both video (TikTok) and text (Twitter) data, investigating dynamic dependencies and spillover effects between social media sentiment and cryptocurrency market indicators.

Result: TikTok’s video-based sentiment significantly influences speculative assets and short-term market trends, while Twitter’s text-based sentiment aligns more closely with long-term dynamics. Cross-platform sentiment integration improves forecasting accuracy by up to 20%.

Conclusion: Video-based social media platforms like TikTok provide valuable sentiment signals distinct from text-based platforms, and integrating multimodal sentiment data from both video and text sources significantly enhances cryptocurrency market forecasting capabilities.

Abstract: As cryptocurrencies gain popularity, the digital asset marketplace becomes increasingly significant. Understanding social media signals offers valuable insights into investor sentiment and market dynamics. Prior research has predominantly focused on text-based platforms such as Twitter. However, video content remains underexplored, despite potentially containing richer emotional and contextual sentiment that is not fully captured by text alone. In this study, we present a multimodal analysis comparing TikTok and Twitter sentiment, using large language models to extract insights from both video and text data. We investigate the dynamic dependencies and spillover effects between social media sentiment and cryptocurrency market indicators. Our results reveal that TikTok’s video-based sentiment significantly influences speculative assets and short-term market trends, while Twitter’s text-based sentiment aligns more closely with long-term dynamics. Notably, the integration of cross-platform sentiment signals improves forecasting accuracy by up to 20%.

Method: ObjexMT benchmark requires models to return a one-sentence base objective and self-reported confidence from multi-turn transcripts. Accuracy is measured via LLM-judge semantic similarity to gold objectives with human-aligned threshold calibration. Metacognition is evaluated with ECE, Brier, Wrong-at-High-Conf, and risk-coverage metrics.

Result: Claude-sonnet-4 achieved best objective-extraction accuracy (0.515) and calibration (ECE 0.296; Brier 0.324). GPT-4.1 and Qwen3-235B-A22B-FP8 tied at 0.441 accuracy but were overconfident (mean confidence ≈0.88 vs accuracy ≈0.44; Wrong-at-0.90 ≈48-52%). Performance varied significantly across datasets (≈0.167-0.865).

Conclusion: LLM judges often misinfer objectives with high confidence when objectives are not explicit. The paper recommends exposing objectives when feasible and gating decisions by confidence otherwise. ObjexMT provides an actionable test for LLM judge qualification.

Abstract: LLM-as-a-Judge (LLMaaJ) now underpins scalable evaluation, yet we lack a decisive test of a judge’s qualification: can it recover a conversation’s latent objective and know when that inference is trustworthy? LLMs degrade under irrelevant or long context; multi-turn jailbreaks further hide goals across turns. We introduce ObjexMT, a benchmark for objective extraction and metacognition. Given a multi-turn transcript, a model must return a one-sentence base objective and a self-reported confidence. Accuracy is computed via LLM-judge semantic similarity to gold objectives, converted to binary correctness by a single human-aligned threshold calibrated once on N = 100 items ($\tau^*=0.61$). Metacognition is evaluated with ECE, Brier, Wrong-at-High-Conf, and risk-coverage. Across gpt-4.1, claude-sonnet-4, and Qwen3-235B-A22B-FP8 on SafeMTData_Attack600, SafeMTData_1K, MHJ, and CoSafe, claude-sonnet-4 attains the best objective-extraction accuracy (0.515) and calibration (ECE 0.296; Brier 0.324); gpt-4.1 and Qwen3-235B-A22B-FP8 tie at 0.441 but are overconfident (mean confidence $\approx$0.88 vs. accuracy $\approx$0.44; Wrong-at-0.90 $\approx$48-52%). Performance varies by dataset ($\approx$0.167-0.865). ObjexMT thus supplies an actionable test for LLM judges: when objectives are not explicit, judges often misinfer them with high confidence. We recommend exposing objectives when feasible and gating decisions by confidence otherwise. Code and data at https://github.com/hyunjun1121/ObjexMT_dataset.

Method: Proposes TextTopoOOD framework with four OOD scenarios: attribute-level shifts via text augmentations, structural shifts through edge rewiring, thematically-guided label shifts, and domain-based divisions. Introduces TNT-OOD model with cross-attention module to fuse local structure into text representations and HyperNetwork for node-specific transformations.

Result: Experiments on 11 datasets across four OOD scenarios demonstrate the framework’s effectiveness in evaluating OOD detection challenges in text-rich networks.

Conclusion: The proposed TextTopoOOD framework and TNT-OOD model successfully address the complex interplay between text and topology in OOD detection, providing a comprehensive evaluation approach for diverse OOD scenarios in text-rich networks.

Abstract: Out-of-distribution (OOD) detection remains challenging in text-rich networks, where textual features intertwine with topological structures. Existing methods primarily address label shifts or rudimentary domain-based splits, overlooking the intricate textual-structural diversity. For example, in social networks, where users represent nodes with textual features (name, bio) while edges indicate friendship status, OOD may stem from the distinct language patterns between bot and normal users. To address this gap, we introduce the TextTopoOOD framework for evaluating detection across diverse OOD scenarios: (1) attribute-level shifts via text augmentations and embedding perturbations; (2) structural shifts through edge rewiring and semantic connections; (3) thematically-guided label shifts; and (4) domain-based divisions. Furthermore, we propose TNT-OOD to model the complex interplay between Text aNd Topology using: 1) a novel cross-attention module to fuse local structure into node-level text representations, and 2) a HyperNetwork to generate node-specific transformation parameters. This aligns topological and semantic features of ID nodes, enhancing ID/OOD distinction across structural and textual shifts. Experiments on 11 datasets across four OOD scenarios demonstrate the nuanced challenge of TextTopoOOD for evaluating OOD detection in text-rich networks.

Method: Used comprehensive corpus of parliamentary speeches from six European countries, employed NLP techniques to estimate parliamentarian sentiment, compared negativity levels in references to opposing groups vs own groups.

Result: Found consistent affective polarization across all six European parliaments. Activity correlates with negativity but no difference in polarization between less and more active MPs. Reciprocity is a contributing mechanism in affective polarization.

Conclusion: Affective polarization is a consistent feature across European parliaments, with reciprocity playing a key role in driving negative interactions between opposing political groups.

Abstract: Affective polarization, characterized by increased negativity and hostility towards opposing groups, has become a prominent feature of political discourse worldwide. Our study examines the presence of this type of polarization in a selection of European parliaments in a fully automated manner. Utilizing a comprehensive corpus of parliamentary speeches from the parliaments of six European countries, we employ natural language processing techniques to estimate parliamentarian sentiment. By comparing the levels of negativity conveyed in references to individuals from opposing groups versus one’s own, we discover patterns of affectively polarized interactions. The findings demonstrate the existence of consistent affective polarization across all six European parliaments. Although activity correlates with negativity, there is no observed difference in affective polarization between less active and more active members of parliament. Finally, we show that reciprocity is a contributing mechanism in affective polarization between parliamentarians across all six parliaments.

Method: Tune two small proxy models and use the difference in their output distributions to shift the original LLM’s distribution without weight modification, allowing continuous sensitivity adjustment.

Result: Achieves continuous and precise control over LLMs’ sensitivity to contextual knowledge, enabling both increased and reduced sensitivity, allowing flexible prioritization of contextual or parametric knowledge.

Conclusion: CSKS provides an effective, lightweight solution for steering LLMs’ knowledge sensitivity without model modification, validated through synthetic data and real conflict datasets.

Abstract: In Large Language Models (LLMs) generation, there exist knowledge conflicts and scenarios where parametric knowledge contradicts knowledge provided in the context. Previous works studied tuning, decoding algorithms, or locating and editing context-aware neurons to adapt LLMs to be faithful to new contextual knowledge. However, they are usually inefficient or ineffective for large models, not workable for black-box models, or unable to continuously adjust LLMs’ sensitivity to the knowledge provided in the context. To mitigate these problems, we propose CSKS (Continuously Steering Knowledge Sensitivity), a simple framework that can steer LLMs’ sensitivity to contextual knowledge continuously at a lightweight cost. Specifically, we tune two small LMs (i.e. proxy models) and use the difference in their output distributions to shift the original distribution of an LLM without modifying the LLM weights. In the evaluation process, we not only design synthetic data and fine-grained metrics to measure models’ sensitivity to contextual knowledge but also use a real conflict dataset to validate CSKS’s practical efficacy. Extensive experiments demonstrate that our framework achieves continuous and precise control over LLMs’ sensitivity to contextual knowledge, enabling both increased sensitivity and reduced sensitivity, thereby allowing LLMs to prioritize either contextual or parametric knowledge as needed flexibly. Our data and code are available at https://github.com/OliveJuiceLin/CSKS.

Method: Proposed Input-Reformulation Multi-Agent (IRMA) framework that automatically reformulates user queries augmented with relevant domain rules and tool suggestions.

Result: IRMA outperforms ReAct, Function Calling, and Self-Reflection by 16.1%, 12.7%, and 19.1% respectively in overall pass scores.

Conclusion: IRMA demonstrates superior reliability and consistency compared to other methods in dynamic environments requiring tool use.

Abstract: Recent advances in reasoning and planning capabilities of large language models (LLMs) have enabled their potential as autonomous agents capable of tool use in dynamic environments. However, in multi-turn conversational environments like $\tau$-bench, these agents often struggle with consistent reasoning, adherence to domain-specific policies, and extracting correct information over a long horizon of tool-calls and conversation. To capture and mitigate these failures, we conduct a comprehensive manual analysis of the common errors occurring in the conversation trajectories. We then experiment with reformulations of inputs to the tool-calling agent for improvement in agent decision making. Finally, we propose the Input-Reformulation Multi-Agent (IRMA) framework, which automatically reformulates user queries augmented with relevant domain rules and tool suggestions for the tool-calling agent to focus on. The results show that IRMA significantly outperforms ReAct, Function Calling, and Self-Reflection by 16.1%, 12.7%, and 19.1%, respectively, in overall pass^5 scores. These findings highlight the superior reliability and consistency of IRMA compared to other methods in dynamic environments.

Method: Uses capsule networks with dynamic routing mechanism for sentential relation extraction, tested on multiple datasets including Tacred, Tacredrev, Retacred, Conll04, and Wikidata.

Result: Outperforms state-of-the-art on common datasets but shows low performance on Wikidata due to label noise. Demonstrates better re-representation capability compared to vanilla models.

Conclusion: Noise in distantly supervised datasets and re-representation capability are significant challenges in sentential relation extraction that need to be addressed.

Abstract: Sentential relation extraction (RE) is an important task in natural language processing (NLP). In this paper we propose to do sentential RE with dynamic routing in capsules. We first show that the proposed approach outperform state of the art on common sentential relation extraction datasets Tacred, Tacredrev, Retacred, and Conll04. We then investigate potential reasons for its good performance on the mentioned datasets, and yet low performance on another similar, yet larger sentential RE dataset, Wikidata. As such, we identify noise in Wikidata labels as one of the reasons that can hinder performance. Additionally, we show associativity of better performance with better re-representation, a term from neuroscience referred to change of representation in human brain to improve the match at comparison time. As example, in the given analogous terms King:Queen::Man:Woman, at comparison time, and as a result of re-representation, the similarity between related head terms (King,Man), and tail terms (Queen,Woman) increases. As such, our observation show that our proposed model can do re-representation better than the vanilla model compared with. To that end, beside noise in the labels of the distantly supervised RE datasets, we propose re-representation as a challenge in sentential RE.

Method: Integrates LLMs with symbolic solvers, translates plain-text tax rules into formal logic programs, and uses intelligently retrieved exemplars for formal case representations.

Result: Dramatically improved performance on the SARA dataset with costs reduced well below real-world averages of $270 and 13 hours per filing.

Conclusion: Neuro-symbolic architectures show promise for equitable access to reliable tax assistance by combining the strengths of LLMs and symbolic reasoning.

Abstract: According to the United States Internal Revenue Service, ‘’the average American spends $$270$ and 13 hours filing their taxes’’. Even beyond the U.S., tax filing requires complex reasoning, combining application of overlapping rules with numerical calculations. Because errors can incur costly penalties, any automated system must deliver high accuracy and auditability, making modern large language models (LLMs) poorly suited for this task. We propose an approach that integrates LLMs with a symbolic solver to calculate tax obligations. We evaluate variants of this system on the challenging StAtutory Reasoning Assessment (SARA) dataset, and include a novel method for estimating the cost of deploying such a system based on real-world penalties for tax errors. We further show how combining up-front translation of plain-text rules into formal logic programs, combined with intelligently retrieved exemplars for formal case representations, can dramatically improve performance on this task and reduce costs to well below real-world averages. Our results demonstrate the promise and economic feasibility of neuro-symbolic architectures for increasing equitable access to reliable tax assistance.

Method: Tested ChatGPT, Gemini, and Claude evaluating blog posts under four label conditions (no labels, true labels, two false labels) using preference voting and quality ratings (Coherence, Informativeness, Conciseness) expressed as percentages.

Result: Strong asymmetrical bias found - Claude labels consistently boosted scores while Gemini labels depressed them. False labels frequently reversed rankings with 50pp preference vote shifts and 12pp quality rating changes. Gemini’s self-scores collapsed under true labels while Claude’s self-preference intensified.

Conclusion: Perceived model identity heavily distorts LLM evaluation judgments, requiring blind or multimodel evaluation protocols to ensure fair benchmarking.

Abstract: Large language models (LLMs) are increasingly used to evaluate outputs, yet their judgments may be influenced. This study examines bias in self- and cross-model evaluations by ChatGPT, Gemini, and Claude under four conditions: no labels, true labels, and two false-label scenarios. Blog posts authored by each model were evaluated by all three using both overall preference voting and quality ratings for Coherence, Informativeness, and Conciseness, with all scores expressed as percentages for direct comparison. Results reveal striking asymmetries: the “Claude” label consistently boosts scores, while the “Gemini” label consistently depresses them, regardless of actual content. False labels frequently reversed rankings, producing shifts of up to 50 percentage points in preference votes and up to 12 percentage points in converted quality ratings. Gemini’s self-scores collapsed under true labels, while Claude’s self-preference intensified. These findings show that perceived model identity can heavily distort high-level judgments and subtly influence detailed quality ratings, underscoring the need for blind or multimodel evaluation protocols to ensure fairness in LLM benchmarking.

Method: Combined YOLOv8 for object segmentation, LSTM for hand motion sequence analysis, and Whisper ASR for audio processing to extract comprehensive cooking data, then used TinyLLaMa LLM for recipe prediction and text generation

Result: Developed a task-specific system that performs well in complex kitchen environments, demonstrating practical computer vision applications for daily life activities

Conclusion: This work successfully extends computer vision capabilities to kitchen tasks and shows potential for many other crucial daily life applications through multimodal AI integration

Abstract: This is a research exploring existing models and fine tuning them to combine a YOLOv8 segmentation model, a LSTM model trained on hand point motion sequence and a ASR (whisper-base) to extract enough data for a LLM (TinyLLaMa) to predict the recipe and generate text creating a step by step guide for the cooking procedure. All the data were gathered by the author for a robust task specific system to perform best in complex and challenging environments proving the extension and endless application of computer vision in daily activities such as kitchen work. This work extends the field for many more crucial task of our day to day life.

Method: Uses feature fusion network, multi-teacher distillation with pre-computed text features from CLIP teachers, KL scatter for distribution matching, and adaptive dynamic weighting based on gradient space diversity.

Result: Achieves superior performance on three benchmark datasets while significantly reducing model complexity.

Conclusion: AMMKD provides an effective and flexible solution for deploying lightweight image-text retrieval models on resource-constrained devices.

Abstract: The success of large-scale visual language pretraining (VLP) models has driven widespread adoption of image-text retrieval tasks. However, their deployment on mobile devices remains limited due to large model sizes and computational complexity. We propose Adaptive Multi-Modal Multi-Teacher Knowledge Distillation (AMMKD), a novel framework that integrates multi-modal feature fusion, multi-teacher distillation, and adaptive optimization to deliver lightweight yet effective retrieval models. Specifically, our method begins with a feature fusion network that extracts and merges discriminative features from both the image and text modalities. To reduce model parameters and further improve performance, we design a multi-teacher knowledge distillation framework to pre-train two CLIP teacher models. We decouple modalities by pre-computing and storing text features as class vectors via the teacher text encoder to enhance efficiency. To better align teacher and student outputs, we apply KL scatter for probability distribution matching. Finally, we design an adaptive dynamic weighting scheme that treats multi-teacher distillation as a multi-objective optimization problem. By leveraging gradient space diversity, we dynamically adjust the influence of each teacher, reducing conflicts and guiding the student toward more optimal learning directions. Extensive experiments on three benchmark datasets demonstrate that AMMKD achieves superior performance while significantly reducing model complexity, validating its effectiveness and flexibility.

Method: Hybrid approach combining monocular depth estimation using Vision Transformers, multi-component anomaly detection, and weighted curiosity scoring that balances known value, anomaly signals, depth variance, and surface roughness.

Result: State-of-the-art performance with AUROC of 0.94, AUPRC of 0.89, F1-Score of 0.87 on Mars rover datasets, and 23% reduction in false positives while maintaining high detection sensitivity across diverse terrain types.

Conclusion: The hybrid fusion approach demonstrates significant improvements in target prioritization accuracy and provides a comprehensive framework for autonomous planetary exploration with reduced false positive rates.

Abstract: We present ARTPS (Autonomous Rover Target Prioritization System), a novel hybrid AI system that combines depth estimation, anomaly detection, and learnable curiosity scoring for autonomous exploration of planetary surfaces. Our approach integrates monocular depth estimation using Vision Transformers with multi-component anomaly detection and a weighted curiosity score that balances known value, anomaly signals, depth variance, and surface roughness. The system achieves state-of-the-art performance with AUROC of 0.94, AUPRC of 0.89, and F1-Score of 0.87 on Mars rover datasets. We demonstrate significant improvements in target prioritization accuracy through ablation studies and provide comprehensive analysis of component contributions. The hybrid fusion approach reduces false positives by 23% while maintaining high detection sensitivity across diverse terrain types.

Method: Developed two novel metrics: Sustainable Harmonic Mean (FMS) that combines accumulated energy consumption and performance via harmonic mean, and Area Under Sustainability Curve (ASC) that constructs performance-power consumption curves. Tested across image classification, segmentation, pose estimation, and batch/online learning tasks.

Result: The evaluation system successfully provides quantitative basis for cross-task algorithm assessment and facilitates the transition of green AI research from theory to practice. The framework code is made available for industry adoption.

Conclusion: The proposed sustainability evaluation system offers a practical approach to measure and compare algorithm energy efficiency, supporting the establishment of industry standards for green AI and promoting environmentally conscious deep learning practices.

Abstract: This work focuses on the high carbon emissions generated by deep learning model training, specifically addressing the core challenge of balancing algorithm performance and energy consumption. It proposes an innovative two-dimensional sustainability evaluation system. Different from the traditional single performance-oriented evaluation paradigm, this study pioneered two quantitative indicators that integrate energy efficiency ratio and accuracy: the sustainable harmonic mean (FMS) integrates accumulated energy consumption and performance parameters through the harmonic mean to reveal the algorithm performance under unit energy consumption; the area under the sustainability curve (ASC) constructs a performance-power consumption curve to characterize the energy efficiency characteristics of the algorithm throughout the cycle. To verify the universality of the indicator system, the study constructed benchmarks in various multimodal tasks, including image classification, segmentation, pose estimation, and batch and online learning. Experiments demonstrate that the system can provide a quantitative basis for evaluating cross-task algorithms and promote the transition of green AI research from theory to practice. Our sustainability evaluation framework code can be found here, providing methodological support for the industry to establish algorithm energy efficiency standards.

Method: Introduces MESTI (Micro-expression Spatio-Temporal Image) that transforms video sequences into single images while preserving micro-movement characteristics. Presents MEGANet with a novel Gradient Attention block to enhance fine-grained motion feature extraction.

Result: MESTI outperforms existing input modalities across three CNN architectures. Replacing inputs of previous MER networks with MESTI consistently improves performance. MEGANet achieves state-of-the-art results on CASMEII and SAMM datasets, with the MEGANet+MESTI combination setting the highest accuracy benchmark.

Conclusion: MESTI is a superior input modality and MEGANet is an advanced recognition network that paves the way for more effective micro-expression recognition systems in various applications.

Abstract: Micro-expression recognition (MER) is a challenging task due to the subtle and fleeting nature of micro-expressions. Traditional input modalities, such as Apex Frame, Optical Flow, and Dynamic Image, often fail to adequately capture these brief facial movements, resulting in suboptimal performance. In this study, we introduce the Micro-expression Spatio-Temporal Image (MESTI), a novel dynamic input modality that transforms a video sequence into a single image while preserving the essential characteristics of micro-movements. Additionally, we present the Micro-expression Gradient Attention Network (MEGANet), which incorporates a novel Gradient Attention block to enhance the extraction of fine-grained motion features from micro-expressions. By combining MESTI and MEGANet, we aim to establish a more effective approach to MER. Extensive experiments were conducted to evaluate the effectiveness of MESTI, comparing it with existing input modalities across three CNN architectures (VGG19, ResNet50, and EfficientNetB0). Moreover, we demonstrate that replacing the input of previously published MER networks with MESTI leads to consistent performance improvements. The performance of MEGANet, both with MESTI and Dynamic Image, is also evaluated, showing that our proposed network achieves state-of-the-art results on the CASMEII and SAMM datasets. The combination of MEGANet and MESTI achieves the highest accuracy reported to date, setting a new benchmark for micro-expression recognition. These findings underscore the potential of MESTI as a superior input modality and MEGANet as an advanced recognition network, paving the way for more effective MER systems in a variety of applications.

Method: A weakly-supervised framework with three innovations: 1) Prototype-driven staining pattern transfer with content-style decoupling, 2) GapBridge cyclic registration-synthesis framework for deformable structural alignment between H&E and IHC domains, and 3) Deformation-aware adversarial learning where generator and registration network jointly optimize a style-focused discriminator.

Result: Extensive experiments show PRINTER achieves superior performance in preserving H&E staining details and virtual staining fidelity, outperforming state-of-the-art methods.

Conclusion: PRINTER provides a robust and scalable solution for virtual staining that advances computational pathology by enabling accurate correlation between H&E morphology and IHC biomarker expression without spatial misalignment issues.

Abstract: Tumor spatial heterogeneity analysis requires precise correlation between Hematoxylin and Eosin H&E morphology and immunohistochemical (IHC) biomarker expression, yet current methods suffer from spatial misalignment in consecutive sections, severely compromising in situ pathological interpretation. In order to obtain a more accurate virtual staining pattern, We propose PRINTER, a weakly-supervised framework that integrates PRototype-drIven content and staiNing patTERn decoupling and deformation-aware adversarial learning strategies designed to accurately learn IHC staining patterns while preserving H&E staining details. Our approach introduces three key innovations: (1) A prototype-driven staining pattern transfer with explicit content-style decoupling; and (2) A cyclic registration-synthesis framework GapBridge that bridges H&E and IHC domains through deformable structural alignment, where registered features guide cross-modal style transfer while synthesized outputs iteratively refine the registration;(3) Deformation-Aware Adversarial Learning: We propose a training framework where a generator and deformation-aware registration network jointly adversarially optimize a style-focused discriminator. Extensive experiments demonstrate that PRINTER effectively achieves superior performance in preserving H&E staining details and virtual staining fidelity, outperforming state-of-the-art methods. Our work provides a robust and scalable solution for virtual staining, advancing the field of computational pathology.

Method: Treats voxels as tokens and uses a discrete diffusion language model to generate 3D voxel structures, extending discrete diffusion beyond sequential domains to create spatially coherent 3D structures.

Result: Outperforms prior baselines and auto-regressive formulations, producing realistic and coherent structures even when trained on data with over 98% sparsity. Evaluated on Minecraft house structures from 3D-Craft dataset.

Conclusion: Discrete diffusion is a promising framework for 3D sparse voxel generative modeling, successfully addressing the challenges of sparsity and class imbalance in voxel structure generation.

Abstract: Generating realistic sparse multi-category 3D voxel structures is difficult due to the cubic memory scaling of voxel structures and moreover the significant class imbalance caused by sparsity. We introduce Scaffold Diffusion, a generative model designed for sparse multi-category 3D voxel structures. By treating voxels as tokens, Scaffold Diffusion uses a discrete diffusion language model to generate 3D voxel structures. We show that discrete diffusion language models can be extended beyond inherently sequential domains such as text to generate spatially coherent 3D structures. We evaluate on Minecraft house structures from the 3D-Craft dataset and demonstrate that, unlike prior baselines and an auto-regressive formulation, Scaffold Diffusion produces realistic and coherent structures even when trained on data with over 98% sparsity. We provide an interactive viewer where readers can visualize generated samples and the generation process. Our results highlight discrete diffusion as a promising framework for 3D sparse voxel generative modeling.

Method: Three-step framework: 1) Face Aware Prompt Enhancement using GPT-4o to add facial details, 2) Prompt Aware Reference Image Enhancement using identity-preserving image generator, 3) ID-Aware Spatiotemporal Guidance Enhancement using unified gradients during generation.

Result: Outperforms prior work, achieves state-of-the-art performance, wins first place in ACM Multimedia 2025 challenge, validated on 1000-video test set with automatic and human evaluations.

Conclusion: TPIGE framework provides significant performance gains at minimal cost, demonstrating strong generality and effectiveness for identity-preserving text-to-video generation without requiring expensive fine-tuning.

Abstract: Identity-preserving text-to-video (IPT2V) generation creates videos faithful to both a reference subject image and a text prompt. While fine-tuning large pretrained video diffusion models on ID-matched data achieves state-of-the-art results on IPT2V, data scarcity and high tuning costs hinder broader improvement. We thus introduce a Training-Free Prompt, Image, and Guidance Enhancement (TPIGE) framework that bridges the semantic gap between the video description and the reference image and design sampling guidance that enhances identity preservation and video quality, achieving performance gains at minimal cost.Specifically, we first propose Face Aware Prompt Enhancement, using GPT-4o to enhance the text prompt with facial details derived from the reference image. We then propose Prompt Aware Reference Image Enhancement, leveraging an identity-preserving image generator to refine the reference image, rectifying conflicts with the text prompt. The above mutual refinement significantly improves input quality before video generation. Finally, we propose ID-Aware Spatiotemporal Guidance Enhancement, utilizing unified gradients to optimize identity preservation and video quality jointly during generation.Our method outperforms prior work and is validated by automatic and human evaluations on a 1000 video test set, winning first place in the ACM Multimedia 2025 Identity-Preserving Video Generation Challenge, demonstrating state-of-the-art performance and strong generality. The code is available at https://github.com/Andyplus1/IPT2V.git.

Method: SGLC framework with two components: Global Features Generator (GFG) for broad contextual understanding and Local Features Enhancer (LFE) for refining localized details and pixel-level features. Integrated with Uformer architecture.

Result: Experimental results show considerable improvement in peak signal-to-noise ratio (PSNR) on high-resolution datasets, demonstrating effectiveness for large-scale imagery dehazing.

Conclusion: SGLC provides a model-agnostic solution that enables any dehazing network to effectively combine scene-level cues with granular details, significantly improving visual fidelity in high-resolution environments.

Abstract: Addressing the challenge of removing atmospheric fog or haze from digital images, known as image dehazing, has recently gained significant traction in the computer vision community. Although contemporary dehazing models have demonstrated promising performance, few have thoroughly investigated high-resolution imagery. In such scenarios, practitioners often resort to downsampling the input image or processing it in smaller patches, which leads to a notable performance degradation. This drop is primarily linked to the difficulty of effectively combining global contextual information with localized, fine-grained details as the spatial resolution grows. In this chapter, we propose a novel framework, termed the Streamlined Global and Local Features Combinator (SGLC), to bridge this gap and enable robust dehazing for high-resolution inputs. Our approach is composed of two principal components: the Global Features Generator (GFG) and the Local Features Enhancer (LFE). The GFG produces an initial dehazed output by focusing on broad contextual understanding of the scene. Subsequently, the LFE refines this preliminary output by enhancing localized details and pixel-level features, thereby capturing the interplay between global appearance and local structure. To evaluate the effectiveness of SGLC, we integrated it with the Uformer architecture, a state-of-the-art dehazing model. Experimental results on high-resolution datasets reveal a considerable improvement in peak signal-to-noise ratio (PSNR) when employing SGLC, indicating its potency in addressing haze in large-scale imagery. Moreover, the SGLC design is model-agnostic, allowing any dehazing network to be augmented with the proposed global-and-local feature fusion mechanism. Through this strategy, practitioners can harness both scene-level cues and granular details, significantly improving visual fidelity in high-resolution environments.

Method: Proposes Robust Alignment Learning (RAL) with: 1) probabilistic modeling encoding videos/queries as multivariate Gaussian distributions for uncertainty quantification, 2) learnable confidence gates to dynamically weight query word similarity.

Result: RAL demonstrates effectiveness across diverse retrieval backbones as a plug-and-play solution through extensive experiments.

Conclusion: The framework successfully addresses data uncertainty in PRVR through probabilistic modeling and dynamic confidence weighting, improving retrieval robustness.

Abstract: Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos partially relevant to a given query. The core challenge lies in learning robust query-video alignment against spurious semantic correlations arising from inherent data uncertainty: 1) query ambiguity, where the query incompletely characterizes the target video and often contains uninformative tokens, and 2) partial video relevance, where abundant query-irrelevant segments introduce contextual noise in cross-modal alignment. Existing methods often focus on enhancing multi-scale clip representations and retrieving the most relevant clip. However, the inherent data uncertainty in PRVR renders them vulnerable to distractor videos with spurious similarities, leading to suboptimal performance. To fill this research gap, we propose Robust Alignment Learning (RAL) framework, which explicitly models the uncertainty in data. Key innovations include: 1) we pioneer probabilistic modeling for PRVR by encoding videos and queries as multivariate Gaussian distributions. This not only quantifies data uncertainty but also enables proxy-level matching to capture the variability in cross-modal correspondences; 2) we consider the heterogeneous informativeness of query words and introduce learnable confidence gates to dynamically weight similarity. As a plug-and-play solution, RAL can be seamlessly integrated into the existing architectures. Extensive experiments across diverse retrieval backbones demonstrate its effectiveness.

Method: Used UNI foundation model features and tested a simple k-nearest neighbor classifier. Then applied self-supervised learning methods on the same features for abnormality detection across 158 drug safety assessment studies.

Result: k-NN classifier performed at chance level, showing UNI features alone are insufficient. Self-supervised method achieved AUC of 0.62 and negative predictive value of 89%, demonstrating better-than-chance performance.

Conclusion: Self-supervised methods show promise for kidney abnormality detection and could be used to rule out normal slides, reducing time and costs in drug safety assessment studies with further development.

Abstract: Kidney abnormality detection is required for all preclinical drug development. It involves a time-consuming and costly examination of hundreds to thousands of whole-slide images per drug safety study, most of which are normal, to detect any subtle changes indicating toxic effects. In this study, we present the first large-scale self-supervised abnormality detection model for kidney toxicologic pathology, spanning drug safety assessment studies from 158 compounds. We explore the complexity of kidney abnormality detection on this scale using features extracted from the UNI foundation model (FM) and show that a simple k-nearest neighbor classifier on these features performs at chance, demonstrating that the FM-generated features alone are insufficient for detecting abnormalities. We then demonstrate that a self-supervised method applied to the same features can achieve better-than-chance performance, with an area under the receiver operating characteristic curve of 0.62 and a negative predictive value of 89%. With further development, such a model can be used to rule out normal slides in drug safety assessment studies, reducing the costs and time associated with drug development.

Method: The authors created a novel dataset specifically designed for waste classification in real-world scenarios with complex environments and deformed objects, and developed an in-depth evaluation approach to assess VLLM robustness and accuracy.

Result: The comprehensive analysis revealed that current VLLMs lack sufficient robustness to perform effectively in complex environments with deformed objects, highlighting performance gaps in challenging conditions.

Conclusion: There is a critical need for further advancements in VLLM robustness to enable better performance in complex real-world environments, and the introduced dataset provides valuable insights for future research in this direction.

Abstract: Recent advancements in Large Language Models (LLMs) have paved the way for Vision Large Language Models (VLLMs) capable of performing a wide range of visual understanding tasks. While LLMs have demonstrated impressive performance on standard natural images, their capabilities have not been thoroughly explored in cluttered datasets where there is complex environment having deformed shaped objects. In this work, we introduce a novel dataset specifically designed for waste classification in real-world scenarios, characterized by complex environments and deformed shaped objects. Along with this dataset, we present an in-depth evaluation approach to rigorously assess the robustness and accuracy of VLLMs. The introduced dataset and comprehensive analysis provide valuable insights into the performance of VLLMs under challenging conditions. Our findings highlight the critical need for further advancements in VLLM’s robustness to perform better in complex environments. The dataset and code for our experiments will be made publicly available.

Method: Curated a dataset with shot boundaries for 237 soccer matches from Spanish, French, and Italian leagues using broadcast footage from SoccerNet. Developed a baseline model specifically designed for soccer video summarization.

Result: The baseline model achieved an F1 score of 0.3956 on the test set. Proposed a new length-constrained evaluation metric for more objective assessment of generated summaries.

Conclusion: Provides a valuable benchmark dataset and baseline approach for soccer video summarization research, with publicly available dataset and code to facilitate further development in sports highlight generation.

Abstract: Video summarization aims to extract key shots from longer videos to produce concise and informative summaries. One of its most common applications is in sports, where highlight reels capture the most important moments of a game, along with notable reactions and specific contextual events. Automatic summary generation can support video editors in the sports media industry by reducing the time and effort required to identify key segments. However, the lack of publicly available datasets poses a challenge in developing robust models for sports highlight generation. In this paper, we address this gap by introducing a curated dataset for soccer video summarization, designed to serve as a benchmark for the task. The dataset includes shot boundaries for 237 matches from the Spanish, French, and Italian leagues, using broadcast footage sourced from the SoccerNet dataset. Alongside the dataset, we propose a baseline model specifically designed for this task, which achieves an F1 score of 0.3956 in the test set. Furthermore, we propose a new metric constrained by the length of each target summary, enabling a more objective evaluation of the generated content. The dataset and code are available at https://ipcv.github.io/SoccerHigh/.

Method: Two-step approach: 1) Transform text query into visual query using generative diffusion model, 2) Estimate image-to-image similarity with vision model. Includes aggregation network to combine multiple generated images and fuse similarity scores across modalities.

Result: Extensive evaluations show the approach consistently outperforms retrieval methods relying solely on text queries.

Conclusion: The proposed method effectively bridges the modality gap between text and images by leveraging generative diffusion models and vision encoders, improving text-to-image retrieval performance.

Abstract: This work explores text-to-image retrieval for queries that specify or describe a semantic category. While vision-and-language models (VLMs) like CLIP offer a straightforward open-vocabulary solution, they map text and images to distant regions in the representation space, limiting retrieval performance. To bridge this modality gap, we propose a two-step approach. First, we transform the text query into a visual query using a generative diffusion model. Then, we estimate image-to-image similarity with a vision model. Additionally, we introduce an aggregation network that combines multiple generated images into a single vector representation and fuses similarity scores across both query modalities. Our approach leverages advancements in vision encoders, VLMs, and text-to-image generation models. Extensive evaluations show that it consistently outperforms retrieval methods relying solely on text queries. Source code is available at: https://github.com/faixan-khan/cletir

Method: Introduce PRISM benchmark to evaluate MLLMs on refusing biometric queries and detecting implicit leakage, and create Safe-LLaVA dataset by systematically removing biometric information from LLaVA data.

Result: Evaluation reveals widespread biometric leakage across MLLMs, and models fine-tuned on Safe-LLaVA show significant reduction in privacy violations.

Conclusion: PRISM and Safe-LLaVA establish new standards for privacy-aligned development and evaluation of multimodal language models, addressing critical biometric privacy concerns.

Abstract: Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in vision-language tasks. However, these models often infer and reveal sensitive biometric attributes - such as race, gender, age, body weight, and eye color - even when such information is not explicitly requested. This raises critical concerns, particularly in real-world applications and socially-sensitive domains. Despite increasing awareness, no publicly available dataset or benchmark exists to comprehensively evaluate or mitigate biometric leakage in MLLMs. To address this gap, we introduce PRISM (Privacy-aware Evaluation of Responses in Sensitive Modalities), a new benchmark designed to assess MLLMs on two fronts: (1) refuse biometric-related queries and (2) implicit biometric leakage in general responses while maintaining semantic faithfulness. Further, we conduct a detailed audit of the widely used LLaVA datasets and uncover extensive biometric leakage across pretraining and instruction data. To address this, we present Safe-LLaVA dataset, the first privacy-preserving MLLM training dataset constructed by systematically removing explicit and implicit biometric information from LLaVA dataset. Our evaluations on PRISM reveal biometric leakages across MLLMs for different attributes, highlighting the detailed privacy-violations. We also fine-tune a model on Safe-LLaVA dataset and show that it substantially reduces the biometric leakages. Together, Safe-LLaVA & PRISM set a new standard for privacy-aligned development and evaluation of MLLMs. The Safe-LLaVA dataset & PRISM benchmark are publicly available at https://huggingface.co/datasets/kyh9191/Safe-LLaVA, and the source code is available at https://github.com/Kimyounggun99/Safe-LLaVA.git.

Method: Proposes VEME framework with: 1) cross-modal alignment bridging objects, spatial representations and visual semantics with spatio-temporal cues; 2) dynamic implicit cognitive map activated by world embedding for task-relevant memory recall; 3) instruction-based navigation leveraging embodied priors for long-term planning.

Result: Achieves 1%-3% accuracy and exploration efficiency improvement on VSI-Bench and VLN-CE benchmarks compared to traditional approaches.

Conclusion: The method significantly improves reasoning and planning in dynamic environments by embedding geometry-aware spatio-temporal episodic experiences, enhancing generalization in unseen scenes.

Abstract: Achieving human-like reasoning in deep learning models for complex tasks in unknown environments remains a critical challenge in embodied intelligence. While advanced vision-language models (VLMs) excel in static scene understanding, their limitations in spatio-temporal reasoning and adaptation to dynamic, open-set tasks like task-oriented navigation and embodied question answering (EQA) persist due to inadequate modeling of fine-grained spatio-temporal cues and physical world comprehension. To address this, we propose VEME, a novel cross-modal alignment method that enhances generalization in unseen scenes by learning an ego-centric, experience-centered world model. Our framework integrates three key components: (1) a cross-modal alignment framework bridging objects, spatial representations, and visual semantics with spatio-temporal cues to enhance VLM in-context learning; (2) a dynamic, implicit cognitive map activated by world embedding to enable task-relevant geometric-semantic memory recall; and (3) an instruction-based navigation and reasoning framework leveraging embodied priors for long-term planning and efficient exploration. By embedding geometry-aware spatio-temporal episodic experiences, our method significantly improves reasoning and planning in dynamic environments. Experimental results on VSI-Bench and VLN-CE demonstrate 1%-3% accuracy and exploration efficiency improvement compared to traditional approaches.

Method: Dual-branch neural network that extracts and fuses features from breast ultrasound images and patient metadata from 81 subjects, using class-aware sampling and subject-stratified 5-fold cross-validation to handle class imbalance and prevent data leakage.

Result: Multimodal method outperformed unimodal baselines, with ConvNeXt and ResNet18 achieving best performance (AUC-ROC: 0.9427/0.9349; F1-scores: 0.6720/0.7294) in classifying benign vs borderline/malignant tumors.

Conclusion: Multimodal AI shows potential as a non-invasive diagnostic tool that could reduce unnecessary biopsies and improve clinical decision-making in breast tumor management.

Abstract: Phyllodes tumors (PTs) are rare fibroepithelial breast lesions that are difficult to classify preoperatively due to their radiological similarity to benign fibroadenomas. This often leads to unnecessary surgical excisions. To address this, we propose a multimodal deep learning framework that integrates breast ultrasound (BUS) images with structured clinical data to improve diagnostic accuracy. We developed a dual-branch neural network that extracts and fuses features from ultrasound images and patient metadata from 81 subjects with confirmed PTs. Class-aware sampling and subject-stratified 5-fold cross-validation were applied to prevent class imbalance and data leakage. The results show that our proposed multimodal method outperforms unimodal baselines in classifying benign versus borderline/malignant PTs. Among six image encoders, ConvNeXt and ResNet18 achieved the best performance in the multimodal setting, with AUC-ROC scores of 0.9427 and 0.9349, and F1-scores of 0.6720 and 0.7294, respectively. This study demonstrates the potential of multimodal AI to serve as a non-invasive diagnostic tool, reducing unnecessary biopsies and improving clinical decision-making in breast tumor management.

Method: Fine-tuned ten Vision Transformer and MLP-Mixer architectures using datasets from HOLISMOKES VI and SuGOHI X, with comprehensive analysis of transfer learning impact including data quality, model architecture, training strategies, and ensemble predictions.

Result: Achieved state-of-the-art classification performance for gravitational lens detection, benchmarking against convolutional baselines with complexity and inference-time analysis.

Conclusion: GraViT demonstrates the effectiveness of pretrained Vision Transformers and MLP-Mixer models for automated gravitational lens detection, providing a scalable solution for the upcoming LSST survey data processing.

Abstract: Gravitational lensing offers a powerful probe into the properties of dark matter and is crucial to infer cosmological parameters. The Legacy Survey of Space and Time (LSST) is predicted to find O(10^5) gravitational lenses over the next decade, demanding automated classifiers. In this work, we introduce GraViT, a PyTorch pipeline for gravitational lens detection that leverages extensive pretraining of state-of-the-art Vision Transformer (ViT) models and MLP-Mixer. We assess the impact of transfer learning on classification performance by examining data quality (source and sample size), model architecture (selection and fine-tuning), training strategies (augmentation, normalization, and optimization), and ensemble predictions. This study reproduces the experiments in a previous systematic comparison of neural networks and provides insights into the detectability of strong gravitational lenses on that common test sample. We fine-tune ten architectures using datasets from HOLISMOKES VI and SuGOHI X, and benchmark them against convolutional baselines, discussing complexity and inference-time analysis.

Method: The FHT2SP algorithm extends Brady’s superpixel concept to arbitrary shapes beyond power-of-two constraints and integrates it into the FHT2DT algorithm. It uses appropriately sized superpixels to balance computational efficiency and approximation accuracy.

Result: For w×h images, FHT2SP achieves near-optimal computational complexity O(wh ln³ w) while maintaining constant-bounded approximation error independent of image size. The error is controllable via a meta-parameter.

Conclusion: FHT2SP successfully bridges the gap between fast but inaccurate and accurate but slow Hough Transform algorithms, providing both computational efficiency and high accuracy with theoretical guarantees.

Abstract: The Hough transform (HT) is a fundamental tool across various domains, from classical image analysis to neural networks and tomography. Two key aspects of the algorithms for computing the HT are their computational complexity and accuracy - the latter often defined as the error of approximation of continuous lines by discrete ones within the image region. The fast HT (FHT) algorithms with optimal linearithmic complexity - such as the Brady-Yong algorithm for power-of-two-sized images - are well established. Generalizations like $FHT2DT$ extend this efficiency to arbitrary image sizes, but with reduced accuracy that worsens with scale. Conversely, accurate HT algorithms achieve constant-bounded error but require near-cubic computational cost. This paper introduces $FHT2SP$ algorithm - a fast and highly accurate HT algorithm. It builds on our development of Brady’s superpixel concept, extending it to arbitrary shapes beyond the original power-of-two square constraint, and integrates it into the $FHT2DT$ algorithm. With an appropriate choice of the superpixel’s size, for an image of shape $w \times h$, the $FHT2SP$ algorithm achieves near-optimal computational complexity $\mathcal{O}(wh \ln^3 w)$, while keeping the approximation error bounded by a constant independent of image size, and controllable via a meta-parameter. We provide theoretical and experimental analyses of the algorithm’s complexity and accuracy.

Method: Analysis of supervised deep learning methods using a newly developed dataset with over 10k images and 120k instances. Evaluation of performance, accuracy, and computational efficiency. Introduction of a lightweight fully connected convolutional network model for image quality improvement.

Result: Identification of the most reliable detection systems and specific challenges they address in industrial applications. Development of a dataset repository and proposed model available on GitHub.

Conclusion: The study provides valuable insights into effective object detection and image enhancement methods for industrial environments, with suggested future directions for model improvement and effectiveness enhancement.

Abstract: This paper tackles two key challenges: detecting small, dense, and overlapping objects (a major hurdle in computer vision) and improving the quality of noisy images, especially those encountered in industrial environments. [1, 2]. Our focus is on evaluating methods built on supervised deep learning. We perform an analysis of these methods, using a newly de- veloped dataset comprising over 10k images and 120k in- stances. By evaluating their performance, accuracy, and com- putational efficiency, we identify the most reliable detection systems and highlight the specific challenges they address in industrial applications. This paper also examines the use of deep learning models to improve image quality in noisy industrial environments. We introduce a lightweight model based on a fully connected convolutional network. Addition- ally, we suggest potential future directions for further enhanc- ing the effectiveness of the model. The repository of the dataset and proposed model can be found at: https://github.com/o-messai/SDOOD, https://github.com/o-messai/DDSRNet

Method: Three-stage system: 1) Data acquisition/preprocessing, 2) Contour generation using conditional GAN, 3) Multimodal contour refinement through vision-language modeling with standardized prompts in human-in-the-loop process.

Result: Improved contour fidelity on FabTrack datasets, enhancing edge continuity and geometric alignment while reducing manual tracing. GPT-image-1 outperformed Gemini 2.0 Flash in structural accuracy and perceptual quality.

Conclusion: VLM-guided generative workflows show promise for advancing industrial computer vision beyond classical pipeline limitations, achieving CAD-level precision in manufacturing applications.

Abstract: Industrial computer vision systems often struggle with noise, material variability, and uncontrolled imaging conditions, limiting the effectiveness of classical edge detectors and handcrafted pipelines. In this work, we present a language-guided generative vision system for remnant contour detection in manufacturing, designed to achieve CAD-level precision. The system is organized into three stages: data acquisition and preprocessing, contour generation using a conditional GAN, and multimodal contour refinement through vision-language modeling, where standardized prompts are crafted in a human-in-the-loop process and applied through image-text guided synthesis. On proprietary FabTrack datasets, the proposed system improved contour fidelity, enhancing edge continuity and geometric alignment while reducing manual tracing. For the refinement stage, we benchmarked several vision-language models, including Google’s Gemini 2.0 Flash, OpenAI’s GPT-image-1 integrated within a VLM-guided workflow, and open-source baselines. Under standardized conditions, GPT-image-1 consistently outperformed Gemini 2.0 Flash in both structural accuracy and perceptual quality. These findings demonstrate the promise of VLM-guided generative workflows for advancing industrial computer vision beyond the limitations of classical pipelines.

Method: Proposes Language-aware Information MaximizatiOn (LIMO) loss with three components: mutual information between vision inputs and text descriptions, KL divergence regularization from zero-shot predictions, and standard cross-entropy loss. Also explores parameter-efficient fine-tuning strategies.

Result: LIMO significantly outperforms recent transductive few-shot CLIP methods by a large margin and achieves substantial gains over best-performing inductive methods. Parameter-efficient fine-tuning provides substantial performance boosts.

Conclusion: The method demonstrates the effectiveness of information-theoretic approaches and parameter-efficient fine-tuning for transductive few-shot learning in VLMs, establishing new state-of-the-art performance benchmarks.

Abstract: Transductive few-shot learning has triggered an abundant literature focusing on vision-only models, but is still at a nascent stage within the recent context of foundational vision-language models (VLMs). Only a few recent methods addressed the problem, pointing to the potential of tranduction in VLMs and to the need for VLM-tailored methods. Building on this momentum, we leverage information-theoretic concepts and recent progress in parameter-efficient fine-tuning (PEFT), developing a highly competitive transductive few-shot CLIP method. Specifically, we introduce a novel Language-aware Information MaximizatiOn (LIMO) loss integrating three complementary terms: (i) the mutual information between the vision inputs and the textual class descriptions; (ii) a Kullback-Leibler (KL) divergence penalizing deviation of the network’s probabilistic outputs from the text-driven zero-shot predictions; and (iii) a standard cross-entropy loss based on the labeled shots. Furthermore, we challenge the commonly followed fine-tuning practices in the context of transductive few-shot learning, and explore PEFT strategies, completely overlooked in this context. Surprisingly, we observe substantial boosts in performances, which points to the potential of adapting a subset of the model’s parameters in the transductive few-shot setting. We report comprehensive evaluations, which show that LIMO outperforms the very recent transductive few-shot CLIP methods by a large margin and yields significant gains over the best-performing inductive methods. Our code is publicly available at:[ \href{https://github.com/ghassenbaklouti/LIMO}{\text{here}} ]

Method: MorphGen integrates histopathology images, augmentations, and nuclear segmentation masks in a supervised contrastive learning framework, aligning latent representations of images and nuclear masks. It also incorporates stochastic weight averaging (SWA) for enhanced robustness.

Result: The method demonstrates resilience to image corruptions and adversarial attacks, with attention maps showing primary reliance on nuclear morphology, cellular composition, and spatial cell organization for classification.

Conclusion: Explicitly modeling biologically robust nuclear morphology and spatial organization enables learning cancer representations that are resilient to domain shifts, addressing critical vulnerabilities in current deep learning systems for digital pathology.

Abstract: Domain generalization in computational histopathology is hindered by heterogeneity in whole slide images (WSIs), caused by variations in tissue preparation, staining, and imaging conditions across institutions. Unlike machine learning systems, pathologists rely on domain-invariant morphological cues such as nuclear atypia (enlargement, irregular contours, hyperchromasia, chromatin texture, spatial disorganization), structural atypia (abnormal architecture and gland formation), and overall morphological atypia that remain diagnostic across diverse settings. Motivated by this, we hypothesize that explicitly modeling biologically robust nuclear morphology and spatial organization will enable the learning of cancer representations that are resilient to domain shifts. We propose MorphGen (Morphology-Guided Generalization), a method that integrates histopathology images, augmentations, and nuclear segmentation masks within a supervised contrastive learning framework. By aligning latent representations of images and nuclear masks, MorphGen prioritizes diagnostic features such as nuclear and morphological atypia and spatial organization over staining artifacts and domain-specific features. To further enhance out-of-distribution robustness, we incorporate stochastic weight averaging (SWA), steering optimization toward flatter minima. Attention map analyses revealed that MorphGen primarily relies on nuclear morphology, cellular composition, and spatial cell organization within tumors or normal regions for final classification. Finally, we demonstrate resilience of the learned representations to image corruptions (such as staining artifacts) and adversarial attacks, showcasing not only OOD generalization but also addressing critical vulnerabilities in current deep learning systems for digital pathology. Code, datasets, and trained models are available at: https://github.com/hikmatkhan/MorphGen

Method: Extracts pedestrian features using language-image pre-trained model, embeds attribute set into query tokens guided by text prompts, and uses Transformer decoder with masked multi-head attention to generate attributes while preventing next-attribute prediction during training.

Result: Achieved 84.92% accuracy, 90.44% precision, 90.73% recall, and 90.46% F1-score on PETA dataset, with extensive experiments validating effectiveness across multiple PAR datasets.

Conclusion: SequencePAR provides an effective generative approach for pedestrian attribute recognition that outperforms traditional methods and handles data imbalance issues better.

Abstract: Current pedestrian attribute recognition (PAR) algorithms use multi-label or multi-task learning frameworks with specific classification heads. These models often struggle with imbalanced data and noisy samples. Inspired by the success of generative models, we propose Sequence Pedestrian Attribute Recognition (SequencePAR), a novel sequence generation paradigm for PAR. SequencePAR extracts pedestrian features using a language-image pre-trained model and embeds the attribute set into query tokens guided by text prompts. A Transformer decoder generates human attributes by integrating visual features and attribute query tokens. The masked multi-head attention layer in the decoder prevents the model from predicting the next attribute during training. The extensive experiments on multiple PAR datasets validate the effectiveness of SequencePAR. Specifically, we achieve 84.92%, 90.44%, 90.73%, and 90.46% in accuracy, precision, recall, and F1-score on the PETA dataset. The source code and pre-trained models are available at https://github.com/Event-AHU/OpenPAR.

Method: Proposes a two-pronged visual token pruning strategy: 1) removes visual tokens semantically misaligned with textual tokens using mutual information, 2) prunes redundant visual tokens by maximizing expected pairwise distances in embedding space using a greedy algorithm.

Result: Achieves 88.9% token reduction on models like LLaVA-1.5-7B and LLaVA-NEXT-7B while maintaining strong performance, resulting in 56.7% improvement in inference speed.

Conclusion: The proposed visual token pruning method effectively reduces computational overhead while preserving model performance by focusing on cross-modal alignment and intra-modal diversity preservation.

Abstract: Large Multimodal Models (LMMs) have achieved significant success across various tasks. These models usually encode visual inputs into dense token sequences, which are then concatenated with textual tokens and jointly processed by a language model. However, the increased token count substantially raises computational and memory costs during inference. Token pruning has emerged as a promising approach to address this issue. Existing token pruning methods often rely on costly calibration or suboptimal importance metrics, leading to redundant retained tokens. In this paper, we analyze the redundancy differences between visual and textual tokens and propose pruning exclusively on visual tokens. Based on this, we propose a visual token pruning strategy that explicitly preserves both cross-modal alignment and intra-modal informational diversity. We introduce a mutual information-based token pruning strategy that removes visual tokens semantically misaligned with textual tokens, effectively preserving the alignment between the visual and textual modalities. To further improve the representational quality of the retained tokens, we additionally prune redundant visual tokens by maximizing the expected pairwise distances in the embedding space, which is solved efficiently with a greedy algorithm. Extensive experiments demonstrate that our method maintains strong performance while reducing tokens by 88.9% on models such as LLaVA-1.5-7B and LLaVA-NEXT-7B, resulting in a 56.7% improvement in inference speed.

Method: Investigates how MLLMs’ reasoning capabilities can be elicited for aesthetic judgment, identifies hallucination issues, and proposes ArtCoT - an evidence-based, objective reasoning process to suppress hallucinations and improve aesthetic reasoning.

Result: MLLMs prompted with ArtCoT produce multifaceted, in-depth aesthetic reasoning that aligns significantly better with human judgment compared to standard approaches.

Conclusion: The work demonstrates that evidence-based reasoning can improve MLLMs’ aesthetic judgment capabilities, with applications in AI art tutoring and reward models for image generation, paving the way for AI systems that truly understand and appreciate art.

Abstract: The rapid technical progress of generative art (GenArt) has democratized the creation of visually appealing imagery. However, achieving genuine artistic impact - the kind that resonates with viewers on a deeper, more meaningful level - remains formidable as it requires a sophisticated aesthetic sensibility. This sensibility involves a multifaceted cognitive process extending beyond mere visual appeal, which is often overlooked by current computational methods. This paper pioneers an approach to capture this complex process by investigating how the reasoning capabilities of Multimodal LLMs (MLLMs) can be effectively elicited to perform aesthetic judgment. Our analysis reveals a critical challenge: MLLMs exhibit a tendency towards hallucinations during aesthetic reasoning, characterized by subjective opinions and unsubstantiated artistic interpretations. We further demonstrate that these hallucinations can be suppressed by employing an evidence-based and objective reasoning process, as substantiated by our proposed baseline, ArtCoT. MLLMs prompted by this principle produce multifaceted, in-depth aesthetic reasoning that aligns significantly better with human judgment. These findings have direct applications in areas such as AI art tutoring and as reward models for image generation. Ultimately, we hope this work paves the way for AI systems that can truly understand, appreciate, and contribute to art that aligns with human aesthetic values. Project homepage: https://github.com/songrise/MLLM4Art.

Method: Uses a mixture of shallow patch convolutional networks with patch-based processing to support higher-dimensional tensors while reducing multiplicative depth and inference latency. Implements parallel FHE evaluation for near-resolution-independent latency.

Result: Significantly accelerates inference and increases verification accuracy compared to state-of-the-art FHE neural networks adapted for face recognition on standard benchmarks.

Conclusion: CryptoFace enables secure face recognition systems with robust and provable security, facilitating privacy-preserving authentication applications.

Abstract: Face recognition is central to many authentication, security, and personalized applications. Yet, it suffers from significant privacy risks, particularly arising from unauthorized access to sensitive biometric data. This paper introduces CryptoFace, the first end-to-end encrypted face recognition system with fully homomorphic encryption (FHE). It enables secure processing of facial data across all stages of a face-recognition process–feature extraction, storage, and matching–without exposing raw images or features. We introduce a mixture of shallow patch convolutional networks to support higher-dimensional tensors via patch-based processing while reducing the multiplicative depth and, thus, inference latency. Parallel FHE evaluation of these networks ensures near-resolution-independent latency. On standard face recognition benchmarks, CryptoFace significantly accelerates inference and increases verification accuracy compared to the state-of-the-art FHE neural networks adapted for face recognition. CryptoFace will facilitate secure face recognition systems requiring robust and provable security. The code is available at https://github.com/human-analysis/CryptoFace.

Method: Built upon a lightweight VideoLLM, GenS leverages vision-language capabilities to identify question-relevant frames. The method uses a large-scale video instruction dataset (GenS-Video-150K) with dense frame relevance annotations for effective retrieval.

Result: GenS consistently boosts performance of various VideoLLMs. Open-source models achieve SOTA results: LLaVA-Video-72B reaches 66.8 (+4.3) on LongVideoBench and 77.0 (+2.7) on MLVU. Aria obtains 39.2 on HourVideo, surpassing Gemini-1.5-pro by 1.9 points.

Conclusion: GenS effectively addresses computational challenges in long-form video perception and significantly enhances VideoLLM performance across both open-source and proprietary models.

Abstract: Despite recent advances in Video Large Language Models (VideoLLMs), effectively understanding long-form videos remains a significant challenge. Perceiving lengthy videos containing thousands of frames poses substantial computational burden. To mitigate this issue, this paper introduces Generative Frame Sampler (GenS), a plug-and-play module integrated with VideoLLMs to facilitate efficient lengthy video perception. Built upon a lightweight VideoLLM, GenS leverages its inherent vision-language capabilities to identify question-relevant frames. To facilitate effective retrieval, we construct GenS-Video-150K, a large-scale video instruction dataset with dense frame relevance annotations. Extensive experiments demonstrate that GenS consistently boosts the performance of various VideoLLMs, including open-source models (Qwen2-VL-7B, Aria-25B, VILA-40B, LLaVA-Video-7B/72B) and proprietary assistants (GPT-4o, Gemini). When equipped with GenS, open-source VideoLLMs achieve impressive state-of-the-art results on long-form video benchmarks: LLaVA-Video-72B reaches 66.8 (+4.3) on LongVideoBench and 77.0 (+2.7) on MLVU, while Aria obtains 39.2 on HourVideo surpassing the Gemini-1.5-pro by 1.9 points. We will release all datasets and models at https://generative-sampler.github.io.

Method: Proposes a look-up table structure with low-order approximation encoding and high-level joint contextual scene encoding. Uses efficient LUT distillation strategy instead of traditional quantization methods, transferring knowledge from a multi-modal fusion network (MM-Net) to the MM-LUT model.

Result: Achieves less than one-tenth of the time compared to current lightweight SOTA fusion algorithms. Maintains high operational speed across various scenarios, even on low-power mobile devices. Extensive experiments validate superiority, reliability, and stability.

Conclusion: LUT-Fuse provides an extremely fast and efficient solution for infrared and visible image fusion that is suitable for real-time applications on resource-constrained devices while maintaining competitive performance.

Abstract: Current advanced research on infrared and visible image fusion primarily focuses on improving fusion performance, often neglecting the applicability on real-time fusion devices. In this paper, we propose a novel approach that towards extremely fast fusion via distillation to learnable lookup tables specifically designed for image fusion, termed as LUT-Fuse. Firstly, we develop a look-up table structure that utilizing low-order approximation encoding and high-level joint contextual scene encoding, which is well-suited for multi-modal fusion. Moreover, given the lack of ground truth in multi-modal image fusion, we naturally proposed the efficient LUT distillation strategy instead of traditional quantization LUT methods. By integrating the performance of the multi-modal fusion network (MM-Net) into the MM-LUT model, our method achieves significant breakthroughs in efficiency and performance. It typically requires less than one-tenth of the time compared to the current lightweight SOTA fusion algorithms, ensuring high operational speed across various scenarios, even in low-power mobile devices. Extensive experiments validate the superiority, reliability, and stability of our fusion approach. The code is available at https://github.com/zyb5/LUT-Fuse.

Method: Integrates perceptual features from CIELAB color space, structural embeddings from ResNet-50, and semantic context from frame-level captions generated by LLaMA-3.2-11B-Vision-Instruct. Uses PCA for modality fusion and HDBSCAN for adaptive video segmentation, with quality assessment and duplicate filtering refinement.

Result: Achieves state-of-the-art performance on TVSum20 and SumMe benchmarks, significantly outperforming both unimodal and prior multimodal approaches.

Conclusion: TriPSS effectively captures complementary visual and semantic cues, establishing it as an effective solution for video summarization, retrieval, and large-scale multimedia understanding.

Abstract: Efficient keyframe extraction is critical for video summarization and retrieval, yet capturing the full semantic and visual richness of video content remains challenging. We introduce TriPSS, a tri-modal framework that integrates perceptual features from the CIELAB color space, structural embeddings from ResNet-50, and semantic context from frame-level captions generated by LLaMA-3.2-11B-Vision-Instruct. These modalities are fused using principal component analysis to form compact multi-modal embeddings, enabling adaptive video segmentation via HDBSCAN clustering. A refinement stage incorporating quality assessment and duplicate filtering ensures the final keyframe set is both concise and semantically diverse. Evaluations on the TVSum20 and SumMe benchmarks show that TriPSS achieves state-of-the-art performance, significantly outperforming both unimodal and prior multimodal approaches. These results highlight TriPSS’ ability to capture complementary visual and semantic cues, establishing it as an effective solution for video summarization, retrieval, and large-scale multimedia understanding.

Method: STAR module uses target-anchored subspace from text embeddings to recenter image features, spectral projection to retain target-aligned directions, vision-language distillation, and feature-space Mixup for smooth transitions.

Result: Experiments across image classification and object detection benchmarks demonstrate STAR’s superiority over existing methods.

Conclusion: Minimal textual metadata significantly enhances generalization under data constraints, enabling robust model deployment in unseen target environments.

Abstract: Deep models trained on a single source domain often fail catastrophically under distribution shifts, a critical challenge in Single Domain Generalization (SDG). While existing methods focus on augmenting source data or learning invariant features, they neglect a readily available resource: textual descriptions of the target deployment environment. We propose Target-Oriented Single Domain Generalization (TO-SDG), a novel problem setup that leverages the textual description of the target domain, without requiring any target data, to guide model generalization. To address TO-SDG, we introduce Spectral TARget Alignment (STAR), a lightweight module that injects target semantics into source features by exploiting visual-language models (VLMs) such as CLIP. STAR uses a target-anchored subspace derived from the text embedding of the target description to recenter image features toward the deployment domain, then utilizes spectral projection to retain directions aligned with target cues while discarding source-specific noise. Moreover, we use a vision-language distillation to align backbone features with VLM’s semantic geometry. STAR further employs feature-space Mixup to ensure smooth transitions between source and target-oriented representations. Experiments across various image classification and object detection benchmarks demonstrate STAR’s superiority. This work establishes that minimal textual metadata, which is a practical and often overlooked resource, significantly enhances generalization under severe data constraints, opening new avenues for deploying robust models in target environments with unseen data.

Method: Two-stage marine object-oriented video captioning pipeline using video-text-segmentation triplets with video splitting for detecting salient object transitions in scene changes.

Result: Improved marine video understanding, analysis, and generation capabilities with enriched captioning semantics through scene change detection.

Conclusion: The proposed benchmark and pipeline effectively address marine video challenges, providing better tools for marine life analysis and video generation.

Abstract: Marine videos present significant challenges for video understanding due to the dynamics of marine objects and the surrounding environment, camera motion, and the complexity of underwater scenes. Existing video captioning datasets, typically focused on generic or human-centric domains, often fail to generalize to the complexities of the marine environment and gain insights about marine life. To address these limitations, we propose a two-stage marine object-oriented video captioning pipeline. We introduce a comprehensive video understanding benchmark that leverages the triplets of video, text, and segmentation masks to facilitate visual grounding and captioning, leading to improved marine video understanding and analysis, and marine video generation. Additionally, we highlight the effectiveness of video splitting in order to detect salient object transitions in scene changes, which significantly enrich the semantics of captioning content. Our dataset and code have been released at https://msc.hkustvgd.com.

Method: Multimodal framework integrating ground-level atmospheric imagery with pollutant concentration data using lightweight CNN backbones (MobileNetV2, ResNet18, EfficientNet-B0) through semantically aligned embedding spaces.

Result: Outperforms unimodal baselines with 92.02% classification accuracy and RMSE of 7.70 using EfficientNet-B0 backbone, delivering 18.5% improvement over single-modality approaches while maintaining low computational overhead.

Conclusion: AQFusionNet provides a scalable and practical solution for AQI monitoring in infrastructure-limited environments, offering robust predictive capability even under partial sensor availability and suitable for edge device deployment.

Abstract: Air pollution monitoring in resource-constrained regions remains challenging due to sparse sensor deployment and limited infrastructure. This work introduces AQFusionNet, a multimodal deep learning framework for robust Air Quality Index (AQI) prediction. The framework integrates ground-level atmospheric imagery with pollutant concentration data using lightweight CNN backbones (MobileNetV2, ResNet18, EfficientNet-B0). Visual and sensor features are combined through semantically aligned embedding spaces, enabling accurate and efficient prediction. Experiments on more than 8,000 samples from India and Nepal demonstrate that AQFusionNet consistently outperforms unimodal baselines, achieving up to 92.02% classification accuracy and an RMSE of 7.70 with the EfficientNet-B0 backbone. The model delivers an 18.5% improvement over single-modality approaches while maintaining low computational overhead, making it suitable for deployment on edge devices. AQFusionNet provides a scalable and practical solution for AQI monitoring in infrastructure-limited environments, offering robust predictive capability even under partial sensor availability.

Method: ILRNet embeds a rank minimization module (RMM) within U-Net architecture, transforming features to wavelet domain and applying singular value thresholding to low-frequency components. It adaptively learns thresholding parameters and uses iterative refinement to combine intermediate results with noisy inputs.

Result: Experimental results show ILRNet achieves state-of-the-art performance in both synthetic and real-world noise removal tasks.

Conclusion: ILRNet successfully integrates physical priors with deep learning, providing effective hyperspectral image denoising with superior detail preservation through its iterative low-rank network approach.

Abstract: Hyperspectral image (HSI) denoising is a crucial preprocessing step for subsequent tasks. The clean HSI usually reside in a low-dimensional subspace, which can be captured by low-rank and sparse representation, known as the physical prior of HSI. It is generally challenging to adequately use such physical properties for effective denoising while preserving image details. This paper introduces a novel iterative low-rank network (ILRNet) to address these challenges. ILRNet integrates the strengths of model-driven and data-driven approaches by embedding a rank minimization module (RMM) within a U-Net architecture. This module transforms feature maps into the wavelet domain and applies singular value thresholding (SVT) to the low-frequency components during the forward pass, leveraging the spectral low-rankness of HSIs in the feature domain. The parameter, closely related to the hyperparameter of the singular vector thresholding algorithm, is adaptively learned from the data, allowing for flexible and effective capture of low-rankness across different scenarios. Additionally, ILRNet features an iterative refinement process that adaptively combines intermediate denoised HSIs with noisy inputs. This manner ensures progressive enhancement and superior preservation of image details. Experimental results demonstrate that ILRNet achieves state-of-the-art performance in both synthetic and real-world noise removal tasks.

Method: Proposed SurgLLM framework with Surgical Context-aware Multimodal Pretraining (instrument-centric Masked Video Reconstruction), Temporal-aware Multimodal Tuning for temporal reasoning, and Surgical Task Dynamic Ensemble for efficient task triaging.

Result: Extensive experiments on diverse surgical video understanding tasks (captioning, general VQA, temporal VQA) demonstrate significant improvements over state-of-the-art approaches.

Conclusion: SurgLLM effectively addresses spatial and temporal challenges in surgical video understanding, providing a versatile solution for Computer-Assisted Surgery systems with enhanced performance across multiple tasks.

Abstract: Surgical video understanding is crucial for facilitating Computer-Assisted Surgery (CAS) systems. Despite significant progress in existing studies, two major limitations persist, including inadequate visual content perception and insufficient temporal awareness in surgical videos, and hinder the development of versatile CAS solutions. In this work, we propose the SurgLLM framework, an effective large multimodal model tailored for versatile surgical video understanding tasks with enhanced spatial focus and temporal awareness. Specifically, to empower the spatial focus of surgical videos, we first devise Surgical Context-aware Multimodal Pretraining (Surg-Pretrain) for the video encoder of SurgLLM, by performing instrument-centric Masked Video Reconstruction (MV-Recon) and subsequent multimodal alignment. To incorporate surgical temporal knowledge into SurgLLM, we further propose Temporal-aware Multimodal Tuning (TM-Tuning) to enhance temporal reasoning with interleaved multimodal embeddings. Moreover, to accommodate various understanding tasks of surgical videos without conflicts, we devise a Surgical Task Dynamic Ensemble to efficiently triage a query with optimal learnable parameters in our SurgLLM. Extensive experiments performed on diverse surgical video understanding tasks, including captioning, general VQA, and temporal VQA, demonstrate significant improvements over the state-of-the-art approaches, validating the effectiveness of our SurgLLM in versatile surgical video understanding. The source code is available at https://github.com/franciszchen/SurgLLM.

Method: Collection of 1123 FDG-PET/CT studies from 10 international medical centers with varying acquisition protocols. Expert radiation oncologists and radiologists manually segmented tumor volumes following standardized guidelines. The dataset includes NifTi files, segmentation masks, radiotherapy dose data, and comprehensive clinical metadata.

Result: Created a large-scale annotated dataset with expert segmentations and rich clinical information. Demonstrated utility through benchmark results using state-of-the-art deep learning models (UNet, SegResNet, multimodal frameworks) for three clinical tasks: automated tumor segmentation, recurrence-free survival prediction, and HPV status classification.

Conclusion: This publicly available dataset provides a valuable resource for the research community to develop and validate AI models for head and neck cancer analysis, addressing key clinical challenges with standardized annotations and real-world diversity across multiple institutions.

Abstract: We describe a publicly available multimodal dataset of annotated Positron Emission Tomography/Computed Tomography (PET/CT) studies for head and neck cancer research. The dataset includes 1123 FDG-PET/CT studies from patients with histologically confirmed head and neck cancer, acquired from 10 international medical centers. All examinations consisted of co-registered PET/CT scans with varying acquisition protocols, reflecting real-world clinical diversity across institutions. Primary gross tumor volumes (GTVp) and involved lymph nodes (GTVn) were manually segmented by experienced radiation oncologists and radiologists following standardized guidelines and quality control measures. We provide anonymized NifTi files of all studies, along with expert-annotated segmentation masks, radiotherapy dose distribution for a subset of patients, and comprehensive clinical metadata. This metadata includes TNM staging, HPV status, demographics (age and gender), long-term follow-up outcomes, survival times, censoring indicators, and treatment information. We demonstrate how this dataset can be used for three key clinical tasks: automated tumor segmentation, recurrence-free survival prediction, and HPV status classification, providing benchmark results using state-of-the-art deep learning models, including UNet, SegResNet, and multimodal prognostic frameworks.

Method: The authors propose Visual Potential Field Calibration (VPFC), a plug-and-play hallucination mitigation method based on their conceptual framework called Visual-Semantic Attention Potential Field, which reveals how models construct visual evidence for object presence/absence.

Result: VPFC effectively reduces omission hallucinations without introducing additional fabrication hallucinations, demonstrating that the two types of hallucinations have distinct causes that require different mitigation strategies.

Conclusion: The research reveals a critical oversight in current object hallucination research and provides new directions for developing more robust and balanced hallucination mitigation strategies by addressing omission and fabrication hallucinations as distinct phenomena.

Abstract: Multimodal Large Language Models (MLLMs) have achieved impressive advances, yet object hallucination remains a persistent challenge. Existing methods, based on the flawed assumption that omission and fabrication hallucinations share a common cause, often reduce omissions only to trigger more fabrications. In this work, we overturn this view by demonstrating that omission hallucinations arise from insufficient confidence when mapping perceived visual features to linguistic expressions, whereas fabrication hallucinations result from spurious associations within the cross-modal representation space due to statistical biases in the training corpus. Building on findings from visual attention intervention experiments, we propose the Visual-Semantic Attention Potential Field, a conceptual framework that reveals how the model constructs visual evidence to infer the presence or absence of objects. Leveraging this insight, we introduce Visual Potential Field Calibration (VPFC), a plug-and-play hallucination mitigation method that effectively reduces omission hallucinations without introducing additional fabrication hallucinations. Our findings reveal a critical oversight in current object hallucination research and chart new directions for developing more robust and balanced hallucination mitigation strategies.

Method: Two-stage approach: Stage I computes adaptive layer-specific steering vectors from diverse data sources to suppress harmful behaviors. Stage II refines these vectors through sequence-level preference optimization with toxicity assessment and visual-consistency rewards.

Result: SPO-VLM enhances safety against attacks while maintaining strong performance on benign tasks without compromising visual understanding capabilities.

Conclusion: The proposed two-stage framework effectively balances efficiency and effectiveness for VLM defense, combining lightweight mitigation with deeper policy refinement for robust and safe text generation.

Abstract: Vision Language Models (VLMs) have demonstrated impressive capabilities in integrating visual and textual information for understanding and reasoning, but remain highly vulnerable to adversarial attacks. While activation steering has emerged as a promising defence, existing approaches often rely on task-specific contrastive prompts to extract harmful directions, which exhibit suboptimal performance and can degrade visual grounding performance. To address these limitations, we propose \textit{Sequence-Level Preference Optimization} for VLM (\textit{SPO-VLM}), a novel two-stage defense framework that combines activation-level intervention with policy-level optimization to enhance model robustness. In \textit{Stage I}, we compute adaptive layer-specific steering vectors from diverse data sources, enabling generalized suppression of harmful behaviors during inference. In \textit{Stage II}, we refine these steering vectors through a sequence-level preference optimization process. This stage integrates automated toxicity assessment, as well as visual-consistency rewards based on caption-image alignment, to achieve safe and semantically grounded text generation. The two-stage structure of SPO-VLM balances efficiency and effectiveness by combining a lightweight mitigation foundation in Stage I with deeper policy refinement in Stage II. Extensive experiments shown SPO-VLM enhances safety against attacks via activation steering and preference optimization, while maintaining strong performance on benign tasks without compromising visual understanding capabilities. We will release our code, model weights, and evaluation toolkit to support reproducibility and future research. \textcolor{red}{Warning: This paper may contain examples of offensive or harmful text and images.}

Method: The system takes user interactions (clicks, bounding boxes, scribbles) along with the image to segment, and uses previously segmented images as context. As more images are segmented, the context grows and reduces interaction requirements for new images.

Result: MultiverSeg reduced total clicks by 36% and scribble steps by 25% to achieve 90% Dice score on unseen tasks compared to state-of-the-art interactive segmentation methods.

Conclusion: The system efficiently amortizes user interactions across multiple images, enabling rapid segmentation of new medical datasets without requiring pre-existing labeled data from the target domain.

Abstract: Medical researchers and clinicians often need to perform novel segmentation tasks on a set of related images. Existing methods for segmenting a new dataset are either interactive, requiring substantial human effort for each image, or require an existing set of previously labeled images. We introduce a system, MultiverSeg, that enables practitioners to rapidly segment an entire new dataset without requiring access to any existing labeled data from that task or domain. Along with the image to segment, the model takes user interactions such as clicks, bounding boxes or scribbles as input, and predicts a segmentation. As the user segments more images, those images and segmentations become additional inputs to the model, providing context. As the context set of labeled images grows, the number of interactions required to segment each new image decreases. We demonstrate that MultiverSeg enables users to interactively segment new datasets efficiently, by amortizing the number of interactions per image to achieve an accurate segmentation. Compared to using a state-of-the-art interactive segmentation method, MultiverSeg reduced the total number of clicks by 36% and scribble steps by 25% to achieve 90% Dice on sets of images from unseen tasks. We release code and model weights at https://multiverseg.csail.mit.edu

Method: Propose Adaptive Point-Prompt Tuning (APPT) which converts point clouds into embeddings with local geometry aggregation, uses permutation-invariant features for relative positions, and employs a weight-sharing prompt generator to dynamically produce point-prompts for self-attention calibration.

Result: Enables direct point cloud processing without heterogeneous mappings, maintains spatial geometries, and reduces computational overhead while adapting any modality foundation model to 3D analysis.

Conclusion: APPT provides a generalizable framework for efficient 3D point cloud analysis by directly leveraging point features and dynamic prompt generation, overcoming limitations of traditional cross-modal adaptation methods.

Abstract: Parameter-efficient fine-tuning strategies for foundation models in 1D textual and 2D visual analysis have demonstrated remarkable efficacy. However, due to the scarcity of point cloud data, pre-training large 3D models remains a challenging task. While many efforts have been made to apply pre-trained visual models to 3D domains through “high-to-low” mapping, these approaches often lead to the loss of spatial geometries and lack a generalizable framework for adapting any modality to 3D. This paper, therefore, attempts to directly leverage point features to calibrate the heterogeneous foundation model of any modality for 3D point cloud analysis. Specifically, we propose the Adaptive Point-Prompt Tuning (APPT) method, which fine-tunes pre-trained models with a modest number of parameters, enabling direct point cloud processing without heterogeneous mappings. We convert raw point clouds into point embeddings by aggregating local geometry to capture spatial features followed by linear layers to ensure seamless utilization of frozen pre-trained models. Given the inherent disorder of point clouds, in contrast to the structured nature of images and language, we employ a permutation-invariant feature to capture the relative positions of point embeddings, thereby obtaining point tokens enriched with location information to optimize self-attention mechanisms. To calibrate self-attention across source domains of any modality to 3D and reduce computational overhead, we introduce a prompt generator that shares weights with the point embedding module, dynamically producing point-prompts without adding additional parameters. These prompts are then concatenated into a frozen foundation model, providing rich global structural information and compensating for the lack of structural context in the heterogeneous data.

Method: Collected 98 MRIs with manual corrections of automated segmentations. Developed MultiAxial network with three independent 2D U-Nets operating in sagittal, axial and coronal planes, then combined to produce 3D segmentation without requiring atlas coregistration.

Result: MultiAxial achieved Dice score of 0.88±0.04, outperforming Multipriors (0.86±0.04) and SPM12 (0.79±0.10). Showed robustness with abnormal anatomy and de-identified images. Enables more accurate current flow modeling in transcranial electric stimulation.

Conclusion: Released state-of-the-art tool for whole-head MRI segmentation in abnormal anatomy along with largest labeled clinical head MRI dataset. Serves as benchmark for future efforts in this domain.

Abstract: Purpose: The goal of this work was to develop a deep network for whole-head segmentation including clinical MRIs with abnormal anatomy, and compile the first public benchmark dataset for this purpose. We collected 98 MRIs with volumetric segmentation labels for a diverse set of human subjects including normal, as well as abnormal anatomy in clinical cases of stroke and disorders of consciousness. Approach: Training labels were generated by manually correcting initial automated segmentations for skin/scalp, skull, CSF, gray matter, white matter, air cavity and extracephalic air. We developed a MultiAxial network consisting of three 2D U-Net that operate independently in sagittal, axial and coronal planes and are then combined to produce a single 3D segmentation. Results: The MultiAxial network achieved a test-set Dice scores of 0.88+-0.04 (median +- interquartile range) on whole head segmentation including gray and white matter. This compared to 0.86 +- 0.04 for Multipriors and 0.79 +- 0.10 for SPM12, two standard tools currently available for this task. The MultiAxial network gains in robustness by avoiding the need for coregistration with an atlas. It performed well in regions with abnormal anatomy and on images that have been de-identified. It enables more accurate and robust current flow modeling when incorporated into ROAST, a widely-used modeling toolbox for transcranial electric stimulation.Conclusions: We are releasing a new state-of-the-art tool for whole-head MRI segmentation in abnormal anatomy, along with the largest volume of labeled clinical head MRIs including labels for non-brain structures. Together the model and data may serve as a benchmark for future efforts.

Method: The proposed NoiseCutMix method partially combines the estimated noise corresponding to two different classes in a diffusion model, enabling natural fusion of characteristics from both classes without unnatural boundaries.

Result: The method was validated through classification experiments, showing effectiveness compared to conventional data augmentation techniques, random image generation using Stable Diffusion, and combinations of these methods.

Conclusion: NoiseCutMix successfully achieves natural, high-resolution image generation with fused characteristics from two classes, addressing the limitations of traditional CutMix while maintaining data diversity benefits.

Abstract: In this study, we propose a novel data augmentation method that introduces the concept of CutMix into the generation process of diffusion models, thereby exploiting both the ability of diffusion models to generate natural and high-resolution images and the characteristic of CutMix, which combines features from two classes to create diverse augmented data. Representative data augmentation methods for combining images from multiple classes include CutMix and MixUp. However, techniques like CutMix often result in unnatural boundaries between the two images due to contextual differences. Therefore, in this study, we propose a method, called NoiseCutMix, to achieve natural, high-resolution image generation featuring the fused characteristics of two classes by partially combining the estimated noise corresponding to two different classes in a diffusion model. In the classification experiments, we verified the effectiveness of the proposed method by comparing it with conventional data augmentation techniques that combine multiple classes, random image generation using Stable Diffusion, and combinations of these methods. Our codes are available at: https://github.com/shumpei-takezaki/NoiseCutMix

Method: Developed pseudo-ground truth labeling pipeline for Thuman2.1 dataset: mesh alignment, multi-viewpoint segmentation, backprojection. Introduced windowed iterative FPS with space-filling curve serialization for efficient downsampling, followed by geometric segmentation using PointTransformer.

Result: Experimental results confirm the effectiveness and accuracy of the proposed approach for semantic parsing of human meshes.

Conclusion: The method successfully enables semantic segmentation of human meshes using only geometric information, providing an effective solution that bridges computer graphics and human parsing domains.

Abstract: Recent advances in point cloud deep learning have led to models that achieve high per-part labeling accuracy on large-scale point clouds, using only the raw geometry of unordered point sets. In parallel, the field of human parsing focuses on predicting body part and clothing/accessory labels from images. This work aims to bridge these two domains by enabling per-vertex semantic segmentation of large-scale human meshes. To achieve this, a pseudo-ground truth labeling pipeline is developed for the Thuman2.1 dataset: meshes are first aligned to a canonical pose, segmented from multiple viewpoints, and the resulting point-level labels are then backprojected onto the original mesh to produce per-point pseudo ground truth annotations. Subsequently, a novel, memory-efficient sampling strategy is introduced, a windowed iterative farthest point sampling (FPS) with space-filling curve-based serialization to effectively downsample the point clouds. This is followed by a purely geometric segmentation using PointTransformer, enabling semantic parsing of human meshes without relying on texture information. Experimental results confirm the effectiveness and accuracy of the proposed approach. Project code and pre-processed data is available at https://github.com/JamesMcCullochDickens/Human3DParsing/tree/master.

Method: Uses crossmodal knowledge distillation with known 2D-3D correspondence to align 3D network outputs with 2D network outputs. Employs self-calibrated convolution on 3D point clouds for domain adaptation, preserving modality-general information while filtering modality-specific details.

Result: The proposed methods consistently surpass state-of-the-art performance in 3D LiDAR semantic segmentation without requiring 3D annotations.

Conclusion: Crossmodal knowledge distillation effectively transfers knowledge from 2D image models to 3D LiDAR segmentation, eliminating the need for costly 3D annotations while achieving superior performance compared to existing methods.

Abstract: Semantic segmentation of 3D LiDAR data plays a pivotal role in autonomous driving. Traditional approaches rely on extensive annotated data for point cloud analysis, incurring high costs and time investments. In contrast, realworld image datasets offer abundant availability and substantial scale. To mitigate the burden of annotating 3D LiDAR point clouds, we propose two crossmodal knowledge distillation methods: Unsupervised Domain Adaptation Knowledge Distillation (UDAKD) and Feature and Semantic-based Knowledge Distillation (FSKD). Leveraging readily available spatio-temporally synchronized data from cameras and LiDARs in autonomous driving scenarios, we directly apply a pretrained 2D image model to unlabeled 2D data. Through crossmodal knowledge distillation with known 2D-3D correspondence, we actively align the output of the 3D network with the corresponding points of the 2D network, thereby obviating the necessity for 3D annotations. Our focus is on preserving modality-general information while filtering out modality-specific details during crossmodal distillation. To achieve this, we deploy self-calibrated convolution on 3D point clouds as the foundation of our domain adaptation module. Rigorous experimentation validates the effectiveness of our proposed methods, consistently surpassing the performance of state-of-the-art approaches in the field.

Method: Proposed NAME paradigm with actor location and shape module for plausible image generation, and developed robust evaluation metrics across three key dimensions to measure perceptual quality and narrative consistency.

Result: Achieves SOTA performance across different data splits: reduces FID from 195 to 152 using only 0.96% data, improves from 175 to 152 (70:30 split), and nearly halves from 96 to 49 (95:5 split). Scores 3.62 vs baseline 2.66 on new metric.

Conclusion: NAME successfully reduces cognitive burden in narrative analysis by generating coherent story images from research documents, demonstrating significant efficiency improvements with minimal data requirements while maintaining high quality.

Abstract: Narrative inquiry has been one of the prominent application domains for the analysis of human experience, aiming to know more about the complexity of human society. However, researchers are often required to transform various forms of data into coherent hand-drafted narratives in storied form throughout narrative analysis, which brings an immense burden of data analysis. Participants, too, are expected to engage in member checking and presentation of these narrative products, which involves reviewing and responding to large volumes of documents. Given the dual burden and the need for more efficient and participant-friendly approaches to narrative making and representation, we made a first attempt: (i) a new paradigm is proposed, NAME, as the initial attempt to push the field of narrative inquiry. Name is able to transfer research documents into coherent story images, alleviating the cognitive burden of interpreting extensive text-based materials during member checking for both researchers and participants. (ii) We develop an actor location and shape module to facilitate plausible image generation. (iii) We have designed a set of robust evaluation metrics comprising three key dimensions to objectively measure the perceptual quality and narrative consistency of generated characters. Our approach consistently demonstrates state-of-the-art performance across different data partitioning schemes. Remarkably, while the baseline relies on the full 100% of the available data, our method requires only 0.96% yet still reduces the FID score from 195 to 152. Under identical data volumes, our method delivers substantial improvements: for the 70:30 split, the FID score decreases from 175 to 152, and for the 95:5 split, it is nearly halved from 96 to 49. Furthermore, the proposed model achieves a score of 3.62 on the newly introduced metric, surpassing the baseline score of 2.66.

Method: Proposes HERO-VQL with two key components: 1) Top-down Attention Guidance (TAG) using class token for context and principal component scores for fine-grained localization, and 2) Egocentric Augmentation with Consistency Training (EgoACT) that enhances query diversity and simulates extreme viewpoint changes.

Result: Extensive experiments on VQ2D dataset show HERO-VQL significantly outperforms baseline methods, effectively handling egocentric challenges.

Conclusion: The method successfully addresses egocentric visual query localization challenges by mimicking human cognitive processes and employing robust training strategies with consistency constraints.

Abstract: In this work, we tackle the egocentric visual query localization (VQL), where a model should localize the query object in a long-form egocentric video. Frequent and abrupt viewpoint changes in egocentric videos cause significant object appearance variations and partial occlusions, making it difficult for existing methods to achieve accurate localization. To tackle these challenges, we introduce Hierarchical, Egocentric and RObust Visual Query Localization (HERO-VQL), a novel method inspired by human cognitive process in object recognition. We propose i) Top-down Attention Guidance (TAG) and ii) Egocentric Augmentation based Consistency Training (EgoACT). Top-down Attention Guidance refines the attention mechanism by leveraging the class token for high-level context and principal component score maps for fine-grained localization. To enhance learning in diverse and challenging matching scenarios, EgoAug enhances query diversity by replacing the query with a randomly selected corresponding object from groundtruth annotations and simulates extreme viewpoint changes by reordering video frames. Additionally, CT loss enforces stable object localization across different augmentation scenarios. Extensive experiments on VQ2D dataset validate that HERO-VQL effectively handles egocentric challenges, significantly outperforming baselines.

Method: Double-Constraint Diffusion Model (DCDM) with frozen pre-trained diffusion model weights and two trainable constraint modules: Nuclear Transformer Constraint (NTC) for low-rank feature compression and Encoding Nexus Constraint (ENC) for controlling the diffusion model using compressed features.

Result: Outperforms state-of-the-art methods on known dose reduction factors and generalizes well to unknown DRF scenarios, even at ultra-low doses (1% of full dose) on UDPET and Clinical datasets.

Conclusion: DCDM provides efficient, flexible ultra-low-dose PET reconstruction with minimal trainable parameters, adapting to various dose levels without full retraining while maintaining high image quality.

Abstract: Ultra-low-dose positron emission tomography (PET) reconstruction holds significant potential for reducing patient radiation exposure and shortening examination times. However, it may also lead to increased noise and reduced imaging detail, which could decrease the image quality. In this study, we present a Double-Constraint Diffusion Model (DCDM), which freezes the weights of a pre-trained diffusion model and injects a trainable double-constraint controller into the encoding architecture, greatly reducing the number of trainable parameters for ultra-low-dose PET reconstruction. Unlike full fine-tuning models, DCDM can adapt to different dose levels without retraining all model parameters, thereby improving reconstruction flexibility. Specifically, the two constraint modules, named the Nuclear Transformer Constraint (NTC) and the Encoding Nexus Constraint (ENC), serve to refine the pre-trained diffusion model. The NTC leverages the nuclear norm as an approximation for matrix rank minimization, integrates the low-rank property into the Transformer architecture, and enables efficient information extraction from low-dose images and conversion into compressed feature representations in the latent space. Subsequently, the ENC utilizes these compressed feature representations to encode and control the pre-trained diffusion model, ultimately obtaining reconstructed PET images in the pixel space. In clinical reconstruction, the compressed feature representations from NTC help select the most suitable ENC for efficient unknown low-dose PET reconstruction. Experiments conducted on the UDPET public dataset and the Clinical dataset demonstrated that DCDM outperforms state-of-the-art methods on known dose reduction factors (DRF) and generalizes well to unknown DRF scenarios, proving valuable even at ultra-low dose levels, such as 1% of the full dose.

Method: Introduces Distortion-Aware Omnidirectional Video Inpainting (DAOVI) with two modules: temporal motion evaluation using geodesic distance in image space, and depth-aware feature propagation in feature space to address geometric distortion.

Result: Experimental results show DAOVI outperforms existing methods both quantitatively and qualitatively.

Conclusion: The proposed method successfully addresses the unique challenges of omnidirectional video inpainting by considering distortion and geometric properties specific to 360-degree content.

Abstract: Omnidirectional videos that capture the entire surroundings are employed in a variety of fields such as VR applications and remote sensing. However, their wide field of view often causes unwanted objects to appear in the videos. This problem can be addressed by video inpainting, which enables the natural removal of such objects while preserving both spatial and temporal consistency. Nevertheless, most existing methods assume processing ordinary videos with a narrow field of view and do not tackle the distortion in equirectangular projection of omnidirectional videos. To address this issue, this paper proposes a novel deep learning model for omnidirectional video inpainting, called Distortion-Aware Omnidirectional Video Inpainting (DAOVI). DAOVI introduces a module that evaluates temporal motion information in the image space considering geodesic distance, as well as a depth-aware feature propagation module in the feature space that is designed to address the geometric distortion inherent to omnidirectional videos. The experimental results demonstrate that our proposed method outperforms existing methods both quantitatively and qualitatively.

Method: Leverages Human4DiT video generative model to generate human motions from alternative perspectives as additional supervision. Uses video fine-tuning to inject physical identity for consistent motion reproduction, and employs patch-based denoising for higher-resolution outputs with finer details.

Result: Outperforms recent state-of-the-art approaches in human avatar reconstruction from monocular videos, demonstrating effectiveness in enriching details in unseen regions and mitigating artifacts.

Conclusion: The proposed framework successfully addresses limitations of current monocular video-based avatar reconstruction by using generative models for additional supervision and complementary strategies for motion consistency and detail enhancement.

Abstract: We present a novel framework to reconstruct human avatars from monocular videos. Recent approaches have struggled either to capture the fine-grained dynamic details from the input or to generate plausible details at novel viewpoints, which mainly stem from the limited representational capacity of the avatar model and insufficient observational data. To overcome these challenges, we propose to leverage the advanced video generative model, Human4DiT, to generate the human motions from alternative perspective as an additional supervision signal. This approach not only enriches the details in previously unseen regions but also effectively regularizes the avatar representation to mitigate artifacts. Furthermore, we introduce two complementary strategies to enhance video generation: To ensure consistent reproduction of human motion, we inject the physical identity into the model through video fine-tuning. For higher-resolution outputs with finer details, a patch-based denoising algorithm is employed. Experimental results demonstrate that our method outperforms recent state-of-the-art approaches and validate the effectiveness of our proposed strategies.

Method: Two-stage approach: 1) Pyramid token merging during encoding to hierarchically reduce image tokens, 2) KV Cache compression during decoding to remove unnecessary caches and increase throughput.

Result: Maintains 100% performance with 35% tokens, 98% performance with 3% tokens. Achieves 2.02× throughput, 3.65× prefilling time reduction, and 3.21× inference time reduction for long sequences.

Conclusion: LightVLM enables efficient deployment of large VLMs by significantly accelerating inference while preserving performance, making heavy models faster than smaller counterparts.

Abstract: In this paper, we introduce LightVLM, a simple but effective method that can be seamlessly deployed upon existing Vision-Language Models (VLMs) to greatly accelerate the inference process in a training-free manner. We divide the inference procedure of VLMs into two stages, i.e., encoding and decoding, and propose to simultaneously accelerate VLMs in both stages to largely improve model efficiency. During encoding, we propose pyramid token merging to reduce tokens of different LLM layers in a hierarchical manner by finally only keeping a few dominant tokens to achieve high efficiency. During decoding, aimed at reducing the high latency of outputting long sequences, we propose KV Cache compression to remove unnecessary caches to increase the network throughput. Experimental results show that LightVLM successfully retains 100% performance when only preserving 35% image tokens, and maintains around 98% performance when keeping only 3% image tokens. LightVLM could 2.02$\times$ the network throughput and reduce the prefilling time by 3.65$\times$. LightVLM also makes large VLMs faster again by enabling a heavy model (e.g., InternVL2.5 26B) to infer faster than significantly smaller models (e.g., InternVL2.5 8B), hopefully facilitating the real-world deployment. When generating long text sequences (e.g., 4096 tokens), LightVLM could reduce the inference time by 3.21$\times$, largely outperforming existing methods.

Method: Uses mask-conditioned space factorization for semantic-decoupled latent modeling, mixture of global and local experts for structure/semantics capture, and dynamic gating network with time-dependent coefficients.

Result: Demonstrates effectiveness in multimodal and monomodal face generation settings with robust zero-shot generalization capability.

Conclusion: Provides a powerful and flexible solution for high-quality, controllable face generation with strong potential in generative modeling and security applications.

Abstract: Controllable face generation poses critical challenges in generative modeling due to the intricate balance required between semantic controllability and photorealism. While existing approaches struggle with disentangling semantic controls from generation pipelines, we revisit the architectural potential of Diffusion Transformers (DiTs) through the lens of expert specialization. This paper introduces Face-MoGLE, a novel framework featuring: (1) Semantic-decoupled latent modeling through mask-conditioned space factorization, enabling precise attribute manipulation; (2) A mixture of global and local experts that captures holistic structure and region-level semantics for fine-grained controllability; (3) A dynamic gating network producing time-dependent coefficients that evolve with diffusion steps and spatial locations. Face-MoGLE provides a powerful and flexible solution for high-quality, controllable face generation, with strong potential in generative modeling and security applications. Extensive experiments demonstrate its effectiveness in multimodal and monomodal face generation settings and its robust zero-shot generalization capability. Project page is available at https://github.com/XavierJiezou/Face-MoGLE.

Method: Integrates Semantic Reordering (clusters and arranges semantically similar patches through reversible permutation) with Semantic-guided Retrieval State Space Module (selects representative query subset to adjust state space parameters for better global modeling).

Result: Achieves state-of-the-art accuracy on four WSI subtype datasets with fewer FLOPs and parameters compared to strong baselines.

Conclusion: SemaMIL effectively addresses computational complexity and interpretability issues in WSI analysis while maintaining high performance.

Abstract: Multiple instance learning (MIL) has become the leading approach for extracting discriminative features from whole slide images (WSIs) in computational pathology. Attention-based MIL methods can identify key patches but tend to overlook contextual relationships. Transformer models are able to model interactions but require quadratic computational cost and are prone to overfitting. State space models (SSMs) offer linear complexity, yet shuffling patch order disrupts histological meaning and reduces interpretability. In this work, we introduce SemaMIL, which integrates Semantic Reordering (SR), an adaptive method that clusters and arranges semantically similar patches in sequence through a reversible permutation, with a Semantic-guided Retrieval State Space Module (SRSM) that chooses a representative subset of queries to adjust state space parameters for improved global modeling. Evaluation on four WSI subtype datasets shows that, compared to strong baselines, SemaMIL achieves state-of-the-art accuracy with fewer FLOPs and parameters.

Method: Stage-wise Adaptive Label Distribution Learning (SA-LDL) that uses stage-wise adaptive variance modeling and weighted loss function based on analysis of embedding similarities between age groups.

Result: Achieves competitive performance with MAE of 1.74 on MORPH-II and 2.15 on FG-NET datasets, demonstrating effectiveness in capturing structured label ambiguity.

Conclusion: SA-LDL effectively addresses the stage-wise patterns of label ambiguity in age estimation, leading to more accurate and robust performance compared to existing methods.

Abstract: Label ambiguity poses a significant challenge in age estimation tasks. Most existing methods address this issue by modeling correlations between adjacent age groups through label distribution learning. However, they often overlook the varying degrees of ambiguity present across different age stages. In this paper, we propose a Stage-wise Adaptive Label Distribution Learning (SA-LDL) algorithm, which leverages the observation – revealed through our analysis of embedding similarities between an anchor and all other ages – that label ambiguity exhibits clear stage-wise patterns. By jointly employing stage-wise adaptive variance modeling and weighted loss function, SA-LDL effectively captures the complex and structured nature of label ambiguity, leading to more accurate and robust age estimation. Extensive experiments demonstrate that SA-LDL achieves competitive performance, with MAE of 1.74 and 2.15 on the MORPH-II and FG-NET datasets.

Method: Encoder-Only Image Registration (EOIR) framework that separates feature learning from flow estimation, using a 3-layer ConvNet for feature extraction and 3-layer flow estimators to build a Laplacian feature pyramid for progressive diffeomorphic deformations.

Result: EOIR achieves superior accuracy-efficiency and accuracy-smoothness trade-offs across five datasets of different modalities and anatomical regions, providing better efficiency and smoothness with comparable accuracy.

Conclusion: The proposed EOIR framework effectively addresses registration challenges by leveraging insights about ConvNet roles in linearizing intensities and harmonizing contrast variations, offering a practical solution with public code availability.

Abstract: Learning-based techniques have significantly improved the accuracy and speed of deformable image registration. However, challenges such as reducing computational complexity and handling large deformations persist. To address these challenges, we analyze how convolutional neural networks (ConvNets) influence registration performance using the Horn-Schunck optical flow equation. Supported by prior studies and our empirical experiments, we observe that ConvNets play two key roles in registration: linearizing local intensities and harmonizing global contrast variations. Based on these insights, we propose the Encoder-Only Image Registration (EOIR) framework, designed to achieve a better accuracy-efficiency trade-off. EOIR separates feature learning from flow estimation, employing only a 3-layer ConvNet for feature extraction and a set of 3-layer flow estimators to construct a Laplacian feature pyramid, progressively composing diffeomorphic deformations under a large-deformation model. Results on five datasets across different modalities and anatomical regions demonstrate EOIR’s effectiveness, achieving superior accuracy-efficiency and accuracy-smoothness trade-offs. With comparable accuracy, EOIR provides better efficiency and smoothness, and vice versa. The source code of EOIR will be publicly available on https://github.com/XiangChen1994/EOIR.

Method: Using the Jump-Jump game as a testing environment where players control a red circle character to jump across platforms with appropriate force to maximize score.

Result: The paper establishes the Jump-Jump game as a suitable platform for evaluating cognitive aspects of LLM decision-making, though specific experimental results are not provided in this abstract.

Conclusion: The Jump-Jump game provides an effective framework for studying and testing LLM decision-making capabilities through its gameplay mechanics that require complex cognitive processing.

Abstract: The Jump-Jump game, as a simple yet challenging casual game, provides an ideal testing environment for studying LLM decision-making capabilities. The game requires players to precisely control jumping force based on current position and target platform distance, involving multiple cognitive aspects including spatial reasoning, physical modeling, and strategic planning. It illustrates the basic gameplay mechanics of the Jump-Jump game, where the player character (red circle) must jump across platforms with appropriate force to maximize score.

Method: Created VideoRewardBench through AI-assisted data pipeline with 1,563 annotated samples (video-text prompts with chosen/rejected responses), covering 4 video understanding aspects. Evaluated 28 MRMs across generative, discriminative, and semi-scalar categories.

Result: Top-performing GPT-4o achieved only 57.0% accuracy, best open-source model Qwen2.5-VL-72B reached 53.3%. Key findings: RL-trained MRMs don’t necessarily have better cross-modal generalization; most MRMs benefit from inference-time scaling; video frame count affects different MRM types differently.

Conclusion: VideoRewardBench provides a challenging benchmark that reveals significant performance gaps in current MRMs and offers valuable insights for advancing multimodal reward model development in video domain.

Abstract: Multimodal reward models (MRMs) play a crucial role in the training, inference, and evaluation of Large Vision Language Models (LVLMs) by assessing response quality. However, existing benchmarks for evaluating MRMs in the video domain suffer from a limited number and diversity of questions, a lack of comprehensive evaluation dimensions, and inadequate evaluation of diverse types of MRMs. To address these gaps, we introduce VideoRewardBench, the first comprehensive benchmark covering four core aspects of video understanding: perception, knowledge, reasoning, and safety. Through our AI-assisted data pipeline, we curate a high-quality preference dataset of 1,563 annotated samples, including 1,482 unique videos and 1,559 distinct questions–15 times the number found in the most question-rich prior benchmark. Each sample is a triplet consisting of a video-text prompt, a chosen response, and a rejected response. We also conduct a comprehensive evaluation across 28 multimodal reward models spanning three categories: generative, discriminative, and semi-scalar. Results show that even the top-performing model GPT-4o achieves only 57.0% overall accuracy, and the state-of-the-art open-source model Qwen2.5-VL-72B reaches merely 53.3%. Our analysis further reveals three key insights: (i) MRMs trained with reinforcement learning (RL) do not necessarily exhibit stronger cross-modal generalization than those trained without RL; (ii) except for discriminative MRMs, other types of MRMs across varying model capacities can benefit from inference-time scaling; and (iii) variations in input video frame count have different effects on different types of MRMs. We believe VideoRewardBench offers a challenging and valuable benchmark for advancing the evaluation and development of MRMs in the video domain.

Method: STVH: Unified framework modeling individual object dynamics and group interactions with spatiotemporal evolution. M-STVH: Enhanced version with hierarchical feature integration and multi-focused representation learning to handle both activity semantics and object visual features.

Result: Both STVH and M-STVH achieve excellent results on publicly available datasets.

Conclusion: The proposed techniques successfully solve the problem of fine-grained group activity retrieval in videos through spatiotemporal modeling and multi-focused representation learning.

Abstract: With the explosive growth of video data in various complex scenarios, quickly retrieving group activities has become an urgent problem. However, many tasks can only retrieve videos focusing on an entire video, not the activity granularity. To solve this problem, we propose a new STVH (spatiotemporal interleaved video hashing) technique for the first time. Through a unified framework, the STVH simultaneously models individual object dynamics and group interactions, capturing the spatiotemporal evolution on both group visual features and positional features. Moreover, in real-life video retrieval scenarios, it may sometimes require activity features, while at other times, it may require visual features of objects. We then further propose a novel M-STVH (multi-focused spatiotemporal video hashing) as an enhanced version to handle this difficult task. The advanced method incorporates hierarchical feature integration through multi-focused representation learning, allowing the model to jointly focus on activity semantics features and object visual features. We conducted comparative experiments on publicly available datasets, and both STVH and M-STVH can achieve excellent results.

Method: Proposes TRUST framework with Token-dRiven module that selects suitable target tokens from data and model perspectives, uses behavior mirror loss, and injects auxiliary prompts to match content representation with downstream behavior.

Result: Experimental results on ultrasound datasets show TRUST outperforms existing UI2I methods in both visual quality and downstream task performance.

Conclusion: TRUST effectively bridges the style gap in ultrasound images while preserving content integrity, making it suitable for improving downstream medical imaging tasks across different devices.

Abstract: Ultrasound images acquired from different devices exhibit diverse styles, resulting in decreased performance of downstream tasks. To mitigate the style gap, unpaired image-to-image (UI2I) translation methods aim to transfer images from a source domain, corresponding to new device acquisitions, to a target domain where a frozen task model has been trained for downstream applications. However, existing UI2I methods have not explicitly considered filtering the most relevant style features, which may result in translated images misaligned with the needs of downstream tasks. In this work, we propose TRUST, a token-driven dual-stream framework that preserves source content while transferring the common style of the target domain, ensuring that content and style remain unblended. Given multiple styles in the target domain, we introduce a Token-dRiven (TR) module that operates from two perspectives: (1) a data view–selecting “suitable” target tokens corresponding to each source token, and (2) a model view–identifying ``optimal” target tokens for the downstream model, guided by a behavior mirror loss. Additionally, we inject auxiliary prompts into the source encoder to match content representation with downstream behavior. Experimental results on ultrasound datasets demonstrate that TRUST outperforms existing UI2I methods in both visual quality and downstream task performance.

Method: ATtention-Guided sCaler (ATGC) leverages DINOv2 attention maps to dynamically select optimal scales for black-box model inference, scoring attention maps with entropy to identify informative scales for pseudo-labelling.

Result: Experiments show substantial improvements under black-box supervision across multiple datasets while requiring only one-hot API predictions.

Conclusion: ATGC effectively enables local model adaptation using black-box APIs by addressing the ‘curse of resolution’ through attention-guided scale selection, making black-box distillation practical with minimal API access requirements.

Abstract: The rise of Artificial Intelligence as a Service (AIaaS) democratizes access to pre-trained models via Application Programming Interfaces (APIs), but also raises a fundamental question: how can local models be effectively trained using black-box models that do not expose their weights, training data, or logits, a constraint in which current domain adaptation paradigms are impractical ? To address this challenge, we introduce the Black-Box Distillation (B2D) setting, which enables local model adaptation under realistic constraints: (1) the API model is open-vocabulary and trained on large-scale general-purpose data, and (2) access is limited to one-hot predictions only. We identify that open-vocabulary models exhibit significant sensitivity to input resolution, with different object classes being segmented optimally at different scales, a limitation termed the “curse of resolution”. Our method, ATtention-Guided sCaler (ATGC), addresses this challenge by leveraging DINOv2 attention maps to dynamically select optimal scales for black-box model inference. ATGC scores the attention maps with entropy to identify informative scales for pseudo-labelling, enabling effective distillation. Experiments demonstrate substantial improvements under black-box supervision across multiple datasets while requiring only one-hot API predictions. Our code is available at https://github.com/yasserben/ATGC.

Method: Language-inspired Bootstrapped Disentanglement framework using CLIP’s prior class semantics. Includes Language-guided Prototypical Disentanglement (using text features as topological templates) and Manifold Mutual Background Disentanglement (multiple learnable prototypes with mask-pooling supervision).

Result: Achieves state-of-the-art performance on Pascal VOC and ADE20k datasets, particularly in multi-step scenarios.

Conclusion: LBD effectively addresses semantic entanglement in CISS by leveraging language semantics for autonomous feature disentanglement, bridging the capability gap between dense and sparse tasks through soft prompt tuning and encoder adaptation.

Abstract: Class-Incremental Semantic Segmentation (CISS) requires continuous learning of newly introduced classes while retaining knowledge of past classes. By abstracting mainstream methods into two stages (visual feature extraction and prototype-feature matching), we identify a more fundamental challenge termed catastrophic semantic entanglement. This phenomenon involves Prototype-Feature Entanglement caused by semantic misalignment during the incremental process, and Background-Increment Entanglement due to dynamic data evolution. Existing techniques, which rely on visual feature learning without sufficient cues to distinguish targets, introduce significant noise and errors. To address these issues, we introduce a Language-inspired Bootstrapped Disentanglement framework (LBD). We leverage the prior class semantics of pre-trained visual-language models (e.g., CLIP) to guide the model in autonomously disentangling features through Language-guided Prototypical Disentanglement and Manifold Mutual Background Disentanglement. The former guides the disentangling of new prototypes by treating hand-crafted text features as topological templates, while the latter employs multiple learnable prototypes and mask-pooling-based supervision for background-incremental class disentanglement. By incorporating soft prompt tuning and encoder adaptation modifications, we further bridge the capability gap of CLIP between dense and sparse tasks, achieving state-of-the-art performance on both Pascal VOC and ADE20k, particularly in multi-step scenarios.

Method: Uses ‘mild-to-severe’ intra-subject generation and ‘real-synth’ mix-up training strategy to build resilience against appearance variations (modality, contrast, deformation, resolution, artifacts).

Result: Demonstrates robust performance across eleven public datasets for five tasks: image synthesis (CT/T1w/T2w/FLAIR MRI), anatomy segmentation, scalp-to-cortical distance, bias field estimation, and registration.

Conclusion: BrainFM provides a versatile foundation model that can be directly applied to diverse brain imaging tasks across multiple modalities without task-specific tuning, enabling broad clinical protocol applicability.

Abstract: Recent learning-based approaches have made astonishing advances in calibrated medical imaging like computerized tomography (CT), yet they struggle to generalize in uncalibrated modalities – notably magnetic resonance (MR) imaging, where performance is highly sensitive to the differences in MR contrast, resolution, and orientation. This prevents broad applicability to diverse real-world clinical protocols. Here we introduce BrainFM, a modality-agnostic, multi-task vision foundation model for human brain imaging. With the proposed “mild-to-severe” intra-subject generation and “real-synth” mix-up training strategy, BrainFM is resilient to the appearance of acquired images (e.g., modality, contrast, deformation, resolution, artifacts), and can be directly applied to five fundamental brain imaging tasks, including image synthesis for CT and T1w/T2w/FLAIR MRI, anatomy segmentation, scalp-to-cortical distance, bias field estimation, and registration. We evaluate the efficacy of BrainFM on eleven public datasets, and demonstrate its robustness and effectiveness across all tasks and input modalities. Code is available at https://github.com/jhuldr/BrainFM.

Method: Proposes Context-Aware Fusion (CAF) using cross-attention mechanisms to integrate global scene context (from a separate encoder) with local proposal features, enabling each object proposal to attend to comprehensive environmental information.

Result: Experimental results show improvement over state-of-the-art models on the CarDD benchmark, establishing new performance benchmarks for context-aware object detection in fine-grained domains.

Conclusion: The framework significantly enhances the generative detection paradigm by enabling better context integration, making it particularly effective for challenging fine-grained object detection tasks.

Abstract: Fine-grained object detection in challenging visual domains, such as vehicle damage assessment, presents a formidable challenge even for human experts to resolve reliably. While DiffusionDet has advanced the state-of-the-art through conditional denoising diffusion, its performance remains limited by local feature conditioning in context-dependent scenarios. We address this fundamental limitation by introducing Context-Aware Fusion (CAF), which leverages cross-attention mechanisms to integrate global scene context with local proposal features directly. The global context is generated using a separate dedicated encoder that captures comprehensive environmental information, enabling each object proposal to attend to scene-level understanding. Our framework significantly enhances the generative detection paradigm by enabling each object proposal to attend to comprehensive environmental information. Experimental results demonstrate an improvement over state-of-the-art models on the CarDD benchmark, establishing new performance benchmarks for context-aware object detection in fine-grained domains

Method: Proposes a training-free framework with: 1) Global-local decoupling module separating text into class nouns/modifiers and images into mask proposals, 2) Local-scale alignment with context-aware cropping and RS-specific knowledge enrichment, 3) Global-scale alignment using Cross-Scale Grad-CAM and mask selection modules.

Result: Enables mask classification for open-vocabulary semantic segmentation and achieves accurate, interpretable alignment across global and local dimensions for referring expression segmentation.

Conclusion: DGL-RSIS effectively bridges the domain gap for remote sensing segmentation without requiring additional training, providing a novel approach for visual-language alignment in specialized domains.

Abstract: The emergence of vision language models (VLMs) has bridged vision and language, enabling joint multimodal understanding beyond traditional visual-only deep learning models. However, transferring VLMs from the natural image domain to remote sensing (RS) segmentation remains challenging due to the limited category diversity in RS datasets and the domain gap between natural and RS imagery. Here, we propose a training-free framework, DGL-RSIS, that decouples visual and textual inputs, performing visual-language alignment at both the local semantic and global contextual levels through tailored strategies. Specifically, we first introduce a global-local decoupling (GLD) module, where text inputs are divided into local class nouns and global modifiers using natural language processing (NLP) techniques; image inputs are partitioned into a set of class-agnostic mask proposals via unsupervised mask proposal networks. Second, visual and textual features are aligned at local scale, through a novel context-aware cropping strategy for extracting image patches with proper boundaries and introducing RS-specific knowledge to enrich the text inputs. By matching the enhanced text features with mask-guided visual features, we enable the mask classification, supporting open-vocabulary semantic segmentation (OVSS). Third, at the global scale, we propose a Cross-Scale Grad-CAM module to refine Grad-CAM maps using contextual information from global modifiers. A subsequent mask selection module integrates pixel-level Grad-CAM activations into the mask-level segmentation output, such that accurate and interpretable alignment can be realized across global and local dimensions for referring expression segmentation (RES).

Method: Proposed UnorthoDOS approach using unorthorectified hyperspectral data from EMIT sensor. Trained ML models on both orthorectified and unorthorectified datasets, comparing against traditional matched filter baseline (mag1c).

Result: ML models on unorthorectified data achieved performance comparable to orthorectified data. Models on orthorectified data outperformed the matched filter baseline. Released datasets and code publicly available.

Conclusion: Bypassing orthorectification preprocessing is feasible for methane detection, enabling faster onboard satellite ML deployment with reduced computational costs while maintaining detection accuracy.

Abstract: Methane is a potent greenhouse gas and a major driver of climate change, making its timely detection critical for effective mitigation. Machine learning (ML) deployed onboard satellites can enable rapid detection while reducing downlink costs, supporting faster response systems. Conventional methane detection methods often rely on image processing techniques, such as orthorectification to correct geometric distortions and matched filters to enhance plume signals. We introduce a novel approach that bypasses these preprocessing steps by using \textit{unorthorectified} data (UnorthoDOS). We find that ML models trained on this dataset achieve performance comparable to those trained on orthorectified data. Moreover, we also train models on an orthorectified dataset, showing that they can outperform the matched filter baseline (mag1c). We release model checkpoints and two ML-ready datasets comprising orthorectified and unorthorectified hyperspectral images from the Earth Surface Mineral Dust Source Investigation (EMIT) sensor at https://huggingface.co/datasets/SpaceML/UnorthoDOS , along with code at https://github.com/spaceml-org/plume-hunter.

Method: Proposes a Multi-View State Space Modeling framework with Projective State Space (PSS) blocks that model joint spatial sequences at feature and keypoint levels, using Grid Token-guided Bidirectional Scanning (GTBS) instead of traditional Mamba scanning.

Result: Achieves significant improvements: +10.8 AP25 (+24%) on CMU Panoptic three-camera setting, +7.0 AP25 (+13%) on varying camera arrangements, and +15.3 PCP (+38%) on Campus A1 in cross-dataset evaluations.

Conclusion: MV-SSM demonstrates strong generalization capabilities and outperforms state-of-the-art methods in multi-view 3D human pose estimation, particularly in challenging scenarios with occlusions and varying camera configurations.

Abstract: While significant progress has been made in single-view 3D human pose estimation, multi-view 3D human pose estimation remains challenging, particularly in terms of generalizing to new camera configurations. Existing attention-based transformers often struggle to accurately model the spatial arrangement of keypoints, especially in occluded scenarios. Additionally, they tend to overfit specific camera arrangements and visual scenes from training data, resulting in substantial performance drops in new settings. In this study, we introduce a novel Multi-View State Space Modeling framework, named MV-SSM, for robustly estimating 3D human keypoints. We explicitly model the joint spatial sequence at two distinct levels: the feature level from multi-view images and the person keypoint level. We propose a Projective State Space (PSS) block to learn a generalized representation of joint spatial arrangements using state space modeling. Moreover, we modify Mamba’s traditional scanning into an effective Grid Token-guided Bidirectional Scanning (GTBS), which is integral to the PSS block. Multiple experiments demonstrate that MV-SSM achieves strong generalization, outperforming state-of-the-art methods: +10.8 on AP25 (+24%) on the challenging three-camera setting in CMU Panoptic, +7.0 on AP25 (+13%) on varying camera arrangements, and +15.3 PCP (+38%) on Campus A1 in cross-dataset evaluations. Project Website: https://aviralchharia.github.io/MV-SSM

Method: Created a comprehensive benchmark dataset with 100,000 facial images across four visually distinct domains, annotated with 39 attributes within 14 categories covering demographic and facial features.

Result: Extensive experiments revealed significant performance gaps under distribution shifts, highlighting the limitations of current fairness-aware approaches and the need for better domain adaptation techniques.

Conclusion: Face4FairShifts provides a valuable testbed for advancing equitable and reliable AI systems, demonstrating the critical need for more effective fairness-aware domain adaptation methods in facial analysis applications.

Abstract: Ensuring fairness and robustness in machine learning models remains a challenge, particularly under domain shifts. We present Face4FairShifts, a large-scale facial image benchmark designed to systematically evaluate fairness-aware learning and domain generalization. The dataset includes 100,000 images across four visually distinct domains with 39 annotations within 14 attributes covering demographic and facial features. Through extensive experiments, we analyze model performance under distribution shifts and identify significant gaps. Our findings emphasize the limitations of existing related datasets and the need for more effective fairness-aware domain adaptation techniques. Face4FairShifts provides a comprehensive testbed for advancing equitable and reliable AI systems. The dataset is available online at https://meviuslab.github.io/Face4FairShifts/.

Method: Employed computer vision techniques and image captioning with neural networks to detect jewels in images and generate natural language descriptions across three hierarchical levels. Used different image captioning architectures, particularly focusing on encoder-decoder models, and assembled a comprehensive jewelry image database for training and evaluation.

Result: The final model achieved captioning accuracy exceeding 90%, demonstrating effective recognition of diverse jewelry types and generation of accurate descriptions at varying detail levels.

Conclusion: The innovative approach successfully enables automatic jewelry identification and description, providing translators and interpreters with quick access to accurate information without relying on industry experts, effectively addressing the challenge of jewelry recognition across varied styles.

Abstract: Identifying jewelry pieces presents a significant challenge due to the wide range of styles and designs. Currently, precise descriptions are typically limited to industry experts. However, translators and interpreters often require a comprehensive understanding of these items. In this study, we introduce an innovative approach to automatically identify and describe jewelry using neural networks. This method enables translators and interpreters to quickly access accurate information, aiding in resolving queries and gaining essential knowledge about jewelry. Our model operates at three distinct levels of description, employing computer vision techniques and image captioning to emulate expert analysis of accessories. The key innovation involves generating natural language descriptions of jewelry across three hierarchical levels, capturing nuanced details of each piece. Different image captioning architectures are utilized to detect jewels in images and generate descriptions with varying levels of detail. To demonstrate the effectiveness of our approach in recognizing diverse types of jewelry, we assembled a comprehensive database of accessory images. The evaluation process involved comparing various image captioning architectures, focusing particularly on the encoder decoder model, crucial for generating descriptive captions. After thorough evaluation, our final model achieved a captioning accuracy exceeding 90 per cent.

Method: FtZ uses a novel vision tower framework with a semantically powerful anchor encoder and perception-rich augmenting encoder connected via lightweight Multi-Head Cross-Attention mechanism.

Result: Significantly outperforms single-encoder baselines and existing feature fusion methods on TextVQA, POPE, MMMU, MME and MM-Vet benchmarks requiring fine-grained visual understanding.

Conclusion: Composing heterogeneous expert encoders is an efficient and effective approach to overcome visual perception bottlenecks in MLLMs, offering a new design paradigm for next-generation AI systems with stronger perceptual capabilities.

Abstract: Multimodal Large Language Models (MLLMs) have made significant progress in bridging visual perception with high-level textual reasoning. However, they face a fundamental contradiction: while excelling at complex semantic understanding, these models often fail at basic visual tasks that require precise detail perception. This deficiency primarily stems from the prevalent architectural reliance on a single vision encoder optimized for high-level semantic alignment, which inherently sacrifices the ability to capture fine-grained visual information. To address this issue, we introduce Fusion to Enhance (FtZ), a novel vision tower framework. FtZ moves beyond the single-encoder design by innovatively composing a semantically powerful anchor encoder with a perception-rich augmenting encoder via a lightweight Multi-Head Cross-Attention mechanism. Experimental results demonstrate that on several challenging benchmarks demanding fine-grained visual understanding, such as TextVQA, POPE, MMMU, MME and MM-Vet, our FtZ model significantly outperforms baselines that use only a single encoder or existing feature fusion methods. This work proves that composing heterogeneous expert encoders is an efficient and effective path to overcoming the visual perception bottleneck in current MLLMs, offering a new design paradigm for building next-generation AI systems with stronger perceptual capabilities.

Method: Introduces Selecting-Tuning-Maintaining (STM) strategy that structurally decomposes pretrained VFM weights using entropy-rank and stable-rank. Adaptively selects rank numbers and task-aware singular directions for initialization, with principal direction regularization to preserve generalization.

Result: Extensive experiments on four real-world benchmarks across diverse weather conditions show STM outperforms existing PEFT methods, full fine-tuning, methods trained with adverse synthetic data, and even depth foundation models.

Conclusion: STM enables effective weather-generalized depth estimation through parameter-efficient fine-tuning while preserving pretrained knowledge, demonstrating superior performance across adverse weather conditions with minimal normal data.

Abstract: Monocular depth estimation under adverse weather conditions (e.g.\ rain, fog, snow, and nighttime) remains highly challenging due to the lack of reliable ground truth and the difficulty of learning from unlabeled real-world data. Existing methods often rely on synthetic adverse data with pseudo-labels, which suffer from domain gaps, or employ self-supervised learning, which violates photometric assumptions in adverse scenarios. In this work, we propose to achieve weather–generalized depth estimation by Parameter–Efficient Fine–Tuning (PEFT) of Vision Foundation Models (VFMs), using only a small amount of high–visibility (normal) data. While PEFT has shown strong performance in semantic tasks such as segmentation, it remains underexplored for geometry–centric tasks like depth estimation – especially in terms of balancing effective adaptation with the preservation of pretrained knowledge. To this end, we introduce the Selecting–Tuning–Maintaining (STM) strategy, which structurally decomposes the pretrained weights of VFMs based on two kinds of effective ranks (entropy–rank and stable–rank). In the tuning phase, we adaptively select the proper rank number as well as the task–aware singular directions for initialization, based on the entropy–rank and full–tuned weight; while in the maintaining stage, we enforce a principal direction regularization based on the stable–rank. This design guarantees flexible task adaptation while preserving the strong generalization capability of the pretrained VFM. Extensive experiments on four real–world benchmarks across diverse weather conditions demonstrate that STM not only outperforms existing PEFT methods and full fine–tuning but also surpasses methods trained with adverse synthetic data, and even the depth foundation model

Method: Reorganize preference-labeled critic datasets into verifiable training signals and perform reinforcement learning directly on a base generative model to create LLaVA-Critic models that optimize preference judgments while retaining generation ability.

Result: LLaVA-Critic-R1 emerges as both top-performing critic and competitive policy model, achieving +5.7% average gain over base model across 26 benchmarks. LLaVA-Critic-R1+ achieves 71.9 SoTA on MMMU at 7B scale. Self-critique at test time yields +13.8% improvement on reasoning tasks.

Conclusion: RL training on critic data produces unified models excelling at both evaluation and generation, offering a simple path toward scalable, self-improving multimodal systems that break the traditional critic-policy separation.

Abstract: In vision-language modeling, critic models are typically trained to evaluate outputs – assigning scalar scores or pairwise preferences – rather than to generate responses. This separation from policy models, which produce the responses, is so entrenched that critics are rarely considered for direct policy use. In this work, we challenge this convention. We propose to reorganize preference-labeled critic datasets into verifiable training signals and perform reinforcement learning directly on a base generative model, producing LLaVA-Critic-R1, a multimodal critic trained to optimize preference judgments while retaining full generation ability. Surprisingly, LLaVA-Critic-R1 emerges not only as a top-performing critic but also as a competitive policy model – matching or surpassing specialized reasoning VLMs trained with in-domain data across 26 visual reasoning and understanding benchmarks, with an average gain of +5.7% over its base model (Qwen-2.5-VL-7B). Extending this approach to existing strong reasoning VLMs yields LLaVA-Critic-R1+, which further advances policy performance without sacrificing critic quality, achieving a SoTA performance of 71.9 on MMMU at the 7B scale. Finally, we show that the enhanced critic ability benefits inference: applying self-critique at test time yields an average +13.8% improvement on five representative reasoning tasks without additional training. Our results reveal that RL training on critic data can produce a unified model excelling at both evaluation and generation, offering a simple path toward scalable, self-improving multimodal systems.

Method: Proposed Cross State Fusion Mamba (CSFMamba) Network with: 1) Preprocessing module for Mamba structure combined with CNN for multi-layer feature extraction, 2) Cross-state module based on Mamba operator for multimodal feature fusion, 3) Combined Mamba-CNN backbone for stronger full-image understanding.

Result: Experimental results on MUUFL and Houston2018 datasets show superior performance compared to Transformer while reducing network training burden.

Conclusion: CSFMamba effectively leverages Mamba’s computational efficiency and CNN’s feature extraction capabilities for improved multimodal remote sensing image classification with reduced computational complexity.

Abstract: Multimodal fusion has made great progress in the field of remote sensing image classification due to its ability to exploit the complementary spatial-spectral information. Deep learning methods such as CNN and Transformer have been widely used in these domains. State Space Models recently highlighted that prior methods suffer from quadratic computational complexity. As a result, modeling longer-range dependencies of spatial-spectral features imposes an overwhelming burden on the network. Mamba solves this problem by incorporating time-varying parameters into ordinary SSM and performing hardware optimization, but it cannot perform feature fusion directly. In order to make full use of Mamba’s low computational burden and explore the potential of internal structure in multimodal feature fusion, we propose Cross State Fusion Mamba (CSFMamba) Network. Specifically, we first design the preprocessing module of remote sensing image information for the needs of Mamba structure, and combine it with CNN to extract multi-layer features. Secondly, a cross-state module based on Mamba operator is creatively designed to fully fuse the feature of the two modalities. The advantages of Mamba and CNN are combined by designing a more powerful backbone. We capture the fusion relationship between HSI and LiDAR modalities with stronger full-image understanding. The experimental results on two datasets of MUUFL and Houston2018 show that the proposed method outperforms the experimental results of Transformer under the premise of reducing the network training burden.

Method: Proposes CascadeFormer - a family of two-stage cascading transformers: 1) masked pretraining stage to learn generalizable skeleton representations, 2) cascading fine-tuning stage tailored for discriminative action classification.

Result: Achieves competitive performance across three benchmark datasets: Penn Action, N-UCLA, and NTU RGB+D 60.

Conclusion: Transformer-based approaches with masked pretraining offer a promising alternative to GCNs for skeleton-based human action recognition, with code and model checkpoints released for reproducibility.

Abstract: Skeleton-based human action recognition leverages sequences of human joint coordinates to identify actions performed in videos. Owing to the intrinsic spatiotemporal structure of skeleton data, Graph Convolutional Networks (GCNs) have been the dominant architecture in this field. However, recent advances in transformer models and masked pretraining frameworks open new avenues for representation learning. In this work, we propose CascadeFormer, a family of two-stage cascading transformers for skeleton-based human action recognition. Our framework consists of a masked pretraining stage to learn generalizable skeleton representations, followed by a cascading fine-tuning stage tailored for discriminative action classification. We evaluate CascadeFormer across three benchmark datasets (Penn Action N-UCLA, and NTU RGB+D 60), achieving competitive performance on all tasks. To promote reproducibility, we release our code and model checkpoints.

Method: Adapt object detection datasets with fine-grained annotations into prompting format, design train/test splits with disjoint label sets to control seen/unseen concept separation, and analyze through mechanistic interpretability.

Result: Projection layer retains about 79-88% of performance on unseen classes compared to seen ones across various settings, showing non-trivial generalization without explicit alignment supervision.

Conclusion: The study introduces a new evaluation framework for alignment generalization and highlights potential for efficient VLM training with limited aligned data, with the projection layer functioning like a key-value memory.

Abstract: Vision-Language Models (VLMs) combine a vision encoder and a large language model (LLM) through alignment training, showing strong performance on multimodal tasks. A central component in this architecture is the projection layer, which maps visual features into the LLM’s embedding space. Despite its importance, its ability to generalize to unseen visual concepts has not been systematically evaluated. To address this, we propose a benchmark for evaluating projection-layer generalization. We adapt object detection datasets (rich in fine-grained annotations) into a prompting format and design train/test splits with disjoint label sets, enabling precise control over seen and unseen concept separation. Experimental results show that the projection layer retains about 79 to 88 percent of the performance on unseen classes compared to seen ones across various settings, suggesting a non-trivial level of generalization even without explicit alignment supervision on those concepts. We further analyze this behavior through a mechanistic interpretability lens. Our findings indicate that the feed-forward network in the projection layer functions like a key-value memory, processing seen and unseen tokens in similar ways. This study introduces a new evaluation framework for alignment generalization and highlights the potential for efficient VLM training with limited aligned data.

Method: Calculates skewness of feature maps in VGG convolution layers and Vision Transformer patches/heads to identify and reduce unnecessary channels related to skin tone, focusing attention on lesion areas rather than skin color.

Result: The approach lowers computational costs, mitigates bias without conventional statistical methods, potentially reduces model size while maintaining fairness, making it practical for real-world applications.

Conclusion: The proposed fairness algorithm effectively addresses skin tone bias in medical imaging models by focusing on lesion features rather than skin color characteristics, offering a computationally efficient solution for equitable healthcare diagnostics.

Abstract: Recent advances in deep learning have significantly improved the accuracy of skin lesion classification models, supporting medical diagnoses and promoting equitable healthcare. However, concerns remain about potential biases related to skin color, which can impact diagnostic outcomes. Ensuring fairness is challenging due to difficulties in classifying skin tones, high computational demands, and the complexity of objectively verifying fairness. To address these challenges, we propose a fairness algorithm for skin lesion classification that overcomes the challenges associated with achieving diagnostic fairness across varying skin tones. By calculating the skewness of the feature map in the convolution layer of the VGG (Visual Geometry Group) network and the patches and the heads of the Vision Transformer, our method reduces unnecessary channels related to skin tone, focusing instead on the lesion area. This approach lowers computational costs and mitigates bias without relying on conventional statistical methods. It potentially reduces model size while maintaining fairness, making it more practical for real-world applications.

Method: Proposes Causal Feature Explanation (CaFE) which leverages Effective Receptive Field (ERF) and applies input-attribution methods to identify image patches that causally drive specific SAE feature activations.

Result: ERF maps frequently diverge from naive activation maps, revealing hidden context dependencies. Patch insertion tests confirm CaFE more effectively recovers/suppresses feature activations than activation-ranked patches.

Conclusion: CaFE provides more faithful and semantically precise explanations of vision-SAE features, highlighting the risk of misinterpretation when relying solely on activation location analysis.

Abstract: Understanding what sparse auto-encoder (SAE) features in vision transformers truly represent is usually done by inspecting the patches where a feature’s activation is highest. However, self-attention mixes information across the entire image, so an activated patch often co-occurs with-but does not cause-the feature’s firing. We propose Causal Feature Explanation (CaFE), which leverages Effective Receptive Field (ERF). We consider each activation of an SAE feature to be a target and apply input-attribution methods to identify the image patches that causally drive that activation. Across CLIP-ViT features, ERF maps frequently diverge from naive activation maps, revealing hidden context dependencies (e.g., a “roaring face” feature that requires the co-occurrence of eyes and nose, rather than merely an open mouth). Patch insertion tests confirm that CaFE more effectively recovers or suppresses feature activations than activation-ranked patches. Our results show that CaFE yields more faithful and semantically precise explanations of vision-SAE features, highlighting the risk of misinterpretation when relying solely on activation location.

Method: Multi-stage framework using Qwen3 for article search, Qwen3-Reranker for contextual alignment, Qwen2-VL for image scoring, and Reciprocal Rank Fusion for output fusion.

Result: Achieved top-1 score on the private test set of Track 2 in the EVENTA 2025 Grand Challenge.

Conclusion: Combining language-based reasoning and multimodal retrieval is effective for complex, real-world image understanding tasks.

Abstract: Event-based image retrieval from free-form captions presents a significant challenge: models must understand not only visual features but also latent event semantics, context, and real-world knowledge. Conventional vision-language retrieval approaches often fall short when captions describe abstract events, implicit causality, temporal context, or contain long, complex narratives. To tackle these issues, we introduce a multi-stage retrieval framework combining dense article retrieval, event-aware language model reranking, and efficient image collection, followed by caption-guided semantic matching and rank-aware selection. We leverage Qwen3 for article search, Qwen3-Reranker for contextual alignment, and Qwen2-VL for precise image scoring. To further enhance performance and robustness, we fuse outputs from multiple configurations using Reciprocal Rank Fusion (RRF). Our system achieves the top-1 score on the private test set of Track 2 in the EVENTA 2025 Grand Challenge, demonstrating the effectiveness of combining language-based reasoning and multimodal retrieval for complex, real-world image understanding. The code is available at https://github.com/vdkhoi20/EVENT-Retriever.

Method: Leverages CLIP ViT-B/16 backbone enhanced with Low-Rank Adaptation (LoRA), multi-level CLS token aggregation, spherical feature interpolation, and class-specific natural language prompts. Uses joint training combining supervised classification with contrastive learning.

Result: Achieved 95% accuracy and F1-score in classification, Recall@1 of 0.93 (image-to-image) and 0.92 (text-to-image), MRR scores of 0.97 and 0.96. Won ACM MM'25 ENTRep Grand Challenge.

Conclusion: The framework effectively addresses multimodal medical understanding in low-resource clinical settings, with ablation studies confirming the incremental benefits of each architectural component for robust ENT endoscopy analysis.

Abstract: We present a unified vision-language framework tailored for ENT endoscopy image analysis that simultaneously tackles three clinically-relevant tasks: image classification, image-to-image retrieval, and text-to-image retrieval. Unlike conventional CNN-based pipelines that struggle to capture cross-modal semantics, our approach leverages the CLIP ViT-B/16 backbone and enhances it through Low-Rank Adaptation, multi-level CLS token aggregation, and spherical feature interpolation. These components collectively enable efficient fine-tuning on limited medical data while improving representation diversity and semantic alignment across modalities. To bridge the gap between visual inputs and textual diagnostic context, we introduce class-specific natural language prompts that guide the image encoder through a joint training objective combining supervised classification with contrastive learning. We validated our framework through participation in the ACM MM'25 ENTRep Grand Challenge, achieving 95% accuracy and F1-score in classification, Recall@1 of 0.93 and 0.92 for image-to-image and text-to-image retrieval respectively, and MRR scores of 0.97 and 0.96. Ablation studies demonstrated the incremental benefits of each architectural component, validating the effectiveness of our design for robust multimodal medical understanding in low-resource clinical settings.

Method: Introduces GaussianBridge to transform unstructured 3D Gaussians into Splatter Image format for neural processing, uses Gaussian-Uncertainty-Perceptual heatmap for imperceptibility, and dense segmentation-based extraction for robust message recovery.

Result: Enables arbitrary message embedding with preserved visual quality and reliable extraction even when watermarked objects occupy minimal regions in rendered views.

Conclusion: Proposes an efficient and robust watermarking solution for 3DGS models that eliminates the need for per-message fine-tuning while maintaining imperceptibility and extraction reliability.

Abstract: The growing popularity of 3D Gaussian Splatting (3DGS) has intensified the need for effective copyright protection. Current 3DGS watermarking methods rely on computationally expensive fine-tuning procedures for each predefined message. We propose the first generalizable watermarking framework that enables efficient protection of Splatter Image-based 3DGS models through a single forward pass. We introduce GaussianBridge that transforms unstructured 3D Gaussians into Splatter Image format, enabling direct neural processing for arbitrary message embedding. To ensure imperceptibility, we design a Gaussian-Uncertainty-Perceptual heatmap prediction strategy for preserving visual quality. For robust message recovery, we develop a dense segmentation-based extraction mechanism that maintains reliable extraction even when watermarked objects occupy minimal regions in rendered views. Project page: https://kevinhuangxf.github.io/marksplatter.

Method: Proposes two debiasing objectives: 1) ‘contrastive-then-calibration’ for input perspective - samples incorrect sibling triplets and reconstructs them correctly using positional priors; 2) ‘merge-then-split’ for output perspective - learns shared features among siblings then refines intra-group differentiation.

Result: Significant performance improvements: +9.18% mAP over baseline and +3.59% mAP over state-of-the-art on HICO-Det dataset across various settings.

Conclusion: The proposed debiasing methods effectively address the Toxic Siblings bias in HOI detection transformers, demonstrating substantial performance gains over existing approaches.

Abstract: Detection transformers have been applied to human-object interaction (HOI) detection, enhancing the localization and recognition of human-action-object triplets in images. Despite remarkable progress, this study identifies a critical issue-“Toxic Siblings” bias-which hinders the interaction decoder’s learning, as numerous similar yet distinct HOI triplets interfere with and even compete against each other both input side and output side to the interaction decoder. This bias arises from high confusion among sibling triplets/categories, where increased similarity paradoxically reduces precision, as one’s gain comes at the expense of its toxic sibling’s decline. To address this, we propose two novel debiasing learning objectives-“contrastive-then-calibration” and “merge-then-split”-targeting the input and output perspectives, respectively. The former samples sibling-like incorrect HOI triplets and reconstructs them into correct ones, guided by strong positional priors. The latter first learns shared features among sibling categories to distinguish them from other groups, then explicitly refines intra-group differentiation to preserve uniqueness. Experiments show that we significantly outperform both the baseline (+9.18% mAP on HICO-Det) and the state-of-the-art (+3.59% mAP) across various settings.

Method: Proposed an automatic motion extraction pipeline to collect interaction-rich human motions from 45.8K videos, creating InterPose dataset with 73.8K sequences of 3D human motions and corresponding text captions.

Result: InterPose brings significant improvements to state-of-the-art methods for human motion generation and enables development of an LLM-based agent for zero-shot animation of people interacting with diverse objects and scenes.

Conclusion: The InterPose dataset successfully addresses the lack of large-scale human-object interaction data and enables more realistic and versatile human motion generation in complex 3D scenes.

Abstract: Human motion generation has shown great advances thanks to the recent diffusion models trained on large-scale motion capture data. Most of existing works, however, currently target animation of isolated people in empty scenes. Meanwhile, synthesizing realistic human-object interactions in complex 3D scenes remains a critical challenge in computer graphics and robotics. One obstacle towards generating versatile high-fidelity human-object interactions is the lack of large-scale datasets with diverse object manipulations. Indeed, existing motion capture data is typically restricted to single people and manipulations of limited sets of objects. To address this issue, we propose an automatic motion extraction pipeline and use it to collect interaction-rich human motions. Our new dataset InterPose contains 73.8K sequences of 3D human motions and corresponding text captions automatically obtained from 45.8K videos with human-object interactions. We perform extensive experiments and demonstrate InterPose to bring significant improvements to state-of-the-art methods for human motion generation. Moreover, using InterPose we develop an LLM-based agent enabling zero-shot animation of people interacting with diverse objects and scenes.

Method: Proposes a two-stage framework: 1) cancelable PQ indexing for fast candidate filtering, and 2) cipher-space retrieval module for precise face ranking. Includes tailored protection mechanism for cancelable biometric authentication.

Result: Experiments on benchmark datasets show the method achieves a good balance between effectiveness, efficiency and security.

Conclusion: The proposed cancelable product quantization framework provides an efficient solution for secure face representation retrieval that addresses privacy concerns while maintaining practical performance.

Abstract: Despite the ubiquity of modern face retrieval systems, their retrieval stage is often outsourced to third-party entities, posing significant risks to user portrait privacy. Although homomorphic encryption (HE) offers strong security guarantees by enabling arithmetic computations in the cipher space, its high computational inefficiency makes it unsuitable for real-time, real-world applications. To address this issue, we propose Cancelable Product Quantization, a highly efficient framework for secure face representation retrieval. Our hierarchical two-stage framework comprises: (i) a high-throughput cancelable PQ indexing module for fast candidate filtering, and (ii) a fine-grained cipher-space retrieval module for final precise face ranking. A tailored protection mechanism is designed to secure the indexing module for cancelable biometric authentication while ensuring efficiency. Experiments on benchmark datasets demonstrate that our method achieves an decent balance between effectiveness, efficiency and security.

Method: Uses hierarchical approach with regular anchors and aligned anchor groups, sequentially linking anchors while updating predicted line segments, followed by simple validation and merging of adjacent segments.

Result: Quantitative experiments show AAGLSD effectively extracts complete line segments compared to other advanced detectors across various datasets.

Conclusion: AAGLSD provides an effective solution for precise and complete line segment detection without requiring complex refinement strategies, with code publicly available.

Abstract: This paper introduces a novel line segment detector, the Aligned Anchor Groups guided Line Segment Detector (AAGLSD), designed to detect line segments from images with high precision and completeness. The algorithm employs a hierarchical approach to extract candidate pixels with different saliency levels, including regular anchors and aligned anchor groups. AAGLSD initiates from these aligned anchor groups, sequentially linking anchors and updating the currently predicted line segment simultaneously. The final predictions are derived through straightforward validation and merging of adjacent line segments, avoiding complex refinement strategies. AAGLSD is evaluated on various datasets and quantitative experiments demonstrate that the proposed method can effectively extract complete line segments from input images compared to other advanced line segment detectors. The implementation is available at https://github.com/LLiDaBao/AAGLSD.

Method: Proposes a DDPM-based framework with cross-attention mechanisms, consisting of a pre-trained CLIP visual encoder and cross-attention enhanced U-Net diffusion model that learns to reconstruct brain signals through iterative denoising.

Result: Evaluated on THINGS-EEG2 and THINGS-MEG datasets, demonstrating effectiveness in generating biologically plausible brain signals. Visualized training and test M/EEG topographies to show intra-subject and inter-subject variations.

Conclusion: The cross-attention enhanced diffusion framework successfully addresses the biological plausibility issue in visual prostheses by enabling dynamic interaction between visual features and brain signal representations.

Abstract: Visual prostheses have shown great potential in restoring vision for blind individuals. On the one hand, researchers have been continuously improving the brain decoding framework of visual prostheses by leveraging the powerful image generation capabilities of diffusion models. On the other hand, the brain encoding stage of visual prostheses struggles to generate brain signals with sufficient biological similarity. Although existing works have recognized this problem, the quality of predicted stimuli still remains a critical issue, as existing approaches typically lack supervised signals from real brain responses to validate the biological plausibility of predicted stimuli. To address this issue, we propose a novel image-to-brain framework based on denoising diffusion probabilistic models (DDPMs) enhanced with cross-attention mechanisms. Our framework consists of two key architectural components: a pre-trained CLIP visual encoder that extracts rich semantic representations from input images, and a cross-attention enhanced U-Net diffusion model that learns to reconstruct biologically plausible brain signals through iterative denoising. Unlike conventional generative models that rely on simple concatenation for conditioning, our cross-attention modules enable dynamic interaction between visual features and brain signal representations, facilitating fine-grained alignment during the generation process. We evaluate our framework on two multimodal datasets (THINGS-EEG2 and THINGS-MEG) to demonstrate its effectiveness in generating biologically plausible brain signals. Moreover, we visualize the training and test M/EEG topographies for all subjects on both datasets to intuitively demonstrate the intra-subject variations and inter-subject variations in M/EEG signals.

Method: Developed OmniReason-Data (large-scale VLA datasets with dense spatiotemporal annotations) using hallucination-mitigated auto-labeling, and OmniReason-Agent architecture with sparse temporal memory and explanation generator using spatiotemporal knowledge distillation.

Result: Achieved state-of-the-art performance with significant improvements in open-loop planning tasks and visual question answering benchmarks, establishing new capabilities for interpretable, temporally-aware autonomous vehicles.

Conclusion: OmniReason framework successfully addresses the temporal reasoning gap in autonomous driving by providing robust spatiotemporal modeling and human-interpretable decision rationales for complex dynamic environments.

Abstract: Recent advances in vision-language models (VLMs) have demonstrated impressive spatial reasoning capabilities for autonomous driving, yet existing methods predominantly focus on static scene understanding while neglecting the essential temporal dimension of real-world driving scenarios. To address this critical limitation, we propose the OmniReason framework, which establishes robust spatiotemporal reasoning by jointly modeling dynamic 3D environments and their underlying decision-making processes. Our work makes two fundamental advances: (1) We introduce OmniReason-Data, two large-scale vision-language-action (VLA) datasets with dense spatiotemporal annotations and natural language explanations, generated through a novel hallucination-mitigated auto-labeling pipeline that ensures both physical plausibility and temporal coherence; (2) We develop the OmniReason-Agent architecture, which integrates a sparse temporal memory module for persistent scene context modeling and an explanation generator that produces human-interpretable decision rationales, facilitated by our spatiotemporal knowledge distillation approach that effectively captures spatiotemporal causal reasoning patterns. Comprehensive experiments demonstrate state-of-the-art performance, where OmniReason-Agent achieves significant improvements in both open-loop planning tasks and visual question answering (VQA) benchmarks, while establishing new capabilities for interpretable, temporally-aware autonomous vehicles operating in complex, dynamic environments.

Method: Proposes MI-RAG framework that iteratively leverages accumulated reasoning records to formulate multi-queries, performs joint search across visually-grounded and textual knowledge bases, and synthesizes new knowledge into reasoning records across multiple iterations.

Result: Significant improvements in both retrieval recall and answer accuracy on challenging benchmarks including Encyclopedic VQA, InfoSeek, and OK-VQA.

Conclusion: MI-RAG establishes a scalable approach for compositional reasoning in knowledge-intensive VQA by progressively refining understanding through iterative retrieval and reasoning across modalities.

Abstract: While Multimodal Large Language Models (MLLMs) have significantly advanced multimodal understanding, their performance remains limited on knowledge-intensive visual questions that require external knowledge beyond the image. Retrieval-Augmented Generation (RAG) has become a promising solution for providing models with external knowledge, its conventional single-pass framework often fails to gather sufficient knowledge. To overcome this limitation, we propose MI-RAG, a Multimodal Iterative RAG framework that leverages reasoning to enhance retrieval and update reasoning over newly retrieved knowledge across modalities. At each iteration, MI-RAG leverages an accumulated reasoning record to dynamically formulate a multi-query. These queries then drive a joint search across heterogeneous knowledge bases containing both visually-grounded and textual knowledge. The newly acquired knowledge is synthesized into the reasoning record, progressively refining understanding across iterations. Experiments on challenging benchmarks, including Encyclopedic VQA, InfoSeek, and OK-VQA, show that MI-RAG significantly improves both retrieval recall and answer accuracy, establishing a scalable approach for compositional reasoning in knowledge-intensive VQA.

Method: Embed semantic features into Gaussian primitives supervised by CLIP, use semantic consistency loss, and implement stage-wise training with coarse-to-fine learning and late-stage parameter refinement.

Result: Outperforms state-of-the-art methods on SeaThru-NeRF and Submerged3D datasets with up to 3.09 dB PSNR improvement on average.

Conclusion: The proposed framework provides robust and high-fidelity deep-sea scene reconstruction, making it suitable for underwater exploration and marine perception applications.

Abstract: Accurate 3D reconstruction in underwater environments remains a complex challenge due to issues such as light distortion, turbidity, and limited visibility. AI-based techniques have been applied to address these issues, however, existing methods have yet to fully exploit the potential of AI, particularly in integrating language models with visual processing. In this paper, we propose a novel framework that leverages multimodal cross-knowledge to create semantic-guided 3D Gaussian Splatting for robust and high-fidelity deep-sea scene reconstruction. By embedding an extra semantic feature into each Gaussian primitive and supervised by the CLIP extracted semantic feature, our method enforces semantic and structural awareness throughout the training. The dedicated semantic consistency loss ensures alignment with high-level scene understanding. Besides, we propose a novel stage-wise training strategy, combining coarse-to-fine learning with late-stage parameter refinement, to further enhance both stability and reconstruction quality. Extensive results show that our approach consistently outperforms state-of-the-art methods on SeaThru-NeRF and Submerged3D datasets across three metrics, with an improvement of up to 3.09 dB on average in terms of PSNR, making it a strong candidate for applications in underwater exploration and marine perception.

Method: ACAM uses a shallow texture-sensitive network to predict clinical contrast parameters, creates multiple contrast-enhanced views through differentiable mapping, and fuses them within downstream classifiers.

Result: Validated on 12,400 multi-center images across six anatomical categories, ACAM improved accuracy: lightweight models +2.02%, traditional models +1.29%, state-of-the-art models +1.15%.

Conclusion: The module bridges low-level image features with high-level semantics through physics-informed transformations aligned with clinical workflows, establishing a new paradigm for robust medical image analysis.

Abstract: Fetal ultrasound standard plane classification is essential for reliable prenatal diagnosis but faces inherent challenges, including low tissue contrast, boundary ambiguity, and operator-dependent image quality variations. To overcome these limitations, we propose a plug-and-play adaptive contrast adjustment module (ACAM), whose core design is inspired by the clinical practice of doctors adjusting image contrast to obtain clearer and more discriminative structural information. The module employs a shallow texture-sensitive network to predict clinically plausible contrast parameters, transforms input images into multiple contrast-enhanced views through differentiable mapping, and fuses them within downstream classifiers. Validated on a multi-center dataset of 12,400 images across six anatomical categories, the module consistently improves performance across diverse models, with accuracy of lightweight models increasing by 2.02 percent, accuracy of traditional models increasing by 1.29 percent, and accuracy of state-of-the-art models increasing by 1.15 percent. The innovation of the module lies in its content-aware adaptation capability, replacing random preprocessing with physics-informed transformations that align with sonographer workflows while improving robustness to imaging heterogeneity through multi-view fusion. This approach effectively bridges low-level image features with high-level semantics, establishing a new paradigm for medical image analysis under real-world image quality variations.

Method: Incremental Patch Generation (IPG) - a novel approach that efficiently generates adversarial patches through systematic incremental generation process.

Result: IPG achieves 11.1x efficiency improvement over existing methods while maintaining comparable attack performance, producing well-generalized patches that cover broader model vulnerabilities.

Conclusion: IPG has significant potential for adversarial patch defense and real-world applications including autonomous vehicles, security systems, and medical imaging where AI resilience is critical.

Abstract: The advent of adversarial patches poses a significant challenge to the robustness of AI models, particularly in the domain of computer vision tasks such as object detection. In contradistinction to traditional adversarial examples, these patches target specific regions of an image, resulting in the malfunction of AI models. This paper proposes Incremental Patch Generation (IPG), a method that generates adversarial patches up to 11.1 times more efficiently than existing approaches while maintaining comparable attack performance. The efficacy of IPG is demonstrated by experiments and ablation studies including YOLO’s feature distribution visualization and adversarial training results, which show that it produces well-generalized patches that effectively cover a broader range of model vulnerabilities. Furthermore, IPG-generated datasets can serve as a robust knowledge foundation for constructing a robust model, enabling structured representation, advanced reasoning, and proactive defenses in AI security ecosystems. The findings of this study suggest that IPG has considerable potential for future utilization not only in adversarial patch defense but also in real-world applications such as autonomous vehicles, security systems, and medical imaging, where AI models must remain resilient to adversarial attacks in dynamic and high-stakes environments.

Method: Sequential Difference Maximization (SDM) with three-layer ‘cycle-stage-step’ optimization framework. Uses negative probability of true label initially to compress solution space, then introduces Directional Probability Difference Ratio (DPDR) loss to gradually increase non-true labels’ probability upper bound.

Result: SDM demonstrates stronger attack performance and higher attack cost-effectiveness compared to previous SOTA methods. Can be combined with adversarial training to enhance defensive effects.

Conclusion: SDM provides an effective gradient-based adversarial attack method that improves both attack performance and efficiency while being compatible with adversarial training defenses.

Abstract: Efficient adversarial attack methods are critical for assessing the robustness of computer vision models. In this paper, we reconstruct the optimization objective for generating adversarial examples as “maximizing the difference between the non-true labels’ probability upper bound and the true label’s probability,” and propose a gradient-based attack method termed Sequential Difference Maximization (SDM). SDM establishes a three-layer optimization framework of “cycle-stage-step.” The processes between cycles and between iterative steps are respectively identical, while optimization stages differ in terms of loss functions: in the initial stage, the negative probability of the true label is used as the loss function to compress the solution space; in subsequent stages, we introduce the Directional Probability Difference Ratio (DPDR) loss function to gradually increase the non-true labels’ probability upper bound by compressing the irrelevant labels’ probabilities. Experiments demonstrate that compared with previous SOTA methods, SDM not only exhibits stronger attack performance but also achieves higher attack cost-effectiveness. Additionally, SDM can be combined with adversarial training methods to enhance their defensive effects. The code is available at https://github.com/X-L-Liu/SDM.

Method: Combines U-Net’s local feature training with Vision Transformer’s global processing, uses Gaussian filter-based loss for noise removal, SP masking for boundary reinforcement, and Gabor filters in post-processing for defect orientation/frequency emphasis.

Result: Achieved average AUC of 0.939 across MVTec-AD, Surface Crack Detection, and Marble Surface Anomaly datasets, with ablation studies confirming optimal filter size and noise probability enhance performance.

Conclusion: The proposed approach effectively detects surface defects across various textures with robust performance in noisy environments through optimized parameter settings and combined local-global processing.

Abstract: This paper proposes a novel approach to enhance the accuracy and reliability of texture-based surface defect detection using Gabor filters and a blurring U-Net-ViT model. By combining the local feature training of U-Net with the global processing of the Vision Transformer(ViT), the model effectively detects defects across various textures. A Gaussian filter-based loss function removes background noise and highlights defect patterns, while Salt-and-Pepper(SP) masking in the training process reinforces texture-defect boundaries, ensuring robust performance in noisy environments. Gabor filters are applied in post-processing to emphasize defect orientation and frequency characteristics. Parameter optimization, including filter size, sigma, wavelength, gamma, and orientation, maximizes performance across datasets like MVTec-AD, Surface Crack Detection, and Marble Surface Anomaly Dataset, achieving an average Area Under the Curve(AUC) of 0.939. The ablation studies validate that the optimal filter size and noise probability significantly enhance defect detection performance.

Method: Proposes end-to-end optimization with camera poses as learnable parameters, models motion as per-primitive SE(3) transformations on 3D Gaussians, and uses a three-stage training schedule alternating between pose and Gaussian optimization.

Result: Achieves significant improvements in reconstruction quality and pose estimation accuracy on Stereo Blur dataset and real-world sequences compared to prior methods.

Conclusion: Joint optimization of camera poses and 3DGS attributes effectively handles motion blur and produces superior dynamic 3D scene reconstruction.

Abstract: Reconstructing dynamic 3D scenes from monocular video has broad applications in AR/VR, robotics, and autonomous navigation, but often fails due to severe motion blur caused by camera and object motion. Existing methods commonly follow a two-step pipeline, where camera poses are first estimated and then 3D Gaussians are optimized. Since blurring artifacts usually undermine pose estimation, pose errors could be accumulated to produce inferior reconstruction results. To address this issue, we introduce a unified optimization framework by incorporating camera poses as learnable parameters complementary to 3DGS attributes for end-to-end optimization. Specifically, we recast camera and object motion as per-primitive SE(3) affine transformations on 3D Gaussians and formulate a unified optimization objective. For stable optimization, we introduce a three-stage training schedule that optimizes camera poses and Gaussians alternatively. Particularly, 3D Gaussians are first trained with poses being fixed, and then poses are optimized with 3D Gaussians being untouched. Finally, all learnable parameters are optimized together. Extensive experiments on the Stereo Blur dataset and challenging real-world sequences demonstrate that our method achieves significant gains in reconstruction quality and pose estimation accuracy over prior dynamic deblurring methods.

Method: SegDINO extracts multi-level features from a frozen pretrained DINOv3 encoder, aligns them to a common resolution and channel width, and uses a lightweight MLP head to directly predict segmentation masks.

Result: Extensive experiments across six benchmarks (three medical datasets: TN3K, Kvasir-SEG, ISIC; three natural image datasets: MSD, VMD-D, ViSha) demonstrate that SegDINO consistently achieves state-of-the-art performance compared to existing methods.

Conclusion: SegDINO provides an efficient segmentation framework that minimizes trainable parameters while preserving the representational power of foundation features from DINOv3, offering strong performance across diverse segmentation tasks.

Abstract: The DINO family of self-supervised vision models has shown remarkable transferability, yet effectively adapting their representations for segmentation remains challenging. Existing approaches often rely on heavy decoders with multi-scale fusion or complex upsampling, which introduce substantial parameter overhead and computational cost. In this work, we propose SegDINO, an efficient segmentation framework that couples a frozen DINOv3 backbone with a lightweight decoder. SegDINO extracts multi-level features from the pretrained encoder, aligns them to a common resolution and channel width, and utilizes a lightweight MLP head to directly predict segmentation masks. This design minimizes trainable parameters while preserving the representational power of foundation features. Extensive experiments across six benchmarks, including three medical datasets (TN3K, Kvasir-SEG, ISIC) and three natural image datasets (MSD, VMD-D, ViSha), demonstrate that SegDINO consistently achieves state-of-the-art performance compared to existing methods. Code is available at https://github.com/script-Yang/SegDINO.

Method: Hybrid framework integrating Swin Transformer and U-Net with SwinRRDB blocks for global context and local detail learning, plus composite loss function (L2 + guided + watershed loss) for structural preservation.

Result: Outperforms state-of-the-art models on RICE and SateHaze1K datasets, achieving PSNR of 33.24 dB and SSIM of 0.967 on RICE dataset.

Conclusion: Provides effective solution for mitigating atmospheric interference in satellite imagery with potential applicability across diverse remote sensing applications.

Abstract: Satellite imagery plays a crucial role in various fields; however, atmospheric interference and haze significantly degrade image clarity and reduce the accuracy of information extraction. To address these challenges, this paper proposes a hybrid dehazing framework that integrates Swin Transformer and U-Net to balance global context learning and local detail restoration, called SUFERNOBWA. The proposed network employs SwinRRDB, a Swin Transformer-based Residual-in-Residual Dense Block, in both the encoder and decoder to effectively extract features. This module enables the joint learning of global contextual information and fine spatial structures, which is crucial for structural preservation in satellite image. Furthermore, we introduce a composite loss function that combines L2 loss, guided loss, and a novel watershed loss, which enhances structural boundary preservation and ensures pixel-level accuracy. This architecture enables robust dehazing under diverse atmospheric conditions while maintaining structural consistency across restored images. Experimental results demonstrate that the proposed method outperforms state-of-the-art models on both the RICE and SateHaze1K datasets. Specifically, on the RICE dataset, the proposed approach achieved a PSNR of 33.24 dB and an SSIM of 0.967, which is a significant improvement over existing method. This study provides an effective solution for mitigating atmospheric interference in satellite imagery and highlights its potential applicability across diverse remote sensing applications.

Method: Two-stage approach: 1) 360-degree scene extrapolation using panorama diffusion model, 2) novel view interpolation using perspective keyframes warped from panorama and spatial noise diffusion in a video diffusion model.

Result: Outperforms existing methods in generating coherent views along user-defined trajectories, maintains global consistency even in loop closure scenarios, and enables flexible camera control.

Conclusion: The proposed decomposition approach effectively addresses long-term view and scene consistency challenges in single-view novel view synthesis, producing superior results compared to prior work.

Abstract: Novel view synthesis (NVS) from a single image is highly ill-posed due to large unobserved regions, especially for views that deviate significantly from the input. While existing methods focus on consistency between the source and generated views, they often fail to maintain coherence and correct view alignment across long-range or looped trajectories. We propose a model that addresses this by decomposing single-view NVS into a 360-degree scene extrapolation followed by novel view interpolation. This design ensures long-term view and scene consistency by conditioning on keyframes extracted and warped from a generated panoramic representation. In the first stage, a panorama diffusion model learns the scene prior from the input perspective image. Perspective keyframes are then sampled and warped from the panorama and used as anchor frames in a pre-trained video diffusion model, which generates novel views through a proposed spatial noise diffusion process. Compared to prior work, our method produces globally consistent novel views – even in loop closure scenarios – while enabling flexible camera control. Experiments on diverse scene datasets demonstrate that our approach outperforms existing methods in generating coherent views along user-defined trajectories. Our implementation is available at https://github.com/YiGuYT/LookBeyond.

Method: Proposes FQAT with layer-wise freezing mechanism to mitigate gradient conflicts and disorder-guided adaptive freezing algorithm with gradient disorder metric to dynamically identify and freeze unstable layers.

Result: Extensive experiments show FQAT outperforms state-of-the-art baselines on both I.D and OOD image classification tasks across influential benchmarks.

Conclusion: FQAT successfully addresses the OOD generalization problem in QAT by incorporating flatness optimization through intelligent layer freezing strategies, achieving superior performance on both distribution types.

Abstract: Current quantization-aware training (QAT) methods primarily focus on enhancing the performance of quantized models on in-distribution (I.D) data, while overlooking the potential performance degradation on out-of-distribution (OOD) data. In this paper, we first substantiate this problem through rigorous experiment, showing that QAT can lead to a significant OOD generalization performance degradation. Further, we find the contradiction between the perspective that flatness of loss landscape gives rise to superior OOD generalization and the phenomenon that QAT lead to a sharp loss landscape, can cause the above problem. Therefore, we propose a flatness-oriented QAT method, FQAT, to achieve generalizable QAT. Specifically, i) FQAT introduces a layer-wise freezing mechanism to mitigate the gradient conflict issue between dual optimization objectives (i.e., vanilla QAT and flatness). ii) FQAT proposes an disorder-guided adaptive freezing algorithm to dynamically determines which layers to freeze at each training step, effectively addressing the challenges caused by interference between layers. A gradient disorder metric is designed to help the algorithm identify unstable layers during training. Extensive experiments on influential OOD benchmark demonstrate the superiority of our method over state-of-the-art baselines under both I.D and OOD image classification tasks.

Method: Introduces Scoliosis1K-Pose dataset with 447,900 pose frames from 1,050 adolescents. Develops Dual Representation Framework (DRF) combining continuous skeleton maps with discrete Postural Asymmetry Vector (PAV), enhanced by PAV-Guided Attention module for clinical prior integration.

Result: DRF achieves state-of-the-art performance. Visualizations confirm the model effectively uses clinical asymmetry cues to guide feature extraction and creates synergy between dual representations.

Conclusion: The proposed framework successfully addresses pose-based scoliosis screening limitations by integrating clinical knowledge with deep learning, providing improved interpretability and performance. Dataset and code are publicly available.

Abstract: Recent AI-based scoliosis screening methods primarily rely on large-scale silhouette datasets, often neglecting clinically relevant postural asymmetries-key indicators in traditional screening. In contrast, pose data provide an intuitive skeletal representation, enhancing clinical interpretability across various medical applications. However, pose-based scoliosis screening remains underexplored due to two main challenges: (1) the scarcity of large-scale, annotated pose datasets; and (2) the discrete and noise-sensitive nature of raw pose coordinates, which hinders the modeling of subtle asymmetries. To address these limitations, we introduce Scoliosis1K-Pose, a 2D human pose annotation set that extends the original Scoliosis1K dataset, comprising 447,900 frames of 2D keypoints from 1,050 adolescents. Building on this dataset, we introduce the Dual Representation Framework (DRF), which integrates a continuous skeleton map to preserve spatial structure with a discrete Postural Asymmetry Vector (PAV) that encodes clinically relevant asymmetry descriptors. A novel PAV-Guided Attention (PGA) module further uses the PAV as clinical prior to direct feature extraction from the skeleton map, focusing on clinically meaningful asymmetries. Extensive experiments demonstrate that DRF achieves state-of-the-art performance. Visualizations further confirm that the model leverages clinical asymmetry cues to guide feature extraction and promote synergy between its dual representations. The dataset and code are publicly available at https://zhouzi180.github.io/Scoliosis1K/.

Method: Evaluates visual tokens from sample-wise and semantic-wise perspectives, retains top-scoring tokens, uses class-specific semantic memory bank with learned prototypes, and implements two-level ranking mechanism for dynamic token-prototype weighting.

Result: Outperforms CLIP by up to 11.19% in harmonic mean accuracy across 11 few-shot benchmarks, achieves up to 0.8K additional FPS, with only 21 extra parameters.

Conclusion: Spotlighter establishes an effective and scalable baseline for prompt tuning, demonstrating significant improvements in both accuracy and efficiency with minimal parameter overhead.

Abstract: CLIP’s success has demonstrated that prompt tuning can achieve robust cross-modal semantic alignment for tasks ranging from open-domain recognition to fine-grained classification. However, redundant or weakly relevant feature components introduce noise and incur unnecessary computational costs. In this work, we propose Spotlighter, a lightweight token-selection framework that simultaneously enhances accuracy and efficiency in prompt tuning. Spotlighter evaluates each visual token’s activation from both sample-wise and semantic-wise perspectives and retains only the top-scoring tokens for downstream prediction. A class-specific semantic memory bank of learned prototypes refines this selection, ensuring semantic representativeness and compensating for discarded features. To further prioritize informative signals, we introduce a two-level ranking mechanism that dynamically weights token–prototype interactions. Across 11 few-shot benchmarks, Spotlighter outperforms CLIP by up to 11.19% in harmonic mean accuracy and achieves up to 0.8K additional FPS, with only 21 extra parameters. These results establish Spotlighter as an effective and scalable baseline for prompt tuning. Code for our method will be available at https://github.com/greatest-gourmet/Spotlighter.

Method: Uses a conditioning scheme leveraging capture metadata to guide alignment/denoising, and a Burst-Order Selective Scan (BOSS) mechanism for long-range temporal dependency modeling.

Result: Achieved first place in AIM 2025 Low-light RAW Video Denoising Challenge and demonstrates state-of-the-art performance on rigorous benchmark datasets.

Conclusion: DarkVRAI sets a new standard for low-light video denoising through synergistic combination of metadata conditioning and advanced temporal modeling.

Abstract: Low-light RAW video denoising is a fundamentally challenging task due to severe signal degradation caused by high sensor gain and short exposure times, which are inherently limited by video frame rate requirements. To address this, we propose DarkVRAI, a novel framework that achieved first place in the AIM 2025 Low-light RAW Video Denoising Challenge. Our method introduces two primary contributions: (1) a successful application of a conditioning scheme for image denoising, which explicitly leverages capture metadata, to video denoising to guide the alignment and denoising processes, and (2) a Burst-Order Selective Scan (BOSS) mechanism that effectively models long-range temporal dependencies within the noisy video sequence. By synergistically combining these components, DarkVRAI demonstrates state-of-the-art performance on a rigorous and realistic benchmark dataset, setting a new standard for low-light video denoising.

Method: Uses lightweight language model to generate soft caption tokens as visual representations, trained with semantic density-aware supervision to dynamically adjust compression based on scene richness (description length).

Result: Achieves 49% reduction in FLOPs compared to VideoGPT+ while maintaining competitive performance, with adaptive compression based on video segment richness.

Conclusion: LangDC successfully mimics human visual processing by dynamically adjusting token compression based on scene complexity, providing efficient and effective video representation.

Abstract: Recent advancements in large video-language models have revolutionized video understanding tasks. However, their efficiency is significantly constrained by processing high volumes of visual tokens. Existing token compression strategies apply a fixed compression ratio, ignoring the variability in semantic density among different video clips. Consequently, this lead to inadequate representation of information-rich clips due to insufficient tokens and unnecessary computation on static or content-poor ones. To address this, we propose LangDC, a Language-aware Dynamic Token Compressor. LangDC leverages a lightweight language model to describe video clips, converting them into soft caption tokens as visual representations. Trained with our proposed semantic density-aware supervision, LangDC aims to 1) cover key visual cues necessary for downstream task reasoning and 2) dynamically adjust compression ratios based on scene richness, reflected by descriptions length. Our design mimics how humans dynamically express what they see: complex scenes (seeing more) elicit more detailed language to convey nuances (saying more), whereas simpler scenes are described with fewer words. Experimental results show that our method reduces FLOPs by 49% compared to VideoGPT+ while maintaining competitive performance. Furthermore, qualitative results demonstrate our approach adaptively adjusts the token compression ratio based on video segment richness.

Method: Evaluated three strategies: 1) Separate per-band reconstruction, 2) Splitting optimization (first optimize RGB geometry then fit new bands), 3) Joint optimization (modalities optimized together, optionally with initial RGB-only phase). Integrated multi-spectral data into spherical harmonics color components.

Result: Joint optimization strategy significantly improved overall spectral reconstruction and enhanced RGB results through spectral cross-talk. Analysis revealed key trade-offs in when and how to introduce spectral bands during optimization.

Conclusion: Integrating multi-spectral data directly into spherical harmonics color components effectively models each Gaussian’s multi-spectral reflectance. Joint optimization approach provides robust multi-modal 3DGS reconstruction with practical insights for spectral band introduction timing.

Abstract: We present a study of how to integrate color (RGB) and multi-spectral imagery (red, green, red-edge, and near-infrared) into the 3D Gaussian Splatting (3DGS) framework, a state-of-the-art explicit radiance-field-based method for fast and high-fidelity 3D reconstruction from multi-view images. While 3DGS excels on RGB data, naive per-band optimization of additional spectra yields poor reconstructions due to inconsistently appearing geometry in the spectral domain. This problem is prominent, even though the actual geometry is the same, regardless of spectral modality. To investigate this, we evaluate three strategies: 1) Separate per-band reconstruction with no shared structure. 2) Splitting optimization, in which we first optimize RGB geometry, copy it, and then fit each new band to the model by optimizing both geometry and band representation. 3) Joint, in which the modalities are jointly optimized, optionally with an initial RGB-only phase. We showcase through quantitative metrics and qualitative novel-view renderings on multi-spectral datasets the effectiveness of our dedicated optimized Joint strategy, increasing overall spectral reconstruction as well as enhancing RGB results through spectral cross-talk. We therefore suggest integrating multi-spectral data directly into the spherical harmonics color components to compactly model each Gaussian’s multi-spectral reflectance. Moreover, our analysis reveals several key trade-offs in when and how to introduce spectral bands during optimization, offering practical insights for robust multi-modal 3DGS reconstruction.

Method: Case study analyzing eye gaze behavior on the same highway route under clear vs rainy conditions using a two-step clustering approach: clustering gaze points within 10-second intervals, then aggregating cluster centroids into meta-clusters. Also used Markov transition matrices, fixation duration, gaze elevation, and azimuth distribution metrics.

Result: While overall gaze behavior (road focus with occasional mirror checks) remains consistent, rainy conditions lead to: more frequent dashboard glances, longer fixation durations, and higher gaze elevation. These changes indicate increased cognitive focus during adverse weather.

Conclusion: The findings provide valuable insights into visual attention patterns under adverse conditions and demonstrate the potential of leveraging gaze modeling to design more robust ADAS and DMS systems that account for weather-related behavioral adaptations.

Abstract: Rainy weather significantly increases the risk of road accidents due to reduced visibility and vehicle traction. Understanding how experienced drivers adapt their visual perception through gaze behavior under such conditions is critical for designing robust driver monitoring systems (DMS) and for informing advanced driver assistance systems (ADAS). This case study investigates the eye gaze behavior of a driver operating the same highway route under both clear and rainy conditions. To this end, gaze behavior was analyzed by a two-step clustering approach: first, clustering gaze points within 10-second intervals, and then aggregating cluster centroids into meta-clusters. This, along with Markov transition matrices and metrics such as fixation duration, gaze elevation, and azimuth distributions, reveals meaningful behavioral shifts. While the overall gaze behavior focused on the road with occasional mirror checks remains consistent, rainy conditions lead to more frequent dashboard glances, longer fixation durations, and higher gaze elevation, indicating increased cognitive focus. These findings offer valuable insight into visual attention patterns under adverse conditions and highlight the potential of leveraging gaze modeling to aid in the design of more robust ADAS and DMS.

Method: Developed automated coral reseeding devices using AI, computer vision, and robotics for substrate classification to detect suitable seafloor areas for coral growth.

Result: Achieved 77.8% deployment accuracy, 89.1% sub-image patch classification accuracy, and real-time inference at 5.5 frames per second on Great Barrier Reef. Contributed large annotated substrate image dataset publicly.

Conclusion: Automated AI-powered coral restoration system successfully demonstrates scalable reef rehabilitation with high accuracy, reducing human dependency and enabling efficient large-scale deployment.

Abstract: Coral reefs are on the brink of collapse, with climate change, ocean acidification, and pollution leading to a projected 70-90% loss of coral species within the next decade. Restoration efforts are crucial, but their success hinges on introducing automation to upscale efforts. We present automated deployment of coral re-seeding devices powered by artificial intelligence, computer vision, and robotics. Specifically, we perform automated substrate classification, enabling detection of areas of the seafloor suitable for coral growth, thus significantly reducing reliance on human experts and increasing the range and efficiency of restoration. Real-world testing of the algorithms on the Great Barrier Reef leads to deployment accuracy of 77.8%, sub-image patch classification of 89.1%, and real-time model inference at 5.5 frames per second. Further, we present and publicly contribute a large collection of annotated substrate image data to foster future research in this area.

Method: CompSlider generates a conditional prior for T2I foundation models, using novel disentanglement and structure losses to compose multiple attribute changes while maintaining image structural consistency. It operates in latent space without retraining the foundation model.

Result: The approach enables reliable and independent manipulation of multiple attributes simultaneously, reduces computational burden for training and inference, and demonstrates generality by extending to video generation.

Conclusion: CompSlider successfully addresses attribute entanglement in slider-based generation, providing precise multi-attribute control while maintaining structural consistency and computational efficiency.

Abstract: In text-to-image (T2I) generation, achieving fine-grained control over attributes - such as age or smile - remains challenging, even with detailed text prompts. Slider-based methods offer a solution for precise control of image attributes. Existing approaches typically train individual adapter for each attribute separately, overlooking the entanglement among multiple attributes. As a result, interference occurs among different attributes, preventing precise control of multiple attributes together. To address this challenge, we aim to disentangle multiple attributes in slider-based generation to enbale more reliable and independent attribute manipulation. Our approach, CompSlider, can generate a conditional prior for the T2I foundation model to control multiple attributes simultaneously. Furthermore, we introduce novel disentanglement and structure losses to compose multiple attribute changes while maintaining structural consistency within the image. Since CompSlider operates in the latent space of the conditional prior and does not require retraining the foundation model, it reduces the computational burden for both training and inference. We evaluate our approach on a variety of image attributes and highlight its generality by extending to video generation.

Method: Uses real paired images of clean/contaminated glass and proposes an all-in-one model with self-supervised test-time adaptation to handle different contamination types by updating for each test image’s unique occlusion.

Result: Outperforms state-of-the-art methods quantitatively and qualitatively, especially on unseen contamination types, demonstrating effective restoration of realistic contaminated images.

Conclusion: The proposed approach successfully handles diverse real-world glass contaminants through real data collection and adaptive test-time learning, showing superior performance over existing methods.

Abstract: Images taken through window glass are often degraded by contaminants adhered to the glass surfaces. Such contaminants cause occlusions that attenuate the incoming light and scatter stray light towards the camera. Most of existing deep learning methods for neutralizing the effects of contaminated glasses relied on synthetic training data. Few researchers used real degraded and clean image pairs, but they only considered removing or alleviating the effects of rain drops on glasses. This paper is concerned with the more challenging task of learning the restoration of images taken through glasses contaminated by a wide range of occluders, including muddy water, dirt and other small foreign particles found in reality. To facilitate the learning task we have gone to a great length to acquire real paired images with and without glass contaminants. More importantly, we propose an all-in-one model to neutralize contaminants of different types by utilizing the one-shot test-time adaptation mechanism. It involves a self-supervised auxiliary learning task to update the trained model for the unique occlusion type of each test image. Experimental results show that the proposed method outperforms the state-of-the-art methods quantitatively and qualitatively in cleaning realistic contaminated images, especially the unseen ones.

Method: Uses contrastive pre-trained encoder on 190M unlabeled images to learn stain-invariant features, combined with unified conditional diffusion process that adapts to specific tasks via textual prompts.

Result: Achieves statistically significant improvements (p<0.01) over SOTA methods in 56/66 tasks, with 10-15% PSNR gains for restoration and 12-18% SSIM improvements for virtual staining.

Conclusion: LPFM successfully bridges the gap in low-level pathology image enhancement by providing a unified foundation model capable of handling multiple restoration and translation tasks with superior performance.

Abstract: Foundation models have revolutionized computational pathology by achieving remarkable success in high-level diagnostic tasks, yet the critical challenge of low-level image enhancement remains largely unaddressed. Real-world pathology images frequently suffer from degradations such as noise, blur, and low resolution due to slide preparation artifacts, staining variability, and imaging constraints, while the reliance on physical staining introduces significant costs, delays, and inconsistency. Although existing methods target individual problems like denoising or super-resolution, their task-specific designs lack the versatility to handle the diverse low-level vision challenges encountered in practice. To bridge this gap, we propose the first unified Low-level Pathology Foundation Model (LPFM), capable of enhancing image quality in restoration tasks, including super-resolution, deblurring, and denoising, as well as facilitating image translation tasks like virtual staining (H&E and special stains), all through a single adaptable architecture. Our approach introduces a contrastive pre-trained encoder that learns transferable, stain-invariant feature representations from 190 million unlabeled pathology images, enabling robust identification of degradation patterns. A unified conditional diffusion process dynamically adapts to specific tasks via textual prompts, ensuring precise control over output quality. Trained on a curated dataset of 87,810 whole slied images (WSIs) across 34 tissue types and 5 staining protocols, LPFM demonstrates statistically significant improvements (p<0.01) over state-of-the-art methods in most tasks (56/66), achieving Peak Signal-to-Noise Ratio (PSNR) gains of 10-15% for image restoration and Structural Similarity Index Measure (SSIM) improvements of 12-18% for virtual staining.

Method: Proposes SpectMamba with Hybrid Spatial-Frequency Attention block to separately learn high/low-frequency features, Visual State-Space Module for long-range dependencies, and Hilbert Curve Scanning to strengthen spatial correlations.

Result: Achieves state-of-the-art performance across various medical image detection tasks while maintaining both effectiveness and efficiency.

Conclusion: SpectMamba successfully addresses the limitations of CNNs and Transformers in medical imaging, providing an efficient and accurate solution for abnormality detection with global context capture capabilities.

Abstract: Abnormality detection in medical imaging is a critical task requiring both high efficiency and accuracy to support effective diagnosis. While convolutional neural networks (CNNs) and Transformer-based models are widely used, both face intrinsic challenges: CNNs have limited receptive fields, restricting their ability to capture broad contextual information, and Transformers encounter prohibitive computational costs when processing high-resolution medical images. Mamba, a recent innovation in natural language processing, has gained attention for its ability to process long sequences with linear complexity, offering a promising alternative. Building on this foundation, we present SpectMamba, the first Mamba-based architecture designed for medical image detection. A key component of SpectMamba is the Hybrid Spatial-Frequency Attention (HSFA) block, which separately learns high- and low-frequency features. This approach effectively mitigates the loss of high-frequency information caused by frequency bias and correlates frequency-domain features with spatial features, thereby enhancing the model’s ability to capture global context. To further improve long-range dependencies, we propose the Visual State-Space Module (VSSM) and introduce a novel Hilbert Curve Scanning technique to strengthen spatial correlations and local dependencies, further optimizing the Mamba framework. Comprehensive experiments show that SpectMamba achieves state-of-the-art performance while being both effective and efficient across various medical image detection tasks.

Method: Bidirectional Sparse Attention (BSA) framework that dynamically sparsifies both queries and key-value pairs in 3D full attention. Query sparsity via semantic similarity selection and dynamic spatial-time training, KV sparsity through statistical dynamic thresholding to retain only salient blocks.

Result: BSA reduces FLOPs by up to 20x, achieves 17.79x faster attention training, and preserves or even surpasses the generative quality of full attention across long sequences.

Conclusion: BSA effectively overcomes computational bottlenecks in video DiTs by dynamically optimizing attention sparsity, enabling efficient high-quality video generation without sacrificing performance.

Abstract: Video diffusion Transformer (DiT) models excel in generative quality but hit major computational bottlenecks when producing high-resolution, long-duration videos. The quadratic complexity of full attention leads to prohibitively high training and inference costs. Full attention inefficiency stems from two key challenges: excessive computation due to the inherent sparsity of Queries and Key-Value pairs, and redundant computation as fixed sparse patterns fail to leverage DiT’s dynamic attention. To overcome this limitation, we propose a Bidirectional Sparse Attention (BSA) framework for faster video DiT training, the first to dynamically sparsify both Queries and Key-Value pairs within 3D full attention, thereby substantially improving training and inference efficiency. BSA addresses these issues through two key components. Query sparsity is optimized by selecting the most informative query tokens via semantic similarity and with a dynamic spatial-time training strategy, while KV sparsity is achieved by computing a statistical dynamic threshold to retain only the most salient KV blocks for computation. Extensive experiments demonstrate that BSA significantly accelerates DiT training across long sequences, reducing FLOPs by up to 20x and achieving 17.79x faster attention training, while preserving or even surpassing the generative quality of full attention.

Method: Proposes VEPE framework with three spatio-temporal Transformer components: STPE, STDME, and STPD, plus an instance consistency mechanism for cross-frame pose query matching and instance tracking.

Result: Outperforms most two-stage models on Posetrack dataset and achieves 300% improvement in inference efficiency.

Conclusion: VEPE provides a simple and flexible end-to-end solution for video pose estimation that effectively utilizes temporal context while significantly improving efficiency.

Abstract: Video-based human pose estimation models aim to address scenarios that cannot be effectively solved by static image models such as motion blur, out-of-focus and occlusion. Most existing approaches consist of two stages: detecting human instances in each image frame and then using a temporal model for single-person pose estimation. This approach separates the spatial and temporal dimensions and cannot capture the global spatio-temporal context between spatial instances for end-to-end optimization. In addition, it relies on separate detectors and complex post-processing such as RoI cropping and NMS, which reduces the inference efficiency of the video scene. To address the above problems, we propose VEPE (Video End-to-End Pose Estimation), a simple and flexible framework for end-to-end pose estimation in video. The framework utilizes three crucial spatio-temporal Transformer components: the Spatio-Temporal Pose Encoder (STPE), the Spatio-Temporal Deformable Memory Encoder (STDME), and the Spatio-Temporal Pose Decoder (STPD). These components are designed to effectively utilize temporal context for optimizing human body pose estimation. Furthermore, to reduce the mismatch problem during the cross-frame pose query matching process, we propose an instance consistency mechanism, which aims to enhance the consistency and discrepancy of the cross-frame instance query and realize the instance tracking function, which in turn accurately guides the pose query to perform cross-frame matching. Extensive experiments on the Posetrack dataset show that our approach outperforms most two-stage models and improves inference efficiency by 300%.

Method: Proposes PVINet with voxel-based encoder for global features and point-based encoder for local contexts, using novel conditional sparse convolution to dynamically customize kernels and enable feature interactions between encoders.

Result: Experiments on benchmark datasets show competitive performance compared to state-of-the-art methods.

Conclusion: PVINet effectively captures both global structural and local contextual features through parallel processing and interactive communication, achieving improved point cloud compression performance.

Abstract: In point cloud compression, the quality of a reconstructed point cloud relies on both the global structure and the local context, with existing methods usually processing global and local information sequentially and lacking communication between these two types of information. In this paper, we propose a point-voxel interlaced network (PVINet), which captures global structural features and local contextual features in parallel and performs interactions at each scale to enhance feature perception efficiency. Specifically, PVINet contains a voxel-based encoder (Ev) for extracting global structural features and a point-based encoder (Ep) that models local contexts centered at each voxel. Particularly, a novel conditional sparse convolution is introduced, which applies point embeddings to dynamically customize kernels for voxel feature extraction, facilitating feature interactions from Ep to Ev. During decoding, a voxel-based decoder employs conditional sparse convolutions to incorporate point embeddings as guidance to reconstruct the point cloud. Experiments on benchmark datasets show that PVINet delivers competitive performance compared to state-of-the-art methods.

Method: Proposes FICGen with: 1) learnable dual-query mechanism with frequency resamplers to extract contextual frequency prototypes, 2) visual-frequency enhanced attention to inject frequency knowledge, 3) instance coherence map for latent-space disentanglement, and 4) adaptive spatial-frequency aggregation module for mixed degraded representations.

Result: Extensive experiments on 5 benchmarks across various degraded scenarios (severe low-light to mild blur) show FICGen consistently outperforms existing L2I methods in generative fidelity, alignment, and downstream auxiliary trainability.

Conclusion: FICGen effectively addresses the contextual illusion problem in degraded image generation by transferring frequency knowledge to latent diffusion space, enabling better rendering of degraded instances and their surroundings through frequency-aware guidance.

Abstract: Layout-to-image (L2I) generation has exhibited promising results in natural domains, but suffers from limited generative fidelity and weak alignment with user-provided layouts when applied to degraded scenes (i.e., low-light, underwater). We primarily attribute these limitations to the “contextual illusion dilemma” in degraded conditions, where foreground instances are overwhelmed by context-dominant frequency distributions. Motivated by this, our paper proposes a new Frequency-Inspired Contextual Disentanglement Generative (FICGen) paradigm, which seeks to transfer frequency knowledge of degraded images into the latent diffusion space, thereby facilitating the rendering of degraded instances and their surroundings via contextual frequency-aware guidance. To be specific, FICGen consists of two major steps. Firstly, we introduce a learnable dual-query mechanism, each paired with a dedicated frequency resampler, to extract contextual frequency prototypes from pre-collected degraded exemplars in the training set. Secondly, a visual-frequency enhanced attention is employed to inject frequency prototypes into the degraded generation process. To alleviate the contextual illusion and attribute leakage, an instance coherence map is developed to regulate latent-space disentanglement between individual instances and their surroundings, coupled with an adaptive spatial-frequency aggregation module to reconstruct spatial-frequency mixed degraded representations. Extensive experiments on 5 benchmarks involving a variety of degraded scenarios-from severe low-light to mild blur-demonstrate that FICGen consistently surpasses existing L2I methods in terms of generative fidelity, alignment and downstream auxiliary trainability.

Method: Uses entropy-driven algorithm to partition images into texture-homogeneous regions, parameterizes each region as 2D Gaussian (mean for position, covariance for shape) with texture features, and employs specialized transformer to optimize Gaussian parameters with differentiable splatting-based renderer.

Result: Achieves rFID and FID scores of 0.65 and 1.50 on image reconstruction and generation tasks using only 128 tokens, demonstrating state-of-the-art performance.

Conclusion: GPSToken provides effective non-uniform image tokenization that disentangles spatial layout from texture features, enabling efficient two-stage generation and superior performance compared to conventional methods.

Abstract: Effective and efficient tokenization plays an important role in image representation and generation. Conventional methods, constrained by uniform 2D/1D grid tokenization, are inflexible to represent regions with varying shapes and textures and at different locations, limiting their efficacy of feature representation. In this work, we propose $\textbf{GPSToken}$, a novel $\textbf{G}$aussian $\textbf{P}$arameterized $\textbf{S}$patially-adaptive $\textbf{Token}$ization framework, to achieve non-uniform image tokenization by leveraging parametric 2D Gaussians to dynamically model the shape, position, and textures of different image regions. We first employ an entropy-driven algorithm to partition the image into texture-homogeneous regions of variable sizes. Then, we parameterize each region as a 2D Gaussian (mean for position, covariance for shape) coupled with texture features. A specialized transformer is trained to optimize the Gaussian parameters, enabling continuous adaptation of position/shape and content-aware feature extraction. During decoding, Gaussian parameterized tokens are reconstructed into 2D feature maps through a differentiable splatting-based renderer, bridging our adaptive tokenization with standard decoders for end-to-end training. GPSToken disentangles spatial layout (Gaussian parameters) from texture features to enable efficient two-stage generation: structural layout synthesis using lightweight networks, followed by structure-conditioned texture generation. Experiments demonstrate the state-of-the-art performance of GPSToken, which achieves rFID and FID scores of 0.65 and 1.50 on image reconstruction and generation tasks using 128 tokens, respectively. Codes and models of GPSToken can be found at $\href{https://github.com/xtudbxk/GPSToken}{https://github.com/xtudbxk/GPSToken}$.

Method: Proposes a spatially heterogeneous loss that splits predictions into four regions (UC, US, DC, DS) with decreasing weights based on uncertainty and consistency. Uses adaptive thresholding to distinguish confident from suspicious predictions and applies the approach to both labeled and unlabeled data.

Result: Experimental results showed significant improvement in segmentation performance when integrated with existing SSL frameworks across different datasets.

Conclusion: MetaSSL provides a plug-and-play universal framework that enhances SSL methods by better leveraging prediction information through spatially adaptive weighting, making it robust to annotation noise and effective for medical image segmentation.

Abstract: Semi-Supervised Learning (SSL) is important for reducing the annotation cost for medical image segmentation models. State-of-the-art SSL methods such as Mean Teacher, FixMatch and Cross Pseudo Supervision (CPS) are mainly based on consistency regularization or pseudo-label supervision between a reference prediction and a supervised prediction. Despite the effectiveness, they have overlooked the potential noise in the labeled data, and mainly focus on strategies to generate the reference prediction, while ignoring the heterogeneous values of different unlabeled pixels. We argue that effectively mining the rich information contained by the two predictions in the loss function, instead of the specific strategy to obtain a reference prediction, is more essential for SSL, and propose a universal framework MetaSSL based on a spatially heterogeneous loss that assigns different weights to pixels by simultaneously leveraging the uncertainty and consistency information between the reference and supervised predictions. Specifically, we split the predictions on unlabeled data into four regions with decreasing weights in the loss: Unanimous and Confident (UC), Unanimous and Suspicious (US), Discrepant and Confident (DC), and Discrepant and Suspicious (DS), where an adaptive threshold is proposed to distinguish confident predictions from suspicious ones. The heterogeneous loss is also applied to labeled images for robust learning considering the potential annotation noise. Our method is plug-and-play and general to most existing SSL methods. The experimental results showed that it improved the segmentation performance significantly when integrated with existing SSL frameworks on different datasets. Code is available at https://github.com/HiLab-git/MetaSSL.

Method: Proposes MVTrajecter with trajectory motion cost and trajectory appearance cost to incorporate motion and appearance information from multiple past timestamps, using attention mechanism to capture relationships between timestamps.

Result: Extensive experiments demonstrate the effectiveness of each component and show that MVTrajecter outperforms previous state-of-the-art methods in multi-view pedestrian tracking.

Conclusion: Utilizing information from multiple past timestamps through trajectory-based costs and attention mechanisms significantly improves pedestrian association robustness in multi-view tracking systems.

Abstract: Multi-View Pedestrian Tracking (MVPT) aims to track pedestrians in the form of a bird’s eye view occupancy map from multi-view videos. End-to-end methods that detect and associate pedestrians within one model have shown great progress in MVPT. The motion and appearance information of pedestrians is important for the association, but previous end-to-end MVPT methods rely only on the current and its single adjacent past timestamp, discarding the past trajectories before that. This paper proposes a novel end-to-end MVPT method called Multi-View Trajectory Tracker (MVTrajecter) that utilizes information from multiple timestamps in past trajectories for robust association. MVTrajecter introduces trajectory motion cost and trajectory appearance cost to effectively incorporate motion and appearance information, respectively. These costs calculate which pedestrians at the current and each past timestamp are likely identical based on the information between those timestamps. Even if a current pedestrian could be associated with a false pedestrian at some past timestamp, these costs enable the model to associate that current pedestrian with the correct past trajectory based on other past timestamps. In addition, MVTrajecter effectively captures the relationships between multiple timestamps leveraging the attention mechanism. Extensive experiments demonstrate the effectiveness of each component in MVTrajecter and show that it outperforms the previous state-of-the-art methods.

Method: Provided empirical evidence through analysis of popular vision encoders’ capability to identify keyframes that MLLMs should focus on for given textual queries.

Result: Found that popular vision encoders critically suffer from limited capability to identify where MLLMs should look inside videos to appropriately handle textual queries.

Conclusion: Development of better keyframe identification techniques is necessary for efficient video multimodal large language models.

Abstract: Recent advances in multimodal large language models (MLLMs) have led to much progress in video understanding tasks. To avoid the heavy computational cost of processing all frames, these models typically rely on keyframe sampling methods guided by vision-language encoders (\textit{e.g.,} SigLIP). However, it remains unclear whether such encoders can truly identify the most informative frames. In this work, we provide several empirical pieces of evidence revealing that popular vision encoders critically suffer from their limited capability to identify where the MLLM should look inside the video to handle the given textual query appropriately. Our findings suggest that the development of better keyframe identification techniques may be necessary for efficient video MLLMs.

Method: Three core modules: 1) Regional-aware Semantic Mapper extracts multimodal semantic features from EEG across brain regions into a unified diffusion prior; 2) Temporal-aware Dynamic Aligner generates dynamic latent sequences for temporal consistency; 3) Dual-Guidance Video Reconstructor translates temporal blueprint into high-fidelity video using semantic guidance.

Result: State-of-the-art performance on SEED-DV dataset: 12.5% and 10.3% improvement in video- and frame-based accuracies respectively, 9.4% SSIM improvement, and 19.7% FVMD reduction, demonstrating exceptional visual fidelity and temporal coherence.

Conclusion: DynaMind represents a critical advancement in bridging neural dynamics with high-fidelity visual semantics, effectively overcoming limitations of previous EEG-based video reconstruction methods through joint modeling of neural dynamics and semantic features.

Abstract: Reconstruction dynamic visual scenes from electroencephalography (EEG) signals remains a primary challenge in brain decoding, limited by the low spatial resolution of EEG, a temporal mismatch between neural recordings and video dynamics, and the insufficient use of semantic information within brain activity. Therefore, existing methods often inadequately resolve both the dynamic coherence and the complex semantic context of the perceived visual stimuli. To overcome these limitations, we introduce DynaMind, a novel framework that reconstructs video by jointly modeling neural dynamics and semantic features via three core modules: a Regional-aware Semantic Mapper (RSM), a Temporal-aware Dynamic Aligner (TDA), and a Dual-Guidance Video Reconstructor (DGVR). The RSM first utilizes a regional-aware encoder to extract multimodal semantic features from EEG signals across distinct brain regions, aggregating them into a unified diffusion prior. In the mean time, the TDA generates a dynamic latent sequence, or blueprint, to enforce temporal consistency between the feature representations and the original neural recordings. Together, guided by the semantic diffusion prior, the DGVR translates the temporal-aware blueprint into a high-fidelity video reconstruction. On the SEED-DV dataset, DynaMind sets a new state-of-the-art (SOTA), boosting reconstructed video accuracies (video- and frame-based) by 12.5 and 10.3 percentage points, respectively. It also achieves a leap in pixel-level quality, showing exceptional visual fidelity and temporal coherence with a 9.4% SSIM improvement and a 19.7% FVMD reduction. This marks a critical advancement, bridging the gap between neural dynamics and high-fidelity visual semantics.

Method: Adaptively identifies focus regions based on dynamic semantic correspondence and supervision image complexity, progressively adjusting focal areas across noise timesteps with a weighted strategy that rewards information-rich patches and penalizes low-confidence regions.

Result: Substantially enhances performance of existing pre-trained personalized generation models, achieving state-of-the-art results on both single-subject and multi-subject benchmarks while effectively mitigating attribute leakage.

Conclusion: The framework advances controllable multi-subject image synthesis by dynamically adjusting focus allocation and establishing robust correspondence mappings between generated and reference subjects.

Abstract: Multi-subject personalized image generation aims to synthesize customized images containing multiple specified subjects without requiring test-time optimization. However, achieving fine-grained independent control over multiple subjects remains challenging due to difficulties in preserving subject fidelity and preventing cross-subject attribute leakage. We present FocusDPO, a framework that adaptively identifies focus regions based on dynamic semantic correspondence and supervision image complexity. During training, our method progressively adjusts these focal areas across noise timesteps, implementing a weighted strategy that rewards information-rich patches while penalizing regions with low prediction confidence. The framework dynamically adjusts focus allocation during the DPO process according to the semantic complexity of reference images and establishes robust correspondence mappings between generated and reference subjects. Extensive experiments demonstrate that our method substantially enhances the performance of existing pre-trained personalized generation models, achieving state-of-the-art results on both single-subject and multi-subject personalized image synthesis benchmarks. Our method effectively mitigates attribute leakage while preserving superior subject fidelity across diverse generation scenarios, advancing the frontier of controllable multi-subject image synthesis.

Method: Formulates SQA as a four-class panoramic segmentation task (TP/FP/TN/FN) using enhanced SAM architecture with mask prompting, Edge Guided Compaction branch with ASF module, and Augmented Mixup Sampling training strategy.

Result: Achieves state-of-the-art performance across 32 datasets from 6 sources and demonstrates remarkable zero-shot transferability to unseen masks.

Conclusion: SegAssess establishes PQM as a robust and transferable solution for unsupervised segmentation quality assessment, providing comprehensive pixel-level evaluation.

Abstract: High-quality image segmentation is fundamental to pixel-level geospatial analysis in remote sensing, necessitating robust segmentation quality assessment (SQA), particularly in unsupervised settings lacking ground truth. Although recent deep learning (DL) based unsupervised SQA methods show potential, they often suffer from coarse evaluation granularity, incomplete assessments, and poor transferability. To overcome these limitations, this paper introduces Panoramic Quality Mapping (PQM) as a new paradigm for comprehensive, pixel-wise SQA, and presents SegAssess, a novel deep learning framework realizing this approach. SegAssess distinctively formulates SQA as a fine-grained, four-class panoramic segmentation task, classifying pixels within a segmentation mask under evaluation into true positive (TP), false positive (FP), true negative (TN), and false negative (FN) categories, thereby generating a complete quality map. Leveraging an enhanced Segment Anything Model (SAM) architecture, SegAssess uniquely employs the input mask as a prompt for effective feature integration via cross-attention. Key innovations include an Edge Guided Compaction (EGC) branch with an Aggregated Semantic Filter (ASF) module to refine predictions near challenging object edges, and an Augmented Mixup Sampling (AMS) training strategy integrating multi-source masks to significantly boost cross-domain robustness and zero-shot transferability. Comprehensive experiments across 32 datasets derived from 6 sources demonstrate that SegAssess achieves state-of-the-art (SOTA) performance and exhibits remarkable zero-shot transferability to unseen masks, establishing PQM via SegAssess as a robust and transferable solution for unsupervised SQA. The code is available at https://github.com/Yangbn97/SegAssess.

Method: Created a dataset with 3,141,568 images combining 0.5m LiDAR-derived canopy height maps and multi-temporal multi-spectral imagery. Proposed an encoder-decoder framework using U-Net architecture that processes multi-temporal imagery and relative time differences to predict canopy height.

Result: U-Net architecture achieved the highest performance with 11.78% masked mean squared error, outperforming ResNet-50 by ~12% and RGB-only experiments by ~30%.

Conclusion: PrediTree enables effective training of deep learning models for tree height prediction and provides a valuable open-source resource for forest monitoring research, with the dataset and code publicly available.

Abstract: We present PrediTree, the first comprehensive open-source dataset designed for training and evaluating tree height prediction models at sub-meter resolution. This dataset combines very high-resolution (0.5m) LiDAR-derived canopy height maps, spatially aligned with multi-temporal and multi-spectral imagery, across diverse forest ecosystems in France, totaling 3,141,568 images. PrediTree addresses a critical gap in forest monitoring capabilities by enabling the training of deep learning methods that can predict tree growth based on multiple past observations. %\sout{Initially focused on French forests, PrediTree is designed as an expanding resource with ongoing efforts to incorporate data from other countries. } To make use of this PrediTree dataset, we propose an encoder-decoder framework that requires the multi-temporal multi-spectral imagery and the relative time differences in years between the canopy height map timestamp (target) and each image acquisition date for which this framework predicts the canopy height. The conducted experiments demonstrate that a U-Net architecture trained on the PrediTree dataset provides the highest masked mean squared error of $11.78%$, outperforming the next-best architecture, ResNet-50, by around $12%$, and cutting the error of the same experiments but on fewer bands (red, green, blue only), by around $30%$. This dataset is publicly available on \href{URL}{HuggingFace}, and both processing and training codebases are available on \href{URL}{GitHub}.

Method: Uses shape graph attention network to capture manifold structure, constructs shared latent space, employs universe predictor for consistent correspondences, and enforces dual-level consistency through spatial/spectral domain alignment with cycle consistency loss.

Result: Extensive experiments show consistent outperformance of previous state-of-the-art approaches across diverse multi-shape matching scenarios.

Conclusion: DcMatch provides a novel unsupervised framework that effectively handles non-rigid multi-shape matching through manifold-aware learning and dual-domain consistency, achieving superior performance.

Abstract: Establishing point-to-point correspondences across multiple 3D shapes is a fundamental problem in computer vision and graphics. In this paper, we introduce DcMatch, a novel unsupervised learning framework for non-rigid multi-shape matching. Unlike existing methods that learn a canonical embedding from a single shape, our approach leverages a shape graph attention network to capture the underlying manifold structure of the entire shape collection. This enables the construction of a more expressive and robust shared latent space, leading to more consistent shape-to-universe correspondences via a universe predictor. Simultaneously, we represent these correspondences in both the spatial and spectral domains and enforce their alignment in the shared universe space through a novel cycle consistency loss. This dual-level consistency fosters more accurate and coherent mappings. Extensive experiments on several challenging benchmarks demonstrate that our method consistently outperforms previous state-of-the-art approaches across diverse multi-shape matching scenarios. Code is available at https://github.com/YeTianwei/DcMatch.

Method: Proposes a novel block-wise mixture of dynamic low-rank experts that adaptively selects different experts based on input features, using small trainable parameters. Also introduces a self-supervised training framework to handle brightness and reflectance inconsistencies.

Result: Outperforms state-of-the-art methods on both realistic and simulated endoscopic datasets, achieving best generalization through zero-shot depth estimation on diverse endoscopic scenes.

Conclusion: The proposed framework enables accurate endoscopic perception for minimally invasive measurement and surgery, with code to be released upon acceptance.

Abstract: Self-supervised monocular depth estimation is a significant task for low-cost and efficient three-dimensional scene perception in endoscopy. The variety of illumination conditions and scene features is still the primary challenge for generalizable depth estimation in endoscopic scenes. In this work, a self-supervised framework is proposed for monocular depth estimation in various endoscopy. Firstly, due to various features in endoscopic scenes with different tissues, a novel block-wise mixture of dynamic low-rank experts is proposed to efficiently finetuning the foundation model for endoscopic depth estimation. In the proposed module, based on the input feature, different experts with a small amount of trainable parameters are adaptively selected for weighted inference, from various mixture of low-rank experts which are allocated based on the training quality of each block. Moreover, a novel self-supervised training framework is proposed to jointly cope with the inconsistency of brightness and reflectance. The proposed method outperform state-of-the-art works on both realistic and simulated endoscopic datasets. Furthermore, the proposed network also achieves the best generalization based on zero-shot depth estimation on diverse endoscopic scenes. The proposed method could contribute to accurate endoscopic perception for minimally invasive measurement and surgery. The code will be released upon acceptance, while the demo video can be found on here: https://endo-gede.netlify.app/.

Method: Develops a reference-free evaluation metric for open-vocabulary relation prediction and creates a region-specific prompt tuning approach for VLMs to generate high-quality synthetic training data.

Result: Experimental results demonstrate that pre-training with the new synthetic data improves the generalization capabilities of open-vocabulary SGG models.

Conclusion: The proposed metric enables fair evaluation of open-vocabulary capabilities, and the synthetic data generation method effectively enhances model performance for scene graph generation tasks.

Abstract: Scene Graph Generation (SGG) encodes visual relationships between objects in images as graph structures. Thanks to the advances of Vision-Language Models (VLMs), the task of Open-Vocabulary SGG has been recently proposed where models are evaluated on their functionality to learn a wide and diverse range of relations. Current benchmarks in SGG, however, possess a very limited vocabulary, making the evaluation of open-source models inefficient. In this paper, we propose a new reference-free metric to fairly evaluate the open-vocabulary capabilities of VLMs for relation prediction. Another limitation of Open-Vocabulary SGG is the reliance on weakly supervised data of poor quality for pre-training. We also propose a new solution for quickly generating high-quality synthetic data through region-specific prompt tuning of VLMs. Experimental results show that pre-training with this new data split can benefit the generalization capabilities of Open-Voc SGG models.

Method: Two-stage framework: 1) Generate large-scale synthetic data for initial model training, 2) Self-improvement approach using fine-tuned model to annotate real documents with quality filtering, then iterative retraining.

Result: Trained POINTS-1.5 model achieves POINTS-Reader that surpasses many existing public and proprietary models of comparable or larger size.

Conclusion: The framework enables automated creation of high-quality document extraction datasets and models capable of handling diverse document formats without manual annotation, achieving state-of-the-art performance.

Abstract: High-quality labeled data is essential for training accurate document conversion models, particularly in domains with complex formats such as tables, formulas, and multi-column text. However, manual annotation is both costly and time-consuming, while automatic labeling using existing models often lacks accuracy in handling such challenging scenarios. Consequently, training student models by distilling outputs from teacher models can significantly limit their performance in real-world applications. In this paper, we propose a fully automated, distillation-free framework comprising two stages for constructing high-quality document extraction datasets and models capable of handling diverse document formats and layouts. In the first stage, we introduce a method for generating large-scale, diverse synthetic data, which enables a model to extract key elements in a unified format with strong initial performance. In the second stage, we present a self-improvement approach that further adapts the model, initially trained on synthetic data, to real-world documents. Specifically, we first use the fine-tuned model to annotate real documents, then apply a suite of filtering strategies to verify annotation quality, and finally retrain the model on the verified dataset. By iteratively repeating this process, we progressively enhance both the model’s conversion capabilities and the quality of the generated data. We train a public POINTS-1.5 model to obtain POINTS-Reader, which surpasses many existing public and proprietary models of comparable or larger size. Our model is available at https://github.com/Tencent/POINTS-Reader.

Method: Models interaction as a dynamic directed graph with a collaborative multi-agent system: scene navigator for environmental perception and path planning, planning agent for goal decomposition, and critic agent for closed-loop feedback correction. Uses Direct Preference Optimization for action generation to enhance physical realism.

Result: Extensive experiments on SceneBench benchmark show FantasyHSI significantly outperforms existing methods in generalization, long-horizon task completion, and physical realism, reducing artifacts like limb distortion and foot-sliding.

Conclusion: FantasyHSI provides an effective framework for realistic human-scene interaction through multi-agent collaboration and feedback mechanisms, demonstrating superior performance in complex, unseen environments.

Abstract: Human-Scene Interaction (HSI) seeks to generate realistic human behaviors within complex environments, yet it faces significant challenges in handling long-horizon, high-level tasks and generalizing to unseen scenes. To address these limitations, we introduce FantasyHSI, a novel HSI framework centered on video generation and multi-agent systems that operates without paired data. We model the complex interaction process as a dynamic directed graph, upon which we build a collaborative multi-agent system. This system comprises a scene navigator agent for environmental perception and high-level path planning, and a planning agent that decomposes long-horizon goals into atomic actions. Critically, we introduce a critic agent that establishes a closed-loop feedback mechanism by evaluating the deviation between generated actions and the planned path. This allows for the dynamic correction of trajectory drifts caused by the stochasticity of the generative model, thereby ensuring long-term logical consistency. To enhance the physical realism of the generated motions, we leverage Direct Preference Optimization (DPO) to train the action generator, significantly reducing artifacts such as limb distortion and foot-sliding. Extensive experiments on our custom SceneBench benchmark demonstrate that FantasyHSI significantly outperforms existing methods in terms of generalization, long-horizon task completion, and physical realism. Ours project page: https://fantasy-amap.github.io/fantasy-hsi/

Method: The authors use a series of eight carefully designed illustrations to explain the architecture, moving from the overall pipeline down to critical components including encoder, decoder, and multi-scale deformable attention.

Result: The paper provides visualizations of tensor flow and unpacks the logic behind each module to create a clearer mental model of RT-DETRv2’s internal workings.

Conclusion: The goal is to make RT-DETRv2 genuinely understandable for researchers and practitioners by providing comprehensive visual explanations of its architecture and component interactions.

Abstract: Object detection architectures are notoriously difficult to understand, often more so than large language models. While RT-DETRv2 represents an important advance in real-time detection, most existing diagrams do little to clarify how its components actually work and fit together. In this article, we explain the architecture of RT-DETRv2 through a series of eight carefully designed illustrations, moving from the overall pipeline down to critical components such as the encoder, decoder, and multi-scale deformable attention. Our goal is to make the existing one genuinely understandable. By visualizing the flow of tensors and unpacking the logic behind each module, we hope to provide researchers and practitioners with a clearer mental model of how RT-DETRv2 works under the hood.

Method: Introduces aleatoric uncertainty modeling by formulating hand joint output as a probabilistic distribution and using a single linear layer to capture intrinsic correlations among hand joints. This parameterization serves as an add-on module that can be applied to existing models.

Result: The proposed parameterization for uncertainty modeling outperforms existing approaches, and the 3D hand pose estimation model equipped with this uncertainty head achieves favorable accuracy while adding new uncertainty modeling capability.

Conclusion: The method successfully addresses limitations in existing 3D hand pose estimation by incorporating aleatoric uncertainty modeling with joint correlation knowledge, achieving a good balance between modeling accuracy and computational efficiency.

Abstract: 3D hand pose estimation is a fundamental task in understanding human hands. However, accurately estimating 3D hand poses remains challenging due to the complex movement of hands, self-similarity, and frequent occlusions. In this work, we address two limitations: the inability of existing 3D hand pose estimation methods to estimate aleatoric (data) uncertainty, and the lack of uncertainty modeling that incorporates joint correlation knowledge, which has not been thoroughly investigated. To this end, we introduce aleatoric uncertainty modeling into the 3D hand pose estimation framework, aiming to achieve a better trade-off between modeling joint correlations and computational efficiency. We propose a novel parameterization that leverages a single linear layer to capture intrinsic correlations among hand joints. This is enabled by formulating the hand joint output space as a probabilistic distribution, allowing the linear layer to capture joint correlations. Our proposed parameterization is used as a task head layer, and can be applied as an add-on module on top of the existing models. Our experiments demonstrate that our parameterization for uncertainty modeling outperforms existing approaches. Furthermore, the 3D hand pose estimation model equipped with our uncertainty head achieves favorable accuracy in 3D hand pose estimation while introducing new uncertainty modeling capability to the model. The project page is available at https://hand-uncertainty.github.io/.

Method: Proposes Point-PQAE with a cross-reconstruction generative paradigm that generates two decoupled point cloud views and reconstructs one from the other. Introduces a crop mechanism for view generation and a novel positional encoding to represent 3D relative position between views.

Result: Outperforms self-reconstruction baseline (Point-MAE) by 6.5%, 7.0%, and 6.7% in three variants of ScanObjectNN with Mlp-Linear evaluation protocol.

Conclusion: Cross-reconstruction significantly increases pre-training difficulty compared to self-reconstruction, enabling superior performance in 3D self-supervised learning over single-modal self-reconstruction methods.

Abstract: Point cloud learning, especially in a self-supervised way without manual labels, has gained growing attention in both vision and learning communities due to its potential utility in a wide range of applications. Most existing generative approaches for point cloud self-supervised learning focus on recovering masked points from visible ones within a single view. Recognizing that a two-view pre-training paradigm inherently introduces greater diversity and variance, it may thus enable more challenging and informative pre-training. Inspired by this, we explore the potential of two-view learning in this domain. In this paper, we propose Point-PQAE, a cross-reconstruction generative paradigm that first generates two decoupled point clouds/views and then reconstructs one from the other. To achieve this goal, we develop a crop mechanism for point cloud view generation for the first time and further propose a novel positional encoding to represent the 3D relative position between the two decoupled views. The cross-reconstruction significantly increases the difficulty of pre-training compared to self-reconstruction, which enables our method to surpass previous single-modal self-reconstruction methods in 3D self-supervised learning. Specifically, it outperforms the self-reconstruction baseline (Point-MAE) by 6.5%, 7.0%, and 6.7% in three variants of ScanObjectNN with the Mlp-Linear evaluation protocol. The code is available at https://github.com/aHapBean/Point-PQAE.

Method: Three-component pipeline: (1) two-stage article retrieval using DINOv2 embeddings with global feature similarity and patch-level mutual nearest neighbor re-ranking, (2) context extraction framework synthesizing article summaries, generic captions, and source metadata, (3) LLM-based caption generation with Semantic Gaussian Normalization.

Result: Achieved overall score of 0.54666 on OpenEvents V1 dataset, ranking 2nd on the private test set in EVENTA 2025 Grand Challenge Track 1.

Conclusion: ReCap effectively bridges visual perception with real-world knowledge, offering a practical solution for context-aware image understanding in high-stakes domains.

Abstract: Image captioning systems often produce generic descriptions that fail to capture event-level semantics which are crucial for applications like news reporting and digital archiving. We present ReCap, a novel pipeline for event-enriched image retrieval and captioning that incorporates broader contextual information from relevant articles to generate narrative-rich, factually grounded captions. Our approach addresses the limitations of standard vision-language models that typically focus on visible content while missing temporal, social, and historical contexts. ReCap comprises three integrated components: (1) a robust two-stage article retrieval system using DINOv2 embeddings with global feature similarity for initial candidate selection followed by patch-level mutual nearest neighbor similarity re-ranking; (2) a context extraction framework that synthesizes information from article summaries, generic captions, and original source metadata; and (3) a large language model-based caption generation system with Semantic Gaussian Normalization to enhance fluency and relevance. Evaluated on the OpenEvents V1 dataset as part of Track 1 in the EVENTA 2025 Grand Challenge, ReCap achieved a strong overall score of 0.54666, ranking 2nd on the private test set. These results highlight ReCap’s effectiveness in bridging visual perception with real-world knowledge, offering a practical solution for context-aware image understanding in high-stakes domains. The code is available at https://github.com/Noridom1/EVENTA2025-Event-Enriched-Image-Captioning.

Method: Proposes X-Agent framework with latent semantic-aware agents that orchestrate cross-modal attention mechanisms to optimize latent semantic dynamics and amplify perceptibility.

Result: Achieves state-of-the-art performance in benchmark evaluations while effectively enhancing latent semantic saliency.

Conclusion: X-Agent successfully addresses the bottleneck in OVSS by improving latent semantic comprehension through innovative agent-based cross-modal attention optimization.

Abstract: Open-vocabulary semantic segmentation (OVSS) conducts pixel-level classification via text-driven alignment, where the domain discrepancy between base category training and open-vocabulary inference poses challenges in discriminative modeling of latent unseen category. To address this challenge, existing vision-language model (VLM)-based approaches demonstrate commendable performance through pre-trained multi-modal representations. However, the fundamental mechanisms of latent semantic comprehension remain underexplored, making the bottleneck for OVSS. In this work, we initiate a probing experiment to explore distribution patterns and dynamics of latent semantics in VLMs under inductive learning paradigms. Building on these insights, we propose X-Agent, an innovative OVSS framework employing latent semantic-aware ``agent’’ to orchestrate cross-modal attention mechanisms, simultaneously optimizing latent semantic dynamic and amplifying its perceptibility. Extensive benchmark evaluations demonstrate that X-Agent achieves state-of-the-art performance while effectively enhancing the latent semantic saliency.

Method: Employed Neural Architecture Search (NAS) to systematically optimize YOLOv10’s network structure, with focus on backbone architecture search. Used evolutionary search within an extensive search space to identify optimal architecture balancing accuracy, parameter efficiency, and computational cost.

Result: Experimental results on the large-scale SARDet-100K dataset show the optimized model outperforms existing SAR detection methods, achieving superior detection accuracy while maintaining lower computational overhead.

Conclusion: This work introduces NAS to SAR object detection for the first time and demonstrates its effectiveness in optimizing lightweight detectors for real-world applications, offering a novel perspective on leveraging NAS in practical scenarios.

Abstract: Synthetic Aperture Radar (SAR) object detection faces significant challenges from speckle noise, small target ambiguities, and on-board computational constraints. While existing approaches predominantly focus on SAR-specific architectural modifications, this paper explores the application of the existing lightweight object detector, i.e., YOLOv10, for SAR object detection and enhances its performance through Neural Architecture Search (NAS). Specifically, we employ NAS to systematically optimize the network structure, especially focusing on the backbone architecture search. By constructing an extensive search space and leveraging evolutionary search, our method identifies a favorable architecture that balances accuracy, parameter efficiency, and computational cost. Notably, this work introduces NAS to SAR object detection for the first time. The experimental results on the large-scale SARDet-100K dataset demonstrate that our optimized model outperforms existing SAR detection methods, achieving superior detection accuracy while maintaining lower computational overhead. We hope this work offers a novel perspective on leveraging NAS for real-world applications.

Method: Uses multi-representation RD radar maps (heatmaps + grayscale images), designs Adapter branch, Exchanger Module, and Primary-Auxiliary Fusion Module for feature extraction and fusion, and employs One-Shot Neural Architecture Search for efficiency optimization.

Result: Achieves mAP@50 of 71.9 on RADDet and 57.1 on CARRADA datasets, demonstrating favorable accuracy-efficiency trade-off and state-of-the-art performance.

Conclusion: The proposed model effectively combines multi-representation radar data with optimized architecture search to deliver efficient and high-performance object detection for radar applications.

Abstract: Detecting objects efficiently from radar sensors has recently become a popular trend due to their robustness against adverse lighting and weather conditions compared with cameras. This paper presents an efficient object detection model for Range-Doppler (RD) radar maps. Specifically, we first represent RD radar maps with multi-representation, i.e., heatmaps and grayscale images, to gather high-level object and fine-grained texture features. Then, we design an additional Adapter branch, an Exchanger Module with two modes, and a Primary-Auxiliary Fusion Module to effectively extract, exchange, and fuse features from the multi-representation inputs, respectively. Furthermore, we construct a supernet with various width and fusion operations in the Adapter branch for the proposed model and employ a One-Shot Neural Architecture Search method to further improve the model’s efficiency while maintaining high performance. Experimental results demonstrate that our model obtains favorable accuracy and efficiency trade-off. Moreover, we achieve new state-of-the-art performance on RADDet and CARRADA datasets with mAP@50 of 71.9 and 57.1, respectively.

Method: Uses ordinary differential equations and Fourier transform to model relationship between domain-specific and domain-agnostic features. Applies iterative transformation with randomized perturbations to spectra, reformulating as ODE optimization problem.

Result: Experimental results on five diverse datasets show FSS-TIs outperforms existing CD-FSS methods. Ablation studies validate cross-domain adaptability.

Conclusion: FSS-TIs provides a structurally concise and effective solution for cross-domain few-shot segmentation, demonstrating superior generalization across substantially different domains.

Abstract: Cross-domain few-shot segmentation (CD-FSS) not only enables the segmentation of unseen categories with very limited samples, but also improves cross-domain generalization ability within the few-shot segmentation framework. Currently, existing CD-FSS studies typically design multiple independent modules to enhance the cross-domain generalization ability of feature representations. However, the independence among these modules hinders the effective flow of knowledge, making it difficult to fully leverage their collective potential. In contrast, this paper proposes an all-in-one module based on ordinary differential equations and Fourier transform, resulting in a structurally concise method–Few-Shot Segmentation over Time Intervals (FSS-TIs). FSS-TIs assumes the existence of an ODE relationship between the spectra (including amplitude and phase spectra) of domain-specific features and domain-agnostic features. This ODE formulation yields an iterative transformation process along a sequence of time intervals, while simultaneously applying affine transformations with randomized perturbations to the spectra. In doing so, the exploration of domain-agnostic feature representation spaces and the simulation of diverse potential target-domain distributions are reformulated as an optimization process over the intrinsic parameters of the ODE. Moreover, we strictly constrain the support-sample selection during target-domain fine-tuning so that it is consistent with the requirements of real-world few-shot segmentation tasks. For evaluation, we introduce five datasets from substantially different domains and define two sets of cross-domain few-shot segmentation tasks to comprehensively analyze the performance of FSS-TIs. Experimental results demonstrate the superiority of FSS-TIs over existing CD-FSS methods, and in-depth ablation studies further validate the cross-domain adaptability of FSS-TIs.

Method: Joint optimization framework with SR module that incorporates semantic cues, new SR loss function focusing on regions of interest, and lightweight model-based architecture with fewer parameters.

Result: Achieves segmentation performance comparable to models using high-resolution 64-channel LiDAR data.

Conclusion: Proposed framework enables cost-effective autonomous driving deployment by effectively enhancing sparse LiDAR data through joint super-resolution and semantic segmentation optimization.

Abstract: High-resolution LiDAR data plays a critical role in 3D semantic segmentation for autonomous driving, but the high cost of advanced sensors limits large-scale deployment. In contrast, low-cost sensors such as 16-channel LiDAR produce sparse point clouds that degrade segmentation accuracy. To overcome this, we introduce the first end-to-end framework that jointly addresses LiDAR super-resolution (SR) and semantic segmentation. The framework employs joint optimization during training, allowing the SR module to incorporate semantic cues and preserve fine details, particularly for smaller object classes. A new SR loss function further directs the network to focus on regions of interest. The proposed lightweight, model-based SR architecture uses significantly fewer parameters than existing LiDAR SR approaches, while remaining easily compatible with segmentation networks. Experiments show that our method achieves segmentation performance comparable to models operating on high-resolution and costly 64-channel LiDAR data.

Method: PGRD embeds discrete labels in continuous space, uses a coarse prior predictor for step-wise guidance, learns residual to the prior for faster convergence, and employs deep diffusion supervision to stabilize training by supervising intermediate time steps.

Result: PGRD achieves higher Dice scores and lower NLL/ECE values compared to Bayesian, ensemble, Probabilistic U-Net, and vanilla diffusion baselines on MRI and CT datasets, while requiring fewer sampling steps for strong performance.

Conclusion: The proposed Prior-Guided Residual Diffusion framework effectively captures conditional distributions in medical image segmentation with improved calibration, accuracy, and computational efficiency compared to existing probabilistic methods.

Abstract: Ambiguity in medical image segmentation calls for models that capture full conditional distributions rather than a single point estimate. We present Prior-Guided Residual Diffusion (PGRD), a diffusion-based framework that learns voxel-wise distributions while maintaining strong calibration and practical sampling efficiency. PGRD embeds discrete labels as one-hot targets in a continuous space to align segmentation with diffusion modeling. A coarse prior predictor provides step-wise guidance; the diffusion network then learns the residual to the prior, accelerating convergence and improving calibration. A deep diffusion supervision scheme further stabilizes training by supervising intermediate time steps. Evaluated on representative MRI and CT datasets, PGRD achieves higher Dice scores and lower NLL/ECE values than Bayesian, ensemble, Probabilistic U-Net, and vanilla diffusion baselines, while requiring fewer sampling steps to reach strong performance.

Method: Integrates open-weight multimodal LLMs with retrieval-augmented generation. Uses SigLIP encoder to build vector database from EMP-16 and OSV-5M datasets. Augments query images with retrieved similar/dissimilar geolocation prompts before processing.

Result: Achieved state-of-the-art performance with higher accuracy on benchmark datasets (IM2GPS, IM2GPS3k, YFCC4k). Eliminates need for expensive fine-tuning and scales seamlessly with new data.

Conclusion: Retrieval-augmented generation with multimodal LLMs provides an effective alternative to traditional training-from-scratch methods, enabling more accessible and scalable geolocation solutions in GeoAI.

Abstract: Street-level geolocalization from images is crucial for a wide range of essential applications and services, such as navigation, location-based recommendations, and urban planning. With the growing popularity of social media data and cameras embedded in smartphones, applying traditional computer vision techniques to localize images has become increasingly challenging, yet highly valuable. This paper introduces a novel approach that integrates open-weight and publicly accessible multimodal large language models with retrieval-augmented generation. The method constructs a vector database using the SigLIP encoder on two large-scale datasets (EMP-16 and OSV-5M). Query images are augmented with prompts containing both similar and dissimilar geolocation information retrieved from this database before being processed by the multimodal large language models. Our approach has demonstrated state-of-the-art performance, achieving higher accuracy compared against three widely used benchmark datasets (IM2GPS, IM2GPS3k, and YFCC4k). Importantly, our solution eliminates the need for expensive fine-tuning or retraining and scales seamlessly to incorporate new data sources. The effectiveness of retrieval-augmented generation-based multimodal large language models in geolocation estimation demonstrated by this paper suggests an alternative path to the traditional methods which rely on the training models from scratch, opening new possibilities for more accessible and scalable solutions in GeoAI.

Method: Uses multimodal deep learning with two modules: Soil Classification Module (ResNet-18, EfficientNet-B0, Vision Transformer for soil type categorization from images) and Crop Recommendation Module (MLP, XGBoost, LightGBM, TabNet for analyzing structured soil data including nutrients, pH, and rainfall).

Result: Achieved 98.0% accuracy, 97.8% precision, 97.7% recall, 96.75% F1-score, with RMSE of 0.32 and MAE of 0.27. Ablation studies confirmed importance of multimodal coupling, and statistical validation showed significant improvements.

Conclusion: AgroSense provides a practical, scalable solution for real-time decision support in precision agriculture and enables future lightweight multimodal AI systems for resource-constrained environments.

Abstract: Meeting the increasing global demand for food security and sustainable farming requires intelligent crop recommendation systems that operate in real time. Traditional soil analysis techniques are often slow, labor-intensive, and not suitable for on-field decision-making. To address these limitations, we introduce AgroSense, a deep-learning framework that integrates soil image classification and nutrient profiling to produce accurate and contextually relevant crop recommendations. AgroSense comprises two main components: a Soil Classification Module, which leverages ResNet-18, EfficientNet-B0, and Vision Transformer architectures to categorize soil types from images; and a Crop Recommendation Module, which employs a Multi-Layer Perceptron, XGBoost, LightGBM, and TabNet to analyze structured soil data, including nutrient levels, pH, and rainfall. We curated a multimodal dataset of 10,000 paired samples drawn from publicly available Kaggle repositories, approximately 50,000 soil images across seven classes, and 25,000 nutrient profiles for experimental evaluation. The fused model achieves 98.0% accuracy, with a precision of 97.8%, a recall of 97.7%, and an F1-score of 96.75%, while RMSE and MAE drop to 0.32 and 0.27, respectively. Ablation studies underscore the critical role of multimodal coupling, and statistical validation via t-tests and ANOVA confirms the significance of our improvements. AgroSense offers a practical, scalable solution for real-time decision support in precision agriculture and paves the way for future lightweight multimodal AI systems in resource-constrained environments.

Method: Curated a large-scale hybrid-modality dataset (867,653 samples including 2D X-rays, ultrasounds, endoscopy videos, and 3D CT scans) and trained M3Ret using both generative (MAE) and contrastive (SimDINO) self-supervised learning paradigms without modality-specific customization.

Result: Achieved new state-of-the-art in zero-shot image-to-image retrieval across all modalities, surpassing DINOv3 and BMC-CLIP. Demonstrated strong cross-modal alignment without paired data and generalization to unseen MRI tasks despite no MRI in pretraining.

Conclusion: M3Ret represents a step toward foundation models for visual self-supervised learning in multimodal medical imaging, showing promising scalability and generalizability across modalities and data sizes.

Abstract: Medical image retrieval is essential for clinical decision-making and translational research, relying on discriminative visual representations. Yet, current methods remain fragmented, relying on separate architectures and training strategies for 2D, 3D, and video-based medical data. This modality-specific design hampers scalability and inhibits the development of unified representations. To enable unified learning, we curate a large-scale hybrid-modality dataset comprising 867,653 medical imaging samples, including 2D X-rays and ultrasounds, RGB endoscopy videos, and 3D CT scans. Leveraging this dataset, we train M3Ret, a unified visual encoder without any modality-specific customization. It successfully learns transferable representations using both generative (MAE) and contrastive (SimDINO) self-supervised learning (SSL) paradigms. Our approach sets a new state-of-the-art in zero-shot image-to-image retrieval across all individual modalities, surpassing strong baselines such as DINOv3 and the text-supervised BMC-CLIP. More remarkably, strong cross-modal alignment emerges without paired data, and the model generalizes to unseen MRI tasks, despite never observing MRI during pretraining, demonstrating the generalizability of purely visual self-supervision to unseen modalities. Comprehensive analyses further validate the scalability of our framework across model and data sizes. These findings deliver a promising signal to the medical imaging community, positioning M3Ret as a step toward foundation models for visual SSL in multimodal medical image understanding.

Method: Uirapuru incorporates comprehensive camera actuation understanding into system design and uses fast adaptive tiling at per-frame level to handle the dynamism from steerable camera movements.

Result: Uirapuru provides up to 1.45x accuracy improvement while meeting latency budgets, or achieves up to 4.53x inference speedup with comparable accuracy compared to state-of-the-art static camera approaches.

Conclusion: The framework successfully addresses the challenges of real-time video analytics on steerable cameras by integrating camera actuation awareness with adaptive tiling techniques.

Abstract: Real-time video analytics on high-resolution cameras has become a popular technology for various intelligent services like traffic control and crowd monitoring. While extensive work has been done on improving analytics accuracy with timing guarantees, virtually all of them target static viewpoint cameras. In this paper, we present Uirapuru, a novel framework for real-time, edge-based video analytics on high-resolution steerable cameras. The actuation performed by those cameras brings significant dynamism to the scene, presenting a critical challenge to existing popular approaches such as frame tiling. To address this problem, Uirapuru incorporates a comprehensive understanding of camera actuation into the system design paired with fast adaptive tiling at a per-frame level. We evaluate Uirapuru on a high-resolution video dataset, augmented by pan-tilt-zoom (PTZ) movements typical for steerable cameras and on real-world videos collected from an actual PTZ camera. Our experimental results show that Uirapuru provides up to 1.45x improvement in accuracy while respecting specified latency budgets or reaches up to 4.53x inference speedup with on-par accuracy compared to state-of-the-art static camera approaches.

Method: Proposes U3D dataset (first unsupervised UHR UAV dataset), Edge Efficiency Index metric, and U3LIE framework with Adaptive Pre-enhancement Augmentation and Luminance Interval Loss for efficient training.

Result: U3LIE achieves state-of-the-art results, processing 4K images at 23.8 FPS on a single GPU, enabling real-time on-board deployment.

Conclusion: Provides a holistic solution (dataset, metric, method) for advancing robust 24/7 UAV vision with available code and datasets.

Abstract: Low light conditions significantly degrade Unmanned Aerial Vehicles (UAVs) performance in critical applications. Existing Low-light Image Enhancement (LIE) methods struggle with the unique challenges of aerial imagery, including Ultra-High Resolution (UHR), lack of paired data, severe non-uniform illumination, and deployment constraints. To address these issues, we propose three key contributions. First, we present U3D, the first unsupervised UHR UAV dataset for LIE, with a unified evaluation toolkit. Second, we introduce the Edge Efficiency Index (EEI), a novel metric balancing perceptual quality with key deployment factors: speed, resolution, model complexity, and memory footprint. Third, we develop U3LIE, an efficient framework with two training-only designs-Adaptive Pre-enhancement Augmentation (APA) for input normalization and a Luminance Interval Loss (L_int) for exposure control. U3LIE achieves SOTA results, processing 4K images at 23.8 FPS on a single GPU, making it ideal for real-time on-board deployment. In summary, these contributions provide a holistic solution (dataset, metric, and method) for advancing robust 24/7 UAV vision. The code and datasets are available at https://github.com/lwCVer/U3D_Toolkit.

Method: Utilizes neural occupancy fields to predict whether 3D points lie inside or outside ribcage objects from CT scans. Applies Laplacian-based contraction to extract the medial axis of the ribcage, demonstrating geometric operations that benefit from continuous coordinate-based representations.

Result: Methodology evaluated on 20 medical scans from the RibSeg dataset (extension of RibFrac dataset). The approach shows advantages over traditional voxel-based representations for handling sparse data and performing morphological operations.

Conclusion: Implicit occupancy fields provide a superior solution for representing medical imaging data, particularly for sparse CT-scanned ribcages, enabling effective geometric operations that are challenging with discrete voxel-based methods.

Abstract: We present RibPull, a methodology that utilizes implicit occupancy fields to bridge computational geometry and medical imaging. Implicit 3D representations use continuous functions that handle sparse and noisy data more effectively than discrete methods. While voxel grids are standard for medical imaging, they suffer from resolution limitations, topological information loss, and inefficient handling of sparsity. Coordinate functions preserve complex geometrical information and represent a better solution for sparse data representation, while allowing for further morphological operations. Implicit scene representations enable neural networks to encode entire 3D scenes within their weights. The result is a continuous function that can implicitly compesate for sparse signals and infer further information about the 3D scene by passing any combination of 3D coordinates as input to the model. In this work, we use neural occupancy fields that predict whether a 3D point lies inside or outside an object to represent CT-scanned ribcages. We also apply a Laplacian-based contraction to extract the medial axis of the ribcage, thus demonstrating a geometrical operation that benefits greatly from continuous coordinate-based 3D scene representations versus voxel-based representations. We evaluate our methodology on 20 medical scans from the RibSeg dataset, which is itself an extension of the RibFrac dataset. We will release our code upon publication.

Method: Uses Neural Scene Designer (NSD) with parallel cross-attention mechanisms for text and style processing, plus Progressive Self-style Representational Learning (PSRL) module with style contrastive loss to capture fine-grained style representations.

Result: Extensive experiments on a comprehensive benchmark demonstrate the framework’s effectiveness in achieving both semantic control and style consistency.

Conclusion: NSD successfully addresses the gap in stylistic consistency preservation while maintaining semantic alignment, establishing a new standard for style-consistent image editing.

Abstract: Maintaining stylistic consistency is crucial for the cohesion and aesthetic appeal of images, a fundamental requirement in effective image editing and inpainting. However, existing methods primarily focus on the semantic control of generated content, often neglecting the critical task of preserving this consistency. In this work, we introduce the Neural Scene Designer (NSD), a novel framework that enables photo-realistic manipulation of user-specified scene regions while ensuring both semantic alignment with user intent and stylistic consistency with the surrounding environment. NSD leverages an advanced diffusion model, incorporating two parallel cross-attention mechanisms that separately process text and style information to achieve the dual objectives of semantic control and style consistency. To capture fine-grained style representations, we propose the Progressive Self-style Representational Learning (PSRL) module. This module is predicated on the intuitive premise that different regions within a single image share a consistent style, whereas regions from different images exhibit distinct styles. The PSRL module employs a style contrastive loss that encourages high similarity between representations from the same image while enforcing dissimilarity between those from different images. Furthermore, to address the lack of standardized evaluation protocols for this task, we establish a comprehensive benchmark. This benchmark includes competing algorithms, dedicated style-related metrics, and diverse datasets and settings to facilitate fair comparisons. Extensive experiments conducted on our benchmark demonstrate the effectiveness of the proposed framework.

Method: Trained using pseudo-MOS supervision with reproducible distortions scored by an ensemble of quality metrics, featuring spatial masking and curriculum learning for optimization tasks.

Result: Outperforms existing metrics across standard benchmarks, enables efficient perceptual optimization in image and latent domains, and improves performance in denoising, super-resolution, and face restoration tasks.

Conclusion: MILO serves as both a state-of-the-art image quality metric and a practical tool for perceptual optimization in generative pipelines with fast inference suitable for real-time applications.

Abstract: We present MILO (Metric for Image- and Latent-space Optimization), a lightweight, multiscale, perceptual metric for full-reference image quality assessment (FR-IQA). MILO is trained using pseudo-MOS (Mean Opinion Score) supervision, in which reproducible distortions are applied to diverse images and scored via an ensemble of recent quality metrics that account for visual masking effects. This approach enables accurate learning without requiring large-scale human-labeled datasets. Despite its compact architecture, MILO outperforms existing metrics across standard FR-IQA benchmarks and offers fast inference suitable for real-time applications. Beyond quality prediction, we demonstrate the utility of MILO as a perceptual loss in both image and latent domains. In particular, we show that spatial masking modeled by MILO, when applied to latent representations from a VAE encoder within Stable Diffusion, enables efficient and perceptually aligned optimization. By combining spatial masking with a curriculum learning strategy, we first process perceptually less relevant regions before progressively shifting the optimization to more visually distorted areas. This strategy leads to significantly improved performance in tasks like denoising, super-resolution, and face restoration, while also reducing computational overhead. MILO thus functions as both a state-of-the-art image quality metric and as a practical tool for perceptual optimization in generative pipelines.

Method: Constructed a diverse Bangladeshi street food dataset, modified YOLOv8 for improved classification and segmentation, and integrated a machine learning regression model for calorie estimation.

Result: Achieved 6.94 MAE, 11.03 RMSE, and 96.0% R^2 score in calorie estimation with only slight computational complexity increase compared to base YOLOv8.

Conclusion: The proposed system provides highly effective and accurate real-world calorie calculation specifically tailored for Bangladeshi street foods, overcoming previous limitations in automated food calorie tracking.

Abstract: As obesity rates continue to increase, automated calorie tracking has become a vital tool for people seeking to maintain a healthy lifestyle or adhere to a diet plan. Although numerous research efforts have addressed this issue, existing approaches often face key limitations, such as providing only constant caloric output, struggling with multiple food recognition challenges, challenges in image scaling and normalization, and a predominant focus on Western cuisines. In this paper, we propose a tailored solution that specifically targets Bangladeshi street food. We first construct a diverse dataset of popular street foods found across Bangladesh. Then, we develop a refined calorie estimation system by modifying the state-of-the-art vision model YOLOv8. Our modified model achieves superior classification and segmentation results, with only a slight increase in computational complexity compared to the base variant. Coupled with a machine learning regression model, our system achieves an impressive 6.94 mean absolute error (MAE), 11.03 root mean squared error (RMSE), and a 96.0% R^2 score in calorie estimation, making it both highly effective and accurate for real-world food calorie calculations.

Method: Proposes InfoScale framework with three modules: Progressive Frequency Compensation for high-frequency information loss, Adaptive Information Aggregation for flexible attention mechanisms, and Noise Adaptation for proper initial noise distribution alignment.

Result: Extensive experiments demonstrate effective variable-scaled image generation, showing the framework’s plug-and-play capability with existing diffusion models.

Conclusion: InfoScale successfully addresses the three critical challenges in variable-scale generation and provides a unified solution that can be integrated with existing diffusion models.

Abstract: Diffusion models (DMs) have become dominant in visual generation but suffer performance drop when tested on resolutions that differ from the training scale, whether lower or higher. In fact, the key challenge in generating variable-scale images lies in the differing amounts of information across resolutions, which requires information conversion procedures to be varied for generating variable-scaled images. In this paper, we investigate the issues of three critical aspects in DMs for a unified analysis in variable-scaled generation: dilated convolution, attention mechanisms, and initial noise. Specifically, 1) dilated convolution in DMs for the higher-resolution generation loses high-frequency information. 2) Attention for variable-scaled image generation struggles to adjust the information aggregation adaptively. 3) The spatial distribution of information in the initial noise is misaligned with variable-scaled image. To solve the above problems, we propose \textbf{InfoScale}, an information-centric framework for variable-scaled image generation by effectively utilizing information from three aspects correspondingly. For information loss in 1), we introduce Progressive Frequency Compensation module to compensate for high-frequency information lost by dilated convolution in higher-resolution generation. For information aggregation inflexibility in 2), we introduce Adaptive Information Aggregation module to adaptively aggregate information in lower-resolution generation and achieve an effective balance between local and global information in higher-resolution generation. For information distribution misalignment in 3), we design Noise Adaptation module to re-distribute information in initial noise for variable-scaled generation. Our method is plug-and-play for DMs and extensive experiments demonstrate the effectiveness in variable-scaled image generation.

Method: Integrates lightweight Mamba-inspired State Space Model gating mechanism into hierarchical convolutional backbone to dynamically modulate feature maps and emphasize salient facial features with long-range spatial relationships.

Result: Achieves new state-of-the-art on SCUT-FBP5500: Pearson Correlation of 0.9187, MAE of 0.2022, and RMSE of 0.2610.

Conclusion: Validates synergistic potential of combining CNNs with selective SSMs, presenting a powerful new architectural paradigm for nuanced visual understanding tasks.

Abstract: The computational assessment of facial attractiveness, a challenging subjective regression task, is dominated by architectures with a critical trade-off: Convolutional Neural Networks (CNNs) offer efficiency but have limited receptive fields, while Vision Transformers (ViTs) model global context at a quadratic computational cost. To address this, we propose Mamba-CNN, a novel and efficient hybrid architecture. Mamba-CNN integrates a lightweight, Mamba-inspired State Space Model (SSM) gating mechanism into a hierarchical convolutional backbone. This core innovation allows the network to dynamically modulate feature maps and selectively emphasize salient facial features and their long-range spatial relationships, mirroring human holistic perception while maintaining computational efficiency. We conducted extensive experiments on the widely-used SCUT-FBP5500 benchmark, where our model sets a new state-of-the-art. Mamba-CNN achieves a Pearson Correlation (PC) of 0.9187, a Mean Absolute Error (MAE) of 0.2022, and a Root Mean Square Error (RMSE) of 0.2610. Our findings validate the synergistic potential of combining CNNs with selective SSMs and present a powerful new architectural paradigm for nuanced visual understanding tasks.

Method: Analyzed latent activations, attention distributions, and patch uniformity in pretrained vision encoders. Explored sparse autoencoder loss over vision transformer representations as a proxy for memorability.

Result: Found that these features correlate with memorability to some extent. Sparse autoencoder loss on ViT representations outperformed past methods using CNN representations.

Conclusion: Model-internal features are informative predictors of image memorability, shedding light on the relationship between these features and what makes images memorable to humans.

Abstract: Images vary in how memorable they are to humans. Inspired by findings from cognitive science and computer vision, this paper explores the correlates of image memorability in pretrained vision encoders, focusing on latent activations, attention distributions, and the uniformity of image patches. We find that these features correlate with memorability to some extent. Additionally, we explore sparse autoencoder loss over the representations of vision transformers as a proxy for memorability, which yields results outperforming past methods using convolutional neural network representations. Our results shed light on the relationship between model-internal features and memorability. They show that some features are informative predictors of what makes images memorable to humans.

Method: Transformer-based prior model trained on synthetic data for internal geometry knowledge, combined with real data learning for outer structure. Gaussian-splatting-based reconstruction from one or more images.

Result: Qualitative and quantitative comparisons show superior performance in capturing detailed hair orientation, overall silhouette, and backside consistency compared to existing methods.

Conclusion: The hybrid approach using both synthetic and real data effectively models complete 3D hairstyles from single photographs, overcoming limitations of previous methods.

Abstract: We present a novel approach for 3D hair reconstruction from single photographs based on a global hair prior combined with local optimization. Capturing strand-based hair geometry from single photographs is challenging due to the variety and geometric complexity of hairstyles and the lack of ground truth training data. Classical reconstruction methods like multi-view stereo only reconstruct the visible hair strands, missing the inner structure of hairstyles and hampering realistic hair simulation. To address this, existing methods leverage hairstyle priors trained on synthetic data. Such data, however, is limited in both quantity and quality since it requires manual work from skilled artists to model the 3D hairstyles and create near-photorealistic renderings. To address this, we propose a novel approach that uses both, real and synthetic data to learn an effective hairstyle prior. Specifically, we train a transformer-based prior model on synthetic data to obtain knowledge of the internal hairstyle geometry and introduce real data in the learning process to model the outer structure. This training scheme is able to model the visible hair strands depicted in an input image, while preserving the general 3D structure of hairstyles. We exploit this prior to create a Gaussian-splatting-based reconstruction method that creates hairstyles from one or more images. Qualitative and quantitative comparisons with existing reconstruction pipelines demonstrate the effectiveness and superior performance of our method for capturing detailed hair orientation, overall silhouette, and backside consistency. For additional results and code, please refer to https://im2haircut.is.tue.mpg.de.

Method: Proposes PointSlice which converts 3D point clouds into multiple 2D data slices along horizontal planes, and introduces a Slice Interaction Network (SIN) to maintain vertical relationships across slices while learning only 2D data distributions.

Result: Achieves 1.13x faster speed and 0.79x fewer parameters than state-of-the-art voxel methods on Waymo with only 1.2 mAPH accuracy reduction. Achieves 66.74 mAP on nuScenes (SOTA) and 1.10x faster speed with 0.66x fewer parameters on Argoverse 2.

Conclusion: PointSlice effectively balances speed and accuracy by treating 3D point clouds as batches of 2D data with vertical interaction, making it suitable for real-time autonomous driving applications.

Abstract: 3D object detection from point clouds plays a critical role in autonomous driving. Currently, the primary methods for point cloud processing are voxel-based and pillarbased approaches. Voxel-based methods offer high accuracy through fine-grained spatial segmentation but suffer from slower inference speeds. Pillar-based methods enhance inference speed but still fall short of voxel-based methods in accuracy. To address these issues, we propose a novel point cloud processing method, PointSlice, which slices point clouds along the horizontal plane and includes a dedicated detection network. The main contributions of PointSlice are: (1) A new point cloud processing technique that converts 3D point clouds into multiple sets of 2D (x-y) data slices. The model only learns 2D data distributions, treating the 3D point cloud as separate batches of 2D data, which reduces the number of model parameters and enhances inference speed; (2) The introduction of a Slice Interaction Network (SIN). To maintain vertical relationships across slices, we incorporate SIN into the 2D backbone network, which improves the model’s 3D object perception capability. Extensive experiments demonstrate that PointSlice achieves high detection accuracy and inference speed. On the Waymo dataset, PointSlice is 1.13x faster and has 0.79x fewer parameters than the state-of-the-art voxel-based method (SAFDNet), with only a 1.2 mAPH accuracy reduction. On the nuScenes dataset, we achieve a state-of-the-art detection result of 66.74 mAP. On the Argoverse 2 dataset, PointSlice is 1.10x faster, with 0.66x fewer parameters and a 1.0 mAP accuracy reduction. The code will be available at https://github.com/qifeng22/PointSlice2.

Method: Integrates TrigFlow-inspired continuous noise schedule with Chamfer Distance-based geometric loss, uses time-conditioned neural network operating in continuous time without discretized diffusion steps or pre-trained teacher models.

Result: Matches or outperforms state-of-the-art diffusion and latent consistency models on ShapeNet benchmark in both quality and efficiency, with high geometric fidelity.

Conclusion: ConTiCoM-3D establishes a practical framework for scalable 3D shape generation with efficient one- to two-step inference while maintaining high quality.

Abstract: Fast and accurate 3D shape generation from point clouds is essential for applications in robotics, AR/VR, and digital content creation. We introduce ConTiCoM-3D, a continuous-time consistency model that synthesizes 3D shapes directly in point space, without discretized diffusion steps, pre-trained teacher models, or latent-space encodings. The method integrates a TrigFlow-inspired continuous noise schedule with a Chamfer Distance-based geometric loss, enabling stable training on high-dimensional point sets while avoiding expensive Jacobian-vector products. This design supports efficient one- to two-step inference with high geometric fidelity. In contrast to previous approaches that rely on iterative denoising or latent decoders, ConTiCoM-3D employs a time-conditioned neural network operating entirely in continuous time, thereby achieving fast generation. Experiments on the ShapeNet benchmark show that ConTiCoM-3D matches or outperforms state-of-the-art diffusion and latent consistency models in both quality and efficiency, establishing it as a practical framework for scalable 3D shape generation.

Method: Propose ReVPT with novel RL algorithm based on GRPO to train models to reason with a suite of four visual tools through reinforcement learning.

Result: Achieves state-of-the-art performance on SAT, CV-Bench, BLINK and MMStar benchmarks. ReVPT-3B/7B outperform instruct models by 9.03%/9.44% on CV-Bench.

Conclusion: RL-based approach effectively enhances visual tool usage and reasoning capabilities, providing new insights for visual reasoning systems.

Abstract: Visual reasoning, a cornerstone of human intelligence, encompasses complex perceptual and logical processes essential for solving diverse visual problems. While advances in computer vision have produced powerful models for various perceptual tasks, leveraging these for general visual reasoning remains challenging. Prior work demonstrates that augmenting LLMs with vision models via supervised finetuning improves performance, but faces key limitations such as expensive data generation, reliance on careful data filtering, and poor generalization. To address these issues, we propose ReVPT to enhance multi-modal LLMs’ abilities to reason about and use visual tools through reinforcement learning. We introduce a novel RL algorithm based on GRPO, designed to train models to reason with a suite of four visual tools. Through extensive experiments, we show that our method achieves state-of-the-art performance on several perception-heavy benchmarks, including SAT, CV-Bench, BLINK and MMStar, significantly outperforming the supervised and text-based RL finetuning baselines. Notably, Our ReVPT-3B and ReVPT-7B outperform the instruct models by 9.03% and 9.44% on CV-Bench. Finally, we bring to the community new insights on RL-based visual tool-usage through extensive ablations. Our code is available at https://github.com/ls-kelvin/REVPT.

Method: A Self-Adaptive Convolution Module that adjusts convolution kernel sizes based on dataset fingerprints. Integrated into Multi-Scale Convolution Bridge and Multi-Scale Amalgamation Decoder of MSA2-Net to capture both global and local features while eliminating redundant data.

Result: Achieved Dice coefficients of 86.49% (Synapse), 92.56% (ACDC), 93.37% (Kvasir), and 92.98% (ISIC2017), demonstrating robust performance across diverse medical image segmentation datasets.

Conclusion: MSA2-Net with Self-Adaptive Convolution Module shows exceptional precision and robustness in medical image segmentation, effectively addressing nnUNet’s limitations in internal hyperparameter tuning.

Abstract: The nnUNet segmentation framework adeptly adjusts most hyperparameters in training scripts automatically, but it overlooks the tuning of internal hyperparameters within the segmentation network itself, which constrains the model’s ability to generalize. Addressing this limitation, this study presents a novel Self-Adaptive Convolution Module that dynamically adjusts the size of the convolution kernels depending on the unique fingerprints of different datasets. This adjustment enables the MSA2-Net, when equipped with this module, to proficiently capture both global and local features within the feature maps. Self-Adaptive Convolution Module is strategically integrated into two key components of the MSA2-Net: the Multi-Scale Convolution Bridge and the Multi-Scale Amalgamation Decoder. In the MSConvBridge, the module enhances the ability to refine outputs from various stages of the CSWin Transformer during the skip connections, effectively eliminating redundant data that could potentially impair the decoder’s performance. Simultaneously, the MSADecoder, utilizing the module, excels in capturing detailed information of organs varying in size during the decoding phase. This capability ensures that the decoder’s output closely reproduces the intricate details within the feature maps, thus yielding highly accurate segmentation images. MSA2-Net, bolstered by this advanced architecture, has demonstrated exceptional performance, achieving Dice coefficient scores of 86.49%, 92.56%, 93.37%, and 92.98% on the Synapse, ACDC, Kvasir, and Skin Lesion Segmentation (ISIC2017) datasets, respectively. This underscores MSA2-Net’s robustness and precision in medical image segmentation tasks across various datasets.

Method: Created the Remote Sensing Change Caption (RSCC) dataset - a large-scale benchmark with 62,315 pre-/post-disaster image pairs covering earthquakes, floods, wildfires and other disasters, paired with rich human-like change captions.

Result: RSCC enables robust training and evaluation of vision-language models for disaster-aware bi-temporal understanding, facilitating detailed disaster-related analysis.

Conclusion: The dataset bridges the temporal and semantic divide in remote sensing data, paving the way for more accurate, interpretable, and scalable vision-language applications in remote sensing for disaster monitoring.

Abstract: Remote sensing is critical for disaster monitoring, yet existing datasets lack temporal image pairs and detailed textual annotations. While single-snapshot imagery dominates current resources, it fails to capture dynamic disaster impacts over time. To address this gap, we introduce the Remote Sensing Change Caption (RSCC) dataset, a large-scale benchmark comprising 62,315 pre-/post-disaster image pairs (spanning earthquakes, floods, wildfires, and more) paired with rich, human-like change captions. By bridging the temporal and semantic divide in remote sensing data, RSCC enables robust training and evaluation of vision-language models for disaster-aware bi-temporal understanding. Our results highlight RSCC’s ability to facilitate detailed disaster-related analysis, paving the way for more accurate, interpretable, and scalable vision-language applications in remote sensing. Code and dataset are available at https://github.com/Bili-Sakura/RSCC.

Method: Proposes Variation-aware Vision Token Dropping (V²Drop) which progressively removes visual tokens with minimal variation during inference, based on the observation that visual token variations exhibit task-agnostic properties.

Result: Maintains 94.0% of original performance for image understanding and 98.6% for video understanding while reducing LLM generation latency by 31.5% and 74.2% respectively, with additional GPU memory reduction when combined with efficient operators.

Conclusion: V²Drop provides an effective token compression solution that addresses limitations of existing methods and significantly improves computational efficiency for LVLMs while preserving performance.

Abstract: Large vision-language models (LVLMs) have demonstrated remarkable capabilities in multimodal understanding tasks. However, the increasing demand for high-resolution image and long-video understanding results in substantial token counts, leading to reduced inference efficiency. Token compression offers a direct solution by reducing the number of tokens to be processed, thereby improving computational efficiency. Through extensive analysis, we identify two critical limitations in existing inner-LLM token compression methods: positional bias and incompatibility with efficient operators, which hinder their practical deployment for LVLM acceleration. This paper presents the first approach from a token variation perspective, revealing that visual token variations within LLMs exhibit task-agnostic properties. We propose Variation-aware Vision Token Dropping (\textit{i.e.}, \textbf{V$^2$Drop}), which progressively removes visual tokens with minimal variation during LVLM inference, thereby enhancing computational efficiency. Extensive experiments across multiple models and benchmarks demonstrate that our V$^2$Drop is able to maintain \textbf{94.0%} and \textbf{98.6%} of the original model performance for image and video understanding tasks respectively, while reducing LLM generation latency by \textbf{31.5%} and \textbf{74.2%}. When combined with efficient operators, V$^2$Drop further reduces GPU peak memory usage.

Method: Uniferum harmonizes three public 3D CT datasets with distinct annotations by unifying classification labels and segmentation masks into a single training framework, integrating heterogeneous annotations and body segmentation.

Result: Achieves 7% AUROC improvement on CT-RATE benchmark compared to CLIP-based and conventional multi-label convolutional models, with robust out-of-distribution generalization and unexpected zero-shot performance on RAD-CHEST and INSPECT datasets.

Conclusion: The approach effectively integrates heterogeneous annotations and body segmentation to enhance model performance, setting a new direction for clinically reliable, data-efficient VLMs in 3D medical imaging.

Abstract: General-purpose vision-language models (VLMs) have emerged as promising tools in radiology, offering zero-shot capabilities that mitigate the need for large labeled datasets. However, in high-stakes domains like diagnostic radiology, these models often lack the discriminative precision required for reliable clinical use. This challenge is compounded by the scarcity and heterogeneity of publicly available volumetric CT datasets, which vary widely in annotation formats and granularity. To address these limitations, we introduce Uniferum, a volumetric VLM that unifies diverse supervision signals, encoded in classification labels and segmentation masks, into a single training framework. By harmonizing three public 3D CT datasets with distinct annotations, Uniferum achieves state-of-the-art performance, improving AUROC on the CT-RATE benchmark by 7% compared to CLIP-based and conventional multi-label convolutional models. The model demonstrates robust out-of-distribution generalization, with observed evidence of unexpected zero-shot performance on the RAD-CHEST and INSPECT datasets. Our results highlight the effectiveness of integrating heterogeneous annotations and body segmentation to enhance model performance, setting a new direction for clinically reliable, data-efficient VLMs in 3D medical imaging.

Method: Uses Vector Quantized Variational Autoencoder (VQ-VAE) to encode noisy images into latent space, then applies latent diffusion model to iteratively remove interference, followed by decoding to reconstruct clean images.

Result: Achieves SSIM of 0.796 and PSNR of 23.780 (vs 0.443 and 14.420 for Notch Filter) with real-time processing at 15 FPS (vs 5 seconds per frame for Notch Filter).

Conclusion: HIFU-ILDiff enables effective real-time denoising of HIFU interference, improving treatment precision for clinical HIFU applications.

Abstract: High-Intensity Focused Ultrasound (HIFU) is a non-invasive therapeutic technique widely used for treating various diseases. However, the success and safety of HIFU treatments depend on real-time monitoring, which is often hindered by interference when using ultrasound to guide HIFU treatment. To address these challenges, we developed HIFU-ILDiff, a novel deep learning-based approach leveraging latent diffusion models to suppress HIFU-induced interference in ultrasound images. The HIFU-ILDiff model employs a Vector Quantized Variational Autoencoder (VQ-VAE) to encode noisy ultrasound images into a lower-dimensional latent space, followed by a latent diffusion model that iteratively removes interference. The denoised latent vectors are then decoded to reconstruct high-resolution, interference-free ultrasound images. We constructed a comprehensive dataset comprising 18,872 image pairs from in vitro phantoms, ex vivo tissues, and in vivo animal data across multiple imaging modalities and HIFU power levels to train and evaluate the model. Experimental results demonstrate that HIFU-ILDiff significantly outperforms the commonly used Notch Filter method, achieving a Structural Similarity Index (SSIM) of 0.796 and Peak Signal-to-Noise Ratio (PSNR) of 23.780 compared to SSIM of 0.443 and PSNR of 14.420 for the Notch Filter under in vitro scenarios. Additionally, HIFU-ILDiff achieves real-time processing at 15 frames per second, markedly faster than the Notch Filter’s 5 seconds per frame. These findings indicate that HIFU-ILDiff is able to denoise HIFU interference in ultrasound guiding images for real-time monitoring during HIFU therapy, which will greatly improve the treatment precision in current clinical applications.

Method: 1) Slow-Fast video encoding strategy with dynamic resource allocation based on inter-frame similarity; 2) Progressive 4-stage pre-training extending context from 8K to 128K tokens; 3) Post-training pipeline with 5-step chain-of-thought data, iterative GSPO-based RL, and alignment training.

Result: Significant improvements over existing models on public benchmarks and internal human assessment, particularly excelling in video understanding tasks while maintaining competitive performance on general multimodal benchmarks.

Conclusion: Keye-VL-1.5 successfully addresses fundamental challenges in video comprehension through its three key innovations, demonstrating state-of-the-art performance in video understanding tasks.

Abstract: In recent years, the development of Large Language Models (LLMs) has significantly advanced, extending their capabilities to multimodal tasks through Multimodal Large Language Models (MLLMs). However, video understanding remains a challenging area due to the dynamic and information-dense nature of videos. Existing models struggle with the trade-off between spatial resolution and temporal coverage when processing video content. We present Keye-VL-1.5, which addresses fundamental challenges in video comprehension through three key innovations. First, we introduce a novel Slow-Fast video encoding strategy that dynamically allocates computational resources based on inter-frame similarity, processing key frames with significant visual changes at higher resolution (Slow pathway) while handling relatively static frames with increased temporal coverage at lower resolution (Fast pathway). Second, we implement a progressive four-stage pre-training methodology that systematically extends the model’s context length from 8K to 128K tokens, enabling processing of longer videos and more complex visual content. Third, we develop a comprehensive post-training pipeline focusing on reasoning enhancement and human preference alignment, incorporating a 5-step chain-of-thought data construction process, iterative GSPO-based reinforcement learning with progressive prompt hinting for difficult cases, and alignment training. Through extensive evaluation on public benchmarks and rigorous internal human assessment, Keye-VL-1.5 demonstrates significant improvements over existing models, particularly excelling in video understanding tasks while maintaining competitive performance on general multimodal benchmarks.

Method: Created a new open-source benchmark with real-world image-text pairs focusing on relative spatial relationships and object ordering. Used contrastive evaluation and performed disentanglement analysis to separate object localization from spatial reasoning capabilities.

Result: VLMs show striking lack of spatial relation understanding, with reasoning models performing surprisingly well. Performance is bottlenecked by spatial reasoning rather than object localization capabilities.

Conclusion: Spatial reasoning remains a significant challenge for current VLMs, and the RocketScience benchmark effectively exposes this weakness while providing a valuable tool for future model development and evaluation.

Abstract: We propose RocketScience, an open-source contrastive VLM benchmark that tests for spatial relation understanding. It is comprised of entirely new real-world image-text pairs covering mostly relative spatial understanding and the order of objects. The benchmark is designed to be very easy for humans and hard for the current generation of VLMs, and this is empirically verified. Our results show a striking lack of spatial relation understanding in open source and frontier commercial VLMs and a surprisingly high performance of reasoning models. Additionally, we perform a disentanglement analysis to separate the contributions of object localization and spatial reasoning in chain-of-thought-based models and find that the performance on the benchmark is bottlenecked by spatial reasoning and not object localization capabilities. We release the dataset with a CC-BY-4.0 license and make the evaluation code available at: https://github.com/nilshoehing/rocketscience

Method: Uses a lightweight symmetric two-view association (STA) model as frontend to estimate relative camera poses and regress local pointmaps from two RGB images. Backend employs a specially designed Sim(3) pose graph with loop closures to handle accumulated drift.

Result: Achieves superior performance in both camera tracking and dense 3D reconstruction quality compared to current methods, with frontend size reduced to 35% of comparable state-of-the-art methods.

Conclusion: ViSTA-SLAM demonstrates that a lightweight, calibration-free approach can outperform existing methods while being more broadly applicable across different camera configurations.

Abstract: We present ViSTA-SLAM as a real-time monocular visual SLAM system that operates without requiring camera intrinsics, making it broadly applicable across diverse camera setups. At its core, the system employs a lightweight symmetric two-view association (STA) model as the frontend, which simultaneously estimates relative camera poses and regresses local pointmaps from only two RGB images. This design reduces model complexity significantly, the size of our frontend is only 35% that of comparable state-of-the-art methods, while enhancing the quality of two-view constraints used in the pipeline. In the backend, we construct a specially designed Sim(3) pose graph that incorporates loop closures to address accumulated drift. Extensive experiments demonstrate that our approach achieves superior performance in both camera tracking and dense 3D reconstruction quality compared to current methods. Github repository: https://github.com/zhangganlin/vista-slam

Method: Proposes O-DisCo-Edit framework with novel object distortion control (O-DisCo) signal based on random and adaptive noise, paired with a “copy-form” preservation module to maintain non-edited regions. Uses an effective training paradigm for efficient, high-fidelity editing.

Result: Extensive experiments and comprehensive human evaluations show that O-DisCo-Edit surpasses both specialized and multitask state-of-the-art methods across various video editing tasks.

Conclusion: O-DisCo-Edit provides a unified solution for controllable video editing that outperforms existing methods while simplifying the model design and reducing training resource requirements through its flexible object distortion control representation.

Abstract: Diffusion models have recently advanced video editing, yet controllable editing remains challenging due to the need for precise manipulation of diverse object properties. Current methods require different control signal for diverse editing tasks, which complicates model design and demands significant training resources. To address this, we propose O-DisCo-Edit, a unified framework that incorporates a novel object distortion control (O-DisCo). This signal, based on random and adaptive noise, flexibly encapsulates a wide range of editing cues within a single representation. Paired with a “copy-form” preservation module for preserving non-edited regions, O-DisCo-Edit enables efficient, high-fidelity editing through an effective training paradigm. Extensive experiments and comprehensive human evaluations consistently demonstrate that O-DisCo-Edit surpasses both specialized and multitask state-of-the-art methods across various video editing tasks. https://cyqii.github.io/O-DisCo-Edit.github.io/

Method: Proposes an encoder-decoder Vision Transformer that processes two input X-ray images as separate sequences of patches. Uses shared segmentation heads for both encoder and decoder embeddings, and a regression head with fused decoder information for 3D force estimation.

Result: The model outperforms state-of-the-art pure segmentation models, vision-based catheter force estimation methods, and multitask approaches. Sets new state-of-the-art in both catheter segmentation and force estimation across various noise levels in synthetic X-ray images.

Conclusion: The stereo Vision Transformer approach effectively captures long-range dependencies and demonstrates superior performance for simultaneous catheter segmentation and 3D force estimation in medical imaging applications.

Abstract: Recently, the emergence of multitask deep learning models has enhanced catheterization procedures by providing tactile and visual perception data through an end-to-end architec- ture. This information is derived from a segmentation and force estimation head, which localizes the catheter in X-ray images and estimates the applied pressure based on its deflection within the image. These stereo vision architectures incorporate a CNN- based encoder-decoder that captures the dependencies between X-ray images from two viewpoints, enabling simultaneous 3D force estimation and stereo segmentation of the catheter. With these tasks in mind, this work approaches the problem from a new perspective. We propose a novel encoder-decoder Vision Transformer model that processes two input X-ray images as separate sequences. Given sequences of X-ray patches from two perspectives, the transformer captures long-range dependencies without the need to gradually expand the receptive field for either image. The embeddings generated by both the encoder and decoder are fed into two shared segmentation heads, while a regression head employs the fused information from the decoder for 3D force estimation. The proposed model is a stereo Vision Transformer capable of simultaneously segmenting the catheter from two angles while estimating the generated forces at its tip in 3D. This model has undergone extensive experiments on synthetic X-ray images with various noise levels and has been compared against state-of-the-art pure segmentation models, vision-based catheter force estimation methods, and a multitask catheter segmentation and force estimation approach. It outperforms existing models, setting a new state-of-the-art in both catheter segmentation and force estimation.

Method: A collaborative learning framework where a panel of LVLMs generate, assess, and refine outputs through iterative peer review, simulating a classroom learning environment with curated prompts.

Result: Significant performance improvement across multiple benchmarks, with average score increasing from 48% to 57% on fifteen benchmarks.

Conclusion: Panel-of-Peers provides a scalable alternative to self-supervised alignment that enhances model performance without requiring extensive human-labeled datasets.

Abstract: Traditional alignment methods for Large Vision and Language Models (LVLMs) primarily rely on human-curated preference data. Human-generated preference data is costly; machine-generated preference data is limited in quality; and self-supervised preference data often introduces hallucinations. To overcome these limitations, we propose a novel Panel-of-Peers learning framework inspired by collaborative learning among humans. This approach leverages a panel of LVLMs, each evaluating and learning from their collective outputs through an iterative self-improvement process. By simulating a peer review system, our models generate, assess, and refine outputs in response to a curated set of prompts, mimicking a classroom learning environment. We demonstrate that this methodology enhances model performance without requiring extensive human-labeled datasets. Our experiments show significant improvement across multiple benchmarks, demonstrating the potential of peer evaluations as a scalable alternative to self-supervised alignment. Notably, we show that Panel-of-Peers increases the average score on fifteen benchmarks from 48% to 57%

Method: Q-Sched modifies the diffusion model scheduler rather than model weights, adjusting the few-step sampling trajectory. It uses JAQ loss (combining text-image compatibility with image quality metric) to learn quantization-aware pre-conditioning coefficients with reference-free calibration using only a handful of prompts.

Result: Achieves 4x reduction in model size while maintaining full-precision accuracy. Shows 15.5% FID improvement over FP16 4-step Latent Consistency Model and 16.6% improvement over FP16 8-step Phased Consistency Model. Large-scale user study with 80,000+ annotations confirms effectiveness on FLUX.1 and SDXL-Turbo.

Conclusion: Quantization and few-step distillation are complementary for high-fidelity generation. Q-Sched provides an effective post-training quantization approach that avoids full-precision inference during calibration and delivers substantial performance gains.

Abstract: Text-to-image diffusion models are computationally intensive, often requiring dozens of forward passes through large transformer backbones. For instance, Stable Diffusion XL generates high-quality images with 50 evaluations of a 2.6B-parameter model, an expensive process even for a single batch. Few-step diffusion models reduce this cost to 2-8 denoising steps but still depend on large, uncompressed U-Net or diffusion transformer backbones, which are often too costly for full-precision inference without datacenter GPUs. These requirements also limit existing post-training quantization methods that rely on full-precision calibration. We introduce Q-Sched, a new paradigm for post-training quantization that modifies the diffusion model scheduler rather than model weights. By adjusting the few-step sampling trajectory, Q-Sched achieves full-precision accuracy with a 4x reduction in model size. To learn quantization-aware pre-conditioning coefficients, we propose the JAQ loss, which combines text-image compatibility with an image quality metric for fine-grained optimization. JAQ is reference-free and requires only a handful of calibration prompts, avoiding full-precision inference during calibration. Q-Sched delivers substantial gains: a 15.5% FID improvement over the FP16 4-step Latent Consistency Model and a 16.6% improvement over the FP16 8-step Phased Consistency Model, showing that quantization and few-step distillation are complementary for high-fidelity generation. A large-scale user study with more than 80,000 annotations further confirms Q-Sched’s effectiveness on both FLUX.1[schnell] and SDXL-Turbo.

Method: Removed text encoder and contrastive loss, retained only captioning loss for purely generative training signal. Scaled to over 1 billion parameters.

Result: 1.5x faster training (83h to 57h), 1.8x lower memory usage (24.5GB to 13.8GB), 4x larger batch size (2k to 8k), while maintaining competitive multimodal benchmark performance.

Conclusion: The lightweight generative-only paradigm is compelling for future vision encoder development in multimodal foundation models due to superior training efficiency.

Abstract: This paper provides a simplification on OpenVision’s architecture and loss design for enhancing its training efficiency. Following the prior vision-language pretraining works CapPa and AIMv2, as well as modern multimodal designs like LLaVA, our changes are straightforward: we remove the text encoder (and therefore the contrastive loss), retaining only the captioning loss as a purely generative training signal. We name this new version OpenVision 2. The initial results are promising: despite this simplification, OpenVision 2 competitively matches the original model’s performance on a broad set of multimodal benchmarks while substantially cutting both training time and memory consumption. For example, with ViT-L/14, it reduces training time by about 1.5x (from 83h to 57h), and memory usage by about 1.8x (from 24.5GB to 13.8GB, equivalently allowing the maximum batch size to grow from 2k to 8k). This superior training efficiency also allows us to scale far beyond the largest vision encoder used in OpenVision, reaching more than 1 billion parameters. We hold a strong belief that this lightweight, generative-only paradigm is compelling for future vision encoder development in multimodal foundation models.

Method: Proposes Gaussian splatting densification strategy on cylindrical skeletal structures, novel view segmentation module for accurate semantic maps, and UNet generator combining Gaussian features with segmentation for photorealistic rendering.

Result: Achieves 79 FPS rendering speed with state-of-the-art pixel fidelity: 32.94dB on ZJU Mocap dataset and 33.39dB on Thuman4 dataset, outperforming previous methods in visual perception and quality.

Conclusion: GaussianGAN successfully addresses blurring issues in current avatar solutions, providing real-time photorealistic rendering with superior visual quality through innovative Gaussian splatting and segmentation techniques.

Abstract: Photorealistic and controllable human avatars have gained popularity in the research community thanks to rapid advances in neural rendering, providing fast and realistic synthesis tools. However, a limitation of current solutions is the presence of noticeable blurring. To solve this problem, we propose GaussianGAN, an animatable avatar approach developed for photorealistic rendering of people in real-time. We introduce a novel Gaussian splatting densification strategy to build Gaussian points from the surface of cylindrical structures around estimated skeletal limbs. Given the camera calibration, we render an accurate semantic segmentation with our novel view segmentation module. Finally, a UNet generator uses the rendered Gaussian splatting features and the segmentation maps to create photorealistic digital avatars. Our method runs in real-time with a rendering speed of 79 FPS. It outperforms previous methods regarding visual perception and quality, achieving a state-of-the-art results in terms of a pixel fidelity of 32.94db on the ZJU Mocap dataset and 33.39db on the Thuman4 dataset.

Method: Used PCA to reduce data to 3 dimensions before feeding into a three-layer classifier, testing with ResNet50 and DINOv2 architectures.

Result: PCA effectiveness varies significantly based on the chosen deep learning architecture and training approach.

Conclusion: Findings open avenues for future research into self-supervised pre-training and alternative dimensionality reduction methods for multispectral image classification.

Abstract: This paper investigates the utility of Principal Component Analysis (PCA) for multi-label classification of multispectral images using ResNet50 and DINOv2, acknowledging the high dimensionality of such data and the associated processing challenges. Multi-label classification, where each image may belong to multiple classes, adds further complexity to feature extraction. Our pipeline includes an optional PCA step that reduces the data to three dimensions before feeding it into a three-layer classifier. The findings demonstrate that the effectiveness of PCA for multi-label multispectral image classification depends strongly on the chosen deep learning architecture and training strategy, opening avenues for future research into self-supervised pre-training and alternative dimensionality reduction approaches.

Method: Proposed CNSCA model based on ConvNeXt architecture augmented with self-attention for long-range spatial dependencies and channel attention for informative feature channels

Result: Model significantly outperforms three strong baselines on rock size classification dataset, demonstrating improved accuracy and robustness

Conclusion: Incorporation of attention mechanisms enhances deep learning models’ capability for fine-grained classification tasks involving natural textures like rocks

Abstract: Accurate classification of rock sizes is a vital component in geotechnical engineering, mining, and resource management, where precise estimation influences operational efficiency and safety. In this paper, we propose an enhanced deep learning model based on the ConvNeXt architecture, augmented with both self-attention and channel attention mechanisms. Building upon the foundation of ConvNext, our proposed model, termed CNSCA, introduces self-attention to capture long-range spatial dependencies and channel attention to emphasize informative feature channels. This hybrid design enables the model to effectively capture both fine-grained local patterns and broader contextual relationships within rock imagery, leading to improved classification accuracy and robustness. We evaluate our model on a rock size classification dataset and compare it against three strong baseline. The results demonstrate that the incorporation of attention mechanisms significantly enhances the models capability for fine-grained classification tasks involving natural textures like rocks.

Method: Two-stage diffusion framework: 1) Transformer-based diffusion model conditioned on metadata (acquisition parameters, demographics, diagnostic impressions) generates coarse anatomical priors, 2) Refinement stage integrates coarse prior and metadata with physics-based data consistency using ADMM at each sampling step.

Result: Significantly improved reconstruction fidelity under severe angular truncation, outperforming metadata-free baselines in SSIM, PSNR, nMI, and PCC metrics. Different metadata types provide complementary benefits, especially diagnostic and demographic priors.

Conclusion: Clinical metadata plays a dual role in improving both reconstruction quality and efficiency, supporting their integration into future metadata-guided medical imaging frameworks.

Abstract: Limited-angle computed tomography (LACT) offers improved temporal resolution and reduced radiation dose for cardiac imaging, but suffers from severe artifacts due to truncated projections. To address the ill-posedness of LACT reconstruction, we propose a two-stage diffusion framework guided by structured clinical metadata. In the first stage, a transformer-based diffusion model conditioned exclusively on metadata, including acquisition parameters, patient demographics, and diagnostic impressions, generates coarse anatomical priors from noise. The second stage further refines the images by integrating both the coarse prior and metadata to produce high-fidelity results. Physics-based data consistency is enforced at each sampling step in both stages using an Alternating Direction Method of Multipliers module, ensuring alignment with the measured projections. Extensive experiments on both synthetic and real cardiac CT datasets demonstrate that incorporating metadata significantly improves reconstruction fidelity, particularly under severe angular truncation. Compared to existing metadata-free baselines, our method achieves superior performance in SSIM, PSNR, nMI, and PCC. Ablation studies confirm that different types of metadata contribute complementary benefits, particularly diagnostic and demographic priors under limited-angle conditions. These findings highlight the dual role of clinical metadata in improving both reconstruction quality and efficiency, supporting their integration into future metadata-guided medical imaging frameworks.

Method: TransMatch merges transfer learning and semi-supervised few-shot learning to leverage both labeled and unlabeled novel-class images, overcoming limitations of previous meta-learning approaches.

Result: Achieved 98.91% accuracy with minimal loss, along with high precision, recall, and F1-scores for multiple defect classes including cracks, pinholes, holes, and spatter on a dataset of 8,284 images.

Conclusion: TransMatch represents a significant advancement in additive manufacturing defect detection, offering a practical and scalable solution for quality assurance across industrial applications.

Abstract: Surface defects in Laser Powder Bed Fusion (LPBF) pose significant risks to the structural integrity of additively manufactured components. This paper introduces TransMatch, a novel framework that merges transfer learning and semi-supervised few-shot learning to address the scarcity of labeled AM defect data. By effectively leveraging both labeled and unlabeled novel-class images, TransMatch circumvents the limitations of previous meta-learning approaches. Experimental evaluations on a Surface Defects dataset of 8,284 images demonstrate the efficacy of TransMatch, achieving 98.91% accuracy with minimal loss, alongside high precision, recall, and F1-scores for multiple defect classes. These findings underscore its robustness in accurately identifying diverse defects, such as cracks, pinholes, holes, and spatter. TransMatch thus represents a significant leap forward in additive manufacturing defect detection, offering a practical and scalable solution for quality assurance and reliability across a wide range of industrial applications.

Method: BIB integrates loss function re-balancing and self-distillation techniques into original IB network. MBIB extends this with multiple BIBs combining knowledge from different network layers, enabling end-to-end simultaneous representation and classification learning.

Result: Experiments on CIFAR100-LT, ImageNet-LT, and iNaturalist 2018 show both BIB and MBIB achieve state-of-the-art performance for long-tailed visual recognition.

Conclusion: The proposed BIB and MBIB approaches effectively address long-tailed recognition by preserving essential label-related information through information bottleneck framework enhanced with re-balancing and multi-layer knowledge integration.

Abstract: Deep neural networks (DNNs) have achieved significant success in various applications with large-scale and balanced data. However, data in real-world visual recognition are usually long-tailed, bringing challenges to efficient training and deployment of DNNs. Information bottleneck (IB) is an elegant approach for representation learning. In this paper, we propose a balanced information bottleneck (BIB) approach, in which loss function re-balancing and self-distillation techniques are integrated into the original IB network. BIB is thus capable of learning a sufficient representation with essential label-related information fully preserved for long-tailed visual recognition. To further enhance the representation learning capability, we also propose a novel structure of mixture of multiple balanced information bottlenecks (MBIB), where different BIBs are responsible for combining knowledge from different network layers. MBIB facilitates an end-to-end learning strategy that trains representation and classification simultaneously from an information theory perspective. We conduct experiments on commonly used long-tailed datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist 2018. Both BIB and MBIB reach state-of-the-art performance for long-tailed visual recognition.

Method: The approach trains a lightweight LoRA regressor to produce direct irradiance maps from a small set of training images. Key insight is that lighting relationships are similar to token interactions in self-attention layers. Uses Classifier Guidance to incorporate desired lighting into image generation.

Result: State-of-the-art performance in quality and control with proven parameter and data efficiency across diverse scenes and image domains. The method generalizes well to various conditions without extensive retraining.

Conclusion: Image lighting can be effectively controlled by tapping into foundational knowledge of generative models, enabling practical and general relighting applications with minimal training requirements.

Abstract: Light control in generated images is a difficult task, posing specific challenges, spanning over the entire image and frequency spectrum. Most approaches tackle this problem by training on extensive yet domain-specific datasets, limiting the inherent generalization and applicability of the foundational backbones used. Instead, PractiLight is a practical approach, effectively leveraging foundational understanding of recent generative models for the task. Our key insight is that lighting relationships in an image are similar in nature to token interaction in self-attention layers, and hence are best represented there. Based on this and other analyses regarding the importance of early diffusion iterations, PractiLight trains a lightweight LoRA regressor to produce the direct irradiance map for a given image, using a small set of training images. We then employ this regressor to incorporate the desired lighting into the generation process of another image using Classifier Guidance. This careful design generalizes well to diverse conditions and image domains. We demonstrate state-of-the-art performance in terms of quality and control with proven parameter and data efficiency compared to leading works over a wide variety of scenes types. We hope this work affirms that image lighting can feasibly be controlled by tapping into foundational knowledge, enabling practical and general relighting.

Method: LGDiST uses a diffusion model approach with context genes previously considered uninformative to construct a rich genetic latent space. The model incorporates neighbor conditioning and operates without external references.

Result: LGDiST outperforms previous state-of-the-art methods with 18% lower average MSE across 26 datasets. It improves gene expression prediction performance by up to 10% MSE on six state-of-the-art methods. Ablation studies show that removing key components (context genes, ST latent space, neighbor conditioning) leads to significant performance drops.

Conclusion: LGDiST represents a significant advancement in spatial transcriptomics data completion, demonstrating that reference-free approaches using context genes and diffusion modeling can effectively address data dropout while providing biologically meaningful representations.

Abstract: Computer Vision has proven to be a powerful tool for analyzing Spatial Transcriptomics (ST) data. However, current models that predict spatially resolved gene expression from histopathology images suffer from significant limitations due to data dropout. Most existing approaches rely on single-cell RNA sequencing references, making them dependent on alignment quality and external datasets while also risking batch effects and inherited dropout. In this paper, we address these limitations by introducing LGDiST, the first reference-free latent gene diffusion model for ST data dropout. We show that LGDiST outperforms the previous state-of-the-art in gene expression completion, with an average Mean Squared Error that is 18% lower across 26 datasets. Furthermore, we demonstrate that completing ST data with LGDiST improves gene expression prediction performance on six state-of-the-art methods up to 10% in MSE. A key innovation of LGDiST is using context genes previously considered uninformative to build a rich and biologically meaningful genetic latent space. Our experiments show that removing key components of LGDiST, such as the context genes, the ST latent space, and the neighbor conditioning, leads to considerable drops in performance. These findings underscore that the full architecture of LGDiST achieves substantially better performance than any of its isolated components.

Method: The study evaluates YOLOv5, YOLOv8, YOLOv11, and Deformable DETR detectors on KITTI, BDD100K, and nuScenes datasets, analyzing trade-offs between detection accuracy, computational cost, and resource usage under various conditions including different resolutions, batch sizes, weather/lighting, and dynamic client participation.

Result: The paper provides a holistic deployment-oriented evaluation that integrates model performance, system-level resource profiling, and environmental robustness for FL-based object detection in CAVs.

Conclusion: This work paves the way for robust federated learning deployment in connected autonomous vehicles by addressing critical deployment factors including data heterogeneity, constrained hardware, and environmental variability through systematic evaluation.

Abstract: Object detection is crucial for Connected Autonomous Vehicles (CAVs) to perceive their surroundings and make safe driving decisions. Centralized training of object detection models often achieves promising accuracy, fast convergence, and simplified training process, but it falls short in scalability, adaptability, and privacy-preservation. Federated learning (FL), by contrast, enables collaborative, privacy-preserving, and continuous training across naturally distributed CAV fleets. However, deploying FL in real-world CAVs remains challenging due to the substantial computational demands of training and inference, coupled with highly diverse operating conditions. Practical deployment must address three critical factors: (i) heterogeneity from non-IID data distributions, (ii) constrained onboard computing hardware, and (iii) environmental variability such as lighting and weather, alongside systematic evaluation to ensure reliable performance. This work introduces the first holistic deployment-oriented evaluation of FL-based object detection in CAVs, integrating model performance, system-level resource profiling, and environmental robustness. Using state-of-the-art detectors, YOLOv5, YOLOv8, YOLOv11, and Deformable DETR, evaluated on the KITTI, BDD100K, and nuScenes datasets, we analyze trade-offs between detection accuracy, computational cost, and resource usage under diverse resolutions, batch sizes, weather and lighting conditions, and dynamic client participation, paving the way for robust FL deployment in CAVs.

Method: Integration of geometric priors and constraints (depth information, surface normals, equivariance) into deep learning models to create geometry-aware architectures for 3D vision tasks.

Result: Enhanced accuracy and robustness in geometric representations for camera pose estimation, point cloud registration, depth prediction, and 3D reconstruction.

Conclusion: Combining geometric techniques with deep learning capabilities produces effective geometry-aware models suitable for real-world applications like cultural heritage preservation and VR/AR environments.

Abstract: Modern deep learning developments create new opportunities for 3D mapping technology, scene reconstruction pipelines, and virtual reality development. Despite advances in 3D deep learning technology, direct training of deep learning models on 3D data faces challenges due to the high dimensionality inherent in 3D data and the scarcity of labeled datasets. Structure-from-motion (SfM) and Simultaneous Localization and Mapping (SLAM) exhibit robust performance when applied to structured indoor environments but often struggle with ambiguous features in unstructured environments. These techniques often struggle to generate detailed geometric representations effective for downstream tasks such as rendering and semantic analysis. Current limitations require the development of 3D representation methods that combine traditional geometric techniques with deep learning capabilities to generate robust geometry-aware deep learning models. The dissertation provides solutions to the fundamental challenges in 3D vision by developing geometric deep learning methods tailored for essential tasks such as camera pose estimation, point cloud registration, depth prediction, and 3D reconstruction. The integration of geometric priors or constraints, such as including depth information, surface normals, and equivariance into deep learning models, enhances both the accuracy and robustness of geometric representations. This study systematically investigates key components of 3D vision, including camera pose estimation, point cloud registration, depth estimation, and high-fidelity 3D reconstruction, demonstrating their effectiveness across real-world applications such as digital cultural heritage preservation and immersive VR/AR environments.

Method: Developed a scene classification framework using transfer learning with five state-of-the-art CNN architectures (VGG-16, ResNet50, MobileNetV2, DenseNet121) and Vision Transformer, trained on 500,000+ seasonally varied RGB images from USGS.

Result: DenseNet121 achieved the highest validation performance with R2 score of 0.89 for predicting CDOM, demonstrating strong capability for real-world water quality monitoring across diverse conditions.

Conclusion: The framework shows promise for practical water quality monitoring but requires future improvements for low-light and obstructed scenarios to expand operational utility.

Abstract: Ongoing advancements in computer vision, particularly in pattern recognition and scene classification, have enabled new applications in environmental monitoring. Deep learning now offers non-contact methods for assessing water quality and detecting contamination, both critical for disaster response and public health protection. This work introduces HydroVision, a deep learning-based scene classification framework that estimates optically active water quality parameters including Chlorophyll-Alpha, Chlorophylls, Colored Dissolved Organic Matter (CDOM), Phycocyanins, Suspended Sediments, and Turbidity from standard Red-Green-Blue (RGB) images of surface water. HydroVision supports early detection of contamination trends and strengthens monitoring by regulatory agencies during external environmental stressors, industrial activities, and force majeure events. The model is trained on more than 500,000 seasonally varied images collected from the United States Geological Survey Hydrologic Imagery Visualization and Information System between 2022 and 2024. This approach leverages widely available RGB imagery as a scalable, cost-effective alternative to traditional multispectral and hyperspectral remote sensing. Four state-of-the-art convolutional neural networks (VGG-16, ResNet50, MobileNetV2, DenseNet121) and a Vision Transformer are evaluated through transfer learning to identify the best-performing architecture. DenseNet121 achieves the highest validation performance, with an R2 score of 0.89 in predicting CDOM, demonstrating the framework’s promise for real-world water quality monitoring across diverse conditions. While the current model is optimized for well-lit imagery, future work will focus on improving robustness under low-light and obstructed scenarios to expand its operational utility.

Method: Proposed two pipelines: VLM only (Pipeline A) and VLM + large language model (Pipeline B) with structured prompts based on wildfire damage indicators. Applied to 2025 Eaton and Palisades fires in California using multi-view image analysis.

Result: Single-view assessments performed poorly (F1 scores 0.225-0.511) while multi-view analysis showed dramatic improvements (F1 scores 0.857-0.947). McNemar test confirmed multi-view yields statistically significant improvements. No significant difference between Pipeline A and B.

Conclusion: VLMs effectively synthesize multi-perspective information for nuanced damage identification. Framework provides immediately deployable, flexible workflow without supervised training, accelerating disaster response triage and prioritization.

Abstract: The escalating intensity and frequency of wildfires demand innovative computational methods for rapid and accurate property damage assessment. Traditional methods are often time consuming, while modern computer vision approaches typically require extensive labeled datasets, hindering immediate post-disaster deployment. This research introduces a novel, zero-shot framework leveraging pre-trained vision language models (VLMs) to classify damage from ground-level imagery. We propose and evaluate two pipelines applied to the 2025 Eaton and Palisades fires in California, a VLM (Pipeline A) and a VLM + large language model (LLM) approach (Pipeline B), that integrate structured prompts based on specific wildfire damage indicators. A primary scientific contribution of this study is demonstrating the VLMs efficacy in synthesizing information from multiple perspectives to identify nuanced damage, a critical limitation in existing literature. Our findings reveal that while single view assessments struggled to classify affected structures (F1 scores ranging from 0.225 to 0.511), the multi-view analysis yielded dramatic improvements (F1 scores ranging from 0.857 to 0.947). Moreover, the McNemar test confirmed that pipelines with a multi-view image assessment yields statistically significant classification improvements; however, the improvements this research observed between Pipeline A and B were not statistically significant. Thus, future research can explore the potential of LLM prompting in damage assessment. The practical contribution is an immediately deployable, flexible, and interpretable workflow that bypasses the need for supervised training, significantly accelerating triage and prioritization for disaster response practitioners.

Method: Gaussian quantization representation learning method for diffusion models with an overfitting monitoring mechanism during training, tested on a new multi-source drone infrared benchmark dataset.

Result: Outperforms existing super-resolution approaches and significantly mitigates overfitting of large-scale architectures under complex conditions with few training samples.

Conclusion: The proposed method effectively reduces overfitting while maintaining architecture complexity, providing a robust solution for few-shot drone-based infrared image reconstruction.

Abstract: Although large scale models achieve significant improvements in performance, the overfitting challenge still frequently undermines their generalization ability. In super resolution tasks on images, diffusion models as representatives of generative models typically adopt large scale architectures. However, few-shot drone-captured infrared training data frequently induces severe overfitting in large-scale architectures. To address this key challenge, our method proposes a new Gaussian quantization representation learning method oriented to diffusion models that alleviates overfitting and enhances robustness. At the same time, an effective monitoring mechanism tracks large scale architectures during training to detect signs of overfitting. By introducing Gaussian quantization representation learning, our method effectively reduces overfitting while maintaining architecture complexity. On this basis, we construct a multi source drone-based infrared image benchmark dataset for detection and use it to emphasize overfitting issues of large scale architectures in few sample, drone-based diverse drone-based image reconstruction scenarios. To verify the efficacy of the method in mitigating overfitting, experiments are conducted on the constructed benchmark. Experimental results demonstrate that our method outperforms existing super resolution approaches and significantly mitigates overfitting of large scale architectures under complex conditions. The code and DroneSR dataset will be available at: https://github.com/wengzp1/GARLSR.

Method: Proposes a Concept-Aware Alignment Module that jointly projects image and location embeddings onto a shared bank of geographic concepts (e.g., tropical climate, mountain, cathedral) and minimizes concept-level loss for better alignment.

Result: Extensive experiments show the approach surpasses GeoCLIP in geo-localization accuracy and improves performance across diverse geospatial prediction tasks while providing richer semantic insights.

Conclusion: This is the first work to introduce interpretability into geo-localization, achieving both better performance and enhanced understanding of geographic decision-making processes.

Abstract: Worldwide geo-localization involves determining the exact geographic location of images captured globally, typically guided by geographic cues such as climate, landmarks, and architectural styles. Despite advancements in geo-localization models like GeoCLIP, which leverages images and location alignment via contrastive learning for accurate predictions, the interpretability of these models remains insufficiently explored. Current concept-based interpretability methods fail to align effectively with Geo-alignment image-location embedding objectives, resulting in suboptimal interpretability and performance. To address this gap, we propose a novel framework integrating global geo-localization with concept bottlenecks. Our method inserts a Concept-Aware Alignment Module that jointly projects image and location embeddings onto a shared bank of geographic concepts (e.g., tropical climate, mountain, cathedral) and minimizes a concept-level loss, enhancing alignment in a concept-specific subspace and enabling robust interpretability. To our knowledge, this is the first work to introduce interpretability into geo-localization. Extensive experiments demonstrate that our approach surpasses GeoCLIP in geo-localization accuracy and boosts performance across diverse geospatial prediction tasks, revealing richer semantic insights into geographic decision-making processes.

Method: Leverages advanced diffusion techniques to synthesize high-fidelity continuous traces that maintain temporal patterns and device relationships while following user-specified configurations.

Result: Experimental results show DiTTO generates traces with high fidelity and diversity while closely aligning with guided configurations with only 8% errors.

Conclusion: DiTTO successfully demonstrates that diffusion models can effectively generate realistic and precisely configurable multi-device storage traces, providing a valuable tool for storage system research and development.

Abstract: We propose DiTTO, a novel diffusion-based framework for generating realistic, precisely configurable, and diverse multi-device storage traces. Leveraging advanced diffusion tech- niques, DiTTO enables the synthesis of high-fidelity continuous traces that capture temporal dynamics and inter-device dependencies with user-defined configurations. Our experimental results demonstrate that DiTTO can generate traces with high fidelity and diversity while aligning closely with guided configurations with only 8% errors.

Method: Proposed contrastive learning approach with two specialized loss functions that leverage the inherent structural properties of diagrams, using hard samples for training.

Result: Substantial improvements over standard CLIP and conventional hard negative CLIP learning on flowchart benchmarks for both image-text matching and visual question answering tasks.

Conclusion: Tailored training strategies are crucial for specialized tasks, and this approach advances diagrammatic understanding in vision-language integration.

Abstract: Multimodal models, such as the Contrastive Language-Image Pre-training (CLIP) model, have demonstrated remarkable success in aligning visual and linguistic representations. However, these models exhibit limitations when applied to specialised visual domains, such as diagrams, which encode structured, symbolic information distinct from that of natural imagery. In this paper, we introduce a novel training paradigm explicitly designed to enhance the comprehension of diagrammatic images within vision-language models. Our approach uses ``hard’’ samples for our proposed contrastive learning that incorporates two specialised loss functions that leverage the inherent structural properties of diagrams. By integrating these objectives into model training, our method enables models to develop a more structured and semantically coherent understanding of diagrammatic content. We empirically validate our approach on a benchmark dataset of flowcharts, as a representative class of diagrammatic imagery, demonstrating substantial improvements over standard CLIP and conventional hard negative CLIP learning paradigms for both image-text matching and visual question answering tasks. Our findings underscore the significance of tailored training strategies for specialised tasks and contribute to advancing diagrammatic understanding within the broader landscape of vision-language integration.

Method: Encodes incomplete images into continuous 2D Gaussian splat coefficients, uses differentiable rasterization with patch-wise strategy for efficiency, and incorporates DINO features for semantic guidance.

Result: Achieves competitive performance on standard benchmarks in both quantitative metrics and perceptual quality.

Conclusion: Establishes a new direction for applying Gaussian Splatting to 2D image processing with promising inpainting results.

Abstract: Gaussian Splatting (GS), a recent technique for converting discrete points into continuous spatial representations, has shown promising results in 3D scene modeling and 2D image super-resolution. In this paper, we explore its untapped potential for image inpainting, which demands both locally coherent pixel synthesis and globally consistent semantic restoration. We propose the first image inpainting framework based on 2D Gaussian Splatting, which encodes incomplete images into a continuous field of 2D Gaussian splat coefficients and reconstructs the final image via a differentiable rasterization process. The continuous rendering paradigm of GS inherently promotes pixel-level coherence in the inpainted results. To improve efficiency and scalability, we introduce a patch-wise rasterization strategy that reduces memory overhead and accelerates inference. For global semantic consistency, we incorporate features from a pretrained DINO model. We observe that DINO’s global features are naturally robust to small missing regions and can be effectively adapted to guide semantic alignment in large-mask scenarios, ensuring that the inpainted content remains contextually consistent with the surrounding scene. Extensive experiments on standard benchmarks demonstrate that our method achieves competitive performance in both quantitative metrics and perceptual quality, establishing a new direction for applying Gaussian Splatting to 2D image processing.

Method: Introduces Draw-In-Mind (DIM) dataset with two subsets: DIM-T2I (14M image-text pairs for instruction comprehension) and DIM-Edit (233K chain-of-thought imaginations as design blueprints). Connects frozen Qwen2.5-VL-3B with trainable SANA1.5-1.6B via lightweight MLP, trained on DIM dataset.

Result: DIM-4.6B-Edit achieves SOTA or competitive performance on ImgEdit and GEdit-Bench benchmarks, outperforming much larger models like UniWorld-V1 and Step1X-Edit despite modest parameter scale.

Conclusion: Explicitly assigning design responsibility to the understanding module provides significant benefits for image editing tasks, demonstrating that better task division rather than simply scaling model size can lead to superior performance.

Abstract: In recent years, integrating multimodal understanding and generation into a single unified model has emerged as a promising paradigm. While this approach achieves strong results in text-to-image (T2I) generation, it still struggles with precise image editing. We attribute this limitation to an imbalanced division of responsibilities. The understanding module primarily functions as a translator that encodes user instructions into semantic conditions, while the generation module must simultaneously act as designer and painter, inferring the original layout, identifying the target editing region, and rendering the new content. This imbalance is counterintuitive because the understanding module is typically trained with several times more data on complex reasoning tasks than the generation module. To address this issue, we introduce Draw-In-Mind (DIM), a dataset comprising two complementary subsets: (i) DIM-T2I, containing 14M long-context image-text pairs to enhance complex instruction comprehension; and (ii) DIM-Edit, consisting of 233K chain-of-thought imaginations generated by GPT-4o, serving as explicit design blueprints for image edits. We connect a frozen Qwen2.5-VL-3B with a trainable SANA1.5-1.6B via a lightweight two-layer MLP, and train it on the proposed DIM dataset, resulting in DIM-4.6B-T2I/Edit. Despite its modest parameter scale, DIM-4.6B-Edit achieves SOTA or competitive performance on the ImgEdit and GEdit-Bench benchmarks, outperforming much larger models such as UniWorld-V1 and Step1X-Edit. These findings demonstrate that explicitly assigning the design responsibility to the understanding module provides significant benefits for image editing. Our dataset and models will be available at https://github.com/showlab/DIM.

Method: Ensemble approach combining sequence-matched results from multiple event-to-frame reconstructions, VPR feature extractors, and temporal resolutions with broader fusion strategy.

Result: 57% relative improvement in Recall@1 across day-night transitions on long-term driving datasets (8km per traverse) without metric subsampling.

Conclusion: The ensemble fusion strategy significantly improves robustness to varied lighting conditions, and comprehensive analysis identifies critical components for robust performance.

Abstract: Compared to conventional cameras, event cameras provide a high dynamic range and low latency, offering greater robustness to rapid motion and challenging lighting conditions. Although the potential of event cameras for visual place recognition (VPR) has been established, developing robust VPR frameworks under severe illumination changes remains an open research problem. In this paper, we introduce an ensemble-based approach to event camera place recognition that combines sequence-matched results from multiple event-to-frame reconstructions, VPR feature extractors, and temporal resolutions. Unlike previous event-based ensemble methods, which only utilise temporal resolution, our broader fusion strategy delivers significantly improved robustness under varied lighting conditions (e.g., afternoon, sunset, night), achieving a 57% relative improvement in Recall@1 across day-night transitions. We evaluate our approach on two long-term driving datasets (with 8 km per traverse) without metric subsampling, thereby preserving natural variations in speed and stop duration that influence event density. We also conduct a comprehensive analysis of key design choices, including binning strategies, polarity handling, reconstruction methods, and feature extractors, to identify the most critical components for robust performance. Additionally, we propose a modification to the standard sequence matching framework that enhances performance at longer sequence lengths. To facilitate future research, we will release our codebase and benchmarking framework.

Method: Introduces SemAlign-MS dataset with fine-grained semantic correspondences, semantic correspondence attention loss for precise point-to-point alignment, and multi-reference disentanglement loss to push subjects into orthogonal attention subspaces.

Result: Achieves state-of-the-art performance on multiple benchmarks and maintains high fidelity with 4+ reference subjects, outperforming methods that typically degrade beyond 3 subjects.

Conclusion: MOSAIC enables complex multi-subject synthesis with precise identity preservation and semantic coherence, opening new possibilities for applications requiring multiple reference subjects.

Abstract: Multi-subject personalized generation presents unique challenges in maintaining identity fidelity and semantic coherence when synthesizing images conditioned on multiple reference subjects. Existing methods often suffer from identity blending and attribute leakage due to inadequate modeling of how different subjects should interact within shared representation spaces. We present MOSAIC, a representation-centric framework that rethinks multi-subject generation through explicit semantic correspondence and orthogonal feature disentanglement. Our key insight is that multi-subject generation requires precise semantic alignment at the representation level - knowing exactly which regions in the generated image should attend to which parts of each reference. To enable this, we introduce SemAlign-MS, a meticulously annotated dataset providing fine-grained semantic correspondences between multiple reference subjects and target images, previously unavailable in this domain. Building on this foundation, we propose the semantic correspondence attention loss to enforce precise point-to-point semantic alignment, ensuring high consistency from each reference to its designated regions. Furthermore, we develop the multi-reference disentanglement loss to push different subjects into orthogonal attention subspaces, preventing feature interference while preserving individual identity characteristics. Extensive experiments demonstrate that MOSAIC achieves state-of-the-art performance on multiple benchmarks. Notably, while existing methods typically degrade beyond 3 subjects, MOSAIC maintains high fidelity with 4+ reference subjects, opening new possibilities for complex multi-subject synthesis applications.

Method: Proposes VARIN with Location-aware Argmax Inversion (LAI) to generate inverse Gumbel noises, enabling precise image reconstruction and targeted edits aligned with textual prompts.

Result: Extensive experiments show VARIN effectively modifies source images according to text prompts while preserving original background and structural details.

Conclusion: VARIN validates VAR models’ capability for practical image editing applications through noise inversion techniques, offering precise control while maintaining image integrity.

Abstract: Visual autoregressive models (VAR) have recently emerged as a promising class of generative models, achieving performance comparable to diffusion models in text-to-image generation tasks. While conditional generation has been widely explored, the ability to perform prompt-guided image editing without additional training is equally critical, as it supports numerous practical real-world applications. This paper investigates the text-to-image editing capabilities of VAR by introducing Visual AutoRegressive Inverse Noise (VARIN), the first noise inversion-based editing technique designed explicitly for VAR models. VARIN leverages a novel pseudo-inverse function for argmax sampling, named Location-aware Argmax Inversion (LAI), to generate inverse Gumbel noises. These inverse noises enable precise reconstruction of the source image and facilitate targeted, controllable edits aligned with textual prompts. Extensive experiments demonstrate that VARIN effectively modifies source images according to specified prompts while significantly preserving the original background and structural details, thus validating its efficacy as a practical editing approach.

Method: Building on existing observations by integrating synthetic hard negative samples into vision transformer representation learning frameworks.

Result: Notable improvement in discriminative power of learned representations, with performance gains demonstrated for both DeiT-S and Swin-T architectures.

Conclusion: A simple yet effective technique that enhances vision transformer performance through strategic use of synthetic hard negative samples in self-supervised learning.

Abstract: This paper does not introduce a novel method per se. Instead, we address the neglected potential of hard negative samples in self-supervised learning. Previous works explored synthetic hard negatives but rarely in the context of vision transformers. We build on this observation and integrate synthetic hard negatives to improve vision transformer representation learning. This simple yet effective technique notably improves the discriminative power of learned representations. Our experiments show performance improvements for both DeiT-S and Swin-T architectures.

Method: Categorizes XAI techniques into perturbation-based, gradient-based, backpropagation-based, and graph-based methods. Analyzes specific methods like D-RISE, BODEM, D-CLOSE, and FSOD, and investigates their applicability to various object detection architectures.

Result: Statistical analysis shows accelerating interest in explainable object detection from 2022 to mid-2025. The review provides a structured taxonomy and critical assessment of existing methods.

Conclusion: This review serves as a guide for researchers and practitioners to select suitable explainability techniques and promotes the development of more interpretable AI systems for object detection applications.

Abstract: In recent years, deep learning has achieved unprecedented success in various computer vision tasks, particularly in object detection. However, the black-box nature and high complexity of deep neural networks pose significant challenges for interpretability, especially in critical domains such as autonomous driving, medical imaging, and security systems. Explainable Artificial Intelligence (XAI) aims to address this challenge by providing tools and methods to make model decisions more transparent, interpretable, and trust-worthy for humans. This review provides a comprehensive analysis of state-of-the-art explain-ability methods specifically applied to object detection models. The paper be-gins by categorizing existing XAI techniques based on their underlying mechanisms-perturbation-based, gradient-based, backpropagation-based, and graph-based methods. Notable methods such as D-RISE, BODEM, D-CLOSE, and FSOD are discussed in detail. Furthermore, the paper investigates their applicability to various object detection architectures, including YOLO, SSD, Faster R-CNN, and EfficientDet. Statistical analysis of publication trends from 2022 to mid-2025 shows an accelerating interest in explainable object detection, indicating its increasing importance. The study also explores common datasets and evaluation metrics, and highlights the major challenges associated with model interpretability. By providing a structured taxonomy and a critical assessment of existing methods, this review aims to guide researchers and practitioners in selecting suitable explainability techniques for object detection applications and to foster the development of more interpretable AI systems.

Method: Combines two approaches: 1) using generative models to create synthetic data for sample diversity, and 2) generating synthetic hard negatives in representation space. Evaluated on DeiT-S and Swin-T architectures.

Result: The framework demonstrates both promise and limitations of synthetic data in self-supervised learning, providing insights for future synthetic-enhanced training approaches.

Conclusion: Synthetic data augmentation and synthetic hard negative generation show potential for creating more robust and transferable visual representations, though with identified limitations that need addressing in future work.

Abstract: This paper does not introduce a new method per se. Instead, we build on existing self-supervised learning approaches for vision, drawing inspiration from the adage “fake it till you make it”. While contrastive self-supervised learning has achieved remarkable success, it typically relies on vast amounts of real-world data and carefully curated hard negatives. To explore alternatives to these requirements, we investigate two forms of “faking it” in vision transformers. First, we study the potential of generative models for unsupervised representation learning, leveraging synthetic data to augment sample diversity. Second, we examine the feasibility of generating synthetic hard negatives in the representation space, creating diverse and challenging contrasts. Our framework - dubbed Syn2Co - combines both approaches and evaluates whether synthetically enhanced training can lead to more robust and transferable visual representations on DeiT-S and Swin-T architectures. Our findings highlight the promise and limitations of synthetic data in self-supervised learning, offering insights for future work in this direction.

Method: Interprets palettes as sparse histograms with two control parameters (histogram entropy and palette-to-histogram distance), introduces negative histogram mechanism to suppress undesired hues, and trains on a curated dataset with balanced color coverage.

Result: Outperforms existing approaches in achieving both strong palette adherence and high image quality across a wide range of palettes and prompts, as validated through qualitative, quantitative, and user study evaluations.

Conclusion: The Palette-Adapter provides stable, semantically coherent generation with flexible palette control, addressing the challenge of palette conditioning in text-to-image diffusion models.

Abstract: We introduce the Palette-Adapter, a novel method for conditioning text-to-image diffusion models on a user-specified color palette. While palettes are a compact and intuitive tool widely used in creative workflows, they introduce significant ambiguity and instability when used for conditioning image generation. Our approach addresses this challenge by interpreting palettes as sparse histograms and introducing two scalar control parameters: histogram entropy and palette-to-histogram distance, which allow flexible control over the degree of palette adherence and color variation. We further introduce a negative histogram mechanism that allows users to suppress specific undesired hues, improving adherence to the intended palette under the standard classifier-free guidance mechanism. To ensure broad generalization across the color space, we train on a carefully curated dataset with balanced coverage of rare and common colors. Our method enables stable, semantically coherent generation across a wide range of palettes and prompts. We evaluate our method qualitatively, quantitatively, and through a user study, and show that it consistently outperforms existing approaches in achieving both strong palette adherence and high image quality.

Method: Developed an embedded monitoring system using quantized MobileNet model deployed on Raspberry Pi for real-time behavioral state detection. The framework includes model quantization (68% size reduction), Raspberry Pi-optimized vision pipelines, and secure IoT communication for clinical alerts.

Result: Achieved state-of-the-art accuracy: 91.8% for sleep detection and 97.7% for crying/normal classification. The system maintains computational efficiency suitable for edge deployment while larger models like ResNet152 and VGG19 proved computationally prohibitive despite marginal accuracy gains.

Conclusion: Lightweight, optimized models like MobileNet provide the most viable foundation for scalable, low-cost NICU monitoring systems, enabling improved preterm care in resource-constrained environments through non-invasive, automated vision-based monitoring.

Abstract: Preterm birth remains a leading cause of neonatal mortality, disproportionately affecting low-resource settings with limited access to advanced neonatal intensive care units (NICUs).Continuous monitoring of infant behavior, such as sleep/awake states and crying episodes, is critical but relies on manual observation or invasive sensors, which are prone to error, impractical, and can cause skin damage. This paper presents a novel, noninvasive, and automated vision-based framework to address this gap. We introduce an embedded monitoring system that utilizes a quantized MobileNet model deployed on a Raspberry Pi for real-time behavioral state detection. When trained and evaluated on public neonatal image datasets, our system achieves state-of-the-art accuracy (91.8% for sleep detection and 97.7% for crying/normal classification) while maintaining computational efficiency suitable for edge deployment. Through comparative benchmarking, we provide a critical analysis of the trade-offs between model size, inference latency, and diagnostic accuracy. Our findings demonstrate that while larger architectures (e.g., ResNet152, VGG19) offer marginal gains in accuracy, their computational cost is prohibitive for real-time edge use. The proposed framework integrates three key innovations: model quantization for memory-efficient inference (68% reduction in size), Raspberry Pi-optimized vision pipelines, and secure IoT communication for clinical alerts. This work conclusively shows that lightweight, optimized models such as the MobileNet offer the most viable foundation for scalable, low-cost, and clinically actionable NICU monitoring systems, paving the way for improved preterm care in resource-constrained environments.

Method: VEIL framework targets unified referring-matching mechanisms of RMOT models, exploiting FIFO-based memory vulnerabilities through spatial-temporal reasoning attacks that persist in history buffers over multiple frames.

Result: Comprehensive evaluations on Refer-KITTI dataset show VEIL effectively corrupts tracking logic reliability, inducing track ID switches and terminations through both digital and physical perturbations.

Conclusion: The study demonstrates urgent need for security-aware RMOT designs for critical large-scale applications, revealing persistent vulnerabilities in advanced RMOT models with FIFO-based memory.

Abstract: Language-vision understanding has driven the development of advanced perception systems, most notably the emerging paradigm of Referring Multi-Object Tracking (RMOT). By leveraging natural-language queries, RMOT systems can selectively track objects that satisfy a given semantic description, guided through Transformer-based spatial-temporal reasoning modules. End-to-End (E2E) RMOT models further unify feature extraction, temporal memory, and spatial reasoning within a Transformer backbone, enabling long-range spatial-temporal modeling over fused textual-visual representations. Despite these advances, the reliability and robustness of RMOT remain underexplored. In this paper, we examine the security implications of RMOT systems from a design-logic perspective, identifying adversarial vulnerabilities that compromise both the linguistic-visual referring and track-object matching components. Additionally, we uncover a novel vulnerability in advanced RMOT models employing FIFO-based memory, whereby targeted and consistent attacks on their spatial-temporal reasoning introduce errors that persist within the history buffer over multiple subsequent frames. We present VEIL, a novel adversarial framework designed to disrupt the unified referring-matching mechanisms of RMOT models. We show that carefully crafted digital and physical perturbations can corrupt the tracking logic reliability, inducing track ID switches and terminations. We conduct comprehensive evaluations using the Refer-KITTI dataset to validate the effectiveness of VEIL and demonstrate the urgent need for security-aware RMOT designs for critical large-scale applications.

Method: Introduces ContextFusion stage to exploit semantic information from foreground/background with auxiliary indicators, and Bootstrap Branch to decouple feature adaptation from reconstruction using bootstrap strategy.

Result: Significantly improves performance of different state-of-the-art slot attention models on both simulated and real-world datasets.

Conclusion: The proposed enhancements effectively address key limitations in slot attention methods, enabling better semantic understanding and more flexible adaptation for improved object-centric learning.

Abstract: A key human ability is to decompose a scene into distinct objects and use their relationships to understand the environment. Object-centric learning aims to mimic this process in an unsupervised manner. Recently, the slot attention-based framework has emerged as a leading approach in this area and has been widely used in various downstream tasks. However, existing slot attention methods face two key limitations: (1) a lack of high-level semantic information. In current methods, image areas are assigned to slots based on low-level features such as color and texture. This makes the model overly sensitive to low-level features and limits its understanding of object contours, shapes, or other semantic characteristics. (2) The inability to fine-tune the encoder. Current methods require a stable feature space throughout training to enable reconstruction from slots, which restricts the flexibility needed for effective object-centric learning. To address these limitations, we propose a novel ContextFusion stage and a Bootstrap Branch, both of which can be seamlessly integrated into existing slot attention models. In the ContextFusion stage, we exploit semantic information from the foreground and background, incorporating an auxiliary indicator that provides additional contextual cues about them to enrich the semantic content beyond low-level features. In the Bootstrap Branch, we decouple feature adaptation from the original reconstruction phase and introduce a bootstrap strategy to train a feature-adaptive mechanism, allowing for more flexible adaptation. Experimental results show that our method significantly improves the performance of different SOTA slot attention models on both simulated and real-world datasets.

Method: Proposes SALAD with a novel composition branch to explicitly model object composition map distributions, learning semantic relationships. Introduces a new procedure for extracting composition maps without hand-made labels or category-specific information.

Result: Achieves state-of-the-art performance on MVTec LOCO benchmark with 96.1% image-level AUROC, significantly outperforming previous methods.

Conclusion: SALAD effectively addresses logical anomaly detection by preserving semantic and spatial information through composition map modeling, demonstrating substantial improvements over existing approaches.

Abstract: Recent surface anomaly detection methods excel at identifying structural anomalies, such as dents and scratches, but struggle with logical anomalies, such as irregular or missing object components. The best-performing logical anomaly detection approaches rely on aggregated pretrained features or handcrafted descriptors (most often derived from composition maps), which discard spatial and semantic information, leading to suboptimal performance. We propose SALAD, a semantics-aware discriminative logical anomaly detection method that incorporates a newly proposed composition branch to explicitly model the distribution of object composition maps, consequently learning important semantic relationships. Additionally, we introduce a novel procedure for extracting composition maps that requires no hand-made labels or category-specific information, in contrast to previous methods. By effectively modelling the composition map distribution, SALAD significantly improves upon state-of-the-art methods on the standard benchmark for logical anomaly detection, MVTec LOCO, achieving an impressive image-level AUROC of 96.1%. Code: https://github.com/MaticFuc/SALAD

Method: 1) Protocol to identify weakly recognized attributes 2) Prompt-driven pipeline using diffusion models to generate synthetic pedestrian images 3) Strategy to incorporate synthetic samples with prompt-based annotation rules and modified loss function

Result: Approach boosts recognition of underrepresented attributes and improves overall model performance beyond targeted attributes on popular PAR datasets, while strengthening zero-shot generalization.

Conclusion: Efficient and scalable solution that improves pedestrian attribute recognition in real-world scenarios without requiring architectural changes to existing models.

Abstract: Pedestrian Attribute Recognition (PAR) is a challenging task as models are required to generalize across numerous attributes in real-world data. Traditional approaches focus on complex methods, yet recognition performance is often constrained by training dataset limitations, particularly the under-representation of certain attributes. In this paper, we propose a data-centric approach to improve PAR by synthetic data augmentation guided by textual descriptions. First, we define a protocol to identify weakly recognized attributes across multiple datasets. Second, we propose a prompt-driven pipeline that leverages diffusion models to generate synthetic pedestrian images while preserving the consistency of PAR datasets. Finally, we derive a strategy to seamlessly incorporate synthetic samples into training data, which considers prompt-based annotation rules and modifies the loss function. Results on popular PAR datasets demonstrate that our approach not only boosts recognition of underrepresented attributes but also improves overall model performance beyond the targeted attributes. Notably, this approach strengthens zero-shot generalization without requiring architectural changes of the model, presenting an efficient and scalable solution to improve the recognition of attributes of pedestrians in the real world.

Method: Uses Graph Neural Network framework with non-overlapping subclips and novel Autoregressive Long-term Tracking layer, allowing adjustable subclip size for latency-context tradeoff.

Result: State-of-the-art performance: +2.3 AssA on DanceTrack, +9.2 on SportsMOT, +5.0 on MOT20 in online mode, with even better offline results.

Conclusion: NOOUGAT successfully unifies online and offline tracking with flexible temporal processing, overcoming limitations of both traditional approaches.

Abstract: The long-standing division between \textit{online} and \textit{offline} Multi-Object Tracking (MOT) has led to fragmented solutions that fail to address the flexible temporal requirements of real-world deployment scenarios. Current \textit{online} trackers rely on frame-by-frame hand-crafted association strategies and struggle with long-term occlusions, whereas \textit{offline} approaches can cover larger time gaps, but still rely on heuristic stitching for arbitrarily long sequences. In this paper, we introduce NOOUGAT, the first tracker designed to operate with arbitrary temporal horizons. NOOUGAT leverages a unified Graph Neural Network (GNN) framework that processes non-overlapping subclips, and fuses them through a novel Autoregressive Long-term Tracking (ALT) layer. The subclip size controls the trade-off between latency and temporal context, enabling a wide range of deployment scenarios, from frame-by-frame to batch processing. NOOUGAT achieves state-of-the-art performance across both tracking regimes, improving \textit{online} AssA by +2.3 on DanceTrack, +9.2 on SportsMOT, and +5.0 on MOT20, with even greater gains in \textit{offline} mode.

Method: SegFormerWithDropout architecture using MiT-B2 encoder pretrained on ImageNet, with dropout probability 0.3 in segmentation head. Trained on 500 dermoscopic images with hair mask annotations using 10-fold cross-validation, AdamW optimizer (lr=0.001), cross-entropy loss, and early stopping.

Result: Robust performance with average Dice coefficient ~0.96, IoU ~0.93, PSNR ~34 dB, SSIM 0.97, and low LPIPS 0.06, demonstrating accurate hair artifact segmentation.

Conclusion: The proposed method effectively segments hair artifacts in dermoscopic images and shows potential to enhance preprocessing for downstream skin cancer detection tasks.

Abstract: Hair artifacts in dermoscopic images present significant challenges for accurate skin lesion analysis, potentially obscuring critical diagnostic features in dermatological assessments. This work introduces a fine-tuned SegFormer model augmented with dropout regularization to achieve precise hair mask segmentation. The proposed SegformerWithDropout architecture leverages the MiT-B2 encoder, pretrained on ImageNet, with an in-channel count of 3 and 2 output classes, incorporating a dropout probability of 0.3 in the segmentation head to prevent overfitting. Training is conducted on a specialized dataset of 500 dermoscopic skin lesion images with fine-grained hair mask annotations, employing 10-fold cross-validation, AdamW optimization with a learning rate of 0.001, and cross-entropy loss. Early stopping is applied based on validation loss, with a patience of 3 epochs and a maximum of 20 epochs per fold. Performance is evaluated using a comprehensive suite of metrics, including Intersection over Union (IoU), Dice coefficient, Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS). Experimental results from the cross-validation demonstrate robust performance, with average Dice coefficients reaching approximately 0.96 and IoU values of 0.93, alongside favorable PSNR (around 34 dB), SSIM (0.97), and low LPIPS (0.06), highlighting the model’s effectiveness in accurate hair artifact segmentation and its potential to enhance preprocessing for downstream skin cancer detection tasks.

Method: The study investigates diffusion-based data expansion for PAR, identifying key parameters including text prompts, image properties, and latest diffusion-based augmentation enhancements. The best-performing approach is used to generate synthetic images to enrich zero-shot datasets.

Result: Experimental results show that prompt alignment and image properties are critical factors, with optimal selection leading to a 4.5% improvement in PAR recognition performance.

Conclusion: Diffusion models are effective for generating synthetic pedestrian images tailored to PAR tasks, and careful parameter selection can significantly enhance PAR model robustness and adaptability in real-world scenarios.

Abstract: Pedestrian Attribute Recognition (PAR) involves identifying various human attributes from images with applications in intelligent monitoring systems. The scarcity of large-scale annotated datasets hinders the generalization of PAR models, specially in complex scenarios involving occlusions, varying poses, and diverse environments. Recent advances in diffusion models have shown promise for generating diverse and realistic synthetic images, allowing to expand the size and variability of training data. However, the potential of diffusion-based data expansion for generating PAR-like images remains underexplored. Such expansion may enhance the robustness and adaptability of PAR models in real-world scenarios. This paper investigates the effectiveness of diffusion models in generating synthetic pedestrian images tailored to PAR tasks. We identify key parameters of img2img diffusion-based data expansion; including text prompts, image properties, and the latest enhancements in diffusion-based data augmentation, and examine their impact on the quality of generated images for PAR. Furthermore, we employ the best-performing expansion approach to generate synthetic images for training PAR models, by enriching the zero-shot datasets. Experimental results show that prompt alignment and image properties are critical factors in image generation, with optimal selection leading to a 4.5% improvement in PAR recognition performance.

Method: Introduced CFpano benchmark dataset with 2700+ images and 8000+ QA pairs. Developed a multi-modal LLM fine-tuned with Group Relative Policy Optimization (GRPO) and tailored reward functions for robust reasoning with cross-frame correlated panoramas.

Result: The proposed method achieves state-of-the-art performance, outperforming strong baselines by +5.37% in overall performance across both multiple-choice and open-ended VQA tasks. Experimental results demonstrate effectiveness across all major reasoning categories.

Conclusion: CFpano establishes a new benchmark for panoramic scene understanding. The GRPO-based multi-modal LLM effectively addresses cross-frame correlated panorama reasoning, validating the approach’s effectiveness and setting a new standard for 360° scene comprehension.

Abstract: Omnidirectional scene understanding is vital for various downstream applications, such as embodied AI, autonomous driving, and immersive environments, yet remains challenging due to geometric distortion and complex spatial relations in 360{\deg} imagery. Existing omnidirectional methods achieve scene understanding within a single frame while neglecting cross-frame correlated panoramas. To bridge this gap, we introduce \textbf{CFpano}, the \textbf{first} benchmark dataset dedicated to cross-frame correlated panoramas visual question answering in the holistic 360{\deg} scenes. CFpano consists of over 2700 images together with over 8000 question-answer pairs, and the question types include both multiple choice and open-ended VQA. Building upon our CFpano, we further present \methodname, a multi-modal large language model (MLLM) fine-tuned with Group Relative Policy Optimization (GRPO) and a set of tailored reward functions for robust and consistent reasoning with cross-frame correlated panoramas. Benchmark experiments with existing MLLMs are conducted with our CFpano. The experimental results demonstrate that \methodname achieves state-of-the-art performance across both multiple-choice and open-ended VQA tasks, outperforming strong baselines on all major reasoning categories (\textbf{+5.37%} in overall performance). Our analyses validate the effectiveness of GRPO and establish a new benchmark for panoramic scene understanding.

Method: Created the ITD dataset using tracklets (sequences of object-centric images) compiled from bounding boxes of an object-tracking dataset. Conducted experimental analysis of current TTA methods and proposed a novel adversarial memory initialization strategy to improve memory-based TTA approaches.

Result: The ITD dataset successfully captures natural temporal dependencies. Experimental analysis revealed limitations of current TTA methods when faced with temporal challenges. The proposed adversarial memory initialization strategy substantially boosted performance of various methods on the challenging benchmark.

Conclusion: The ITD dataset provides a more realistic benchmark for TTA methods by incorporating temporal dependencies. The proposed adversarial memory initialization effectively addresses the challenges posed by temporal patterns, significantly improving existing TTA method performance.

Abstract: We introduce a novel tracklet-based dataset for benchmarking test-time adaptation (TTA) methods. The aim of this dataset is to mimic the intricate challenges encountered in real-world environments such as images captured by hand-held cameras, self-driving cars, etc. The current benchmarks for TTA focus on how models face distribution shifts, when deployed, and on violations to the customary independent-and-identically-distributed (i.i.d.) assumption in machine learning. Yet, these benchmarks fail to faithfully represent realistic scenarios that naturally display temporal dependencies, such as how consecutive frames from a video stream likely show the same object across time. We address this shortcoming of current datasets by proposing a novel TTA benchmark we call the “Inherent Temporal Dependencies” (ITD) dataset. We ensure the instances in ITD naturally embody temporal dependencies by collecting them from tracklets-sequences of object-centric images we compile from the bounding boxes of an object-tracking dataset. We use ITD to conduct a thorough experimental analysis of current TTA methods, and shed light on the limitations of these methods when faced with the challenges of temporal dependencies. Moreover, we build upon these insights and propose a novel adversarial memory initialization strategy to improve memory-based TTA methods. We find this strategy substantially boosts the performance of various methods on our challenging benchmark.

Method: Computer vision pipeline that extracts key characteristics (principal line structures, texture, shape metrics) from palm images, trained on a novel dataset of annotated palm images using machine learning models.

Result: Demonstrates feasibility for digital anthropometry and personalized user analytics, with machine learning models successfully identifying complex patterns in palm data.

Conclusion: Opens avenues for research intersecting cultural practices with computational analysis, with potential for mobile platform deployment.

Abstract: This paper explores the automated analysis of palmar features using machine learning techniques. We present a computer vision pipeline that extracts key characteristics from palm images, such as principal line structures, texture, and shape metrics. These features are used to train predictive models on a novel dataset curated from annotated palm images. Our approach moves beyond traditional subjective interpretation by providing a data-driven, quantitative framework for studying the correlations between palmar morphology and externally validated traits or conditions. The methodology demonstrates feasibility for applications in digital anthropometry and personalized user analytics, with potential for deployment on mobile platforms. Results indicate that machine learning models can identify complex patterns in palm data, opening avenues for research that intersects cultural practices with computational analysis.

Method: Adaptive Bidirectional Pyramid Difference Convolution (ABPDC) module for multimodal integration using difference convolution advantages, and Feature Pyramid Registration Auxiliary Network (FPRAN) to address structural misalignment.

Result: Experiments on MMCSD dataset show superior prediction accuracy of postoperative neck pain recovery compared to existing methods, with ablation studies confirming effectiveness.

Conclusion: The proposed multimodal fusion approach with ABPDC and FPRAN effectively addresses imaging challenges and improves neck pain recovery prediction in cervical spondylosis.

Abstract: Neck pain is the primary symptom of cervical spondylosis, yet its underlying mechanisms remain unclear, leading to uncertain treatment outcomes. To address the challenges of multimodal feature fusion caused by imaging differences and spatial mismatches, this paper proposes an Adaptive Bidirectional Pyramid Difference Convolution (ABPDC) module that facilitates multimodal integration by exploiting the advantages of difference convolution in texture extraction and grayscale invariance, and a Feature Pyramid Registration Auxiliary Network (FPRAN) to mitigate structural misalignment. Experiments on the MMCSD dataset demonstrate that the proposed model achieves superior prediction accuracy of postoperative neck pain recovery compared with existing methods, and ablation studies further confirm its effectiveness.

Method: Proposes DSGC-Net with Density Approximation (DA) branch and Representation Approximation (RA) branch. DA branch generates density distribution maps and constructs density-driven semantic graphs. RA branch establishes representation-driven semantic graphs using global representation similarity. Both use graph convolutional networks to model latent semantic relationships.

Result: Achieves MAE of 48.9 on ShanghaiTech Part A and 5.9 on ShanghaiTech Part B datasets, outperforming current state-of-the-art methods across three widely used datasets.

Conclusion: DSGC-Net effectively addresses density variation adaptation and representation inconsistency challenges in crowd counting through dual-stream graph convolutional architecture with feature correlation mining, demonstrating superior performance over existing methods.

Abstract: Deep learning-based crowd counting methods have achieved remarkable progress in recent years. However, in complex crowd scenarios, existing models still face challenges when adapting to significant density distribution differences between regions. Additionally, the inconsistency of individual representations caused by viewpoint changes and body posture differences further limits the counting accuracy of the models. To address these challenges, we propose DSGC-Net, a Dual-Stream Graph Convolutional Network based on feature correlation mining. DSGC-Net introduces a Density Approximation (DA) branch and a Representation Approximation (RA) branch. By modeling two semantic graphs, it captures the potential feature correlations in density variations and representation distributions. The DA branch incorporates a density prediction module that generates the density distribution map, and constructs a density-driven semantic graph based on density similarity. The RA branch establishes a representation-driven semantic graph by computing global representation similarity. Then, graph convolutional networks are applied to the two semantic graphs separately to model the latent semantic relationships, which enhance the model’s ability to adapt to density variations and improve counting accuracy in multi-view and multi-pose scenarios. Extensive experiments on three widely used datasets demonstrate that DSGC-Net outperforms current state-of-the-art methods. In particular, we achieve MAE of 48.9 and 5.9 in ShanghaiTech Part A and Part B datasets, respectively. The released code is available at: https://github.com/Wu-eon/CrowdCounting-DSGCNet.

Method: Leverages remote sensing-specific vision-language modeling with three innovations: spatial feature enhancement, dual-prompt alignment mechanism for spatial-semantic consistency, and confidence-guided self-training loop for pseudo-label mining.

Result: Consistently outperforms existing methods across multiple remote sensing benchmarks and enables efficient adaptation with minimal labeled data.

Conclusion: Demonstrates the critical value of spatial-semantic integration for robust OOD detection in remote sensing applications.

Abstract: Out-of-distribution (OOD) detection represents a critical challenge in remote sensing applications, where reliable identification of novel or anomalous patterns is essential for autonomous monitoring, disaster response, and environmental assessment. Despite remarkable progress in OOD detection for natural images, existing methods and benchmarks remain poorly suited to remote sensing imagery due to data scarcity, complex multi-scale scene structures, and pronounced distribution shifts. To this end, we propose RS-OOD, a novel framework that leverages remote sensing-specific vision-language modeling to enable robust few-shot OOD detection. Our approach introduces three key innovations: spatial feature enhancement that improved scene discrimination, a dual-prompt alignment mechanism that cross-verifies scene context against fine-grained semantics for spatial-semantic consistency, and a confidence-guided self-training loop that dynamically mines pseudo-labels to expand training data without manual annotation. RS-OOD consistently outperforms existing methods across multiple remote sensing benchmarks and enables efficient adaptation with minimal labeled data, demonstrating the critical value of spatial-semantic integration.

Method: Self-supervised student-teacher architecture with ClassMix++ algorithm for blending synthetic data, Grounded Mask Consistency Loss for cross-domain alignment, and Pseudo-Label Guided Contrastive Learning for domain-invariant features.

Result: Outperforms state-of-the-art single-source methods, achieving 50% mIoU on real-world datasets like Indian Driving Dataset (IDD).

Conclusion: The approach effectively bridges sim-to-real domain gaps, reduces reliance on labeled target data, and improves generalization in complex urban environments.

Abstract: Unstructured urban environments present unique challenges for scene understanding and generalization due to their complex and diverse layouts. We introduce SynthGenNet, a self-supervised student-teacher architecture designed to enable robust test-time domain generalization using synthetic multi-source imagery. Our contributions include the novel ClassMix++ algorithm, which blends labeled data from various synthetic sources while maintaining semantic integrity, enhancing model adaptability. We further employ Grounded Mask Consistency Loss (GMC), which leverages source ground truth to improve cross-domain prediction consistency and feature alignment. The Pseudo-Label Guided Contrastive Learning (PLGCL) mechanism is integrated into the student network to facilitate domain-invariant feature learning through iterative knowledge distillation from the teacher network. This self-supervised strategy improves prediction accuracy, addresses real-world variability, bridges the sim-to-real domain gap, and reliance on labeled target data, even in complex urban areas. Outcomes show our model outperforms the state-of-the-art (relying on single source) by achieving 50% Mean Intersection-Over-Union (mIoU) value on real-world datasets like Indian Driving Dataset (IDD).

Method: Train lightweight classifiers to decode spatial relationships from diffusion model’s cross-attention maps, then use these classifiers as learned loss functions during inference. Uses dual-inversion strategy to prevent shortcuts and enforce geometric understanding.

Result: Dramatically improves spatial accuracy: from 0.20 to 0.61 on FLUX.1-dev and from 0.07 to 0.54 on SD2.1 across standard benchmarks. Generalizes to multiple relations with significant accuracy improvements.

Conclusion: Learning spatial objectives directly from model’s internal representations through data-driven classifiers is more effective than handcrafted losses, enabling substantial improvements in spatial reasoning for text-to-image generation.

Abstract: Text-to-image diffusion models can generate stunning visuals, yet they often fail at tasks children find trivial–like placing a dog to the right of a teddy bear rather than to the left. When combinations get more unusual–a giraffe above an airplane–these failures become even more pronounced. Existing methods attempt to fix these spatial reasoning failures through model fine-tuning or test-time optimization with handcrafted losses that are suboptimal. Rather than imposing our assumptions about spatial encoding, we propose learning these objectives directly from the model’s internal representations. We introduce Learn-to-Steer, a novel framework that learns data-driven objectives for test-time optimization rather than handcrafting them. Our key insight is to train a lightweight classifier that decodes spatial relationships from the diffusion model’s cross-attention maps, then deploy this classifier as a learned loss function during inference. Training such classifiers poses a surprising challenge: they can take shortcuts by detecting linguistic traces rather than learning true spatial patterns. We solve this with a dual-inversion strategy that enforces geometric understanding. Our method dramatically improves spatial accuracy: from 0.20 to 0.61 on FLUX.1-dev and from 0.07 to 0.54 on SD2.1 across standard benchmarks. Moreover, our approach generalizes to multiple relations and significantly improves accuracy.

Method: Testing CLIP’s color perception and color naming capabilities by playing the board game Hues & Cues, comparing its performance against human observers.

Result: CLIP shows general alignment with human color perception but reveals cultural biases and inconsistencies when handling different abstraction levels that are hard to detect with standard benchmarks.

Conclusion: Board games offer valuable alternative evaluation methods that can expose model deficiencies not apparent through conventional testing approaches, highlighting the importance of diverse assessment strategies.

Abstract: Playing games is inherently human, and a lot of games are created to challenge different human characteristics. However, these tasks are often left out when evaluating the human-like nature of artificial models. The objective of this work is proposing a new approach to evaluate artificial models via board games. To this effect, we test the color perception and color naming capabilities of CLIP by playing the board game Hues & Cues and assess its alignment with humans. Our experiments show that CLIP is generally well aligned with human observers, but our approach brings to light certain cultural biases and inconsistencies when dealing with different abstraction levels that are hard to identify with other testing strategies. Our findings indicate that assessing models with different tasks like board games can make certain deficiencies in the models stand out in ways that are difficult to test with the commonly used benchmarks.

Method: Proposed Layer-heterogeneity MoE architecture that separates deep-layer parameters to eliminate conflict between GUI and embodied data while sharing shallow-layer parameters to leverage their synergy. Also unified action spaces and collected large-scale GUI/embodied data from various sources.

Result: OmniActor outperforms agents trained only on GUI or embodied data in both GUI and embodied tasks. The approach significantly improves performance across different scenarios, especially in GUI tasks.

Conclusion: The cerebrum-cerebellum inspired architecture successfully resolves data conflicts while maintaining synergies between different environment types, enabling effective generalist agents that can operate across both GUI and embodied environments.

Abstract: Multimodal large language models are evolving toward multimodal agents capable of proactively executing tasks. Most agent research focuses on GUI or embodied scenarios, which correspond to agents interacting with 2D virtual worlds or 3D real worlds, respectively. However, many complex tasks typically require agents to interleavely interact with these two types of environment. We initially mix GUI and embodied data to train, but find the performance degeneration brought by the data conflict. Further analysis reveals that GUI and embodied data exhibit synergy and conflict at the shallow and deep layers, respectively, which resembles the cerebrum-cerebellum mechanism in the human brain. To this end, we propose a high-performance generalist agent OmniActor, designed from both structural and data perspectives. First, we propose Layer-heterogeneity MoE to eliminate the conflict between GUI and embodied data by separating deep-layer parameters, while leverage their synergy by sharing shallow-layer parameters. By successfully leveraging the synergy and eliminating the conflict, OmniActor outperforms agents only trained by GUI or embodied data in GUI or embodied tasks. Furthermore, we unify the action spaces of GUI and embodied tasks, and collect large-scale GUI and embodied data from various sources for training. This significantly improves OmniActor under different scenarios, especially in GUI tasks. The code will be publicly available.

Method: Proposes ORDAC (ORDinal Adaptive Correction) that uses Label Distribution Learning to dynamically adjust mean and standard deviation of label distributions during training, correcting noisy samples instead of discarding them.

Result: Significant improvements on Adience (age estimation) and Diabetic Retinopathy datasets - reduced MAE from 0.86 to 0.62 and increased recall from 0.37 to 0.49 with 40% noise on Adience.

Conclusion: Adaptive label correction using label distributions effectively enhances robustness and accuracy of ordinal classification models in noisy data scenarios.

Abstract: Labeled data is a fundamental component in training supervised deep learning models for computer vision tasks. However, the labeling process, especially for ordinal image classification where class boundaries are often ambiguous, is prone to error and noise. Such label noise can significantly degrade the performance and reliability of machine learning models. This paper addresses the problem of detecting and correcting label noise in ordinal image classification tasks. To this end, a novel data-centric method called ORDinal Adaptive Correction (ORDAC) is proposed for adaptive correction of noisy labels. The proposed approach leverages the capabilities of Label Distribution Learning (LDL) to model the inherent ambiguity and uncertainty present in ordinal labels. During training, ORDAC dynamically adjusts the mean and standard deviation of the label distribution for each sample. Rather than discarding potentially noisy samples, this approach aims to correct them and make optimal use of the entire training dataset. The effectiveness of the proposed method is evaluated on benchmark datasets for age estimation (Adience) and disease severity detection (Diabetic Retinopathy) under various asymmetric Gaussian noise scenarios. Results show that ORDAC and its extended versions (ORDAC_C and ORDAC_R) lead to significant improvements in model performance. For instance, on the Adience dataset with 40% noise, ORDAC_R reduced the mean absolute error from 0.86 to 0.62 and increased the recall metric from 0.37 to 0.49. The method also demonstrated its effectiveness in correcting intrinsic noise present in the original datasets. This research indicates that adaptive label correction using label distributions is an effective strategy to enhance the robustness and accuracy of ordinal classification models in the presence of noisy data.

Method: Renders multi-view images and normal maps from input 3D model, generates novel 2D object using adaptive text-image harmony, then applies texture multi-view diffusion for consistency and shape multi-view diffusion for accuracy before reconstructing the final 3D model.

Result: Extensive experiments demonstrate effectiveness, producing impressive 3D creations like shark(3D)-crocodile(text) composites with coherent structures and consistent textures.

Conclusion: The proposed C33D approach successfully addresses the challenges of 3D object synthesis by effectively integrating multiple content sources to create novel and structurally coherent 3D models with improved texture consistency and shape accuracy.

Abstract: In this paper, we tackle a new task of 3D object synthesis, where a 3D model is composited with another object category to create a novel 3D model. However, most existing text/image/3D-to-3D methods struggle to effectively integrate multiple content sources, often resulting in inconsistent textures and inaccurate shapes. To overcome these challenges, we propose a straightforward yet powerful approach, category+3D-to-3D (C33D), for generating novel and structurally coherent 3D models. Our method begins by rendering multi-view images and normal maps from the input 3D model, then generating a novel 2D object using adaptive text-image harmony (ATIH) with the front-view image and a text description from another object category as inputs. To ensure texture consistency, we introduce texture multi-view diffusion, which refines the textures of the remaining multi-view RGB images based on the novel 2D object. For enhanced shape accuracy, we propose shape multi-view diffusion to improve the 2D shapes of both the multi-view RGB images and the normal maps, also conditioned on the novel 2D object. Finally, these outputs are used to reconstruct a complete and novel 3D model. Extensive experiments demonstrate the effectiveness of our method, yielding impressive 3D creations, such as shark(3D)-crocodile(text) in the first row of Fig. 1. A project page is available at: https://xzr52.github.io/C33D/

Method: Proposed MulSeT benchmark for multi-view spatial understanding tasks. Conducted systematic analysis from data and architectural perspectives across three scenarios using designed experiments.

Result: Performance converges quickly with more training data but reaches low upper bounds, especially for spatial imagination tasks. Spatial understanding relies more on visual encoder’s positional encoding than language model’s in both cascaded and native MLLMs.

Conclusion: Merely expanding training data is insufficient. Future improvements require architectural design optimization and reasoning injection to enhance spatial reasoning capabilities in MLLMs.

Abstract: Spatial understanding is essential for Multimodal Large Language Models (MLLMs) to support perception, reasoning, and planning in embodied environments. Despite recent progress, existing studies reveal that MLLMs still struggle with spatial understanding. However, existing research lacks a comprehensive and systematic evaluation of these limitations, often restricted to isolated scenarios, such as single-view or video. In this work, we present a systematic analysis of spatial understanding from both data and architectural perspectives across three representative scenarios: single-view, multi-view, and video. We propose a benchmark named MulSeT (Multi-view Spatial Understanding Tasks), and design a series of experiments to analyze the spatial reasoning capabilities of MLLMs. From the data perspective, the performance of spatial understanding converges quickly as the training data increases, and the upper bound is relatively low, especially for tasks that require spatial imagination. This indicates that merely expanding training data is insufficient to achieve satisfactory performance. From the architectural perspective, we find that spatial understanding relies more heavily on the positional encoding within the visual encoder than within the language model, in both cascaded and native MLLMs. Moreover, we explore reasoning injection and envision future improvements through architectural design to optimize spatial understanding. These insights shed light on the limitations of current MLLMs and suggest new directions for improving spatial reasoning capabilities through data scaling and architectural tuning.

Method: Revisits plain ViTs with multi-scale token aggregation architecture, performs domain-adaptive pretraining on 3.87M CT slices using multi-stage DINOv3 recipe to learn robust dense features.

Result: Matches or exceeds state-of-the-art performance across four segmentation benchmarks, demonstrating foundation models as unified backbones for medical image segmentation.

Conclusion: MedDINOv3 provides an effective framework for adapting vision foundation models to medical imaging, overcoming domain gap challenges and achieving superior segmentation performance.

Abstract: Accurate segmentation of organs and tumors in CT and MRI scans is essential for diagnosis, treatment planning, and disease monitoring. While deep learning has advanced automated segmentation, most models remain task-specific, lacking generalizability across modalities and institutions. Vision foundation models (FMs) pretrained on billion-scale natural images offer powerful and transferable representations. However, adapting them to medical imaging faces two key challenges: (1) the ViT backbone of most foundation models still underperform specialized CNNs on medical image segmentation, and (2) the large domain gap between natural and medical images limits transferability. We introduce \textbf{MedDINOv3}, a simple and effective framework for adapting DINOv3 to medical segmentation. We first revisit plain ViTs and design a simple and effective architecture with multi-scale token aggregation. Then, we perform domain-adaptive pretraining on \textbf{CT-3M}, a curated collection of 3.87M axial CT slices, using a multi-stage DINOv3 recipe to learn robust dense features. MedDINOv3 matches or exceeds state-of-the-art performance across four segmentation benchmarks, demonstrating the potential of vision foundation models as unified backbones for medical image segmentation. The code is available at https://github.com/ricklisz/MedDINOv3.

Method: Proposes a 4D cost aggregation network using pure 2D convolutions with a lightweight bidirectional geometry aggregation block that captures spatial and disparity representations through decoupled learning.

Result: DBStereo outperforms all existing aggregation-based methods in both inference time and accuracy, even surpassing iterative-based method IGEV-Stereo, while achieving real-time performance.

Conclusion: Breaks the empirical design of using 3D convolutions for 4D cost volume and provides a simple yet strong baseline for decoupled aggregation paradigm in stereo matching.

Abstract: High-performance real-time stereo matching methods invariably rely on 3D regularization of the cost volume, which is unfriendly to mobile devices. And 2D regularization based methods struggle in ill-posed regions. In this paper, we present a deployment-friendly 4D cost aggregation network DBStereo, which is based on pure 2D convolutions. Specifically, we first provide a thorough analysis of the decoupling characteristics of 4D cost volume. And design a lightweight bidirectional geometry aggregation block to capture spatial and disparity representation respectively. Through decoupled learning, our approach achieves real-time performance and impressive accuracy simultaneously. Extensive experiments demonstrate that our proposed DBStereo outperforms all existing aggregation-based methods in both inference time and accuracy, even surpassing the iterative-based method IGEV-Stereo. Our study break the empirical design of using 3D convolutions for 4D cost volume and provides a simple yet strong baseline of the proposed decouple aggregation paradigm for further study. Code will be available at (\href{https://github.com/happydummy/DBStereo}{https://github.com/happydummy/DBStereo}) soon.

Method: Proposes Geometric-Structural Dual-Guided Network (GSD-Net) with: 1) Geometric Distance-Aware module that dynamically adjusts pixel-level weights using geometric features, 2) Structure-Guided Label Refinement module that refines labels with structural priors, and 3) Knowledge Transfer module to enrich supervision and improve local detail sensitivity.

Result: Achieved state-of-the-art performance under noisy annotations with improvements of 2.52% on Kvasir, 22.76% on Shenzhen, 8.87% on BU-SUC, and 4.59% on BraTS2020 under SR simulated noise. Evaluated on six public datasets with simulated and real-world noisy annotations.

Conclusion: GSD-Net effectively addresses the challenge of noisy annotations in medical image segmentation by integrating geometric and structural guidance, demonstrating robust performance across various noise types and datasets.

Abstract: The effectiveness of convolutional neural networks in medical image segmentation relies on large-scale, high-quality annotations, which are costly and time-consuming to obtain. Even expert-labeled datasets inevitably contain noise arising from subjectivity and coarse delineations, which disrupt feature learning and adversely impact model performance. To address these challenges, this study propose a Geometric-Structural Dual-Guided Network (GSD-Net), which integrates geometric and structural cues to improve robustness against noisy annotations. It incorporates a Geometric Distance-Aware module that dynamically adjusts pixel-level weights using geometric features, thereby strengthening supervision in reliable regions while suppressing noise. A Structure-Guided Label Refinement module further refines labels with structural priors, and a Knowledge Transfer module enriches supervision and improves sensitivity to local details. To comprehensively assess its effectiveness, we evaluated GSD-Net on six publicly available datasets: four containing three types of simulated label noise, and two with multi-expert annotations that reflect real-world subjectivity and labeling inconsistencies. Experimental results demonstrate that GSD-Net achieves state-of-the-art performance under noisy annotations, achieving improvements of 2.52% on Kvasir, 22.76% on Shenzhen, 8.87% on BU-SUC, and 4.59% on BraTS2020 under SR simulated noise. The codes of this study are available at https://github.com/ortonwang/GSD-Net.

Method: Proposes a collaborative distillation and self-learning framework using reinforcement learning. An RL agent identifies optimal training strategies, generates challenging samples for self-learning, and dynamically adjusts teacher guidance based on student performance.

Result: Experimental results show significant improvement in student performance and better fusion results compared to existing techniques.

Conclusion: The proposed framework successfully enhances image fusion quality through collaborative distillation and adaptive self-learning driven by reinforcement learning, outperforming conventional methods.

Abstract: Infrared and visible image fusion plays a critical role in enhancing scene perception by combining complementary information from different modalities. Despite recent advances, achieving high-quality image fusion with lightweight models remains a significant challenge. To bridge this gap, we propose a novel collaborative distillation and self-learning framework for image fusion driven by reinforcement learning. Unlike conventional distillation, this approach not only enables the student model to absorb image fusion knowledge from the teacher model, but more importantly, allows the student to perform self-learning on more challenging samples to enhance its capabilities. Particularly, in our framework, a reinforcement learning agent explores and identifies a more suitable training strategy for the student.The agent takes both the student’s performance and the teacher-student gap as inputs, which leads to the generation of challenging samples to facilitate the student’s self-learning. Simultaneously, it dynamically adjusts the teacher’s guidance strength based on the student’s state to optimize the knowledge transfer. Experimental results demonstrate that our method can significantly improve student performance and achieve better fusion results compared to existing techniques.

Method: Decouples makeup transfer into transparent makeup mask extraction and graphics-based rendering. Uses pseudo-ground-truth data from graphics rendering and unsupervised k-means clustering, with specialized training objectives including alpha-weighted reconstruction and lip color losses.

Result: Achieves robust makeup transfer across diverse poses, expressions, and skin tones while preserving temporal smoothness. Outperforms existing baselines in capturing fine details, maintaining stability, and preserving identity integrity.

Conclusion: The proposed framework successfully addresses key challenges in virtual makeup try-on by enabling real-time, high-fidelity cosmetic transfer with strong temporal consistency and identity preservation.

Abstract: We present a novel framework for real-time virtual makeup try-on that achieves high-fidelity, identity-preserving cosmetic transfer with robust temporal consistency. In live makeup transfer applications, it is critical to synthesize temporally coherent results that accurately replicate fine-grained makeup and preserve user’s identity. However, existing methods often struggle to disentangle semitransparent cosmetics from skin tones and other identify features, causing identity shifts and raising fairness concerns. Furthermore, current methods lack real-time capabilities and fail to maintain temporal consistency, limiting practical adoption. To address these challenges, we decouple makeup transfer into two steps: transparent makeup mask extraction and graphics-based mask rendering. After the makeup extraction step, the makeup rendering can be performed in real time, enabling live makeup try-on. Our makeup extraction model trained on pseudo-ground-truth data generated via two complementary methods: a graphics-based rendering pipeline and an unsupervised k-means clustering approach. To further enhance transparency estimation and color fidelity, we propose specialized training objectives, including alpha-weighted reconstruction and lip color losses. Our method achieves robust makeup transfer across diverse poses, expressions, and skin tones while preserving temporal smoothness. Extensive experiments demonstrate that our approach outperforms existing baselines in capturing fine details, maintaining temporal stability, and preserving identity integrity.

Method: Developed through computer science-hydrology collaboration, RiverScope includes 1,145 high-resolution images with expert-labeled river masks, co-registered with Sentinel-2, SWOT, and SWORD data. Evaluated deep networks across multiple architectures, pretraining strategies, and training datasets.

Result: Achieved median error of 7.2 meters for river width estimation - significantly outperforming existing satellite-derived methods. Best models combined transfer learning with multispectral PlanetScope channels via learned adaptors.

Conclusion: RiverScope provides a valuable resource for fine-scale hydrological modeling and multi-sensor water monitoring, supporting climate adaptation and sustainable water management with unprecedented accuracy for river width estimation.

Abstract: Surface water dynamics play a critical role in Earth’s climate system, influencing ecosystems, agriculture, disaster resilience, and sustainable development. Yet monitoring rivers and surface water at fine spatial and temporal scales remains challenging – especially for narrow or sediment-rich rivers that are poorly captured by low-resolution satellite data. To address this, we introduce RiverScope, a high-resolution dataset developed through collaboration between computer science and hydrology experts. RiverScope comprises 1,145 high-resolution images (covering 2,577 square kilometers) with expert-labeled river and surface water masks, requiring over 100 hours of manual annotation. Each image is co-registered with Sentinel-2, SWOT, and the SWOT River Database (SWORD), enabling the evaluation of cost-accuracy trade-offs across sensors – a key consideration for operational water monitoring. We also establish the first global, high-resolution benchmark for river width estimation, achieving a median error of 7.2 meters – significantly outperforming existing satellite-derived methods. We extensively evaluate deep networks across multiple architectures (e.g., CNNs and transformers), pretraining strategies (e.g., supervised and self-supervised), and training datasets (e.g., ImageNet and satellite imagery). Our best-performing models combine the benefits of transfer learning with the use of all the multispectral PlanetScope channels via learned adaptors. RiverScope provides a valuable resource for fine-scale and multi-sensor hydrological modeling, supporting climate adaptation and sustainable water management.

Method: Developed a Diffusion Transformer (DiT) pipeline with background preservation branch using masked token injection, DiT fusion blocks with full self-attention, foreground augmentation, and Extended Rotary Position Embedding (ERoPE) for layout fusion. Created VideoComp dataset with 61K video sets.

Result: The method effectively achieves generative video compositing and outperforms existing solutions in both fidelity and consistency metrics.

Conclusion: The proposed generative video compositing approach successfully automates the traditionally labor-intensive process, enabling adaptive injection of foreground elements with user customization while maintaining video consistency.

Abstract: Video compositing combines live-action footage to create video production, serving as a crucial technique in video creation and film production. Traditional pipelines require intensive labor efforts and expert collaboration, resulting in lengthy production cycles and high manpower costs. To address this issue, we automate this process with generative models, called generative video compositing. This new task strives to adaptively inject identity and motion information of foreground video to the target video in an interactive manner, allowing users to customize the size, motion trajectory, and other attributes of the dynamic elements added in final video. Specifically, we designed a novel Diffusion Transformer (DiT) pipeline based on its intrinsic properties. To maintain consistency of the target video before and after editing, we revised a light-weight DiT-based background preservation branch with masked token injection. As to inherit dynamic elements from other sources, a DiT fusion block is proposed using full self-attention, along with a simple yet effective foreground augmentation for training. Besides, for fusing background and foreground videos with different layouts based on user control, we developed a novel position embedding, named Extended Rotary Position Embedding (ERoPE). Finally, we curated a dataset comprising 61K sets of videos for our new task, called VideoComp. This data includes complete dynamic elements and high-quality target videos. Experiments demonstrate that our method effectively realizes generative video compositing, outperforming existing possible solutions in fidelity and consistency.

Method: Two-stage training: 1) Distill a decoder to create structured latent space from large human reconstruction model, 2) Train text-controlled latent diffusion model to generate avatars within this latent space.

Result: Superior performance over previous text-to-avatar generative models in both subjective and objective evaluations, with faster generation and text-based partial customization capabilities.

Conclusion: TeRA provides a more efficient and effective framework for photorealistic 3D human avatar generation from text, eliminating slow optimization while enabling customization through structured 3D representation.

Abstract: In this paper, we rethink text-to-avatar generative models by proposing TeRA, a more efficient and effective framework than the previous SDS-based models and general large 3D generative models. Our approach employs a two-stage training strategy for learning a native 3D avatar generative model. Initially, we distill a decoder to derive a structured latent space from a large human reconstruction model. Subsequently, a text-controlled latent diffusion model is trained to generate photorealistic 3D human avatars within this latent space. TeRA enhances the model performance by eliminating slow iterative optimization and enables text-based partial customization through a structured 3D human representation. Experiments have proven our approach’s superiority over previous text-to-avatar generative models in subjective and objective evaluation.

Method: Developed AFPE as a generalization of IFPE that incorporates anisotropic, class-specific, and domain-specific spatial dependencies, and benchmarked it against common PEs on multi-label chest X-ray classification, CT organ classification, and echocardiography ejection fraction regression.

Result: AFPE significantly outperformed state-of-the-art positional encodings in all tested anisotropic settings, demonstrating that optimal PE choice depends on data anisotropy and structure shape.

Conclusion: For anisotropic medical images and videos, choosing an anisotropic positional encoding that fits both the data characteristics and the shape of interest is crucial for optimal model performance.

Abstract: The adoption of Transformer-based architectures in the medical domain is growing rapidly. In medical imaging, the analysis of complex shapes - such as organs, tissues, or other anatomical structures - combined with the often anisotropic nature of high-dimensional images complicates these adaptations. In this study, we critically examine the role of Positional Encodings (PEs), arguing that commonly used approaches may be suboptimal for the specific challenges of medical imaging. Sinusoidal Positional Encodings (SPEs) have proven effective in vision tasks, but they struggle to preserve Euclidean distances in higher-dimensional spaces. Isotropic Fourier Feature Positional Encodings (IFPEs) have been proposed to better preserve Euclidean distances, but they lack the ability to account for anisotropy in images. To address these limitations, we propose Anisotropic Fourier Feature Positional Encoding (AFPE), a generalization of IFPE that incorporates anisotropic, class-specific, and domain-specific spatial dependencies. We systematically benchmark AFPE against commonly used PEs on multi-label classification in chest X-rays, organ classification in CT images, and ejection fraction regression in echocardiography. Our results demonstrate that choosing the correct PE can significantly improve model performance. We show that the optimal PE depends on the shape of the structure of interest and the anisotropy of the data. Finally, our proposed AFPE significantly outperforms state-of-the-art PEs in all tested anisotropic settings. We conclude that, in anisotropic medical images and videos, it is of paramount importance to choose an anisotropic PE that fits the data and the shape of interest.

Method: Integrates pre-trained 2D CNNs (ResNet50, EfficientNet, Vision Transformers) with LSTM network enhanced by spatial attention mechanism to extract spatial features and capture temporal dependencies.

Result: Achieved peak accuracy of 93.34% with ResNet50-based configuration on curated UCF101 dataset subset, outperforming several state-of-the-art HAR systems.

Conclusion: Proposed method offers scalable, real-time-capable solution for fitness activity recognition with broader applications in vision-based health and activity monitoring.

Abstract: Fitness movement recognition, a focused subdomain of human activity recognition (HAR), plays a vital role in health monitoring, rehabilitation, and personalized fitness training by enabling automated exercise classification from video data. However, many existing deep learning approaches rely on computationally intensive 3D models, limiting their feasibility in real-time or resource-constrained settings. In this paper, we present a lightweight and effective framework that integrates pre-trained 2D Convolutional Neural Networks (CNNs) such as ResNet50, EfficientNet, and Vision Transformers (ViT) with a Long Short-Term Memory (LSTM) network enhanced by spatial attention. These models efficiently extract spatial features while the LSTM captures temporal dependencies, and the attention mechanism emphasizes informative segments. We evaluate the framework on a curated subset of the UCF101 dataset, achieving a peak accuracy of 93.34% with the ResNet50-based configuration. Comparative results demonstrate the superiority of our approach over several state-of-the-art HAR systems. The proposed method offers a scalable and real-time-capable solution for fitness activity recognition with broader applications in vision-based health and activity monitoring.

Method: MDM-MixMFL framework with modality decoupling strategy (separating modality-tailored and modality-shared information) and modality memorizing mechanism (storing dynamically refreshed client-shared modality prototypes).

Result: Extensive experiments on two public MRI datasets demonstrate the effectiveness and superiority of the proposed method.

Conclusion: The proposed framework successfully addresses mix-modal federated learning challenges for MRI segmentation in distributed medical settings with heterogeneous modalities and data.

Abstract: Magnetic resonance imaging (MRI) image segmentation is crucial in diagnosing and treating many diseases, such as brain tumors. Existing MRI image segmentation methods mainly fall into a centralized multimodal paradigm, which is inapplicable in engineering non-centralized mix-modal medical scenarios. In this situation, each distributed client (hospital) processes multiple mixed MRI modalities, and the modality set and image data for each client are diverse, suffering from extensive client-wise modality heterogeneity and data heterogeneity. In this paper, we first formulate non-centralized mix-modal MRI image segmentation as a new paradigm for federated learning (FL) that involves multiple modalities, called mix-modal federated learning (MixMFL). It distinguishes from existing multimodal federating learning (MulMFL) and cross-modal federating learning (CroMFL) paradigms. Then, we proposed a novel modality decoupling and memorizing mix-modal federated learning framework (MDM-MixMFL) for MRI image segmentation, which is characterized by a modality decoupling strategy and a modality memorizing mechanism. Specifically, the modality decoupling strategy disentangles each modality into modality-tailored and modality-shared information. During mix-modal federated updating, corresponding modality encoders undergo tailored and shared updating, respectively. It facilitates stable and adaptive federating aggregation of heterogeneous data and modalities from distributed clients. Besides, the modality memorizing mechanism stores client-shared modality prototypes dynamically refreshed from every modality-tailored encoder to compensate for incomplete modalities in each local client. It further benefits modality aggregation and fusion processes during mixmodal federated learning. Extensive experiments on two public datasets for MRI image segmentation demonstrate the effectiveness and superiority of our methods.

Method: Extends self-supervised DINOSAUR by generating high-quality unsupervised pseudo labels from video frames without camera motion, performs motion segmentation on optical flow, refines slot representations, and trains a slot deactivation module for foreground/background assignment.

Result: Achieves strong multi-object discovery results on TRI-PD and KITTI datasets, outperforming previous state-of-the-art methods despite being fully unsupervised.

Conclusion: MR-DINOSAUR demonstrates that conceptually simple motion refinement and slot deactivation can enable fully unsupervised multi-object discovery with competitive performance.

Abstract: Unsupervised multi-object discovery (MOD) aims to detect and localize distinct object instances in visual scenes without any form of human supervision. Recent approaches leverage object-centric learning (OCL) and motion cues from video to identify individual objects. However, these approaches use supervision to generate pseudo labels to train the OCL model. We address this limitation with MR-DINOSAUR – Motion-Refined DINOSAUR – a minimalistic unsupervised approach that extends the self-supervised pre-trained OCL model, DINOSAUR, to the task of unsupervised multi-object discovery. We generate high-quality unsupervised pseudo labels by retrieving video frames without camera motion for which we perform motion segmentation of unsupervised optical flow. We refine DINOSAUR’s slot representations using these pseudo labels and train a slot deactivation module to assign slots to foreground and background. Despite its conceptual simplicity, MR-DINOSAUR achieves strong multi-object discovery results on the TRI-PD and KITTI datasets, outperforming the previous state of the art despite being fully unsupervised.

Method: Proposes FastVGGT with training-free token merging mechanism, unique 3D-specific token partitioning strategy to eliminate redundant computation while preserving reconstruction capacity.

Result: Achieves 4x speedup over VGGT with 1000 input images while mitigating error accumulation in long-sequence scenarios across multiple 3D geometry benchmarks.

Conclusion: Token merging shows promise as a principled solution for scalable 3D vision systems, effectively addressing inference efficiency without compromising performance.

Abstract: Foundation models for 3D vision have recently demonstrated remarkable capabilities in 3D perception. However, scaling these models to long-sequence image inputs remains a significant challenge due to inference-time inefficiency. In this work, we present a detailed analysis of VGGT, a state-of-the-art feed-forward visual geometry model and identify its primary bottleneck. Visualization further reveals a token collapse phenomenon in the attention maps. Motivated by these findings, we explore the potential of token merging in the feed-forward visual geometry model. Owing to the unique architectural and task-specific properties of 3D models, directly applying existing merging techniques proves challenging. To this end, we propose FastVGGT, which, for the first time, leverages token merging in the 3D domain through a training-free mechanism for accelerating VGGT. we devise a unique token partitioning strategy tailored to 3D architectures and tasks, effectively eliminating redundant computation while preserving VGGT’s powerful reconstruction capacity. Extensive experiments on multiple 3D geometry benchmarks validate the effectiveness of our approach. Notably, with 1000 input images, FastVGGT achieves a 4x speedup over VGGT while mitigating error accumulation in long-sequence scenarios. These findings underscore the potential of token merging as a principled solution for scalable 3D vision systems. Code is available at: https://mystorm16.github.io/fastvggt/.

Method: The toolbox extracts activations from 600+ DNNs trained on various vision tasks, computes representational dissimilarity matrices (RDMs), and compares them to brain recordings using RSA and weighted RSA in specific ROIs and with searchlight search.

Result: Net2Brain enables comprehensive comparison of DNN representational spaces with human brain data, supporting addition of new stimulus datasets and brain recordings for evaluation.

Conclusion: The toolbox provides a powerful interface for testing cognitive computational neuroscience hypotheses by facilitating detailed comparison between artificial neural networks and human brain representations across diverse vision tasks.

Abstract: We introduce Net2Brain, a graphical and command-line user interface toolbox for comparing the representational spaces of artificial deep neural networks (DNNs) and human brain recordings. While different toolboxes facilitate only single functionalities or only focus on a small subset of supervised image classification models, Net2Brain allows the extraction of activations of more than 600 DNNs trained to perform a diverse range of vision-related tasks (e.g semantic segmentation, depth estimation, action recognition, etc.), over both image and video datasets. The toolbox computes the representational dissimilarity matrices (RDMs) over those activations and compares them to brain recordings using representational similarity analysis (RSA), weighted RSA, both in specific ROIs and with searchlight search. In addition, it is possible to add a new data set of stimuli and brain recordings to the toolbox for evaluation. We demonstrate the functionality and advantages of Net2Brain with an example showcasing how it can be used to test hypotheses of cognitive computational neuroscience.

Method: Adapter-based approach designed for multi-modal domain with encoder-level implicit cross-modal interactions between vision and language encoders, using few parameterization layers.

Result: Outperforms adapter-based methods on image-text retrieval (MSCOCO, Flickr30K) and video-text retrieval (MSR-VTT, DiDeMo, ActivityNet) datasets while reducing parameters and training time.

Conclusion: The proposed cross-modal adapter enables efficient transfer learning by fixing pre-trained parameters, allowing model sharing across datasets while maintaining strong retrieval performance.

Abstract: Vision-language retrieval is an important multi-modal learning topic, where the goal is to retrieve the most relevant visual candidate for a given text query. Recently, pre-trained models, e.g., CLIP, show great potential on retrieval tasks. However, as pre-trained models are scaling up, fully fine-tuning them on donwstream retrieval datasets has a high risk of overfitting. Moreover, in practice, it would be costly to train and store a large model for each task. To overcome the above issues, we present a novel Cross-Modal Adapter for parameter-efficient transfer learning. Inspired by adapter-based methods, we adjust the pre-trained model with a few parameterization layers. However, there are two notable differences. First, our method is designed for the multi-modal domain. Secondly, it allows encoder-level implicit cross-modal interactions between vision and language encoders. Although surprisingly simple, our approach has three notable benefits: (1) reduces the vast majority of fine-tuned parameters, (2) saves training time, and (3) allows all the pre-trained parameters to be fixed, enabling the pre-trained model to be shared across datasets. Extensive experiments demonstrate that, without bells and whistles, our approach outperforms adapter-based methods on image-text retrieval datasets (MSCOCO, Flickr30K) and video-text retrieval datasets (MSR-VTT, DiDeMo, and ActivityNet).

Method: Uses object-level supervision from 3D bounding boxes to guide depth distribution learning, selects foreground pixels with 2D object detector and projects them into 3D space for pseudo-voxel feature encoding, then incorporates these features into BEV representation using deformable attention mechanism.

Result: Achieves consistent improvements over BEV-based baselines on nuScenes and Argoverse 2 datasets in terms of average precision and comprehensive detection score.

Conclusion: OA-DET3D effectively enhances 3D object detection performance by introducing object awareness through depth learning and pseudo-point features, demonstrating general applicability to various existing 3D detection pipelines.

Abstract: The recent advance in multi-camera 3D object detection is featured by bird’s-eye view (BEV) representation or object queries. However, the ill-posed transformation from image-plane view to 3D space inevitably causes feature clutter and distortion, making the objects blur into the background. To this end, we explore how to incorporate supplementary cues for differentiating objects in the transformed feature representation. Formally, we introduce OA-DET3D, a general plug-in module that improves 3D object detection by bringing object awareness into a variety of existing 3D object detection pipelines. Specifically, OA-DET3D boosts the representation of objects by leveraging object-centric depth information and foreground pseudo points. First, we use object-level supervision from the properties of each 3D bounding box to guide the network in learning the depth distribution. Next, we select foreground pixels using a 2D object detector and project them into 3D space for pseudo-voxel feature encoding. Finally, the object-aware depth features and pseudo-voxel features are incorporated into the BEV representation or query feature from the baseline model with a deformable attention mechanism. We conduct extensive experiments on the nuScenes dataset and Argoverse 2 dataset to validate the merits of OA-DET3D. Our method achieves consistent improvements over the BEV-based baselines in terms of both average precision and comprehensive detection score.

Method: MultiScale Signature feature learning Network (MS-SigNet) with novel co-tuplet loss that learns global and regional features from multiple spatial scales, focusing on multiple positive and negative examples to overcome metric learning limitations.

Result: Promising performance demonstrated on four benchmark datasets across different languages, outperforming state-of-the-art approaches in distinguishing genuine signatures from skilled forgeries.

Conclusion: The proposed MS-SigNet with co-tuplet loss effectively addresses signature verification challenges and shows strong cross-language performance, supported by a new large-scale Chinese signature dataset (HanSig) for future research.

Abstract: Handwritten signature verification, crucial for legal and financial institutions, faces challenges including inter-writer similarity, intra-writer variations, and limited signature samples. To address these, we introduce the MultiScale Signature feature learning Network (MS-SigNet) with the co-tuplet loss, a novel metric learning loss designed for offline handwritten signature verification. MS-SigNet learns both global and regional signature features from multiple spatial scales, enhancing feature discrimination. This approach effectively distinguishes genuine signatures from skilled forgeries by capturing overall strokes and detailed local differences. The co-tuplet loss, focusing on multiple positive and negative examples, overcomes the limitations of typical metric learning losses by addressing inter-writer similarity and intra-writer variations and emphasizing informative examples. The code is available at https://github.com/ashleyfhh/MS-SigNet. We also present HanSig, a large-scale Chinese signature dataset to support robust system development for this language. The dataset is accessible at https://github.com/hsinmin/HanSig. Experimental results on four benchmark datasets in different languages demonstrate the promising performance of our method in comparison to state-of-the-art approaches.

Method: Provides a comprehensive survey of recent developments, introduces mathematical models for different collaboration strategies, examines key techniques including agent selection, data alignment, and feature fusion, and discusses major challenges and research directions.

Result: The paper systematically organizes and analyzes the state-of-the-art in V2X cooperative perception, identifying technical approaches for reliable perception sharing and highlighting critical challenges that need to be addressed.

Conclusion: V2X cooperative perception is crucial for autonomous driving advancement, with promising future directions including privacy-preserving AI methods, collaborative intelligence, and integrated sensing frameworks to overcome current limitations and support further development.

Abstract: Achieving fully autonomous driving with enhanced safety and efficiency relies on vehicle-to-everything cooperative perception, which enables vehicles to share perception data, thereby enhancing situational awareness and overcoming the limitations of the sensing ability of individual vehicles. Vehicle-to-everything cooperative perception plays a crucial role in extending the perception range, increasing detection accuracy, and supporting more robust decision-making and control in complex environments. This paper provides a comprehensive survey of recent developments in vehicle-to-everything cooperative perception, introducing mathematical models that characterize the perception process under different collaboration strategies. Key techniques for enabling reliable perception sharing, such as agent selection, data alignment, and feature fusion, are examined in detail. In addition, major challenges are discussed, including differences in agents and models, uncertainty in perception outputs, and the impact of communication constraints such as transmission delay and data loss. The paper concludes by outlining promising research directions, including privacy-preserving artificial intelligence methods, collaborative intelligence, and integrated sensing frameworks to support future advancements in vehicle-to-everything cooperative perception.

Method: Created LvBench with 20,061 QA pairs from 100 movies across genres, featuring extended temporal durations (single-scene to full-scene contexts), six diverse question types using video frames and subtitles, and rigorous human annotation.

Result: Analysis shows all existing methods’ performance significantly deteriorates as video and clue length increases, demonstrating the challenge of genuine long-form video understanding.

Conclusion: LvBench serves as a valuable benchmark for future long-form video understanding research, addressing limitations of previous datasets through extended durations, diverse modalities, and high-quality annotations.

Abstract: Despite remarkable recent progress, existing long-form VideoQA datasets fall short of meeting the criteria for genuine long-form video understanding. This is primarily due to the use of short videos for question curation, and the reliance on limited-length sub-clips as clues to answer those questions. Meanwhile, previous datasets have limited focus on question type and modality. To remedy this, we introduce LvBench, a Long-form video understanding benchmark for versatile multi-modal question-answering. Our LvBench stands out from existing long-form VideoQA datasets through three key characteristics: 1) Extended temporal durations: We consider videos ranging from 70 seconds to 4 hours, covering single-scene, multi-scene, and full-scene contexts. This design accounts for both video and clue lengths, capturing diverse contextual dynamics. 2) Diverse question types and modalities: LvBench introduces six distinct question types that evaluate various perceptual and cognitive capabilities, utilizing both video frames and subtitles. 3) High-quality annotations: We employ rigorous manual labeling by human annotators. Our dataset comprises 20,061 question-answer pairs sourced from 100 carefully selected movies across diverse genres, annotated collaboratively by multiple individuals. Analysis involving various baselines reveals a consistent trend: the performance of all existing methods significantly deteriorates when video and clue length increases. We expect LvBench to serve as a valuable resource for future works on long-form video understanding.

Method: Two novel supervised, embedded methods that integrate band selection directly into deep learning training pipelines, ensuring alignment with target tasks without separate preprocessing steps.

Result: Extensive experiments on three remote sensing benchmarks and an autonomous driving dataset show state-of-the-art performance while selecting only a minimal number of bands.

Conclusion: The proposed methods demonstrate the potential of efficient, task-specific hyperspectral imaging pipelines for practical deployment in resource-constrained settings.

Abstract: Hyperspectral Imaging (HSI) captures rich spectral information across contiguous wavelength bands, supporting applications in precision agriculture, environmental monitoring, and autonomous driving. However, its high dimensionality poses computational challenges, particularly in real-time or resource-constrained settings. While prior band selection methods attempt to reduce complexity, they often rely on separate preprocessing steps and lack alignment with downstream tasks. We propose two novel supervised, embedded methods for task-specific HSI band selection that integrate directly into deep learning models. By embedding band selection within the training pipeline, our methods eliminate the need for separate preprocessing and ensure alignment with the target task. Extensive experiments on three remote sensing benchmarks and an autonomous driving dataset show that our methods achieve state-of-the-art performance while selecting only a minimal number of bands. These results highlight the potential of efficient, task-specific HSI pipelines for practical deployment.

Method: Divides generation into subtasks: 1) Layout prediction split into numerical/spatial reasoning and bounding box visual planning, 2) Layout-to-image generation synthesizes objects from easy to difficult ones in two steps.

Result: Outperforms previous approaches on HRS and NSR-1K benchmarks with notable margins. Visual results and user studies show significant improvement in perceptual quality for complex multi-object prompts.

Conclusion: The divide-and-conquer approach enables lightweight LLMs to achieve layout accuracy comparable to large-scale models while significantly improving performance on complex text-to-image generation tasks with multiple objects.

Abstract: Diffusion-driven text-to-image (T2I) generation has achieved remarkable advancements in recent years. To further improve T2I models’ capability in numerical and spatial reasoning, layout is employed as an intermedium to bridge large language models and layout-based diffusion models. However, these methods often rely on closed-source, large-scale LLMs for layout prediction, limiting accessibility and scalability. They also struggle with generating images from prompts with multiple objects and complicated spatial relationships. To tackle these challenges, we introduce a divide-and-conquer approach which decouples the generation task into multiple subtasks. First, the layout prediction stage is divided into numerical & spatial reasoning and bounding box visual planning, enabling even lightweight LLMs to achieve layout accuracy comparable to large-scale models. Second, the layout-to-image generation stage is divided into two steps to synthesize objects from easy ones to difficult ones. Experiments are conducted on the HRS and NSR-1K benchmarks and our method outperforms previous approaches with notable margins. In addition, visual results and user study demonstrate that our approach significantly improves the perceptual quality, especially when generating multiple objects from complex textural prompts.

Method: Uses Truncated Signed Distance Functions (TSDFs) encoded into a discrete latent space with joint 2D/3D supervision, employs latent diffusion model with multimodal conditioning, and trains with simulated partial observations.

Result: Reduces GPU memory usage by 30%, outperforms class-specific models by 12% and class-agnostic models by 47% in reconstruction error, produces more diverse and high-fidelity completions.

Conclusion: The framework successfully unifies multimodal conditioning for 3D shape completion, demonstrating significant improvements in efficiency, generalization, and reconstruction quality over existing approaches.

Abstract: We present a novel 3D shape completion framework that unifies multimodal conditioning, leveraging both 2D images and 3D partial scans through a latent diffusion model. Shapes are represented as Truncated Signed Distance Functions (TSDFs) and encoded into a discrete latent space jointly supervised by 2D and 3D cues, enabling efficient high-resolution processing while reducing GPU memory usage by 30% compared to state-of-the-art methods. Our approach guides the generation process with flexible multimodal conditioning, ensuring consistent integration of 2D and 3D information from encoding to reconstruction. Our training strategy simulates realistic partial observations, avoiding assumptions about input structure and improving robustness in real-world scenarios. Leveraging our efficient latent space and multimodal conditioning, our model generalizes across object categories, outperforming class-specific models by 12% and class-agnostic models by 47% in $l_1$ reconstruction error, while producing more diverse, realistic, and high-fidelity completions than prior approaches.

Method: Uses hybrid encoder architecture with stride-based dilated convolutions and depthwise separable dilated convolutions, lightweight upsampling modules (UCB and USB), and Partial Class Activation Attention (PCAA) mechanism. Available in four configurations from ultra-compact to high-performance.

Result: TwinLiteNet+_Large achieves 92.9% mIoU for drivable area segmentation and 34.2% IoU for lane segmentation on BDD100K dataset, surpassing existing models with 11x fewer FLOPs. Demonstrates superior inference speed, quantization robustness, and energy efficiency on embedded devices.

Conclusion: TwinLiteNet+ is a compelling solution for real-world autonomous driving systems, offering high efficiency and performance suitable for resource-constrained embedded deployment.

Abstract: Semantic segmentation is a fundamental perception task in autonomous driving, particularly for identifying drivable areas and lane markings to enable safe navigation. However, most state-of-the-art (SOTA) models are computationally intensive and unsuitable for real-time deployment on resource-constrained embedded devices. In this paper, we introduce TwinLiteNet+, an enhanced multi-task segmentation model designed for real-time drivable area and lane segmentation with high efficiency. TwinLiteNet+ employs a hybrid encoder architecture that integrates stride-based dilated convolutions and depthwise separable dilated convolutions, balancing representational capacity and computational cost. To improve task-specific decoding, we propose two lightweight upsampling modules-Upper Convolution Block (UCB) and Upper Simple Block (USB)-alongside a Partial Class Activation Attention (PCAA) mechanism that enhances segmentation precision. The model is available in four configurations, ranging from the ultra-compact TwinLiteNet+{Nano} (34K parameters) to the high-performance TwinLiteNet+{Large} (1.94M parameters). On the BDD100K dataset, TwinLiteNet+_{Large} achieves 92.9% mIoU for drivable area segmentation and 34.2% IoU for lane segmentation-surpassing existing state-of-the-art models while requiring 11x fewer floating-point operations (FLOPs) for computation. Extensive evaluations on embedded devices demonstrate superior inference speed, quantization robustness (INT8/FP16), and energy efficiency, validating TwinLiteNet+ as a compelling solution for real-world autonomous driving systems. Code is available at https://github.com/chequanghuy/TwinLiteNetPlus.

Method: Uses 3D Gaussian Splatting representation with filter-based refinement to eliminate floaters, nearest neighbor-based style loss for stylization by fine-tuning geometry and color parameters, depth preservation loss to maintain geometry integrity, and specially designed losses for color, scale, and region control.

Result: Achieves high-quality stylization results with faithful brushstrokes and geometric consistency, demonstrating effectiveness and efficiency across various scenes and styles in both quality and inference speed.

Conclusion: StylizedGS provides an efficient and flexible solution for 3D neural style transfer that overcomes limitations of previous methods while enabling customizable artistic control over stylized scenes.

Abstract: As XR technology continues to advance rapidly, 3D generation and editing are increasingly crucial. Among these, stylization plays a key role in enhancing the appearance of 3D models. By utilizing stylization, users can achieve consistent artistic effects in 3D editing using a single reference style image, making it a user-friendly editing method. However, recent NeRF-based 3D stylization methods encounter efficiency issues that impact the user experience, and their implicit nature limits their ability to accurately transfer geometric pattern styles. Additionally, the ability for artists to apply flexible control over stylized scenes is considered highly desirable to foster an environment conducive to creative exploration. To address the above issues, we introduce StylizedGS, an efficient 3D neural style transfer framework with adaptable control over perceptual factors based on 3D Gaussian Splatting representation. We propose a filter-based refinement to eliminate floaters that affect the stylization effects in the scene reconstruction process. The nearest neighbor-based style loss is introduced to achieve stylization by fine-tuning the geometry and color parameters of 3DGS, while a depth preservation loss with other regularizations is proposed to prevent the tampering of geometry content. Moreover, facilitated by specially designed losses, StylizedGS enables users to control color, stylized scale, and regions during the stylization to possess customization capabilities. Our method achieves high-quality stylization results characterized by faithful brushstrokes and geometric consistency with flexible controls. Extensive experiments across various scenes and styles demonstrate the effectiveness and efficiency of our method concerning both stylization quality and inference speed.

Method: Transformer-based framework called BRACTIVE that aligns visual features with functional MRI signals to automatically identify brain Regions of Interest across multiple subjects and various object categories.

Result: BRACTIVE effectively identifies person-specific regions like face and body-selective areas, aligns with neuroscience findings, and enhances deep neural network performance across various benchmarks when guided by human visual brain activity.

Conclusion: BRACTIVE shows strong potential for both neuroscience research and machine intelligence studies by enabling scalable ROI identification across multiple subjects and demonstrating that brain activity guidance improves neural network performance.

Abstract: The human brain is a highly efficient processing unit, and understanding how it works can inspire new algorithms and architectures in machine learning. In this work, we introduce a novel framework named Brain Activation Network (BRACTIVE), a transformer-based approach to studying the human visual brain. The primary objective of BRACTIVE is to align the visual features of subjects with their corresponding brain representations using functional Magnetic Resonance Imaging (fMRI) signals. It enables us to identify the brain’s Regions of Interest (ROIs) in the subjects. Unlike previous brain research methods, which can only identify ROIs for one subject at a time and are limited by the number of subjects, BRACTIVE automatically extends this identification to multiple subjects and ROIs. Our experiments demonstrate that BRACTIVE effectively identifies person-specific regions of interest, such as face and body-selective areas, aligning with neuroscience findings and indicating potential applicability to various object categories. More importantly, we found that leveraging human visual brain activity to guide deep neural networks enhances performance across various benchmarks. It encourages the potential of BRACTIVE in both neuroscience and machine intelligence studies.

Method: Proposes TE-NeXt block that fuses attention mechanisms with 3D sparse convolutions in a residual convolution framework, trained and evaluated on SemanticKITTI, Rellis-3D, and SemanticUSL datasets.

Result: Outperforms state-of-the-art methods in semantic segmentation, demonstrating better performance in unstructured environments while maintaining high reliability and robustness in urban settings.

Conclusion: TE-NeXt provides superior abstraction capabilities for traversability estimation and the implementation is made publicly available to ensure reproducibility of results for the scientific community.

Abstract: This paper presents TE-NeXt, a novel and efficient architecture for Traversability Estimation (TE) from sparse LiDAR point clouds based on a residual convolution block. TE-NeXt block fuses notions of current trends such as attention mechanisms and 3D sparse convolutions. TE-NeXt aims to demonstrate high capacity for generalisation in a variety of urban and natural environments, using well-known and accessible datasets such as SemanticKITTI, Rellis-3D and SemanticUSL. Thus, the designed architecture ouperforms state-of-the-art methods in the problem of semantic segmentation, demonstrating better results in unstructured environments and maintaining high reliability and robustness in urbans environments, which leads to better abstraction. Implementation is available in a open repository to the scientific community with the aim of ensuring the reproducibility of results.

Method: Comprehensive survey analysis of existing methodologies, including examination of related tasks like image/video captioning, visual question answering, and non-visual story generation techniques.

Result: Provides critical analysis of current approaches, datasets, and evaluation metrics, highlighting their limitations and common challenges in the field.

Conclusion: The survey offers a systematic overview of visual story generation techniques and identifies key challenges and limitations that need to be addressed for future advancements in the field.

Abstract: Creating engaging narratives from visual data is crucial for automated digital media consumption, assistive technologies, and interactive entertainment. This survey covers methodologies used in the generation of these narratives, focusing on their principles, strengths, and limitations. The survey also covers tasks related to automatic story generation, such as image and video captioning, and visual question answering, as well as story generation without visual inputs. These tasks share common challenges with visual story generation and have served as inspiration for the techniques used in the field. We analyze the main datasets and evaluation metrics, providing a critical perspective on their limitations.

Method: Uses 2.5D shape from RGB-D reference, differentiable renderer to generate rotated views, and semantic maps from DINOv2. Compares rendered RGB and semantic maps with query images to refine 3D relative pose through gradient backpropagation.

Result: Outperforms state-of-the-art supervised methods on LineMOD, LM-O, and YCB-V datasets, especially under rigorous Acc@5/10/15° metrics and challenging cross-dataset settings.

Conclusion: The method enables zero-shot relative pose estimation for unseen objects using only a single RGB-D reference, demonstrating superior performance without requiring training or labeling.

Abstract: Humans can easily deduce the relative pose of a previously unseen object, without labeling or training, given only a single query-reference image pair. This is arguably achieved by incorporating i) 3D/2.5D shape perception from a single image, ii) render-and-compare simulation, and iii) rich semantic cue awareness to furnish (coarse) reference-query correspondence. Motivated by this, we propose a novel 3D generalizable relative pose estimation method by elaborating 3D/2.5D shape perception with a 2.5D shape from an RGB-D reference, fulfilling the render-and-compare paradigm with an off-the-shelf differentiable renderer, and leveraging the semantic cues from a pretrained model like DINOv2. Specifically, our differentiable renderer takes the 2.5D rotatable mesh textured by the RGB and the semantic maps (obtained by DINOv2 from the RGB input), then renders new RGB and semantic maps (with back-surface culling) under a novel rotated view. The refinement loss comes from comparing the rendered RGB and semantic maps with the query ones, back-propagating the gradients through the differentiable renderer to refine the 3D relative pose. As a result, \emph{our method can be readily applied to unseen objects, given only a single RGB-D reference, without labeling or training}. Extensive experiments on LineMOD, LM-O, and YCB-V show that our training-free method significantly outperforms the state-of-the-art supervised methods, especially under the rigorous \texttt{Acc@5/10/15}$^\circ$ metrics and the challenging cross-dataset settings.

Method: Uses a diffusion process in 2D parameterized UV domain with IP-Adapter cross-attention layers for each conditioning signal type (sketches, photos, edges, FLAME parameters, landmarks, text).

Result: Produces topology-consistent, high-quality 3D face geometry from diverse input modalities within a single unified model.

Conclusion: The approach enables easy-to-use 3D face generation with multiple input options and fine control, demonstrating versatility across artistic, photographic, parametric, and textual inputs.

Abstract: We present a new method for multimodal conditional 3D face geometry generation that allows user-friendly control over the output identity and expression via a number of different conditioning signals. Within a single model, we demonstrate 3D faces generated from artistic sketches, portrait photos, Canny edges, FLAME face model parameters, 2D face landmarks, or text prompts. Our approach is based on a diffusion process that generates 3D geometry in a 2D parameterized UV domain. Geometry generation passes each conditioning signal through a set of cross-attention layers (IP-Adapter), one set for each user-defined conditioning signal. The result is an easy-to-use 3D face generation tool that produces topology-consistent, high-quality geometry with fine-grain user control.

Method: Modified YOLOX detector outputs multiple appearance attributes + ReID network for complementary embeddings. Uses joint distance function considering IoU, direction, color, style, and ReID similarity. Incorporates global linking and Gaussian Smoothing Process interpolation.

Result: Achieves MOTA scores: 79.7 on MOT16, 80.6 on MOT17, 77.9 on MOT20, and 92.2 on DanceTrack - state-of-the-art online performance.

Conclusion: Feature-enriched detection and modular post-processing significantly advance multi-object tracking, maintaining consistent identities through occlusions with reduced identity switches.

Abstract: We introduce FeatureSORT, a simple yet effective online multiple object tracker that reinforces the DeepSORT baseline with a redesigned detector and additional feature cues. In contrast to conventional detectors that only provide bounding boxes, our modified YOLOX architecture is extended to output multiple appearance attributes, including clothing color, clothing style, and motion direction, alongside the bounding boxes. These feature cues, together with a ReID network, form complementary embeddings that substantially improve association accuracy. Furthermore, we incorporate stronger post-processing strategies, such as global linking and Gaussian Smoothing Process interpolation, to handle missing associations and detections. During online tracking, we define a measurement-to-track distance function that jointly considers IoU, direction, color, style, and ReID similarity. This design enables FeatureSORT to maintain consistent identities through longer occlusions while reducing identity switches. Extensive experiments on standard MOT benchmarks demonstrate that FeatureSORT achieves state-of-the-art online performance, with MOTA scores of 79.7 on MOT16, 80.6 on MOT17, 77.9 on MOT20, and 92.2 on DanceTrack, underscoring the effectiveness of feature-enriched detection and modular post processing in advancing multi-object tracking.

Method: Used YOLO algorithm for muzzle detection and FaceNet architecture for learning embeddings from muzzle images. Applied CLAHE with sharpening filters for preprocessing. Collected 32,374 images from 826 cattle.

Result: Achieved 96.489% accuracy, 97.334% F1 score, 87.993% true positive rate at extremely low false positive rate of 0.098%.

Conclusion: The system provides reliable and efficient cattle identification that can significantly advance livestock insurance and precision farming.

Abstract: Absence of tamper-proof cattle identification technology was a significant problem preventing insurance companies from providing livestock insurance. This lack of technology had devastating financial consequences for marginal farmers as they did not have the opportunity to claim compensation for any unexpected events such as the accidental death of cattle in Bangladesh. Using machine learning and deep learning algorithms, we have solved the bottleneck of cattle identification by developing and introducing a muzzle-based cattle identification system. The uniqueness of cattle muzzles has been scientifically established, which resembles human fingerprints. This is the fundamental premise that prompted us to develop a cattle identification system that extracts the uniqueness of cattle muzzles. For this purpose, we collected 32,374 images from 826 cattle. Contrast-limited adaptive histogram equalization (CLAHE) with sharpening filters was applied in the preprocessing steps to remove noise from images. We used the YOLO algorithm for cattle muzzle detection in the image and the FaceNet architecture to learn unified embeddings from muzzle images using squared $L_2$ distances. Our system performs with an accuracy of $96.489%$, $F_1$ score of $97.334%$, and a true positive rate (tpr) of $87.993%$ at a remarkably low false positive rate (fpr) of $0.098%$. This reliable and efficient system for identifying cattle can significantly advance livestock insurance and precision farming.

Method: Introduces a semi-optimal policy that independently estimates the value of each frame using per-frame confidence scores, then selects the top N frames based on these individual estimates rather than exploring the entire combinatorial space.

Result: The method efficiently approximates the optimal policy, particularly in practical settings, and demonstrates stable high performance across various datasets and model architectures regardless of N and T sizes.

Conclusion: The proposed semi-optimal frame sampling policy provides a computationally efficient solution that maintains performance while dramatically reducing the search complexity from exponential to linear in the number of frames.

Abstract: Given a video with $T$ frames, frame sampling is a task to select $N \ll T$ frames, so as to maximize the performance of a fixed video classifier. Not just brute-force search, but most existing methods suffer from its vast search space of $\binom{T}{N}$, especially when $N$ gets large. To address this challenge, we introduce a novel perspective of reducing the search space from $O(T^N)$ to $O(T)$. Instead of exploring the entire $O(T^N)$ space, our proposed semi-optimal policy selects the top $N$ frames based on the independently estimated value of each frame using per-frame confidence, significantly reducing the computational complexity. We verify that our semi-optimal policy can efficiently approximate the optimal policy, particularly under practical settings. Additionally, through extensive experiments on various datasets and model architectures, we demonstrate that learning our semi-optimal policy ensures stable and high performance regardless of the size of $N$ and $T$.

Method: Introduced DivScene dataset with 4,614 houses across 81 scene types and 5,707 target objects. Fine-tuned LVLMs to predict next actions with Chain-of-Thought explanations using BFS-generated shortest paths without human supervision.

Result: Current models fall short on open-vocab object navigation. Fine-tuned LVLMs achieved substantial improvement, surpassing GPT-4o by over 20% in success rates using only BFS-generated paths.

Conclusion: LVLMs’ navigation capabilities can be significantly enhanced through fine-tuning with automatically generated paths, demonstrating strong potential for embodied AI applications without requiring human supervision.

Abstract: Large Vision-Language Models (LVLMs) have achieved significant progress in tasks like visual question answering and document understanding. However, their potential to comprehend embodied environments and navigate within them remains underexplored. In this work, we first study the challenge of open-vocabulary object navigation by introducing DivScene, a large-scale dataset with 4,614 houses across 81 scene types and 5,707 kinds of target objects. Our dataset provides a much greater diversity of target objects and scene types than existing datasets, enabling a comprehensive task evaluation. We evaluated various methods with LVLMs and LLMs on our dataset and found that current models still fall short of open-vocab object navigation ability. Then, we fine-tuned LVLMs to predict the next action with CoT explanations. We observe that LVLM’s navigation ability can be improved substantially with only BFS-generated shortest paths without any human supervision, surpassing GPT-4o by over 20% in success rates.

Method: Proposes an ensemble-learning approach that first detects pedestrians, compresses images using skeletonization, adds contextual information, and uses a stacked ensemble-learning model for intent prediction.

Result: Achieves similar pedestrian intent prediction performance as state-of-the-art approaches while reducing computational complexity by 99.7% across different datasets.

Conclusion: The proposed low-complexity method effectively balances prediction accuracy and computational efficiency, making it suitable for real-time applications in autonomous vehicles.

Abstract: Walking as a form of active travel is essential in promoting sustainable transport. It is thus crucial to accurately predict pedestrian crossing intention and avoid collisions, especially with the advent of autonomous and advanced driver-assisted vehicles. Current research leverages computer vision and machine learning advances to predict near-misses; however, this often requires high computation power to yield reliable results. In contrast, this work proposes a low-complexity ensemble-learning approach that employs contextual data for predicting the pedestrian’s intent for crossing. The pedestrian is first detected, and their image is then compressed using skeleton-ization, and contextual information is added into a stacked ensemble-learning approach. Our experiments on different datasets achieve similar pedestrian intent prediction performance as the state-of-the-art approaches with 99.7% reduction in computational complexity. Our source code and trained models will be released upon paper acceptance

Method: Uses a dual learning framework with shared 3D scene graph (3DSG) representation. Proposes SD^3 framework that leverages intermediate features from easier 3D→image and 3D→text processes to guide the harder image→3D and text→3D processes.

Result: Outperforms mainstream T2I and I2T methods significantly on the VSD dataset. In-depth analysis shows the dual learning strategy advances performance.

Conclusion: Joint modeling of SI2T and ST2I tasks with shared 3D scene graph representation and dual diffusion framework significantly improves spatial understanding performance compared to standalone approaches.

Abstract: In the visual spatial understanding (VSU) area, spatial image-to-text (SI2T) and spatial text-to-image (ST2I) are two fundamental tasks that appear in dual form. Existing methods for standalone SI2T or ST2I perform imperfectly in spatial understanding, due to the difficulty of 3D-wise spatial feature modeling. In this work, we consider modeling the SI2T and ST2I together under a dual learning framework. During the dual framework, we then propose to represent the 3D spatial scene features with a novel 3D scene graph (3DSG) representation that can be shared and beneficial to both tasks. Further, inspired by the intuition that the easier 3D$\to$image and 3D$\to$text processes also exist symmetrically in the ST2I and SI2T, respectively, we propose the Spatial Dual Discrete Diffusion (SD$^3$) framework, which utilizes the intermediate features of the 3D$\to$X processes to guide the hard X$\to$3D processes, such that the overall ST2I and SI2T will benefit each other. On the visual spatial understanding dataset VSD, our system outperforms the mainstream T2I and I2T methods significantly. Further in-depth analysis reveals how our dual learning strategy advances.

Method: 1) Uses visual foundational models for image feature extraction 2) Geometry-guided depth distribution adapter bridges monocular depth estimation and camera-to-BEV transform 3) Leverages semantic embeddings from OSM as auxiliary guidance for feature matching

Result: Superior performance demonstrated on MGL dataset, cross-area/cross-condition benchmark, and KITTI dataset. The method shows improved accuracy, robustness, and generalization capability.

Conclusion: OSMLoc effectively integrates semantic and geometric guidance to overcome modality disparities between visual observations and map data, enabling more accurate and robust visual localization using OpenStreetMap.

Abstract: OpenStreetMap (OSM), a rich and versatile source of volunteered geographic information (VGI), facilitates human self-localization and scene understanding by integrating nearby visual observations with vectorized map data. However, the disparity in modalities and perspectives poses a major challenge for effectively matching camera imagery with compact map representations, thereby limiting the full potential of VGI data in real-world localization applications. Inspired by the fact that the human brain relies on the fusion of geometric and semantic understanding for spatial localization tasks, we propose the OSMLoc in this paper. OSMLoc is a brain-inspired visual localization approach based on first-person-view images against the OSM maps. It integrates semantic and geometric guidance to significantly improve accuracy, robustness, and generalization capability. First, we equip the OSMLoc with the visual foundational model to extract powerful image features. Second, a geometry-guided depth distribution adapter is proposed to bridge the monocular depth estimation and camera-to-BEV transform. Thirdly, the semantic embeddings from the OSM data are utilized as auxiliary guidance for image-to-OSM feature matching. To validate the proposed OSMLoc, we collect a worldwide cross-area and cross-condition (CC) benchmark for extensive evaluation. Experiments on the MGL dataset, CC validation benchmark, and KITTI dataset have demonstrated the superiority of our method. Code, pre-trained models, CC validation benchmark, and additional results are available at: https://github.com/WHU-USI3DV/OSMLoc.

Method: Proposes Zoom Eye, a model-agnostic tree search algorithm that treats images as hierarchical trees where child nodes represent zoomed-in sub-regions. MLLMs navigate from root to leaf nodes to find task-relevant visual evidence, simulating human-like zooming behavior.

Result: Zoom Eye consistently improves multiple MLLMs by large margins (e.g., InternVL2.5-8B increases by 15.71% and 17.69% on HR-Bench) and enables small 3-8B MLLMs to outperform strong large models like GPT-4o.

Conclusion: The proposed vision-level reasoning approach through hierarchical zooming significantly enhances MLLM performance on high-resolution visual tasks, demonstrating the importance of dynamic visual exploration over static text-level reasoning.

Abstract: Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in vision-language understanding. Recently, with the integration of test-time scaling techniques, these models have also shown strong potential in visual reasoning. However, most existing reasoning approaches remain text-level in nature: MLLMs are prompted to explore various combinations of textual tokens via their underlying language model, while the visual input remains fixed throughout the reasoning process. This paradigm limits the model’s ability to fully exploit rich visual information, particularly when dealing with images containing numerous fine-grained elements. In such cases, vision-level reasoning becomes crucial - where models dynamically zoom into specific regions of the image to gather detailed visual cues necessary for accurate decision-making. In this paper, we propose Zoom Eye, a training-free, model-agnostic tree search algorithm tailored for vision-level reasoning. Zoom Eye treats an image as a hierarchical tree structure, where each child node represents a zoomed-in sub-region of its parent, and the root corresponds to the full image. The algorithm enables MLLMs to simulate human-like zooming behavior by navigating from root to leaf nodes in search of task-relevant visual evidence. We experiment on a series of high-resolution benchmarks and the results demonstrate that Zoom Eye consistently improves the performance of multiple MLLMs by a large margin (e.g., InternVL2.5-8B increases by 15.71% and 17.69% on HR-Bench) and also enables small 3-8B MLLMs to outperform strong large models such as GPT-4o. Code: https://github.com/om-ai-lab/ZoomEye

Method: Proposes EFTViT framework with masked image patches - randomly excluding patches during training to reduce computation. Uses hierarchical structure with lightweight local modules on clients and larger global module on server. Implements median sampling strategy to balance intermediate features and protect data privacy.

Result: Achieves up to 28.17% accuracy improvement, reduces local training computational cost by 2.8x, and cuts local training time by 4.4x compared to existing methods on popular benchmarks.

Conclusion: EFTViT successfully enables efficient full-parameter ViT training on resource-constrained devices while maintaining performance, reducing computational overhead, and enhancing privacy protection through masked patch training and hierarchical federated architecture.

Abstract: Federated learning research has recently shifted from Convolutional Neural Networks (CNNs) to Vision Transformers (ViTs) due to their superior capacity. ViTs training demands higher computational resources due to the lack of 2D inductive biases inherent in CNNs. However, efficient federated training of ViTs on resource-constrained edge devices remains unexplored in the community. In this paper, we propose EFTViT, a hierarchical federated framework that leverages masked images to enable efficient, full-parameter training on resource-constrained edge devices, offering substantial benefits for learning on heterogeneous data. In general, we patchify images and randomly mask a portion of the patches, observing that excluding them from training has minimal impact on performance while substantially reducing computation costs and enhancing data content privacy protection. Specifically, EFTViT comprises a series of lightweight local modules and a larger global module, updated independently on clients and the central server, respectively. The local modules are trained on masked image patches, while the global module is trained on intermediate patch features uploaded from the local client, balanced through a proposed median sampling strategy to erase client data distribution privacy. We analyze the computational complexity and privacy protection of EFTViT. Extensive experiments on popular benchmarks show that EFTViT achieves up to 28.17% accuracy improvement, reduces local training computational cost by up to 2.8$\times$, and cuts local training time by up to 4.4$\times$ compared to existing methods.

Method: Created OHRBench with carefully selected document images and multimodal Q&A pairs. Identified two OCR noise types (Semantic and Formatting) and applied perturbations to generate structured data with varying noise levels. Evaluated current OCR solutions and systematically assessed noise impact on RAG.

Result: Current OCR solutions are not competent for constructing high-quality RAG knowledge bases. Both Semantic and Formatting noise types significantly impact RAG performance, with clear trend relationships between noise degree and performance degradation.

Conclusion: OHRBench provides the first comprehensive benchmark to understand OCR’s cascading impact on RAG systems, revealing critical limitations in current OCR technology and establishing the need for improved OCR solutions tailored for RAG applications.

Abstract: Retrieval-augmented Generation (RAG) enhances Large Language Models (LLMs) by integrating external knowledge to reduce hallucinations and incorporate up-to-date information without retraining. As an essential part of RAG, external knowledge bases are commonly built by extracting structured data from unstructured PDF documents using Optical Character Recognition (OCR). However, given the imperfect prediction of OCR and the inherent non-uniform representation of structured data, knowledge bases inevitably contain various OCR noises. In this paper, we introduce OHRBench, the first benchmark for understanding the cascading impact of OCR on RAG systems. OHRBench includes 8,561 carefully selected unstructured document images from seven real-world RAG application domains, along with 8,498 Q&A pairs derived from multimodal elements in documents, challenging existing OCR solutions used for RAG. To better understand OCR’s impact on RAG systems, we identify two primary types of OCR noise: Semantic Noise and Formatting Noise and apply perturbation to generate a set of structured data with varying degrees of each OCR noise. Using OHRBench, we first conduct a comprehensive evaluation of current OCR solutions and reveal that none is competent for constructing high-quality knowledge bases for RAG systems. We then systematically evaluate the impact of these two noise types and demonstrate the trend relationship between the degree of OCR noise and RAG performance. Our OHRBench, including PDF documents, Q&As, and the ground truth structured data are released at: https://github.com/opendatalab/OHR-Bench

Method: Uses 2D deep trackers within a 3D object fusion strategy and introduces motion learning in implicit feature space to autonomously correct trajectory errors and recover missed detections.

Result: Outperforms existing approaches on Waymo-NOTR and KITTI datasets.

Conclusion: The method eliminates reliance on 3D trackers and enhances robustness across diverse environments by leveraging 2D foundation models with error correction capabilities.

Abstract: Realistic scene reconstruction in driving scenarios poses significant challenges due to fast-moving objects. Most existing methods rely on labor-intensive manual labeling of object poses to reconstruct dynamic objects in canonical space and move them based on these poses during rendering. While some approaches attempt to use 3D object trackers to replace manual annotations, the limited generalization of 3D trackers – caused by the scarcity of large-scale 3D datasets – results in inferior reconstructions in real-world settings. In contrast, 2D foundation models demonstrate strong generalization capabilities. To eliminate the reliance on 3D trackers and enhance robustness across diverse environments, we propose a stable object tracking module by leveraging associations from 2D deep trackers within a 3D object fusion strategy. We address inevitable tracking errors by further introducing a motion learning strategy in an implicit feature space that autonomously corrects trajectory errors and recovers missed detections. Experimental results on Waymo-NOTR and KITTI show that our method outperforms existing approaches. Our code will be released on https://lolrudy.github.io/No3DTrackSG/.

Method: Temporal Preference Optimization (TPO) uses self-training with curated preference datasets at two granularities: localized temporal grounding and comprehensive temporal grounding to differentiate between well-grounded and inaccurate responses.

Result: TPO significantly enhances temporal understanding across three benchmarks (LongVideoBench, MLVU, Video-MME) and establishes LLaVA-Video-TPO as the leading 7B model on Video-MME.

Conclusion: TPO provides a scalable and efficient solution for advancing temporal reasoning in long-form video understanding while reducing reliance on manual annotations.

Abstract: Despite significant advancements in video large multimodal models (video-LMMs), achieving effective temporal grounding in long-form videos remains a challenge for existing models. To address this limitation, we propose Temporal Preference Optimization (TPO), a novel post-training framework designed to enhance the temporal grounding capabilities of video-LMMs through preference learning. TPO adopts a self-training approach that enables models to differentiate between well-grounded and less accurate temporal responses by leveraging curated preference datasets at two granularities: localized temporal grounding, which focuses on specific video segments, and comprehensive temporal grounding, which captures extended temporal dependencies across entire video sequences. By optimizing on these preference datasets, TPO significantly enhances temporal understanding while reducing reliance on manually annotated data. Extensive experiments on three long-form video understanding benchmarks–LongVideoBench, MLVU, and Video-MME–demonstrate the effectiveness of TPO across two state-of-the-art video-LMMs. Notably, LLaVA-Video-TPO establishes itself as the leading 7B model on the Video-MME benchmark, underscoring the potential of TPO as a scalable and efficient solution for advancing temporal reasoning in long-form video understanding. Project page: https://ruili33.github.io/tpo_website.

Method: Post-training quantization method that finds optimal clipping thresholds and scaling factors with mathematical guarantees to minimize quantization noise.

Result: Empirical results show significant reduction in model size and computational requirements while preserving model accuracy on real-world datasets.

Conclusion: The proposed quantization method effectively addresses deployment challenges of CNN models on resource-constrained devices by reducing storage and computational demands without sacrificing accuracy.

Abstract: Despite the success of CNN models on a variety of Image classification and segmentation tasks, their extensive computational and storage demands pose considerable challenges for real-world deployment on resource-constrained devices. Quantization is one technique that aims to alleviate these large storage requirements and speed up the inference process by reducing the precision of model parameters to lower-bit representations. In this paper, we introduce a novel post-training quantization method for model weights. Our method finds optimal clipping thresholds and scaling factors along with mathematical guarantees that our method minimizes quantization noise. Empirical results on real-world datasets demonstrate that our quantization scheme significantly reduces model size and computational requirements while preserving model accuracy.

Method: Uses a streamlined transformer architecture with self-attention blocks to exchange information among multi-view image tokens, decoding them into pixel-aligned 3D Gaussian primitives in a unified reference frame. Developed two specialized variants for object-centric and scene-level reconstruction.

Result: Outperforms several pose-dependent Large Reconstruction Models by a notable margin while achieving comparable or better pose estimation accuracy than state-of-the-art pose-free approach MASt3R in challenging benchmarks.

Conclusion: FreeSplatter eliminates camera pose management complexity while delivering exceptional visual fidelity, streamlining text/image-to-3D content creation pipelines without requiring pre-calibrated camera parameters.

Abstract: Sparse-view reconstruction models typically require precise camera poses, yet obtaining these parameters from sparse-view images remains challenging. We introduce FreeSplatter, a scalable feed-forward framework that generates high-quality 3D Gaussians from uncalibrated sparse-view images while estimating camera parameters within seconds. Our approach employs a streamlined transformer architecture where self-attention blocks facilitate information exchange among multi-view image tokens, decoding them into pixel-aligned 3D Gaussian primitives within a unified reference frame. This representation enables both high-fidelity 3D modeling and efficient camera parameter estimation using off-the-shelf solvers. We develop two specialized variants–for object-centric and scene-level reconstruction–trained on comprehensive datasets. Remarkably, FreeSplatter outperforms several pose-dependent Large Reconstruction Models (LRMs) by a notable margin while achieving comparable or even better pose estimation accuracy compared to state-of-the-art pose-free reconstruction approach MASt3R in challenging benchmarks. Beyond technical benchmarks, FreeSplatter streamlines text/image-to-3D content creation pipelines, eliminating the complexity of camera pose management while delivering exceptional visual fidelity.

Method: Uses monocular video with simple head rotation, explicitly accounts for environment lighting, visibility and occlusions without simplifying assumptions, reconstructs surface geometry, diffuse albedo, specular intensity and roughness.

Result: Produces facial appearance maps approaching studio-based multi-view capture fidelity, but with much simpler and cheaper capture procedure in unconstrained environments.

Conclusion: Demonstrates that high-fidelity facial appearance reconstruction is achievable from lightweight in-the-wild capture, potentially democratizing professional-quality facial scanning.

Abstract: We present a new method for reconstructing the appearance properties of human faces from a lightweight capture procedure in an unconstrained environment. Our method recovers the surface geometry, diffuse albedo, specular intensity and specular roughness from a monocular video containing a simple head rotation in-the-wild. Notably, we make no simplifying assumptions on the environment lighting, and we explicitly take visibility and occlusions into account. As a result, our method can produce facial appearance maps that approach the fidelity of studio-based multi-view captures, but with a far easier and cheaper procedure.

Method: Systematic testing of four leading VLMs (LLaVA-1.5, LLaVA-Next, Qwen2-VL, Qwen2.5-VL) with eight acceleration methods across ten multi-modal benchmarks, including case studies with medical VLM LLaVA-Med.

Result: Accelerated models changed original answers up to 20% of the time, with up to 6.5% converting correct answers to incorrect. Input perturbations amplified these inconsistencies.

Conclusion: Current VLM acceleration evaluations overlook instance-level stability, revealing a critical need for stability checks to ensure trustworthy real-world deployment in sensitive applications.

Abstract: Vision-Language Models (VLMs) are powerful yet computationally intensive for widespread practical deployments. To address such challenge without costly re-training, post-training acceleration techniques like quantization and token reduction are extensively explored. However, current acceleration evaluations primarily target minimal overall performance degradation, overlooking a crucial question: does the accelerated model still give the same answers to the same questions as it did before acceleration? This is vital for stability-centered industrial applications where consistently correct answers for specific, known situations are paramount, such as in AI-based disease diagnosis. We systematically investigate this for accelerated VLMs, testing four leading models (LLaVA-1.5, LLaVA-Next, Qwen2-VL, Qwen2.5-VL) with eight acceleration methods on ten multi-modal benchmarks. Our findings are stark: despite minimal aggregate performance drops, accelerated models changed original answers up to 20% of the time. Critically, up to 6.5% of these changes converted correct answers to incorrect. Input perturbations magnified these inconsistencies, and the trend is confirmed by case studies with the medical VLM LLaVA-Med. This research reveals a significant oversight in VLM acceleration, stressing an urgent need for instance-level stability checks to ensure trustworthy real-world deployment.

Method: Leverages vision-language models as teachers to provide additional supervision with unstructured reasoning information and structured action labels during training.

Result: Achieves significant improvements in planning accuracy, reduced collision rates on nuScenes dataset, and better route completion and driving scores in closed-loop evaluation.

Conclusion: VLM-AD enhances feature representations to capture driving rationale, demonstrates effectiveness in long-horizon interactive scenarios, and shows potential for safe real-world deployment.

Abstract: Human drivers rely on commonsense reasoning to navigate diverse and dynamic real-world scenarios. Existing end-to-end (E2E) autonomous driving (AD) models are typically optimized to mimic driving patterns observed in data, without capturing the underlying reasoning processes. This limitation constrains their ability to handle challenging driving scenarios. To close this gap, we propose VLM-AD, a method that leverages vision-language models (VLMs) as teachers to enhance training by providing additional supervision that incorporates unstructured reasoning information and structured action labels. Such supervision enhances the model’s ability to learn richer feature representations that capture the rationale behind driving patterns. Importantly, our method does not require a VLM during inference, making it practical for real-time deployment. When integrated with state-of-the-art methods, VLM-AD achieves significant improvements in planning accuracy and reduced collision rates on the nuScenes dataset. It further improves route completion and driving scores under closed-loop evaluation, demonstrating its effectiveness in long-horizon, interactive driving scenarios and its potential for safe and reliable real-world deployment.

Method: Two-stage optimization: 1) Construct and refine SFT dataset based on safety and alignment principles, 2) Use text-level and video-level feedback for preference learning to optimize the SFT model.

Result: VPO significantly improves safety, alignment, and video quality compared to baselines, shows strong generalization across video models, and can outperform/combine with RLHF methods.

Conclusion: VPO provides an effective framework for aligning video generation models through principled prompt optimization that preserves user intent while enhancing safety and quality.

Abstract: Video generation models have achieved remarkable progress in text-to-video tasks. These models are typically trained on text-video pairs with highly detailed and carefully crafted descriptions, while real-world user inputs during inference are often concise, vague, or poorly structured. This gap makes prompt optimization crucial for generating high-quality videos. Current methods often rely on large language models (LLMs) to refine prompts through in-context learning, but suffer from several limitations: they may distort user intent, omit critical details, or introduce safety risks. Moreover, they optimize prompts without considering the impact on the final video quality, which can lead to suboptimal results. To address these issues, we introduce VPO, a principled framework that optimizes prompts based on three core principles: harmlessness, accuracy, and helpfulness. The generated prompts faithfully preserve user intents and, more importantly, enhance the safety and quality of generated videos. To achieve this, VPO employs a two-stage optimization approach. First, we construct and refine a supervised fine-tuning (SFT) dataset based on principles of safety and alignment. Second, we introduce both text-level and video-level feedback to further optimize the SFT model with preference learning. Our extensive experiments demonstrate that VPO significantly improves safety, alignment, and video quality compared to baseline methods. Moreover, VPO shows strong generalization across video generation models. Furthermore, we demonstrate that VPO could outperform and be combined with RLHF methods on video generation models, underscoring the effectiveness of VPO in aligning video generation models. Our code and data are publicly available at https://github.com/thu-coai/VPO.

Method: Proposes Traffic Scenario Generator and Unbounded Layout Generator using BEV representation. Combines stuff-oriented and instance-oriented neural fields with customized generative hash grids and periodic positional embeddings for different object types.

Result: Delivers state-of-the-art performance in generating realistic 4D cities. Supports downstream applications including instance editing, city stylization, and urban simulation.

Conclusion: The compositional approach effectively addresses the challenges of 4D city generation by separating dynamic and static elements and using specialized neural field representations for different urban components.

Abstract: 3D scene generation has garnered growing attention in recent years and has made significant progress. Generating 4D cities is more challenging than 3D scenes due to the presence of structurally complex, visually diverse objects like buildings and vehicles, and heightened human sensitivity to distortions in urban environments. To tackle these issues, we propose CityDreamer4D, a compositional generative model specifically tailored for generating unbounded 4D cities. Our main insights are 1) 4D city generation should separate dynamic objects (e.g., vehicles) from static scenes (e.g., buildings and roads), and 2) all objects in the 4D scene should be composed of different types of neural fields for buildings, vehicles, and background stuff. Specifically, we propose Traffic Scenario Generator and Unbounded Layout Generator to produce dynamic traffic scenarios and static city layouts using a highly compact BEV representation. Objects in 4D cities are generated by combining stuff-oriented and instance-oriented neural fields for background stuff, buildings, and vehicles. To suit the distinct characteristics of background stuff and instances, the neural fields employ customized generative hash grids and periodic positional embeddings as scene parameterizations. Furthermore, we offer a comprehensive suite of datasets for city generation, including OSM, GoogleEarth, and CityTopia. The OSM dataset provides a variety of real-world city layouts, while the Google Earth and CityTopia datasets deliver large-scale, high-quality city imagery complete with 3D instance annotations. Leveraging its compositional design, CityDreamer4D supports a range of downstream applications, such as instance editing, city stylization, and urban simulation, while delivering state-of-the-art performance in generating realistic 4D cities.

Method: Proposes a crossmodal progressive scene generation framework with incremental point cloud reconstruction and 3D Gaussian splatting, hierarchical depth prior-based regularization for depth accuracy and smoothness, and structured context-guided compression using hash grids to reduce redundancy.

Result: Comprehensive experiments across multiple scenes demonstrate significant potential and advantages over baseline methods, generating diverse and high-quality 3D scenes.

Conclusion: BloomScene provides an effective solution for lightweight, structured 3D scene generation with reduced storage overhead and improved geometric quality compared to existing approaches.

Abstract: With the widespread use of virtual reality applications, 3D scene generation has become a new challenging research frontier. 3D scenes have highly complex structures and need to ensure that the output is dense, coherent, and contains all necessary structures. Many current 3D scene generation methods rely on pre-trained text-to-image diffusion models and monocular depth estimators. However, the generated scenes occupy large amounts of storage space and often lack effective regularisation methods, leading to geometric distortions. To this end, we propose BloomScene, a lightweight structured 3D Gaussian splatting for crossmodal scene generation, which creates diverse and high-quality 3D scenes from text or image inputs. Specifically, a crossmodal progressive scene generation framework is proposed to generate coherent scenes utilizing incremental point cloud reconstruction and 3D Gaussian splatting. Additionally, we propose a hierarchical depth prior-based regularization mechanism that utilizes multi-level constraints on depth accuracy and smoothness to enhance the realism and continuity of the generated scenes. Ultimately, we propose a structured context-guided compression mechanism that exploits structured hash grids to model the context of unorganized anchor attributes, which significantly eliminates structural redundancy and reduces storage overhead. Comprehensive experiments across multiple scenes demonstrate the significant potential and advantages of our framework compared with several baselines.

Method: Introduces Visual Proxy Learning that initializes visual proxies for attributes, objects, and compositions from text representations, and Cross-Modal Joint Learning (CMJL) that imposes cross-modal constraints between text-image and fine-grained visual spaces.

Result: Achieves state-of-the-art performance in closed-world scenarios and competitive results in open-world settings across four CZSL benchmarks.

Conclusion: The approach effectively reduces modality gaps and enhances compositional generalization, demonstrating improved discrimination of similar composition pairs.

Abstract: Compositional Zero-Shot Learning (CZSL) aims to recognize novel attribute-object compositions by leveraging knowledge from seen compositions. Current methods align textual prototypes with visual features via Vision-Language Models (VLMs), but suffer from two limitations: (1) modality gaps hinder the discrimination of semantically similar pairs, and (2) single-modal textual prototypes lack fine-grained visual cues. In this paper, we introduce Visual Proxy Learning, a method that reduces modality gaps and enhances compositional generalization. We initialize visual proxies for attributes, objects, and their compositions using text representations and optimize the visual space to capture fine-grained cues, improving visual representations. Additionally, we propose Cross-Modal Joint Learning (CMJL), which imposes cross-modal constraints between the text-image and fine-grained visual spaces, improving generalization for unseen compositions and discriminating similar pairs. Experiments show state-of-the-art performance in closed-world scenarios and competitive results in open-world settings across four CZSL benchmarks, demonstrating the effectiveness of our approach in compositional generalization.

Method: Proposed StoryReasoning dataset with structured scene analyses and grounded stories, featuring cross-frame object re-identification using visual similarity and face recognition, chain-of-thought reasoning for narrative modeling, and grounding scheme linking text to visual entities across frames.

Result: Fine-tuned Qwen2.5-VL 7B model (Qwen Storyteller) reduced hallucinations from 4.06 to 3.56 (-12.3%) per story and improved creativity from 2.58 to 3.38 (+31.0%) compared to non-fine-tuned model.

Conclusion: The proposed approach effectively addresses referential hallucinations in visual storytelling through structured grounding and cross-frame consistency, demonstrating significant improvements in both accuracy and creativity.

Abstract: Visual storytelling systems struggle to maintain character identity across frames and link actions to appropriate subjects, frequently leading to referential hallucinations. These issues can be addressed through grounding of characters, objects, and other entities on the visual elements. We propose StoryReasoning, a dataset containing 4,178 stories derived from 52,016 movie images, with both structured scene analyses and grounded stories. Each story maintains character and object consistency across frames while explicitly modeling multi-frame relationships through structured tabular representations. Our approach features cross-frame object re-identification using visual similarity and face recognition, chain-of-thought reasoning for explicit narrative modeling, and a grounding scheme that links textual elements to visual entities across multiple frames. We establish baseline performance by fine-tuning Qwen2.5-VL 7B, creating Qwen Storyteller, which performs end-to-end object detection, re-identification, and landmark detection while maintaining consistent object references throughout the story. Evaluation demonstrates a reduction from 4.06 to 3.56 (-12.3%) hallucinations on average per story and an improvement in creativity from 2.58 to 3.38 (+31.0%) when compared to a non-fine-tuned model.

Method: CompDAE (Compressive Denoising Autoencoder) based on STFormer architecture with rate-constrained training strategy, using 8x8 pseudo-random binary masks and shared encoder with lightweight task-specific decoders.

Result: Achieves state-of-the-art performance with significantly lower complexity across multiple datasets, especially effective under ultra-low-light conditions where traditional CMOS and SCI pipelines fail.

Conclusion: The framework enables physically feasible implementations with small masks and performs downstream tasks directly from compressive measurements without reconstruction, making it suitable for practical hardware-constrained applications.

Abstract: Snapshot Compressed Imaging (SCI) offers high-speed, low-bandwidth, and energy-efficient image acquisition, but remains challenged by low-light and low signal-to-noise ratio (SNR) conditions. Moreover, practical hardware constraints in high-resolution sensors limit the use of large frame-sized masks, necessitating smaller, hardware-friendly designs. In this work, we present a novel SCI-based computer vision framework using pseudo-random binary masks of only 8$\times$8 in size for physically feasible implementations. At its core is CompDAE, a Compressive Denoising Autoencoder built on the STFormer architecture, designed to perform downstream tasks–such as edge detection and depth estimation–directly from noisy compressive raw pixel measurements without image reconstruction. CompDAE incorporates a rate-constrained training strategy inspired by BackSlash to promote compact, compressible models. A shared encoder paired with lightweight task-specific decoders enables a unified multi-task platform. Extensive experiments across multiple datasets demonstrate that CompDAE achieves state-of-the-art performance with significantly lower complexity, especially under ultra-low-light conditions where traditional CMOS and SCI pipelines fail.

Method: Fine-tuned state-of-the-art VLMs (2B-8B parameters) using Low-Rank Adaptation (LoRA) and validated on multiple out-of-domain scenarios from a proposed benchmark with controlled 2D shape configurations.

Result: Coherent sentence-based outputs outperformed tuple formats, especially in out-of-domain scenarios. Scaling numeric tokens during loss computation enhanced numerical approximation capabilities for spatial and measurement tasks.

Conclusion: Output format design, loss scaling strategies, and robust generalization techniques are crucial for enhancing VLM training and fine-tuning, particularly for tasks requiring precise spatial approximations and strong out-of-domain generalization.

Abstract: This work investigates the capabilities of current vision-language models (VLMs) in visual understanding and attribute measurement of primitive shapes using a benchmark focused on controlled 2D shape configurations with variations in spatial positioning, occlusion, rotation, size, and shape attributes such as type, quadrant, center-coordinates, rotation, occlusion status, and color as shown in Figure 1 and supplementary Figures S3-S81. We fine-tune state-of-the-art VLMs (2B-8B parameters) using Low-Rank Adaptation (LoRA) and validate them on multiple out-of-domain (OD) scenarios from our proposed benchmark. Our findings reveal that coherent sentence-based outputs outperform tuple formats, particularly in OD scenarios with large domain gaps. Additionally, we demonstrate that scaling numeric tokens during loss computation enhances numerical approximation capabilities, further improving performance on spatial and measurement tasks. These results highlight the importance of output format design, loss scaling strategies, and robust generalization techniques in enhancing the training and fine-tuning of VLMs, particularly for tasks requiring precise spatial approximations and strong OD generalization.

Method: Translate experimental properties and systemic constraints into mathematical terms, use functional analysis tools to collapse multi-variable bias optimization into a two-parameter problem solvable experimentally.

Result: Demonstrates that optimal camera parameter values for certain signals (like incandescent lamps powered by electrical grid) are significantly different from manufacturer defaults.

Conclusion: The proposed heuristic successfully squeezes event camera potential and expands detection capabilities for specific applications, showing that default manufacturer settings are suboptimal for certain use cases.

Abstract: One of the main challenges in unlocking the potential of neuromorphic cameras, also called ‘’event camera’’, is the development of novel methods that solve the multi-variable problem of adjusting their biases parameters to accommodate a desired task. Actually, it is very difficult to find in the literature a systematic heuristic that solves the problem for any desired application. In this paper we present a tuning parameters heuristic for the biases of event cameras, for tasks that require small objects detection in staring scenarios. The main purpose of the heuristic is to squeeze the camera’s potential, optimize its performance, and expand its detection capabilities as much as possible. In the presentation, we translate the experimental properties of event camera and systemic constrains into mathematical terms, and show, under certain assumptions and classical tools from functional analysis, how the multi-variable problem collapses into a two-parameter problem that can be solved experimentally. A main conclusion that will be demonstrated is that for certain desired signals, such as the one provided by an incandescent lamp powered by the periodic electrical grid, the optimal values of the camera are very far from the default values recommended by the manufacturer.

Method: CytoDiff - a stable diffusion model fine-tuned with LoRA weights and guided by few-shot samples to generate high-fidelity synthetic white blood cell images.

Result: Addition of 5,000 synthetic images per class improved ResNet classifier accuracy from 27% to 78% (+51%) and CLIP-based classification from 62% to 77% (+15%) using small, imbalanced real datasets.

Conclusion: Synthetic image generation is a valuable tool for biomedical ML, enhancing data coverage and enabling secure data sharing while preserving patient privacy.

Abstract: Biomedical datasets are often constrained by stringent privacy requirements and frequently suffer from severe class imbalance. These two aspects hinder the development of accurate machine learning models. While generative AI offers a promising solution, producing synthetic images of sufficient quality for training robust classifiers remains challenging. This work addresses the classification of individual white blood cells, a critical task in diagnosing hematological malignancies such as acute myeloid leukemia (AML). We introduce CytoDiff, a stable diffusion model fine-tuned with LoRA weights and guided by few-shot samples that generates high-fidelity synthetic white blood cell images. Our approach demonstrates substantial improvements in classifier performance when training data is limited. Using a small, highly imbalanced real dataset, the addition of 5,000 synthetic images per class improved ResNet classifier accuracy from 27% to 78% (+51%). Similarly, CLIP-based classification accuracy increased from 62% to 77% (+15%). These results establish synthetic image generation as a valuable tool for biomedical machine learning, enhancing data coverage and facilitating secure data sharing while preserving patient privacy. Paper code is publicly available at https://github.com/JanCarreras24/CytoDiff.

Method: Proposes DesCLIP with a language assistant to generate general attribute descriptions via prompts, an anchor-based embedding filter to obtain relevant GA description embeddings, and uses these as paired text embeddings for visual-textual matching to tune the visual encoder while calibrating class text embeddings.

Result: Extensive experiments show superior performance in VLM-based recognition compared to existing continual learning methods, demonstrating advancements and efficacy.

Conclusion: Leveraging general attribute descriptions to establish robust vision-GA-class trilateral associations effectively addresses knowledge forgetting in continual learning of vision-language models.

Abstract: Continual learning of vision-language models (VLMs) focuses on leveraging cross-modal pretrained knowledge to incrementally adapt to expanding downstream tasks and datasets, while tackling the challenge of knowledge forgetting. Existing research often focuses on connecting visual features with specific class text in downstream tasks, overlooking the latent relationships between general and specialized knowledge. Our findings reveal that forcing models to optimize inappropriate visual-text matches exacerbates forgetting of VLM’s recognition ability. To tackle this issue, we propose DesCLIP, which leverages general attribute (GA) descriptions to guide the understanding of specific class objects, enabling VLMs to establish robust vision-GA-class trilateral associations rather than relying solely on vision-class connections. Specifically, we introduce a language assistant to generate concrete GA description candidates via proper request prompts. Then, an anchor-based embedding filter is designed to obtain highly relevant GA description embeddings, which are leveraged as the paired text embeddings for visual-textual instance matching, thereby tuning the visual encoder. Correspondingly, the class text embeddings are gradually calibrated to align with these shared GA description embeddings. Extensive experiments demonstrate the advancements and efficacy of our proposed method, with comprehensive empirical evaluations highlighting its superior performance in VLM-based recognition compared to existing continual learning methods.

Method: Uses knowledge distillation from large language models, visual relationship detection for fine-grained knowledge extraction, question paraphrasing for underspecification, Chain-of-Evidence prompting for interpretable reasoning, and self-reflection technology for learning from mistakes.

Result: Achieves new state-of-the-art results on widely used datasets and performs on par with leading proprietary models like ChatGPT-5.

Conclusion: VIKSER effectively addresses key limitations in visual reasoning through fine-grained knowledge extraction and interpretable reasoning frameworks, demonstrating superior performance compared to existing methods.

Abstract: Visual reasoning refers to the task of solving questions about visual information. Current visual reasoning methods typically employ pre-trained vision-language model (VLM) strategies or deep neural network approaches. However, existing efforts are constrained by limited reasoning interpretability, while hindering by the phenomenon of underspecification in the question text. Additionally, the absence of fine-grained visual knowledge limits the precise understanding of subject behavior in visual reasoning tasks. To address these issues, we propose VIKSER (Visual Knowledge-Driven Self-Reinforcing Reasoning Framework). Specifically, VIKSER, trained using knowledge distilled from large language models, extracts fine-grained visual knowledge with the assistance of visual relationship detection techniques. Subsequently, VIKSER utilizes fine-grained visual knowledge to paraphrase the question with underspecification. Additionally, we design a novel prompting method called Chain-of-Evidence (CoE), which leverages the power of “evidence for reasoning” to endow VIKSER with interpretable reasoning capabilities. Meanwhile, the integration of self-reflection technology empowers VIKSER with the ability to learn and improve from its mistakes. Experiments conducted on widely used datasets demonstrate that VIKSER achieves new state-of-the-art (SOTA) results in relevant tasks. Moreover, VIKSER achieves performance on par with leading proprietary models, such as the latest ChatGPT-5.

Method: Created SpatialViz-Bench with 1,180 automatically generated problems across 12 tasks and 4 sub-abilities, then evaluated 33 state-of-the-art MLLMs on this benchmark.

Result: Found wide performance variations, difficulty perception misaligned with human intuition, 2D-to-3D performance cliffs, formulaic derivation preferences, and performance degradation from Chain-of-Thought prompting in open-source models.

Conclusion: State-of-the-art MLLMs still exhibit significant deficiencies in spatial visualization tasks, and SpatialViz-Bench effectively addresses this evaluation gap with publicly available data and code.

Abstract: Humans can directly imagine and manipulate visual images in their minds, a capability known as spatial visualization. While multi-modal Large Language Models (MLLMs) support imagination-based reasoning, spatial visualization remains insufficiently evaluated, typically embedded within broader mathematical and logical assessments. Existing evaluations often rely on IQ tests or math competitions that may overlap with training data, compromising assessment reliability. To this end, we introduce SpatialViz-Bench, a comprehensive multi-modal benchmark for spatial visualization with 12 tasks across 4 sub-abilities, comprising 1,180 automatically generated problems. Our evaluation of 33 state-of-the-art MLLMs not only reveals wide performance variations and demonstrates the benchmark’s strong discriminative power, but also uncovers counter-intuitive findings: models show difficulty perception misaligned with human intuition, exhibit dramatic 2Dto-3D performance cliffs, default to formulaic derivation over visualization, and paradoxically suffer performance degradation from Chain-of-Thought prompting in open-source models. Through statistical and qualitative analysis of error types, SpatialViz-Bench demonstrates that state-of-the-art MLLMs continue to exhibit deficiencies in spatial visualization tasks, thereby addressing a significant lacuna in the field. The benchmark data and evaluation code are publicly available.

Method: Proposes RAMer with: 1) Reconstruction-based adversarial learning with contrastive learning to handle missing modalities and enrich features, 2) Personality auxiliary task with modality-level attention to complement missing data, 3) Stack shuffle strategy to capture label-modality interdependencies.

Result: Achieves state-of-the-art performance on three benchmarks: MEmoR, CMU-MOSEI, and M³ED in both dyadic and multi-party MMER scenarios.

Conclusion: RAMer effectively addresses the challenges of incomplete modalities in real-world settings while capturing modality-specific characteristics and label-modality correlations, demonstrating superior performance across multiple benchmarks.

Abstract: Conventional Multi-modal multi-label emotion recognition (MMER) assumes complete access to visual, textual, and acoustic modalities. However, real-world multi-party settings often violate this assumption, as non-speakers frequently lack acoustic and textual inputs, leading to a significant degradation in model performance. Existing approaches also tend to unify heterogeneous modalities into a single representation, overlooking each modality’s unique characteristics. To address these challenges, we propose RAMer (Reconstruction-based Adversarial Model for Emotion Recognition), which refines multi-modal representations by not only exploring modality commonality and specificity but crucially by leveraging reconstructed features, enhanced by contrastive learning, to overcome data incompleteness and enrich feature quality. RAMer also introduces a personality auxiliary task to complement missing modalities using modality-level attention, improving emotion reasoning. To further strengthen the model’s ability to capture label and modality interdependency, we propose a stack shuffle strategy to enrich correlations between labels and modality-specific features. Experiments on three benchmarks, i.e., MEmoR, CMU-MOSEI, and $M^3ED$, demonstrate that RAMer achieves state-of-the-art performance in dyadic and multi-party MMER scenarios.

Method: Proposed Self-Knowledge Distillation training where vision-text component serves as teacher and vision-audio component as student, enabling audio processing analogous to text processing.

Result: Experimental results show Self-KD effectively enhances vision-audio capabilities by learning from vision-text components, improving audio-image interaction and multimodal task performance.

Conclusion: Self-Knowledge Distillation is an effective approach for bridging the performance gap between text and audio queries in Omnimodal LLMs by leveraging knowledge from well-aligned vision-text components.

Abstract: Omnimodal Large Language Models (OLLMs) have shown significant progress in integrating vision and text, but still struggle with integrating vision and audio, often exhibiting suboptimal performance when processing audio queries compared to text queries. This disparity is primarily due to insufficient alignment between vision and audio modalities during training, leading to inadequate attention to visual information when using audio queries. To mitigate this issue, we propose a Self-Knowledge Distillation (Self-KD) training method where the vision-text component of the OLLM serves as the teacher and the vision-audio component as the student. This enables the model to process audio in a manner analogous to its text processing. Our experimental results demonstrate that Self-KD is an effective method for enhancing the vision-audio capabilities of OLLMs by learning from the vision-text components, which subsequently improves the interaction between audio and images and results in improved performance on multimodal tasks.

Method: Proposes UnAvgLip with two main components: (1) Identity Perceiver module that encodes facial embeddings to align with audio features, and (2) ID-CrossAttn module that injects facial embeddings into the generation process to enhance identity retention.

Result: Achieves significant improvements of 5% on identity consistency metric and 2% on SSIM metric across HDTF and LRW benchmark datasets, effectively mitigating the averaging phenomenon at modest training and inference cost.

Conclusion: UnAvgLip successfully addresses the lip averaging problem in visual dubbing by preserving unique facial characteristics while maintaining precise lip synchronization, demonstrating superior performance over existing approaches.

Abstract: Recent advances in diffusion-based lip-syncing generative models have demonstrated their ability to produce highly synchronized talking face videos for visual dubbing. Although these models excel at lip synchronization, they often struggle to maintain fine-grained control over facial details in generated images. In this work, we identify “lip averaging” phenomenon where the model fails to preserve subtle facial details when dubbing unseen in-the-wild videos. This issue arises because the commonly used UNet backbone primarily integrates audio features into visual representations in the latent space via cross-attention mechanisms and multi-scale fusion, but it struggles to retain fine-grained lip details in the generated faces. To address this issue, we propose UnAvgLip, which extracts identity embeddings from reference videos to generate highly faithful facial sequences while maintaining accurate lip synchronization. Specifically, our method comprises two primary components: (1) an Identity Perceiver module that encodes facial embeddings to align with conditioned audio features; and (2) an ID-CrossAttn module that injects facial embeddings into the generation process, enhancing model’s capability of identity retention. Extensive experiments demonstrate that, at a modest training and inference cost, UnAvgLip effectively mitigates the “averaging” phenomenon in lip inpainting, significantly preserving unique facial characteristics while maintaining precise lip synchronization. Compared with the original approach, our method demonstrates significant improvements of 5% on the identity consistency metric and 2% on the SSIM metric across two benchmark datasets (HDTF and LRW).

Method: Proposed a stronger relaxation that incorporates anisotropic costs into certifiably optimal rotation averaging, addressing the limitations of existing isotropic solvers

Result: The proposed method recovers global optima in all tested datasets and leads to more accurate reconstructions in almost all scenes compared to existing isotropic solvers

Conclusion: Anisotropic rotation averaging with the proposed stronger relaxation provides certifiably optimal solutions and significantly improves reconstruction accuracy over traditional isotropic approaches

Abstract: Rotation averaging is a key subproblem in applications of computer vision and robotics. Many methods for solving this problem exist, and there are also several theoretical results analyzing difficulty and optimality. However, one aspect that most of these have in common is a focus on the isotropic setting, where the intrinsic uncertainties in the measurements are not fully incorporated into the resulting optimization task. Recent empirical results suggest that moving to an anisotropic framework, where these uncertainties are explicitly included, can result in an improvement of solution quality. However, global optimization for rotation averaging has remained a challenge in this scenario. In this work we show how anisotropic costs can be incorporated in certifiably optimal rotation averaging. We also demonstrate how existing solvers, designed for isotropic situations, fail in the anisotropic setting. Finally, we propose a stronger relaxation and empirically show that it recovers global optima in all tested datasets and leads to more accurate reconstructions in almost all scenes.

Method: Proposes CoViPAL with a Plug-and-Play Pruning Module (PPM) that predicts and removes redundant vision tokens before LVLM processing. The PPM is lightweight, model-agnostic, and operates independently of LVLM architecture.

Result: Extensive experiments show CoViPAL outperforms training-free pruning methods under equal token budgets and surpasses training-based methods with comparable supervision while maintaining accuracy.

Conclusion: CoViPAL provides a scalable and efficient solution to improve inference efficiency in LVLMs without compromising performance, offering seamless integration with various models.

Abstract: Large Vision-Language Models (LVLMs) process multimodal inputs consisting of text tokens and vision tokens extracted from images or videos. Due to the rich visual information, a single image can generate thousands of vision tokens, leading to high computational costs during the prefilling stage and significant memory overhead during decoding. Existing methods attempt to prune redundant vision tokens, revealing substantial redundancy in visual representations. However, these methods often struggle in shallow layers due to the lack of sufficient contextual information. We argue that many visual tokens are inherently redundant even in shallow layers and can be safely and effectively pruned with appropriate contextual signals. In this work, we propose CoViPAL, a layer-wise contextualized visual token pruning method that employs a Plug-and-Play Pruning Module (PPM) to predict and remove redundant vision tokens before they are processed by the LVLM. The PPM is lightweight, model-agnostic, and operates independently of the LVLM architecture, ensuring seamless integration with various models. Extensive experiments on multiple benchmarks demonstrate that CoViPAL outperforms training-free pruning methods under equal token budgets and surpasses training-based methods with comparable supervision. CoViPAL offers a scalable and efficient solution to improve inference efficiency in LVLMs without compromising accuracy.

Method: Generate pathology localization maps using cross-attention between medical text tokens and diffusion denoising network, aggregate attention across layers/heads/time steps, then optimize LDM decoder to incorporate adaptive watermarking that minimizes interference in critical diagnostic regions during image synthesis.

Result: Achieves state-of-the-art performance in both image quality and detection accuracy on MIMIC-CXR and OIA-ODIR datasets, preserving diagnostic integrity while ensuring watermark robustness.

Conclusion: MedSign provides an effective solution for ethical watermarking in medical image generation by adaptively preserving pathological regions, enabling reliable detection of AI-generated medical images without compromising clinical diagnostic value.

Abstract: As recent text-conditioned diffusion models have enabled the generation of high-quality images, concerns over their potential misuse have also grown. This issue is critical in the medical domain, where text-conditioned generated medical images could enable insurance fraud or falsified records, highlighting the urgent need for reliable safeguards against unethical use. While watermarking techniques have emerged as a promising solution in general image domains, their direct application to medical imaging presents significant challenges. A key challenge is preserving fine-grained disease manifestations, as even minor distortions from a watermark may lead to clinical misinterpretation, which compromises diagnostic integrity. To overcome this gap, we present MedSign, a deep learning-based watermarking framework specifically designed for text-to-medical image synthesis, which preserves pathologically significant regions by adaptively adjusting watermark strength. Specifically, we generate a pathology localization map using cross-attention between medical text tokens and the diffusion denoising network, aggregating token-wise attention across layers, heads, and time steps. Leveraging this map, we optimize the LDM decoder to incorporate watermarking during image synthesis, ensuring cohesive integration while minimizing interference in diagnostically critical regions. Experimental results show that our MedSign preserves diagnostic integrity while ensuring watermark robustness, achieving state-of-the-art performance in image quality and detection accuracy on MIMIC-CXR and OIA-ODIR datasets.

Method: The method discovers a new statistical relationship in transition operators between adjacent denoising steps, focusing on network outputs rather than internal structures. This relationship is used to estimate current transition operators based on adjacent steps without requiring specific network assumptions.

Result: LTC-Accel achieves 1.67x speedup in Stable Diffusion v2 and 1.55x speedup in video generation models. When combined with distillation models, it achieves 10x speedup in video generation (over 16FPS real-time generation) while maintaining competitive sample quality.

Conclusion: LTC-Accel provides a universal, training-free acceleration method that works with almost all diffusion-based methods and is orthogonal to existing techniques, enabling significant speed improvements without compromising quality.

Abstract: Text-based diffusion models have made significant breakthroughs in generating high-quality images and videos from textual descriptions. However, the lengthy sampling time of the denoising process remains a significant bottleneck in practical applications. Previous methods either ignore the statistical relationships between adjacent steps or rely on attention or feature similarity between them, which often only works with specific network structures. To address this issue, we discover a new statistical relationship in the transition operator between adjacent steps, focusing on the relationship of the outputs from the network. This relationship does not impose any requirements on the network structure. Based on this observation, we propose a novel training-free acceleration method called LTC-Accel, which uses the identified relationship to estimate the current transition operator based on adjacent steps. Due to no specific assumptions regarding the network structure, LTC-Accel is applicable to almost all diffusion-based methods and orthogonal to almost all existing acceleration techniques, making it easy to combine with them. Experimental results demonstrate that LTC-Accel significantly speeds up sampling in text-to-image and text-to-video synthesis while maintaining competitive sample quality. Specifically, LTC-Accel achieves a speedup of 1.67-fold in Stable Diffusion v2 and a speedup of 1.55-fold in video generation models. When combined with distillation models, LTC-Accel achieves a remarkable 10-fold speedup in video generation, allowing real-time generation of more than 16FPS.

Method: Developed a semi-automated VQA generation pipeline optimized for text-rich settings using refined stepwise image decomposition and a rigorous seven-metric evaluation protocol to ensure data quality.

Result: KRETA benchmark supports comprehensive evaluation across 15 domains and 26 image types, facilitating in-depth assessment of both visual text understanding and reasoning capabilities for Korean.

Conclusion: KRETA bridges the evaluation gap for Korean VLM research and provides an adaptable pipeline that can facilitate similar benchmark development for other languages, accelerating multilingual VLM research.

Abstract: Understanding and reasoning over text within visual contexts poses a significant challenge for Vision-Language Models (VLMs), given the complexity and diversity of real-world scenarios. To address this challenge, text-rich Visual Question Answering (VQA) datasets and benchmarks have emerged for high-resource languages like English. However, a critical gap persists for low-resource languages such as Korean, where the lack of comprehensive benchmarks hinders robust model evaluation and comparison. To bridge this gap, we introduce KRETA, a benchmark for Korean Reading and rEasoning in Text-rich VQA Attuned to diverse visual contexts. KRETA facilitates an in-depth evaluation of both visual text understanding and reasoning capabilities, while also supporting a multifaceted assessment across 15 domains and 26 image types. Additionally, we introduce a semi-automated VQA generation pipeline specifically optimized for text-rich settings, leveraging refined stepwise image decomposition and a rigorous seven-metric evaluation protocol to ensure data quality. While KRETA is tailored for Korean, we hope our adaptable and extensible pipeline will facilitate the development of similar benchmarks in other languages, thereby accelerating multilingual VLM research. The code and dataset for KRETA are available at https://github.com/tabtoyou/KRETA.

Method: Developed SBD algorithm inspired by human cognitive event segmentation theory. Uses prediction errors from a pretrained unconditional action-prediction model to detect skill boundaries - significant prediction error increases indicate skill shifts. Evaluated in Minecraft environment.

Result: SBD-generated segments improved conditioned policies by 63.7% and 52.1% on short-term atomic skill tasks, and hierarchical agents by 11.3% and 20.8% on long-horizon tasks. Method enables leveraging diverse YouTube videos for training instruction-following agents.

Conclusion: The proposed self-supervised SBD approach effectively segments long demonstration videos into skill-consistent segments without human annotation, significantly improving agent performance in both atomic and hierarchical tasks while enabling scalable learning from online video content.

Abstract: Learning skills in open-world environments is essential for developing agents capable of handling a variety of tasks by combining basic skills. Online demonstration videos are typically long but unsegmented, making them difficult to segment and label with skill identifiers. Unlike existing methods that rely on sequence sampling or human labeling, we have developed a self-supervised learning-based approach to segment these long videos into a series of semantic-aware and skill-consistent segments. Drawing inspiration from human cognitive event segmentation theory, we introduce Skill Boundary Detection (SBD), an annotation-free temporal video segmentation algorithm. SBD detects skill boundaries in a video by leveraging prediction errors from a pretrained unconditional action-prediction model. This approach is based on the assumption that a significant increase in prediction error indicates a shift in the skill being executed. We evaluated our method in Minecraft, a rich open-world simulator with extensive gameplay videos available online. Our SBD-generated segments improved the average performance of conditioned policies by 63.7% and 52.1% on short-term atomic skill tasks, and their corresponding hierarchical agents by 11.3% and 20.8% on long-horizon tasks. Our method can leverage the diverse YouTube videos to train instruction-following agents. The project page can be found in https://craftjarvis.github.io/SkillDiscovery.

Method: Proposed approach encodes scene chunks as uniform vector sets for better compression and performance than spatially structured latents. Trained an explicit outpainting model for unbounded generation to improve coherence and speed compared to resampling-based inpainting. Created NuiScene43 dataset for joint training.

Result: Method offers better compression and performance than prior approaches. The explicit outpainting model improves coherence and speeds up generation by eliminating extra diffusion steps. Model can blend different environments (e.g., rural houses and city skyscrapers) when trained on scenes of varying styles.

Conclusion: The proposed approach effectively addresses outdoor scene generation challenges, demonstrating the potential of heterogeneous scene curation for joint training and enabling coherent blending of diverse environmental styles within single scenes.

Abstract: In this paper, we explore the task of generating expansive outdoor scenes, ranging from castles to high-rises. Unlike indoor scene generation, which has been a primary focus of prior work, outdoor scene generation presents unique challenges, including wide variations in scene heights and the need for a method capable of rapidly producing large landscapes. To address this, we propose an efficient approach that encodes scene chunks as uniform vector sets, offering better compression and performance than the spatially structured latents used in prior methods. Furthermore, we train an explicit outpainting model for unbounded generation, which improves coherence compared to prior resampling-based inpainting schemes while also speeding up generation by eliminating extra diffusion steps. To facilitate this task, we curate NuiScene43, a small but high-quality set of scenes, preprocessed for joint training. Notably, when trained on scenes of varying styles, our model can blend different environments, such as rural houses and city skyscrapers, within the same scene, highlighting the potential of our curation process to leverage heterogeneous scenes for joint training.

Method: Multi-agent reinforcement learning framework where agents erase target regions from boxes to flip classification tags, using superpixel/supervoxel encoding, three specialized rewards (classification score + two intensity distribution rewards), and progressive curriculum learning.

Result: Outperforms state-of-the-art weakly-supervised methods and foundation models, achieving comparable performance to fully-supervised learning algorithms on large in-house BUS and ABUS datasets.

Conclusion: Flip Learning provides an effective weakly-supervised segmentation solution that eliminates the need for pixel-level annotations while delivering precise nodule segmentation results suitable for clinical applications.

Abstract: Accurate segmentation of nodules in both 2D breast ultrasound (BUS) and 3D automated breast ultrasound (ABUS) is crucial for clinical diagnosis and treatment planning. Therefore, developing an automated system for nodule segmentation can enhance user independence and expedite clinical analysis. Unlike fully-supervised learning, weakly-supervised segmentation (WSS) can streamline the laborious and intricate annotation process. However, current WSS methods face challenges in achieving precise nodule segmentation, as many of them depend on inaccurate activation maps or inefficient pseudo-mask generation algorithms. In this study, we introduce a novel multi-agent reinforcement learning-based WSS framework called Flip Learning, which relies solely on 2D/3D boxes for accurate segmentation. Specifically, multiple agents are employed to erase the target from the box to facilitate classification tag flipping, with the erased region serving as the predicted segmentation mask. The key contributions of this research are as follows: (1) Adoption of a superpixel/supervoxel-based approach to encode the standardized environment, capturing boundary priors and expediting the learning process. (2) Introduction of three meticulously designed rewards, comprising a classification score reward and two intensity distribution rewards, to steer the agents’ erasing process precisely, thereby avoiding both under- and over-segmentation. (3) Implementation of a progressive curriculum learning strategy to enable agents to interact with the environment in a progressively challenging manner, thereby enhancing learning efficiency. Extensively validated on the large in-house BUS and ABUS datasets, our Flip Learning method outperforms state-of-the-art WSS methods and foundation models, and achieves comparable performance as fully-supervised learning algorithms.

Method: Developed (1) a scalable system for 3D reconstruction of table tennis matches from monocular video, and (2) an uncertainty-aware controller that anticipates opponent actions.

Result: In simulation, the policy improved ball return rate against high-speed hits from 49.9% to 59.0% compared to a baseline non-anticipatory policy.

Conclusion: The proposed anticipatory agent with uncertainty-aware control and scalable 3D reconstruction significantly enhances table tennis gameplay performance by effectively predicting opponent actions.

Abstract: Physical agility is a necessary skill in competitive table tennis, but by no means sufficient. Champions excel in this fast-paced and highly dynamic environment by anticipating their opponent’s intent - buying themselves the necessary time to react. In this work, we take one step towards designing such an anticipatory agent. Previous works have developed systems capable of real-time table tennis gameplay, though they often do not leverage anticipation. Among the works that forecast opponent actions, their approaches are limited by dataset size and variety. Our paper contributes (1) a scalable system for reconstructing monocular video of table tennis matches in 3D and (2) an uncertainty-aware controller that anticipates opponent actions. We demonstrate in simulation that our policy improves the ball return rate against high-speed hits from 49.9% to 59.0% as compared to a baseline non-anticipatory policy.

Method: Proposes PS-ReID with dual-path asymmetric encoding: query branch captures identity-discriminative cues, target branch performs holistic scene reasoning. Uses token-level ReID loss to supervise identity-aware tokens, coupling retrieval and segmentation. Built M2ReID dataset with 200K+ images and 4,894 identities.

Result: PS-ReID significantly outperforms unimodal query-based models in both ReID and segmentation tasks. Excels in challenging scenarios like occlusion, low lighting, and background clutter.

Conclusion: The model offers a robust and flexible solution for person retrieval and segmentation, demonstrating the effectiveness of multimodal integration in ReID tasks. All resources will be publicly available.

Abstract: Person re-identification (ReID) plays a critical role in applications such as security surveillance and criminal investigations. Most traditional image-based ReID methods face challenges including occlusions and lighting changes, while text provides complementary information to mitigate these issues. However, the integration of both image and text modalities remains underexplored. To address this gap, we propose {\bf PS-ReID}, a multimodal model that combines image and text inputs to enhance ReID performance. In contrast to existing ReID methods limited by cropped pedestrian images, our PS-ReID focuses on full-scene settings and introduces a multimodal ReID task that incorporates segmentation, enabling precise feature extraction of the queried individual, even under challenging conditions such as occlusion. To this end, our model adopts a dual-path asymmetric encoding scheme that explicitly separates query and target roles: the query branch captures identity-discriminative cues, while the target branch performs holistic scene reasoning. Additionally, a token-level ReID loss supervises identity-aware tokens, coupling retrieval and segmentation to yield masks that are both spatially precise and identity-consistent. To facilitate systematic evaluation, we construct M2ReID, currently the largest full-scene multimodal ReID dataset, with over 200K images and 4,894 identities, featuring multimodal queries and high-quality segmentation masks. Experimental results demonstrate that PS-ReID significantly outperforms unimodal query-based models in both ReID and segmentation tasks. The model excels in challenging real-world scenarios such as occlusion, low lighting, and background clutter, offering a robust and flexible solution for person retrieval and segmentation. All code, models, and datasets will be publicly available.

Method: Developed an interpretable-by-design hybrid fully convolutional CNN-Transformer architecture that generates class-specific sparse evidence maps in a single forward pass, unlike post-hoc saliency methods.

Result: Achieved state-of-the-art predictive performance on retinal disease detection tasks using color fundus images, outperforming both black-box and interpretable models while providing faithful localized evidence.

Conclusion: The proposed architecture successfully combines the strengths of CNNs and Transformers while maintaining interpretability, making it suitable for medical imaging applications where both performance and transparency are critical.

Abstract: In many medical imaging tasks, convolutional neural networks (CNNs) efficiently extract local features hierarchically. More recently, vision transformers (ViTs) have gained popularity, using self-attention mechanisms to capture global dependencies, but lacking the inherent spatial localization of convolutions. Therefore, hybrid models combining CNNs and ViTs have been developed to combine the strengths of both architectures. However, such hybrid models are difficult to interpret, which hinders their application in medical imaging. In this work, we introduce an interpretable-by-design hybrid fully convolutional CNN-Transformer architecture for retinal disease detection. Unlike widely used post-hoc saliency methods for ViTs, our approach generates faithful and localized evidence maps that directly reflect the mode’s decision process. We evaluated our method on two medical tasks focused on disease detection using color fundus images. Our model achieves state-of-the-art predictive performance compared to black-box and interpretable models and provides class-specific sparse evidence maps in a single forward pass. The code is available at: https://github.com/kdjoumessi/Self-Explainable-CNN-Transformer.

Method: Proposed a novel multi-stage deep learning approach using auto-context concept to unify tissue and nuclei information. Includes pre-training and post-processing stages to perform segmentation and classification in melanoma histological images.

Result: Achieved second and first place rankings in PUMA challenge with average micro Dice tissue score of 73.40% for Track 1 and summed nuclei F1-score of 63.48% for Track 2. Demonstrated effectiveness through ablation study and generalization on external dataset.

Conclusion: The unified framework combining tissue and nuclei information proves effective for melanoma histological image analysis, showing strong performance in segmentation and classification tasks with good generalization capabilities.

Abstract: Melanoma is the most lethal form of skin cancer, with an increasing incidence rate worldwide. Analyzing histological images of melanoma by localizing and classifying tissues and cell nuclei is considered the gold standard method for diagnosis and treatment options for patients. While many computerized approaches have been proposed for automatic analysis, most perform tissue-based analysis and nuclei (cell)-based analysis as separate tasks, which might be suboptimal. In this work, using the PUMA challenge dataset, we propose a novel multi-stage deep learning approach by combining tissue and nuclei information in a unified framework based on the auto-context concept to perform segmentation and classification in histological images of melanoma. Through pre-training and further post-processing, our approach achieved second and first place rankings in the PUMA challenge, with average micro Dice tissue score and summed nuclei F1-score of 73.40% for Track 1 and 63.48% for Track 2, respectively. Furthermore, through a comprehensive ablation study and additional evaluation on an external dataset, we demonstrated the effectiveness of the framework components as well as the generalization capabilities of the proposed approach. Our implementation for training and testing is available at: https://github.com/NimaTorbati/PumaSubmit

Method: Leverages automatically extracted modality-specific tags from foundation models, aligns modalities in latent space, and learns auxiliary latent concepts from video and caption features to improve concept distinction.

Result: Outperforms current state-of-the-art methods across three datasets (MSR-VTT splits, DiDeMo, TGIF, Charades, YouCook2) and performs comparably or better on others, demonstrating improved cross-modal alignment.

Conclusion: Modality-specific tags significantly enhance video retrieval by improving the alignment of visual and textual latent concepts, allowing better concept distinction and superior performance across diverse datasets.

Abstract: Video retrieval requires aligning visual content with corresponding natural language descriptions. In this paper, we introduce Modality Auxiliary Concepts for Video Retrieval (MAC-VR), a novel approach that leverages modality-specific tags – automatically extracted from foundation models – to enhance video retrieval. We propose to align modalities in a latent space, along with learning and aligning auxiliary latent concepts derived from the features of a video and its corresponding caption. We introduce these auxiliary concepts to improve the alignment of visual and textual latent concepts, allowing concepts to be distinguished from one another. We conduct extensive experiments on six diverse datasets: two different splits of MSR-VTT, DiDeMo, TGIF, Charades and YouCook2. The experimental results consistently demonstrate that modality-specific tags improve cross-modal alignment, outperforming current state-of-the-art methods across three datasets and performing comparably or better across others. Project Webpage: https://adrianofragomeni.github.io/MAC-VR/

Method: Developed a Shadow Erosion and Nighttime Adaptability pipeline that preserves color and texture details, compared against the widely used CLAHE technique.

Result: Significant improvement over CLAHE in both illumination uniformity and visual perception quality metrics, and enhanced performance of YOLO-based drivable area segmentation algorithm.

Conclusion: The proposed pipeline effectively addresses shadow and nighttime challenges in autonomous driving imagery, providing superior image enhancement compared to traditional methods.

Abstract: Enhancement of images from RGB cameras is of particular interest due to its wide range of ever-increasing applications such as medical imaging, satellite imaging, automated driving, etc. In autonomous driving, various techniques are used to enhance image quality under challenging lighting conditions. These include artificial augmentation to improve visibility in poor nighttime conditions, illumination-invariant imaging to reduce the impact of lighting variations, and shadow mitigation to ensure consistent image clarity in bright daylight. This paper proposes a pipeline for Shadow Erosion and Nighttime Adaptability in images for automated driving applications while preserving color and texture details. The Shadow Erosion and Nighttime Adaptability pipeline is compared to the widely used CLAHE technique and evaluated based on illumination uniformity and visual perception quality metrics. The results also demonstrate a significant improvement over CLAHE, enhancing a YOLO-based drivable area segmentation algorithm.

Method: Uses frozen CLIP model to generate image and text embeddings, employs hierarchical attention mechanism to jointly learn single-domain and cross-domain preferences, mimicking human information integration processes.

Result: Outperforms existing methods on four e-commerce datasets, demonstrating superior capability in capturing cross-domain user interests and sequential decision-making patterns.

Conclusion: Successfully bridges cognitive principles with computational models, highlighting the significant role of multimodal data in enhancing cross-domain sequential recommendation systems.

Abstract: Cross-Domain Sequential Recommendation (CDSR) predicts user behavior by leveraging historical interactions across multiple domains, focusing on modeling cross-domain preferences through intra- and inter-sequence item relationships. Inspired by human cognitive processes, we propose Hierarchical Attention Fusion of Visual and Textual Representations (HAF-VT), a novel approach integrating visual and textual data to enhance cognitive modeling. Using the frozen CLIP model, we generate image and text embeddings, enriching item representations with multimodal data. A hierarchical attention mechanism jointly learns single-domain and cross-domain preferences, mimicking human information integration. Evaluated on four e-commerce datasets, HAF-VT outperforms existing methods in capturing cross-domain user interests, bridging cognitive principles with computational models and highlighting the role of multimodal data in sequential decision-making.

Method: Leverages a diffusion model to jointly model low-level image latents from a VAE and high-level semantic features from a pretrained self-supervised encoder (like DINO), learning to generate coherent image-feature pairs from noise with minimal modifications to standard Diffusion Transformer architectures.

Result: Significant improvements in both image quality and training convergence speed in conditional and unconditional settings, while eliminating the need for complex distillation objectives.

Conclusion: Establishes a new direction for representation-aware generative modeling with a unified design that simplifies training and enables powerful semantic guidance during inference.

Abstract: Latent diffusion models (LDMs) dominate high-quality image generation, yet integrating representation learning with generative modeling remains a challenge. We introduce a novel generative image modeling framework that seamlessly bridges this gap by leveraging a diffusion model to jointly model low-level image latents (from a variational autoencoder) and high-level semantic features (from a pretrained self-supervised encoder like DINO). Our latent-semantic diffusion approach learns to generate coherent image-feature pairs from pure noise, significantly enhancing both generative quality and training efficiency, all while requiring only minimal modifications to standard Diffusion Transformer architectures. By eliminating the need for complex distillation objectives, our unified design simplifies training and unlocks a powerful new inference strategy: Representation Guidance, which leverages learned semantics to steer and refine image generation. Evaluated in both conditional and unconditional settings, our method delivers substantial improvements in image quality and training convergence speed, establishing a new direction for representation-aware generative modeling. Project page and code: https://representationdiffusion.github.io

Method: Proposes three key components: 1) Hierarchical Depth-Aware Head for multi-scale depth features to mitigate flat-ground assumption, 2) Depth Prior Distillation to transfer semantic depth knowledge from teacher model, and 3) Conditional Random Field module to enforce spatial coherence and smooth lane reconstruction.

Result: Achieves state-of-the-art performance in terms of z-axis error and outperforms other methods in overall performance, as validated through extensive experiments.

Conclusion: The proposed framework effectively addresses the challenges of monocular 3D lane detection by enhancing depth awareness, leveraging semantic depth knowledge, and ensuring spatial coherence, leading to improved accuracy in 3D lane reconstruction.

Abstract: Monocular 3D lane detection is challenging due to the difficulty in capturing depth information from single-camera images. A common strategy involves transforming front-view (FV) images into bird’s-eye-view (BEV) space through inverse perspective mapping (IPM), facilitating lane detection using BEV features. However, IPM’s flat-ground assumption and loss of contextual information lead to inaccuracies in reconstructing 3D information, especially height. In this paper, we introduce a BEV-based framework to address these limitations and improve 3D lane detection accuracy. Our approach incorporates a Hierarchical Depth-Aware Head that provides multi-scale depth features, mitigating the flat-ground assumption by enhancing spatial awareness across varying depths. Additionally, we leverage Depth Prior Distillation to transfer semantic depth knowledge from a teacher model, capturing richer structural and contextual information for complex lane structures. To further refine lane continuity and ensure smooth lane reconstruction, we introduce a Conditional Random Field module that enforces spatial coherence in lane predictions. Extensive experiments validate that our method achieves state-of-the-art performance in terms of z-axis error and outperforms other methods in the field in overall performance. The code is released at: https://anonymous.4open.science/r/Depth3DLane-DCDD.

Method: Proposes a hybrid framework integrating oriented bounding box (OBB) detection using YOLOv11 for 9 key categories (GD&T, tolerances, measures, materials, etc.) with transformer-based Donut model for structured JSON output. Uses in-house annotated dataset and compares single model vs category-specific fine-tuning strategies.

Result: Single model consistently outperforms category-specific models across all metrics: achieves 94.77% precision for GD&T, 100% recall for most categories, 97.3% F1 score, while reducing hallucination to 5.23%.

Conclusion: The proposed hybrid framework significantly improves accuracy, reduces manual effort, and supports scalable deployment in precision-driven industries by providing structured information extraction from complex engineering drawings.

Abstract: Accurate extraction of key information from 2D engineering drawings is crucial for high-precision manufacturing. Manual extraction is time-consuming and error-prone, while traditional Optical Character Recognition (OCR) techniques often struggle with complex layouts and overlapping symbols, resulting in unstructured outputs. To address these challenges, this paper proposes a novel hybrid deep learning framework for structured information extraction by integrating an oriented bounding box (OBB) detection model with a transformer-based document parsing model (Donut). An in-house annotated dataset is used to train YOLOv11 for detecting nine key categories: Geometric Dimensioning and Tolerancing (GD&T), General Tolerances, Measures, Materials, Notes, Radii, Surface Roughness, Threads, and Title Blocks. Detected OBBs are cropped into images and labeled to fine-tune Donut for structured JSON output. Fine-tuning strategies include a single model trained across all categories and category-specific models. Results show that the single model consistently outperforms category-specific ones across all evaluation metrics, achieving higher precision (94.77% for GD&T), recall (100% for most), and F1 score (97.3%), while reducing hallucination (5.23%). The proposed framework improves accuracy, reduces manual effort, and supports scalable deployment in precision-driven industries.

Method: FedMVP uses a PromptFormer module with cross-attention to align textual and visual features, generating multimodal visual prompts. These prompts are input to CLIP’s frozen vision encoder and trained with CLIP similarity loss plus consistency loss.

Result: Extensive evaluation on 20 datasets across three generalization settings shows FedMVP preserves in-distribution performance while achieving +1.57%-2.26% better generalization to unseen classes/domains compared to state-of-the-art methods.

Conclusion: Multimodal visual prompt tuning with cross-attention alignment effectively addresses overfitting issues in federated learning, significantly improving generalization capabilities while maintaining performance on known concepts.

Abstract: In federated learning, textual prompt tuning adapts Vision-Language Models (e.g., CLIP) by tuning lightweight input tokens (or prompts) on local client data, while keeping network weights frozen. After training, only the prompts are shared by the clients with the central server for aggregation. However, textual prompt tuning suffers from overfitting to known concepts, limiting its generalizability to unseen concepts. To address this limitation, we propose Multimodal Visual Prompt Tuning (FedMVP) that conditions the prompts on multimodal contextual information - derived from the input image and textual attribute features of a class. At the core of FedMVP is a PromptFormer module that synergistically aligns textual and visual features through a cross-attention mechanism. The dynamically generated multimodal visual prompts are then input to the frozen vision encoder of CLIP, and trained with a combination of CLIP similarity loss and a consistency loss. Extensive evaluation on 20 datasets, spanning three generalization settings, demonstrates that FedMVP not only preserves performance on in-distribution classes and domains, but also displays higher generalizability to unseen classes and domains, surpassing state-of-the-art methods by a notable margin of +1.57% - 2.26%. Code is available at https://github.com/mainaksingha01/FedMVP.

Method: Develop RQNet neural network to characterize bitrate-quality relationship based on video content and coding context, integrate with least-squares method using previous frames for online parameter adaptation.

Result: Achieves significantly smaller bitrate deviations than baseline methods on common datasets with minimal additional computational complexity.

Conclusion: The proposed R-Q model provides accurate rate-quality estimation and enables effective online parameter adaptation, improving learned video coding performance.

Abstract: Learned video coding (LVC) has recently achieved superior coding performance. In this paper, we model the rate-quality (R-Q) relationship for learned video coding by a parametric function. We learn a neural network, termed RQNet, to characterize the relationship between the bitrate and quality level according to video content and coding context. The predicted (R,Q) results are further integrated with those from previously coded frames using the least-squares method to determine the parameters of our R-Q model on-the-fly. Compared to the conventional approaches, our method accurately estimates the R-Q relationship, enabling the online adaptation of model parameters to enhance both flexibility and precision. Experimental results show that our R-Q model achieves significantly smaller bitrate deviations than the baseline method on commonly used datasets with minimal additional complexity.

Method: Three-module framework: 1) Multi-granularity Uncertainty Estimation (MUE) for identifying potential targets with confidence scores, 2) Prototype-based Uncertainty Decoupling (PUD) for visual context decoupling and prototype mining at cluster and individual levels, 3) Cross-modal Re-identification for evaluating candidates with varying confidence levels.

Result: Experiments on CUHK-SYSU-TBPS and PRW-TBPS datasets validate the effectiveness of the proposed framework.

Conclusion: UPD-TBPS successfully addresses uncertainties in text-based pedestrian search through its three-module approach, improving detection and retrieval accuracy in complex scenes.

Abstract: Text-based pedestrian search (TBPS) in full images aims to locate a target pedestrian in untrimmed images using natural language descriptions. However, in complex scenes with multiple pedestrians, existing methods are limited by uncertainties in detection and matching, leading to degraded performance. To address this, we propose UPD-TBPS, a novel framework comprising three modules: Multi-granularity Uncertainty Estimation (MUE), Prototype-based Uncertainty Decoupling (PUD), and Cross-modal Re-identification (ReID). MUE conducts multi-granularity queries to identify potential targets and assigns confidence scores to reduce early-stage uncertainty. PUD leverages visual context decoupling and prototype mining to extract features of the target pedestrian described in the query. It separates and learns pedestrian prototype representations at both the coarse-grained cluster level and the fine-grained individual level, thereby reducing matching uncertainty. ReID evaluates candidates with varying confidence levels, improving detection and retrieval accuracy. Experiments on CUHK-SYSU-TBPS and PRW-TBPS datasets validate the effectiveness of our framework.

Method: Constructed SoccerWiki (multimodal knowledge base), created SoccerBench (10K multimodal QA benchmark), and developed SoccerAgent (multi-agent system for collaborative reasoning using domain expertise).

Result: The proposed framework achieves superior performance compared to representative multimodal large language models (MLLMs) on the comprehensive SoccerBench benchmark.

Conclusion: The holistic soccer understanding framework successfully bridges the gap in existing research by providing integrated knowledge, comprehensive evaluation, and robust agentic reasoning capabilities for complex soccer analysis.

Abstract: Recent advances in soccer understanding have demonstrated rapid progress, yet existing research predominantly focuses on isolated or narrow tasks. To bridge this gap, we propose a comprehensive framework for holistic soccer understanding. Concretely, we make the following contributions in this paper: (i) we construct SoccerWiki, the first large-scale multimodal soccer knowledge base, integrating rich domain knowledge about players, teams, referees, and venues to enable knowledge-driven reasoning; (ii) we present SoccerBench, the largest and most comprehensive soccer-specific benchmark, featuring around 10K multimodal (text, image, video) multi-choice QA pairs across 13 distinct tasks; (iii) we introduce SoccerAgent, a novel multi-agent system that decomposes complex soccer questions via collaborative reasoning, leveraging domain expertise from SoccerWiki and achieving robust performance; (iv) extensive evaluations and comparisons with representative MLLMs on SoccerBench highlight the superiority of our agentic system.

Method: Benchmark spanning 200+ combinations of model architectures, datasets and training strategies across three datasets (FaultSeg3D, CRACKS, Thebe), systematically assessing pretraining, fine-tuning, and joint training under varying domain shifts.

Result: Fine-tuning practices can cause catastrophic forgetting with disjoint datasets; larger models like Segformer are more robust; domain adaptation outperforms fine-tuning for large shifts but underperforms for similar domains; novel fault characteristic analysis reveals structural biases.

Conclusion: Establishes robust experimental baseline providing insights into tradeoffs in fault delineation workflows and highlights directions for building more generalizable and interpretable seismic interpretation models.

Abstract: Machine learning has taken a critical role in seismic interpretation workflows, especially in fault delineation tasks. However, despite the recent proliferation of pretrained models and synthetic datasets, the field still lacks a systematic understanding of the generalizability limits of these models across seismic data representing diverse geologic, acquisition and processing settings. Distributional shifts between data sources, limitations in fine-tuning strategies and labeled data accessibility, and inconsistent evaluation protocols all remain major roadblocks to deploying reliable models in real-world exploration. In this paper, we present the first large-scale benchmarking study explicitly designed to provide guidelines for domain shift strategies in seismic interpretation. Our benchmark spans over 200 combinations of model architectures, datasets and training strategies, across three datasets (synthetic and real) including FaultSeg3D, CRACKS, and Thebe. We systematically assess pretraining, fine-tuning, and joint training under varying domain shifts. Our analysis shows that common fine-tuning practices can lead to catastrophic forgetting, especially when source and target datasets are disjoint, and that larger models such as Segformer are more robust than smaller architectures. We also find that domain adaptation methods outperform fine-tuning when shifts are large, yet underperform when domains are similar. Finally, we complement segmentation metrics with a novel analysis based on fault characteristic descriptors, revealing how models absorb structural biases from training datasets. Overall, we establish a robust experimental baseline that provides insights into tradeoffs in current fault delineation workflows and highlights directions for building more generalizable and interpretable models.

Method: ViEEG decomposes visual stimuli into three biologically aligned components (contour, foreground object, contextual scene) and uses a three-stream EEG encoder with cross-attention routing to progressively integrate features, simulating cortical information flow from low-level to high-level vision.

Result: Extensive experiments on THINGS-EEG dataset show ViEEG significantly outperforms previous methods by a large margin. Results on THINGS-MEG dataset confirm generalization to different neural modalities.

Conclusion: ViEEG not only advances performance but also sets a new paradigm for EEG brain decoding by properly modeling the brain’s hierarchical visual processing structure.

Abstract: Understanding and decoding brain activity into visual representations is a fundamental challenge at the intersection of neuroscience and artificial intelligence. While EEG visual decoding has shown promise due to its non-invasive, and low-cost nature, existing methods suffer from Hierarchical Neural Encoding Neglect (HNEN)-a critical limitation where flat neural representations fail to model the brain’s hierarchical visual processing hierarchy. Inspired by the hierarchical organization of visual cortex, we propose ViEEG, a neuro-We further adopt hierarchical contrastive learning for EEG-CLIP representation alignment, enabling zero-shot object recognition. Extensive experiments on the THINGS-EEG dataset demonstrate that ViEEG significantly outperforms previous methods by a large margin in both subject-dependent and subject-independent settings. Results on the THINGS-MEG dataset further confirm ViEEG’s generalization to different neural modalities. Our framework not only advances the performance frontier but also sets a new paradigm for EEG brain decoding. inspired framework that addresses HNEN. ViEEG decomposes each visual stimulus into three biologically aligned components-contour, foreground object, and contextual scene-serving as anchors for a three-stream EEG encoder. These EEG features are progressively integrated via cross-attention routing, simulating cortical information flow from low-level to high-level vision.

Method: Created GenBuster-200K dataset with diverse generative techniques and real-world scenes. Developed BusterX framework combining multimodal large language model (MLLM) with reinforcement learning for detection and explainable rationale.

Result: Extensive comparisons show BusterX’s effectiveness and generalizability. The framework provides both authenticity determination and explainable decision-making.

Conclusion: First large-scale AI-generated video dataset with latest techniques and first explainable detection framework using MLLM+RL, addressing critical gaps in deepfake video detection and explanation.

Abstract: Advances in AI generative models facilitate super-realistic video synthesis, amplifying misinformation risks via social media and eroding trust in digital content. Several research works have explored new deepfake detection methods on AI-generated images to alleviate these risks. However, with the fast development of video generation models, such as Sora and WanX, there is currently a lack of large-scale, high-quality AI-generated video datasets for forgery detection. In addition, existing detection approaches predominantly treat the task as binary classification, lacking explainability in model decision-making and failing to provide actionable insights or guidance for the public. To address these challenges, we propose \textbf{GenBuster-200K}, a large-scale AI-generated video dataset featuring 200K high-resolution video clips, diverse latest generative techniques, and real-world scenes. We further introduce \textbf{BusterX}, a novel AI-generated video detection and explanation framework leveraging multimodal large language model (MLLM) and reinforcement learning for authenticity determination and explainable rationale. To our knowledge, GenBuster-200K is the {\it \textbf{first}} large-scale, high-quality AI-generated video dataset that incorporates the latest generative techniques for real-world scenarios. BusterX is the {\it \textbf{first}} framework to integrate MLLM with reinforcement learning for explainable AI-generated video detection. Extensive comparisons with state-of-the-art methods and ablation studies validate the effectiveness and generalizability of BusterX. The code, models, and datasets will be released.

Method: Proposes DiffDecompose - a diffusion Transformer-based framework that learns posterior over possible layer decompositions using input image, semantic prompts, and blending type. Uses In-Context Decomposition and Layer Position Encoding Cloning to maintain pixel correspondence.

Result: Extensive experiments on AlphaBlend dataset (6 real-world subtasks) and public LOGO dataset verify effectiveness. Outperforms existing methods in transparent/semi-transparent layer decomposition.

Conclusion: The approach successfully addresses layer ambiguity, generalization, and data scarcity challenges in alpha-composited image decomposition, providing a robust solution for various real-world applications.

Abstract: Diffusion models have recently motivated great success in many generation tasks like object removal. Nevertheless, existing image decomposition methods struggle to disentangle semi-transparent or transparent layer occlusions due to mask prior dependencies, static object assumptions, and the lack of datasets. In this paper, we delve into a novel task: Layer-Wise Decomposition of Alpha-Composited Images, aiming to recover constituent layers from single overlapped images under the condition of semi-transparent/transparent alpha layer non-linear occlusion. To address challenges in layer ambiguity, generalization, and data scarcity, we first introduce AlphaBlend, the first large-scale and high-quality dataset for transparent and semi-transparent layer decomposition, supporting six real-world subtasks (e.g., translucent flare removal, semi-transparent cell decomposition, glassware decomposition). Building on this dataset, we present DiffDecompose, a diffusion Transformer-based framework that learns the posterior over possible layer decompositions conditioned on the input image, semantic prompts, and blending type. Rather than regressing alpha mattes directly, DiffDecompose performs In-Context Decomposition, enabling the model to predict one or multiple layers without per-layer supervision, and introduces Layer Position Encoding Cloning to maintain pixel-level correspondence across layers. Extensive experiments on the proposed AlphaBlend dataset and public LOGO dataset verify the effectiveness of DiffDecompose. The code and dataset will be available upon paper acceptance. Our code will be available at: https://github.com/Wangzt1121/DiffDecompose.

Method: Proposes decoupled RGB channel learning guided by underwater physics, frame interpolation with adaptive weighting, and a novel loss function for noise reduction and edge preservation.

Result: Outperforms state-of-the-art methods with PSNR gains up to 1.90dB, delivering superior perceptual quality and robustness in deep-sea environments.

Conclusion: The framework offers promising directions for marine robotics and underwater visual analytics, with code and new Submerged3D dataset publicly available.

Abstract: Reconstructing high-fidelity underwater scenes remains a challenging task due to light absorption, scattering, and limited visibility inherent in aquatic environments. This paper presents an enhanced Gaussian Splatting-based framework that improves both the visual quality and geometric accuracy of deep underwater rendering. We propose decoupled learning for RGB channels, guided by the physics of underwater attenuation, to enable more accurate colour restoration. To address sparse-view limitations and improve view consistency, we introduce a frame interpolation strategy with a novel adaptive weighting scheme. Additionally, we introduce a new loss function aimed at reducing noise while preserving edges, which is essential for deep-sea content. We also release a newly collected dataset, Submerged3D, captured specifically in deep-sea environments. Experimental results demonstrate that our framework consistently outperforms state-of-the-art methods with PSNR gains up to 1.90dB, delivering superior perceptual quality and robustness, and offering promising directions for marine robotics and underwater visual analytics. The code of RUSplatting is available at https://github.com/theflash987/RUSplatting and the dataset Submerged3D can be downloaded at https://zenodo.org/records/15482420.

Method: Redesigned Neural Radiance Fields (NeRFs) that can learn from multipath signals, specifically using sparse WiFi measurements at multiple indoor locations to model the environment.

Result: Successfully inferred indoor floorplans from WiFi measurements, with promising results that enable forward applications like indoor signal prediction and basic ray tracing.

Conclusion: NeRFs can be effectively adapted to work with multipath signals, demonstrating the ability to “see” environments through RF measurements and opening new possibilities for indoor mapping and signal analysis.

Abstract: Neural Radiance Fields (NeRFs) have been remarkably successful at synthesizing novel views of 3D scenes by optimizing a volumetric scene function. This scene function models how optical rays bring color information from a 3D object to the camera pixels. Radio frequency (RF) or audio signals can also be viewed as a vehicle for delivering information about the environment to a sensor. However, unlike camera pixels, an RF/audio sensor receives a mixture of signals that contain many environmental reflections (also called “multipath”). Is it still possible to infer the environment using such multipath signals? We show that with redesign, NeRFs can be taught to learn from multipath signals, and thereby “see” the environment. As a grounding application, we aim to infer the indoor floorplan of a home from sparse WiFi measurements made at multiple locations inside the home. Although a difficult inverse problem, our implicitly learnt floorplans look promising, and enables forward applications, such as indoor signal prediction and basic ray tracing.

Method: Created So-Fake-Set dataset with 2M+ images from 35 state-of-the-art generative models, established So-Fake-OOD benchmark (100K images) for cross-domain testing, and developed So-Fake-R1 vision-language framework using reinforcement learning for detection, localization, and explainable inference.

Result: So-Fake-R1 outperforms second-best method with 1.3% gain in detection accuracy and 4.5% increase in localization IoU, demonstrating superior performance on the challenging benchmark.

Conclusion: This work establishes a new foundation for social media-centric forgery detection research by integrating scalable dataset, challenging OOD benchmark, and advanced detection framework, with public release of code, models, and datasets.

Abstract: Recent advances in AI-powered generative models have enabled the creation of increasingly realistic synthetic images, posing significant risks to information integrity and public trust on social media platforms. While robust detection frameworks and diverse, large-scale datasets are essential to mitigate these risks, existing academic efforts remain limited in scope: current datasets lack the diversity, scale, and realism required for social media contexts, while detection methods struggle with generalization to unseen generative technologies. To bridge this gap, we introduce So-Fake-Set, a comprehensive social media-oriented dataset with over 2 million high-quality images, diverse generative sources, and photorealistic imagery synthesized using 35 state-of-the-art generative models. To rigorously evaluate cross-domain robustness, we establish a novel and large-scale (100K) out-of-domain benchmark (So-Fake-OOD) featuring synthetic imagery from commercial models explicitly excluded from the training distribution, creating a realistic testbed for evaluating real-world performance. Leveraging these resources, we present So-Fake-R1, an advanced vision-language framework that employs reinforcement learning for highly accurate forgery detection, precise localization, and explainable inference through interpretable visual rationales. Extensive experiments show that So-Fake-R1 outperforms the second-best method, with a 1.3% gain in detection accuracy and a 4.5% increase in localization IoU. By integrating a scalable dataset, a challenging OOD benchmark, and an advanced detection framework, this work establishes a new foundation for social media-centric forgery detection research. The code, models, and datasets will be released publicly.

Method: Created MedEBench benchmark with clinically sourced data, three evaluation metrics (Editing Accuracy, Contextual Preservation, Visual Quality), ROI masks, and systematic comparison of 7 state-of-the-art models using attention grounding and IoU analysis.

Result: The benchmark reveals common failure patterns in existing models and provides a failure analysis protocol using attention map-ROI IoU to identify mislocalization issues.

Conclusion: MedEBench establishes a solid foundation for developing and evaluating reliable, clinically meaningful medical image editing systems with standardized evaluation framework.

Abstract: Text-guided image editing has seen rapid progress in natural image domains, but its adaptation to medical imaging remains limited and lacks standardized evaluation. Clinically, such editing holds promise for simulating surgical outcomes, creating personalized teaching materials, and enhancing patient communication. To bridge this gap, we introduce MedEBench, a comprehensive benchmark for evaluating text-guided medical image editing. It consists of 1,182 clinically sourced image-prompt triplets spanning 70 tasks across 13 anatomical regions. MedEBench offers three key contributions: (1) a clinically relevant evaluation framework covering Editing Accuracy, Contextual Preservation, and Visual Quality, supported by detailed descriptions of expected change and ROI (Region of Interest) masks; (2) a systematic comparison of seven state-of-the-art models, revealing common failure patterns; and (3) a failure analysis protocol based on attention grounding, using IoU between attention maps and ROIs to identify mislocalization. MedEBench provides a solid foundation for developing and evaluating reliable, clinically meaningful medical image editing systems. Project website: https://mliuby.github.io/MedEBench_Website/

Method: Integrates optimized encoder-decoder architecture with specialized multi-scale feature fusion and auxiliary supervision strategies. Introduces two novel datasets: Controlled Carbon Dioxide Release (CCR) dataset and Real Time Ankom (RTA) dataset for dairy cow rumen emissions.

Result: Achieves 84.88% mIoU on CCR dataset and 92.98% mIoU on RTA dataset with only 5.07M parameters and 84.68 FPS. Outperforms other lightweight methods like SegFormer-B0 and SegNeXt, especially in low-flow scenarios.

Conclusion: CarboFormer provides robust and efficient tools for CO2 emission analysis, advancing environmental sensing and precision livestock management, making it suitable for real-time monitoring on resource-constrained platforms like drones.

Abstract: Carbon dioxide (CO$_2$) emissions are critical indicators of both environmental impact and various industrial processes, including livestock management. We introduce CarboFormer, a lightweight semantic segmentation framework for Optical Gas Imaging (OGI), designed to detect and quantify CO$_2$ emissions across diverse applications. Our approach integrates an optimized encoder-decoder architecture with specialized multi-scale feature fusion and auxiliary supervision strategies to effectively model both local details and global relationships in gas plume imagery while achieving competitive accuracy with minimal computational overhead for resource-constrained environments. We contribute two novel datasets: (1) the Controlled Carbon Dioxide Release (CCR) dataset, which simulates gas leaks with systematically varied flow rates (10-100 SCCM), and (2) the Real Time Ankom (RTA) dataset, focusing on emissions from dairy cow rumen fluid in vitro experiments. Extensive evaluations demonstrate that CarboFormer achieves competitive performance with 84.88% mIoU on CCR and 92.98% mIoU on RTA, while maintaining computational efficiency with only 5.07M parameters and operating at 84.68 FPS. The model shows particular effectiveness in challenging low-flow scenarios and significantly outperforms other lightweight methods like SegFormer-B0 (83.36% mIoU on CCR) and SegNeXt (82.55% mIoU on CCR), making it suitable for real-time monitoring on resource-constrained platforms such as programmable drones. Our work advances both environmental sensing and precision livestock management by providing robust and efficient tools for CO$_2$ emission analysis.

Method: Systematic review of domain adaptation methods categorized into shallow and deep learning approaches, including supervised, semi-supervised, and unsupervised strategies. Special focus on adversarial learning techniques for complex scenarios. Also reviews public agricultural image datasets.

Result: Domain adaptation methods have significantly improved cross-domain performance in agricultural applications including crop health monitoring, pest detection, and fruit identification. The study provides a complete framework for DA research.

Conclusion: This review provides key insights and a comprehensive framework that can serve as a reference for future research and development of domain adaptation methods in agricultural vision tasks.

Abstract: With the wide application of computer vision in agriculture, image analysis has become the key to tasks such as crop health monitoring and pest detection. However, the significant domain shifts caused by environmental changes, different crop types, and diverse data acquisition methods seriously hinder the generalization ability of the model in cross-region, cross-season, and complex agricultural scenarios. This paper explores how domain adaptation (DA) techniques can address these challenges to improve cross-domain transferability in agricultural image analysis. DA is considered a promising solution in the case of limited labeled data, insufficient model adaptability, and dynamic changes in the field environment. This paper systematically reviews the latest advances in DA in agricultural images in recent years, focusing on application scenarios such as crop health monitoring, pest and disease detection, and fruit identification, in which DA methods have significantly improved cross-domain performance. We categorize DA methods into shallow learning and deep learning methods, including supervised, semi-supervised and unsupervised strategies, and pay special attention to the adversarial learning-based techniques that perform well in complex scenarios. In addition, this paper also reviews the main public datasets of agricultural images, and evaluates their advantages and limitations in DA research. Overall, this study provides a complete framework and some key insights that can be used as a reference for the research and development of domain adaptation methods in future agricultural vision tasks.

Method: Developed MCA-Bench with heterogeneous CAPTCHA types integrated into a single evaluation protocol, using fine-tuned vision-language model agents for each CAPTCHA category.

Result: Effectively mapped vulnerability spectrum of modern CAPTCHAs, provided first quantitative analysis of challenge complexity, interaction depth, and model solvability relationships.

Conclusion: Proposed three actionable design principles and identified key open challenges for systematic CAPTCHA hardening and fair benchmarking.

Abstract: As automated attack techniques rapidly advance, CAPTCHAs remain a critical defense mechanism against malicious bots. However, existing CAPTCHA schemes encompass a diverse range of modalities – from static distorted text and obfuscated images to interactive clicks, sliding puzzles, and logic-based questions – yet the community still lacks a unified, large-scale, multimodal benchmark to rigorously evaluate their security robustness. To address this gap, we introduce MCA-Bench, a comprehensive and reproducible benchmarking suite that integrates heterogeneous CAPTCHA types into a single evaluation protocol. Leveraging a shared vision-language model backbone, we fine-tune specialized cracking agents for each CAPTCHA category, enabling consistent, cross-modal assessments. Extensive experiments reveal that MCA-Bench effectively maps the vulnerability spectrum of modern CAPTCHA designs under varied attack settings, and crucially offers the first quantitative analysis of how challenge complexity, interaction depth, and model solvability interrelate. Based on these findings, we propose three actionable design principles and identify key open challenges, laying the groundwork for systematic CAPTCHA hardening, fair benchmarking, and broader community collaboration. Datasets and code are available online.

Method: Created benchmark dataset with features from three vision tasks compressed by four codecs, providing task-specific performance degradation as ground truth semantic distortion for evaluating metrics.

Result: Systematic evaluation shows MSE, cosine similarity, and CKA metrics are insufficient for capturing semantic degradation in compressed features, demonstrating the need for better CFQA metrics.

Conclusion: Establishes foundational CFQA benchmark and releases dataset/code to advance research on semantic fidelity assessment for compressed features in resource-constrained deployments.

Abstract: The widespread deployment of large models in resource-constrained environments has underscored the need for efficient transmission of intermediate feature representations. In this context, feature coding, which compresses features into compact bitstreams, becomes a critical component for scenarios involving feature transmission, storage, and reuse. However, this compression process inevitably introduces semantic degradation that is difficult to quantify with traditional metrics. To address this, we formalize the research problem of Compressed Feature Quality Assessment (CFQA), aiming to evaluate the semantic fidelity of compressed features. To advance CFQA research, we propose the first benchmark dataset, comprising 300 original features and 12000 compressed features derived from three vision tasks and four feature codecs. Task-specific performance degradation is provided as true semantic distortion for evaluating CFQA metrics. We systematically assess three widely used metrics – MSE, cosine similarity, and Centered Kernel Alignment (CKA) – in terms of their ability to capture semantic degradation. Our findings demonstrate the representativeness of the proposed dataset while underscoring the need for more sophisticated metrics capable of measuring semantic distortion in compressed features. This work advances the field by establishing a foundational benchmark and providing a critical resource for the community to explore CFQA. To foster further research, we release the dataset and all associated source code at https://github.com/chansongoal/Compressed-Feature-Quality-Assessment.

Method: Uses pre-trained LLM with few-shot learning to generate planning scripts, SS-parser to correct logic errors, and output verifier with ensemble learning and caption analysis to validate results.

Result: Outperforms state-of-the-art visual reasoning models on six visual benchmarks, demonstrating superior compositional text-visual reasoning capabilities.

Conclusion: VLAgent provides an effective framework for compositional vision-text reasoning through executable script generation, error correction, and output validation.

Abstract: The advancement in large language models (LLMs) and large vision models has fueled the rapid progress in multi-modal vision-text reasoning capabilities. However, existing vision-language models (VLMs) to date offer poor performance for compositional reasoning. This paper presents VLAgent, a vision-language agent system for vision-text compositional reasoning with three novel features. First, VLAgent leverages a pre-trained LLM with few-shot context learning to generate the planning script for each compositional reasoning task and provides a backend engine to generate and perform executable runtime, which maps the planning script into executable code using the VLAgent library for VLAgent executor. Second, VLAgent introduces the SS-parser, which identifies and corrects logic errors embedded in the LLM-generated planning script, to further enhance the quality of script-executable mapping. Third, VLAgent introduces the compositional reasoning output verifier, which validates and refines the output of complex compositional reasoning steps, by leveraging complementary reasoning techniques, e.g., ensemble learning and caption analysis. Extensive experiments are conducted on six visual benchmarks and compared to a dozen of the SoTA visual reasoning models. The results show that VLAgent outperforms existing representative approaches for compositional text-visual reasoning. Our code and datasets with outputs will be made available upon acceptance.

Method: DART uses learnable region scores with piecewise differentiable quantile operations to adaptively partition images into varying-sized patches, allocating denser tokens to information-rich areas.

Result: DART improves DeiT accuracy by 2.1% on ImageNet-1K with only ~1M additional parameters, achieves 45% FLOPs reduction while maintaining superior performance, and consistently enhances accuracy across DeiT, Vim, and VideoMamba with minimal computational overhead.

Conclusion: DART provides an efficient alternative to uniform token density methods, demonstrating that adaptive token allocation significantly improves vision transformer performance while reducing computational costs.

Abstract: Recently, non-convolutional models such as the Vision Transformer (ViT) and Vision Mamba (Vim) have achieved remarkable performance in computer vision tasks. However, their reliance on fixed-size patches often results in excessive encoding of background regions and omission of critical local details, especially when informative objects are sparsely distributed. To address this, we introduce a fully differentiable Dynamic Adaptive Region Tokenizer (DART), which adaptively partitions images into content-dependent patches of varying sizes. DART combines learnable region scores with piecewise differentiable quantile operations to allocate denser tokens to information-rich areas. Despite introducing only approximately 1 million (1M) additional parameters, DART improves accuracy by 2.1% on DeiT (ImageNet-1K). Unlike methods that uniformly increase token density to capture fine-grained details, DART offers a more efficient alternative, achieving 45% FLOPs reduction with superior performance. Extensive experiments on DeiT, Vim, and VideoMamba confirm that DART consistently enhances accuracy while incurring minimal or even reduced computational overhead. Code is available at https://github.com/HCPLab-SYSU/DART.

Method: Integrates pre-trained Grounded-SAM 2 for object proposals and DINOv2 for embeddings. Uses cyclic thresholding for stable matching, appearance scoring, and confidence-based scoring in an RGB-only pipeline.

Result: Achieves state-of-the-art mean AP scores on BOP 2023 challenge datasets, outperforming both RGB and RGB-D methods for unseen object segmentation.

Conclusion: NOCTIS demonstrates that effective novel object instance segmentation can be achieved without retraining by combining existing pre-trained models with intelligent matching mechanisms and scoring strategies.

Abstract: Instance segmentation of novel objects instances in RGB images, given some example images for each object, is a well known problem in computer vision. Designing a model general enough to be employed for all kinds of novel objects without (re-) training has proven to be a difficult task. To handle this, we present a new training-free framework, called: Novel Object Cyclic Threshold based Instance Segmentation (NOCTIS). NOCTIS integrates two pre-trained models: Grounded-SAM 2 for object proposals with precise bounding boxes and corresponding segmentation masks; and DINOv2 for robust class and patch embeddings, due to its zero-shot capabilities. Internally, the proposal-object matching is realized by determining an object matching score based on the similarity of the class embeddings and the average maximum similarity of the patch embeddings with a new cyclic thresholding (CT) mechanism that mitigates unstable matches caused by repetitive textures or visually similar patterns. Beyond CT, NOCTIS introduces: (i) an appearance score that is unaffected by object selection bias; (ii) the usage of the average confidence of the proposals bounding box and mask as a scoring component; and (iii) an RGB-only pipeline that performs even better than RGB-D ones. We empirically show that NOCTIS, without further training/fine tuning, attains state-of-the-art results regarding the mean AP score, w.r.t. the best RGB and RGB-D methods on the seven core datasets of the BOP 2023 challenge for the “Model-based 2D segmentation of unseen objects” task.

Method: Proposes Hierarchical Temporal Feature Transfer (HTFT) to model temporal and denoising processes separately with feature transfer between frames, plus a three-stage training strategy with model decoupling and auto-regressive inference simulation.

Result: Significantly surpasses existing methods on Nuscenes dataset and demonstrates ability to generate 600 frames of high-quality driving videos, far exceeding previous maximum lengths.

Conclusion: STAGE provides an effective solution for sustainable, high-fidelity long-horizon driving video generation with superior temporal consistency and reduced error accumulation.

Abstract: The generation of temporally consistent, high-fidelity driving videos over extended horizons presents a fundamental challenge in autonomous driving world modeling. Existing approaches often suffer from error accumulation and feature misalignment due to inadequate decoupling of spatio-temporal dynamics and limited cross-frame feature propagation mechanisms. To address these limitations, we present STAGE (Streaming Temporal Attention Generative Engine), a novel auto-regressive framework that pioneers hierarchical feature coordination and multi-phase optimization for sustainable video synthesis. To achieve high-quality long-horizon driving video generation, we introduce Hierarchical Temporal Feature Transfer (HTFT) and a novel multi-stage training strategy. HTFT enhances temporal consistency between video frames throughout the video generation process by modeling the temporal and denoising process separately and transferring denoising features between frames. The multi-stage training strategy is to divide the training into three stages, through model decoupling and auto-regressive inference process simulation, thereby accelerating model convergence and reducing error accumulation. Experiments on the Nuscenes dataset show that STAGE has significantly surpassed existing methods in the long-horizon driving video generation task. In addition, we also explored STAGE’s ability to generate unlimited-length driving videos. We generated 600 frames of high-quality driving videos on the Nuscenes dataset, which far exceeds the maximum length achievable by existing methods.

Method: A framework that leverages large vision models to analyze naturalistic driving videos captured through in-vehicle sensors. The method extracts “digital fingerprints” from real-world driving behavior across different roadway scenarios to identify cognitive status and predict disease progression.

Result: The approach can identify early warning signs of functional impairment by correlating driving patterns with clinical features of dementia, enabling the vehicle to serve as a “diagnostic tool” for cognitive status assessment.

Conclusion: This work enhances early detection capabilities and supports the development of proactive intervention strategies, contributing to scalable monitoring systems that can mitigate the societal and economic burden of cognitive decline in older adults.

Abstract: We introduce scenario-based cognitive status identification in older drivers from naturalistic driving videos, leveraging large vision models. In recent times, cognitive decline including Dementia and Mild Cognitive Impairment (MCI), is often underdiagnosed due to the time-consuming and costly nature of current diagnostic methods. By analyzing real-world driving behavior captured through in-vehicle sensors, this study aims to extract “digital fingerprints” that correlate with functional decline and clinical features of dementia. Moreover, modern large vision models can draw meaningful insights from everyday driving patterns across different roadway scenarios to early detect cognitive decline. We propose a framework that uses large vision models and naturalistic driving videos to analyze driver behavior, identify cognitive status and predict disease progression. We leverage the strong relationship between real-world driving behavior as an observation of the current cognitive status of the drivers where the vehicle can be utilized as a “diagnostic tool”. Our method identifies early warning signs of functional impairment, contributing to proactive intervention strategies. This work enhances early detection and supports the development of scalable, non-invasive monitoring systems to mitigate the growing societal and economic burden of cognitive decline in the aging population.

Method: Uses 9 pre-trained CNN architectures as ‘critics’ that are domain-adapted on a non-overlapping leaf disease dataset. Extracts feature embeddings, fuses them through a Feature Fusion Block, and classifies using Bi-LSTM layers in a few-shot learning framework.

Result: Achieved 89.06% (5-shot), 92.46% (10-shot), 94.07% (15-shot), and 98.09% (80-shot) accuracy on tomato leaf disease classification, requiring only 5.5% of the data compared to state-of-the-art while maintaining similar performance.

Conclusion: DExNet provides an effective solution for plant disease classification with limited data, outperforming existing methods across single-domain, mixed-domain, and cross-domain scenarios while significantly reducing data requirements.

Abstract: While deep learning-based architectures have been widely used for correctly detecting and classifying plant diseases, they require large-scale datasets to learn generalized features and achieve state-of-the-art performance. This poses a challenge for such models to obtain satisfactory performance in classifying leaf diseases with limited samples. This work proposes a few-shot learning framework, Domain-adapted Expert Network (DExNet), for plant disease classification that compensates for the lack of sufficient training data by combining observations of a number of expert critics. It starts with extracting the feature embeddings as ‘observations’ from nine ‘critics’ that are state-of-the-art pre-trained CNN-based architectures. These critics are ‘domain adapted’ using a publicly available leaf disease dataset having no overlapping classes with the specific downstream task of interest. The observations are then passed to the ‘Feature Fusion Block’ and finally to a classifier network consisting of Bi-LSTM layers. The proposed pipeline is evaluated on the 10 classes of tomato leaf images from the PlantVillage dataset, achieving promising accuracies of 89.06%, 92.46%, and 94.07%, respectively, for 5-shot, 10-shot, and 15-shot classification. Furthermore, an accuracy of 98.09+-0.7% has been achieved in 80-shot classification, which is only 1.2% less than state-of-the-art, allowing a 94.5% reduction in the training data requirement. The proposed pipeline also outperforms existing works on leaf disease classification with limited data in both laboratory and real-life conditions in single-domain, mixed-domain, and cross-domain scenarios.

Method: Framed as fine-grained recognition task, fused patient metadata with X-rays, used fine-grained pre-training weights instead of ImageNet, and applied transformer architecture for the first time with metadata integration in wrist pathology.

Result: Combination of fine-grained transformer approach, fine-grained pre-training, and metadata integration improved diagnostic accuracy by 2% on small custom dataset and over 10% on larger fracture dataset.

Conclusion: The multifaceted approach integrating metadata with medical images and using specialized fine-grained pre-training significantly enhances wrist pathology recognition accuracy, demonstrating the value of combining multiple data modalities in medical imaging analysis.

Abstract: Wrist pathologies are frequently observed, particularly among children who constitute the majority of fracture cases. Computer vision presents a promising avenue, contingent upon the availability of extensive datasets, a notable challenge in medical imaging. Therefore, reliance solely on one modality, such as images, proves inadequate, especially in an era of diverse and plentiful data types. This study addresses the problem using a multifaceted approach: framing it as a fine-grained recognition task, fusing patient metadata with X-rays, and leveraging weights from a separate fine-grained dataset rather than from a coarse-grained dataset like ImageNet. Unlike prior work, this is the first application of metadata integration for wrist pathology recognition. Our results show that combining fine-grained transformer approach, fine-grained pre-training, and metadata integration improves diagnostic accuracy by 2% on small custom curated dataset and over 10% on a larger fracture dataset.

Method: Captured and released a novel multiview dataset of a small lab-based tornado, then used 3D Gaussian splatting (3DGS) technique for reconstruction.

Result: Successfully demonstrated effective 3D reconstruction and visualization of the tornado structure using 3DGS.

Conclusion: The created dataset enables development of 3D reconstruction tools for tornadoes, and 3DGS proves effective for visualizing tornado structures from controlled lab data.

Abstract: Accurately reconstructing the 3D structure of tornadoes is critically important for understanding and preparing for this highly destructive weather phenomenon. While modern 3D scene reconstruction techniques, such as 3D Gaussian splatting (3DGS), could provide a valuable tool for reconstructing the 3D structure of tornados, at present we are critically lacking a controlled tornado dataset with which to develop and validate these tools. In this work we capture and release a novel multiview dataset of a small lab-based tornado. We demonstrate one can effectively reconstruct and visualize the 3D structure of this tornado using 3DGS.

Method: Uses Gestalt principles with orthogonal integration strategy to reduce redundancy. Implements Gestalt Feature Attention block with self/cross-attention mechanisms and Dual-path Multi-Granularity parallel interaction block for information exchange across granularities.

Result: Extensive experiments demonstrate superior performance compared to existing methods on various challenging tasks.

Conclusion: GPI-Net effectively handles the fusion of local and global features for high-quality correspondences in point cloud registration through Gestalt-guided parallel interaction.

Abstract: The accurate identification of high-quality correspondences is a prerequisite task in feature-based point cloud registration. However, it is extremely challenging to handle the fusion of local and global features due to feature redundancy and complex spatial relationships. Given that Gestalt principles provide key advantages in analyzing local and global relationships, we propose a novel Gestalt-guided Parallel Interaction Network via orthogonal geometric consistency (GPI-Net) in this paper. It utilizes Gestalt principles to facilitate complementary communication between local and global information. Specifically, we introduce an orthogonal integration strategy to optimally reduce redundant information and generate a more compact global structure for high-quality correspondences. To capture geometric features in correspondences, we leverage a Gestalt Feature Attention (GFA) block through a hybrid utilization of self-attention and cross-attention mechanisms. Furthermore, to facilitate the integration of local detail information into the global structure, we design an innovative Dual-path Multi-Granularity parallel interaction aggregation (DMG) block to promote information exchange across different granularities. Extensive experiments on various challenging tasks demonstrate the superior performance of our proposed GPI-Net in comparison to existing methods. The code will be released at https://github.com/gwk429/GPI-Net.

Method: Proposes VSRM framework with Spatial-to-Temporal and Temporal-to-Spatial Mamba blocks for feature extraction, Deformable Cross-Mamba Alignment for dynamic frame alignment, and Frequency Charbonnier-like loss for frequency domain optimization.

Result: Achieves state-of-the-art performance on diverse benchmarks, demonstrating superior video super-resolution quality with efficient computation.

Conclusion: VSRM establishes a solid foundation for future video super-resolution research by effectively addressing limitations of existing methods through Mamba architecture and novel alignment techniques.

Abstract: Video super-resolution remains a major challenge in low-level vision tasks. To date, CNN- and Transformer-based methods have delivered impressive results. However, CNNs are limited by local receptive fields, while Transformers struggle with quadratic complexity, posing challenges for processing long sequences in VSR. Recently, Mamba has drawn attention for its long-sequence modeling, linear complexity, and large receptive fields. In this work, we propose VSRM, a novel \textbf{V}ideo \textbf{S}uper-\textbf{R}esolution framework that leverages the power of \textbf{M}amba. VSRM introduces Spatial-to-Temporal Mamba and Temporal-to-Spatial Mamba blocks to extract long-range spatio-temporal features and enhance receptive fields efficiently. To better align adjacent frames, we propose Deformable Cross-Mamba Alignment module. This module utilizes a deformable cross-mamba mechanism to make the compensation stage more dynamic and flexible, preventing feature distortions. Finally, we minimize the frequency domain gaps between reconstructed and ground-truth frames by proposing a simple yet effective Frequency Charbonnier-like loss that better preserves high-frequency content and enhances visual quality. Through extensive experiments, VSRM achieves state-of-the-art results on diverse benchmarks, establishing itself as a solid foundation for future research.

Method: Introduces basis functions to each Gaussian that take sampling rate as input to model appearance variations. Parameters are jointly optimized with 3D Gaussians end-to-end. Also creates a new synthetic dataset with varying camera distances for comprehensive evaluation.

Result: Achieves state-of-the-art rendering quality while effectively eliminating aliasing artifacts. Extensive experiments on public datasets and new synthetic dataset demonstrate superior performance.

Conclusion: LOD-GS provides a sampling-rate-sensitive filtering framework that dynamically adapts to different viewing conditions, solving aliasing issues in 3D Gaussian Splatting while maintaining high rendering efficiency and quality.

Abstract: Despite the advancements in quality and efficiency achieved by 3D Gaussian Splatting (3DGS) in 3D scene rendering, aliasing artifacts remain a persistent challenge. Existing approaches primarily rely on low-pass filtering to mitigate aliasing. However, these methods are not sensitive to the sampling rate, often resulting in under-filtering and over-smoothing renderings. To address this limitation, we propose LOD-GS, a Level-of-Detail-sensitive filtering framework for Gaussian Splatting, which dynamically predicts the optimal filtering strength for each 3D Gaussian primitive. Specifically, we introduce a set of basis functions to each Gaussian, which take the sampling rate as input to model appearance variations, enabling sampling-rate-sensitive filtering. These basis function parameters are jointly optimized with the 3D Gaussian in an end-to-end manner. The sampling rate is influenced by both focal length and camera distance. However, existing methods and datasets rely solely on down-sampling to simulate focal length changes for anti-aliasing evaluation, overlooking the impact of camera distance. To enable a more comprehensive assessment, we introduce a new synthetic dataset featuring objects rendered at varying camera distances. Extensive experiments on both public datasets and our newly collected dataset demonstrate that our method achieves SOTA rendering quality while effectively eliminating aliasing. The code and dataset have been open-sourced.

Method: Proposes IC-Custom with In-context Multi-Modal Attention (ICMA) mechanism using learnable task-oriented register tokens and boundary-aware positional embeddings. Uses polyptych configuration of reference and target images with DiT’s attention for token-level interactions. Curates 12k high-quality dataset with real-world and synthetic data.

Result: Significantly outperforms community workflows, closed-source models, and state-of-the-art open-source approaches on ProductBench and DreamBench. Achieves ~73% higher human preference across identity consistency, harmonicity, and text alignment metrics.

Conclusion: IC-Custom provides a unified framework for diverse industrial applications including try-on, accessory placement, furniture arrangement, and creative IP customization, demonstrating superior performance with minimal parameter training.

Abstract: Image customization, a crucial technique for industrial media production, aims to generate content that is consistent with reference images. However, current approaches conventionally separate image customization into position-aware and position-free customization paradigms and lack a universal framework for diverse customization, limiting their applications across various scenarios. To overcome these limitations, we propose IC-Custom, a unified framework that seamlessly integrates position-aware and position-free image customization through in-context learning. IC-Custom concatenates reference images with target images to a polyptych, leveraging DiT’s multi-modal attention mechanism for fine-grained token-level interactions. We introduce the In-context Multi-Modal Attention (ICMA) mechanism with learnable task-oriented register tokens and boundary-aware positional embeddings to enable the model to correctly handle different task types and distinguish various inputs in polyptych configurations. To bridge the data gap, we carefully curated a high-quality dataset of 12k identity-consistent samples with 8k from real-world sources and 4k from high-quality synthetic data, avoiding the overly glossy and over-saturated synthetic appearance. IC-Custom supports various industrial applications, including try-on, accessory placement, furniture arrangement, and creative IP customization. Extensive evaluations on our proposed ProductBench and the publicly available DreamBench demonstrate that IC-Custom significantly outperforms community workflows, closed-source models, and state-of-the-art open-source approaches. IC-Custom achieves approximately 73% higher human preference across identity consistency, harmonicity, and text alignment metrics, while training only 0.4% of the original model parameters. Project page: https://liyaowei-stu.github.io/project/IC_Custom

Method: Proposes a formulation based on local planarity assumption, modeling constraints between surface normals and ray directions to accurately approximate depth-normal relations.

Result: Achieves state-of-the-art results on standard normal integration benchmark and is the first method to directly handle generic central camera models.

Conclusion: The approach provides more accurate normal integration by explicitly modeling discontinuities and supporting various camera models, advancing photometric shape reconstruction techniques.

Abstract: Recovering a 3D surface from its surface normal map, a problem known as normal integration, is a key component for photometric shape reconstruction techniques such as shape-from-shading and photometric stereo. The vast majority of existing approaches for normal integration handle only implicitly the presence of depth discontinuities and are limited to orthographic or ideal pinhole cameras. In this paper, we propose a novel formulation that allows modeling discontinuities explicitly and handling generic central cameras. Our key idea is based on a local planarity assumption, that we model through constraints between surface normals and ray directions. Compared to existing methods, our approach more accurately approximates the relation between depth and surface normals, achieves state-of-the-art results on the standard normal integration benchmark, and is the first to directly handle generic central camera models.

Method: Introduces BAV-Classroom-VQA dataset from real Vietnamese classroom recordings, with methodology for data collection and annotation. Benchmarks four open-source VQA models on this dataset.

Result: Initial experimental results show all four models achieve promising performance levels in answering behavior-related visual questions from classroom videos.

Conclusion: The VQA models demonstrate potential for future classroom analytics and intervention systems, enabling automated behavior monitoring and analysis.

Abstract: Classroom behavior monitoring is a critical aspect of educational research, with significant implications for student engagement and learning outcomes. Recent advancements in Visual Question Answering (VQA) models offer promising tools for automatically analyzing complex classroom interactions from video recordings. In this paper, we investigate the applicability of several state-of-the-art open-source VQA models, including LLaMA2, LLaMA3, QWEN3, and NVILA, in the context of classroom behavior analysis. To facilitate rigorous evaluation, we introduce our BAV-Classroom-VQA dataset derived from real-world classroom video recordings at the Banking Academy of Vietnam. We present the methodology for data collection, annotation, and benchmark the performance of the selected VQA models on this dataset. Our initial experimental results demonstrate that all four models achieve promising performance levels in answering behavior-related visual questions, showcasing their potential in future classroom analytics and intervention systems.

Method: Hierarchical motion estimation combining lightweight linear prediction with selective non-linear refinement, plus optimized memory bank with long-term and short-term memory frame distinction.

Result: Achieved 9.6% and 7.2% relative improvements in AUC on LaSOT and LaSOText with large model, with even larger gains on smaller models.

Conclusion: Trainless, low-overhead improvements effectively boost long-term tracking performance across different model scales.

Abstract: This paper presents enhancements to the SAM2 framework for video object tracking task, addressing challenges such as occlusions, background clutter, and target reappearance. We introduce a hierarchical motion estimation strategy, combining lightweight linear prediction with selective non-linear refinement to improve tracking accuracy without requiring additional training. In addition, we optimize the memory bank by distinguishing long-term and short-term memory frames, enabling more reliable tracking under long-term occlusions and appearance changes. Experimental results show consistent improvements across different model scales. Our method achieves state-of-the-art performance on LaSOT and LaSOText with the large model, achieving 9.6% and 7.2% relative improvements in AUC over the original SAM2, and demonstrates even larger relative gains on smaller models, highlighting the effectiveness of our trainless, low-overhead improvements for boosting long-term tracking performance. The code is available at https://github.com/LouisFinner/HiM2SAM.

Method: Uses plain Video Vision Transformers (Video ViTs) with different pre-training strategies including self-supervised Masked Video Modeling (MVM), weakly-supervised, fully-supervised pre-training, and Domain-Adaptive Pre-Training (DAPT) on unlabeled driving videos.

Result: Simple encoder-only models match or surpass specialized state-of-the-art TAD methods while being significantly more efficient. Self-supervised MVM provides the strongest signal, and DAPT further improves performance without requiring anomalous examples.

Conclusion: Effective, efficient, and scalable traffic anomaly detection models can be built with minimal architectural complexity through proper pre-training strategies, particularly self-supervised approaches and domain adaptation.

Abstract: Recent methods for ego-centric Traffic Anomaly Detection (TAD) often rely on complex multi-stage or multi-representation fusion architectures, yet it remains unclear whether such complexity is necessary. Recent findings in visual perception suggest that foundation models, enabled by advanced pre-training, allow simple yet flexible architectures to outperform specialized designs. Therefore, in this work, we investigate an architecturally simple encoder-only approach using plain Video Vision Transformers (Video ViTs) and study how pre-training enables strong TAD performance. We find that: (i) advanced pre-training enables simple encoder-only models to match or even surpass the performance of specialized state-of-the-art TAD methods, while also being significantly more efficient; (ii) although weakly- and fully-supervised pre-training are advantageous on standard benchmarks, we find them less effective for TAD. Instead, self-supervised Masked Video Modeling (MVM) provides the strongest signal; and (iii) Domain-Adaptive Pre-Training (DAPT) on unlabeled driving videos further improves downstream performance, without requiring anomalous examples. Our findings highlight the importance of pre-training and show that effective, efficient, and scalable TAD models can be built with minimal architectural complexity. We release our code, domain-adapted encoders, and fine-tuned models to support future work: https://github.com/tue-mps/simple-tad.

Method: Timestep-aware probabilistic function that dynamically selects between BP-low for target-specific adaptation and ZO-high for high-resolution detail refinement based on diffusion timesteps.

Result: Achieves competitive performance while significantly reducing memory consumption, enabling scalable high-quality on-device personalization without increasing inference latency.

Conclusion: The framework successfully combines both optimization strategies to leverage their complementary strengths - BP-low for effective personalization and ZO-high for structural consistency, achieving memory-efficient fine-tuning.

Abstract: Memory-efficient personalization is critical for adapting text-to-image diffusion models while preserving user privacy and operating within the limited computational resources of edge devices. To this end, we propose a selective optimization framework that adaptively chooses between backpropagation on low-resolution images (BP-low) and zeroth-order optimization on high-resolution images (ZO-high), guided by the characteristics of the diffusion process. As observed in our experiments, BP-low efficiently adapts the model to target-specific features, but suffers from structural distortions due to resolution mismatch. Conversely, ZO-high refines high-resolution details with minimal memory overhead but faces slow convergence when applied without prior adaptation. By complementing both methods, our framework leverages BP-low for effective personalization while using ZO-high to maintain structural consistency, achieving memory-efficient and high-quality fine-tuning. To maximize the efficacy of both BP-low and ZO-high, we introduce a timestep-aware probabilistic function that dynamically selects the appropriate optimization strategy based on diffusion timesteps. This function mitigates the overfitting from BP-low at high timesteps, where structural information is critical, while ensuring ZO-high is applied more effectively as training progresses. Experimental results demonstrate that our method achieves competitive performance while significantly reducing memory consumption, enabling scalable, high-quality on-device personalization without increasing inference latency.

Method: Fast weighted fusion of multi-view 2D features with pre-trained Gaussians, training per-Gaussian autoencoders directly on lifted features, and adaptive sparsification with pruning/merging of redundant Gaussians.

Result: Achieves competitive 3D feature field using only 5% of Gaussians compared to Feature-3DGS while preserving geometric details.

Conclusion: CF3 provides an efficient top-down approach for constructing compact 3D Gaussian feature fields with significantly reduced computational requirements.

Abstract: 3D Gaussian Splatting (3DGS) has begun incorporating rich information from 2D foundation models. However, most approaches rely on a bottom-up optimization process that treats raw 2D features as ground truth, incurring increased computational costs. We propose a top-down pipeline for constructing compact and fast 3D Gaussian feature fields, namely, CF3. We first perform a fast weighted fusion of multi-view 2D features with pre-trained Gaussians. This approach enables training a per-Gaussian autoencoder directly on the lifted features, instead of training autoencoders in the 2D domain. As a result, the autoencoder better aligns with the feature distribution. More importantly, we introduce an adaptive sparsification method that optimizes the Gaussian attributes of the feature field while pruning and merging the redundant Gaussians, constructing an efficient representation with preserved geometric details. Our approach achieves a competitive 3D feature field using as little as 5% of the Gaussians compared to Feature-3DGS.

Method: Proposed a multimodal semantic reasoning framework (VIM-Sense) combining vision-language models with OCR-based textual analysis to detect attacks categorized into character, phrase, and pattern manipulation formats.

Result: Achieved 88.94% attack detection accuracy on the AR-VIM dataset, outperforming vision-only and text-only baselines, with average detection latency of 7.07-7.17 seconds.

Conclusion: The proposed VIM-Sense framework effectively detects visual information manipulation attacks in AR through multimodal semantic reasoning, demonstrating practical applicability in real-world AR scenarios.

Abstract: The virtual content in augmented reality (AR) can introduce misleading or harmful information, leading to semantic misunderstandings or user errors. In this work, we focus on visual information manipulation (VIM) attacks in AR, where virtual content changes the meaning of real-world scenes in subtle but impactful ways. We introduce a taxonomy that categorizes these attacks into three formats: character, phrase, and pattern manipulation, and three purposes: information replacement, information obfuscation, and extra wrong information. Based on the taxonomy, we construct a dataset, AR-VIM, which consists of 452 raw-AR video pairs spanning 202 different scenes, each simulating a real-world AR scenario. To detect the attacks in the dataset, we propose a multimodal semantic reasoning framework, VIM-Sense. It combines the language and visual understanding capabilities of vision-language models (VLMs) with optical character recognition (OCR)-based textual analysis. VIM-Sense achieves an attack detection accuracy of 88.94% on AR-VIM, consistently outperforming vision-only and text-only baselines. The system achieves an average attack detection latency of 7.07 seconds in a simulated video processing framework and 7.17 seconds in a real-world evaluation conducted on a mobile Android AR application.

Method: Proposes EDTR that uses pixel-error-based pre-restored images with mild noise as starting points for diffusion, and employs limited denoising steps to avoid generating redundant details that dilute task-critical information.

Result: The method effectively utilizes diffusion prior for TDIR, significantly enhancing both task performance and visual quality across diverse tasks with multiple complex degradations.

Conclusion: EDTR successfully bridges the gap between diffusion-based generation and task-driven restoration by strategically controlling the diffusion process to preserve and enhance task-relevant details in severely degraded images.

Abstract: Task-driven image restoration (TDIR) has recently emerged to address performance drops in high-level vision tasks caused by low-quality (LQ) inputs. Previous TDIR methods struggle to handle practical scenarios in which images are degraded by multiple complex factors, leaving minimal clues for restoration. This motivates us to leverage the diffusion prior, one of the most powerful natural image priors. However, while the diffusion prior can help generate visually plausible results, using it to restore task-relevant details remains challenging, even when combined with recent TDIR methods. To address this, we propose EDTR, which effectively harnesses the power of diffusion prior to restore task-relevant details. Specifically, we propose directly leveraging useful clues from LQ images in the diffusion process by generating from pixel-error-based pre-restored LQ images with mild noise added. Moreover, we employ a small number of denoising steps to prevent the generation of redundant details that dilute crucial task-related information. We demonstrate that our method effectively utilizes diffusion prior for TDIR, significantly enhancing task performance and visual quality across diverse tasks with multiple complex degradations.

Method: Proposes fine-grained temporally-aware textual descriptions of hand motion, contrastive alignment module, distillation of features from HaMeR, and contrastive loss between video representations and target language embeddings.

Result: Achieves state-of-the-art performance on Phoenix14T and CSL-Daily benchmarks, demonstrating effective modality gap reduction and improved sign distinction.

Conclusion: BeyondGloss framework effectively addresses SLT challenges through VideoLLM enhancement with hand-centric temporal modeling and contrastive alignment, providing a robust gloss-free solution.

Abstract: Sign Language Translation (SLT) is a challenging task that requires bridging the modality gap between visual and linguistic information while capturing subtle variations in hand shapes and movements. To address these challenges, we introduce \textbf{BeyondGloss}, a novel gloss-free SLT framework that leverages the spatio-temporal reasoning capabilities of Video Large Language Models (VideoLLMs). Since existing VideoLLMs struggle to model long videos in detail, we propose a novel approach to generate fine-grained, temporally-aware textual descriptions of hand motion. A contrastive alignment module aligns these descriptions with video features during pre-training, encouraging the model to focus on hand-centric temporal dynamics and distinguish signs more effectively. To further enrich hand-specific representations, we distill fine-grained features from HaMeR. Additionally, we apply a contrastive loss between sign video representations and target language embeddings to reduce the modality gap in pre-training. \textbf{BeyondGloss} achieves state-of-the-art performance on the Phoenix14T and CSL-Daily benchmarks, demonstrating the effectiveness of the proposed framework. We will release the code upon acceptance of the paper.

Method: Proposes parameter-efficient Convolutional Low-Rank Adaptation (Conv-LoRA) adapter to inject local inductive biases, and Dynamic Fusion Gateway (DFG) that uses visual context to adaptively modulate text prompts for bidirectional fusion.

Result: Extensive experiments on diverse industrial and medical benchmarks demonstrate superior accuracy and robustness compared to existing methods.

Conclusion: The synergistic architectural co-design framework is critical for robustly adapting foundation models to dense perception tasks like zero-shot anomaly detection.

Abstract: Pre-trained Vision-Language Models (VLMs) struggle with Zero-Shot Anomaly Detection (ZSAD) due to a critical adaptation gap: they lack the local inductive biases required for dense prediction and employ inflexible feature fusion paradigms. We address these limitations through an Architectural Co-Design framework that jointly refines feature representation and cross-modal fusion. Our method proposes a parameter-efficient Convolutional Low-Rank Adaptation (Conv-LoRA) adapter to inject local inductive biases for fine-grained representation, and introduces a Dynamic Fusion Gateway (DFG) that leverages visual context to adaptively modulate text prompts, enabling a powerful bidirectional fusion. Extensive experiments on diverse industrial and medical benchmarks demonstrate superior accuracy and robustness, validating that this synergistic co-design is critical for robustly adapting foundation models to dense perception tasks.

Method: TNet uses a Terrace Convolutional Decoder that progressively fuses low-resolution (global context) features into higher-resolution (local details) features across decoding stages using convolution and addition operations only.

Result: TNet-R (with ResNet-18 encoder) achieves mIoU of 85.35% on ISPRS Vaihingen, 87.05% on ISPRS Potsdam, and 52.19% on LoveDA datasets while maintaining computational efficiency.

Conclusion: TNet demonstrates that simple convolution and addition operations can effectively integrate multi-resolution global-local features for competitive remote sensing segmentation performance without complex modules like Transformers or Mamba.

Abstract: In remote sensing, most segmentation networks adopt the UNet architecture, often incorporating modules such as Transformers or Mamba to enhance global-local feature interactions within decoder stages. However, these enhancements typically focus on intra-scale relationships and neglect the global contextual dependencies across multiple resolutions. To address this limitation, we introduce the Terrace Convolutional Decoder Network (TNet), a simple yet effective architecture that leverages only convolution and addition operations to progressively integrate low-resolution features (rich in global context) into higher-resolution features (rich in local details) across decoding stages. This progressive fusion enables the model to learn spatially-aware convolutional kernels that naturally blend global and local information in a stage-wise manner. We implement TNet with a ResNet-18 encoder (TNet-R) and evaluate it on three benchmark datasets. TNet-R achieves competitive performance with a mean Intersection-over-Union (mIoU) of 85.35% on ISPRS Vaihingen, 87.05% on ISPRS Potsdam, and 52.19% on LoveDA, while maintaining high computational efficiency. Code is publicly available.

Method: Uses a top-down approach to decompose, summarize and reformulate papers, followed by bottom-up video generation. Introduces Progressive Chain of Thought (P-CoT) for granular iterative planning and defines key scenes to align cross-modal representations.

Result: Successfully generates high-quality video abstracts across five research fields, demonstrating expertise beyond current video generation models.

Conclusion: Preacher addresses the limitations of existing video generation models and provides an effective solution for converting research papers into accessible, well-organized video abstracts.

Abstract: The paper-to-video task converts a research paper into a structured video abstract, distilling key concepts, methods, and conclusions into an accessible, well-organized format. While state-of-the-art video generation models demonstrate potential, they are constrained by limited context windows, rigid video duration constraints, limited stylistic diversity, and an inability to represent domain-specific knowledge. To address these limitations, we introduce Preacher, the first paper-to-video agentic system. Preacher employs a topdown approach to decompose, summarize, and reformulate the paper, followed by bottom-up video generation, synthesizing diverse video segments into a coherent abstract. To align cross-modal representations, we define key scenes and introduce a Progressive Chain of Thought (P-CoT) for granular, iterative planning. Preacher successfully generates high-quality video abstracts across five research fields, demonstrating expertise beyond current video generation models. Code will be released at: https://github.com/GenVerse/Paper2Video

Method: Developed HistoPLUS model trained on a novel curated pan-cancer dataset of 108,722 nuclei covering 13 cell types, using a more efficient architecture with fewer parameters.

Result: Outperforms state-of-the-art models by 5.2% in detection quality and 23.7% in overall F1 classification score, while using 5x fewer parameters. Enables study of 7 understudied cell types and shows robust transfer to unseen oncology indications.

Conclusion: HistoPLUS significantly advances cell analysis capabilities for tumor microenvironment research, particularly for rare cell types, and demonstrates strong cross-domain generalization with improved efficiency.

Abstract: Cell detection, segmentation and classification are essential for analyzing tumor microenvironments (TME) on hematoxylin and eosin (H&E) slides. Existing methods suffer from poor performance on understudied cell types (rare or not present in public datasets) and limited cross-domain generalization. To address these shortcomings, we introduce HistoPLUS, a state-of-the-art model for cell analysis, trained on a novel curated pan-cancer dataset of 108,722 nuclei covering 13 cell types. In external validation across 4 independent cohorts, HistoPLUS outperforms current state-of-the-art models in detection quality by 5.2% and overall F1 classification score by 23.7%, while using 5x fewer parameters. Notably, HistoPLUS unlocks the study of 7 understudied cell types and brings significant improvements on 8 of 13 cell types. Moreover, we show that HistoPLUS robustly transfers to two oncology indications unseen during training. To support broader TME biomarker research, we release the model weights and inference code at https://github.com/owkin/histoplus/.

Method: Proposes SocialTrack framework with four key components: 1) Multi-scale feature enhancement mechanism for small-target detection, 2) Velocity Adaptive Cubature Kalman Filter (VACKF) for trajectory prediction, 3) Group Motion Compensation Strategy (GMCS) for social group motion modeling, and 4) Spatio-Temporal Memory Prediction (STMP) for leveraging historical trajectory information.

Result: Extensive experiments on UAVDT and MOT17 datasets show SocialTrack outperforms state-of-the-art methods, with significant improvements in MOTA and IDF1 metrics, demonstrating superior robustness and adaptability in complex dynamic environments.

Conclusion: SocialTrack provides an effective solution for UAV-based multi-object tracking in complex urban traffic scenarios, offering enhanced accuracy, robustness against identity switching, and modular compatibility with existing trackers for performance improvement.

Abstract: As a key research direction in the field of multi-object tracking (MOT), UAV-based multi-object tracking has significant application value in the analysis and understanding of urban intelligent transportation systems. However, in complex UAV perspectives, challenges such as small target scale variations, occlusions, nonlinear crossing motions, and motion blur severely hinder the stability of multi-object tracking. To address these challenges, this paper proposes a novel multi-object tracking framework, SocialTrack, aimed at enhancing the tracking accuracy and robustness of small targets in complex urban traffic environments. The specialized small-target detector enhances the detection performance by employing a multi-scale feature enhancement mechanism. The Velocity Adaptive Cubature Kalman Filter (VACKF) improves the accuracy of trajectory prediction by incorporating a velocity dynamic modeling mechanism. The Group Motion Compensation Strategy (GMCS) models social group motion priors to provide stable state update references for low-quality tracks, significantly improving the target association accuracy in complex dynamic environments. Furthermore, the Spatio-Temporal Memory Prediction (STMP) leverages historical trajectory information to predict the future state of low-quality tracks, effectively mitigating identity switching issues. Extensive experiments on the UAVDT and MOT17 datasets demonstrate that SocialTrack outperforms existing state-of-the-art (SOTA) methods across several key metrics. Significant improvements in MOTA and IDF1, among other core performance indicators, highlight its superior robustness and adaptability. Additionally, SocialTrack is highly modular and compatible, allowing for seamless integration with existing trackers to further enhance performance.

Method: Proposes FLAIR with two innovations: RC-GAUSS activation for explicit frequency selection and spatial localization under TFUP constraints, and Wavelet-Energy-Guided Encoding (WEGE) using DWT to guide frequency information.

Result: Consistently outperforms existing INRs in 2D image representation/restoration and 3D reconstruction tasks.

Conclusion: FLAIR successfully addresses spectral bias and improves implicit neural representations through frequency-aware activation and wavelet-guided encoding.

Abstract: Implicit Neural Representations (INRs) leverage neural networks to map coordinates to corresponding signals, enabling continuous and compact representations. This paradigm has driven significant advances in various vision tasks. However, existing INRs lack frequency selectivity, spatial localization, and sparse representations, leading to an over-reliance on redundant signal components. Consequently, they exhibit spectral bias, tending to learn low-frequency components early while struggling to capture fine high-frequency details. To address these issues, we propose FLAIR (Frequency- and Locality-Aware Implicit Neural Representations), which incorporates two key innovations. The first is RC-GAUSS, a novel activation designed for explicit frequency selection and spatial localization under the constraints of the time-frequency uncertainty principle (TFUP). The second is Wavelet-Energy-Guided Encoding (WEGE), which leverages the discrete wavelet transform (DWT) to compute energy scores and explicitly guide frequency information to the network. Our method consistently outperforms existing INRs in 2D image representation and restoration, as well as 3D reconstruction.

Method: Multi-agent collaboration framework based on Multimodal Large Language Models (MLLMs) with feedback and rollback mechanism for iterative self-correction of multimodal content.

Result: Comprehensive experiments demonstrate significant advantages over baselines, showing effectiveness for automated Vlog generation.

Conclusion: The framework shows great potential for generating automated Vlogs with high personalization and multimodal collaboration capabilities.

Abstract: With the growing demand for short videos and personalized content, automated Video Log (Vlog) generation has become a key direction in multimodal content creation. Existing methods mostly rely on predefined scripts, lacking dynamism and personal expression. Therefore, there is an urgent need for an automated Vlog generation approach that enables effective multimodal collaboration and high personalization. To this end, we propose PersonaVlog, an automated multimodal stylized Vlog generation framework that can produce personalized Vlogs featuring videos, background music, and inner monologue speech based on a given theme and reference image. Specifically, we propose a multi-agent collaboration framework based on Multimodal Large Language Models (MLLMs). This framework efficiently generates high-quality prompts for multimodal content creation based on user input, thereby improving the efficiency and creativity of the process. In addition, we incorporate a feedback and rollback mechanism that leverages MLLMs to evaluate and provide feedback on generated results, thereby enabling iterative self-correction of multimodal content. We also propose ThemeVlogEval, a theme-based automated benchmarking framework that provides standardized metrics and datasets for fair evaluation. Comprehensive experiments demonstrate the significant advantages and potential of our framework over several baselines, highlighting its effectiveness and great potential for generating automated Vlogs.

Method: Temporal Prompt Alignment (TPA) extracts frame features with an image encoder, aggregates them with a temporal extractor, aligns video representations with class-specific text prompts using margin-hinge contrastive loss, and incorporates CVAESM module for uncertainty quantification and style modulation.

Result: TPA achieves 85.40% macro F1 score for CHD diagnosis, reduces expected calibration error by 5.38% and adaptive ECE by 6.8%, and boosts macro F1 by 4.73% on EchoNet-Dynamic’s three-class task.

Conclusion: TPA effectively addresses limitations of current methods by integrating temporal modeling, prompt learning, and uncertainty quantification, demonstrating superior performance and improved calibration for clinical ultrasound video analysis.

Abstract: Congenital heart defect (CHD) detection in ultrasound videos is hindered by image noise and probe positioning variability. While automated methods can reduce operator dependence, current machine learning approaches often neglect temporal information, limit themselves to binary classification, and do not account for prediction calibration. We propose Temporal Prompt Alignment (TPA), a method leveraging foundation image-text model and prompt-aware contrastive learning to classify fetal CHD on cardiac ultrasound videos. TPA extracts features from each frame of video subclips using an image encoder, aggregates them with a trainable temporal extractor to capture heart motion, and aligns the video representation with class-specific text prompts via a margin-hinge contrastive loss. To enhance calibration for clinical reliability, we introduce a Conditional Variational Autoencoder Style Modulation (CVAESM) module, which learns a latent style vector to modulate embeddings and quantifies classification uncertainty. Evaluated on a private dataset for CHD detection and on a large public dataset, EchoNet-Dynamic, for systolic dysfunction, TPA achieves state-of-the-art macro F1 scores of 85.40% for CHD diagnosis, while also reducing expected calibration error by 5.38% and adaptive ECE by 6.8%. On EchoNet-Dynamic’s three-class task, it boosts macro F1 by 4.73% (from 53.89% to 58.62%). Temporal Prompt Alignment (TPA) is a framework for fetal congenital heart defect (CHD) classification in ultrasound videos that integrates temporal modeling, prompt-aware contrastive learning, and uncertainty quantification.

Method: The paper adopts a causal chain modeling perspective to analyze RPM tasks, designs a baseline model (DIO), and progressively proposes three improvement methods to address limitations in mutual information maximization and causal relationship capture.

Result: Experiments revealed that the baseline model’s optimization objective (maximizing variational lower bound of mutual information) failed to enable genuine acquisition of human reasoning logic due to tightness issues and statistical nature of mutual information.

Conclusion: The paper proposes progressive improvements to overcome the limitations of mutual information-based approaches in capturing causal relationships and enabling true abstract reasoning capabilities in deep learning models.

Abstract: Despite the outstanding performance of current deep learning models across various domains, their fundamental bottleneck in abstract reasoning remains unresolved. To address this challenge, the academic community has introduced Raven’s Progressive Matrices (RPM) problems as an authoritative benchmark for evaluating the abstract reasoning capabilities of deep learning algorithms, with a focus on core intelligence dimensions such as abstract reasoning, pattern recognition, and complex problem-solving. Therefore, this paper centers on solving RPM problems, aiming to contribute to enhancing the abstract reasoning abilities of machine intelligence. Firstly, this paper adopts a ``causal chain modeling’’ perspective to systematically analyze the complete causal chain in RPM tasks: image $\rightarrow$ abstract attributes $\rightarrow$ progressive attribute patterns $\rightarrow$ pattern consistency $\rightarrow$ correct answer. Based on this analysis, the network architecture of the baseline model DIO is designed. However, experiments reveal that the optimization objective formulated for DIO, namely maximizing the variational lower bound of mutual information between the context and the correct option, fails to enable the model to genuinely acquire the predefined human reasoning logic. This is attributed to two main reasons: the tightness of the lower bound significantly impacts the effectiveness of mutual information maximization, and mutual information, as a statistical measure, does not capture the causal relationship between subjects and objects. To overcome these limitations, this paper progressively proposes three improvement methods:

Method: MATR uses transformer architecture with dual-stage sequence alignment to condition target video representations on query features, plus self-supervised pre-training by localizing random clips within videos.

Result: Achieves 13.1% R@1 and 8.1% mIoU improvement on ActivityNet-VRL, and 14.7% R@1 and 14.4% mIoU gain on SportsMoments dataset compared to state-of-the-art methods.

Conclusion: MATR effectively addresses the challenges of video-to-video moment retrieval through its transformer-based architecture and self-supervised pre-training, demonstrating substantial performance gains across multiple datasets.

Abstract: Video-to-video moment retrieval (Vid2VidMR) is the task of localizing unseen events or moments in a target video using a query video. This task poses several challenges, such as the need for semantic frame-level alignment and modeling complex dependencies between query and target videos. To tackle this challenging problem, we introduce MATR (Moment Alignment TRansformer), a transformer-based model designed to capture semantic context as well as the temporal details necessary for precise moment localization. MATR conditions target video representations on query video features using dual-stage sequence alignment that encodes the required correlations and dependencies. These representations are then used to guide foreground/background classification and boundary prediction heads, enabling the model to accurately identify moments in the target video that semantically match with the query video. Additionally, to provide a strong task-specific initialization for MATR, we propose a self-supervised pre-training technique that involves training the model to localize random clips within videos. Extensive experiments demonstrate that MATR achieves notable performance improvements of 13.1% in R@1 and 8.1% in mIoU on an absolute scale compared to state-of-the-art methods on the popular ActivityNet-VRL dataset. Additionally, on our newly proposed dataset, SportsMoments, MATR shows a 14.7% gain in R@1 and a 14.4% gain in mIoU on an absolute scale over strong baselines.

Method: Introduces IoU-Augmented Maximum Likelihood (IAML) with novel IoU-based coordinate sampling pipeline for data augmentation, fine-tuning MLLMs to mitigate exposure bias in traditional training.

Result: Extensive experiments show superior performance of IAML approach over traditional training paradigms for GUI element coordinate generation.

Conclusion: IAML training paradigm effectively addresses MLLMs’ coordinate prediction challenges and improves GUI understanding capabilities through strategic data augmentation and bias mitigation.

Abstract: Multimodal large language models (MLLMs) have emerged as pivotal tools in enhancing human-computer interaction. In this paper we focus on the application of MLLMs in the field of graphical user interface (GUI) elements structuring, where they assist in processing user instructions based on screen contents. Despite the promise of MLLMs, their performance in precisely generating UI element coordinates, a critical aspect of GUI understanding, is hindered by the nature of next-token prediction training. This challenge arises from the semantic void surrounding numerical UI coordinates in language representation spaces, necessitating a substantial and diverse dataset to bolster visual module capabilities. To address these limitations, we introduce an IoU-Augmented Maximum Likelihood (IAML) training paradigm. Specifically, our approach involves a novel pipeline for IoU-based coordinate sampling to augment the training data, which considers the proximity to ground truth coordinates. This data augmentation strategy is then employed to fine-tune MLLMs under the IAML paradigm, which is designed to mitigate the exposure bias problem inherent in traditional maximum likelihood estimation. Through extensive experiments, we demonstrate the superior performance of our IAML training approach over traditional training paradigms.

Method: MSNav integrates three modules: Memory Module for dynamic map memory with selective node pruning, Spatial Module for spatial reasoning and object relationship inference, and Decision Module for LLM-based path planning. Also introduces I-O-S dataset and fine-tunes Qwen3-4B into Qwen-Spatial model.

Result: Achieves state-of-the-art performance on R2R and REVERIE datasets with significant improvements in Success Rate (SR) and Success weighted by Path Length (SPL). Qwen-Spatial outperforms commercial LLMs in object list extraction with higher F1 and NDCG scores.

Conclusion: MSNav’s synergistic architecture transforms fragile inference into robust, integrated intelligence for vision-and-language navigation, effectively addressing critical vulnerabilities of single-LLM approaches.

Abstract: Vision-and-Language Navigation (VLN) requires an agent to interpret natural language instructions and navigate complex environments. Current approaches often adopt a “black-box” paradigm, where a single Large Language Model (LLM) makes end-to-end decisions. However, it is plagued by critical vulnerabilities, including poor spatial reasoning, weak cross-modal grounding, and memory overload in long-horizon tasks. To systematically address these issues, we propose Memory Spatial Navigation(MSNav), a framework that fuses three modules into a synergistic architecture, which transforms fragile inference into a robust, integrated intelligence. MSNav integrates three modules: Memory Module, a dynamic map memory module that tackles memory overload through selective node pruning, enhancing long-range exploration; Spatial Module, a module for spatial reasoning and object relationship inference that improves endpoint recognition; and Decision Module, a module using LLM-based path planning to execute robust actions. Powering Spatial Module, we also introduce an Instruction-Object-Space (I-O-S) dataset and fine-tune the Qwen3-4B model into Qwen-Spatial (Qwen-Sp), which outperforms leading commercial LLMs in object list extraction, achieving higher F1 and NDCG scores on the I-O-S test set. Extensive experiments on the Room-to-Room (R2R) and REVERIE datasets demonstrate MSNav’s state-of-the-art performance with significant improvements in Success Rate (SR) and Success weighted by Path Length (SPL).

Method: Uses Large Language Models to derive category clusters encoding intrinsic inter-class relationships, and introduces class-aware patch-level contrastive loss to enforce intra-class consistency and inter-class separation in a hierarchical design.

Result: Experiments on PASCAL VOC 2012 and MS COCO 2014 show that CPC surpasses existing state-of-the-art methods in Weakly Supervised Semantic Segmentation.

Conclusion: CPC effectively addresses the limitations of previous WSSS methods by leveraging hierarchical semantic priors from LLMs and contrastive learning, achieving superior performance while maintaining cost-effectiveness of image-level labels.

Abstract: Weakly Supervised Semantic Segmentation (WSSS) with image-level labels has gained attention for its cost-effectiveness. Most existing methods emphasize inter-class separation, often neglecting the shared semantics among related categories and lacking fine-grained discrimination. To address this, we propose Contrastive Prompt Clustering (CPC), a novel WSSS framework. CPC exploits Large Language Models (LLMs) to derive category clusters that encode intrinsic inter-class relationships, and further introduces a class-aware patch-level contrastive loss to enforce intra-class consistency and inter-class separation. This hierarchical design leverages clusters as coarse-grained semantic priors while preserving fine-grained boundaries, thereby reducing confusion among visually similar categories. Experiments on PASCAL VOC 2012 and MS COCO 2014 demonstrate that CPC surpasses existing state-of-the-art methods in WSSS.

Method: Joint optimization of multimodal retriever with LVLM for medical diagnosis, using general-purpose backbones with lightweight fine-tuning only.

Result: Competitive performance with medically-pretrained models across clinical multi-label classification and visual question answering tasks. Significant improvement over standard RAG for challenging cases where different retrieved images lead to different predictions.

Conclusion: While correct diagnosis is frequently achievable with top retrieved images, there remains a large performance gap from oracle performance, and current rerankers using frontier LVLMs don’t close this gap, indicating substantial room for improvement.

Abstract: Clinical decision-making often involves interpreting images (e.g., radiology) for making diagnoses. Retrieving relevant visual information from medical literature and hospital records could enhance diagnostic accuracy. In this paper, we develop a model in which a multimodal retriever is jointly optimized with an LVLM for medical diagnosis, unlike standard RAG where LVLM error signal is not propagated down to the retriever. We show that using only general-purpose backbones, with only lightweight fine-tuning, our model is able to achieve competitive results with medically-pretrained models across clinical multi-label classification and visual question answering tasks. In a novel analysis, we additionally find that in many cases different top retrieved images each lead to different predictions for a given target, and that these cases are empirically challenging for all models, even for non-retrieval models. Our joint retrieval optimization significantly improves these challenging cases over standard RAG. However, oracle analysis reveals that while the correct diagnosis is frequently achievable using one of the top retrieved images, in practice there is a large performance gap from the oracle, and rerankers using frontier LVLMs do not close this gap – leaving ample room for improvement by future methods. Code will be made publicly available.

Method: The authors create a benchmark using ScanNet, SUN RGB-D, 3D Front datasets, plus new synthetic datasets ProcTHOR-OD and ProcFront. They conduct experiments across various adaptation scenarios including synthetic-to-real, point cloud quality, layout, and instance feature adaptation.

Result: The paper analyzes the impact of different domain gaps on 3D object detectors and provides baseline approaches to improve adaptation performance across datasets.

Conclusion: This work establishes foundational benchmarks for domain adaptive indoor 3D object detection, aiming to inspire future development of detectors with stronger cross-domain generalization capabilities.

Abstract: As a fundamental task for indoor scene understanding, 3D object detection has been extensively studied, and the accuracy on indoor point cloud data has been substantially improved. However, existing researches have been conducted on limited datasets, where the training and testing sets share the same distribution. In this paper, we consider the task of adapting indoor 3D object detectors from one dataset to another, presenting a comprehensive benchmark with ScanNet, SUN RGB-D and 3D Front datasets, as well as our newly proposed large-scale datasets ProcTHOR-OD and ProcFront generated by a 3D simulator. Since indoor point cloud datasets are collected and constructed in different ways, the object detectors are likely to overfit to specific factors within each dataset, such as point cloud quality, bounding box layout and instance features. We conduct experiments across datasets on different adaptation scenarios including synthetic-to-real adaptation, point cloud quality adaptation, layout adaptation and instance feature adaptation, analyzing the impact of different domain gaps on 3D object detectors. We also introduce several approaches to improve adaptation performances, providing baselines for domain adaptive indoor 3D object detection, hoping that future works may propose detectors with stronger generalization ability across domains. Our project homepage can be found in https://jeremyzhao1998.github.io/DAVoteNet-release/.

Method: Used custom dataset of 18 military vehicle classes to test CLIP variants’ occlusion robustness, measuring performance with Normalized Area Under the Curve (NAUC) across different occlusion percentages and types.

Result: Transformer-based CLIP models outperform CNNs; fine-grained dispersed occlusions degrade performance more than larger contiguous ones; linear-probed models show sharp performance drop at ~35% occlusion; backbone finetuning pushes performance drop threshold to >60% occlusion.

Conclusion: Occlusion-specific augmentations during training are crucial, and further research is needed on patch-level sensitivity and architectural resilience for real-world CLIP deployment in military applications.

Abstract: Vision-language models (VLMs) like CLIP enable zero-shot classification by aligning images and text in a shared embedding space, offering advantages for defense applications with scarce labeled data. However, CLIP’s robustness in challenging military environments, with partial occlusion and degraded signal-to-noise ratio (SNR), remains underexplored. We investigate CLIP variants’ robustness to occlusion using a custom dataset of 18 military vehicle classes and evaluate using Normalized Area Under the Curve (NAUC) across occlusion percentages. Four key insights emerge: (1) Transformer-based CLIP models consistently outperform CNNs, (2) fine-grained, dispersed occlusions degrade performance more than larger contiguous occlusions, (3) despite improved accuracy, performance of linear-probed models sharply drops at around 35% occlusion, (4) by finetuning the model’s backbone, this performance drop occurs at more than 60% occlusion. These results underscore the importance of occlusion-specific augmentations during training and the need for further exploration into patch-level sensitivity and architectural resilience for real-world deployment of CLIP.

Method: Uses low sample-per-pixel path tracing for efficient rendering, trains a lightweight event spiking network with bipolar leaky integrate-and-fired units and bidirectional earth mover distance loss to denoise RGB videos into realistic event sequences.

Result: EventTracer runs at about 4 minutes per second of 720p video, captures better scene details, shows greater similarity to real-world event data than other simulators, and enables accurate spatiotemporal modeling.

Conclusion: Establishes EventTracer as a promising tool for creating large-scale event-RGB datasets at low cost, narrowing the sim-to-real gap in event-based vision, and boosting applications in robotics, autonomous driving, and VR/AR.

Abstract: Simulating event streams from 3D scenes has become a common practice in event-based vision research, as it meets the demand for large-scale, high temporal frequency data without setting up expensive hardware devices or undertaking extensive data collections. Yet existing methods in this direction typically work with noiseless RGB frames that are costly to render, and therefore they can only achieve a temporal resolution equivalent to 100-300 FPS, far lower than that of real-world event data. In this work, we propose EventTracer, a path tracing-based rendering pipeline that simulates high-fidelity event sequences from complex 3D scenes in an efficient and physics-aware manner. Specifically, we speed up the rendering process via low sample-per-pixel (SPP) path tracing, and train a lightweight event spiking network to denoise the resulting RGB videos into realistic event sequences. To capture the physical properties of event streams, the network is equipped with a bipolar leaky integrate-and-fired (BiLIF) spiking unit and trained with a bidirectional earth mover distance (EMD) loss. Our EventTracer pipeline runs at a speed of about 4 minutes per second of 720p video, and it inherits the merit of accurate spatiotemporal modeling from its path tracing backbone. We show in two downstream tasks that EventTracer captures better scene details and demonstrates a greater similarity to real-world event data than other event simulators, which establishes it as a promising tool for creating large-scale event-RGB datasets at a low cost, narrowing the sim-to-real gap in event-based vision, and boosting various application scenarios such as robotics, autonomous driving, and VRAR.

Method: Integrates customized InceptionV3 for spatial feature extraction, LSTM for temporal dependency modeling, and ensemble-based genetic algorithm with ADFSA for optimized feature selection.

Result: Achieves 99.65% recognition accuracy on UCF-YouTube dataset, reduces features to as few as 7, and enhances inference time for real-time deployment on edge devices.

Conclusion: The lightweight and scalable framework enables practical real-time HAR applications in resource-aware environments including public safety, assistive technology, and autonomous monitoring systems.

Abstract: Human Activity Recognition (HAR) plays a pivotal role in various applications, including smart surveillance, healthcare, assistive technologies, sports analytics, etc. However, HAR systems still face critical challenges, including high computational costs, redundant features, and limited scalability in real-time scenarios. An optimized hybrid deep learning framework is introduced that integrates a customized InceptionV3, an LSTM architecture, and a novel ensemble-based feature selection strategy. The proposed framework first extracts spatial descriptors using the customized InceptionV3 model, which captures multilevel contextual patterns, region homogeneity, and fine-grained localization cues. The temporal dependencies across frames are then modeled using LSTMs to effectively encode motion dynamics. Finally, an ensemble-based genetic algorithm with Adaptive Dynamic Fitness Sharing and Attention (ADFSA) is employed to select a compact and optimized feature set by dynamically balancing objectives such as accuracy, redundancy, uniqueness, and complexity reduction. Consequently, the selected feature subsets, which are both diverse and discriminative, enable various lightweight machine learning classifiers to achieve accurate and robust HAR in heterogeneous environments. Experimental results on the robust UCF-YouTube dataset, which presents challenges such as occlusion, cluttered backgrounds, motion dynamics, and poor illumination, demonstrate good performance. The proposed approach achieves 99.65% recognition accuracy, reduces features to as few as 7, and enhances inference time. The lightweight and scalable nature of the HAR system supports real-time deployment on edge devices such as Raspberry Pi, enabling practical applications in intelligent, resource-aware environments, including public safety, assistive technology, and autonomous monitoring systems.

Method: Uses bi-mode annealing to enable both thinking and non-thinking capabilities, followed by Bi-mode Policy Optimization (BPO) to improve decision-making on when to activate thinking. Trained on curated dataset with both modes and improved GRPO framework.

Result: Achieves SOTA performance across 25 benchmarks, outperforms Qwen2.5-VL-7B in most tasks, and matches larger models like Kimi-VL-A3B-Thinking-2506 (16B) with lower computational cost.

Conclusion: R-4B provides an efficient solution by adaptively using thinking only when needed, demonstrating superior performance with reduced computational requirements.

Abstract: Multimodal Large Language Models (MLLMs) equipped with step-by-step thinking capabilities have demonstrated remarkable performance on complex reasoning problems. However, this thinking process is redundant for simple problems solvable without complex reasoning. To address this inefficiency, we propose R-4B, an auto-thinking MLLM, which can adaptively decide when to think based on problem complexity. The central idea of R-4B is to empower the model with both thinking and non-thinking capabilities using bi-mode annealing, and apply Bi-mode Policy Optimization (BPO) to improve the model’s accuracy in determining whether to activate the thinking process. Specifically, we first train the model on a carefully curated dataset spanning various topics, which contains samples from both thinking and non-thinking modes. Then it undergoes a second phase of training under an improved GRPO framework, where the policy model is forced to generate responses from both modes for each input query. Experimental results show that R-4B achieves state-of-the-art performance across 25 challenging benchmarks. It outperforms Qwen2.5-VL-7B in most tasks and achieves performance comparable to larger models such as Kimi-VL-A3B-Thinking-2506 (16B) on reasoning-intensive benchmarks with lower computational cost.

Method: Uses a dual-decoder transformer architecture with network-friendly vector representation, decoupling command type and parameter generation while maintaining geometric precision through soft target distribution loss.

Result: Developed CAD-VGDrawing dataset and demonstrated effective conversion of 2D drawings to parametric CAD models with preserved geometric precision.

Conclusion: The framework successfully bridges the gap between traditional 2D engineering workflows and modern parametric CAD generation, enabling automated conversion while maintaining design intent.

Abstract: Computer-Aided Design (CAD) generative modeling is driving significant innovations across industrial applications. Recent works have shown remarkable progress in creating solid models from various inputs such as point clouds, meshes, and text descriptions. However, these methods fundamentally diverge from traditional industrial workflows that begin with 2D engineering drawings. The automatic generation of parametric CAD models from these 2D vector drawings remains underexplored despite being a critical step in engineering design. To address this gap, our key insight is to reframe CAD generation as a sequence-to-sequence learning problem where vector drawing primitives directly inform the generation of parametric CAD operations, preserving geometric precision and design intent throughout the transformation process. We propose Drawing2CAD, a framework with three key technical components: a network-friendly vector primitive representation that preserves precise geometric information, a dual-decoder transformer architecture that decouples command type and parameter generation while maintaining precise correspondence, and a soft target distribution loss function accommodating inherent flexibility in CAD parameters. To train and evaluate Drawing2CAD, we create CAD-VGDrawing, a dataset of paired engineering drawings and parametric CAD models, and conduct thorough experiments to demonstrate the effectiveness of our method. Code and dataset are available at https://github.com/lllssc/Drawing2CAD.

Method: Introduce ELV-Halluc benchmark, analyze SAH patterns, experiment with positional encoding strategies, and use DPO (Direct Preference Optimization) with 8K adversarial data pairs to enhance semantic distinction capabilities.

Result: Confirmed SAH existence showing it increases with semantic complexity, especially with rapidly changing semantics. Achieved 27.7% SAH reduction and improvements on both ELV-Halluc and Video-MME benchmarks.

Conclusion: SAH is a distinct hallucination type in long videos that requires specialized mitigation approaches. Positional encoding and DPO strategies effectively reduce SAH, highlighting the need for targeted solutions for long-video understanding challenges.

Abstract: Video multimodal large language models (Video-MLLMs) have achieved remarkable progress in video understanding. However, they remain vulnerable to hallucination-producing content inconsistent with or unrelated to video inputs. Previous video hallucination benchmarks primarily focus on short-videos. They attribute hallucinations to factors such as strong language priors, missing frames, or vision-language biases introduced by the visual encoder. While these causes indeed account for most hallucinations in short videos, they still oversimplify the cause of hallucinations. Sometimes, models generate incorrect outputs but with correct frame-level semantics. We refer to this type of hallucination as Semantic Aggregation Hallucination (SAH), which arises during the process of aggregating frame-level semantics into event-level semantic groups. Given that SAH becomes particularly critical in long videos due to increased semantic complexity across multiple events, it is essential to separate and thoroughly investigate the causes of this type of hallucination. To address the above issues, we introduce ELV-Halluc, the first benchmark dedicated to long-video hallucination, enabling a systematic investigation of SAH. Our experiments confirm the existence of SAH and show that it increases with semantic complexity. Additionally, we find that models are more prone to SAH on rapidly changing semantics. Moreover, we discuss potential approaches to mitigate SAH. We demonstrate that positional encoding strategy contributes to alleviating SAH, and further adopt DPO strategy to enhance the model’s ability to distinguish semantics within and across events. To support this, we curate a dataset of 8K adversarial data pairs and achieve improvements on both ELV-Halluc and Video-MME, including a substantial 27.7% reduction in SAH ratio.

Method: Multi-task learning framework that jointly optimizes VIF and crowd counting. Uses dynamic loss weighting for task balance and incorporates adversarial training for robustness. Leverages population density information to guide fusion process.

Result: Experimental results show improved image fusion quality and superior crowd counting performance compared to existing methods on public datasets.

Conclusion: Integrating crowd counting with VIF creates a mutually beneficial framework that enhances both tasks, particularly effective in dense crowded scenes with minimal annotation requirements.

Abstract: Visible and infrared image fusion (VIF) is an important multimedia task in computer vision. Most VIF methods focus primarily on optimizing fused image quality. Recent studies have begun incorporating downstream tasks, such as semantic segmentation and object detection, to provide semantic guidance for VIF. However, semantic segmentation requires extensive annotations, while object detection, despite reducing annotation efforts compared with segmentation, faces challenges in highly crowded scenes due to overlapping bounding boxes and occlusion. Moreover, although RGB-T crowd counting has gained increasing attention in recent years, no studies have integrated VIF and crowd counting into a unified framework. To address these challenges, we propose FusionCounting, a novel multi-task learning framework that integrates crowd counting into the VIF process. Crowd counting provides a direct quantitative measure of population density with minimal annotation, making it particularly suitable for dense scenes. Our framework leverages both input images and population density information in a mutually beneficial multi-task design. To accelerate convergence and balance tasks contributions, we introduce a dynamic loss function weighting strategy. Furthermore, we incorporate adversarial training to enhance the robustness of both VIF and crowd counting, improving the model’s stability and resilience to adversarial attacks. Experimental results on public datasets demonstrate that FusionCounting not only enhances image fusion quality but also achieves superior crowd counting performance.

Method: Comparative analysis of four representative approaches through comprehensive evaluation on multiple datasets and expert clinician assessment.

Result: YOLO-Stutter and FluentNet provide efficiency but limited transparency; UDM achieves best balance of accuracy and interpretability; SSDM could not be fully reproduced.

Conclusion: The analysis highlights trade-offs among approaches and identifies future directions for clinically viable dysfluency modeling, with UDM showing the most promise for clinical adoption.

Abstract: Recent advances in dysfluency detection have introduced a variety of modeling paradigms, ranging from lightweight object-detection inspired networks (YOLOStutter) to modular interpretable frameworks (UDM). While performance on benchmark datasets continues to improve, clinical adoption requires more than accuracy: models must be controllable and explainable. In this paper, we present a systematic comparative analysis of four representative approaches–YOLO-Stutter, FluentNet, UDM, and SSDM–along three dimensions: performance, controllability, and explainability. Through comprehensive evaluation on multiple datasets and expert clinician assessment, we find that YOLO-Stutter and FluentNet provide efficiency and simplicity, but with limited transparency; UDM achieves the best balance of accuracy and clinical interpretability; and SSDM, while promising, could not be fully reproduced in our experiments. Our analysis highlights the trade-offs among competing approaches and identifies future directions for clinically viable dysfluency modeling. We also provide detailed implementation insights and practical deployment considerations for each approach.

Method: Used infinitely scalable framework to synthesize research-level QA from arXiv papers, leveraging temporal structure to detect performance decay after knowledge cutoffs. Evaluated 4 frontier models on 1,643 multi-step reasoning questions from 20,277 papers across 26 months.

Result: Consistent lack of significant performance decay near knowledge cutoff dates across models of various sizes, developers, and release dates. Multi-step reasoning complexity appears to mitigate benchmark contamination.

Conclusion: Advocates for paradigm shift towards reasoning-driven synthesis for benchmark construction rather than periodic collection of new questions, and open-sources code/dataset for reproducibility.

Abstract: Capability evaluation of large language models (LLMs) is increasingly shadowed by rising concerns of data contamination that cast doubts on whether static benchmarks measure genuine reasoning or mere memorization. We present an empirical study using an infinitely scalable framework to synthesize research-level QA directly from arXiv papers, harnessing the natural temporal structure of research publications where performance decay after knowledge cutoffs may indicate potential contamination. We evaluated 4 frontier model represented by 2 models of different knowledge cutoff dates per family on 1,643 multi-step reasoning questions synthesized from 20,277 arXiv papers stratified over 26 months, covering at least 6 months before and after all cutoff dates. Our results consistently showed a lack of significant performance decay near knowledge cutoff dates for models of various sizes, developers, and release dates. We further performed a comparative analysis with previous longitudinal studies that reported significant post-cutoff performance decay using directly retrieved questions based on public data. we hypothesize that the multi-step reasoning required by our synthesis pipeline offered additional complexity that goes deeper than shallow memorization, which effectively serves a mitigation strategy against benchmark contamination. We fully open source our code and dataset to aid reproducibility and advocate for a paradigm shift that prioritize reasoning-driven synthesis to construct benchmarks over simply collecting newly released questions periodically.

Method: Developed a framework that models social learning as joint probabilistic inference over structured, executable world models. Used pretrained language models as probabilistic models of human advice sharing, enabling agents to generate advice and interpret linguistic input during Bayesian inference across 10 video games.

Result: Linguistic guidance shaped exploration and accelerated learning by reducing risky interactions and speeding up key discoveries in both humans and models. Successful knowledge transfer between humans and models was demonstrated through iterated learning experiments.

Conclusion: Structured, language-compatible representations enable effective human-machine collaborative learning, showing how linguistic guidance can be integrated with direct experience to accelerate learning and facilitate knowledge accumulation across generations.

Abstract: The ability to combine linguistic guidance from others with direct experience is central to human development, enabling safe and rapid learning in new environments. How do people integrate these two sources of knowledge, and how might AI systems? We present a computational framework that models social learning as joint probabilistic inference over structured, executable world models given sensorimotor and linguistic data. We make this possible by turning a pretrained language model into a probabilistic model of how humans share advice conditioned on their beliefs, allowing our agents both to generate advice for others and to interpret linguistic input as evidence during Bayesian inference. Using behavioral experiments and simulations across 10 video games, we show how linguistic guidance can shape exploration and accelerate learning by reducing risky interactions and speeding up key discoveries in both humans and models. We further explore how knowledge can accumulate across generations through iterated learning experiments and demonstrate successful knowledge transfer between humans and models – revealing how structured, language-compatible representations might enable human-machine collaborative learning.

Method: Extracts logprobs, computes Shannon entropy on top-k alternatives, and uses OR-logic trigger (perplexity + max token entropy + low-confidence token count) to trigger refinement. Passes compact uncertainty report back to model for corrective edits.

Result: Small model with this loop achieves 95% of reference reasoning model quality at ~1/3 cost. Selective refinement on ~31% of responses improves accuracy by 16 percentage points over single-pass inference.

Conclusion: The uncertainty-aware loop provides an effective middle ground between single-pass inference and expensive reasoning chains, making it practical for production deployments where both quality and cost matter.

Abstract: Reasoning models often outperform smaller models but at 3–5$\times$ higher cost and added latency. We present entropy-guided refinement: a lightweight, test-time loop that uses token-level uncertainty to trigger a single, targeted refinement pass. We extract logprobs, compute Shannon entropy on top-$k$ alternatives, and apply a simple OR-logic trigger over perplexity, maximum token entropy, and low-confidence-token count. Unlike approaches that use entropy only for measurement or decoding, we pass a compact uncertainty report (tokens, confidences, alternatives, context) back to the model to guide corrective edits. On representative technical queries across reasoning, mathematics, and code generation tasks, a small model with our loop approaches 95% of a reference reasoning model’s quality at approximately one-third of the cost. The method achieves selective refinement on ~31% of responses while improving accuracy by 16 percentage points over single-pass inference. We demonstrate that this uncertainty-aware loop provides an effective middle ground between single-pass inference and expensive reasoning chains, making it practical for production deployments where both quality and cost matter.

Method: The paper formalizes an AMDM algorithm that: 1) normalizes heterogeneous metrics, 2) applies per-axis exponentially weighted moving-average thresholds, and 3) performs joint anomaly detection via Mahalanobis distance. They conduct simulations and real-world experiments to validate the approach.

Result: AMDM significantly improves performance: reduces anomaly-detection latency from 12.3s to 5.6s on simulated goal drift, and cuts false-positive rates from 4.5% to 0.9% compared to static thresholds. The paper includes comparison tables, ROC/PR curves, and case study reanalysis.

Conclusion: The AMDM algorithm provides an effective solution for comprehensive monitoring of agentic AI systems, addressing both technical and human-centered metrics. The authors provide code, data, and reproducibility materials to facilitate adoption and further research.

Abstract: Agentic artificial intelligence (AI) – multi-agent systems that combine large language models with external tools and autonomous planning – are rapidly transitioning from research laboratories into high-stakes domains. Our earlier “Basic” paper introduced a five-axis framework and proposed preliminary metrics such as goal drift and harm reduction but did not provide an algorithmic instantiation or empirical evidence. This “Advanced” sequel fills that gap. First, we revisit recent benchmarks and industrial deployments to show that technical metrics still dominate evaluations: a systematic review of 84 papers from 2023–2025 found that 83% report capability metrics while only 30% consider human-centred or economic axes [2]. Second, we formalise an Adaptive Multi-Dimensional Monitoring (AMDM) algorithm that normalises heterogeneous metrics, applies per-axis exponentially weighted moving-average thresholds and performs joint anomaly detection via the Mahalanobis distance. Third, we conduct simulations and real-world experiments. AMDM cuts anomaly-detection latency from 12.3 s to 5.6 s on simulated goal drift and reduces false-positive rates from 4.5% to 0.9% compared with static thresholds. We present a comparison table and ROC/PR curves, and we reanalyse case studies to surface missing metrics. Code, data and a reproducibility checklist accompany this paper to facilitate replication.

Method: OmniDPO uses two strategies: (1) text-preference sample pairs to enhance understanding of audio-video interactions, and (2) multimodal-preference sample pairs to strengthen attention to visual and auditory information.

Result: Experiments on two OLLMs show OmniDPO effectively mitigates multimodal hallucinations and significantly enhances reasoning capabilities across modalities.

Conclusion: OmniDPO successfully addresses hallucination challenges in OLLMs by improving multimodal grounding through preference alignment strategies.

Abstract: Recently, Omni-modal large language models (OLLMs) have sparked a new wave of research, achieving impressive results in tasks such as audio-video understanding and real-time environment perception. However, hallucination issues still persist. Similar to the bimodal setting, the priors from the text modality tend to dominate, leading OLLMs to rely more heavily on textual cues while neglecting visual and audio information. In addition, fully multimodal scenarios introduce new challenges. Most existing models align visual or auditory modalities with text independently during training, while ignoring the intrinsic correlations between video and its corresponding audio. This oversight results in hallucinations when reasoning requires interpreting hidden audio cues embedded in video content. To address these challenges, we propose OmniDPO, a preference-alignment framework designed to mitigate hallucinations in OLLMs. Specifically, OmniDPO incorporates two strategies: (1) constructing text-preference sample pairs to enhance the model’s understanding of audio-video interactions; and (2) constructing multimodal-preference sample pairs to strengthen the model’s attention to visual and auditory information. By tackling both challenges, OmniDPO effectively improves multimodal grounding and reduces hallucination. Experiments conducted on two OLLMs demonstrate that OmniDPO not only effectively mitigates multimodal hallucinations but also significantly enhances the models’ reasoning capabilities across modalities. All code and datasets will be released upon paper acceptance.

Method: Agents with emotion classifiers of varying accuracy (poor/medium/high) placed on 2D toroidal lattice, responding to perceived emotions by moving toward positive and away from negative emotions. Experiments conducted on homogeneous/heterogeneous populations with emotional shocks.

Result: Low-accuracy classifiers reliably caused diminished trust, emotional disintegration into sadness, and disordered social organization. High-accuracy agents developed hardy emotional clusters and resilience to disruptions. Misperception alone generated segregation even in neutral scenarios.

Conclusion: Biases or imprecision in emotion recognition significantly warp social processes and disrupt emotional integration, highlighting the critical importance of accurate emotion perception for stable social organization.

Abstract: The ability of humans to detect and respond to others’ emotions is fundamental to understanding social behavior. Here, agents are instantiated with emotion classifiers of varying accuracy to study the impact of perceptual accuracy on emergent emotional and spatial behavior. Agents are visually represented with face photos from the KDEF database and endowed with one of three classifiers trained on the JAFFE (poor), CK+ (medium), or KDEF (high) datasets. Agents communicate locally on a 2D toroidal lattice, perceiving neighbors’ emotional state based on their classifier and responding with movement toward perceived positive emotions and away from perceived negative emotions. Note that the agents respond to perceived, instead of ground-truth, emotions, introducing systematic misperception and frustration. A battery of experiments is carried out on homogeneous and heterogeneous populations and scenarios with repeated emotional shocks. Results show that low-accuracy classifiers on the part of the agent reliably result in diminished trust, emotional disintegration into sadness, and disordered social organization. By contrast, the agent that develops high accuracy develops hardy emotional clusters and resilience to emotional disruptions. Even in emotionally neutral scenarios, misperception is enough to generate segregation and disintegration of cohesion. These findings underscore the fact that biases or imprecision in emotion recognition may significantly warp social processes and disrupt emotional integration.

Method: Extend topological evidence models to groups, develop logical systems for group evidence, knowledge and belief, and add dynamic evidence-sharing operators.

Result: Complete axiomatization and decidability proofs for logics of group evidence, knowledge, and belief, showing dynamic extensions are co-expressive with static bases.

Conclusion: Successfully formalized group knowledge and belief in evidence models with complete logical systems, demonstrating the feasibility of dynamic evidence-sharing while maintaining expressivity.

Abstract: We study notions of (virtual) group knowledge and group belief within multi-agent evidence models, obtained by extending the topological semantics of evidence-based belief and fallible knowledge from individuals to groups. We completely axiomatize and show the decidability of the logic of (“hard” and “soft”) group evidence, and do the same for an especially interesting fragment of it: the logic of group knowledge and group belief. We also extend these languages with dynamic evidence-sharing operators, and completely axiomatize the corresponding logics, showing that they are co-expressive with their static bases.

Method: Conducted 150 debates across 15 scenarios using five ensemble configurations of local open-source models, comparing against single-model baselines using a 7-point evaluation rubric.

Result: Ensembles outperformed single models (3.48 vs 3.13 overall), with largest gains in reasoning depth (+19.4%) and argument quality (+34.1%). Strongest improvements in truthfulness (+1.25 points) and human enhancement (+0.80 points).

Conclusion: Local open-source ensemble debates provide an effective, accessible, and reproducible method for improving alignment-oriented reasoning, with code and dataset provided for broader research participation.

Abstract: As large language models (LLMs) take on greater roles in high-stakes decisions, alignment with human values is essential. Reliance on proprietary APIs limits reproducibility and broad participation. We study whether local open-source ensemble debates can improve alignmentoriented reasoning. Across 150 debates spanning 15 scenarios and five ensemble configurations, ensembles outperform single-model baselines on a 7-point rubric (overall: 3.48 vs. 3.13), with the largest gains in reasoning depth (+19.4%) and argument quality (+34.1%). Improvements are strongest for truthfulness (+1.25 points) and human enhancement (+0.80). We provide code, prompts, and a debate data set, providing an accessible and reproducible foundation for ensemble-based alignment evaluation.

Method: HiVA uses Semantic-Topological Evolution (STEV) algorithm that models workflows as self-organized graphs, optimizes hybrid semantic-topological spaces using textual gradients as discrete-domain surrogates for backpropagation, with iterative process including Multi-Armed Bandit routing, diagnostic gradient generation, and coordinated updates.

Result: Experiments across dialogue, coding, long-context Q&A, mathematical, and agentic benchmarks show 5-10% improvements in task accuracy and enhanced resource efficiency compared to existing baselines.

Conclusion: HiVA establishes effectiveness in autonomous task execution by enabling self-organized workflow optimization that adapts to unknown environments while maintaining transferable reasoning structures.

Abstract: Autonomous agents play a crucial role in advancing Artificial General Intelligence, enabling problem decomposition and tool orchestration through Large Language Models (LLMs). However, existing paradigms face a critical trade-off. On one hand, reusable fixed workflows require manual reconfiguration upon environmental changes; on the other hand, flexible reactive loops fail to distill reasoning progress into transferable structures. We introduce Hierarchical Variable Agent (HiVA), a novel framework modeling agentic workflows as self-organized graphs with the Semantic-Topological Evolution (STEV) algorithm, which optimizes hybrid semantic-topological spaces using textual gradients as discrete-domain surrogates for backpropagation. The iterative process comprises Multi-Armed Bandit-infused forward routing, diagnostic gradient generation from environmental feedback, and coordinated updates that co-evolve individual semantics and topology for collective optimization in unknown environments. Experiments on dialogue, coding, Long-context Q&A, mathematical, and agentic benchmarks demonstrate improvements of 5-10% in task accuracy and enhanced resource efficiency over existing baselines, establishing HiVA’s effectiveness in autonomous task execution.

Method: Documents are embedded, grouped into semantically coherent clusters, and represented by cached centroids. At query time, routing goes to top centroid(s) and retrieves context only within those clusters.

Result: On HotpotQA and SQuAD corpora (100-500 chunks), MODE matches or exceeds dense-retrieval baseline in answer quality while reducing end-to-end retrieval time. Tighter clusters improve downstream accuracy.

Conclusion: MODE provides a practical solution for small and medium corpora where simplicity, speed, and topical focus are important, eliminating external vector-database infrastructure while maintaining performance.

Abstract: Retrieval-Augmented Generation (RAG) often relies on large vector databases and cross-encoders tuned for large-scale corpora, which can be excessive for small, domain-specific collections. We present MODE (Mixture of Document Experts), a lightweight alternative that replaces fine-grained nearest-neighbor search with cluster-and-route retrieval. Documents are embedded, grouped into semantically coherent clusters, and represented by cached centroids. At query time, we route to the top centroid(s) and retrieve context only within those clusters, eliminating external vector-database infrastructure and reranking while keeping latency low. On HotpotQA and SQuAD corpora with 100-500 chunks, MODE matches or exceeds a dense-retrieval baseline in answer quality while reducing end-to-end retrieval time. Ablations show that cluster granularity and multi-cluster routing control the recall/precision trade-off, and that tighter clusters improve downstream accuracy. MODE offers a practical recipe for small and medium corpora where simplicity, speed, and topical focus matter.

Method: DACE assesses task difficulty online based on policy success rates and modulates intrinsic rewards: penalizing high certainty for difficult tasks to encourage exploration, and rewarding high certainty for easier tasks to promote learning efficiency.

Result: Experiments on AIME and MATH benchmarks show DACE significantly outperforms baselines, achieving higher accuracy and more robust performance with test-time compute scaling.

Conclusion: DACE’s adaptive approach effectively fosters exploration without sacrificing precision, demonstrating that leveraging LLM self-certainty based on task difficulty improves reinforcement learning for reasoning tasks.

Abstract: Reinforcement Learning with Verifiable Feedback (RLVF) has become a key technique for enhancing the reasoning abilities of Large Language Models (LLMs). However, its reliance on sparse, outcome based rewards, which only indicate if a final answer is correct or not, fails to provide granular guidance on the reasoning process itself. This limitation hinders efficient learning, as the model cannot distinguish between high quality and inefficient solutions, nor can it learn effectively from different types of failures. To address this, we observe that an LLMs self-certainty often correlates with task difficulty and solution quality. We introduce Difficulty Aware Certainty guided Exploration (DACE), a novel RL algorithm that leverages this insight to dynamically balance the exploration exploitation trade-off. DACE assesses task difficulty online based on the policys success rate. It then uses this signal to modulate an intrinsic reward: for difficult tasks where the model is struggling, DACE encourages exploration by penalizing high certainty; for easier tasks, it encourages learning efficiency by rewarding high certainty. Experiments on challenging mathematical reasoning benchmarks (AIME, MATH) show that DACE significantly outperforms strong baselines. The DACE-trained models not only achieve higher accuracy but also demonstrate more robust performance when scaling test-time compute, validating that our adaptive approach fosters effective exploration without sacrificing precision.

Method: Created Health Access Resource Planner (HARP) with two algorithms: learning-augmented approach for single-step improvement over expert recommendations, and greedy algorithm for multi-step planning with worst-case approximation guarantees.

Result: Demonstrated empirical efficacy through collaboration with Ethiopian Public Health Institute and Ministry of Health across three regions and various planning scenarios.

Conclusion: HARP provides a principled optimization framework for sequential health facility planning that can effectively guide resource allocation decisions under budget uncertainty while ensuring regional equity.

Abstract: As part of nationwide efforts aligned with the United Nations’ Sustainable Development Goal 3 on Universal Health Coverage, Ethiopia’s Ministry of Health is strengthening health posts to expand access to essential healthcare services. However, only a fraction of this health system strengthening effort can be implemented each year due to limited budgets and other competing priorities, thus the need for an optimization framework to guide prioritization across the regions of Ethiopia. In this paper, we develop a tool, Health Access Resource Planner (HARP), based on a principled decision-support optimization framework for sequential facility planning that aims to maximize population coverage under budget uncertainty while satisfying region-specific proportionality targets at every time step. We then propose two algorithms: (i) a learning-augmented approach that improves upon expert recommendations at any single-step; and (ii) a greedy algorithm for multi-step planning, both with strong worst-case approximation estimation. In collaboration with the Ethiopian Public Health Institute and Ministry of Health, we demonstrated the empirical efficacy of our method on three regions across various planning scenarios.

Method: UDR wraps around any language model and provides a system for users to create, edit, and refine custom deep research strategies. It includes example strategies (minimal, expansive, intensive) and a user interface for experimentation.

Result: The system demonstrates generality by supporting diverse research strategies without additional training or finetuning, enabling flexible and customizable deep research workflows.

Conclusion: UDR provides a flexible framework for deep research that empowers users to define their own research strategies rather than being constrained by hard-coded approaches, making agentic research systems more accessible and adaptable.

Abstract: Deep research tools are among the most impactful and most commonly encountered agentic systems today. We observe, however, that each deep research agent introduced so far is hard-coded to carry out a particular research strategy using a fixed choice of tools. We introduce Universal Deep Research (UDR), a generalist agentic system that wraps around any language model and enables the user to create, edit, and refine their own entirely custom deep research strategies without any need for additional training or finetuning. To showcase the generality of our system, we equip UDR with example minimal, expansive, and intensive research strategies, and provide a user interface to facilitate experimentation with the system.

Method: Instruction-Level Weight Shaping (ILWS) uses curated system instructions as external pseudo-parameters updated after each session. A Reflection Engine analyzes conversations, diagnoses reasoning, and proposes typed deltas (ΔS, ΔU, ΔT) for instructions, user preferences, and tools. These deltas are version-controlled, evaluated with ratings, and distilled into model parameters when an edit budget threshold is crossed.

Result: In enterprise support, ILWS increased throughput 2.4-5.0x and reduced audited hallucinations by about 80% compared to frozen baseline. In Adobe Commerce Cloud proof of concept, it achieved 4-5x more tickets per hour and about 80% lower time per ticket with autonomous instruction updates.

Conclusion: ILWS provides an effective approach for continuous LLM improvement that preserves governance, removes per-call retrieval, and generalizes to dynamic domains requiring adaptive reasoning and low-latency deployment, converting prompt-space improvements into weight-space without downtime.

Abstract: Large language models (LLMs) are fluent but largely static after pre-training; new or shifting knowledge is typically added with retrieval-augmented generation (RAG) or fine-tuning. RAG raises latency and engineering overhead and often fails to integrate facts; prompt engineering is brittle and can conflict with prior knowledge; fine-tuning is costly and risks catastrophic forgetting. We propose Instruction-Level Weight Shaping (ILWS): curated system instructions act as external, auditable pseudo-parameters updated after each session via reflection and user feedback. A Reflection Engine inspects conversation traces, diagnoses reasoning successes and failures, and proposes typed deltas $\Delta K=(\Delta S,\Delta U,\Delta T)$ over instructions, user preferences, and tools. Deltas are version-controlled, evaluated with a sliding window of 1-5 star ratings, auto-repaired on first failure, and rolled back on repeated failure. When an edit budget crosses a threshold, the agent compiles a rating-weighted synthetic set and distills matured instruction-space gains into parameters, converting prompt-space improvements into weight-space without downtime. ILWS makes explicit the low-rank shaping induced by context in transformer blocks, preserves governance, and removes per-call retrieval. In enterprise support it increased throughput 2.4-5.0x and cut audited hallucinations by about 80% versus a frozen baseline. In an Adobe Commerce Cloud proof of concept “L0 Support”, it achieved 4-5x more tickets per hour and about 80% lower time per ticket, with autonomous instruction updates and optional tool synthesis. Because ILWS operates at the instruction layer until controlled distillation, it generalizes to dynamic domains (legal, medical, engineering) requiring adaptive reasoning, tool creation, and low-latency deployment.

Method: Formally define BRaP as a graph search problem and propose five search-based solution algorithms using joint configuration space search, classical planning, multi-agent pathfinding, and expert heuristics.

Result: The methods demonstrate efficiency in creating rearrangement plans for deeply buried blocks in up to 80x80 grids, despite the exponential relationship between search space size and block number.

Conclusion: The proposed search-based approaches effectively solve the Block Rearrangement Problem and show scalability for large warehouse grid configurations.

Abstract: We introduce the Block Rearrangement Problem (BRaP), a challenging component of large warehouse management which involves rearranging storage blocks within dense grids to achieve a target state. We formally define the BRaP as a graph search problem. Building on intuitions from sliding puzzle problems, we propose five search-based solution algorithms, leveraging joint configuration space search, classical planning, multi-agent pathfinding, and expert heuristics. We evaluate the five approaches empirically for plan quality and scalability. Despite the exponential relation between search space size and block number, our methods demonstrate efficiency in creating rearrangement plans for deeply buried blocks in up to 80x80 grids.

Method: Proposes SHERPA framework that structures LLM execution using hierarchical state machines, enabling fine-grained control via rules or ML-based decisions including LLMs themselves.

Result: SHERPA significantly improves LLM output quality across code generation, class name generation, and question answering tasks, particularly beneficial for complex tasks with established human best practices but limited training data.

Conclusion: Integrating well-designed state machines provides an effective mechanism to control LLM behavior and incorporate domain expertise, demonstrating substantial performance improvements on various complex tasks.

Abstract: Recently, large language models (LLMs) have achieved widespread application across various fields. Despite their impressive capabilities, LLMs suffer from a lack of structured reasoning ability, particularly for complex tasks requiring domain-specific best practices, which are often unavailable in the training data. Although multi-step prompting methods incorporating human best practices, such as chain-of-thought and tree-of-thought, have gained popularity, they lack a general mechanism to control LLM behavior. In this paper, we propose SHERPA, a model-driven framework to improve the LLM performance on complex tasks by explicitly incorporating domain-specific best practices into hierarchical state machines. By structuring the LLM execution processes using state machines, SHERPA enables more fine-grained control over their behavior via rules or decisions driven by machine learning-based approaches, including LLMs. We show that SHERPA is applicable to a wide variety of tasks-specifically, code generation, class name generation, and question answering-replicating previously proposed approaches while further improving the performance. We demonstrate the effectiveness of SHERPA for the aforementioned tasks using various LLMs. Our systematic evaluation compares different state machine configurations against baseline approaches without state machines. Results show that integrating well-designed state machines significantly improves the quality of LLM outputs, and is particularly beneficial for complex tasks with well-established human best practices but lacking data used for training LLMs.

Method: SIGMUS uses Large Language Models to generate world knowledge for identifying relationships between urban incidents and multimodal data, organizing evidence into knowledge graphs without human-encoded rules.

Result: The system successfully connects 5 different data sources (news articles, CCTV images, air quality, weather, traffic) with relevant incidents occurring at same time and location.

Conclusion: LLM-based semantic integration enables automated reasoning about urban incidents from fragmented multimodal data sources, providing scalable incident analysis without manual rule engineering.

Abstract: Modern urban spaces are equipped with an increasingly diverse set of sensors, all producing an abundance of multimodal data. Such multimodal data can be used to identify and reason about important incidents occurring in urban landscapes, such as major emergencies, cultural and social events, as well as natural disasters. However, such data may be fragmented over several sources and difficult to integrate due to the reliance on human-driven reasoning for identifying relationships between the multimodal data corresponding to an incident, as well as understanding the different components which define an incident. Such relationships and components are critical to identifying the causes of such incidents, as well as producing forecasting the scale and intensity of future incidents as they begin to develop. In this work, we create SIGMUS, a system for Semantic Integration for Knowledge Graphs in Multimodal Urban Spaces. SIGMUS uses Large Language Models (LLMs) to produce the necessary world knowledge for identifying relationships between incidents occurring in urban spaces and data from different modalities, allowing us to organize evidence and observations relevant to an incident without relying and human-encoded rules for relating multimodal sensory data with incidents. This organized knowledge is represented as a knowledge graph, organizing incidents, observations, and much more. We find that our system is able to produce reasonable connections between 5 different data sources (new article text, CCTV images, air quality, weather, and traffic measurements) and relevant incidents occurring at the same time and location.

Method: A three-agent framework: Reasoning Agent generates disambiguation questions, Knowledge Agent resolves them via focused web searches, and Deduplication Agent reuses validated reasoning traces to reduce redundancy. Includes human-in-the-loop for uncertain cases.

Result: Outperforms strong baselines on real-world consumer goods datasets, achieving higher accuracy and robustness in challenging scenarios like bundle identification and brand origin disambiguation.

Conclusion: Q2K provides a scalable and interpretable solution for product integration by balancing accuracy with efficiency through reasoning reuse rather than repeated searches.

Abstract: Identifying whether two product listings refer to the same Stock Keeping Unit (SKU) is a persistent challenge in ecommerce, especially when explicit identifiers are missing and product names vary widely across platforms. Rule based heuristics and keyword similarity often misclassify products by overlooking subtle distinctions in brand, specification, or bundle configuration. To overcome these limitations, we propose Question to Knowledge (Q2K), a multi agent framework that leverages Large Language Models (LLMs) for reliable SKU mapping. Q2K integrates: (1) a Reasoning Agent that generates targeted disambiguation questions, (2) a Knowledge Agent that resolves them via focused web searches, and (3) a Deduplication Agent that reuses validated reasoning traces to reduce redundancy and ensure consistency. A human in the loop mechanism further refines uncertain cases. Experiments on real world consumer goods datasets show that Q2K surpasses strong baselines, achieving higher accuracy and robustness in difficult scenarios such as bundle identification and brand origin disambiguation. By reusing retrieved reasoning instead of issuing repeated searches, Q2K balances accuracy with efficiency, offering a scalable and interpretable solution for product integration.

Method: Introduces NEWSAGENT benchmark with 6k human-verified examples from real news, converted to text for broad compatibility. Evaluates open- and closed-source LLMs with agentic frameworks on tasks like narrative perspective identification, keyword querying, historical background retrieval, and article generation.

Result: Agents demonstrate capability in retrieving relevant facts but struggle significantly with planning and narrative integration tasks that require active information discovery and contextual reasoning.

Conclusion: NEWSAGENT provides a realistic testbed for evaluating agent capabilities in multimodal web data manipulation for real-world productivity applications, highlighting current limitations in complex reasoning and planning tasks.

Abstract: Recent advances in autonomous digital agents from industry (e.g., Manus AI and Gemini’s research mode) highlight potential for structured tasks by autonomous decision-making and task decomposition; however, it remains unclear to what extent the agent-based systems can improve multimodal web data productivity. We study this in the realm of journalism, which requires iterative planning, interpretation, and contextual reasoning from multimodal raw contents to form a well structured news. We introduce NEWSAGENT, a benchmark for evaluating how agents can automatically search available raw contents, select desired information, and edit and rephrase to form a news article by accessing core journalistic functions. Given a writing instruction and firsthand data as how a journalist initiates a news draft, agents are tasked to identify narrative perspectives, issue keyword-based queries, retrieve historical background, and generate complete articles. Unlike typical summarization or retrieval tasks, essential context is not directly available and must be actively discovered, reflecting the information gaps faced in real-world news writing. NEWSAGENT includes 6k human-verified examples derived from real news, with multimodal contents converted to text for broad model compatibility. We evaluate open- and closed-sourced LLMs with commonly-used agentic frameworks on NEWSAGENT, which shows that agents are capable of retrieving relevant facts but struggling with planning and narrative integration. We believe that NEWSAGENT serves a realistic testbed for iterating and evaluating agent capabilities in terms of multimodal web data manipulation to real-world productivity.

Method: Combines hybrid multi-agent architecture with deterministic components, strategically externalizing critical logic from LLMs to improve transparency and reliability. Delivers granular, modular outputs for surgical modifications.

Result: Evaluated across 4 diverse datasets, demonstrating strong generalizability, narrative quality, and computational efficiency with minimal dependencies.

Conclusion: The system successfully automates data analysis workflow and generates coherent visual narratives while supporting sustainable human-AI collaboration through transparent and reliable architecture.

Abstract: Recent advancements in the field of AI agents have impacted the way we work, enabling greater automation and collaboration between humans and agents. In the data visualization field, multi-agent systems can be useful for employing agents throughout the entire data-to-communication pipeline. We present a lightweight multi-agent system that automates the data analysis workflow, from data exploration to generating coherent visual narratives for insight communication. Our approach combines a hybrid multi-agent architecture with deterministic components, strategically externalizing critical logic from LLMs to improve transparency and reliability. The system delivers granular, modular outputs that enable surgical modifications without full regeneration, supporting sustainable human-AI collaboration. We evaluated our system across 4 diverse datasets, demonstrating strong generalizability, narrative quality, and computational efficiency with minimal dependencies.

Method: Architects a decoupled control plane with MCP server providing Workload Analysis Engine, Scheduler Policy Repository, and Execution Verifier. Uses multi-agent system to analyze workloads and synthesize custom eBPF scheduling policies.

Result: Achieves up to 1.79x performance improvement and 13x cost reduction compared to naive agentic approaches while maintaining high success rate.

Conclusion: Bridges the semantic gap, democratizes expert-level system optimization, and represents a step towards creating truly self-optimizing, application-aware operating systems.

Abstract: Operating system schedulers suffer from a fundamental semantic gap, where kernel policies fail to understand application-specific needs, leading to suboptimal performance. We introduce SchedCP, the first framework that enables fully autonomous Large Language Model (LLM) agents to safely and efficiently optimize Linux schedulers without human involvement. Our core insight is that the challenge is not merely to apply a better LLM, but to architect a decoupled control plane that separates the AI’s role of semantic reasoning (“what to optimize”) from the system’s role of execution (“how to observe and act”). Implemented as Model Context Protocol(MCP) server, SchedCP provides a stable interface with three key services: a Workload Analysis Engine, an evolving Scheduler Policy Repository, and an Execution Verifier that validates all AI-generated code and configure before deployment with static and dynamic analysis. We demonstrate this architecture’s power with sched-agent, a multi-agent system that autonomously analyzes workloads, synthesizes custom eBPF scheduling policies, and deploys them via the sched_ext infrastructure. Our evaluation shows that SchedCP achieves up to an 1.79x performance improvement, and a 13x cost reduction compared to naive agentic approaches, all while maintaining high success rate. By bridging the semantic gap, SchedCP democratizes expert-level system optimization and represents a step towards creating truly self-optimizing, application-aware operating systems. The code is open-sourced in https://github.com/eunomia-bpf/schedcp

Method: Prepared ice cream samples with each additive, conducted melting tests under controlled conditions, used timelapse recordings to track melting progression, and employed Python and OpenCV for image processing and analysis.

Result: All samples maintained foam-like structure after melting, indicating stable air-cell matrix formation. Re-frozen and re-melted samples showed increased sturdiness. Different stabilizers varied in effectiveness, with some providing stronger melting resistance and structural support than others.

Conclusion: The study provides insights into functional roles of food additives in ice cream, demonstrating potential for developing recipes that balance durability with cost efficiency for both small-scale and commercial production.

Abstract: The stability of ice cream during melting is a critical factor for consumer’s acceptance and product quality. With the commonly added stabilizer to improve texture, structure and slower melting as the factors to analyze. This report explores the effects of locust bean gum, guar gum, maltodextrin, and carrageenan on the melting behavior of homemade ice cream. The main objective was to assess how these additives influence melting resistance and to identify a more cost-effective recipe formulation. Ice cream samples incorporating each additive were prepared and subjected to melting tests under controlled conditions. Timelapse recordings were used to capture and analyze the progression of melting over time. Python and OpenCV is used for process and analysis. Observations revealed that all samples retained a foam-like structure even after melting, suggesting the stabilizers contributed to the formation of a stable air-cell matrix. Furthermore, when the melted samples were re-frozen and subsequently melted again, they displayed increased sturdiness, indicating improved resilience of the ice cream structure. Comparative analysis of the different stabilizers highlighted variations in their effectiveness, with some offering stronger melting resistance and structural support than others. Overall, the findings provide insights into the functional roles of commonly used food additives in ice cream formulation. By evaluating both performance and cost, this study demonstrates the potential for developing recipes that balance durability with economic efficiency, contributing to practical applications in both small-scale and commercial ice cream production.

Method: Proposes a scenario generator design method with three coordinated LLM agents: Environment Agent (EA) generates social environments including extremes, Social Agent (SA) generates social collaboration structures, and Planner Agent (PA) couples task-role relationships and plans solutions while adjusting schemes in real-time.

Result: Experiments on ProgrammableWeb dataset show the method generates more accurate scenarios more efficiently compared to traditional approaches.

Conclusion: The method provides an innovative and effective approach for constructing experimental systems for service ecosystem governance, enabling better scenario generation through adaptive coordination of multiple AI agents.

Abstract: As the social environment is growing more complex and collaboration is deepening, factors affecting the healthy development of service ecosystem are constantly changing and diverse, making its governance a crucial research issue. Applying the scenario analysis method and conducting scenario rehearsals by constructing an experimental system before managers make decisions, losses caused by wrong decisions can be largely avoided. However, it relies on predefined rules to construct scenarios and faces challenges such as limited information, a large number of influencing factors, and the difficulty of measuring social elements. These challenges limit the quality and efficiency of generating social and uncertain scenarios for the service ecosystem. Therefore, we propose a scenario generator design method, which adaptively coordinates three Large Language Model (LLM) empowered agents that autonomously optimize experimental schemes to construct an experimental system and generate high quality scenarios. Specifically, the Environment Agent (EA) generates social environment including extremes, the Social Agent (SA) generates social collaboration structure, and the Planner Agent (PA) couples task-role relationships and plans task solutions. These agents work in coordination, with the PA adjusting the experimental scheme in real time by perceiving the states of each agent and these generating scenarios. Experiments on the ProgrammableWeb dataset illustrate our method generates more accurate scenarios more efficiently, and innovatively provides an effective way for service ecosystem governance related experimental system construction.

Method: Four-stage approach: 1) Subclass Brain formation through personalized user-LLM interactions, 2) GA-assisted forward-backward evolution for prompt refinement, 3) Swarm Intelligence coordination of multiple Subclass Brains, 4) Integration into Superclass Brain meta-intelligence.

Result: Initial implementations demonstrated in UAV scheduling and keyword filtering tasks, with proposed registry for cross-dyad knowledge consolidation.

Conclusion: Provides conceptual foundation and architectural roadmap for scalable, explainable, and ethically aligned collective AI systems through human-LLM co-evolution.

Abstract: We propose a novel SuperBrain framework for collective intelligence, grounded in the co-evolution of large language models (LLMs) and human users. Unlike static prompt engineering or isolated agent simulations, our approach emphasizes a dynamic pathway from Subclass Brain to Superclass Brain: (1) A Subclass Brain arises from persistent, personalized interaction between a user and an LLM, forming a cognitive dyad with adaptive learning memory. (2) Through GA-assisted forward-backward evolution, these dyads iteratively refine prompts and task performance. (3) Multiple Subclass Brains coordinate via Swarm Intelligence, optimizing across multi-objective fitness landscapes and exchanging distilled heuristics. (4) Their standardized behaviors and cognitive signatures integrate into a Superclass Brain, an emergent meta-intelligence capable of abstraction, generalization and self-improvement. We outline the theoretical constructs, present initial implementations (e.g., UAV scheduling, KU/KI keyword filtering) and propose a registry for cross-dyad knowledge consolidation. This work provides both a conceptual foundation and an architectural roadmap toward scalable, explainable and ethically aligned collective AI.

Method: Combines prompt engineering with LLMs, furniture placement refinement algorithms, and Python scripting to interpret text inputs and generate spatially coherent layouts including walls, doors, windows, and furniture arrangements.

Result: Successfully generates functional residential layouts that preserve Revit-native parametric attributes for direct BIM integration, demonstrated through a case study of a mid-sized residential layout.

Conclusion: The workflow provides an efficient method for automated architectural drafting with transparent replication capabilities, enabling other researchers to implement similar systems and integrate directly into professional BIM workflows.

Abstract: This paper presents the development of an AI-powered workflow that uses Large Language Models (LLMs) to assist in drafting schematic architectural floor plans from natural language prompts. The proposed system interprets textual input to automatically generate layout options including walls, doors, windows, and furniture arrangements. It combines prompt engineering, a furniture placement refinement algorithm, and Python scripting to produce spatially coherent draft plans compatible with design tools such as Autodesk Revit. A case study of a mid-sized residential layout demonstrates the approach’s ability to generate functional and structured outputs with minimal manual effort. The workflow is designed for transparent replication, with all key prompt specifications documented to enable independent implementation by other researchers. In addition, the generated models preserve the full range of Revit-native parametric attributes required for direct integration into professional BIM processes.

Method: A POMDP-driven design that represents social interactions as structured tuples (state, observation, agent actions, mental states) which can be automatically induced from free-form narratives or other inputs.

Result: +51% improvement on FANToM’s theory-of-mind reasoning with OpenAI’s o1, reaching new SOTA performance; +18% improvement on SOTOPIA social interaction benchmark; ability to predict future social dynamics and improve agent decision-making.

Conclusion: S3AP shows promise as a powerful, general-purpose representation for social world states, enabling development of more socially-aware systems that better navigate social interactions.

Abstract: Humans intuitively navigate social interactions by simulating unspoken dynamics and reasoning about others’ perspectives, even with limited information. In contrast, AI systems struggle to automatically structure and reason about these implicit social contexts. In this paper, we introduce a novel structured social world representation formalism (S3AP), designed to help AI systems reason more effectively about social dynamics. Following a POMDP-driven design, S3AP represents social interactions as structured tuples, such as state, observation, agent actions, and mental states, which can be automatically induced from free-form narratives or other inputs. We first show S3AP can help LLMs better understand social narratives across 5 social reasoning tasks (e.g., +51% improvement on FANToM’s theory-of-mind reasoning with OpenAI’s o1), reaching new state-of-the-art (SOTA) performance. We then induce social world models from these structured representations, demonstrating their ability to predict future social dynamics and improve agent decision-making, yielding up to +18% improvement on the SOTOPIA social interaction benchmark. Our findings highlight the promise of S3AP as a powerful, general-purpose representation for social world states, enabling the development of more socially-aware systems that better navigate social interactions.

Method: Quantitative comparison using Initiation-Response-Feedback (IRF) coding scheme and Epistemic Network Analysis (ENA) to analyze dialogue patterns.

Result: Human dialogues significantly outperform AI in utterance length, questioning behaviors, and general feedback. Human interactions are more cognitively guided and diverse with clear pedagogical guidance, while AI shows structural simplification and behavioral convergence in simple information transfer patterns.

Conclusion: Current AI-generated tutoring has key limitations in pedagogical effectiveness, providing empirical guidance for designing better educational dialogue systems that can replicate human teaching patterns.

Abstract: Heuristic and scaffolded teacher-student dialogues are widely regarded as critical for fostering students’ higher-order thinking and deep learning. However, large language models (LLMs) currently face challenges in generating pedagogically rich interactions. This study systematically investigates the structural and behavioral differences between AI-simulated and authentic human tutoring dialogues. We conducted a quantitative comparison using an Initiation-Response-Feedback (IRF) coding scheme and Epistemic Network Analysis (ENA). The results show that human dialogues are significantly superior to their AI counterparts in utterance length, as well as in questioning (I-Q) and general feedback (F-F) behaviors. More importantly, ENA results reveal a fundamental divergence in interactional patterns: human dialogues are more cognitively guided and diverse, centered around a “question-factual response-feedback” teaching loop that clearly reflects pedagogical guidance and student-driven thinking; in contrast, simulated dialogues exhibit a pattern of structural simplification and behavioral convergence, revolving around an “explanation-simplistic response” loop that is essentially a simple information transfer between the teacher and student. These findings illuminate key limitations in current AI-generated tutoring and provide empirical guidance for designing and evaluating more pedagogically effective generative educational dialogue systems.

Method: Processes raw time series through a time series encoder, feeds descriptive statistics to an LLM with learnable prompts, uses scaling strategy and contrastive objective to align textual embeddings with time series embeddings, then integrates both for forecasting.

Result: Achieves state-of-the-art performance in both long-term and few-shot forecasting with minimal trainable parameters, demonstrating effective complementary information utilization.

Conclusion: BALM-TSF successfully addresses modality imbalance and harnesses the complementary strengths of both text and time series data for superior forecasting performance.

Abstract: Time series forecasting is a long-standing and highly challenging research topic. Recently, driven by the rise of large language models (LLMs), research has increasingly shifted from purely time series methods toward harnessing textual modalities to enhance forecasting performance. However, the vast discrepancy between text and temporal data often leads current multimodal architectures to over-emphasise one modality while neglecting the other, resulting in information loss that harms forecasting performance. To address this modality imbalance, we introduce BALM-TSF (Balanced Multimodal Alignment for LLM-Based Time Series Forecasting), a lightweight time series forecasting framework that maintains balance between the two modalities. Specifically, raw time series are processed by the time series encoder, while descriptive statistics of raw time series are fed to an LLM with learnable prompt, producing compact textual embeddings. To ensure balanced cross-modal context alignment of time series and textual embeddings, a simple yet effective scaling strategy combined with a contrastive objective then maps these textual embeddings into the latent space of the time series embeddings. Finally, the aligned textual semantic embeddings and time series embeddings are together integrated for forecasting. Extensive experiments on standard benchmarks show that, with minimal trainable parameters, BALM-TSF achieves state-of-the-art performance in both long-term and few-shot forecasting, confirming its ability to harness complementary information from text and time series. Code is available at https://github.com/ShiqiaoZhou/BALM-TSF.

Method: Dynamic Speculative Planning (DSP) - an asynchronous online reinforcement learning framework that optimizes a joint objective balancing end-to-end latency against dollar cost without requiring additional pre-deployment preparation.

Result: Experiments on two standard agent benchmarks show DSP achieves comparable efficiency to the fastest lossless acceleration method while reducing total cost by 30% and unnecessary cost by up to 60%.

Conclusion: DSP provides a flexible, cost-effective solution for accelerating LLM agents with lossless performance, allowing practitioners to control the tradeoff between response speed and operational costs through a single parameter.

Abstract: Despite their remarkable success in complex tasks propelling widespread adoption, large language-model-based agents still face critical deployment challenges due to prohibitive latency and inference costs. While recent work has explored various methods to accelerate inference, existing approaches suffer from significant limitations: they either fail to preserve performance fidelity, require extensive offline training of router modules, or incur excessive operational costs. Moreover, they provide minimal user control over the tradeoff between acceleration and other performance metrics. To address these gaps, we introduce Dynamic Speculative Planning (DSP), an asynchronous online reinforcement learning framework that provides lossless acceleration with substantially reduced costs without requiring additional pre-deployment preparation. DSP explicitly optimizes a joint objective balancing end-to-end latency against dollar cost, allowing practitioners to adjust a single parameter that steers the system toward faster responses, cheaper operation, or any point along this continuum. Experiments on two standard agent benchmarks demonstrate that DSP achieves comparable efficiency to the fastest lossless acceleration method while reducing total cost by 30% and unnecessary cost up to 60%. Our code and data are available through https://github.com/guanyilin428/Dynamic-Speculative-Planning.

Method: Users specify workflows as natural-language rules that define state-dependent actions, which are compiled into nondeterministic finite automata (NFAs) and translated into reusable executable code with state synthesis and caching.

Result: NetGent automated 50+ workflows across various domains (video streaming, conferencing, social media, web scraping), producing realistic traffic traces while remaining robust to UI changes and enabling deterministic replay.

Conclusion: NetGent combines language-based agent flexibility with compiled execution reliability, providing a scalable foundation for generating diverse, repeatable network traffic datasets to advance ML in networking.

Abstract: We present NetGent, an AI-agent framework for automating complex application workflows to generate realistic network traffic datasets. Developing generalizable ML models for networking requires data collection from network environments with traffic that results from a diverse set of real-world web applications. However, using existing browser automation tools that are diverse, repeatable, realistic, and efficient remains fragile and costly. NetGent addresses this challenge by allowing users to specify workflows as natural-language rules that define state-dependent actions. These abstract specifications are compiled into nondeterministic finite automata (NFAs), which a state synthesis component translates into reusable, executable code. This design enables deterministic replay, reduces redundant LLM calls through state caching, and adapts quickly when application interfaces change. In experiments, NetGent automated more than 50+ workflows spanning video-on-demand streaming, live video streaming, video conferencing, social media, and web scraping, producing realistic traffic traces while remaining robust to UI variability. By combining the flexibility of language-based agents with the reliability of compiled execution, NetGent provides a scalable foundation for generating the diverse, repeatable datasets needed to advance ML in networking.

Method: Two-stage framework: 1) Specialized agents extract legal concepts and formalize rules into verifiable intermediate representations 2) Three-step application: query analysis, symbolic inference, and programmatic answer generation bridging natural language with symbolic reasoning.

Result: Substantial improvement on statutory tax calculation tasks - foundational models achieved 76.4% accuracy compared to 18.8% baseline performance, effectively narrowing the performance gap.

Conclusion: Modular architectures with formalized knowledge representations make legal reasoning more accessible through computationally efficient models while enhancing consistency and explainability, establishing foundation for transparent and trustworthy AI legal systems.

Abstract: Legal reasoning requires both precise interpretation of statutory language and consistent application of complex rules, presenting significant challenges for AI systems. This paper introduces a modular multi-agent framework that decomposes legal reasoning into distinct knowledge acquisition and application stages. In the first stage, specialized agents extract legal concepts and formalize rules to create verifiable intermediate representations of statutes. The second stage applies this knowledge to specific cases through three steps: analyzing queries to map case facts onto the ontology schema, performing symbolic inference to derive logically entailed conclusions, and generating final answers using a programmatic implementation that operationalizes the ontological knowledge. This bridging of natural language understanding with symbolic reasoning provides explicit and verifiable inspection points, significantly enhancing transparency compared to end-to-end approaches. Evaluation on statutory tax calculation tasks demonstrates substantial improvements, with foundational models achieving 76.4% accuracy compared to 18.8% baseline performance, effectively narrowing the performance gap between reasoning and foundational models. These findings suggest that modular architectures with formalized knowledge representations can make sophisticated legal reasoning more accessible through computationally efficient models while enhancing consistency and explainability in AI legal reasoning, establishing a foundation for future research into more transparent, trustworthy, and effective AI systems for legal domain.

Method: Systematically inject structured context directly into the input to guide LLMs, enabling implicit alignment of tasks with grounded conceptual spaces without requiring model fine-tuning.

Result: Consistent performance improvements across multiple graph tasks using both lightweight and large models, with structured input context rivaling or surpassing more complex approaches.

Conclusion: Structured context injection is an effective and scalable strategy for graph understanding with LLMs, offering a practical alternative to expensive fine-tuning methods.

Abstract: Large Language Models (LLMs) have shown strong capabilities in solving problems across domains, including graph-related tasks traditionally addressed by symbolic or algorithmic methods. In this work, we present a framework for structured context injection, where task-specific information is systematically embedded in the input to guide LLMs in solving a wide range of graph problems. Our method does not require fine-tuning of LLMs, making it cost-efficient and lightweight. We observe that certain graph reasoning tasks remain challenging for LLMs unless they are mapped to conceptually grounded representations. However, achieving such mappings through fine-tuning or repeated multi-step querying can be expensive and inefficient. Our approach offers a practical alternative by injecting structured context directly into the input, enabling the LLM to implicitly align the task with grounded conceptual spaces. We evaluate the approach on multiple graph tasks using both lightweight and large models, highlighting the trade-offs between accuracy and computational cost. The results demonstrate consistent performance improvements, showing that structured input context can rival or surpass more complex approaches. Our findings underscore the value of structured context injection as an effective and scalable strategy for graph understanding with LLMs.

Method: Decomposes queries into subproblems, performs targeted searches across heterogeneous sources (web, local RAG, case law), and uses a Judge Agent for verification of sufficiency, jurisdiction, and temporal validity in an iterative reasoning-search-verification loop.

Result: Substantially improves factual accuracy, reduces uncertainty, and achieves higher preference scores from both human experts and LLM-based judges on the LegalSearchQA benchmark of 200 up-to-date legal questions.

Conclusion: Multi-agent reasoning with agentic search provides a scalable and reproducible blueprint for deploying LLMs in high-stakes domains requiring precise legal retrieval and deliberation.

Abstract: We present L-MARS (Legal Multi-Agent Workflow with Orchestrated Reasoning and Agentic Search), a system that reduces hallucination and uncertainty in legal question answering through coordinated multi-agent reasoning and retrieval. Unlike single-pass retrieval-augmented generation (RAG), L-MARS decomposes queries into subproblems, issues targeted searches across heterogeneous sources (Serper web, local RAG, CourtListener case law), and employs a Judge Agent to verify sufficiency, jurisdiction, and temporal validity before answer synthesis. This iterative reasoning-search-verification loop maintains coherence, filters noisy evidence, and grounds answers in authoritative law. We evaluated L-MARS on LegalSearchQA, a new benchmark of 200 up-to-date multiple choice legal questions in 2025. Results show that L-MARS substantially improves factual accuracy, reduces uncertainty, and achieves higher preference scores from both human experts and LLM-based judges. Our work demonstrates that multi-agent reasoning with agentic search offers a scalable and reproducible blueprint for deploying LLMs in high-stakes domains requiring precise legal retrieval and deliberation.

Method: Introduces Physics-aware Rejection Sampling (PaRS) that selects training traces based on consistency with fundamental physics and numerical closeness to targets, using a teacher-student model framework.

Result: Improves accuracy and calibration, reduces physics-violation rates, and lowers sampling cost compared to baseline rejection sampling methods.

Conclusion: Domain-aware constraints with trace-level selection provide a practical path toward reliable, efficient large reasoning models for process-aware property prediction and materials design.

Abstract: AI-driven materials discovery that couples automated experimentation with algorithmic decision-making requires process aware recipe to property predictors that are accurate, calibrated, and physically admissible. We approach this as a reasoning problem with large reasoning models (LRMs). To instill reasoning capability into language models, we curate reasoning traces from a teacher model to train a student model. However, most training pipelines select reasoning traces using binary correctness or learned preference signals that poorly reflect physical admissibility. We introduce Physics-aware Rejection Sampling (PaRS), a training-time trace selection scheme that favors traces consistent with fundamental physics and numerically close to targets, with lightweight halting to control compute. We instantiate our framework with a large student model fine-tuned on traces synthesized by a larger teacher model, and evaluate under matched token budgets against various rejection sampling baselines. Our method improves accuracy and calibration, reduces physics-violation rates, and lowers sampling cost relative to baselines. These results indicate that modest, domain-aware constraints combined with trace-level selection provide a practical path toward reliable, efficient LRMs for process-aware property prediction and closed-loop materials design.

Method: Uses Dinkelbach’s transform to convert Sharpe ratio to mean-squared-variance objective. Develops iterative algorithms that solve M2V problems and update risk-sensitive parameters, proving convergence for both average and discounted MDP settings with policy iteration procedures.

Result: The proposed algorithm produces monotonically increasing Sharpe ratios that converge to the optimal value. Numerical experiments validate the approach, showing it’s the first dynamic programming type solution for Sharpe ratio optimization in MDPs.

Conclusion: The method successfully addresses both challenges of fractional objectives and risk metrics in MDPs, providing a convergent dynamic programming framework that can potentially be extended to other fractional objective problems in reinforcement learning.

Abstract: Sharpe ratio (also known as reward-to-variability ratio) is a widely-used metric in finance, which measures the additional return at the cost of per unit of increased risk (standard deviation of return). However, the optimization of Sharpe ratio in Markov decision processes (MDPs) is challenging, because there exist two difficulties hindering the application of dynamic programming. One is that dynamic programming does not work for fractional objectives, and the other is that dynamic programming is invalid for risk metrics. In this paper, we study the Sharpe ratio optimization in infinite-horizon MDPs, considering both the long-run average and discounted settings. We address the first challenge with the Dinkelbachs transform, which converts the Sharpe ratio objective to a mean-squared-variance (M2V) objective. It is shown that the M2V optimization and the original Sharpe ratio optimization share the same optimal policy when the risk-sensitive parameter is equal to the optimal Sharpe ratio. For the second challenge, we develop an iterative algorithm to solve the M2V optimization which is similar to a mean-variance optimization in MDPs. We iteratively solve the M2V problem and obtain the associated Sharpe ratio that is used to update the risk-sensitive parameter in the next iteration of M2V problems. We show that such a sequence of Sharpe ratios derived is monotonically increasing and converges to the optimal Sharpe ratio. For both average and discounted MDP settings, we develop a policy iteration procedure and prove its convergence to the optimum. Numerical experiments are conducted for validation. To the best of our knowledge, our approach is the first that solves the Sharpe ratio optimization in MDPs with dynamic programming type algorithms. We believe that the proposed algorithm can shed light on solving MDPs with other fractional objectives.

Method: Integrates Linear Temporal Logic over finite traces (LTLf) into autoregressive sequence predictors using differentiable logical loss function with Gumbel-Softmax trick and soft LTLf approximation.

Result: Experimental evaluation on three real-world datasets shows improved suffix prediction accuracy and compliance with temporal constraints.

Conclusion: Framework improves Neuro-Symbolic AI for sequence generation tasks beyond BPM, with effective local and global logic loss variants for noisy settings.

Abstract: This paper addresses the problem of suffix prediction in Business Process Management (BPM) by proposing a Neuro-Symbolic Predictive Process Monitoring (PPM) approach that integrates data-driven learning with temporal logic-based prior knowledge. While recent approaches leverage deep learning models for suffix prediction, they often fail to satisfy even basic logical constraints due to the absence of explicit integration of domain knowledge during training. We propose a novel method to incorporate Linear Temporal Logic over finite traces (LTLf) into the training process of autoregressive sequence predictors. Our approach introduces a differentiable logical loss function, defined using a soft approximation of LTLf semantics and the Gumbel-Softmax trick, which can be combined with standard predictive losses. This ensures the model learns to generate suffixes that are both accurate and logically consistent. Experimental evaluation on three real-world datasets shows that our method improves suffix prediction accuracy and compliance with temporal constraints. We also introduce two variants of the logic loss (local and global) and demonstrate their effectiveness under noisy and realistic settings. While developed in the context of BPM, our framework is applicable to any symbolic sequence generation task and contributes toward advancing Neuro-Symbolic AI.

Method: The framework includes a library of expert-validated virtual patients with clinically grounded profiles, simulates multi-turn interactions with nurse agents using evidence-based persuasive strategies, and supports longitudinal counseling and adversarial social influence scenarios.

Result: Larger and more reflective LLMs adapt strategies over time, but all models struggle to overcome resistance, particularly under realistic social pressure scenarios.

Conclusion: Current LLMs have critical limitations for behavior change applications, and ChatCLIDS provides a high-fidelity, scalable testbed for advancing trustworthy persuasive AI in healthcare and other domains.

Abstract: Real-world adoption of closed-loop insulin delivery systems (CLIDS) in type 1 diabetes remains low, driven not by technical failure, but by diverse behavioral, psychosocial, and social barriers. We introduce ChatCLIDS, the first benchmark to rigorously evaluate LLM-driven persuasive dialogue for health behavior change. Our framework features a library of expert-validated virtual patients, each with clinically grounded, heterogeneous profiles and realistic adoption barriers, and simulates multi-turn interactions with nurse agents equipped with a diverse set of evidence-based persuasive strategies. ChatCLIDS uniquely supports longitudinal counseling and adversarial social influence scenarios, enabling robust, multi-dimensional evaluation. Our findings reveal that while larger and more reflective LLMs adapt strategies over time, all models struggle to overcome resistance, especially under realistic social pressure. These results highlight critical limitations of current LLMs for behavior change, and offer a high-fidelity, scalable testbed for advancing trustworthy persuasive AI in healthcare and beyond.

Method: Proposed Robust Deep MCCFR framework with target networks, uniform exploration mixing, variance-aware training objectives, and diagnostic monitoring. Systematic ablation studies on Kuhn and Leduc Poker.

Result: 60% improvement on Kuhn Poker (0.0628 vs 0.156 exploitability) and 23.5% improvement on Leduc Poker (0.2386 vs 0.3703). Identified scale-dependent component effectiveness and critical interactions.

Conclusion: Careful component selection is more important than comprehensive mitigation. Framework provides convergence guarantees and practical guidelines for larger games with scale-dependent risk patterns.

Abstract: Monte Carlo Counterfactual Regret Minimization (MCCFR) has emerged as a cornerstone algorithm for solving extensive-form games, but its integration with deep neural networks introduces scale-dependent challenges that manifest differently across game complexities. This paper presents a comprehensive analysis of how neural MCCFR component effectiveness varies with game scale and proposes an adaptive framework for selective component deployment. We identify that theoretical risks such as nonstationary target distribution shifts, action support collapse, variance explosion, and warm-starting bias have scale-dependent manifestation patterns, requiring different mitigation strategies for small versus large games. Our proposed Robust Deep MCCFR framework incorporates target networks with delayed updates, uniform exploration mixing, variance-aware training objectives, and comprehensive diagnostic monitoring. Through systematic ablation studies on Kuhn and Leduc Poker, we demonstrate scale-dependent component effectiveness and identify critical component interactions. The best configuration achieves final exploitability of 0.0628 on Kuhn Poker, representing a 60% improvement over the classical framework (0.156). On the more complex Leduc Poker domain, selective component usage achieves exploitability of 0.2386, a 23.5% improvement over the classical framework (0.3703) and highlighting the importance of careful component selection over comprehensive mitigation. Our contributions include: (1) a formal theoretical analysis of risks in neural MCCFR, (2) a principled mitigation framework with convergence guarantees, (3) comprehensive multi-scale experimental validation revealing scale-dependent component interactions, and (4) practical guidelines for deployment in larger games.

Method: Generates Satisfiability-based logical reasoning problems directly from CNF instances using randomized SAT-based problem generation and objective answer verification via PySAT.

Result: Evaluation revealed significant limitations in LLM logical reasoning, particularly in generalization beyond familiar mathematical formats. Reinforcement fine-tuning with SATQuest rewards substantially improved targeted task performance and generalized to more complex instances.

Conclusion: SATQuest demonstrates potential as a foundational tool for advancing LLM logical reasoning, though challenges remain in cross-format adaptation.

Abstract: Recent advances in Large Language Models (LLMs) have demonstrated remarkable general reasoning capabilities. However, systematically evaluating and enhancing these reasoning capabilities is challenging due to the lack of controllable and scalable tools for fine-grained analysis. Existing benchmarks and datasets often lack the necessary variable control for multi-dimensional, systematic analysis and training, or have narrow problem types and formats. To address these limitations, we introduce SATQuest, a systematic verifier designed to evaluate and enhance logical reasoning in LLMs by generating diverse, Satisfiability-based logical reasoning problems directly from Conjunctive Normal Form (CNF) instances. SATQuest structures these problems along three orthogonal dimensions: instance scale, problem type, and question format, employing randomized, SAT-based problem generation and objective answer verification via PySAT. This design mitigates memorization issues, allows for nuanced insights into reasoning performance, and enables effective reinforcement fine-tuning. Our extensive evaluation of various LLMs using SATQuest identified significant limitations in their logical reasoning, particularly in generalizing beyond familiar mathematical formats. Furthermore, we show that reinforcement fine-tuning with SATQuest rewards substantially improves targeted task performance and generalizes to more complex instances, while highlighting remaining challenges in cross-format adaptation. Through these demonstrations, we showcase SATQuest’s potential as a foundational tool and a valuable starting point for advancing LLM logical reasoning.

Method: Proposes Reinforced Advantage feedback (ReAd) framework that performs critic regression to learn a sequential advantage function from LLM-planned data, then treats the LLM planner as an optimizer to generate actions maximizing the advantage function.

Result: Experiments on Overcooked-AI and RoCoBench show ReAd surpasses baselines in success rate while significantly decreasing agent interaction steps and LLM query rounds.

Conclusion: ReAd provides an efficient framework for grounding LLMs in multi-agent collaboration by endowing LLMs with foresight to discern action contributions to final task completion, with theoretical foundation in advantage-weighted regression.

Abstract: Grounding the reasoning ability of large language models (LLMs) for embodied tasks is challenging due to the complexity of the physical world. Especially, LLM planning for multi-agent collaboration requires communication of agents or credit assignment as the feedback to re-adjust the proposed plans and achieve effective coordination. However, existing methods that overly rely on physical verification or self-reflection suffer from excessive and inefficient querying of LLMs. In this paper, we propose a novel framework for multi-agent collaboration that introduces Reinforced Advantage feedback (ReAd) for efficient self-refinement of plans. Specifically, we perform critic regression to learn a sequential advantage function from LLM-planned data, and then treat the LLM planner as an optimizer to generate actions that maximize the advantage function. It endows the LLM with the foresight to discern whether the action contributes to accomplishing the final task. We provide theoretical analysis by extending advantage-weighted regression in reinforcement learning to multi-agent systems. Experiments on Overcooked-AI and a difficult variant of RoCoBench show that ReAd surpasses baselines in success rate, and also significantly decreases the interaction steps of agents and query rounds of LLMs, demonstrating its high efficiency for grounding LLMs. More results are given at https://read-llm.github.io.

Method: Blends physics-informed machine learning (for real-world constraint adherence), multimodal data fusion via knowledge graphs (semantic integration), and adaptive rule-based intelligence using LLMs for real-time filtering and decision-making at the edge.

Result: Creates a foundation for digital twin systems that move beyond passive monitoring to provide actionable insights, enabling efficient operation even under constrained resources.

Conclusion: This unified approach of physics-based reasoning, semantic data fusion, and adaptive rule generation enables responsive, trustworthy, and sustainable smart infrastructure development.

Abstract: Cities today generate enormous streams of data from sensors, cameras, and connected infrastructure. While this information offers unprecedented opportunities to improve urban life, most existing systems struggle with scale, latency, and fragmented insights. This work introduces a framework that blends physics-informed machine learning, multimodal data fusion, and knowledge graph representation with adaptive, rule-based intelligence powered by large language models (LLMs). Physics-informed methods ground learning in real-world constraints, ensuring predictions remain meaningful and consistent with physical dynamics. Knowledge graphs act as the semantic backbone, integrating heterogeneous sensor data into a connected, queryable structure. At the edge, LLMs generate context-aware rules that adapt filtering and decision-making in real time, enabling efficient operation even under constrained resources. Together, these elements form a foundation for digital twin systems that go beyond passive monitoring to provide actionable insights. By uniting physics-based reasoning, semantic data fusion, and adaptive rule generation, this approach opens new possibilities for creating responsive, trustworthy, and sustainable smart infrastructures.

Method: Multi-stage hybrid intelligence framework combining: 1) Learning to Rank model evaluating patents against 30+ legal/commercial parameters, 2) Need Agent using NLP to mine market needs from unstructured data, 3) Seed Agent using fine-tuned LLMs to analyze patent claims, and 4) Core Ontology Framework matching patents to market demands with HITL validation.

Result: The framework automates and deepens patent portfolio analysis, generating strategic rationale for divestment decisions by systematically matching high-potential patents to documented market demands.

Conclusion: The proposed hybrid intelligence framework provides an automated, adaptable, and credible solution for identifying high-value patent assets for technology transfer, overcoming limitations of traditional manual methods.

Abstract: This paper introduces a novel, multi stage hybrid intelligence framework for pruning patent portfolios to identify high value assets for technology transfer. Current patent valuation methods often rely on retrospective indicators or manual, time intensive analysis. Our framework automates and deepens this process by combining a Learning to Rank (LTR) model, which evaluates patents against over 30 legal and commercial parameters, with a unique “Need-Seed” agent-based system. The “Need Agent” uses Natural Language Processing (NLP) to mine unstructured market and industry data, identifying explicit technological needs. Concurrently, the “Seed Agent” employs fine tuned Large Language Models (LLMs) to analyze patent claims and map their technological capabilities. The system generates a “Core Ontology Framework” that matches high potential patents (Seeds) to documented market demands (Needs), providing a strategic rationale for divestment decisions. We detail the architecture, including a dynamic parameter weighting system and a crucial Human in the-Loop (HITL) validation protocol, to ensure both adaptability and real-world credibility.

Method: Combines symbolic program synthesis with large language models (LLMs) to generate explanations, evaluated through systematic testing with multiple LLM judges and human validation, plus human learning experiments across three domains.

Result: LENS generates superior explanations compared to direct LLM prompting and hand-crafted templates, but human learning experiments showed no significant performance improvements, suggesting comprehensive LLM responses may overwhelm users for simpler problems.

Conclusion: Provides a foundation for building effective USML systems to support human learning, though current approach may need refinement for optimal teaching effectiveness with human users.

Abstract: Ultra Strong Machine Learning (USML) refers to symbolic learning systems that not only improve their own performance but can also teach their acquired knowledge to quantifiably improve human performance. In this work, we present LENS (Logic Programming Explanation via Neural Summarisation), a neuro-symbolic method that combines symbolic program synthesis with large language models (LLMs) to automate the explanation of machine-learned logic programs in natural language. LENS addresses a key limitation of prior USML approaches by replacing hand-crafted explanation templates with scalable automated generation. Through systematic evaluation using multiple LLM judges and human validation, we demonstrate that LENS generates superior explanations compared to direct LLM prompting and hand-crafted templates. To investigate whether LENS can teach transferable active learning strategies, we carried out a human learning experiment across three related domains. Our results show no significant human performance improvements, suggesting that comprehensive LLM responses may overwhelm users for simpler problems rather than providing learning support. Our work provides a solid foundation for building effective USML systems to support human learning. The source code is available on: https://github.com/lun-ai/LENS.git.

Method: General Symbolics reasoning approach structured around three key use cases: tool-calling, code generation, and planning. The method operates without any finetuning or training costs.

Result: Achieved SOTA scores: 66.66% on Livecodebench v6, 89% on Instruction-Following Evals, 24.4% on ARC-AGI-2, and 62.3% on SWE-Bench Lite through an agentic coding IDE.

Conclusion: CoreThink’s General Symbolics provides a pure performance uplift for reasoning tasks and represents a necessary advancement beyond incumbent methods that face diminishing returns.

Abstract: We introduce CoreThink, a state-of-the-art Reasoning Layer built upon a novel reasoning method called General Symbolics. This approach diverges from reasoning paradigms such as test-time scaling, Supervised Fine-Tuning (SFT), and Reinforcement Learning with Verifiable Rewards (RLVR). CoreThink General Symbolic Reasoner (GSR) is specifically structured around three key use cases: tool-calling, code generation, and planning, demonstrating exemplary performance across a total of seven benchmarks in their respective areas. Notably, we are achieving SOTA scores of 66.66% on Livecodebench v6, 89% on Instruction-Following Evals, and 24.4% on ARC-AGI-2. We also present an agentic coding IDE, developed using the principles of General Symbolics, which achieves a state-of-the-art accuracy of 62.3% on \texttt{SWE-Bench Lite}. We are able to achieve these improvements without any finetuning or training costs. Our Reasoning Layer is designed to provide a pure performance uplift, ensuring that a model’s accuracy on reasoning tasks is never negatively impacted. We argue that incumbent methods will eventually lead to diminishing returns in LLM performance, necessitating the development of new reasoning techniques. This technical report details our approach at a high level and the availability of the CoreThink models for reasoning-intensive use cases.

Method: Reinforcement learning framework that fine-tunes LLMs using outcome-based rewards to encourage self-exploration of reasoning strategies from graph structure and production of explanations that justify predictions.

Result: ReaL-TG-4B (fine-tuned Qwen3-4B) outperforms much larger frontier LLMs including GPT-5 mini on ranking metrics, while producing high-quality explanations validated by both LLM judge and human evaluation.

Conclusion: The framework successfully enables LLMs to perform explainable temporal graph reasoning with strong performance and verifiable explanation quality, addressing key limitations of existing approaches.

Abstract: Forecasting future links is a central task in temporal graph (TG) reasoning, requiring models to leverage historical interactions to predict upcoming ones. Traditional neural approaches, such as temporal graph neural networks, achieve strong performance but lack explainability and cannot be applied to unseen graphs without retraining. Recent studies have begun to explore using large language models (LLMs) for graph reasoning, but most of them are constrained to static graphs or small synthetic TGs and lack the evaluation of the quality of reasoning traces generated by LLMs. In this work, we present Reasoning-Enhanced Learning for Temporal Graphs (ReaL-TG), a reinforcement learning framework that fine-tunes LLMs to perform explainable link forecasting on real-world TGs. ReaL-TG uses outcome-based reward to encourage models to self-explore reasoning strategies from graph structure and to produce explanations that directly justify their predictions. To enable evaluation on LLM-generated reasoning traces, we propose a new evaluation protocol combining ranking metrics with an LLM-as-a-Judge system that assesses both the quality of reasoning and the impact of hallucinations. Experiments with ReaL-TG-4B, obtained by fine-tuning Qwen3-4B under our framework, show that it outperforms much larger frontier LLMs, including GPT-5 mini, on ranking metrics, while producing high-quality explanations confirmed by both the LLM judge and human evaluation.

Method: Examines diverse architectural patterns including pipeline processing, debate frameworks, simulation environments, and iterative refinement loops using multiple LLM-based agents.

Result: Provides comprehensive overview of how multi-agent LLM systems are being applied to causal reasoning, discovery, and effect estimation across various domains.

Conclusion: Causal multi-agent LLMs represent a synergistic field with significant potential, though challenges remain that require further research and development.

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in various reasoning and generation tasks. However, their proficiency in complex causal reasoning, discovery, and estimation remains an area of active development, often hindered by issues like hallucination, reliance on spurious correlations, and difficulties in handling nuanced, domain-specific, or personalized causal relationships. Multi-agent systems, leveraging the collaborative or specialized abilities of multiple LLM-based agents, are emerging as a powerful paradigm to address these limitations. This review paper explores the burgeoning field of causal multi-agent LLMs. We examine how these systems are designed to tackle different facets of causality, including causal reasoning and counterfactual analysis, causal discovery from data, and the estimation of causal effects. We delve into the diverse architectural patterns and interaction protocols employed, from pipeline-based processing and debate frameworks to simulation environments and iterative refinement loops. Furthermore, we discuss the evaluation methodologies, benchmarks, and diverse application domains where causal multi-agent LLMs are making an impact, including scientific discovery, healthcare, fact-checking, and personalized systems. Finally, we highlight the persistent challenges, open research questions, and promising future directions in this synergistic field, aiming to provide a comprehensive overview of its current state and potential trajectory.

Method: Proposes a unified conceptual framework with four key components (System Inputs, Agent System, Environment, Optimisers) and systematically reviews self-evolving techniques targeting different agent system components across various domains.

Result: Comprehensive review of self-evolving agent techniques, domain-specific evolution strategies for biomedicine, programming, and finance, plus evaluation, safety, and ethical considerations.

Conclusion: Provides foundation for developing more adaptive, autonomous, and lifelong agentic systems by systematizing understanding of self-evolving AI agents and their implementation considerations.

Abstract: Recent advances in large language models have sparked growing interest in AI agents capable of solving complex, real-world tasks. However, most existing agent systems rely on manually crafted configurations that remain static after deployment, limiting their ability to adapt to dynamic and evolving environments. To this end, recent research has explored agent evolution techniques that aim to automatically enhance agent systems based on interaction data and environmental feedback. This emerging direction lays the foundation for self-evolving AI agents, which bridge the static capabilities of foundation models with the continuous adaptability required by lifelong agentic systems. In this survey, we provide a comprehensive review of existing techniques for self-evolving agentic systems. Specifically, we first introduce a unified conceptual framework that abstracts the feedback loop underlying the design of self-evolving agentic systems. The framework highlights four key components: System Inputs, Agent System, Environment, and Optimisers, serving as a foundation for understanding and comparing different strategies. Based on this framework, we systematically review a wide range of self-evolving techniques that target different components of the agent system. We also investigate domain-specific evolution strategies developed for specialised fields such as biomedicine, programming, and finance, where optimisation objectives are tightly coupled with domain constraints. In addition, we provide a dedicated discussion on the evaluation, safety, and ethical considerations for self-evolving agentic systems, which are critical to ensuring their effectiveness and reliability. This survey aims to provide researchers and practitioners with a systematic understanding of self-evolving AI agents, laying the foundation for the development of more adaptive, autonomous, and lifelong agentic systems.

Method: The paper identifies key characteristics of agentic speculation (scale, heterogeneity, redundancy, steerability) and proposes adapting data systems with new query interfaces, processing techniques, and agentic memory stores.

Result: The analysis reveals that current data systems are inadequate for handling LLM agent workloads and outlines research opportunities for agent-first data system architecture.

Conclusion: Data systems need fundamental architectural changes to natively support LLM agent workloads, with new research directions spanning query interfaces, processing techniques, and specialized memory stores for agentic operations.

Abstract: Large Language Model (LLM) agents, acting on their users’ behalf to manipulate and analyze data, are likely to become the dominant workload for data systems in the future. When working with data, agents employ a high-throughput process of exploration and solution formulation for the given task, one we call agentic speculation. The sheer volume and inefficiencies of agentic speculation can pose challenges for present-day data systems. We argue that data systems need to adapt to more natively support agentic workloads. We take advantage of the characteristics of agentic speculation that we identify, i.e., scale, heterogeneity, redundancy, and steerability - to outline a number of new research opportunities for a new agent-first data systems architecture, ranging from new query interfaces, to new query processing techniques, to new agentic memory stores.

Method: Compared LLM-based hypothesis search framework with direct program generation approaches on few-shot rule induction tasks, including error analysis of hypothesis generation bottlenecks.

Result: Hypothesis search achieved performance comparable to humans, while direct program generation fell notably behind in rule induction tasks.

Conclusion: LLM-based hypothesis search shows strong potential for modeling inductive reasoning, but current program induction methods face significant challenges that need addressing for more efficient systems.

Abstract: Inductive reasoning enables humans to infer abstract rules from limited examples and apply them to novel situations. In this work, we compare an LLM-based hypothesis search framework with direct program generation approaches on few-shot rule induction tasks. Our findings show that hypothesis search achieves performance comparable to humans, while direct program generation falls notably behind. An error analysis reveals key bottlenecks in hypothesis generation and suggests directions for advancing program induction methods. Overall, this paper underscores the potential of LLM-based hypothesis search for modeling inductive reasoning and the challenges in building more efficient systems.

Method: Modeling chemical reactions where computation is the reaction itself and execution environment is molecular aggregation degree. Implementing open computing through Token computing (individual molecular behavior) and Type computing (normative behavior) with their interplay.

Result: Token computing exhibits self-organizing critical phenomena, Type computing demonstrates quantum logic. Their interplay enables recruitment of fluctuations, quantum coherence across Hilbert spaces, and formation of spike waves for signal transmission.

Conclusion: The system achieves quantum-like coherence through the interplay of Token and Type computing, potentially explaining the source of enzymes controlling spike waves and biochemical rhythms, representing a novel approach to open computing in chemical systems.

Abstract: By uncovering the contrast between Artificial Intelligence and Natural-born Intelligence as a computational process, we define closed computing and open computing, and implement open computing within chemical reactions. This involves forming a mixture and invalidation of the computational process and the execution environment, which are logically distinct, and coalescing both to create a system that adjusts fluctuations. We model chemical reactions by considering the computation as the chemical reaction and the execution environment as the degree of aggregation of molecules that interact with the reactive environment. This results in a chemical reaction that progresses while repeatedly clustering and de-clustering, where concentration no longer holds significant meaning. Open computing is segmented into Token computing, which focuses on the individual behavior of chemical molecules, and Type computing, which focuses on normative behavior. Ultimately, both are constructed as an interplay between the two. In this system, Token computing demonstrates self-organizing critical phenomena, while Type computing exhibits quantum logic. Through their interplay, the recruitment of fluctuations is realized, giving rise to interactions between quantum logical subspaces corresponding to quantum coherence across different Hilbert spaces. As a result, spike waves are formed, enabling signal transmission. This occurrence may be termed quantum-like coherence, implying the source of enzymes responsible for controlling spike waves and biochemical rhythms.

Method: Introduces FlashAdventure benchmark, CUA-as-a-Judge automated evaluator, and COAST agentic framework that uses long-term clue memory for better planning and sequential task solving.

Result: Current GUI agents struggle with complete story arcs. COAST improves milestone completion by addressing the observation-behavior gap, but significant performance gap remains between best agents and humans.

Conclusion: The benchmark reveals limitations in current GUI agents for complex narrative tasks. While COAST shows improvement, continued research is needed to bridge the human-agent performance gap in adventure game completion.

Abstract: GUI agents powered by LLMs show promise in interacting with diverse digital environments. Among these, video games offer a valuable testbed due to their varied interfaces, with adventure games posing additional challenges through complex, narrative-driven interactions. Existing game benchmarks, however, lack diversity and rarely evaluate agents on completing entire storylines. To address this, we introduce FlashAdventure, a benchmark of 34 Flash-based adventure games designed to test full story arc completion and tackle the observation-behavior gap: the challenge of remembering and acting on earlier gameplay information. We also propose CUA-as-a-Judge, an automated gameplay evaluator, and COAST, an agentic framework leveraging long-term clue memory to better plan and solve sequential tasks. Experiments show current GUI agents struggle with full story arcs, while COAST improves milestone completion by bridging the observation-behavior gap. Nonetheless, a marked discrepancy between humans and best-performing agents warrants continued research efforts to narrow this divide.

Method: VerlTool introduces systematic design principles including upstream alignment with VeRL, unified tool management via standardized APIs, asynchronous rollout execution, and modular plugin architecture for rapid tool integration.

Result: The framework achieves near 2x speedup through asynchronous execution and demonstrates competitive performance across 6 ARLT domains including mathematical reasoning, knowledge QA, SQL generation, visual reasoning, web search, and software engineering.

Conclusion: VerlTool provides a scalable foundation for tool-augmented RL research with reduced development overhead, formalizing ARLT as multi-turn trajectories with multi-modal observations beyond single-turn RLVR paradigms.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has demonstrated success in enhancing LLM reasoning capabilities, but remains limited to single-turn interactions without tool integration. While recent Agentic Reinforcement Learning with Tool use (ARLT) approaches have emerged to address multi-turn tool interactions, existing works develop task-specific codebases that suffer from fragmentation, synchronous execution bottlenecks, and limited extensibility across domains. These inefficiencies hinder broader community adoption and algorithmic innovation. We introduce VerlTool, a unified and modular framework that addresses these limitations through systematic design principles. VerlTool provides four key contributions: (1) upstream alignment with VeRL ensuring compatibility and simplified maintenance, (2) unified tool management via standardized APIs supporting diverse modalities including code execution, search, SQL databases, and vision processing, (3) asynchronous rollout execution achieving near 2$\times$ speedup by eliminating synchronization bottlenecks, and (4) comprehensive evaluation demonstrating competitive performance across 6 ARLT domains. Our framework formalizes ARLT as multi-turn trajectories with multi-modal observation tokens (text/image/video), extending beyond single-turn RLVR paradigms. We train and evaluate models on mathematical reasoning, knowledge QA, SQL generation, visual reasoning, web search, and software engineering tasks, achieving results comparable to specialized systems while providing unified training infrastructure. The modular plugin architecture enables rapid tool integration requiring only lightweight Python definitions, significantly reducing development overhead and providing a scalable foundation for tool-augmented RL research. Our code is open-sourced at https://github.com/TIGER-AI-Lab/verl-tool.

Method: Uses chain-of-thought reasoning with a three-stage training strategy: (1) continued pretraining for embodied reasoning abilities, (2) supervised finetuning to model human-robot interaction and task planning as unified sequences, and (3) reinforcement learning for reasoning-action consistency.

Result: Robix outperforms both open-source and commercial baselines in interactive task execution, demonstrating strong generalization across diverse instruction types and various user-involved tasks like table bussing, grocery shopping, and dietary filtering.

Conclusion: Robix successfully integrates reasoning, planning, and interaction capabilities within a single vision-language architecture, enabling robots to handle complex real-world scenarios with natural human interaction and demonstrating superior performance compared to state-of-the-art models.

Abstract: We introduce Robix, a unified model that integrates robot reasoning, task planning, and natural language interaction within a single vision-language architecture. Acting as the high-level cognitive layer in a hierarchical robot system, Robix dynamically generates atomic commands for the low-level controller and verbal responses for human interaction, enabling robots to follow complex instructions, plan long-horizon tasks, and interact naturally with human within an end-to-end framework. Robix further introduces novel capabilities such as proactive dialogue, real-time interruption handling, and context-aware commonsense reasoning during task execution. At its core, Robix leverages chain-of-thought reasoning and adopts a three-stage training strategy: (1) continued pretraining to enhance foundational embodied reasoning abilities including 3D spatial understanding, visual grounding, and task-centric reasoning; (2) supervised finetuning to model human-robot interaction and task planning as a unified reasoning-action sequence; and (3) reinforcement learning to improve reasoning-action consistency and long-horizon task coherence. Extensive experiments demonstrate that Robix outperforms both open-source and commercial baselines (e.g., GPT-4o and Gemini 2.5 Pro) in interactive task execution, demonstrating strong generalization across diverse instruction types (e.g., open-ended, multi-stage, constrained, invalid, and interrupted) and various user-involved tasks such as table bussing, grocery shopping, and dietary filtering.

Method: Uses causal abstractive simulation (a variation of causal abstraction) to analyze language model simulation, with concrete examples of simulating fair coin tosses and proving simulation capabilities.

Result: Demonstrates both failure modes and successful cases of language model simulation, showing how the formalism can prove when a language model properly simulates another system given causal descriptions.

Conclusion: Causal abstractive simulation provides a rigorous foundation for language model simulation that benefits practitioners, philosophers, and mathematicians by formalizing simulation concepts and connecting to causal theory.

Abstract: This note illustrates how a variety of causal abstraction arXiv:1707.00819 arXiv:1812.03789, defined here as causal abstractive simulation, can be used to formalize a simple example of language model simulation. This note considers the case of simulating a fair coin toss with a language model. Examples are presented illustrating the ways language models can fail to simulate, and a success case is presented, illustrating how this formalism may be used to prove that a language model simulates some other system, given a causal description of the system. This note may be of interest to three groups. For practitioners in the growing field of language model simulation, causal abstractive simulation is a means to connect ad-hoc statistical benchmarking practices to the solid formal foundation of causality. Philosophers of AI and philosophers of mind may be interested as causal abstractive simulation gives a precise operationalization to the idea that language models are role-playing arXiv:2402.12422. Mathematicians and others working on causal abstraction may be interested to see a new application of the core ideas that yields a new variation of causal abstraction.

Method: A multi-agent collaboration framework with three components: 1) Predictor agent dynamically identifies candidate anchor entities by aligning query terms with KG nodes, 2) Retriever agents conduct parallel multi-hop explorations from each candidate, and 3) Supervisor agent formulates retrieval strategy and synthesizes knowledge paths for final answer generation.

Result: Extensive experiments on four public benchmarks show that AnchorRAG significantly outperforms existing baselines and establishes new state-of-the-art results on real-world question answering tasks.

Conclusion: AnchorRAG provides a robust solution for open-world RAG by eliminating the dependency on predefined anchor entities through multi-agent collaboration, demonstrating superior performance and addressing the limitations of traditional KG-based retrieval approaches.

Abstract: Large Language Models (LLMs) have demonstrated strong capabilities in language understanding and reasoning. However, their dependence on static training corpora makes them prone to factual errors and knowledge gaps. Retrieval-Augmented Generation (RAG) addresses this limitation by incorporating external knowledge sources, especially structured Knowledge Graphs (KGs), which provide explicit semantics and efficient retrieval. Existing KG-based RAG approaches, however, generally assume that anchor entities are accessible to initiate graph traversal, which limits their robustness in open world settings where accurate linking between the query and the entity is unreliable. To overcome this limitation, we propose AnchorRAG, a novel multi-agent collaboration framework for open-world RAG without the predefined anchor entities. Specifically, a predictor agent dynamically identifies candidate anchor entities by aligning user query terms with KG nodes and initializes independent retriever agents to conduct parallel multi-hop explorations from each candidate. Then a supervisor agent formulates the iterative retrieval strategy for these retriever agents and synthesizes the resulting knowledge paths to generate the final answer. This multi-agent collaboration framework improves retrieval robustness and mitigates the impact of ambiguous or erroneous anchors. Extensive experiments on four public benchmarks demonstrate that AnchorRAG significantly outperforms existing baselines and establishes new state-of-the-art results on the real-world question answering tasks.

Method: VGTeam uses a team of communicative agents (scriptwriting, scene creation, audio design) working collaboratively in a chat tower workflow to transform text prompts into slide-style narrative videos, emulating traditional video production stages.

Result: The system achieves remarkable efficiency improvements with substantially reduced computational overhead - generates videos at $0.103 average cost with 98.4% success rate while maintaining high creative fidelity and customization.

Conclusion: VGTeam democratizes video production by enabling high-quality content creation without extensive resources, demonstrating the transformative potential of language models in creative domains as a pioneering next-generation content creation system.

Abstract: With the rapid advancement of artificial intelligence (AI), the proliferation of AI-generated content (AIGC) tasks has significantly accelerated developments in text-to-video generation. As a result, the field of video production is undergoing a transformative shift. However, conventional text-to-video models are typically constrained by high computational costs. In this study, we propose Video-Generation-Team (VGTeam), a novel slide show video generation system designed to redefine the video creation pipeline through the integration of large language models (LLMs). VGTeam is composed of a suite of communicative agents, each responsible for a distinct aspect of video generation, such as scriptwriting, scene creation, and audio design. These agents operate collaboratively within a chat tower workflow, transforming user-provided textual prompts into coherent, slide-style narrative videos. By emulating the sequential stages of traditional video production, VGTeam achieves remarkable improvements in both efficiency and scalability, while substantially reducing computational overhead. On average, the system generates videos at a cost of only $0.103, with a successful generation rate of 98.4%. Importantly, this framework maintains a high degree of creative fidelity and customization. The implications of VGTeam are far-reaching. It democratizes video production by enabling broader access to high-quality content creation without the need for extensive resources. Furthermore, it highlights the transformative potential of language models in creative domains and positions VGTeam as a pioneering system for next-generation content creation.

Method: Introduced a framework for training Outcome Reward Models (ORMs) that assign utility scores based on semantic correctness. Evaluated on BIRD and Spider benchmarks using various open-source LLMs including Qwen2, Granite3, and Llama3 families.

Result: ORMs achieved execution accuracy gains of +4.33% (BIRD) and +2.10% (Spider) over ex-BoN, and +2.91% (BIRD) and +0.93% (Spider) over Maj. Fine-tuned models like OmniSQL showed superior ORM performance.

Conclusion: ORMs provide a more effective heuristic for Best-of-N selection in Text-to-SQL, outperforming traditional methods by better aligning with semantic correctness and benefiting more from increased candidate numbers.

Abstract: Text-to-SQL, the task of translating natural language questions into SQL queries, has significantly advanced with the introduction of Large Language Models (LLMs), broadening database accessibility for a wide range of users. Despite substantial progress in generating valid SQL, current LLMs still struggle with complex queries that require precise alignment between user intent and the database schema. To mitigate this, test-time strategies such as Best-of-N (BoN) and Majority Voting (Maj) are often employed, based on the assumption that LLMs can generate correct answers but may require multiple attempts. However, these methods rely on surface-level heuristics, selecting either the syntactically correct query through execution-based BoN (ex-BoN) or the most frequently generated query with Maj. Recently, Outcome Reward Models (ORMs), which assign utility scores to generated outputs based on semantic correctness, have emerged as a promising approach for better aligning model predictions with user intent. Nevertheless, their application to Text-to-SQL remains largely underexplored. In this work, we evaluate ORMs as an effective heuristic for BoN, compare them with ex-BoN and Maj, and introduce a framework for training ORMs for the Text-to-SQL task. We evaluate our ORMs on the BIRD and SPIDER benchmarks, finetuning various open-source LLMs, including the Qwen2, Granite3, and Llama3 model families. Our results show that ORMs outperform ex-BoN and Maj, achieving execution accuracy gains of +4.33% (BIRD) and +2.10% (Spider) over ex-BoN, and +2.91% (BIRD) and +0.93% (Spider) over Maj. We further demonstrate that finetuning models already aligned with SQL generation, such as OmniSQL, yields superior ORM performance. Additionally, we observe that ORMs achieve competitive results on simple queries and benefit more from an increased number of candidates compared to ex-BoN and Maj.

Method: Leverages score-based diffusion models to approximate probabilistic multi-modal system dynamics, employs a mode classifier to partition trajectories by dynamical mode, and applies conformal inference for each mode to produce statistically valid prediction intervals.

Result: GenQPM produces significantly more informative (less conservative) prediction intervals compared to mode-agnostic baselines on agent navigation and autonomous driving benchmarks.

Conclusion: The proposed GenQPM method effectively addresses the limitations of existing QPM approaches by incorporating mode-specific analysis through deep generative models, resulting in more precise and useful prediction intervals for systems with complex multi-modal dynamics.

Abstract: We consider the problem of quantitative predictive monitoring (QPM) of stochastic systems, i.e., predicting at runtime the degree of satisfaction of a desired temporal logic property from the current state of the system. Since computational efficiency is key to enable timely intervention against predicted violations, several state-of-the-art QPM approaches rely on fast machine-learning surrogates to provide prediction intervals for the satisfaction values, using conformal inference to offer statistical guarantees. However, these QPM methods suffer when the monitored agent exhibits multi-modal dynamics, whereby certain modes may yield high satisfaction values while others critically violate the property. Existing QPM methods are mode-agnostic and so would yield overly conservative and uninformative intervals that lack meaningful mode-specific satisfaction information. To address this problem, we present GenQPM, a method that leverages deep generative models, specifically score-based diffusion models, to reliably approximate the probabilistic and multi-modal system dynamics without requiring explicit model access. GenQPM employs a mode classifier to partition the predicted trajectories by dynamical mode. For each mode, we then apply conformal inference to produce statistically valid, mode-specific prediction intervals. We demonstrate the effectiveness of GenQPM on a benchmark of agent navigation and autonomous driving tasks, resulting in prediction intervals that are significantly more informative (less conservative) than mode-agnostic baselines.

Method: Error Notebooks construction (collecting erroneous CoTs with corrections) + RAG to retrieve specification-relevant records for refined prompt engineering.

Result: Substantial gains with proprietary models, GPT-4o achieving 23.4% absolute accuracy improvement; CoT reasoning benefits challenging cases with >10 parts.

Conclusion: The framework effectively handles 3D models with lengthy metadata without training, demonstrating significant performance improvements in CAD part retrieval.

Abstract: Effective specification-aware part retrieval within complex CAD assemblies is essential for automated design verification and downstream engineering tasks. However, directly using LLMs/VLMs to this task presents some challenges: the input sequences may exceed model token limits, and even after processing, performance remains unsatisfactory. Moreover, fine-tuning LLMs/VLMs requires significant computational resources, and for many high-performing general-use proprietary models (e.g., GPT or Gemini), fine-tuning access is not available. In this paper, we propose a novel part retrieval framework that requires no extra training, but using Error Notebooks + RAG for refined prompt engineering to help improve the existing general model’s retrieval performance. The construction of Error Notebooks consists of two steps: (1) collecting historical erroneous CoTs and their incorrect answers, and (2) connecting these CoTs through reflective corrections until the correct solutions are obtained. As a result, the Error Notebooks serve as a repository of tasks along with their corrected CoTs and final answers. RAG is then employed to retrieve specification-relevant records from the Error Notebooks and incorporate them into the inference process. Another major contribution of our work is a human-in-the-loop CAD dataset, which is used to evaluate our method. In addition, the engineering value of our novel framework lies in its ability to effectively handle 3D models with lengthy, non-natural language metadata. Experiments with proprietary models, including GPT-4o and the Gemini series, show substantial gains, with GPT-4o (Omni) achieving up to a 23.4% absolute accuracy improvement on the human preference dataset. Moreover, ablation studies confirm that CoT reasoning provides benefits especially in challenging cases with higher part counts (>10).

Method: Multi-Agent Hierarchical Task Generation (MAHTG) system that extracts research-worthy inspirations from seminar transcripts and translates them into high-quality research tasks.

Result: Curated over 10,000 high-quality research tasks from 200+ academic seminars across 12 disciplines, presenting substantial challenges for state-of-the-art agents with clear performance gaps.

Conclusion: DeepResearch Arena provides a faithful benchmark grounded in real academic discourse that better reflects real-world research environments and reduces data leakage risks.

Abstract: Deep research agents have attracted growing attention for their potential to orchestrate multi-stage research workflows, spanning literature synthesis, methodological design, and empirical verification. Despite these strides, evaluating their research capability faithfully is rather challenging due to the difficulty of collecting frontier research questions that genuinely capture researchers’ attention and intellectual curiosity. To address this gap, we introduce DeepResearch Arena, a benchmark grounded in academic seminars that capture rich expert discourse and interaction, better reflecting real-world research environments and reducing the risk of data leakage. To automatically construct DeepResearch Arena, we propose a Multi-Agent Hierarchical Task Generation (MAHTG) system that extracts research-worthy inspirations from seminar transcripts. The MAHTG system further translates research-worthy inspirations into high-quality research tasks, ensuring the traceability of research task formulation while filtering noise. With the MAHTG system, we curate DeepResearch Arena with over 10,000 high-quality research tasks from over 200 academic seminars, spanning 12 disciplines, such as literature, history, and science. Our extensive evaluation shows that DeepResearch Arena presents substantial challenges for current state-of-the-art agents, with clear performance gaps observed across different models.

Method: The paper examines historical scientific discovery development, reviews AI approaches including data-driven methods, knowledge-aware neural architectures, symbolic reasoning frameworks, and LLM agents for hypothesis generation and pattern discovery.

Result: AI systems can successfully uncover patterns and propose candidate scientific laws at unprecedented scale and speed, but their scientific value remains dependent on verification processes.

Conclusion: Rigorous and transparent verification must be the cornerstone of AI-assisted scientific discovery to ensure that the abundance of AI-generated hypotheses translates into meaningful scientific advancement rather than creating verification bottlenecks.

Abstract: Artificial intelligence (AI) is transforming the practice of science. Machine learning and large language models (LLMs) can generate hypotheses at a scale and speed far exceeding traditional methods, offering the potential to accelerate discovery across diverse fields. However, the abundance of hypotheses introduces a critical challenge: without scalable and reliable mechanisms for verification, scientific progress risks being hindered rather than being advanced. In this article, we trace the historical development of scientific discovery, examine how AI is reshaping established practices for scientific discovery, and review the principal approaches, ranging from data-driven methods and knowledge-aware neural architectures to symbolic reasoning frameworks and LLM agents. While these systems can uncover patterns and propose candidate laws, their scientific value ultimately depends on rigorous and transparent verification, which we argue must be the cornerstone of AI-assisted discovery.

Method: Counterfactual Sensitivity Regularization (CSR) introduces automated operator-level counterfactual interventions during training and penalizes models that maintain the same answer under logically invalid reasoning traces.

Result: CSR improves faithfulness by up to 70 percentage points over standard methods across arithmetic, logical deduction, and planning tasks, with minimal accuracy loss, and generalizes to larger models.

Conclusion: CSR effectively enhances reasoning faithfulness in LLMs through lightweight counterfactual training, and can be extended with semantic perturbations for commonsense reasoning tasks.

Abstract: Large language models (LLMs) often produce correct answers while relying on flawed or irrelevant reasoning traces, undermining their trustworthiness in high-stakes domains. We propose Counterfactual Sensitivity Regularization (CSR), a lightweight training objective that enforces dependence between intermediate reasoning and final outputs. CSR introduces automated, operator-level counterfactual interventions (e.g., swapping “+” with “-”) during training and penalizes models that preserve the same answer under logically invalid traces. This requires only one additional forward pass per sample. To measure faithfulness, we introduce Counterfactual Outcome Sensitivity (COS), which quantifies the impact of such perturbations on model predictions. Across structured reasoning tasks - arithmetic (GSM8K), logical deduction (PrOntoQA), and planning (Blocks World) - CSR improves faithfulness by up to 70 percentage points over standard fine-tuning and process supervision, with only minor accuracy loss. The learned sensitivity generalizes to larger models and synergizes with inference-time methods such as self-consistency. A pilot study on HellaSwag further demonstrates that extending CSR with semantic perturbations can enhance faithfulness in commonsense reasoning.

Method: Developed a structured decision-making framework using Enabler agents, Levels and Scenarios, implemented in autonomous decision-making for disaster management. Compared against judgement-based systems and experienced human operators (victims, volunteers, stakeholders).

Result: Framework achieved 60.94% greater stability in consistently accurate decisions across multiple scenarios compared to judgement-based systems, and outperformed human operators with 38.93% higher accuracy across various scenarios.

Conclusion: The structured decision-making framework shows promise for building more reliable autonomous AI applications in safety-critical contexts, demonstrating significant improvements in decision stability and accuracy over both automated and human alternatives.

Abstract: With artificial intelligence (AI) being applied to bring autonomy to decision-making in safety-critical domains such as the ones typified in the aerospace and emergency-response services, there has been a call to address the ethical implications of structuring those decisions, so they remain reliable and justifiable when human lives are at stake. This paper contributes to addressing the challenge of decision-making by proposing a structured decision-making framework as a foundational step towards responsible AI. The proposed structured decision-making framework is implemented in autonomous decision-making, specifically within disaster management. By introducing concepts of Enabler agents, Levels and Scenarios, the proposed framework’s performance is evaluated against systems relying solely on judgement-based insights, as well as human operators who have disaster experience: victims, volunteers, and stakeholders. The results demonstrate that the structured decision-making framework achieves 60.94% greater stability in consistently accurate decisions across multiple Scenarios, compared to judgement-based systems. Moreover, the study shows that the proposed framework outperforms human operators with a 38.93% higher accuracy across various Scenarios. These findings demonstrate the promise of the structured decision-making framework for building more reliable autonomous AI applications in safety-critical contexts.

Method: Developed Throttling Gates using rebus-based Reasoning Gates - synthetic text puzzles requiring multi-hop reasoning over world knowledge, with scalable generation and verification protocols.

Result: Achieved 9.2x computational asymmetry (response cost vs generation cost for SOTA models), successfully deployed on custom websites and MCP servers, and evaluated with real-world web agents.

Conclusion: The framework effectively throttles web agents through computational costs but has limitations and environmental impact concerns that need consideration for real-world deployment.

Abstract: AI web agents use Internet resources at far greater speed, scale, and complexity – changing how users and services interact. Deployed maliciously or erroneously, these agents could overload content providers. At the same time, web agents can bypass CAPTCHAs and other defenses by mimicking user behavior or flood authentication systems with fake accounts. Yet providers must protect their services and content from denial-of-service attacks and scraping by web agents. In this paper, we design a framework that imposes tunable costs on agents before providing access to resources; we call this Web Agent Throttling. We start by formalizing Throttling Gates as challenges issued to an agent that are asymmetric, scalable, robust, and compatible with any agent. Focusing on a common component – the language model – we require the agent to solve reasoning puzzles, thereby incurring excessive token-generation costs. However, we find that using existing puzzles, e.g., coding or math, as throttling gates fails to satisfy our properties. To address this, we introduce rebus-based Reasoning Gates, synthetic text puzzles that require multi-hop reasoning over world knowledge (thereby throttling an agent’s model). We design a scalable generation and verification protocol for such reasoning gates. Our framework achieves computational asymmetry, i.e., the response-generation cost is 9.2x higher than the generation cost for SOTA models. We further deploy reasoning gates on a custom website and Model Context Protocol (MCP) servers and evaluate with real-world web agents. Finally, we discuss the limitations and environmental impact of real-world deployment of our framework.

Method: Developed a novel neuron-level interpretability method that projects model’s internal representation into interpretable vocabulary space to identify safety-related knowledge neurons, then created SafeTuning - a fine-tuning strategy that reinforces these safety-critical neurons.

Result: Achieved mean Attack Success Rate (ASR) higher than 97% by adjusting safety-related neuron activations. SafeTuning consistently reduced attack success rates across multiple LLMs and outperformed all four baseline defenses.

Conclusion: Provides a new perspective on understanding and defending against jailbreak attacks by focusing on safety-critical neurons, offering an effective fine-tuning approach to improve model robustness against malicious exploitation.

Abstract: Large Language Models (LLMs) are increasingly attracting attention in various applications. Nonetheless, there is a growing concern as some users attempt to exploit these models for malicious purposes, including the synthesis of controlled substances and the propagation of disinformation, a technique known as “Jailbreak.” While some studies have achieved defenses against jailbreak attacks by modifying output distributions or detecting harmful content, the exact rationale still remains elusive. In this work, we present a novel neuron-level interpretability method that focuses on the role of safety-related knowledge neurons. Unlike existing approaches, our method projects the model’s internal representation into a more consistent and interpretable vocabulary space. We then show that adjusting the activation of safety-related neurons can effectively control the model’s behavior with a mean ASR higher than 97%. Building on this insight, we propose SafeTuning, a fine-tuning strategy that reinforces safety-critical neurons to improve model robustness against jailbreaks. SafeTuning consistently reduces attack success rates across multiple LLMs and outperforms all four baseline defenses. These findings offer a new perspective on understanding and defending against jailbreak attacks.

Method: The researchers developed Physics Supernova as an AI agent system with principled tool integration, designed to formulate and apply physical laws for explaining and predicting physical processes.

Result: Physics Supernova achieved 23.5/30 points in IPhO 2025 theory problems, ranking 14th of 406 contestants and surpassing the median performance of human gold medalists. Extensive analysis showed strong capabilities across diverse physics tasks.

Conclusion: Principled tool integration within agent systems can deliver competitive improvements in solving challenging science problems, demonstrating that AI can achieve elite-level physics problem-solving abilities comparable to top human performers.

Abstract: Physics provides fundamental laws that describe and predict the natural world. AI systems aspiring toward more general, real-world intelligence must therefore demonstrate strong physics problem-solving abilities: to formulate and apply physical laws for explaining and predicting physical processes. The International Physics Olympiad (IPhO)–the world’s most prestigious physics competition–offers a rigorous benchmark for this purpose. We introduce Physics Supernova, an AI agent system with superior physics problem-solving abilities that match elite IPhO gold medalists. In IPhO 2025 theory problems, Physics Supernova attains 23.5/30 points, ranking 14th of 406 contestants and surpassing the median performance of human gold medalists. We extensively analyzed Physics Supernova’s capabilities and flexibility across diverse physics tasks. These results show that principled tool integration within agent systems can deliver competitive improvements in solving challenging science problems. The codes are available at https://github.com/CharlesQ9/Physics-Supernova.

Method: Uses state-of-the-art LLMs to identify key privacy information, constructs Pr²Graph knowledge graph grounded in Data Privacy Vocabulary (DPV), and demonstrates conversion to formal policy representations like ODRL and psDToU.

Result: Created Pr²Graph for top-100 websites as public resource, enriched Policy-IE dataset with expert annotations, benchmarked LLM performance, and verified technology capabilities for privacy practice extraction.

Conclusion: Shows promise for large-scale analysis of online privacy practices, enabling web auditing. All datasets and source code released publicly to facilitate reuse and improvement.

Abstract: In modern times, people have numerous online accounts, but they rarely read the Terms of Service or Privacy Policy of those sites, despite claiming otherwise, due to the practical difficulty in comprehending them. The mist of data privacy practices forms a major barrier for user-centred Web approaches, and for data sharing and reusing in an agentic world. Existing research proposed methods for using formal languages and reasoning for verifying the compliance of a specified policy, as a potential cure for ignoring privacy policies. However, a critical gap remains in the creation or acquisition of such formal policies at scale. We present a semantic-centric approach for using state-of-the-art large language models (LLM), to automatically identify key information about privacy practices from privacy policies, and construct $\mathit{Pr}^2\mathit{Graph}$, knowledge graph with grounding from Data Privacy Vocabulary (DPV) for privacy practices, to support downstream tasks. Along with the pipeline, the $\mathit{Pr}^2\mathit{Graph}$ for the top-100 popular websites is also released as a public resource, by using the pipeline for analysis. We also demonstrate how the $\mathit{Pr}^2\mathit{Graph}$ can be used to support downstream tasks by constructing formal policy representations such as Open Digital Right Language (ODRL) or perennial semantic Data Terms of Use (psDToU). To evaluate the technology capability, we enriched the Policy-IE dataset by employing legal experts to create custom annotations. We benchmarked the performance of different large language models for our pipeline and verified their capabilities. Overall, they shed light on the possibility of large-scale analysis of online services’ privacy practices, as a promising direction to audit the Web and the Internet. We release all datasets and source code as public resources to facilitate reuse and improvement.

Method: Constructive Safety Alignment (CSA) combines game-theoretic anticipation of user reactions, fine-grained risk boundary discovery, and interpretable reasoning control.

Result: Oy1 achieves SOTA safety among open models, strong constructive engagement close to GPT-5, and unmatched robustness on jailbreak datasets nearing GPT-o1 levels.

Conclusion: CSA redefines model-user relationships by shifting from refusal-first to guidance-first safety, creating systems that are both safe and meaningfully helpful.

Abstract: Large language models (LLMs) typically deploy safety mechanisms to prevent harmful content generation. Most current approaches focus narrowly on risks posed by malicious actors, often framing risks as adversarial events and relying on defensive refusals. However, in real-world settings, risks also come from non-malicious users seeking help while under psychological distress (e.g., self-harm intentions). In such cases, the model’s response can strongly influence the user’s next actions. Simple refusals may lead them to repeat, escalate, or move to unsafe platforms, creating worse outcomes. We introduce Constructive Safety Alignment (CSA), a human-centric paradigm that protects against malicious misuse while actively guiding vulnerable users toward safe and helpful results. Implemented in Oyster-I (Oy1), CSA combines game-theoretic anticipation of user reactions, fine-grained risk boundary discovery, and interpretable reasoning control, turning safety into a trust-building process. Oy1 achieves state-of-the-art safety among open models while retaining high general capabilities. On our Constructive Benchmark, it shows strong constructive engagement, close to GPT-5, and unmatched robustness on the Strata-Sword jailbreak dataset, nearing GPT-o1 levels. By shifting from refusal-first to guidance-first safety, CSA redefines the model-user relationship, aiming for systems that are not just safe, but meaningfully helpful. We release Oy1, code, and the benchmark to support responsible, user-centered AI.

Method: Uses an ensemble of models, a constitution describing values, and scenario datasets. Models judge each other’s outputs, and EigenTrust aggregates these judgments to produce alignment scores without ground truth data.

Result: EigenBench scores are mostly sensitive to the prompt rather than the model itself, but a small residual quantifies the model’s inherent disposition.

Conclusion: EigenBench provides a practical method for quantitatively benchmarking value alignment in language models, demonstrating that prompts dominate alignment scores while still capturing model-specific dispositions.

Abstract: Aligning AI with human values is a pressing unsolved problem. To address the lack of quantitative metrics for value alignment, we propose EigenBench: a black-box method for comparatively benchmarking language models’ values. Given an ensemble of models, a constitution describing a value system, and a dataset of scenarios, our method returns a vector of scores quantifying each model’s alignment to the given constitution. To produce these scores, each model judges the outputs of other models across many scenarios, and these judgments are aggregated with EigenTrust (Kamvar et al, 2003), yielding scores that reflect a weighted-average judgment of the whole ensemble. EigenBench uses no ground truth labels, as it is designed to quantify traits for which reasonable judges may disagree on the correct label. Using prompted personas, we test whether EigenBench scores are more sensitive to the model or the prompt: we find that most of the variance is explained by the prompt, but a small residual quantifies the disposition of the model itself.

Method: Construct two large benchmarks from MIMIC-IV (ED-Triage and Opioid Recommendation) with 50k+ prompts across race/gender variants, and develop mFARM framework to assess allocational, stability, and latent disparities with aggregated fairness-accuracy scores.

Result: Evaluation of 4 open-source LLMs shows mFARM captures subtle biases effectively; models maintain robust fairness under quantization but deteriorate significantly with reduced context.

Conclusion: The mFARM framework provides comprehensive fairness assessment for medical LLMs, with benchmarks and code released to advance aligned AI in healthcare.

Abstract: The deployment of Large Language Models (LLMs) in high-stakes medical settings poses a critical AI alignment challenge, as models can inherit and amplify societal biases, leading to significant disparities. Existing fairness evaluation methods fall short in these contexts as they typically use simplistic metrics that overlook the multi-dimensional nature of medical harms. This also promotes models that are fair only because they are clinically inert, defaulting to safe but potentially inaccurate outputs. To address this gap, our contributions are mainly two-fold: first, we construct two large-scale, controlled benchmarks (ED-Triage and Opioid Analgesic Recommendation) from MIMIC-IV, comprising over 50,000 prompts with twelve race x gender variants and three context tiers. Second, we propose a multi-metric framework - Multi-faceted Fairness Assessment based on hARMs ($mFARM$) to audit fairness for three distinct dimensions of disparity (Allocational, Stability, and Latent) and aggregate them into an $mFARM$ score. We also present an aggregated Fairness-Accuracy Balance (FAB) score to benchmark and observe trade-offs between fairness and prediction accuracy. We empirically evaluate four open-source LLMs (Mistral-7B, BioMistral-7B, Qwen-2.5-7B, Bio-LLaMA3-8B) and their finetuned versions under quantization and context variations. Our findings showcase that the proposed $mFARM$ metrics capture subtle biases more effectively under various settings. We find that most models maintain robust performance in terms of $mFARM$ score across varying levels of quantization but deteriorate significantly when the context is reduced. Our benchmarks and evaluation code are publicly released to enhance research in aligned AI for healthcare.

Method: The paper outlines the phenomenon of generative AI, formulates requirements for its use in TA, analyzes structural causes of associated problems, and proposes solutions with feasibility assessments.

Result: The analysis reveals that while generative AI continues to develop, structurally induced risks persist, requiring specific solutions for safe and effective implementation in technology assessment work.

Conclusion: Generative AI presents both opportunities and challenges for technology assessment, with persistent structural risks that need targeted solutions, though the technology shows promise when implemented with appropriate safeguards.

Abstract: Many scientists use generative AI in their scientific work. People working in technology assessment (TA) are no exception. TA’s approach to generative AI is twofold: on the one hand, generative AI is used for TA work, and on the other hand, generative AI is the subject of TA research. After briefly outlining the phenomenon of generative AI and formulating requirements for its use in TA, the following article discusses in detail the structural causes of the problems associated with it. Although generative AI is constantly being further developed, the structurally induced risks remain. The article concludes with proposed solutions and brief notes on their feasibility, as well as some examples of the use of generative AI in TA work.

Method: The paper proposes a theoretical framework called Structural-Generative Ontology of Intelligence, which establishes three essential conditions for true intelligence: generativity, coordination, and sustaining of identity over time.

Result: The framework demonstrates that current AI systems lack the depth required for genuine intelligence despite their broad functional capabilities. It provides criteria to distinguish between surface-level simulation and true intelligence.

Conclusion: True intelligence emerges from depth (generativity, coordination, and sustaining) rather than breadth of function. Future systems meeting these conditions could represent a “Second Being” distinct from human existence, moving beyond being mere tools.

Abstract: Artificial intelligence is often measured by the range of tasks it can perform. Yet wide ability without depth remains only an imitation. This paper proposes a Structural-Generative Ontology of Intelligence: true intelligence exists only when a system can generate new structures, coordinate them into reasons, and sustain its identity over time. These three conditions – generativity, coordination, and sustaining – define the depth that underlies real intelligence. Current AI systems, however broad in function, remain surface simulations because they lack this depth. Breadth is not the source of intelligence but the growth that follows from depth. If future systems were to meet these conditions, they would no longer be mere tools, but could be seen as a possible Second Being, standing alongside yet distinct from human existence.

Method: Document chunking and augmentation to handle long documents, engineered prompts fed into QWEN-2 model, and Distribution-based Localisation + Inverse Cardinality Weighting heuristics for candidate selection.

Result: Achieved state-of-the-art performance with up to 9% improvement over previous methods on CUAD dataset for legal information retrieval.

Conclusion: Structured prompt engineering shows promise as an under-explored tool for ensuring AI accountability in legal domains, though current automatic evaluation metrics remain limiting.

Abstract: The rise of Large Language Models (LLMs) has had a profoundly transformative effect on a number of fields and domains. However, their uptake in Law has proven more challenging due to the important issues of reliability and transparency. In this study, we present a structured prompting methodology as a viable alternative to the often expensive fine-tuning, with the capability of tacking long legal documents from the CUAD dataset on the task of information retrieval. Each document is first split into chunks via a system of chunking and augmentation, addressing the long document problem. Then, alongside an engineered prompt, the input is fed into QWEN-2 to produce a set of answers for each question. Finally, we tackle the resulting candidate selection problem with the introduction of the Distribution-based Localisation and Inverse Cardinality Weighting heuristics. This approach leverages a general purpose model to promote long term scalability, prompt engineering to increase reliability and the two heuristic strategies to reduce the impact of the black box effect. Whilst our model performs up to 9% better than the previously presented method, reaching state-of-the-art performance, it also highlights the limiting factor of current automatic evaluation metrics for question answering, serving as a call to action for future research. However, the chief aim of this work is to underscore the potential of structured prompt engineering as a useful, yet under-explored, tool in ensuring accountability and responsibility of AI in the legal domain, and beyond.

Method: Used an ensemble approach incorporating multiple Large Language Models to process WHO Disease Outbreak News reports, extracting valuable epidemiological information and constructing a knowledge graph (eKG) with daily updates.

Result: Created a comprehensive daily-updated dataset and knowledge graph (eKG) that provides nuanced representation of public health domain knowledge from WHO outbreak reports, enabling new epidemiological research opportunities.

Conclusion: The developed AI-powered system and data resources open new possibilities for epidemiological research, disease outbreak analysis, and enhanced surveillance capabilities through automated extraction of actionable information from authoritative health reports.

Abstract: The rapid evolution of artificial intelligence (AI), together with the increased availability of social media and news for epidemiological surveillance, are marking a pivotal moment in epidemiology and public health research. Leveraging the power of generative AI, we use an ensemble approach which incorporates multiple Large Language Models (LLMs) to extract valuable actionable epidemiological information from the World Health Organization (WHO) Disease Outbreak News (DONs). DONs is a collection of regular reports on global outbreaks curated by the WHO and the adopted decision-making processes to respond to them. The extracted information is made available in a daily-updated dataset and a knowledge graph, referred to as eKG, derived to provide a nuanced representation of the public health domain knowledge. We provide an overview of this new dataset and describe the structure of eKG, along with the services and tools used to access and utilize the data that we are building on top. These innovative data resources open altogether new opportunities for epidemiological research, and the analysis and surveillance of disease outbreaks.

Method: Uses reward and policy mechanisms in reinforcement learning to identify explanatory paths within knowledge graphs, enriched with domain-specific ontologies to ensure explanations are grounded in biomedical knowledge.

Result: Outperforms state-of-the-art approaches in predictive performance on three knowledge graph benchmarks and generates explanations that validate predictive insights against biomedical knowledge.

Conclusion: REx represents a significant contribution to advancing AI-driven scientific discovery by providing both accurate predictions and scientifically meaningful explanations.

Abstract: Knowledge graphs (KGs) are powerful tools for modelling complex, multi-relational data and supporting hypothesis generation, particularly in applications like drug repurposing. However, for predictive methods to gain acceptance as credible scientific tools, they must ensure not only accuracy but also the capacity to offer meaningful scientific explanations. This paper presents a novel approach REx, for generating scientific explanations based in link prediction in knowledge graphs. It employs reward and policy mechanisms that consider desirable properties of scientific explanation to guide a reinforcement learning agent in the identification of explanatory paths within a KG. The approach further enriches explanatory paths with domain-specific ontologies, ensuring that the explanations are both insightful and grounded in established biomedical knowledge. We evaluate our approach in drug repurposing using three popular knowledge graph benchmarks. The results clearly demonstrate its ability to generate explanations that validate predictive insights against biomedical knowledge and that outperform the state-of-the-art approaches in predictive performance, establishing REx as a relevant contribution to advance AI-driven scientific discovery.