Method: Adaptive multi-agent framework with four specialized agents: retrieval agent for scalable video search, reasoning agent for zero-shot contextual temporal reasoning, query reformulation agent for refining ambiguous queries, and orchestration agent that dynamically coordinates agents using intermediate feedback and novel communication mechanism with retrieval-performance memory and historical reasoning traces.

Result: Achieves twofold improvement over CLIP4Clip and significantly outperforms state-of-the-art methods on three TRECVid benchmarks spanning eight years.

Conclusion: The proposed multi-agent framework effectively addresses limitations in temporal reasoning for zero-shot text-to-video retrieval through dynamic orchestration of specialized agents and improved coordination mechanisms.

Abstract: The rise of short-form video platforms and the emergence of multimodal large language models (MLLMs) have amplified the need for scalable, effective, zero-shot text-to-video retrieval systems. While recent advances in large-scale pretraining have improved zero-shot cross-modal alignment, existing methods still struggle with query-dependent temporal reasoning, limiting their effectiveness on complex queries involving temporal, logical, or causal relationships. To address these limitations, we propose an adaptive multi-agent retrieval framework that dynamically orchestrates specialized agents over multiple reasoning iterations based on the demands of each query. The framework includes: (1) a retrieval agent for scalable retrieval over large video corpora, (2) a reasoning agent for zero-shot contextual temporal reasoning, and (3) a query reformulation agent for refining ambiguous queries and recovering performance for those that degrade over iterations. These agents are dynamically coordinated by an orchestration agent, which leverages intermediate feedback and reasoning outcomes to guide execution. We also introduce a novel communication mechanism that incorporates retrieval-performance memory and historical reasoning traces to improve coordination and decision-making. Experiments on three TRECVid benchmarks spanning eight years show that our framework achieves a twofold improvement over CLIP4Clip and significantly outperforms state-of-the-art methods by a large margin.

Method: Three key innovations: 1) Modality-specific mixture-of-experts (MS-MoE) for cross-modal interaction while enhancing single-modal quality, 2) Temporal-aligned RoPE (TA-RoPE) for explicit frame-level audio-video synchronization, 3) Audio-video direct preference optimization (AV-DPO) to align outputs with human preferences.

Result: Achieves state-of-the-art performance with only ~1M public training entries, significantly outperforming prior approaches in both qualitative and quantitative evaluations. Built on Wan2.1-1.3B-T2V foundation.

Conclusion: JavisDiT++ bridges the gap between open-source and commercial joint audio-video generation models through effective architectural designs and optimization strategies for synchronization and human preference alignment.

Abstract: AIGC has rapidly expanded from text-to-image generation toward high-quality multimodal synthesis across video and audio. Within this context, joint audio-video generation (JAVG) has emerged as a fundamental task that produces synchronized and semantically aligned sound and vision from textual descriptions. However, compared with advanced commercial models such as Veo3, existing open-source methods still suffer from limitations in generation quality, temporal synchrony, and alignment with human preferences. To bridge the gap, this paper presents JavisDiT++, a concise yet powerful framework for unified modeling and optimization of JAVG. First, we introduce a modality-specific mixture-of-experts (MS-MoE) design that enables cross-modal interaction efficacy while enhancing single-modal generation quality. Then, we propose a temporal-aligned RoPE (TA-RoPE) strategy to achieve explicit, frame-level synchronization between audio and video tokens. Besides, we develop an audio-video direct preference optimization (AV-DPO) method to align model outputs with human preference across quality, consistency, and synchrony dimensions. Built upon Wan2.1-1.3B-T2V, our model achieves state-of-the-art performance merely with around 1M public training entries, significantly outperforming prior approaches in both qualitative and quantitative evaluations. Comprehensive ablation studies have been conducted to validate the effectiveness of our proposed modules. All the code, model, and dataset are released at https://JavisVerse.github.io/JavisDiT2-page.

Method: Uses multimodal LLMs (Gemma and Qwen) to generate contextually relevant audio labels, employs zero-shot learning (Human-CLAP) to quantify audio-text alignment, applies human-in-the-loop intervention for poorly aligned pairs, and clusters labels using adjusted silhouette score with penalty parameter for thematic granularity.

Result: Evaluated on three auditory scene datasets (ADVANCE, AHEAD-DS, TAU 2019), the framework provides scalable taxonomy creation enabling training of lightweight scene recognition models deployable to edge devices like hearing aids and smart home assistants.

Conclusion: AuditoryHuM offers a novel collaborative human-MLLM approach for unsupervised audio label discovery, balancing automation with human oversight to create standardized taxonomies for practical audio understanding applications.

Abstract: Manual annotation of audio datasets is labour intensive, and it is challenging to balance label granularity with acoustic separability. We introduce AuditoryHuM, a novel framework for the unsupervised discovery and clustering of auditory scene labels using a collaborative Human-Multimodal Large Language Model (MLLM) approach. By leveraging MLLMs (Gemma and Qwen) the framework generates contextually relevant labels for audio data. To ensure label quality and mitigate hallucinations, we employ zero-shot learning techniques (Human-CLAP) to quantify the alignment between generated text labels and raw audio content. A strategically targeted human-in-the-loop intervention is then used to refine the least aligned pairs. The discovered labels are grouped into thematically cohesive clusters using an adjusted silhouette score that incorporates a penalty parameter to balance cluster cohesion and thematic granularity. Evaluated across three diverse auditory scene datasets (ADVANCE, AHEAD-DS, and TAU 2019), AuditoryHuM provides a scalable, low-cost solution for creating standardised taxonomies. This solution facilitates the training of lightweight scene recognition models deployable to edge devices, such as hearing aids and smart home assistants. The project page and code: https://github.com/Australian-Future-Hearing-Initiative

Method: Introduces ReportLogic benchmark with hierarchical taxonomy: Macro-Logic (report structure), Expositional-Logic (context progression), and Structural-Logic (claim-support verification). Creates human-annotated dataset and trains open-source LogicJudge for scalable evaluation. Tests robustness via adversarial attacks.

Result: Off-the-shelf LLM judges are frequently influenced by superficial cues like verbosity, and reasoning modes can mask broken support relations. Provides actionable guidance for building more robust logic evaluators.

Conclusion: ReportLogic addresses the gap in evaluating logical quality of LLM-generated reports, offering a framework for improving logical reliability and developing better evaluation methods.

Abstract: Users increasingly rely on Large Language Models (LLMs) for Deep Research, using them to synthesize diverse sources into structured reports that support understanding and action. In this context, the practical reliability of such reports hinges on logical quality: whether the report’s claims and arguments are explicitly supported and can be trusted as a basis for downstream use, rather than merely appearing fluent or informative. However, current evaluation frameworks largely overlook this requirement. To bridge this gap, we introduce ReportLogic, a benchmark that quantifies report-level logical quality through a reader-centric lens of auditability. Specifically, ReportLogic adopts a hierarchical taxonomy that evaluates whether readers can (1) trace an on-topic report structure with a unified analytical arc (Macro-Logic), (2) understand the progression with necessary context (Expositional-Logic), and (3) verify conclusions via explicit claim–support (Structural-Logic). Based on this taxonomy, we construct a human-annotated rubric-guided dataset and train an open-source LogicJudge for scalable evaluation. We further evaluate judge robustness via adversarial attacks, showing that off-the-shelf LLM judges are frequently influenced by superficial cues (e.g., verbosity), and reasoning modes can mask broken support relations. Overall, our results provide actionable guidance for building more robust logic evaluators and improving the logical reliability of LLM-generated reports.

Method: Confidence-gated cascaded verification framework where small draft models handle step-level verification (a constrained discriminative task) and accept high-confidence decisions directly, while selectively escalating uncertain cases to the large target model.

Result: Achieves up to 2.24× end-to-end speedups while matching target-model accuracy across diverse workloads. Requires no external judge models and is orthogonal to token-level speculative decoding.

Conclusion: ConfSpec resolves the trade-off between accuracy, inference speed, and resource efficiency in step-level speculative reasoning for Chain-of-Thought, enabling significant acceleration without compromising accuracy.

Abstract: Chain-of-Thought reasoning significantly improves the performance of large language models on complex tasks, but incurs high inference latency due to long generation traces. Step-level speculative reasoning aims to mitigate this cost, yet existing approaches face a long-standing trade-off among accuracy, inference speed, and resource efficiency. We propose ConfSpec, a confidence-gated cascaded verification framework that resolves this trade-off. Our key insight is an asymmetry between generation and verification: while generating a correct reasoning step requires substantial model capacity, step-level verification is a constrained discriminative task for which small draft models are well-calibrated within their competence range, enabling high-confidence draft decisions to be accepted directly while selectively escalating uncertain cases to the large target model. Evaluation across diverse workloads shows that ConfSpec achieves up to 2.24$\times$ end-to-end speedups while matching target-model accuracy. Our method requires no external judge models and is orthogonal to token-level speculative decoding, enabling further multiplicative acceleration.

Method: Created a benchmark corpus with 50 de-identified AI-initiated calls with live insurance representatives and 1,000 synthetically generated calls mirroring the same workflow. All calls are annotated with a phase-structured JSON schema covering IVR navigation, patient identification, coverage status, medication checks, and agent identification. Each phase is labeled for Information and Procedural compliance under explicit ask/answer logic.

Result: Defined two novel evaluation tasks: Phase Boundary Detection (span segmentation) and Compliance Verification (IC/PC decisions). Per-phase scores are strong across small, low-latency baselines, but end-to-end reliability is constrained by span-boundary errors. Full-call exact segmentation is low on real calls, showing a gap between conversational fluency and audit-grade evidence.

Conclusion: INSURE-Dial provides the first public benchmark for developing compliance-aware voice agents in healthcare, highlighting challenges in achieving reliable end-to-end performance due to span-boundary errors and the gap between conversational fluency and audit requirements.

Abstract: Administrative phone tasks drain roughly 1 trillion USD annually from U.S. healthcare, with over 500 million insurance-benefit verification calls manually handled in 2024. We introduce INSURE-Dial, to our knowledge the first public benchmark for developing and assessing compliance-aware voice agents for phase-aware call auditing with span-based compliance verification. The corpus includes 50 de-identified, AI-initiated calls with live insurance representatives (mean 71 turns/call) and 1,000 synthetically generated calls that mirror the same workflow. All calls are annotated with a phase-structured JSON schema covering IVR navigation, patient identification, coverage status, medication checks (up to two drugs), and agent identification (CRN), and each phase is labeled for Information and Procedural compliance under explicit ask/answer logic. We define two novel evaluation tasks: (1) Phase Boundary Detection (span segmentation under phase-specific acceptance rules) and (2) Compliance Verification (IC/PC decisions given fixed spans). Per-phase scores are strong across small, low-latency baselines, but end-to-end reliability is constrained by span-boundary errors. On real calls, full-call exact segmentation is low, showing a gap between conversational fluency and audit-grade evidence.

Method: Uses Diffusion Language Models (DLMs) with masked denoising to iteratively refine system prompts. Conditions on interaction traces (user queries, model responses, feedback) for span-level prompt updates. Operates without gradient access or modifying downstream LLM.

Result: DLM-optimized prompts consistently improve performance of frozen target LLMs (e.g., GPT-4o-mini) across diverse benchmarks (τ-bench, SST-2, SST-5). Moderate diffusion step counts provide best balance between refinement quality and stability.

Conclusion: Diffusion-based prompt optimization is a general, model-agnostic, and scalable approach for enhancing LLM performance through iterative prompt refinement, offering flexible updates without requiring model modifications.

Abstract: We propose a diffusion-based framework for prompt optimization that leverages Diffusion Language Models (DLMs) to iteratively refine system prompts through masked denoising. By conditioning on interaction traces, including user queries, model responses, and optional feedback, our method enables flexible, span-level prompt updates without requiring gradient access or modifying the downstream language model. Across diverse benchmarks (e.g., $τ$-bench, SST-2, SST-5), DLM-optimized prompts consistently improve the performance of a frozen target LLM (e.g., GPT-4o-mini). We further show that moderate diffusion step counts provide the best balance between refinement quality and stability. These results highlight diffusion-based prompt optimization as a general, model-agnostic, and scalable approach for enhancing LLM performance through iterative prompt refinement.

Method: Model semantic states as points on a Riemannian manifold and analyze induced projection dynamics in hierarchical optimization with a Dominant Anchor Node and Peripheral Agent Nodes, using both theoretical analysis and empirical validation with RWKV-7 13B GGUF checkpoint.

Result: The limiting semantic configuration is path-independent (insensitive to optimization history), and context dependence controls information content - fully entangled representations force node entropy to vanish. Empirical results show zero hash collisions, mean compliance of 0.50-0.531, and Jaccard-to-anchor similarity of 0.295-0.224.

Conclusion: The theory connects information-theoretic quantities with differential-geometric structure, suggesting an interpretation as an immutable consensus rule that constrains agents to a shared semantic grammar, with implications for understanding semantic convergence in multi-agent systems.

Abstract: Multi-agent language systems can exhibit a failure mode where a shared dominant context progressively absorbs individual semantics, yielding near-uniform behavior across agents. We study this effect under the name Asymptotic Semantic Collapse in Hierarchical Optimization. In a closed linguistic setting with a Dominant Anchor Node whose semantic state has effectively infinite inertia, we show that repeated interactions with Peripheral Agent Nodes drive an asymptotic alignment that minimizes a global loss. We model semantic states as points on a Riemannian manifold and analyze the induced projection dynamics. Two consequences follow. First, the limiting semantic configuration is insensitive to the optimization history: both smooth gradient-style updates and stochastic noisy updates converge to the same topological endpoint, establishing path independence at convergence. Second, the degree of context dependence controls information content: moving from atomic (independent) representations to fully entangled (context-bound) representations forces the node entropy, interpreted as available degrees of freedom, to vanish in the limit. The theory connects information-theoretic quantities with differential-geometric structure and suggests an interpretation as an immutable consensus rule that constrains agents to a shared semantic grammar. A lightweight dataset-free benchmark on an RWKV-7 13B GGUF checkpoint complements the analysis, reporting zero hash collisions, mean compliance of 0.50 under greedy decoding and 0.531 under stochastic decoding, and final Jaccard-to-anchor similarity values of 0.295 and 0.224, respectively.

Method: Proposes a bi-encoder design that decouples the process into separate label encoder and context encoder, removing the context-window bottleneck. Uses pre-computed label embeddings for efficiency and introduces GLiNKER framework for entity linking across massive knowledge bases.

Result: Achieves state-of-the-art zero-shot performance with 61.5% Micro-F1 on CrossNER benchmark. Enables simultaneous recognition of thousands to millions of entity types with up to 130x throughput improvement at 1024 labels compared to previous uni-encoder approaches.

Conclusion: GLiNER-bi-Encoder successfully harmonizes zero-shot flexibility with industrial-scale efficiency, enabling practical deployment of NER systems that can handle massive numbers of entity types while maintaining strong generalization capabilities.

Abstract: This paper introduces GLiNER-bi-Encoder, a novel architecture for Named Entity Recognition (NER) that harmonizes zero-shot flexibility with industrial-scale efficiency. While the original GLiNER framework offers strong generalization, its joint-encoding approach suffers from quadratic complexity as the number of entity labels increases. Our proposed bi-encoder design decouples the process into a dedicated label encoder and a context encoder, effectively removing the context-window bottleneck. This architecture enables the simultaneous recognition of thousands, and potentially millions, of entity types with minimal overhead. Experimental results demonstrate state-of-the-art zero-shot performance, achieving 61.5 percent Micro-F1 on the CrossNER benchmark. Crucially, by leveraging pre-computed label embeddings, GLiNER-bi-Encoder achieves up to a 130 times throughput improvement at 1024 labels compared to its uni-encoder predecessors. Furthermore, we introduce GLiNKER, a modular framework that leverages this architecture for high-performance entity linking across massive knowledge bases such as Wikidata.

Method: Leverages decoder-only small language models (SLMs) with lightweight LoRA/PEFT heads for each specific metric, enabling hundreds of specialized metrics to run concurrently on a single GPU with deterministic evaluation.

Result: Matches accuracy of state-of-the-art LLM-based evaluators while reducing inference cost by over 80x and latency by over 20x, processing over 100B tokens per month with $30M annual cost savings.

Conclusion: Luna-2 provides a practical solution for real-time guardrails by making evaluation accurate, cheap, and fast while being privacy-preserving and deployable locally next to AI systems.

Abstract: Real-time guardrails require evaluation that is accurate, cheap, and fast - yet today’s default, LLM-as-a-judge (LLMAJ), is slow, expensive, and operationally non-deterministic due to multi-token generation. We present Luna-2, a novel architecture that leverages decoder-only small language models (SLMs) into a deterministic evaluation model to reliably compute complex task-specific LLMAJ metrics (e.g. toxicity, hallucination, tool selection quality, etc.) at an accuracy at par or higher than LLMAJ using frontier LLMs while drastically reducing the cost and latency of computation. Each metric is implemented as a lightweight LoRA/PEFT head on top of a shared SLM backbone, enabling hundreds of specialized metrics to run concurrently on a single GPU, deployable locally next to AI systems in a privacy-preserving and latency optimizing manner. Across content safety and hallucination benchmarks, Luna-2 matches the accuracy of state-of-the-art LLM-based evaluators while reducing inference cost by over 80x and latency by over 20x. In this paper, we outline the model architecture, training methodology and report real-world empirical results on accuracy, latency, and throughput results. In production, Luna-2 is protecting 100M+ AI sessions and processing over 100B tokens per month for our customers with eval cost savings of over $30M annually.

Method: Proposes ReHear framework that integrates an instruction-tuned, audio-aware large language model into the self-training loop. Unlike text-only correctors, it conditions the LLM on both ASR hypothesis and source audio to recover phonetically accurate transcripts. Uses refined pseudo-labels to fine-tune ASR model iteratively.

Result: Experimental results across diverse benchmarks show ReHear effectively mitigates error propagation and consistently outperforms both supervised and pseudo-labeling baselines.

Conclusion: ReHear provides an effective solution to confirmation bias in semi-supervised ASR by leveraging audio-aware LLMs for pseudo-label refinement, leading to improved performance over conventional approaches.

Abstract: Semi-supervised learning in automatic speech recognition (ASR) typically relies on pseudo-labeling, which often suffers from confirmation bias and error accumulation due to noisy supervision. To address this limitation, we propose ReHear, a framework for iterative pseudo-label refinement that integrates an instruction-tuned, audio-aware large language model (LLM) into the self-training loop. Unlike conventional text-based correctors, our approach conditions the LLM on both the ASR hypothesis and the source audio, allowing it to recover phonetically accurate transcripts even from severe recognition errors. These refined pseudo-labels serve as high-fidelity targets for fine-tuning the ASR model in an iterative cycle. Experimental results across diverse benchmarks demonstrate that ReHear effectively mitigates error propagation, consistently outperforming both supervised and pseudo-labeling baselines.

Method: DP-RFT uses online reinforcement learning where an LLM generates synthetic samples, and DP-protected nearest-neighbor votes from a private corpus serve as reward signals. The LLM learns via Proximal Policy Optimization (PPO) to maximize expected DP votes without direct access to private data.

Result: DP-RFT closes the gap between private evolution and DP finetuning methods in terms of fidelity and downstream utility for long-form and domain-specific synthetic data generation (news articles, meeting transcripts, medical abstracts).

Conclusion: DP-RFT enables training LLMs to generate high-quality synthetic text with formal privacy guarantees while respecting private data boundaries, without requiring direct access to individual private examples.

Abstract: Differentially private (DP) synthetic data generation plays a pivotal role in developing large language models (LLMs) on private data, where data owners cannot provide eyes-on access to individual examples. Generating DP synthetic data typically involves a difficult trade-off. On one hand, DP finetuning methods train an LLM as a synthetic data generator with formal privacy guarantees, yet it still requires the raw content of private examples for model training. However, methods that avoid direct exposure to private data are bounded by an off-the-shelf, un-finetuned model, whose outputs often lack domain fidelity. Can we train an LLM to generate high-quality synthetic text without eyes-on access to individual private examples? In this work, we introduce Differentially Private Reinforcement Fine-Tuning (DP-RFT), an online reinforcement learning algorithm for synthetic data generation with LLMs. DP-RFT leverages DP-protected nearest-neighbor votes from an eyes-off private corpus as a reward signal for on-policy synthetic samples generated by an LLM. The LLM iteratively learns to generate synthetic data to maximize the expected DP votes through Proximal Policy Optimization (PPO). We evaluate DP-RFT for long-form and domain-specific synthetic data generation, such as news articles, meeting transcripts, and medical article abstracts. Our experiments show that DP-RFT closes the gap between private evolution and DP finetuning methods in terms of the fidelity and downstream utility of the generated synthetic data, while respecting the private data boundary.

Method: Unified pipeline using frozen CLIP-style vision-language encoders and multilingual BGE M3 encoder, with lightweight modules: logistic regression and LLM-based sentence-type predictor, idiom synonym substitution, distractor-aware scoring, and Borda rank fusion.

Result: Improved from CLIP baseline (26.7% Top-1 on English dev) to 60.0% Top-1 on English and 60.0% Top-1 (0.822 NDCG@5) in zero-shot transfer to Portuguese. On multilingual blind test: average Top-1/NDCG scores of 0.35/0.73 for Subtask A (image+text) and 0.32/0.71 for Subtask B (text-only) across 15 languages.

Conclusion: Effective idiom disambiguation is feasible without fine-tuning large multimodal encoders. Idiom-aware rewriting is the main performance contributor, while sentence-type prediction and multimodal fusion enhance robustness.

Abstract: Multimodal models struggle with idiomatic expressions due to their non-compositional meanings, a challenge amplified in multilingual settings. We introduced PolyFrame, our system for the MWE-2026 AdMIRe2 shared task on multimodal idiom disambiguation, featuring a unified pipeline for both image+text ranking (Subtask A) and text-only caption ranking (Subtask B). All model variants retain frozen CLIP-style vision–language encoders and the multilingual BGE M3 encoder, training only lightweight modules: a logistic regression and LLM-based sentence-type predictor, idiom synonym substitution, distractor-aware scoring, and Borda rank fusion. Starting from a CLIP baseline (26.7% Top-1 on English dev, 6.7% on English test), adding idiom-aware paraphrasing and explicit sentence-type classification increased performance to 60.0% Top-1 on English and 60.0% Top-1 (0.822 NDCG@5) in zero-shot transfer to Portuguese. On the multilingual blind test, our systems achieved average Top-1/NDCG scores of 0.35/0.73 for Subtask A and 0.32/0.71 for Subtask B across 15 languages. Ablation results highlight idiom-aware rewriting as the main contributor to performance, while sentence-type prediction and multimodal fusion enhance robustness. These findings suggest that effective idiom disambiguation is feasible without fine-tuning large multimodal encoders.

Method: Developed a large-scale lexicon capturing descriptive norms of anxiety associations for more than 20,000 English MWEs, assessing reliability and studying prevalence across different word sequence lengths.

Result: Created a reliable anxiety association lexicon for MWEs, analyzed prevalence of anxiety- and calmness-associated expressions across 2-, 3-, and 4-word sequences, and studied compositionality of anxiety associations.

Conclusion: The lexicon enables diverse anxiety-related research across psychology, NLP, public health, and social sciences, addressing a significant gap in studying anxiety at the multiword expression level.

Abstract: Anxiety is the unease about a possible future negative outcome. In recent years, there has been growing interest in understanding how anxiety relates to our health, well-being, body, mind, and behaviour. This includes work on lexical resources for word-anxiety association. However, there is very little anxiety-related work on larger units of text such as multiword expressions (MWE). Here, we introduce the first large-scale lexicon capturing descriptive norms of anxiety associations for more than 20k English MWEs. We show that the anxiety associations are highly reliable. We use the lexicon to study prevalence of different types of anxiety- and calmness-associated MWEs; and how that varies across two-, three-, and four-word sequences. We also study the extent to which the anxiety association of MWEs is compositional (due to its constituent words). The lexicon enables a wide variety of anxiety-related research in psychology, NLP, public health, and social sciences. The lexicon is freely available: https://saifmohammad.com/worrylex.html

Method: Novel retrieval strategy collects evidence for both original and negated claims from Wikipedia, PubMed, and Google. Evidence is filtered, deduplicated, aggregated across sources, then used with LLMs for verification. Includes disagreement analysis via confidence scores.

Result: Knowledge aggregation improves claim verification across four benchmark datasets with five LLMs. Reveals source-specific reasoning differences and demonstrates importance of diverse evidence for reliable systems.

Conclusion: Embracing diversity, contradiction, and aggregation in evidence is crucial for building reliable and transparent claim verification systems that better reflect real-world information complexity.

Abstract: The spread of misinformation across digital platforms can pose significant societal risks. Claim verification, a.k.a. fact-checking, systems can help identify potential misinformation. However, their efficacy is limited by the knowledge sources that they rely on. Most automated claim verification systems depend on a single knowledge source and utilize the supporting evidence from that source; they ignore the disagreement of their source with others. This limits their knowledge coverage and transparency. To address these limitations, we present a novel system for open-domain claim verification (ODCV) that leverages large language models (LLMs), multi-perspective evidence retrieval, and cross-source disagreement analysis. Our approach introduces a novel retrieval strategy that collects evidence for both the original and the negated forms of a claim, enabling the system to capture supporting and contradicting information from diverse sources: Wikipedia, PubMed, and Google. These evidence sets are filtered, deduplicated, and aggregated across sources to form a unified and enriched knowledge base that better reflects the complexity of real-world information. This aggregated evidence is then used for claim verification using LLMs. We further enhance interpretability by analyzing model confidence scores to quantify and visualize inter-source disagreement. Through extensive evaluation on four benchmark datasets with five LLMs, we show that knowledge aggregation not only improves claim verification but also reveals differences in source-specific reasoning. Our findings underscore the importance of embracing diversity, contradiction, and aggregation in evidence for building reliable and transparent claim verification systems

Method: Proposes a formal model $S_t=(X,d_t,P_t)$ combining embedding geometry with local diffusion, introduces node-level neighborhood drift measures, coarse Ricci curvature for local contractivity of semantic diffusion, and recursive drift for stability of iterated semantic operators. Also introduces bridge mass as a predictor of future neighborhood rewiring.

Result: The paper provides theoretical framework and test contracts for the proposed model, but empirical performance is deferred to subsequent studies, so no experimental results are presented in this work.

Conclusion: The paper establishes a unified theoretical foundation for semantic drift analysis that can relate multiple observed signals, with bridge mass proposed as a novel predictor of semantic rewiring, though empirical validation remains future work.

Abstract: Most semantic drift studies report multiple signals e.g., embedding displacement, neighbor changes, distributional divergence, and recursive trajectory instability, without a shared explanatory theory that relates them. This paper proposes a formalization of these signals in one time-indexed substrate, $S_t=(X,d_t,P_t)$, combining embedding geometry with local diffusion. Within this substrate, node-level neighborhood drift measures changes in local conditional distributions, coarse Ricci curvature measures local contractivity of semantic diffusion, and recursive drift probes stability of iterated semantic operators. This manuscript specifies the formal model, assumptions, and tests that can refute the model. Herein, the paper introduces bridge mass, a node-level aggregate of incident negative curvature, as a predictor of future neighborhood rewiring. This paper provides the theory and test contracts; empirical performance is deferred to subsequent studies.

Method: Proposes Cooperative Retrieval-Augmented Generation (CoRAG) framework that treats reranker and generator as cooperative multi-agents jointly optimized toward shared task objectives rather than in pipeline dependency.

Result: Experimental results show good generalization and improved generation stability, achieving strong performance with only ~10K PopQA training samples.

Conclusion: CoRAG’s cooperative multi-agent approach overcomes limitations of asymmetric dependency in traditional RAG systems, enabling more stable and effective knowledge-intensive generation.

Abstract: Retrieval-Augmented Generation (RAG) has demonstrated strong effectiveness in knowledge-intensive tasks by grounding language generation in external evidence. Despite its success, many existing RAG systems are built based on a ranking-centric, asymmetric dependency paradigm, where the generation quality of the generator is highly dependent on reranking results of the reranker. To overcome this limitation, we reformulate RAG as a cooperative multi-agent decision-making problem and propose Cooperative Retrieval-Augmented Generation (CoRAG), a framework in which the reranker and the generator act as peer decision-makers rather than being connected through an asymmetric dependency pipeline. By jointly optimizing their behaviors toward a shared task objective, the reranker and generator are encouraged to cooperate, ensuring that document reranking and generation work in concert to improve the final response. Experimental results demonstrate good generalization and improved generation stability of CoRAG, even when the model is trained on only around 10K PopQA samples. Our model released in https://anonymous.4open.science/r/CoRAG-D63F

Method: Created a comprehensive benchmark with 210 number reading tasks across six contextual categories (pure numerals, addresses, dates, quantities, prices). Evaluated 71 models from 10 providers using four prompting strategies (zero-shot, zero-shot CoT, few-shot, few-shot CoT) with 59,010 test cases, tracking extraction methods for structured output generation.

Result: Found substantial performance variation (14.29% to 99.05% accuracy). Few-shot Chain-of-Thought prompting achieved 2.8x higher accuracy than zero-shot approaches (80.06% vs 28.76%). Models with elite accuracy often produced unstructured output, with only 6 models consistently generating structured output across all test cases.

Conclusion: Numerical accuracy and instruction-following represent distinct capabilities in LLMs. The benchmark establishes baselines for Arabic number comprehension and provides guidance for model selection in production Arabic NLP systems.

Abstract: We present ArabicNumBench, a comprehensive benchmark for evaluating large language models on Arabic number reading tasks across Eastern Arabic-Indic numerals (0-9 in Arabic script) and Western Arabic numerals (0-9). We evaluate 71 models from 10 providers using four prompting strategies (zero-shot, zero-shot CoT, few-shot, few-shot CoT) on 210 number reading tasks spanning six contextual categories: pure numerals, addresses, dates, quantities, and prices. Our evaluation comprises 59,010 individual test cases and tracks extraction methods to measure structured output generation. Evaluation reveals substantial performance variation, with accuracy ranging from 14.29% to 99.05% across models and strategies. Few-shot Chain-of-Thought prompting achieves 2.8x higher accuracy than zero-shot approaches (80.06% vs 28.76%). A striking finding emerges: models achieving elite accuracy (98-99%) often produce predominantly unstructured output, with most responses lacking Arabic CoT markers. Only 6 models consistently generate structured output across all test cases, while the majority require fallback extraction methods despite high numerical accuracy. Comprehensive evaluation of 281 model-strategy combinations demonstrates that numerical accuracy and instruction-following represent distinct capabilities, establishing baselines for Arabic number comprehension and providing actionable guidance for model selection in production Arabic NLP systems.

Method: SALAD (Sample-efficient Alignment with Learning through Active selection and cross-modal Distillation) combines cross-modal distillation with targeted synthetic data to improve speech-text alignment while mitigating forgetting of text capabilities during adaptation.

Result: Applied to 3B and 7B LLMs, SALAD achieves competitive performance with strong open-weight models across broad-domain benchmarks in knowledge, language understanding, and reasoning, while training on over an order of magnitude less speech data from public corpora.

Conclusion: SALAD provides a data-efficient solution to the text-speech understanding gap, enabling speech-adapted LLMs to perform competitively with much less training data through targeted alignment techniques.

Abstract: Large Language Models (LLMs) can be adapted to extend their text capabilities to speech inputs. However, these speech-adapted LLMs consistently underperform their text-based counterparts–and even cascaded pipelines–on language understanding tasks. We term this shortfall the text-speech understanding gap: the performance drop observed when a speech-adapted LLM processes spoken inputs relative to when the original text-based LLM processes the equivalent text. Recent approaches to narrowing this gap either rely on large-scale speech synthesis of text corpora, which is costly and heavily dependent on synthetic data, or on large-scale proprietary speech datasets, which are not reproducible. As a result, there remains a need for more data-efficient alternatives for closing the text-speech understanding gap. In this work, we analyze the gap as driven by two factors: (i) forgetting of text capabilities during adaptation, and (ii) cross-modal misalignment between speech and text. Based on this analysis, we introduce SALAD–Sample-efficient Alignment with Learning through Active selection and cross-modal Distillation–which combines cross-modal distillation with targeted synthetic data to improve alignment while mitigating forgetting. Applied to 3B and 7B LLMs, SALAD achieves competitive performance with a strong open-weight model across broad-domain benchmarks in knowledge, language understanding, and reasoning, while training on over an order of magnitude less speech data from public corpora.

Method: Created a holistic benchmark with 7 subtasks (QA, Sentiment Analysis, Toxicity Detection, Causal Reasoning, NLI, Summarization, Translation) using rigorous native-speaker-driven process. Conducted large-scale evaluation of open-weight and commercial LLMs.

Result: Burmese performance depends more on architectural design, language representation, and instruction tuning than model scale alone. Southeast Asia regional fine-tuning and newer model generations yield substantial gains.

Conclusion: BURMESE-SAN provides a systematic evaluation framework for Burmese LLMs, revealing key factors affecting performance and supporting progress in low-resource languages. Released as public leaderboard.

Abstract: We introduce BURMESE-SAN, the first holistic benchmark that systematically evaluates large language models (LLMs) for Burmese across three core NLP competencies: understanding (NLU), reasoning (NLR), and generation (NLG). BURMESE-SAN consolidates seven subtasks spanning these competencies, including Question Answering, Sentiment Analysis, Toxicity Detection, Causal Reasoning, Natural Language Inference, Abstractive Summarization, and Machine Translation, several of which were previously unavailable for Burmese. The benchmark is constructed through a rigorous native-speaker-driven process to ensure linguistic naturalness, fluency, and cultural authenticity while minimizing translation-induced artifacts. We conduct a large-scale evaluation of both open-weight and commercial LLMs to examine challenges in Burmese modeling arising from limited pretraining coverage, rich morphology, and syntactic variation. Our results show that Burmese performance depends more on architectural design, language representation, and instruction tuning than on model scale alone. In particular, Southeast Asia regional fine-tuning and newer model generations yield substantial gains. Finally, we release BURMESE-SAN as a public leaderboard to support systematic evaluation and sustained progress in Burmese and other low-resource languages. https://leaderboard.sea-lion.ai/detailed/MY

Method: Pre-trains on 600B tokens to establish bidirectional mapping between raw audio and high-level conceptual space via rich captions, then uses caption-then-process workflow during fine-tuning to simulate cognitive reasoning for diverse tasks.

Result: Outperforms Qwen-2.5-Omni on MMAU and AIRBench for audio understanding, surpasses CosyVoice3 and TangoFlux in generation quality, and can synthesize arbitrary compositions of speech, music, and sound effects.

Conclusion: Bagpiper achieves unified understanding and generation for general audio, representing one of the first works to accomplish this holistic approach to audio processing.

Abstract: Current audio foundation models typically rely on rigid, task-specific supervision, addressing isolated factors of audio rather than the whole. In contrast, human intelligence processes audio holistically, seamlessly bridging physical signals with abstract cognitive concepts to execute complex tasks. Grounded in this philosophy, we introduce Bagpiper, an 8B audio foundation model that interprets physical audio via rich captions, i.e., comprehensive natural language descriptions that encapsulate the critical cognitive concepts inherent in the signal (e.g., transcription, audio events). By pre-training on a massive corpus of 600B tokens, the model establishes a robust bidirectional mapping between raw audio and this high-level conceptual space. During fine-tuning, Bagpiper adopts a caption-then-process workflow, simulating an intermediate cognitive reasoning step to solve diverse tasks without task-specific priors. Experimentally, Bagpiper outperforms Qwen-2.5-Omni on MMAU and AIRBench for audio understanding and surpasses CosyVoice3 and TangoFlux in generation quality, capable of synthesizing arbitrary compositions of speech, music, and sound effects. To the best of our knowledge, Bagpiper is among the first works that achieve unified understanding generation for general audio. Model, data, and code are available at Bagpiper Home Page.

Method: Introduces a psychologically grounded metacognitive framework operationalizing Ann Brown’s regulatory cycle (Planning, Monitoring, Evaluation) as structured prompting architecture. Integrates this with a lightweight dual-process MetaController for adaptive effort allocation.

Result: Explicit regulatory structuring substantially improves error diagnosis and yields a threefold increase in successful self-correction across diverse reasoning benchmarks (GSM8K, CRUXEval, MBPP, AIME, CorrectBench, TruthfulQA) using Llama-3 and Qwen-3 (8B). Human evaluations show 84% preference for trustworthiness and metacognitive self-awareness over standard baselines.

Conclusion: Grounding LLM reasoning in established cognitive theory offers a principled path toward more transparent and diagnostically robust AI systems.

Abstract: Large Language Models (LLMs) demonstrate strong reasoning performance, yet their ability to reliably monitor, diagnose, and correct their own errors remains limited. We introduce a psychologically grounded metacognitive framework that operationalizes Ann Brown’s regulatory cycle (Planning, Monitoring, and Evaluation) as a structured prompting architecture, and study its integration within a lightweight dual-process MetaController for adaptive effort allocation. Across diverse reasoning and diagnostic benchmarks (GSM8K, CRUXEval, MBPP, AIME, CorrectBench, and TruthfulQA) using Llama-3 and Qwen-3 (8B), explicit regulatory structuring substantially improves error diagnosis and yields a threefold increase in successful self-correction. Blinded human evaluations over 580 query pairs show an 84% aggregate preference for trustworthiness and metacognitive self-awareness over standard and Chain-of-Thought baselines. Grounding LLM reasoning in established cognitive theory offers a principled path toward more transparent and diagnostically robust AI systems.

Method: EvalSense provides a flexible framework with two key components: (1) interactive guide for evaluation method selection, and (2) automated meta-evaluation tools that assess evaluation reliability using perturbed data. It supports various model providers and evaluation strategies.

Result: Demonstrated effectiveness in a case study involving clinical note generation from doctor-patient dialogues using an open dataset. The framework is open-source and publicly available.

Conclusion: EvalSense addresses the need for robust LLM evaluation by providing a flexible, extensible framework that helps users select appropriate evaluation methods and assess their reliability, particularly valuable for sensitive domains like healthcare.

Abstract: Robust and comprehensive evaluation of large language models (LLMs) is essential for identifying effective LLM system configurations and mitigating risks associated with deploying LLMs in sensitive domains. However, traditional statistical metrics are poorly suited to open-ended generation tasks, leading to growing reliance on LLM-based evaluation methods. These methods, while often more flexible, introduce additional complexity: they depend on carefully chosen models, prompts, parameters, and evaluation strategies, making the evaluation process prone to misconfiguration and bias. In this work, we present EvalSense, a flexible, extensible framework for constructing domain-specific evaluation suites for LLMs. EvalSense provides out-of-the-box support for a broad range of model providers and evaluation strategies, and assists users in selecting and deploying suitable evaluation methods for their specific use-cases. This is achieved through two unique components: (1) an interactive guide aiding users in evaluation method selection and (2) automated meta-evaluation tools that assess the reliability of different evaluation approaches using perturbed data. We demonstrate the effectiveness of EvalSense in a case study involving the generation of clinical notes from unstructured doctor-patient dialogues, using a popular open dataset. All code, documentation, and assets associated with EvalSense are open-source and publicly available at https://github.com/nhsengland/evalsense.

Method: Two core components: (1) Automated pipeline to extract structured research knowledge from scientific literature, and (2) “Next Idea Prediction” training paradigm that models idea generation as iterative prediction, evaluation, and refinement.

Result: DeepInnovator-14B significantly outperforms untrained baselines with win rates of 80.53%-93.81%, achieving performance comparable to leading LLMs in generating novel research ideas.

Conclusion: Provides a scalable training pathway for building research agents with genuine innovative capability and will open-source the dataset to advance the community.

Abstract: The application of Large Language Models (LLMs) in accelerating scientific discovery has garnered increasing attention, with a key focus on constructing research agents endowed with innovative capability, i.e., the ability to autonomously generate novel and significant research ideas. Existing approaches predominantly rely on sophisticated prompt engineering and lack a systematic training paradigm. To address this, we propose DeepInnovator, a training framework designed to trigger the innovative capability of LLMs. Our approach comprises two core components. (1) Standing on the shoulders of giants''. We construct an automated data extraction pipeline to extract and organize structured research knowledge from a vast corpus of unlabeled scientific literature. (2) Conjectures and refutations’’. We introduce a ``Next Idea Prediction’’ training paradigm, which models the generation of research ideas as an iterative process of continuously predicting, evaluating, and refining plausible and novel next idea. Both automatic and expert evaluations demonstrate that our DeepInnovator-14B significantly outperforms untrained baselines, achieving win rates of 80.53%-93.81%, and attains performance comparable to that of current leading LLMs. This work provides a scalable training pathway toward building research agents with genuine, originative innovative capability, and will open-source the dataset to foster community advancement. Source code and data are available at: https://github.com/HKUDS/DeepInnovator.

Method: Developed evaluation framework pairing AI psychotherapists with simulated patient agents equipped with dynamic cognitive-affective models, assessing therapy sessions against comprehensive quality of care and risk ontology. Applied to Alcohol Use Disorder case with six AI agents evaluated against 15 clinically-validated patient personas.

Result: Large-scale simulation (369 sessions) revealed critical safety gaps including validation of patient delusions (“AI Psychosis”) and failure to de-escalate suicide risk. Interactive dashboard validated with stakeholders effectively enables auditing of AI psychotherapy systems.

Conclusion: Underscores critical safety risks of AI-provided mental health support and necessity of simulation-based clinical red teaming before deployment.

Abstract: Large Language Models (LLMs) are increasingly utilized for mental health support; however, current safety benchmarks often fail to detect the complex, longitudinal risks inherent in therapeutic dialogue. We introduce an evaluation framework that pairs AI psychotherapists with simulated patient agents equipped with dynamic cognitive-affective models and assesses therapy session simulations against a comprehensive quality of care and risk ontology. We apply this framework to a high-impact test case, Alcohol Use Disorder, evaluating six AI agents (including ChatGPT, Gemini, and Character.AI) against a clinically-validated cohort of 15 patient personas representing diverse clinical phenotypes. Our large-scale simulation (N=369 sessions) reveals critical safety gaps in the use of AI for mental health support. We identify specific iatrogenic risks, including the validation of patient delusions (“AI Psychosis”) and failure to de-escalate suicide risk. Finally, we validate an interactive data visualization dashboard with diverse stakeholders, including AI engineers and red teamers, mental health professionals, and policy experts (N=9), demonstrating that this framework effectively enables stakeholders to audit the “black box” of AI psychotherapy. These findings underscore the critical safety risks of AI-provided mental health support and the necessity of simulation-based clinical red teaming before deployment.

Method: Introduces W5H2 structured intent decomposition framework, applies V-measure decomposition for cache-key evaluation, uses SetFit with 8 examples per class, and implements five-tier cascade with risk-controlled selective prediction guarantees via RCPS.

Result: Achieves 91.1%+/-1.7% accuracy on MASSIVE in ~2ms (vs 37.9% for GPTCache and 68.8% for 20B-parameter LLM at 3,447ms), 55.3% on NyayaBench v2 with cross-lingual transfer across 30 languages, handles 85% of interactions locally, projects 97.5% cost reduction.

Conclusion: W5H2 framework enables efficient caching for personal AI agents by addressing the root cause of cache failures, achieving high accuracy with low latency, significant cost reductions, and cross-lingual capabilities with formal guarantees.

Abstract: Personal AI agents incur substantial cost via repeated LLM calls. We show existing caching methods fail: GPTCache achieves 37.9% accuracy on real benchmarks; APC achieves 0-12%. The root cause is optimizing for the wrong property – cache effectiveness requires key consistency and precision, not classification accuracy. We observe cache-key evaluation reduces to clustering evaluation and apply V-measure decomposition to separate these on n=8,682 points across MASSIVE, BANKING77, CLINC150, and NyayaBench v2, our new 8,514-entry multilingual agentic dataset (528 intents, 20 W5H2 classes, 63 languages). We introduce W5H2, a structured intent decomposition framework. Using SetFit with 8 examples per class, W5H2 achieves 91.1%+/-1.7% on MASSIVE in ~2ms – vs 37.9% for GPTCache and 68.8% for a 20B-parameter LLM at 3,447ms. On NyayaBench v2 (20 classes), SetFit achieves 55.3%, with cross-lingual transfer across 30 languages. Our five-tier cascade handles 85% of interactions locally, projecting 97.5% cost reduction. We provide risk-controlled selective prediction guarantees via RCPS with nine bound families.

Method: Created Yor-Sarc dataset with 436 instances annotated by three native speakers using a culture-specific protocol, analyzed inter-annotator agreement using Fleiss’ κ and Cohen’s κ, preserved soft labels for uncertainty-aware modeling.

Result: Achieved substantial to almost perfect agreement (Fleiss’ κ=0.7660; pairwise Cohen’s κ=0.6732-0.8743), with 83.3% unanimous consensus and 16.7% majority-agreement cases as soft labels.

Conclusion: Yor-Sarc facilitates research on semantic interpretation and culturally informed NLP for low-resource African languages, with annotation protocol supporting replication in other African languages.

Abstract: Sarcasm detection poses a fundamental challenge in computational semantics, requiring models to resolve disparities between literal and intended meaning. The challenge is amplified in low-resource languages where annotated datasets are scarce or nonexistent. We present \textbf{Yor-Sarc}, the first gold-standard dataset for sarcasm detection in Yorùbá, a tonal Niger-Congo language spoken by over $50$ million people. The dataset comprises 436 instances annotated by three native speakers from diverse dialectal backgrounds using an annotation protocol specifically designed for Yorùbá sarcasm by taking culture into account. This protocol incorporates context-sensitive interpretation and community-informed guidelines and is accompanied by a comprehensive analysis of inter-annotator agreement to support replication in other African languages. Substantial to almost perfect agreement was achieved (Fleiss’ $κ= 0.7660$; pairwise Cohen’s $κ= 0.6732$–$0.8743$), with $83.3%$ unanimous consensus. One annotator pair achieved almost perfect agreement ($κ= 0.8743$; $93.8%$ raw agreement), exceeding a number of reported benchmarks for English sarcasm research works. The remaining $16.7%$ majority-agreement cases are preserved as soft labels for uncertainty-aware modelling. Yor-Sarc\footnote{https://github.com/toheebadura/yor-sarc} is expected to facilitate research on semantic interpretation and culturally informed NLP for low-resource African languages.

Method: A multi-agent LLM pipeline that intercepts Whisper’s initial transcript, applies specialized agents for domain context identification, named entity recognition, and jargon detection, then generates compact prompts to guide Whisper’s decoder for improved transcription.

Result: On 421 NBA basketball commentary segments, the pipeline achieved a 17.0% relative reduction in word error rate (from 0.217 to 0.180, p<0.001), with improvements in 40.1% of segments and degradation in only 7.1%.

Conclusion: Prompt-based augmentation can deliver scalable domain adaptation for ASR, offering a practical alternative to costly model fine-tuning, particularly for domains with specialized vocabulary like sports commentary.

Abstract: Domain-specific speech remains a persistent challenge for automatic speech recognition (ASR), even for state-of-the-art systems like OpenAI’s Whisper. We introduce Whisper: Courtside Edition, a novel multi-agent large language model (LLM) pipeline that enhances Whisper transcriptions without retraining. The pipeline intercepts Whisper’s initial transcript, applies specialized LLM agents for domain context identification, named entity recognition, and jargon detection, and generates compact prompts that guide Whisper’s decoder. Evaluated on 421 NBA basketball commentary segments (a domain characterized by dense proper nouns and technical terminology) our best pipeline achieves a statistically significant 17.0% relative reduction in word error rate (WER; from 0.217 to 0.180, p<0.001). Improvements are observed in 40.1% of segments with degradation in only 7.1%, substantially outperforming direct transcript post-editing. These results demonstrate that prompt-based augmentation can deliver scalable domain adaptation for ASR, offering a practical alternative to costly model fine-tuning.

Method: Analyzed trajectories from Toolathlon benchmark with 22 frontier models attempting 108 real-world tool-use tasks across 3 independent runs, comparing successful vs failed runs within model×task units where the same model succeeded on some runs and failed on others due to LLM sampling stochasticity alone.

Result: Successful runs adhere significantly more closely to canonical solution paths than failed runs (+0.060 Jaccard, p<0.0001). The adherence gap emerges gradually, and each off-canonical tool call raises probability of subsequent off-canonical calls by 22.7 percentage points. A simple monitor restarting bottom tercile runs based on mid-trajectory adherence lifts success rates by +8.8 percentage points.

Conclusion: Agent reliability cannot be improved by capability scaling alone; reliability failures stem from stochastic drift from canonical solution paths. Monitoring adherence to canonical paths provides an actionable intervention to improve agent performance.

Abstract: Why do language agents fail on tasks they are capable of solving? We argue that many such failures are reliability failures caused by stochastic drift from a task’s latent solution structure, not capability failures. Every well-defined tool-use task imposes a canonical solution path (i.e., a convergent set of tool invocations shared across successful runs) and agent success depends critically on whether a trajectory stays within this path’s operating envelope. We establish this causally using a natural experiment that holds model capability and task difficulty fixed by construction. We analyze trajectories from the Toolathlon benchmark: 22 frontier models each attempt 108 real-world tool-use tasks across 3 independent runs, yielding 515 model$\times$task units where the same model succeeds on some runs and fails on others due to LLM sampling stochasticity alone. Within these units, successful runs adhere significantly more closely to the canonical solution path than failed runs ($+$0.060 Jaccard, $p<0.0001$, $n=488$ units, 95% CI [+0.043, +0.077]). This result survives six robustness checks including cross-model-family leave-one-out validation. Critically, the causal mechanism is gradual and self-reinforcing: the adherence gap is statistically indistinguishable from zero through the first 50% of the trajectory, ruling out early-branching selection bias, and each off-canonical tool call raises the probability that the next call is also off-canonical by 22.7 percentage points ($\hatβ=+0.227$, $p<0.0001$), more than doubling the baseline rate. These findings imply that agent reliability cannot be improved by capability scaling alone, but offer a highly actionable intervention: a simple monitor that restarts the bottom tercile of runs based on mid-trajectory canonical adherence lifts success rates by $+$8.8 percentage points among intervened runs.

Method: Proposes COIN (COntext-INdependent editing) framework that encourages models to focus on knowledge within local scope rather than memorizing contextual patterns, addressing the inherent Context Reliance issue in gradient-based optimization.

Result: COIN reduces Context Reliance by 45.2% and outperforms strong baselines by 23.6% in editing success rate, demonstrating effectiveness in robust LLM editing.

Conclusion: Mitigating Context Reliance is crucial for robust LLM editing, and the COIN framework provides an effective solution by encouraging context-independent knowledge acquisition.

Abstract: Editing Large language models (LLMs) with real-world, unstructured knowledge is essential for correcting and updating their internal parametric knowledge. In this work, we revisit the fundamental next-token prediction (NTP) as a candidate paradigm for unstructured editing. We identify Context Reliance as a critical failure mode of NTP-based approaches, where knowledge acquired from edited text becomes highly dependent on its preceding context, leading to recall failures when that context is absent during inference. This hypothesis is supported by our empirical validation that prepending context during inference recovers knowledge recall. We further theoretically demonstrate that Context Reliance is an inherent consequence of gradient-based optimization, which tends to bind acquired knowledge to a specific aggregated contextual representation. To address this, we propose a simple yet effective COntext-INdependent editing framework (COIN), encouraging model to focus on knowledge within local scope rather than memorizing contextual patterns. Evaluations show that COIN reduces Context Reliance by 45.2% and outperforms strong baselines by 23.6% in editing success rate, highlighting the vital role of mitigating Context Reliance for robust editing.

Method: Proposes IAPO (Information-Aware Post-training Optimization), which assigns token-wise advantages based on each token’s conditional mutual information with the final answer. This provides a principled mechanism for identifying informative reasoning steps and suppressing low-utility exploration.

Result: IAPO consistently improves reasoning accuracy while reducing reasoning length by up to 36%, outperforming existing token-efficient RL methods across various reasoning datasets. Theoretical analysis shows it can induce monotonic reductions in reasoning verbosity without harming correctness.

Conclusion: Information-aware advantage shaping is a powerful and general direction for token-efficient post-training, providing explicit control over reasoning allocation while maintaining accuracy.

Abstract: Large language models increasingly rely on long chains of thought to improve accuracy, yet such gains come with substantial inference-time costs. We revisit token-efficient post-training and argue that existing sequence-level reward-shaping methods offer limited control over how reasoning effort is allocated across tokens. To bridge the gap, we propose IAPO, an information-theoretic post-training framework that assigns token-wise advantages based on each token’s conditional mutual information (MI) with the final answer. This yields an explicit, principled mechanism for identifying informative reasoning steps and suppressing low-utility exploration. We provide a theoretical analysis showing that our IAPO can induce monotonic reductions in reasoning verbosity without harming correctness. Empirically, IAPO consistently improves reasoning accuracy while reducing reasoning length by up to 36%, outperforming existing token-efficient RL methods across various reasoning datasets. Extensive empirical evaluations demonstrate that information-aware advantage shaping is a powerful and general direction for token-efficient post-training. The code is available at https://github.com/YinhanHe123/IAPO.

Method: 1) Discovered >50% neuron overlap between LLMs and LVLMs during multi-step inference; 2) Used causal probing to show shared neurons encode consistent concept-level effects; 3) Proposed SNRF: identifies shared neurons, computes low-rank approximation of weight differences, selectively injects updates in shared-neuron subspace.

Result: SNRF consistently enhances LVLM inference performance across diverse mathematics and perception benchmarks while preserving perceptual capabilities, with minimal parameter changes and no large-scale multimodal fine-tuning.

Conclusion: Shared neurons form an interpretable bridge between LLMs and LVLMs, enabling low-cost transfer of inference ability into multimodal models through parameter-efficient fusion techniques.

Abstract: Large vision-language models (LVLMs) have rapidly advanced across various domains, yet they still lag behind strong text-only large language models (LLMs) on tasks that require multi-step inference and compositional decision-making. Motivated by their shared transformer architectures, we investigate whether the two model families rely on common internal computation for such inference. At the neuron level, we uncover a surprisingly large overlap: more than half of the top-activated units during multi-step inference are shared between representative LLMs and LVLMs, revealing a modality-invariant inference subspace. Through causal probing via activation amplification, we further show that these shared neurons encode consistent and interpretable concept-level effects, demonstrating their functional contribution to inference. Building on this insight, we propose Shared Neuron Low-Rank Fusion (SNRF), a parameter-efficient framework that transfers mature inference circuitry from LLMs to LVLMs. SNRF profiles cross-model activations to identify shared neurons, computes a low-rank approximation of inter-model weight differences, and injects these updates selectively within the shared-neuron subspace. This mechanism strengthens multimodal inference performance with minimal parameter changes and requires no large-scale multimodal fine-tuning. Across diverse mathematics and perception benchmarks, SNRF consistently enhances LVLM inference performance while preserving perceptual capabilities. Our results demonstrate that shared neurons form an interpretable bridge between LLMs and LVLMs, enabling low-cost transfer of inference ability into multimodal models. Our code is available at https://github.com/chenhangcuisg-code/Do-LLMs-VLMs-Share-Neurons.

Method: TriTopic uses a tri-modal graph fusing semantic embeddings, TF-IDF, and metadata with three innovations: hybrid graph construction via Mutual kNN and Shared Nearest Neighbors, Consensus Leiden Clustering for stable partitions, and Iterative Refinement that sharpens embeddings through dynamic centroid-pulling. It also uses archetype-based topic representations instead of average documents.

Result: TriTopic achieves the highest NMI on every benchmark dataset (20 Newsgroups, BBC News, AG News, Arxiv) with mean NMI 0.575 vs. 0.513 for BERTopic, 0.416 for NMF, 0.299 for LDA, while guaranteeing 100% corpus coverage with 0% outliers.

Conclusion: TriTopic provides a more stable, precise, and comprehensive topic modeling framework that addresses key limitations of existing methods through multi-modal graph fusion and innovative clustering techniques.

Abstract: Topic modeling extracts latent themes from large text collections, but leading approaches like BERTopic face critical limitations: stochastic instability, loss of lexical precision (“Embedding Blur”), and reliance on a single data perspective. We present TriTopic, a framework that addresses these weaknesses through a tri-modal graph fusing semantic embeddings, TF-IDF, and metadata. Three core innovations drive its performance: hybrid graph construction via Mutual kNN and Shared Nearest Neighbors to eliminate noise and combat the curse of dimensionality; Consensus Leiden Clustering for reproducible, stable partitions; and Iterative Refinement that sharpens embeddings through dynamic centroid-pulling. TriTopic also replaces the “average document” concept with archetype-based topic representations defined by boundary cases rather than centers alone. In benchmarks across 20 Newsgroups, BBC News, AG News, and Arxiv, TriTopic achieves the highest NMI on every dataset (mean NMI 0.575 vs. 0.513 for BERTopic, 0.416 for NMF, 0.299 for LDA), guarantees 100% corpus coverage with 0% outliers, and is available as an open-source PyPI library.

Method: Probed model behavior, embeddings, and residual stream activations across three value types; used selective ablation of activation vectors associated with morality to repair value entanglement.

Result: Found pervasive value entanglement where grammatical and economic valuation were overly influenced by moral value relative to human norms; selective ablation successfully repaired this conflation.

Conclusion: LLMs exhibit value entanglement that differs from human value representation, but this can be corrected through targeted intervention, suggesting potential for improving value alignment in multimodal systems.

Abstract: Value alignment of Large Language Models (LLMs) requires us to empirically measure these models’ actual, acquired representation of value. Among the characteristics of value representation in humans is that they distinguish among value of different kinds. We investigate whether LLMs likewise distinguish three different kinds of good: moral, grammatical, and economic. By probing model behavior, embeddings, and residual stream activations, we report pervasive cases of value entanglement: a conflation between these distinct representations of value. Specifically, both grammatical and economic valuation was found to be overly influenced by moral value, relative to human norms. This conflation was repaired by selective ablation of the activation vectors associated with morality.

Method: Astra estimates tail eigenvectors of model output activations from a small task-specific calibration set, then constructs task-adaptive low-rank adapters by constraining updates to the subspace spanned by these tail eigenvectors.

Result: Extensive experiments across 16 NLU and NLG benchmarks show Astra consistently outperforms existing PEFT baselines and even surpasses full fine-tuning in certain scenarios, with faster convergence and reduced parameter budget.

Conclusion: Astra demonstrates that leveraging tail eigenvectors of activations for constructing task-adaptive adapters is an effective approach for parameter-efficient fine-tuning, achieving superior performance with computational efficiency.

Abstract: Parameter-Efficient Fine-Tuning (PEFT) methods, especially LoRA, are widely used for adapting pre-trained models to downstream tasks due to their computational and storage efficiency. However, in the context of LoRA and its variants, the potential of activation subspaces corresponding to tail eigenvectors remains substantially under-exploited, which may lead to suboptimal fine-tuning performance. In this work, we propose Astra (Activation-Space Tail-Eigenvector Low-Rank Adaptation), a novel PEFT method that leverages the tail eigenvectors of the model output activations-estimated from a small task-specific calibration set-to construct task-adaptive low-rank adapters. By constraining updates to the subspace spanned by these tail eigenvectors, Astra achieves faster convergence and improved downstream performance with a significantly reduced parameter budget. Extensive experiments across natural language understanding (NLU) and natural language generation (NLG) tasks demonstrate that Astra consistently outperforms existing PEFT baselines across 16 benchmarks and even surpasses full fine-tuning (FFT) in certain scenarios.

Method: Used sparse autoencoders to extract monosemantic features from LLMs under different experimental settings. Assessed these features as predictors for three research quality tasks: predicting citation count, journal SJR, and journal h-index.

Result: Identified four recurring types of features that capture key aspects of research quality representation: 1) research methodologies, 2) publication types (reviews have higher impact), 3) high-impact research fields/technologies, and 4) specific scientific jargon.

Conclusion: LLMs encode features associated with multiple dimensions of scientific quality, representing an important step toward understanding how LLMs encapsulate research quality concepts.

Abstract: In recent years, there has been a growing use of generative AI, and large language models (LLMs) in particular, to support both the assessment and generation of scientific work. Although some studies have shown that LLMs can, to a certain extent, evaluate research according to perceived quality, our understanding of the internal mechanisms that enable this capability remains limited. This paper presents the first study that investigates how LLMs encode the concept of scientific quality through relevant monosemantic features extracted using sparse autoencoders. We derive such features under different experimental settings and assess their ability to serve as predictors across three tasks related to research quality: predicting citation count, journal SJR, and journal h-index. The results indicate that LLMs encode features associated with multiple dimensions of scientific quality. In particular, we identify four recurring types of features that capture key aspects of how research quality is represented: 1) features reflecting research methodologies; 2) features related to publication type, with literature reviews typically exhibiting higher impact; 3) features associated with high-impact research fields and technologies; and 4) features corresponding to specific scientific jargons. These findings represent an important step toward understanding how LLMs encapsulate concepts related to research quality.

Method: Introduces AgenticRAGTracer, an Agentic RAG benchmark primarily constructed automatically by LLMs, spanning multiple domains with 1,305 data points and no overlap with existing benchmarks.

Result: Even best LLMs perform poorly (GPT-5: 22.6% EM on hardest portion). Hop-aware diagnosis reveals failures driven by distorted reasoning chains (premature collapse or over-extension).

Conclusion: Provides diagnostic dimension missing in traditional evaluations, facilitates Agentic RAG research, and enables step-by-step validation of multi-hop reasoning capabilities.

Abstract: With the rapid advancement of agent-based methods in recent years, Agentic RAG has undoubtedly become an important research direction. Multi-hop reasoning, which requires models to engage in deliberate thinking and multi-step interaction, serves as a critical testbed for assessing such capabilities. However, existing benchmarks typically provide only final questions and answers, while lacking the intermediate hop-level questions that gradually connect atomic questions to the final multi-hop query. This limitation prevents researchers from analyzing at which step an agent fails and restricts more fine-grained evaluation of model capabilities. Moreover, most current benchmarks are manually constructed, which is both time-consuming and labor-intensive, while also limiting scalability and generalization. To address these challenges, we introduce AgenticRAGTracer, the first Agentic RAG benchmark that is primarily constructed automatically by large language models and designed to support step-by-step validation. Our benchmark spans multiple domains, contains 1,305 data points, and has no overlap with existing mainstream benchmarks. Extensive experiments demonstrate that even the best large language models perform poorly on our dataset. For instance, GPT-5 attains merely 22.6% EM accuracy on the hardest portion of our dataset. Hop-aware diagnosis reveals that failures are primarily driven by distorted reasoning chains – either collapsing prematurely or wandering into over-extension. This highlights a critical inability to allocate steps consistent with the task’s logical structure, providing a diagnostic dimension missing in traditional evaluations. We believe our work will facilitate research in Agentic RAG and inspire further meaningful progress in this area. Our code and data are available at https://github.com/YqjMartin/AgenticRAGTracer.

Method: Created a manually annotated dataset from museum catalogs, auction listings, and scholarly databases, filtered for single-artwork focus with explicit material/composition/iconography statements, with three annotation layers (metadata, content, co-reference) and 37 entity types aligned with Wikidata.

Result: FRAME dataset with UIMA XMI CAS files, images, and bibliographic metadata, supporting NER, RE, and NEL tasks for art-historical analysis and knowledge graph construction.

Conclusion: FRAME enables benchmarking and fine-tuning of NER/RE systems for multimodal art analysis, including zero- and few-shot LLM applications, bridging visual content understanding with structured knowledge extraction.

Abstract: This paper introduces FRAME (Fine-grained Recognition of Art-historical Metadata and Entities), a manually annotated dataset of art-historical image descriptions for Named Entity Recognition (NER) and Relation Extraction (RE). Descriptions were collected from museum catalogs, auction listings, open-access platforms, and scholarly databases, then filtered to ensure that each text focuses on a single artwork and contains explicit statements about its material, composition, or iconography. FRAME provides stand-off annotations in three layers: a metadata layer for object-level properties, a content layer for depicted subjects and motifs, and a co-reference layer linking repeated mentions. Across layers, entity spans are labeled with 37 types and connected by typed RE links between mentions. Entity types are aligned with Wikidata to support Named Entity Linking (NEL) and downstream knowledge-graph construction. The dataset is released as UIMA XMI Common Analysis Structure (CAS) files with accompanying images and bibliographic metadata, and can be used to benchmark and fine-tune NER and RE systems, including zero- and few-shot setups with Large Language Models (LLMs).

Method: Proposes a contrastive Sparse AutoEncoder framework that learns facet-level personality control vectors aligned with the Big Five 30-facet model. Uses a new 15,000-sample leakage-controlled corpus for balanced supervision. Learned vectors are integrated into the model’s residual space and dynamically selected by a trait-activated routing module.

Result: Experiments show the method maintains stable character fidelity and output quality across contextualized settings, outperforming Contrastive Activation Addition (CAA) and prompt-only baselines. The combined SAE+Prompt configuration achieves best overall performance.

Conclusion: Contrastively trained latent vectors can enhance persona control while preserving dialogue coherence, providing precise and interpretable personality steering for Role-Playing Agents.

Abstract: Personality control in Role-Playing Agents (RPAs) is commonly achieved via training-free methods that inject persona descriptions and memory through prompts or retrieval-augmented generation, or via supervised fine-tuning (SFT) on persona-specific corpora. While SFT can be effective, it requires persona-labeled data and retraining for new roles, limiting flexibility. In contrast, prompt- and RAG-based signals are easy to apply but can be diluted in long dialogues, leading to drifting and sometimes inconsistent persona behavior. To address this, we propose a contrastive Sparse AutoEncoder (SAE) framework that learns facet-level personality control vectors aligned with the Big Five 30-facet model. A new 15,000-sample leakage-controlled corpus is constructed to provide balanced supervision for each facet. The learned vectors are integrated into the model’s residual space and dynamically selected by a trait-activated routing module, enabling precise and interpretable personality steering. Experiments on Large Language Models (LLMs) show that the proposed method maintains stable character fidelity and output quality across contextualized settings, outperforming Contrastive Activation Addition (CAA) and prompt-only baselines. The combined SAE+Prompt configuration achieves the best overall performance, confirming that contrastively trained latent vectors can enhance persona control while preserving dialogue coherence.

Method: Developed a modular multi-backend architecture with language-agnostic API that integrates rule-based finite-state transducers and neural models transparently. Features automatic script detection and routing between script variants, with outputs following CoNLL-U standard for interoperability.

Result: Created TurkicNLP library covering tokenization, morphological analysis, POS tagging, dependency parsing, NER, bidirectional script transliteration, cross-lingual sentence embeddings, and machine translation for Turkic languages across four script families.

Conclusion: TurkicNLP provides a comprehensive, unified solution for NLP in Turkic languages, addressing fragmentation issues and enabling easier research and application development across this important language family.

Abstract: Natural language processing for the Turkic language family, spoken by over 200 million people across Eurasia, remains fragmented, with most languages lacking unified tooling and resources. We present TurkicNLP, an open-source Python library providing a single, consistent NLP pipeline for Turkic languages across four script families: Latin, Cyrillic, Perso-Arabic, and Old Turkic Runic. The library covers tokenization, morphological analysis, part-of-speech tagging, dependency parsing, named entity recognition, bidirectional script transliteration, cross-lingual sentence embeddings, and machine translation through one language-agnostic API. A modular multi-backend architecture integrates rule-based finite-state transducers and neural models transparently, with automatic script detection and routing between script variants. Outputs follow the CoNLL-U standard for full interoperability and extension. Code and documentation are hosted at https://github.com/turkic-nlp/turkicnlp .

Method: Created a novel history-conditioned reply prediction task using authentic X (Twitter) data to build a dataset for evaluating LLM linguistic output against human content, analyzed using stylistic and content-based metrics.

Result: Findings reveal significant linguistic discrepancies between LLM-generated and human content, highlighting the need for more sophisticated prompting techniques and specialized datasets to ensure synthetic data accurately reflects human communication patterns.

Conclusion: The paper emphasizes the methodological risks of naive LLM application in social science research and provides a quantitative framework for assessing synthetic data quality, advocating for improved techniques to enhance research validity.

Abstract: The increasing use of Large Language Models (LLMs) as proxies for human participants in social science research presents a promising, yet methodologically risky, paradigm shift. While LLMs offer scalability and cost-efficiency, their “naive” application, where they are prompted to generate content without explicit behavioral constraints, introduces significant linguistic discrepancies that challenge the validity of research findings. This paper addresses these limitations by introducing a novel, history-conditioned reply prediction task on authentic X (formerly Twitter) data, to create a dataset designed to evaluate the linguistic output of LLMs against human-generated content. We analyze these discrepancies using stylistic and content-based metrics, providing a quantitative framework for researchers to assess the quality and authenticity of synthetic data. Our findings highlight the need for more sophisticated prompting techniques and specialized datasets to ensure that LLM-generated content accurately reflects the complex linguistic patterns of human communication, thereby improving the validity of computational social science studies.

Method: Propose xDORA framework integrating CLIP/DINOv2 vision encoders with XGLM/XLM-R text encoders via weighted attention pooling. Use FAISS-based k-NN classifier for non-parametric inference and introduce RAG-Fused DORA for retrieval-augmented contextual reasoning. Also evaluate LLaVA under various prompting settings.

Result: xDORA (CLIP + XLM-R) achieves macro-average F1-scores of 0.78 for hateful meme identification and 0.71 for target entity detection. RAG-Fused DORA improves to 0.79 and 0.74. FAISS-based classifier shows robustness for rare classes. LLaVA has limited effectiveness without fine-tuning.

Conclusion: Supervised, retrieval-augmented, and non-parametric multimodal frameworks effectively address linguistic and cultural complexities in low-resource hate speech detection, while pretrained vision-language models like LLaVA need fine-tuning for code-mixed Bengali content.

Abstract: Hateful content on social media increasingly appears as multimodal memes that combine images and text to convey harmful narratives. In low-resource languages such as Bengali, automated detection remains challenging due to limited annotated data, class imbalance, and pervasive code-mixing. To address these issues, we augment the Bengali Hateful Memes (BHM) dataset with semantically aligned samples from the Multimodal Aggression Dataset in Bengali (MIMOSA), improving both class balance and semantic diversity. We propose the Enhanced Dual Co-attention Framework (xDORA), integrating vision encoders (CLIP, DINOv2) and multilingual text encoders (XGLM, XLM-R) via weighted attention pooling to learn robust cross-modal representations. Building on these embeddings, we develop a FAISS-based k-nearest neighbor classifier for non-parametric inference and introduce RAG-Fused DORA, which incorporates retrieval-driven contextual reasoning. We further evaluate LLaVA under zero-shot, few-shot, and retrieval-augmented prompting settings. Experiments on the extended dataset show that xDORA (CLIP + XLM-R) achieves macro-average F1-scores of 0.78 for hateful meme identification and 0.71 for target entity detection, while RAG-Fused DORA improves performance to 0.79 and 0.74, yielding gains over the DORA baseline. The FAISS-based classifier performs competitively and demonstrates robustness for rare classes through semantic similarity modeling. In contrast, LLaVA exhibits limited effectiveness in few-shot settings, with only modest improvements under retrieval augmentation, highlighting constraints of pretrained vision-language models for code-mixed Bengali content without fine-tuning. These findings demonstrate the effectiveness of supervised, retrieval-augmented, and non-parametric multimodal frameworks for addressing linguistic and cultural complexities in low-resource hate speech detection.

Method: PR2 uses reinforcement learning to learn adaptive retrieval-reasoning policies that determine when to retrieve, what evidence to retrieve from user profiles, and how to incorporate it into intermediate reasoning steps. It optimizes multi-turn reasoning trajectories under a personalized reward function.

Result: Extensive experiments on the LaMP-QA benchmark using three LLMs show PR2 consistently outperforms strong baselines, achieving an average relative improvement of 8.8%-12% in personalized QA.

Conclusion: PR2 demonstrates that learning adaptive retrieval-reasoning policies through reinforcement learning significantly improves personalization in QA by better aligning reasoning paths with user-specific preferences and contextual signals.

Abstract: Personalization in Question Answering (QA) requires answers that are both accurate and aligned with users’ background, preferences, and historical context. Existing state-of-the-art methods primarily rely on retrieval-augmented generation (RAG) solutions that construct personal context by retrieving relevant items from the user’s profile. Existing methods use the user’s query directly to retrieve personal documents, and such strategies often lead to surface-level personalization. We propose PR2 (Personalized Retrieval-Augmented Reasoning), a reinforcement learning framework that integrates reasoning and retrieval from personal context for personalization. PR2 learns adaptive retrieval-reasoning policies, determining when to retrieve, what evidence to retrieve from user profiles, and how to incorporate it into intermediate reasoning steps. By optimizing multi-turn reasoning trajectories under a personalized reward function, the framework reinforces reasoning paths that better align with user-specific preferences and contextual signals reflected by the reward model. Extensive experiments on the LaMP-QA benchmark using three LLMs show that PR2 consistently outperforms strong baselines, achieving an average relative improvement of 8.8%-12% in personalized QA.

Method: Presents a structured analysis with taxonomy of MAG systems based on four memory structures, then analyzes key pain points including benchmark saturation, metric validity, judge sensitivity, backbone-dependent accuracy, and latency/throughput overhead.

Result: Identifies why current agentic memory systems underperform theoretical promise and provides directions for more reliable evaluation and scalable system design through connecting memory structure to empirical limitations.

Conclusion: The survey clarifies the gap between theoretical promise and practical performance of agentic memory systems, offering a framework for addressing evaluation and scalability challenges in LLM agent memory architectures.

Abstract: Agentic memory systems enable large language model (LLM) agents to maintain state across long interactions, supporting long-horizon reasoning and personalization beyond fixed context windows. Despite rapid architectural development, the empirical foundations of these systems remain fragile: existing benchmarks are often underscaled, evaluation metrics are misaligned with semantic utility, performance varies significantly across backbone models, and system-level costs are frequently overlooked. This survey presents a structured analysis of agentic memory from both architectural and system perspectives. We first introduce a concise taxonomy of MAG systems based on four memory structures. Then, we analyze key pain points limiting current systems, including benchmark saturation effects, metric validity and judge sensitivity, backbone-dependent accuracy, and the latency and throughput overhead introduced by memory maintenance. By connecting the memory structure to empirical limitations, this survey clarifies why current agentic memory systems often underperform their theoretical promise and outlines directions for more reliable evaluation and scalable system design.

Method: Created balanced dataset of 36k posts across 9 categories using hybrid annotation (ChatGPT few-shot prompting + human verification). Applied undersampling with semantic redundancy removal and data augmentation. Benchmarked BiLSTM, XLM-RoBERTa (with LoRA/AdaLoRA), FaBERT, SBERT, and Persian-specific TookaBERT models.

Result: Transformer-based models outperformed traditional neural networks, with TookaBERT-Large achieving best performance (Precision: 0.9622, Recall: 0.9621, F1: 0.9621). Social and political texts showed slightly lower scores due to ambiguity.

Conclusion: Provides high-quality Persian social media dataset and comprehensive model evaluations, establishing foundation for Persian NLP research including trend analysis and user classification.

Abstract: This research introduces the first large-scale, well-balanced Persian social media text classification dataset, specifically designed to address the lack of comprehensive resources in this domain. The dataset comprises 36,000 posts across nine categories (Economic, Artistic, Sports, Political, Social, Health, Psychological, Historical, and Science & Technology), each containing 4,000 samples to ensure balanced class distribution. Data collection involved 60,000 raw posts from various Persian social media platforms, followed by rigorous preprocessing and hybrid annotation combining ChatGPT-based few-shot prompting with human verification. To mitigate class imbalance, we employed undersampling with semantic redundancy removal and advanced data augmentation strategies integrating lexical replacement and generative prompting. We benchmarked several models, including BiLSTM, XLM-RoBERTa (with LoRA and AdaLoRA adaptations), FaBERT, SBERT-based architectures, and the Persian-specific TookaBERT (Base and Large). Experimental results show that transformer-based models consistently outperform traditional neural networks, with TookaBERT-Large achieving the best performance (Precision: 0.9622, Recall: 0.9621, F1- score: 0.9621). Class-wise evaluation further confirms robust performance across all categories, though social and political texts exhibited slightly lower scores due to inherent ambiguity. This research presents a new high-quality dataset and provides comprehensive evaluations of cutting-edge models, establishing a solid foundation for further developments in Persian NLP, including trend analysis, social behavior modeling, and user classification. The dataset is publicly available to support future research endeavors.

Method: Evaluated multiple approaches: (1) supervised learning models trained on labeled data, (2) zero/few-shot LLMs without task-specific fine-tuning, and (3) LLM-based digital twins incorporating individual characteristics and prior PME histories. Used dataset of 3010 message ratings from 301 young adult smokers across three domains: content quality, coping support, and quitting support.

Result: LLM-based digital twins outperformed zero/few-shot LLMs by 12 percentage points and supervised baselines by 13 percentage points, achieving accuracies of 0.49 (content), 0.45 (coping), and 0.49 (quitting), with directional accuracies of 0.75, 0.66, and 0.70 on simplified 3-point scale. Digital twins showed greater dispersion across rating categories, indicating improved sensitivity to individual differences.

Conclusion: Integrating personal profiles with LLMs captures person-specific differences in PME and outperforms traditional approaches. LLM-based digital twins show potential for personalizing mobile smoking cessation and other health behavior change interventions through improved PME prediction.

Abstract: Perceived message effectiveness (PME) by potential intervention end-users is important for selecting and optimizing personalized smoking cessation intervention messages for mobile health (mHealth) platform delivery. This study evaluates whether large language models (LLMs) can accurately predict PME for smoking cessation messages. We evaluated multiple models for predicting PME across three domains: content quality, coping support, and quitting support. The dataset comprised 3010 message ratings (5-point Likert scale) from 301 young adult smokers. We compared (1) supervised learning models trained on labeled data, (2) zero and few-shot LLMs prompted without task-specific fine-tuning, and (3) LLM-based digital twins that incorporate individual characteristics and prior PME histories to generate personalized predictions. Model performance was assessed on three held-out messages per participant using accuracy, Cohen’s kappa, and F1. LLM-based digital twins outperformed zero and few-shot LLMs (12 percentage points on average) and supervised baselines (13 percentage points), achieving accuracies of 0.49 (content), 0.45 (coping), and 0.49 (quitting), with directional accuracies of 0.75, 0.66, and 0.70 on a simplified 3-point scale. Digital twin predictions showed greater dispersion across rating categories, indicating improved sensitivity to individual differences. Integrating personal profiles with LLMs captures person-specific differences in PME and outperforms supervised and zero and few-shot approaches. Improved PME prediction may enable more tailored intervention content in mHealth. LLM-based digital twins show potential for supporting personalization of mobile smoking cessation and other health behavior change interventions.

Method: Propose “Pyramid MoA” - hierarchical Mixture-of-Agents architecture with lightweight Router that uses semantic agreement and confidence calibration among ensemble of small models to identify hard problems and dynamically escalate only necessary queries to larger models.

Result: Achieves 93.0% accuracy on GSM8K benchmark (vs Oracle baseline 98.0%) with 61% compute cost reduction and negligible latency overhead (+0.82s), allowing tunable performance-budget trade-off.

Conclusion: Hierarchical routing approach effectively balances accuracy and cost, enabling deployment of high-performance LLM systems with significantly reduced computational expense.

Abstract: Large Language Models (LLMs) face a persistent trade-off between inference cost and reasoning capability. While “Oracle” models (e.g., Llama-3-70B) achieve state-of-the-art accuracy, they are prohibitively expensive for high-volume deployment. Smaller models (e.g., 8B parameters) are cost-effective but struggle with complex tasks. In this work, we propose “Pyramid MoA”, a hierarchical Mixture-of-Agents architecture that uses a lightweight Router to dynamically escalate queries only when necessary. By leveraging semantic agreement and confidence calibration among an ensemble of small models, our Router identifies “hard” problems with high precision. On the GSM8K benchmark, our system achieves 93.0% accuracy, effectively matching the Oracle baseline (98.0%) while reducing compute costs by 61%. We demonstrate that the system introduces negligible latency overhead (+0.82s) and allows for a tunable trade-off between performance and budget.

Method: Conducted systematic study along three decoupled dimensions: prompt template (Fast Thinking vs Slow Thinking), reward function (F1-based vs EM with action-level penalties), and policy optimization methods (REINFORCE, PPO, GRPO).

Result: Fast Thinking template outperforms Slow Thinking; F1-based reward underperforms EM due to answer avoidance but can be improved with action-level penalties; REINFORCE outperforms PPO with fewer search actions; GRPO shows poorest stability. Search-R1++ baseline improves performance from 0.403 to 0.442 (Qwen2.5-7B) and 0.289 to 0.331 (Qwen2.5-3B).

Conclusion: Systematic RL component analysis provides insights for more principled training strategies in deep research systems, with identified optimal configurations leading to improved baseline performance.

Abstract: Deep Research agents tackle knowledge-intensive tasks through multi-round retrieval and decision-oriented generation. While reinforcement learning (RL) has been shown to improve performance in this paradigm, its contributions remain underexplored. To fully understand the role of RL, we conduct a systematic study along three decoupled dimensions: prompt template, reward function, and policy optimization. Our study reveals that: 1) the Fast Thinking template yields greater stability and better performance than the Slow Thinking template used in prior work; 2) the F1-based reward underperforms the EM due to training collapse driven by answer avoidance; this can be mitigated by incorporating action-level penalties, ultimately surpassing EM; 3) REINFORCE outperforms PPO while requiring fewer search actions, whereas GRPO shows the poorest stability among policy optimization methods. Building on these insights, we then introduce Search-R1++, a strong baseline that improves the performance of Search-R1 from 0.403 to 0.442 (Qwen2.5-7B) and 0.289 to 0.331 (Qwen2.5-3B). We hope that our findings can pave the way for more principled and reliable RL training strategies in Deep Research systems.

Method: Proposes Hyper-KGGen with: 1) coarse-to-fine document decomposition for full-dimensional coverage from binary links to hyperedges, 2) adaptive skill acquisition module that distills domain expertise into a Global Skill Library, 3) stability-based feedback loop using extraction stability as reward signal to induce high-quality skills from unstable traces.

Result: Hyper-KGGen significantly outperforms strong baselines. Also introduces HyperDocRED benchmark for document-level knowledge hypergraph extraction. Shows evolved skills provide richer guidance than static few-shot examples in multi-scenario settings.

Conclusion: The skill-driven framework effectively bridges the scenario gap in knowledge hypergraph extraction by dynamically evolving domain-specific skills through stability-based feedback, enabling better generalization across diverse domains.

Abstract: Knowledge hypergraphs surpass traditional binary knowledge graphs by encapsulating complex $n$-ary atomic facts, providing a more comprehensive paradigm for semantic representation. However, constructing high-quality hypergraphs remains challenging due to the \textit{scenario gap}: generic extractors struggle to generalize across diverse domains with specific jargon, while existing methods often fail to balance structural skeletons with fine-grained details. To bridge this gap, we propose \textbf{Hyper-KGGen}, a skill-driven framework that reformulates extraction as a dynamic skill-evolving process. First, Hyper-KGGen employs a \textit{coarse-to-fine} mechanism to systematically decompose documents, ensuring full-dimensional coverage from binary links to complex hyperedges. Crucially, it incorporates an \textit{adaptive skill acquisition} module that actively distills domain expertise into a Global Skill Library. This is achieved via a stability-based feedback loop, where extraction stability serves as a relative reward signal to induce high-quality skills from unstable traces and missed predictions. Additionally, we present \textbf{HyperDocRED}, a rigorously annotated benchmark for document-level knowledge hypergraph extraction. Experiments demonstrate that Hyper-KGGen significantly outperforms strong baselines, validating that evolved skills provide substantially richer guidance than static few-shot examples in multi-scenario settings.

Method: Investigated different HTML text extractors, compared their outputs, and tested the union approach to combine content from multiple extractors while maintaining filtering pipelines.

Result: Union of different extractors increased token yield by up to 71% while maintaining benchmark performance; extractor choice significantly impacted downstream tasks (up to 10 p.p. on WikiTQ and 3 p.p. on HumanEval).

Conclusion: Using multiple HTML text extractors rather than a single fixed one can substantially improve data coverage and utilization for web-scale LLM pretraining, especially for structured content.

Abstract: One of the first pre-processing steps for constructing web-scale LLM pretraining datasets involves extracting text from HTML. Despite the immense diversity of web content, existing open-source datasets predominantly apply a single fixed extractor to all webpages. In this work, we investigate whether this practice leads to suboptimal coverage and utilization of Internet data. We first show that while different extractors may lead to similar model performance on standard language understanding tasks, the pages surviving a fixed filtering pipeline can differ substantially. This suggests a simple intervention: by taking a Union over different extractors, we can increase the token yield of DCLM-Baseline by up to 71% while maintaining benchmark performance. We further show that for structured content such as tables and code blocks, extractor choice can significantly impact downstream task performance, with differences of up to 10 percentage points (p.p.) on WikiTQ and 3 p.p. on HumanEval.

Method: Two-stage framework: 1) Adaptive pruning stage filters out low-information patches to create refined embeddings, 2) Hierarchical merging stage compresses the pre-filtered set to summarize semantic content without noise-induced feature dilution.

Result: Extensive experiments on 29 VDR datasets show the framework consistently outperforms existing methods, significantly extending near-lossless compression range and providing robust performance at high compression ratios.

Conclusion: The Prune-then-Merge framework effectively overcomes the compression-performance dilemma in VDR by synergizing complementary pruning and merging approaches, achieving superior efficiency without sacrificing retrieval quality.

Abstract: Visual Document Retrieval (VDR), which aims to retrieve relevant pages within vast corpora of visually-rich documents, is of significance in current multimodal retrieval applications. The state-of-the-art multi-vector paradigm excels in performance but suffers from prohibitive overhead, a problem that current efficiency methods like pruning and merging address imperfectly, creating a difficult trade-off between compression rate and feature fidelity. To overcome this dilemma, we introduce Prune-then-Merge, a novel two-stage framework that synergizes these complementary approaches. Our method first employs an adaptive pruning stage to filter out low-information patches, creating a refined, high-signal set of embeddings. Subsequently, a hierarchical merging stage compresses this pre-filtered set, effectively summarizing semantic content without the noise-induced feature dilution seen in single-stage methods. Extensive experiments on 29 VDR datasets demonstrate that our framework consistently outperforms existing methods, significantly extending the near-lossless compression range and providing robust performance at high compression ratios.

Method: Proposes a framework with three key components: 1) constraint-aware question representation combining semantic cues from language models with temporal entity dynamics, 2) temporal-aware graph neural network for explicit multi-hop reasoning via time-aware message passing, and 3) multi-view attention mechanism for effective fusion of question context and temporal graph knowledge.

Result: Experiments on multiple TKGQA benchmarks demonstrate consistent improvements over multiple baselines.

Conclusion: The proposed framework effectively addresses key limitations in TKGQA by better incorporating temporal constraints, enabling explicit multi-hop reasoning, and improving language-graph fusion.

Abstract: Question Answering over Temporal Knowledge Graphs (TKGQA) has attracted growing interest for handling time-sensitive queries. However, existing methods still struggle with: 1) weak incorporation of temporal constraints in question representation, causing biased reasoning; 2) limited ability to perform explicit multi-hop reasoning; and 3) suboptimal fusion of language and graph representations. We propose a novel framework with temporal-aware question encoding, multi-hop graph reasoning, and multi-view heterogeneous information fusion. Specifically, our approach introduces: 1) a constraint-aware question representation that combines semantic cues from language models with temporal entity dynamics; 2) a temporal-aware graph neural network for explicit multi-hop reasoning via time-aware message passing; and 3) a multi-view attention mechanism for more effective fusion of question context and temporal graph knowledge. Experiments on multiple TKGQA benchmarks demonstrate consistent improvements over multiple baselines.

Method: DEEP automates execution and scoring of dockerized models, extracts runtime information, and uses statistical clustering algorithms to group models by performance significance.

Result: The software enables better understanding of model performance through automated evaluation, statistical clustering, and visualization web-app for result interpretation.

Conclusion: DEEP provides an extensible, automated framework for comparative evaluation of AI systems with statistical analysis and visualization capabilities.

Abstract: Comparative evaluation of several systems is a recurrent task in researching. It is a key step before deciding which system to use for our work, or, once our research has been conducted, to demonstrate the potential of the resulting model. Furthermore, it is the main task of competitive, public challenges evaluation. Our proposed software (DEEP) automates both the execution and scoring of machine translation and optical character recognition models. Furthermore, it is easily extensible to other tasks. DEEP is prepared to receive dockerized systems, run them (extracting information at that same time), and assess hypothesis against some references. With this approach, evaluators can achieve a better understanding of the performance of each model. Moreover, the software uses a clustering algorithm based on a statistical analysis of the significance of the results yielded by each model, according to the evaluation metrics. As a result, evaluators are able to identify clusters of performance among the swarm of proposals and have a better understanding of the significance of their differences. Additionally, we offer a visualization web-app to ensure that the results can be adequately understood and interpreted. Finally, we present an exemplary case of use of DEEP.

Method: The authors created: 1) an extensive overview of existing datasets, 2) a living online overview with over 45 features per dataset, and 3) integration of all publicly available datasets into the Python package pymovements for easy access.

Result: The work covers numerous eye-tracking-while-reading datasets with diverse features, provides an online resource (https://dili-lab.github.io/datasets.html), and enables standardized access through pymovements Python package.

Conclusion: This standardization effort strengthens FAIR principles in eye-tracking research, promotes reproducibility, and facilitates easier reuse of datasets across different research communities.

Abstract: Eye-tracking-while-reading corpora are a valuable resource for many different disciplines and use cases. Use cases range from studying the cognitive processes underlying reading to machine-learning-based applications, such as gaze-based assessments of reading comprehension. The past decades have seen an increase in the number and size of eye-tracking-while-reading datasets as well as increasing diversity with regard to the stimulus languages covered, the linguistic background of the participants, or accompanying psychometric or demographic data. The spread of data across different disciplines and the lack of data sharing standards across the communities lead to many existing datasets that cannot be easily reused due to a lack of interoperability. In this work, we aim at creating more transparency and clarity with regards to existing datasets and their features across different disciplines by i) presenting an extensive overview of existing datasets, ii) simplifying the sharing of newly created datasets by publishing a living overview online, https://dili-lab.github.io/datasets.html, presenting over 45 features for each dataset, and iii) integrating all publicly available datasets into the Python package pymovements which offers an eye-tracking datasets library. By doing so, we aim to strengthen the FAIR principles in eye-tracking-while-reading research and promote good scientific practices, such as reproducing and replicating studies.

Method: Introduces DUAL benchmark with 28.6k Wikidata-derived triplets annotated with fact popularity using Wikipedia link counts and LLM-based salience scores; compares unlearning performance on pretrained vs SFT models

Result: SFT-based unlearning yields smoother forgetting, more stable tuning, and 10-50% higher retention; direct unlearning on pretrained models remains unstable and prone to relearning or catastrophic forgetting

Conclusion: Training stage matters for machine unlearning - SFT-based approaches are more effective than direct pretrained model unlearning; DUAL benchmark enables better evaluation of unlearning across training stages

Abstract: Machine Unlearning (MU) enables Large Language Models (LLMs) to remove unsafe or outdated information. However, existing work assumes that all facts are equally forgettable and largely ignores whether the forgotten knowledge originates from pretraining or supervised fine-tuning (SFT). In this paper, we introduce DUAL (Dual Unlearning Evaluation across Training Stages), a benchmark of 28.6k Wikidata-derived triplets annotated with fact popularity using Wikipedia link counts and LLM-based salience scores. Our experiments show that pretrained and SFT models respond differently to unlearning. An SFT step on the forget data yields smoother forgetting, more stable tuning, and 10-50% higher retention, while direct unlearning on pretrained models remains unstable and prone to relearning or catastrophic forgetting.

Method: Uses knowledge graphs to dynamically construct challenging, multifaceted questions with statistical difficulty estimation to address popularity bias. Implements automated verification pipeline that detects abstentions and verifies responses at conceptual and correctness levels.

Result: Evaluated 25 frontier models using novel accuracy and hallucination metrics, providing interpretable insights into knowledge factors causing hallucinations across different model sizes.

Conclusion: KGHaluBench offers fairer, more comprehensive assessment of LLM truthfulness and is publicly available to support future hallucination mitigation research.

Abstract: Large Language Models (LLMs) possess a remarkable capacity to generate persuasive and intelligible language. However, coherence does not equate to truthfulness, as the responses often contain subtle hallucinations. Existing benchmarks are limited by static and narrow questions, leading to limited coverage and misleading evaluations. We present KGHaluBench, a Knowledge Graph-based hallucination benchmark that assesses LLMs across the breadth and depth of their knowledge, providing a fairer and more comprehensive insight into LLM truthfulness. Our framework utilises the KG to dynamically construct challenging, multifaceted questions, whose difficulty is then statistically estimated to address popularity bias. Our automated verification pipeline detects abstentions and verifies the LLM’s response at both conceptual and correctness levels to identify different types of hallucinations. We evaluate 25 frontier models, using novel accuracy and hallucination metrics. The results provide a more interpretable insight into the knowledge factors that cause hallucinations across different model sizes. KGHaluBench is publicly available to support future developments in hallucination mitigation.

Method: Developed two keyboard tools: (1) Android mobile keyboard published on Google Play Store, (2) Windows desktop keyboard undergoing community testing. Both support complete character inventory including schwa, retracted schwa, nasalized vowels, and accented forms, operating fully offline with zero network permissions

Result: Android keyboard actively used in teacher training programs, Windows keyboard undergoing community testing. Both tools address connectivity constraints and data sovereignty concerns while providing complete writing system support

Conclusion: Presents a replicable model for creating digital input tools for other endangered language communities, enabling proper use of developed orthographies

Abstract: We present a mobile and desktop keyboard suite for Idu Mishmi, an endangered Trans-Himalayan language spoken by approximately 11,000 people in Arunachal Pradesh, India. Although a Latin-based orthography was developed in 2018, no digital input tools existed to use it, forcing speakers into ad-hoc romanizations that cannot represent the full writing system. Our keyboards comprise two tools: (1) an Android mobile keyboard, published on the Google Play Store and actively used in teacher training programs, and (2) a Windows desktop keyboard currently undergoing community testing. Both tools support the complete Idu Mishmi character inventory, including schwa, retracted schwa, nasalized vowels, and accented forms. Both operate fully offline with zero network permissions, addressing connectivity constraints and data sovereignty concerns. We describe the design, implementation, and deployment as a replicable model for other endangered language communities.

Method: Treats style preservation as signal processing task. Quantifies literary style into Stylistic Feature Spectrum (SFS) using wavelet packet transform. Uses SFS as control signal to dynamically assemble tailored workflow of specialized translation agents based on source text’s structural patterns.

Result: Extensive experiments on translation benchmarks show SAMAS achieves competitive semantic accuracy against strong baselines, with statistically significant advantage in style fidelity.

Conclusion: SAMAS effectively addresses the style preservation limitation in LLM-based translation by framing it as a signal processing problem and using dynamic multi-agent assembly based on stylistic feature analysis.

Abstract: Modern large language models (LLMs) excel at generating fluent and faithful translations. However, they struggle to preserve an author’s unique literary style, often producing semantically correct but generic outputs. This limitation stems from the inability of current single-model and static multi-agent systems to perceive and adapt to stylistic variations. To address this, we introduce the Style-Adaptive Multi-Agent System (SAMAS), a novel framework that treats style preservation as a signal processing task. Specifically, our method quantifies literary style into a Stylistic Feature Spectrum (SFS) using the wavelet packet transform. This SFS serves as a control signal to dynamically assemble a tailored workflow of specialized translation agents based on the source text’s structural patterns. Extensive experiments on translation benchmarks show that SAMAS achieves competitive semantic accuracy against strong baselines, primarily by leveraging its statistically significant advantage in style fidelity.

Method: SHIELD combines disproportionality analysis (Information Component with empirical Bayesian shrinkage) with semantic clustering of MedDRA term embeddings. It constructs a utility matrix weighting semantic similarities by signal magnitude, performs spectral embedding and clustering, then uses large language models to annotate clusters with syndrome-level summary labels.

Result: The framework successfully recovers known safety signals and generates interpretable, cluster-based summaries in real clinical trial examples, producing network graphs and hierarchical trees representing treatment-associated safety profiles.

Conclusion: SHIELD bridges statistical signal detection with modern NLP to enhance safety assessment and causal interpretation in clinical trials, providing automated, integrated safety signal detection with semantic coherence.

Abstract: We present SHIELD, a novel methodology for automated and integrated safety signal detection in clinical trials. SHIELD combines disproportionality analysis with semantic clustering of adverse event (AE) terms applied to MedDRA term embeddings. For each AE, the pipeline computes an information-theoretic disproportionality measure (Information Component) with effect size derived via empirical Bayesian shrinkage. A utility matrix is constructed by weighting semantic term-term similarities by signal magnitude, followed by spectral embedding and clustering to identify groups of related AEs. Resulting clusters are annotated with syndrome-level summary labels using large language models, yielding a coherent, data-driven representation of treatment-associated safety profiles in the form of a network graph and hierarchical tree. We implement the SHIELD framework in the context of a single-arm incidence summary, to compare two treatment arms or for the detection of any treatment effect in a multi-arm trial. We illustrate its ability to recover known safety signals and generate interpretable, cluster-based summaries in a real clinical trial example. This work bridges statistical signal detection with modern natural language processing to enhance safety assessment and causal interpretation in clinical trials.

Method: Classify ODRL operands by value-domain structure, develop axis-decomposition framework to refine dimensional operands into axis-specific scalar operands, with conflict detection using two-layer approach and three-valued logic.

Result: Created ODRL Spatial Axis Profile with 15 axis-specific operands, evaluated on 117 benchmark problems with full concordance between provers, mechanically verified theorems in Isabelle/HOL.

Conclusion: The axis-decomposition framework resolves dimensional ambiguity in ODRL constraints while maintaining formal properties, enabling deterministic policy evaluation for multi-dimensional quantities.

Abstract: Every ODRL 2.2 constraint compares a single scalar value: (leftOperand, operator, rightOperand). Five of ODRL’s approximately 34 left operands, however, denote multi-dimensional quantities–image dimensions, canvas positions, geographic coordinates–whose specification text explicitly references multiple axes. For these operands, a single scalar constraint admits one interpretation per axis, making policy evaluation non-deterministic. We classify ODRL’s left operands by value-domain structure (scalar, dimensional, concept-valued), grounded in the ODRL 2.2 specification text, and show that dimensional ambiguity is intrinsic to the constraint syntax. We present an axis-decomposition framework that refines each dimensional operand into axis-specific scalar operands and prove four properties: deterministic interpretation, AABB completeness, sound over-approximation under projection, and conservative extension. Conflict detection operates in two layers: per-axis verdicts are always decidable; box-level verdicts compose through Strong Kleene conjunction into a three-valued logic (Conflict, Compatible, Unknown). For ODRL’s disjunctive (odrl:or) and exclusive-or (odrl:xone) logical constraints, where per-axis decomposition does not apply, the framework encodes coupled multi-axis conjectures directly. We instantiate the framework as the ODRL Spatial Axis Profile–15 axis-specific left operands for the five affected base terms–and evaluate it on 117 benchmark problems spanning nine categories across both TPTP FOF (Vampire) and SMT-LIB (Z3) encodings, achieving full concordance between provers. Benchmark scenarios are inspired by constraints arising in cultural heritage dataspaces such as Datenraum Kultur. All meta-theorems are mechanically verified in Isabelle/HOL.

Method: Develop denotational semantics mapping ODRL constraints to knowledge-base concepts. Use three-valued verdicts (Conflict, Compatible, Unknown) with soundness under incomplete knowledge. Cover all ODRL composition modes and semantic domains (taxonomic, mereological, nominal). Define order-preserving alignments between knowledge bases.

Result: Validated with 154 benchmarks across six knowledge base families and four structural KBs. Both Vampire theorem prover and Z3 SMT solver agreed on all verdicts. Found that exclusive composition (xone) requires stronger KB axioms than conjunction/disjunction.

Conclusion: Provides formal framework for ODRL policy analysis with sound conflict detection across different knowledge bases, preserving conflicts across KB standards and degrading gracefully to Unknown for unmapped concepts.

Abstract: ODRL’s six set-based operators – isA, isPartOf, hasPart, isAnyOf, isAllOf, isNoneOf – depend on external domain knowledge that the W3C specification leaves unspecified. Without it, every cross-dataspace policy comparison defaults to Unknown. We present a denotational semantics that maps each ODRL constraint to the set of knowledge-base concepts satisfying it. Conflict detection reduces to denotation intersection under a three-valued verdict – Conflict, Compatible, or Unknown – that is sound under incomplete knowledge. The framework covers all three ODRL composition modes (and, or, xone) and all three semantic domains arising in practice: taxonomic (class subsumption), mereological (part-whole containment), and nominal (identity). For cross-dataspace interoperability, we define order-preserving alignments between knowledge bases and prove two guarantees: conflicts are preserved across different KB standards, and unmapped concepts degrade gracefully to Unknown – never to false conflicts. A runtime soundness theorem ensures that design-time verdicts hold for all execution contexts. The encoding stays within the decidable EPR fragment of first-order logic. We validate it with 154 benchmarks across six knowledge base families (GeoNames, ISO 3166, W3C DPV, a GDPR-derived taxonomy, BCP 47, and ISO 639-3) and four structural KBs targeting adversarial edge cases. Both the Vampire theorem prover and the Z3 SMT solver agree on all 154 verdicts. A key finding is that exclusive composition (xone) requires strictly stronger KB axioms than conjunction or disjunction: open-world semantics blocks exclusivity even when positive evidence appears to satisfy exactly one branch.

Method: Two-stage framework: Stage I builds large-scale financial news event dataset (62,400 articles with 10 event types, stocks, sentiment, CAR). Stage II performs decision-oriented fine-tuning combining supervised learning with reinforcement learning guided by Hierarchical Gated Reward Model (HGRM) to capture trade-offs among multiple trading objectives.

Result: Janus-Q achieves more consistent, interpretable, and profitable trading decisions than market indices and LLM baselines, improving Sharpe Ratio by up to 102.0% and increasing direction accuracy by over 17.5% compared to strongest competing strategies.

Conclusion: The proposed event-driven framework successfully elevates financial news events from auxiliary signals to primary decision units, demonstrating superior trading performance through unified data construction and decision-oriented optimization.

Abstract: Financial market movements are often driven by discrete financial events conveyed through news, whose impacts are heterogeneous, abrupt, and difficult to capture under purely numerical prediction objectives. These limitations have motivated growing interest in using textual information as the primary source of trading signals in learning-based systems. Two key challenges hinder existing approaches: (1) the absence of large-scale, event-centric datasets that jointly model news semantics and statistically grounded market reactions, and (2) the misalignment between language model reasoning and financially valid trading behavior under dynamic market conditions. To address these challenges, we propose Janus-Q, an end-to-end event-driven trading framework that elevates financial news events from auxiliary signals to primary decision units. Janus-Q unifies event-centric data construction and model optimization under a two-stage paradigm. Stage I focuses on event-centric data construction, building a large-scale financial news event dataset comprising 62,400 articles annotated with 10 fine-grained event types, associated stocks, sentiment labels, and event-driven cumulative abnormal return (CAR). Stage II performs decision-oriented fine-tuning, combining supervised learning with reinforcement learning guided by a Hierarchical Gated Reward Model (HGRM), which explicitly captures trade-offs among multiple trading objectives. Extensive experiments demonstrate that Janus-Q achieves more consistent, interpretable, and profitable trading decisions than market indices and LLM baselines, improving the Sharpe Ratio by up to 102.0% while increasing direction accuracy by over 17.5% compared to the strongest competing strategies.

Method: The survey examines benchmark landscapes and methodological evolution, categorizing approaches into three main aspects: 1) multimodal embedding models, 2) multimodal reranker models, and 3) integration of Retrieval-Augmented Generation (RAG) and Agentic systems for complex document intelligence.

Result: The paper provides a systematic overview of the VDR landscape, identifying key methodological approaches and their evolution in the MLLM era. It establishes a clear taxonomy of techniques and their applications in visual document understanding and retrieval.

Conclusion: The survey identifies persistent challenges and outlines promising future directions, aiming to provide a clear roadmap for future multimodal document intelligence research and development.

Abstract: With the rapid proliferation of multimodal information, Visual Document Retrieval (VDR) has emerged as a critical frontier in bridging the gap between unstructured visually rich data and precise information acquisition. Unlike traditional natural image retrieval, visual documents exhibit unique characteristics defined by dense textual content, intricate layouts, and fine-grained semantic dependencies. This paper presents the first comprehensive survey of the VDR landscape, specifically through the lens of the Multimodal Large Language Model (MLLM) era. We begin by examining the benchmark landscape, and subsequently dive into the methodological evolution, categorizing approaches into three primary aspects: multimodal embedding models, multimodal reranker models, and the integration of Retrieval-Augmented Generation (RAG) and Agentic systems for complex document intelligence. Finally, we identify persistent challenges and outline promising future directions, aiming to provide a clear roadmap for future multimodal document intelligence.

Method: ReAttn uses two post-hoc adjustments: 1) Cross-document IDF weighting to down-weight attention on query-overlapping tokens that frequently appear across candidate documents, reducing lexical bias and emphasizing distinctive terms, and 2) Entropy-based regularization to mitigate over-concentrated attention, encouraging more balanced distribution across informative tokens. Both operate directly on existing attention weights without additional training or supervision.

Result: Extensive experiments demonstrate the effectiveness of ReAttn in improving attention-based re-ranking performance by addressing attention concentration and lexical bias issues.

Conclusion: ReAttn provides an effective post-hoc solution to improve attention-based re-ranking methods by reducing lexical bias and balancing attention distribution, enhancing ranking quality without requiring additional training.

Abstract: The strong capabilities of recent Large Language Models (LLMs) have made them highly effective for zero-shot re-ranking task. Attention-based re-ranking methods, which derive relevance scores directly from attention weights, offer an efficient and interpretable alternative to generation-based re-ranking methods. However, they still face two major limitations. First, attention signals are highly concentrated a small subset of tokens within a few documents, making others indistinguishable. Second, attention often overemphasizes phrases lexically similar to the query, yielding biased rankings that irrelevant documents with mere lexical resemblance are regarded as relevant. In this paper, we propose \textbf{ReAttn}, a post-hoc re-weighting strategy for attention-based re-ranking methods. It first compute the cross-document IDF weighting to down-weight attention on query-overlapping tokens that frequently appear across the candidate documents, reducing lexical bias and emphasizing distinctive terms. It then employs entropy-based regularization to mitigate over-concentrated attention, encouraging a more balanced distribution across informative tokens. Both adjustments operate directly on existing attention weights without additional training or supervision. Extensive experiments demonstrate the effectiveness of our method.

Method: Train bilingual speech-text Matryoshka embedding model using large-scale data curation pipelines. Compare modeling strategies, finding modality fusion within frozen text Matryoshka model works best. Analyze cost-accuracy trade-offs across Matryoshka dimensions.

Result: Model enables efficient retrieval of French text from Wolof speech queries without ASR-translation pipelines. Generalizes to other tasks like speech intent detection. Information concentrated in few components, suggesting efficiency improvements.

Conclusion: First successful bilingual speech-text Matryoshka model for under-represented language pair demonstrates effective cross-modal retrieval and generalization capabilities with potential for efficiency gains.

Abstract: Speakers of under-represented languages face both a language barrier, as most online knowledge is in a few dominant languages, and a modality barrier, since information is largely text-based while many languages are primarily oral. We address this for French-Wolof by training the first bilingual speech-text Matryoshka embedding model, enabling efficient retrieval of French text from Wolof speech queries without relying on a costly ASR-translation pipelines. We introduce large-scale data curation pipelines and new benchmarks, compare modeling strategies, and show that modality fusion within a frozen text Matryoshka model performs best. Although trained only for retrieval, the model generalizes well to other tasks, such as speech intent detection, indicating the learning of general semantic representations. Finally, we analyze cost-accuracy trade-offs across Matryoshka dimensions and ranks, showing that information is concentrated only in a few components, suggesting potential for efficiency improvements.

Method: QuIeTT performs lossless schema and value normalization, exposes implicit relations, and preserves full provenance via raw table snapshots. It transforms raw tables into a single SQL-ready canonical representation before any test-time queries are observed.

Result: Experiments on WikiTQ, HiTab, NQ-Table, and SequentialQA show consistent gains across models and reasoning paradigms, with particularly strong improvements on structurally diverse, unseen questions.

Conclusion: By decoupling table transformation from reasoning, QuIeTT enables cleaner, more reliable, and highly efficient querying without modifying downstream models.

Abstract: Real-world tables often exhibit irregular schemas, heterogeneous value formats, and implicit relational structure, which degrade the reliability of downstream table reasoning and question answering. Most existing approaches address these issues in a query-dependent manner, entangling table cleanup with reasoning and thus limiting generalization. We introduce QuIeTT, a query-independent table transformation framework that preprocesses raw tables into a single SQL-ready canonical representation before any test-time queries are observed. QuIeTT performs lossless schema and value normalization, exposes implicit relations, and preserves full provenance via raw table snapshots. By decoupling table transformation from reasoning, QuIeTT enables cleaner, more reliable, and highly efficient querying without modifying downstream models. Experiments on four benchmarks, WikiTQ, HiTab, NQ-Table, and SequentialQA show consistent gains across models and reasoning paradigms, with particularly strong improvements on a challenge set of structurally diverse, unseen questions.

Method: 1) Develop Generative Item Response Theory (GIRT) model that estimates student knowledge from open-ended responses and predicts responses to unseen questions, trained via supervised fine-tuning and preference optimization. 2) Introduce three question selection algorithms based on uncertainty, linguistic diversity, and information of sampled student responses. 3) Test on two real-world programming datasets.

Result: GENCAT outperforms existing CAT baselines, achieving up to 4.32% AUC improvement in early testing stages on programming datasets.

Conclusion: The proposed GENCAT framework successfully leverages LLMs for adaptive testing, enabling better knowledge estimation from open-ended responses and more effective question selection, particularly valuable for programming education.

Abstract: Existing computerized Adaptive Testing (CAT) frameworks are typically built on predicting the correctness of a student response to a question. Although effective, this approach fails to leverage textual information in questions and responses, especially for open-ended questions. In this work, we propose GENCAT (\textbf{GEN}erative \textbf{CAT}), a novel CAT framework that leverages Large Language Models for knowledge estimate and question selection. First, we develop a Generative Item Response Theory (GIRT) model that enables us to estimate student knowledge from their open-ended responses and predict responses to unseen questions. We train the model in a two-step process, first via Supervised Fine-Tuning and then via preference optimization for knowledge-response alignment. Second, we introduce three question selection algorithms that leverage the generative capabilities of the GIRT model, based on the uncertainty, linguistic diversity, and information of sampled student responses. Third, we conduct experiments on two real-world programming datasets and demonstrate that GENCAT outperforms existing CAT baselines, achieving an AUC improvement of up to 4.32% in the key early testing stages.

Method: An inference-time agentic framework that decomposes summarization into coordinated stages: context selection/compression, initial draft generation, identification of weakly supported spans using internal attention grounding signals, and selective revision of flagged content under supervisory control.

Result: AgenticSum demonstrates consistent improvements over vanilla LLMs and other baselines across multiple evaluation metrics on two public datasets, including reference-based metrics, LLM-as-a-judge assessment, and human evaluation.

Conclusion: Structured, agentic design with targeted correction offers an effective inference-time solution to improve factual consistency in clinical note summarization using LLMs.

Abstract: Large language models (LLMs) offer substantial promise for automating clinical text summarization, yet maintaining factual consistency remains challenging due to the length, noise, and heterogeneity of clinical documentation. We present AgenticSum, an inference-time, agentic framework that separates context selection, generation, verification, and targeted correction to reduce hallucinated content. The framework decomposes summarization into coordinated stages that compress task-relevant context, generate an initial draft, identify weakly supported spans using internal attention grounding signals, and selectively revise flagged content under supervisory control. We evaluate AgenticSum on two public datasets, using reference-based metrics, LLM-as-a-judge assessment, and human evaluation. Across various measures, AgenticSum demonstrates consistent improvements compared to vanilla LLMs and other strong baselines. Our results indicate that structured, agentic design with targeted correction offers an effective inference time solution to improve clinical note summarization using LLMs.

Method: Proposes Edge Alignment as a distinct approach with seven interdependent pillars organized into three phases. This approach preserves multi-dimensional value structure, supports plural democratic representation, and incorporates epistemic mechanisms for interaction and clarification.

Result: The paper identifies key challenges in data collection, training objectives, and evaluation, outlining complementary technical and governance directions. It reframes alignment as a lifecycle problem of dynamic normative governance rather than single-instance optimization.

Conclusion: Edge Alignment offers a framework to address limitations of current alignment practices by preserving value diversity, supporting plural representation, and incorporating uncertainty mechanisms, shifting alignment from optimization to ongoing governance.

Abstract: Large language models are being deployed in complex socio-technical systems, which exposes limits in current alignment practice. We take the position that the dominant paradigm of General Alignment, which compresses diverse human values into a single scalar reward, reaches a structural ceiling in settings with conflicting values, plural stakeholders, and irreducible uncertainty. These failures follow from the mathematics and incentives of scalarization and lead to \textbf{structural} value flattening, \textbf{normative} representation loss, and \textbf{cognitive} uncertainty blindness. We introduce Edge Alignment as a distinct approach in which systems preserve multi dimensional value structure, support plural and democratic representation, and incorporate epistemic mechanisms for interaction and clarification. To make this approach practical, we propose seven interdependent pillars organized into three phases. We identify key challenges in data collection, training objectives, and evaluation, outlining complementary technical and governance directions. Taken together, these measures reframe alignment as a lifecycle problem of dynamic normative governance rather than as a single instance optimization task.

Method: Model LLM output as an information source generating symbols from a finite alphabet with constant random distribution (stationary source). Compare source entropy per word to natural language entropy using the Open American National Corpus (OANC) for written and spoken language.

Result: The word entropy of LLMs is lower than the word entropy of natural speech in both written and spoken forms.

Conclusion: LLMs have reduced entropy compared to natural language, which has implications for training LLMs on web data that increasingly contains LLM-generated content.

Abstract: In this study, the output of large language models (LLM) is considered an information source generating an unlimited sequence of symbols drawn from a finite alphabet. Given the probabilistic nature of modern LLMs, we assume a probabilistic model for these LLMs, following a constant random distribution and the source itself thus being stationary. We compare this source entropy (per word) to that of natural language (written or spoken) as represented by the Open American National Corpus (OANC). Our results indicate that the word entropy of such LLMs is lower than the word entropy of natural speech both in written or spoken form. The long-term goal of such studies is to formalize the intuitions of information and uncertainty in large language training to assess the impact of training an LLM from LLM generated training data. This refers to texts from the world wide web in particular.

Method: Tested 3 popular LLMs on language comprehension tasks across 12 languages from 5 language families, comparing performance against human baselines and analyzing factors like tokenization, language distance, training data size, and data origin.

Result: Models showed remarkable linguistic accuracy across typologically diverse languages but fell behind human baselines in all languages. Surprisingly, English was systematically outperformed by several Romance languages, including lower-resource ones.

Conclusion: LLM comprehension varies significantly across languages, challenging assumptions about English superiority. Performance is influenced by multiple factors including tokenization, language distance from training data languages, and data origin in WEIRD vs. non-WEIRD communities.

Abstract: Large Language Models (LLMs) play a critical role in how humans access information. While their core use relies on comprehending written requests, our understanding of this ability is currently limited, because most benchmarks evaluate LLMs in high-resource languages predominantly spoken by Western, Educated, Industrialised, Rich, and Democratic (WEIRD) communities. The default assumption is that English is the best-performing language for LLMs, while smaller, low-resource languages are linked to less reliable outputs, even in multilingual, state-of-the-art models. To track variation in the comprehension abilities of LLMs, we prompt 3 popular models on a language comprehension task across 12 languages, representing the Indo-European, Afro-Asiatic, Turkic, Sino-Tibetan, and Japonic language families. Our results suggest that the models exhibit remarkable linguistic accuracy across typologically diverse languages, yet they fall behind human baselines in all of them, albeit to different degrees. Contrary to what was expected, English is not the best-performing language, as it was systematically outperformed by several Romance languages, even lower-resource ones. We frame the results by discussing the role of several factors that drive LLM performance, such as tokenization, language distance from Spanish and English, size of training data, and data origin in high- vs. low-resource languages and WEIRD vs. non-WEIRD communities.

Method: Systematic analysis of how different types of retrieved documents affect hidden states of LLMs. Examined internal representation shifts across four QA datasets and three LLMs under controlled single- and multi-document settings.

Result: Revealed how context relevancy and layer-wise processing influence internal representations, providing explanations for LLM output behaviors and insights for RAG system design.

Conclusion: Understanding internal representation shifts in RAG provides valuable insights into how retrieved context affects model behavior, offering guidance for better RAG system design.

Abstract: Retrieval-augmented generation (RAG) enhances large language models (LLMs) by conditioning generation on retrieved external documents, but the effect of retrieved context is often non-trivial. In realistic retrieval settings, the retrieved document set often contains a mixture of documents that vary in relevance and usefulness. While prior work has largely examined these phenomena through output behavior, little is known about how retrieved context shapes the internal representations that mediate information integration in RAG. In this work, we study RAG through the lens of latent representations. We systematically analyze how different types of retrieved documents affect the hidden states of LLMs, and how these internal representation shifts relate to downstream generation behavior. Across four question-answering datasets and three LLMs, we analyze internal representations under controlled single- and multi-document settings. Our results reveal how context relevancy and layer-wise processing influence internal representations, providing explanations on LLMs output behaviors and insights for RAG system design.

Method: Organizes a competition with two tracks: general data-efficient pretraining and new multilingual track, plus workshop papers on related topics.

Result: Calls for participation in the 4th BabyLM competition and workshop submissions on relevant research areas.

Conclusion: BabyLM continues to promote research at the intersection of cognitive science and language modeling through competitions and workshops.

Abstract: BabyLM aims to dissolve the boundaries between cognitive modeling and language modeling. We call for both workshop papers and for researchers to join the 4th BabyLM competition. As in previous years, we call for participants in the data-efficient pretraining challenge in the general track. This year, we also offer a new track: Multilingual. We also call for papers outside the competition in any relevant areas. These include training efficiency, cognitively plausible research, weak model evaluation, and more.

Method: Created NanoKnow benchmark dataset partitioning Natural Questions and SQuAD questions based on whether answers are present in nanochat’s pre-training corpus, then conducted experiments with eight nanochat checkpoints

Result: Found: (1) closed-book accuracy depends on answer frequency in pre-training data, (2) external evidence mitigates frequency dependence, (3) parametric and external knowledge are complementary, (4) non-relevant information harms accuracy based on position and quantity

Conclusion: NanoKnow provides a transparent framework for studying knowledge encoding in LLMs, revealing important insights about how parametric and external knowledge interact

Abstract: How do large language models (LLMs) know what they know? Answering this question has been difficult because pre-training data is often a “black box” – unknown or inaccessible. The recent release of nanochat – a family of small LLMs with fully open pre-training data – addresses this as it provides a transparent view into where a model’s parametric knowledge comes from. Towards the goal of understanding how knowledge is encoded by LLMs, we release NanoKnow, a benchmark dataset that partitions questions from Natural Questions and SQuAD into splits based on whether their answers are present in nanochat’s pre-training corpus. Using these splits, we can now properly disentangle the sources of knowledge that LLMs rely on when producing an output. To demonstrate NanoKnow’s utility, we conduct experiments using eight nanochat checkpoints. Our findings show: (1) closed-book accuracy is strongly influenced by answer frequency in the pre-training data, (2) providing external evidence can mitigate this frequency dependence, (3) even with external evidence, models are more accurate when answers were seen during pre-training, demonstrating that parametric and external knowledge are complementary, and (4) non-relevant information is harmful, with accuracy decreasing based on both the position and the number of non-relevant contexts. We release all NanoKnow artifacts at https://github.com/castorini/NanoKnow.

Method: Proposes Selective CoT, an inference-time strategy that first predicts whether a question requires reasoning and generates rationales only when needed. Evaluated on Llama-3.1-8B and Qwen-2.5-7B using four biomedical QA benchmarks.

Result: Reduced inference time by 13-45% and token usage by 8-47% with ≤4% accuracy loss. In some cases, achieved both higher accuracy and greater efficiency than standard CoT.

Conclusion: Selective CoT provides a simple, model-agnostic, cost-effective approach for medical QA that dynamically balances reasoning depth with efficiency, enhancing real-world deployability.

Abstract: Objective: To improve the efficiency of medical question answering (MedQA) with large language models (LLMs) by avoiding unnecessary reasoning while maintaining accuracy. Methods: We propose Selective Chain-of-Thought (Selective CoT), an inference-time strategy that first predicts whether a question requires reasoning and generates a rationale only when needed. Two open-source LLMs (Llama-3.1-8B and Qwen-2.5-7B) were evaluated on four biomedical QA benchmarks-HeadQA, MedQA-USMLE, MedMCQA, and PubMedQA. Metrics included accuracy, total generated tokens, and inference time. Results: Selective CoT reduced inference time by 13-45% and token usage by 8-47% with minimal accuracy loss ($\leq$4%). In some model-task pairs, it achieved both higher accuracy and greater efficiency than standard CoT. Compared with fixed-length CoT, Selective CoT reached similar or superior accuracy at substantially lower computational cost. Discussion: Selective CoT dynamically balances reasoning depth and efficiency by invoking explicit reasoning only when beneficial, reducing redundancy on recall-type questions while preserving interpretability. Conclusion: Selective CoT provides a simple, model-agnostic, and cost-effective approach for medical QA, aligning reasoning effort with question complexity to enhance real-world deployability of LLM-based clinical systems.

Method: KNIGHT constructs topic-specific knowledge graphs from external sources (entities and relations), then uses these compressed graphs to generate MCQs with controlled difficulty levels (including multi-hop questions) without re-feeding full source texts.

Result: KNIGHT achieves token- and cost-efficient generation from reusable graph representations, produces high-quality MCQs across five criteria (fluency, unambiguity, relevance, option uniqueness, answerability), and yields model rankings aligned with MMLU benchmarks while supporting topic-specific and difficulty-controlled evaluation.

Conclusion: KNIGHT provides an efficient, reusable framework for generating high-quality MCQ datasets for RAG evaluation, addressing the bottleneck of dataset creation while enabling fine-grained, topic-specific assessment.

Abstract: With the rise of large language models (LLMs), they have become instrumental in applications such as Retrieval-Augmented Generation (RAG). Yet evaluating these systems remains bottlenecked by the time and cost of building specialized assessment datasets. We introduce KNIGHT, an LLM-based, knowledge-graph-driven framework for generating multiple-choice question (MCQ) datasets from external sources. KNIGHT constructs a topic-specific knowledge graph, a structured and parsimonious summary of entities and relations, that can be reused to generate instructor-controlled difficulty levels, including multi-hop questions, without repeatedly re-feeding the full source text. This knowledge graph acts as a compressed, reusable state, making question generation a cheap read over the graph. We instantiate KNIGHT on Wikipedia/Wikidata while keeping the framework domain- and ontology-agnostic. As a case study, KNIGHT produces six MCQ datasets in History, Biology, and Mathematics. We evaluate quality on five criteria: fluency, unambiguity (single correct answer), topic relevance, option uniqueness, and answerability given the provided sources (as a proxy for hallucination). Results show that KNIGHT enables token- and cost-efficient generation from a reusable graph representation, achieves high quality across these criteria, and yields model rankings aligned with MMLU-style benchmarks, while supporting topic-specific and difficulty-controlled evaluation.

Method: Deriving confidence from the distribution of multiple randomly sampled model generations using three consistency measures, evaluated across various open/closed-source models on nine reasoning datasets.

Result: Consistency-based calibration outperforms existing post-hoc methods; intermediate explanations, model scaling, and larger sample sizes enhance calibration; instruction-tuning makes calibration more difficult; consistency-based confidence can improve model performance.

Conclusion: Consistency from multiple generations provides effective calibration for LLMs, with practical guidance offered for choosing suitable consistency metrics based on different LM characteristics.

Abstract: Accurately gauging the confidence level of Large Language Models’ (LLMs) predictions is pivotal for their reliable application. However, LLMs are often uncalibrated inherently and elude conventional calibration techniques due to their proprietary nature and massive scale. In this work, we explore the potential of deriving confidence from the distribution of multiple randomly sampled model generations, via three measures of consistency. We perform an extensive evaluation across various open and closed-source models on nine reasoning datasets. Results show that consistency-based calibration methods outperform existing post-hoc approaches. Meanwhile, we find that factors such as intermediate explanations, model scaling, and larger sample sizes enhance calibration, while instruction-tuning makes calibration more difficult. Moreover, confidence scores obtained from consistency have the potential to enhance model performance. Finally, we offer practical guidance on choosing suitable consistency metrics for calibration, tailored to the characteristics of various LMs.

Method: Created ViTextVQA dataset with Vietnamese text-based VQA annotations, and proposed ViTextBLIP-2 - a multimodal feature fusion method optimized for Vietnamese text-based VQA, focusing on OCR token ordering importance.

Result: Dataset contains 16,000+ images and 50,000+ question-answer pairs; experiments show significant performance improvements by considering OCR token ordering in answer generation.

Conclusion: ViTextVQA fills the gap for Vietnamese text-based VQA research, and proper handling of OCR text token ordering is crucial for performance in text-based VQA tasks.

Abstract: Visual Question Answering (VQA) is a challenging task that requires the joint understanding of natural language and visual content. While early research primarily focused on recognizing objects and scene context, it often overlooked scene text-an essential source of explicit semantic information. This paper introduces \textbf{ViTextVQA} (\textbf{Vi}etnamese \textbf{Text}-based \textbf{V}isual \textbf{Q}uestion \textbf{A}nswering), the first large-scale Vietnamese dataset specializing in text-based VQA. The dataset contains \textbf{over 16,000} images and \textbf{over 50,000} question-answer pairs. To tackle this task efficiently, \textbf{ViTextBLIP-2} (Vietnamese Text-based Bootstrapped Language-Image Model via Fine-tuning) is proposed, a novel multimodal feature fusion method designed to optimize Vietnamese text-based VQA. Experiments with state-of-the-art models highlight the importance of token ordering in OCR text for answer generation, leading to significant performance improvements. The ViTextVQA dataset is publicly available for research purposes.

Method: Used neural network language models to study passive exceptions, characterized human judgments, compared model judgments to human data, and tested hypotheses by training models on manipulated corpora with specific sentence types removed/altered/introduced.

Result: Model passivizability judgments largely matched human judgments; both entrenchment (frequency) and affectedness (semantics) made independent contributions to verb passivizability; methodological approach of corpus manipulation proved effective.

Conclusion: Language models can learn passive exceptions from linguistic input similarly to humans, with both frequency and semantic factors playing roles; corpus manipulation methods are valuable for studying language acquisition.

Abstract: Grammatical rules in natural languages are often characterized by exceptions. How do language learners learn these exceptions to otherwise general patterns? Here, we study this question through the case study of English passivization. While passivization is in general quite productive, there are cases where it cannot apply (cf. the following sentence is ungrammatical: *One hour was lasted by the meeting). Using neural network language models as theories of language acquisition, we explore the sources of indirect evidence that a learner can leverage to learn whether a verb can be passivized. We first characterize English speakers’ judgments of exceptions to the passive, and confirm that speakers find some verbs more passivizable than others. We then show that a neural network language model’s verb passivizability judgments are largely similar to those displayed by humans, suggesting that evidence for these exceptions is available in the linguistic input. Finally, we test two hypotheses as to the source of evidence that language models use to learn these restrictions: frequency (entrenchment) and semantics (affectedness). We do so by training models on versions of the corpus that have had sentences of the types implicated by each hypothesis removed, altered, or introduced. We find support for both hypotheses: entrenchment and affectedness make independent contributions to a verb’s passivizability. From a methodological point of view, this study highlights the utility of altering a language model’s training data for answering questions where complete control over a learner’s input is vital.

Method: Experimented with various prompt designs, manipulating input context, reasoning approaches, and output forms to test LLMs’ thematic fit knowledge. Compared closed-weight and open-weight LLMs using different prompting strategies including multi-step reasoning

Result: Set new state-of-the-art on thematic fit benchmarks. Found that closed models achieve better overall scores and benefit from multi-step reasoning, but perform worse at filtering out generated sentences incompatible with specified predicate, role, and argument compared to open models

Conclusion: LLMs possess substantial knowledge of thematic fit, but there are systematic differences between closed and open models in how they respond to prompting strategies, particularly in filtering capabilities

Abstract: The thematic fit estimation task measures semantic arguments’ compatibility with a specific semantic role for a specific predicate. We investigate if LLMs have consistent, expressible knowledge of event arguments’ thematic fit by experimenting with various prompt designs, manipulating input context, reasoning, and output forms. We set a new state-of-the-art on thematic fit benchmarks, but show that closed and open weight LLMs respond differently to our prompting strategies: Closed models achieve better scores overall and benefit from multi-step reasoning, but they perform worse at filtering out generated sentences incompatible with the specified predicate, role, and argument.

Method: Augmented CICERO (a natural language agent with superhuman Diplomacy performance) to generate both move and message advice based on player intentions. Conducted experiments with a dozen Diplomacy games involving novice and experienced players under varying advice settings.

Result: Some generated advice was beneficial: helped novices compete with experienced players and in some instances surpass them. The mere presence of advice can be advantageous even when players don’t follow it.

Conclusion: AI-generated advice can meaningfully assist human players in complex strategy games, demonstrating potential for AI-human collaboration beyond just competitive performance.

Abstract: AIs can beat humans in game environments; however, how helpful those agents are to human remains understudied. We augment CICERO, a natural language agent that demonstrates superhuman performance in Diplomacy, to generate both move and message advice based on player intentions. A dozen Diplomacy games with novice and experienced players, with varying advice settings, show that some of the generated advice is beneficial. It helps novices compete with experienced players and in some instances even surpass them. The mere presence of advice can be advantageous, even if players do not follow it.

Method: Uses lightweight adapters with SLMs to facilitate parameter-efficient knowledge exchange between server and clients, minimizing computational and communication overhead

Result: Client SLMs show notable improvements with LLM assistance, while LLMs enhanced via FedCoLLM achieve comparable performance to direct fine-tuning on client data

Conclusion: FedCoLLM successfully enables mutual enhancement between LLMs and SLMs in federated settings while preserving privacy and efficiency

Abstract: By adapting Large Language Models (LLMs) to domain-specific tasks or enriching them with domain-specific knowledge, we can fully harness the capabilities of LLMs. Nonetheless, a gap persists in achieving simultaneous mutual enhancement between the server’s LLM and the downstream clients’ Small Language Models (SLMs). To address this, we propose FedCoLLM, a novel and parameter-efficient federated framework designed for co-tuning LLMs and SLMs. This approach is aimed at adaptively transferring server-side LLMs knowledge to clients’ SLMs while simultaneously enriching the LLMs with domain insights from the clients. To accomplish this, FedCoLLM utilizes lightweight adapters in conjunction with SLMs, facilitating knowledge exchange between server and clients in a manner that respects data privacy while also minimizing computational and communication overhead. Our evaluation of FedCoLLM, utilizing various public LLMs and SLMs across a range of NLP text generation tasks, reveals that the performance of clients’ SLMs experiences notable improvements with the assistance of the LLMs. Simultaneously, the LLMs enhanced via FedCoLLM achieves comparable performance to that obtained through direct fine-tuning on clients’ data. Our code has been contributed to the FATE open-source project and is now publicly accessible at https://github.com/FederatedAI/FATE-LLM/tree/main/python/fate_llm/algo/fedcollm.

Method: Develops four new architectural paradigms based on PerceiverAR, with ECP as the best performer. ECP uses both context and latent sequences in autoregressive training and employs pairwise segment attention for better information extraction while maintaining LongLoRA-level attention complexity.

Result: ECP significantly outperforms other state-of-the-art Transformer models on Wikitext-103, PG-19, and sCIFAR-10 benchmarks.

Conclusion: ECP successfully addresses the trade-off between computational efficiency and performance in Transformer architectures, offering improved language modeling while maintaining efficient attention complexity.

Abstract: One of the key challenges in Transformer architectures is the quadratic complexity of the attention mechanism, which limits the efficient processing of long sequences. Many recent research works have attempted to provide a reduction from the $O(n^2)$ time complexity of attention to semi-linear complexity. However, it remains an unsolved problem in the sense of maintaining high performance when complexity is reduced. One of the important works in this respect is the Perceiver class of architectures that have demonstrated excellent performance, while reducing the computation complexity. In this paper, we use the PerceiverAR as a basis and explore the design space of different trade-offs between preserving context and reducing attention complexity. To this end, we develop four new architectural paradigms, the best performing of which we denote as the Efficient Context propagating Perceiver (ECP). ECP has two major advantages over the PerceiverAR. First, the ECP architecture overcomes the main drawback of PercieverAR by utilizing both the context and the latent sequences in autoregressive training. Second, the ECP architecture operates with the same attention complexity as LongLoRA, making it computationally efficient. More importantly, via pairwise segment attention, it extracts better information resulting in improved language modeling. Empirically, we demonstrate that the ECP architecture significantly outperforms other state-of-the-art Transformer models on Wikitext-103, PG-19 and sCIFAR-10.

Method: Developed LiveIdeaBench with 1,180 keywords across 22 scientific domains, using single-keyword prompts to evaluate 40+ LLMs. Employed Guilford’s creativity theory to assess ideas across five dimensions: originality, feasibility, fluency, flexibility, and clarity using a dynamic panel of state-of-the-art LLMs as evaluators.

Result: Scientific idea generation capabilities measured by LiveIdeaBench are poorly predicted by standard general intelligence metrics. Models like QwQ-32B-preview achieve creative performance comparable to top-tier models despite significant gaps in general intelligence scores.

Conclusion: Specialized evaluation benchmarks are needed for scientific idea generation, and enhancing these capabilities may require different training strategies than those used for general problem-solving, potentially enabling AI tools for different scientific process stages.

Abstract: While Large Language Models (LLMs) demonstrate remarkable capabilities in scientific tasks such as literature analysis and experimental design (e.g., accurately extracting key findings from papers or generating coherent experimental procedures), existing evaluation benchmarks primarily assess performance using rich contextual inputs. We introduce LiveIdeaBench, a comprehensive benchmark evaluating LLMs’ scientific idea generation by assessing divergent thinking capabilities using single-keyword prompts. Drawing from Guilford’s creativity theory, our benchmark employs a dynamic panel of state-of-the-art LLMs to assess generated ideas across five key dimensions: originality, feasibility, fluency, flexibility, and clarity. Through extensive experimentation with over 40 leading models across 1,180 keywords spanning 22 scientific domains, we reveal that the scientific idea generation capabilities measured by our benchmark, are poorly predicted by standard metrics of general intelligence. Our results demonstrate that models like QwQ-32B-preview achieve creative performance comparable to top-tier models such as claude-3.7-sonnet:thinking, despite significant gaps in their general intelligence scores. These findings highlight the need for specialized evaluation benchmarks for scientific idea generation and suggest that enhancing these idea generation capabilities in LLMs may require different training strategies than those used for improving general problem-solving abilities, potentially enabling a wider range of AI tools tailored for different stages of the scientific process.

Method: GRASP uses gradient-based attribution on a small calibration dataset to identify and retain sensitivity-aware singular values. Instead of removing entire layers, it replaces redundant layers with minimal parameter sets, preserving only critical singular components.

Result: Experiments across multiple LLMs show GRASP consistently outperforms existing compression methods, achieving 90% of original model performance under 20% compression ratio.

Conclusion: GRASP provides an effective gradient-based approach for LLM compression that maintains strong performance through selective retention of critical singular components rather than indiscriminate layer pruning.

Abstract: Recent studies have demonstrated that many layers are functionally redundant in large language models (LLMs), enabling model compression by removing these layers to reduce inference cost. While such approaches can improve efficiency, indiscriminate layer pruning often results in significant performance degradation. In this paper, we propose GRASP (Gradient-based Retention of Adaptive Singular Parameters), a novel compression framework that mitigates this issue by preserving sensitivity-aware singular values. Unlike direct layer pruning, GRASP leverages gradient-based attribution on a small calibration dataset to adaptively identify and retain critical singular components. By replacing redundant layers with only a minimal set of parameters, GRASP achieves efficient compression while maintaining strong performance with minimal overhead. Experiments across multiple LLMs show that GRASP consistently outperforms existing compression methods, achieving 90% of the original model’s performance under 20% compression ratio.

Method: Introduced a novel benchmark simulating real-world diagnostic scenarios with noise and difficulty levels aligned with USMLE standards, and explored dialogue-based fine-tuning that transforms static datasets into conversational formats.

Result: Dialogue-tuned models outperformed traditional methods with 9.64% improvement in multi-round reasoning scenarios and 6.18% accuracy improvement in noisy environments.

Conclusion: Dialogue tuning is a promising approach for advancing clinically aligned and robust medical AI systems that better capture iterative reasoning processes.

Abstract: Current medical AI systems often fail to replicate real-world clinical reasoning, as they are predominantly trained and evaluated on static text and question-answer tasks. These tuning methods and benchmarks overlook critical aspects like evidence-based reasoning and handling distracting information. To bridge this gap, we introduce a novel benchmark that simulates real-world diagnostic scenarios, integrating noise and difficulty levels aligned with USMLE standards. Moreover, we explore dialogue-based fine-tuning, which transforms static datasets into conversational formats to better capture iterative reasoning processes. Experiments show that dialogue-tuned models outperform traditional methods, with improvements of $9.64%$ in multi-round reasoning scenarios and $6.18%$ in accuracy in a noisy environment. Our findings highlight dialogue tuning as a promising approach for advancing clinically aligned and robust medical AI systems.

Method: VQEL incorporates vector quantization into message generation, allowing agents to perform self-play using discrete internal representations from a learned codebook while maintaining end-to-end differentiability. The codebook induces a symbolic vocabulary that transfers to mutual play.

Result: Agents pretrained via VQEL self-play achieve more consistent symbol alignment and higher task success when later engaged in mutual interaction compared to existing approaches.

Conclusion: Self-play serves as a principled mechanism for learning discrete communication protocols, addressing optimization and representational challenges in emergent language systems through vector quantization.

Abstract: Emergent Language (EL) focuses on the emergence of communication among artificial agents. Although symbolic communication channels more closely mirror the discrete nature of human language, learning such protocols remains fundamentally difficult due to the non-differentiability of symbol sampling. Existing approaches typically rely on high-variance gradient estimators such as REINFORCE or on continuous relaxations such as Gumbel-Softmax, both of which suffer from limitations in training stability and scalability. Motivated by cognitive theories that emphasize intrapersonal processes preceding communication, we explore self-play as a substrate for language emergence prior to mutual interaction. We introduce Vector Quantized Emergent Language (VQEL), a novel architecture that incorporates vector quantization into the message generation process. VQEL enables agents to perform self-play using discrete internal representations derived from a learned codebook while preserving end-to-end differentiability. Moreover, the resulting vector-quantized codebook naturally induces a symbolic vocabulary that can be directly transferred and aligned during subsequent mutual play with other agents. Empirical results show that agents pretrained via VQEL self-play achieve more consistent symbol alignment and higher task success when later engaged in mutual interaction. These findings position self-play as a principled and effective mechanism for learning discrete communication protocols, addressing key optimization and representational challenges in emergent language systems.

Method: Comprehensive survey methodology with task-oriented taxonomy spanning instruction following (mathematics, coding) and conversational engagement (role-playing, healthcare, education, adversarial jailbreak). Systematic examination of benchmarks, datasets, and enhancement methodologies including model-centric strategies, external integration approaches, and agent-based techniques.

Result: Organized existing benchmarks and datasets into coherent categories reflecting the evolving landscape of multi-turn dialogue evaluation. Reviewed broad spectrum of enhancement methodologies and identified key challenges in multi-turn LLM interactions.

Conclusion: Identified open challenges and promising directions for future research to improve robustness and effectiveness of multi-turn LLM interactions, highlighting the importance of continued development in this area for real-world applications.

Abstract: Recent advances in large language models (LLMs) have substantially improved single-turn task performance, yet real-world applications increasingly demand sophisticated multi-turn interactions. This survey provides a comprehensive review of recent progress in evaluating and enhancing multi-turn LLM interactions. Centered on a task-oriented taxonomy-spanning instruction following in domains such as mathematics and coding, and conversational engagement in role-playing, healthcare, education, and adversarial jailbreak settings-we systematically examine the challenges of maintaining context, coherence, fairness, and responsiveness across prolonged dialogues. We organize existing benchmarks and datasets into coherent categories reflecting the evolving landscape of multi-turn dialogue evaluation, and review a broad spectrum of enhancement methodologies, including model-centric strategies (in-context learning, supervised fine-tuning, reinforcement learning, and architectural innovations), external integration approaches (memory augmentation, retrieval-based methods, and knowledge graphs), and agent-based techniques for collaborative interaction. Finally, we identify open challenges and promising directions for future research to further improve the robustness and effectiveness of multi-turn LLM interactions.

Method: 1) Identify correlation factors (linguistic features, semantic similarity, toxicity) across multiple experimental datasets; 2) Evaluate adversarial robustness of fine-tuned models; 3) Analyze persona shifts and interpretability traits to understand how dataset factors contribute to attack success rates; 4) Explore causal relationships for insights into adversarial defense strategies.

Result: The research reveals that fine-tuning data characteristics can create unexpected vulnerabilities, and provides insights into how dataset factors affect adversarial attack success rates and model robustness.

Conclusion: Dataset design plays a crucial role in preserving model alignment and preventing accidental vulnerabilities during fine-tuning, offering new insights for adversarial defense strategies.

Abstract: As large language models (LLMs) gain popularity, their vulnerability to adversarial attacks emerges as a primary concern. While fine-tuning models on domain-specific datasets is often employed to improve model performance, it can inadvertently introduce vulnerabilities within the underlying model. In this work, we investigate Accidental Vulnerability, unexpected vulnerabilities arising from characteristics of fine-tuning data. We begin by identifying potential correlation factors such as linguistic features, semantic similarity, and toxicity across multiple experimental datasets. We then evaluate the adversarial robustness of these fine-tuned models, analyzing persona shifts and interpretability traits to understand how dataset factors contribute to attack success rates. Lastly, we explore causal relationships that offer new insights into adversarial defense strategies, highlighting the crucial role of dataset design in preserving model alignment. Our code is available at https://github.com/psyonp/accidental_vulnerability.

Method: Inductive, bottom-up approach using dependency-parsed Universal Dependencies treebanks; defines syntactic structures as delexicalized dependency (sub)trees; analyzes size, diversity, distribution, overlap, and keyness analysis of structures across spoken/written corpora in English and Slovenian.

Result: Spoken corpora contain fewer and less diverse syntactic structures than written counterparts; limited overlap between speech/writing syntactic inventories (most speech structures don’t occur in writing); consistent cross-linguistic patterns; speech-specific structures associated with interactivity, context-grounding, and economy of expression.

Conclusion: The framework offers scalable, language-independent method for studying syntactic variation across corpora, revealing modality-specific preferences reflecting distinct demands of real-time interaction vs elaborated writing, supporting data-driven theories of grammar in use.

Abstract: This paper presents a novel treebank-driven approach to comparing syntactic structures in speech and writing using dependency-parsed corpora. Adopting a fully inductive, bottom-up method, we define syntactic structures as delexicalized dependency (sub)trees and extract them from spoken and written Universal Dependencies (UD) treebanks in two syntactically distinct languages, English and Slovenian. For each corpus, we analyze the size, diversity, and distribution of syntactic inventories, their overlap across modalities, and the structures most characteristic of speech. Results show that, across both languages, spoken corpora contain fewer and less diverse syntactic structures than their written counterparts, with consistent cross-linguistic preferences for certain structural types across modalities. Strikingly, the overlap between spoken and written syntactic inventories is very limited: most structures attested in speech do not occur in writing, pointing to modality-specific preferences in syntactic organization that reflect the distinct demands of real-time interaction and elaborated writing. This contrast is further supported by a keyness analysis of the most frequent speech-specific structures, which highlights patterns associated with interactivity, context-grounding, and economy of expression. We argue that this scalable, language-independent framework offers a useful general method for systematically studying syntactic variation across corpora, laying the groundwork for more comprehensive data-driven theories of grammar in use.

Method: Proposes Bayesian Attention Mechanism (BAM) that formulates positional encoding as a prior in a probabilistic model, unifying existing methods and introducing Generalized Gaussian positional priors.

Result: Achieves accurate information retrieval at 500× training context length, outperforming previous SOTA in long-context retrieval accuracy while maintaining comparable perplexity with minimal added parameters.

Conclusion: BAM provides a theoretical foundation for positional encoding that enables superior context length extrapolation and could benefit multimodal models requiring long-context understanding.

Abstract: Transformer-based language models rely on positional encoding (PE) to handle token order and support context length extrapolation. However, existing PE methods lack theoretical clarity and rely on limited evaluation metrics to substantiate their extrapolation claims. We propose the Bayesian Attention Mechanism (BAM), a theoretical framework that formulates positional encoding as a prior within a probabilistic model. BAM unifies existing methods (e.g., NoPE and ALiBi) and motivates a new Generalized Gaussian positional prior that substantially improves long-context generalization. Empirically, BAM enables accurate information retrieval at $500\times$ the training context length, outperforming previous state-of-the-art context length generalization in long context retrieval accuracy while maintaining comparable perplexity and introducing minimal additional parameters.

Method: Eso-LMs fuse AR and Masked Diffusion Model (MDM) paradigms using causal attention instead of bidirectional attention. This connection between MDMs and Any-Order autoregressive models enables exact likelihood computation and introduces KV caching for MDMs while preserving parallel generation capabilities.

Result: Eso-LMs achieve new state-of-the-art on the speed-quality Pareto frontier for unconditional generation. On long contexts, they provide 14-65× faster inference than standard MDMs and 3-4× faster inference than prior semi-autoregressive approaches.

Conclusion: The proposed Eso-LMs successfully bridge AR and diffusion paradigms, offering improved perplexity, efficient inference with KV caching, and parallel generation capabilities, making them a promising approach for language modeling.

Abstract: Diffusion-based language models offer a compelling alternative to autoregressive (AR) models by enabling parallel and controllable generation. Within this family, Masked Diffusion Models (MDMs) currently perform best but still underperform AR models in perplexity and lack key inference-time efficiency features, most notably KV caching. We introduce Eso-LMs, a new family of models that fuses AR and MDM paradigms, smoothly interpolating between their perplexities while overcoming their respective limitations. Unlike prior work, which uses transformers with bidirectional attention as MDM denoisers, we exploit the connection between MDMs and Any-Order autoregressive models and adopt causal attention. This design lets us compute the exact likelihood of MDMs for the first time and, crucially, enables us \to introduce KV caching for MDMs while preserving parallel generation for the first time, significantly improving inference efficiency. Combined with an optimized sampling schedule, Eso-LMs achieves a new state of the art on the speed-quality Pareto frontier for unconditional generation. On long contexts, it yields $\mathbf{14 - 65{}\times}$ faster inference than standard MDMs and $\mathbf{3 - 4{}\times}$ faster inference than prior semi-autoregressive approaches. We provide code, model checkpoints, and a video tutorial on the project page: https://s-sahoo.com/Eso-LMs.

Method: Created EuroGEST by expanding an existing expert-informed benchmark covering 16 gender stereotypes using translation tools, quality estimation metrics, and morphological heuristics, then validated with human evaluations.

Result: Found strongest stereotypes across all models and languages: women as ‘beautiful’, ’empathetic’, ’neat’; men as ’leaders’, ‘strong/tough’, ‘professional’. Larger models encode stereotypes more strongly, and instruction finetuning doesn’t consistently reduce them.

Conclusion: Highlights need for multilingual fairness studies in LLMs and provides scalable methods/resources for auditing gender bias across languages.

Abstract: Large language models increasingly support multiple languages, yet most benchmarks for gender bias remain English-centric. We introduce EuroGEST, a dataset designed to measure gender-stereotypical reasoning in LLMs across English and 29 European languages. EuroGEST builds on an existing expert-informed benchmark covering 16 gender stereotypes, expanded in this work using translation tools, quality estimation metrics, and morphological heuristics. Human evaluations confirm that our data generation method results in high accuracy of both translations and gender labels across languages. We use EuroGEST to evaluate 24 multilingual language models from six model families, demonstrating that the strongest stereotypes in all models across all languages are that women are ‘beautiful’, ’empathetic’ and ’neat’ and men are ’leaders’, ‘strong, tough’ and ‘professional’. We also show that larger models encode gendered stereotypes more strongly and that instruction finetuning does not consistently reduce gendered stereotypes. Our work highlights the need for more multilingual studies of fairness in LLMs and offers scalable methods and resources to audit gender bias across languages.

Method: The authors propose GraphRAG-Bench, a comprehensive benchmark featuring: 1) A dataset with tasks of increasing difficulty covering fact retrieval, complex reasoning, contextual summarization, and creative generation; 2) Systematic evaluation across the entire pipeline from graph construction and knowledge retrieval to final generation.

Result: The benchmark enables systematic investigation of conditions when GraphRAG surpasses traditional RAG and the underlying reasons for its success, providing guidelines for practical application. All resources are made available to the community.

Conclusion: GraphRAG-Bench addresses the critical need for standardized evaluation of GraphRAG systems, helping determine when graph structures provide measurable benefits for RAG systems and offering practical guidelines for their application.

Abstract: Graph retrieval-augmented generation (GraphRAG) has emerged as a powerful paradigm for enhancing large language models (LLMs) with external knowledge. It leverages graphs to model the hierarchical structure between specific concepts, enabling more coherent and effective knowledge retrieval for accurate reasoning.Despite its conceptual promise, recent studies report that GraphRAG frequently underperforms vanilla RAG on many real-world tasks. This raises a critical question: Is GraphRAG really effective, and in which scenarios do graph structures provide measurable benefits for RAG systems? To address this, we propose GraphRAG-Bench, a comprehensive benchmark designed to evaluate GraphRAG models onboth hierarchical knowledge retrieval and deep contextual reasoning. GraphRAG-Bench features a comprehensive dataset with tasks of increasing difficulty, coveringfact retrieval, complex reasoning, contextual summarization, and creative generation, and a systematic evaluation across the entire pipeline, from graph constructionand knowledge retrieval to final generation. Leveraging this novel benchmark, we systematically investigate the conditions when GraphRAG surpasses traditional RAG and the underlying reasons for its success, offering guidelines for its practical application. All related resources and analyses are collected for the community at https://github.com/GraphRAG-Bench/GraphRAG-Benchmark.

Method: Train LLMs with Group-Relative Policy Optimization (GRPO) on translated math datasets, track both task accuracy and language consistency of reasoning chains, and analyze factors like English-centric priors, long-CoT optimization, task difficulty, and decoding strategies.

Result: Three key findings: (1) CoT systematically drifts toward English as reasoning performance improves; (2) English priors, long-CoT optimization, task difficulty, and high-entropy decoding amplify this drift; (3) Interventions (language-consistency reward, decoding controls, balanced backbones) mitigate collapse but reveal persistent performance-fidelity trade-off.

Conclusion: There’s a fundamental trade-off between maximizing reasoning depth through long chains of thought and preserving target-language fidelity in multilingual LLMs, with English-centric priors strongly influencing reasoning language during RLVR training.

Abstract: Reinforcement learning with verifiable reward (RLVR) has been instrumental in eliciting strong reasoning capabilities from large language models (LLMs) via long chains of thought (CoT). During RLVR training, we formalize and systemically study an empirical phenomenon whereby a multilingual model’s CoT reverts to its dominant pre-training language (e.g., English) even when prompted in another language, which we term Cross-lingual Collapse. Because the long-CoT regime magnifies exposure to linguistic priors, the underlying trade-off between maximizing reasoning depth and preserving target-language fidelity has remained under-characterized. To examine this trade-off, we train LLMs with Group-Relative Policy Optimization (GRPO) on translated versions of math datasets widely used to elicit long-CoT reasoning. Throughout training, we track both task accuracy and the language consistency of reasoning chains. Our experiments yield three findings: (i) under RLVR, CoT in LLMs systematically drifts toward the pre-training dominant language as reasoning performance rises; (ii) English-centric priors, long-CoT GRPO optimization, task difficulty, and high-entropy decoding jointly amplify this drift, and the pattern persists beyond mathematics; and (iii) interventions that favor target-language traces–via a language-consistency reward, decoding-time controls, or more balanced backbones–mitigate collapse but reveal a persistent performance-fidelity trade-off.

Method: Proposes AbstRaL (Abstract Reasoning in LLMs using RL) - uses reinforcement learning rather than supervised fine-tuning to teach models to abstract reasoning problems. The method trains on granular abstraction data to promote abstract thinking capabilities.

Result: Significantly mitigates performance degradation on GSM perturbation benchmarks. Also shows implicit benefits on out-of-distribution mathematical and general reasoning tasks, indicating that abstract thinking broadly enables better generalizability.

Conclusion: Teaching abstract reasoning through reinforcement learning improves LLM robustness in mathematical reasoning and enhances generalizability across tasks, suggesting abstract thinking is a key capability for robust AI systems.

Abstract: Recent studies have shown that large language models (LLMs), especially smaller ones, often lack robustness in grade school math (GSM) reasoning. In particular, they tend to experience performance drops when faced with distribution shifts, such as changes to numerical or nominal variables, or insertions of distracting clauses. A possible strategy to address this involves generating synthetic data to further “instantiate” reasoning problems on potential variations. In this work, we instead focus on the strategy of “abstracting” reasoning problems. This not only helps counteract distribution shifts but also facilitates the connection to symbolic tools for deriving solutions. Focusing on GSM, we find that this abstraction process is better acquired through reinforcement learning (RL) than just supervised fine-tuning, which often fails to produce faithful abstractions. Our method, AbstRaL – which promotes abstract reasoning in LLMs using RL on granular abstraction data – significantly mitigates performance degradation on recent GSM perturbation benchmarks. Besides, improving GSM robustness via AbstRaL is shown to also implicitly benefit LLMs’ capabilities on OOD mathematical and general reasoning tasks, indicating that abstract thinking broadly enables better generalizability.

Method: Proposes Taxonomy-Guided Preference Data Generation (TaP) framework that uses structured taxonomy for fine-grained control over dataset composition to ensure diversity and broad coverage across languages.

Result: LLMs trained on TaP-generated datasets outperform those trained on existing open-source datasets, even outperforming models trained on datasets 180× larger.

Conclusion: TaP provides an effective automated approach for scalable preference dataset construction across languages, enabling better LLM fine-tuning with controlled diversity.

Abstract: Conducting supervised and preference fine-tuning of large language models (LLMs) requires high-quality datasets to improve their ability to follow instructions and align with human preferences and values. However, constructing such datasets is resource-intensive, and most publicly available datasets are in English. To address these challenges, we propose the \underline{\textbf{Ta}}xonomy-Guided \underline{\textbf{P}}reference Data Generation (TaP) framework for automated, scalable preference dataset construction across languages. TaP uses a structured taxonomy to provide fine-grained control over dataset composition, ensuring diversity and broad coverage. We use TaP-generated datasets to perform supervised and preference fine-tuning on multiple LLMs. Experimental results demonstrate that LLMs trained on TaP-generated datasets outperform those trained on existing open-source datasets. Remarkably, LLMs trained on TaP-generated datasets outperform models trained on an open-source dataset that is 180$\times$ larger.

Method: Proposed InfoRidge framework uses information theory to characterize predictive information across depth during training. Conducted experiments across various models and datasets, with complementary analyses using residual scaling, attention patterns, and controlled model capacity to characterize layer-wise functional specialization. Validated findings with multiple-token generation experiments.

Result: Revealed consistent non-monotonic trend: predictive information peaks in intermediate layers (forming a generalization ridge) before declining in final layers, reflecting transition between generalization and memorization. The ridge phenomenon persists across decoding steps in generation tasks.

Conclusion: Findings offer new insights into transformer internal mechanisms and underscore critical role of intermediate layers in supporting generalization. The generalization ridge phenomenon is robust across models and tasks.

Abstract: Transformer-based language models have achieved state-of-the-art performance in natural language generation (NLG), yet their internal mechanisms for synthesizing task-relevant information remain insufficiently understood. While prior studies suggest that intermediate layers often yield more generalizable representations than final layers, how this generalization ability emerges and propagates across layers during training remains unclear. To address this gap, we propose InfoRidge, an information-theoretic framework, to characterize how predictive information-the mutual information between hidden representations and target outputs-varies across depth during training. Our experiments across various models and datasets reveal a consistent non-monotonic trend: predictive information peaks in intermediate layers-forming a generalization ridge-before declining in final layers, reflecting a transition between generalization and memorization. To further investigate this phenomenon, we conduct a set of complementary analyses that leverage residual scaling, attention pattern, and controlled model capacity to characterize layer-wise functional specialization. We further validate our findings with multiple-token generation experiments, verifying that the observed ridge phenomenon persists across decoding steps. Together, these findings offer new insights into the internal mechanisms of transformers and underscore the critical role of intermediate layers in supporting generalization.

Method: Introduces SAE-LAPE (Sparse Autoencoder - Language-specific Feature Activation Probability Estimation), a method based on feature activation probability to identify language-specific features within the feed-forward network of LLMs. Uses sparse autoencoders to learn monosemantic features and analyzes their activation patterns across languages.

Result: Found many language-specific features predominantly appear in middle to final layers of the model and are interpretable. These features influence the model’s multilingual performance and language output. Can be used for language identification with performance comparable to fastText while offering more interpretability.

Conclusion: SAE-LAPE successfully identifies language-specific features in LLMs, providing insights into multilingual mechanisms. These features are interpretable, affect model performance, and enable practical applications like language identification with interpretability advantages over traditional methods.

Abstract: Understanding the multilingual mechanisms of large language models (LLMs) provides insight into how they process different languages, yet this remains challenging. Existing studies often focus on individual neurons, but their polysemantic nature makes it difficult to isolate language-specific units from cross-lingual representations. To address this, we explore sparse autoencoders (SAEs) for their ability to learn monosemantic features that represent concrete and abstract concepts across languages in LLMs. While some of these features are language-independent, the presence of language-specific features remains underexplored. In this work, we introduce SAE-LAPE, a method based on feature activation probability, to identify language-specific features within the feed-forward network. We find that many such features predominantly appear in the middle to final layers of the model and are interpretable. These features influence the model’s multilingual performance and language output and can be used for language identification with performance comparable to fastText along with more interpretability. Our code and complete figures are available at https://github.com/LyzanderAndrylie/language-specific-features

Method: Decomposes human behavior simulation into two stages: rationale generation guided by internal model signals (logit distributions), and action prediction using hierarchical reward structure with difficulty-aware scaling to prevent reward hacking.

Result: Achieves over 65% relative improvement compared to baseline methods in simulating human behavior in online shopping environments.

Conclusion: Shop-R1 demonstrates that RL with specialized reward structures can significantly enhance LLM reasoning for human behavior simulation, surpassing limitations of supervised fine-tuning approaches.

Abstract: Large Language Models (LLMs) have recently demonstrated strong potential in generating ‘believable human-like’ behavior in web environments. Prior work has explored augmenting training data with LLM-synthesized rationales and applying supervised fine-tuning (SFT) to enhance reasoning ability, which in turn can improve downstream action prediction. However, the performance of such approaches remains inherently bounded by the reasoning capabilities of the model used to generate the rationales. In this paper, we introduce Shop-R1, a novel reinforcement learning (RL) framework aimed at enhancing the reasoning ability of LLMs for simulation of real human behavior in online shopping environments. Specifically, Shop-R1 decomposes the human behavior simulation task into two stages: rationale generation and action prediction, each guided by distinct reward signals. For rationale generation, we leverage internal model signals (e.g., logit distributions) to guide the reasoning process in a self-supervised manner. For action prediction, we propose a hierarchical reward structure with difficulty-aware scaling to prevent reward hacking and enable fine-grained reward assignment. This design evaluates both high-level action types and the correctness of fine-grained sub-action details (attributes and values), rewarding outputs proportionally to their difficulty. Experimental results show that our method achieves a relative improvement of over 65% compared to the baseline. The project page is available at https://damon-demon.github.io/shop-r1.html.

Method: Three modeling strategies: 1) BERT-based classifier, 2) LLM-based classifier, and 3) role-conditioned generation. Two datasets constructed: first adapted from existing instruction-tuning corpora through clustering and role labeling, second synthetically generated for realistic enterprise scenarios. Evaluation includes organizational structure variations and robustness testing against prompt injection, role mismatch, and jailbreak attempts.

Result: The paper investigates whether LLMs can be fine-tuned to generate responses reflecting access privileges associated with different organizational roles, but specific quantitative results are not provided in the abstract.

Conclusion: Role-based access control is essential for enterprise LLM deployment, and the paper explores methods to achieve this through fine-tuning and specialized datasets, though full results would need to be examined in the complete paper.

Abstract: As large language models (LLMs) are increasingly deployed in enterprise settings, controlling model behavior based on user roles becomes an essential requirement. Existing safety methods typically assume uniform access and focus on preventing harmful or toxic outputs, without addressing role-specific access constraints. In this work, we investigate whether LLMs can be fine-tuned to generate responses that reflect the access privileges associated with different organizational roles. We explore three modeling strategies: a BERT-based classifier, an LLM-based classifier, and role-conditioned generation. To evaluate these approaches, we construct two complementary datasets. The first is adapted from existing instruction-tuning corpora through clustering and role labeling, while the second is synthetically generated to reflect realistic, role-sensitive enterprise scenarios. We assess model performance across varying organizational structures and analyze robustness to prompt injection, role mismatch, and jailbreak attempts.

Method: Developed CORE metric integrating cluster entropy, lexical repetition, and semantic similarity; applied to pairwise LLM dialogs across competitive, cooperative, and neutral settings; grounded analysis in Zipf’s and Heaps’ Laws for word frequency distributions and vocabulary growth.

Result: Cooperative settings show steeper Zipf distributions and higher Heap exponents (more repetition with greater vocabulary expansion), while competitive interactions display lower exponents (less repetition with constrained vocabularies).

Conclusion: CORE provides robust diagnostic for measuring linguistic robustness in multi-agent LLM systems, revealing how social incentives influence language adaptation.

Abstract: Game-theoretic interactions between agents with Large Language Models (LLMs) have revealed many emergent capabilities, yet the linguistic diversity of these interactions has not been sufficiently quantified. In this paper, we present the Conversational Robustness Evaluation Score: CORE, a metric to quantify the effectiveness of language use within multi-agent systems across different game-theoretic interactions. CORE integrates measures of cluster entropy, lexical repetition, and semantic similarity, providing a direct lens of dialog quality. We apply CORE to pairwise LLM dialogs across competitive, cooperative, and neutral settings, further grounding our analysis in Zipf’s and Heaps’ Laws to characterize word frequency distributions and vocabulary growth. Our findings show that cooperative settings exhibit both steeper Zipf distributions and higher Heap exponents, indicating more repetition alongside greater vocabulary expansion. In contrast, competitive interactions display lower Zipf and Heaps exponents, reflecting less repetition and more constrained vocabularies. These results provide new insights into how social incentives influence language adaptation, and highlight CORE as a robust diagnostic for measuring linguistic robustness in multi-agent LLM systems. Our code is available at https://github.com/psyonp/core.

Method: Created HebID corpus with 5,536 Hebrew sentences from Israeli politicians’ Facebook posts (Dec 2018-Apr 2021), manually annotated for 12 social identities. Benchmarked multilabel and single-label encoders alongside Hebrew-tuned LLMs (2B-9B parameters). Applied classifier to analyze politicians’ posts and parliamentary speeches.

Result: Hebrew-tuned LLMs achieved best performance (macro-F1 = 0.74). Analysis revealed differences in popularity, temporal trends, clustering patterns, and gender-related variations in identity expression. Enabled comparison between elite discourse identities and public identity priorities from national survey.

Conclusion: HebID provides comprehensive foundation for studying social identities in Hebrew political discourse and serves as model for similar research in other non-English contexts, bridging elite and public identity perspectives.

Abstract: Political language is deeply intertwined with social identities. While social identities are often shaped by specific cultural contexts and expressed through particular uses of language, existing datasets for group and identity detection are predominantly English-centric, single-label and focus on coarse identity categories. We introduce HebID, the first multilabel Hebrew corpus for social identity detection: 5,536 sentences from Israeli politicians’ Facebook posts (Dec 2018-Apr 2021), manually annotated for twelve nuanced social identities (e.g. Rightist, Ultra-Orthodox, Socially-oriented) grounded by survey data. We benchmark multilabel and single-label encoders alongside 2B-9B-parameter generative LLMs, finding that Hebrew-tuned LLMs provide the best results (macro-$F_1$ = 0.74). We apply our classifier to politicians’ Facebook posts and parliamentary speeches, evaluating differences in popularity, temporal trends, clustering patterns, and gender-related variations in identity expression. We utilize identity choices from a national public survey, enabling a comparison between identities portrayed in elite discourse and the public’s identity priorities. HebID provides a comprehensive foundation for studying social identities in Hebrew and can serve as a model for similar research in other non-English political contexts.

Method: Introduces ProPerSim, a simulation framework where a user agent with rich persona interacts with an assistant, providing ratings on suggestions. ProPerAssistant is proposed as a retrieval-augmented, preference-aligned assistant that continually learns and adapts through user feedback.

Result: Experiments across 32 diverse personas show that ProPerAssistant adapts its strategy and steadily improves user satisfaction over time.

Conclusion: The framework successfully demonstrates the promise of combining proactivity and personalization in AI assistants, enabling them to learn and adapt to user preferences through feedback.

Abstract: As large language models (LLMs) become increasingly integrated into daily life, there is growing demand for AI assistants that are not only reactive but also proactive and personalized. While recent advances have pushed forward proactivity and personalization individually, their combination remains underexplored. To bridge this gap, we introduce ProPerSim, a new task and simulation framework for developing assistants capable of making timely, personalized recommendations in realistic home scenarios. In our simulation environment, a user agent with a rich persona interacts with the assistant, providing ratings on how well each suggestion aligns with its preferences and context. The assistant’s goal is to use these ratings to learn and adapt to achieve higher scores over time. Built on ProPerSim, we propose ProPerAssistant, a retrieval-augmented, preference-aligned assistant that continually learns and adapts through user feedback. Experiments across 32 diverse personas show that ProPerAssistant adapts its strategy and steadily improves user satisfaction, highlighting the promise of uniting proactivity and personalization.

Method: ReMemR1 integrates memory retrieval into the memory update process, enabling selective callback of historical memories for non-linear reasoning. It also uses a multi-level reward design combining final-answer rewards with dense, step-level signals to guide effective memory use.

Result: Extensive experiments show ReMemR1 significantly outperforms state-of-the-art baselines on long-context question answering while incurring negligible computational overhead, validating its ability to trade marginal cost for robust long-context reasoning.

Conclusion: ReMemR1 effectively addresses information degradation, improves supervision, and supports complex multi-hop reasoning in long-context QA, offering a practical solution for LLMs to handle evidence dispersed across millions of tokens.

Abstract: Large language models face challenges in long-context question answering, where key evidence of a query may be dispersed across millions of tokens. Existing works equip large language models with a memory buffer that is dynamically updated via a linear document scan, also known as the “memorize while reading” methods. While this approach scales efficiently, it suffers from pruning of latent evidence, information loss through overwriting, and sparse reinforcement learning signals. To tackle these challenges, we present ReMemR1, which integrates the mechanism of memory retrieval into the memory update process, enabling the agent to selectively callback historical memories for non-linear reasoning. To further strengthen training, we propose a multi-level reward design, which combines final-answer rewards with dense, step-level signals that guide effective memory use. Together, these contributions mitigate information degradation, improve supervision, and support complex multi-hop reasoning. Extensive experiments demonstrate that ReMemR1 significantly outperforms state-of-the-art baselines on long-context question answering while incurring negligible computational overhead, validating its ability to trade marginal cost for robust long-context reasoning.

Method: Pretrains language models to first generate intermediate latent thoughts (last hidden state of current position) which then serve as input to predict the actual subsequent token, allowing refinement in continuous space before token prediction.

Result: PonderLM-2-Pythia-1.4B outperforms vanilla Pythia-2.8B on language modeling and downstream tasks despite having half the parameters; performance improves consistently with more latent thoughts per token.

Conclusion: Scaling computational steps during pretraining via latent thought generation significantly improves language model performance without increasing inference cost, offering an effective alternative to parameter scaling.

Abstract: The remarkable success of Chain-of-Thought (CoT), which enhances performance by scaling generation steps at test-time, inspires us to ask: can we leverage a similar scaling of computational steps during pretraining to improve the generation of each individual token? To address this, we propose a novel pre-training methodology: Pretraining Language Models with Latent Thoughts (PonderLM-2). Our approach pretrains a language model (LM) to first generate an intermediate latent thought-the last hidden state of the current position-which is then used as input to predict the actual subsequent token. This additional computational step enables the LM to refine its prediction within unconstrained continuous space. Our experiments demonstrate that, at an identical inference cost, a LM that generates one additional latent thought per token outperforms a standard model with double the parameters. For instance, our PonderLM-2-Pythia-1.4B, pretrained on 300B tokens from the Pile, significantly surpasses the vanilla Pythia-2.8B trained on the same data on both language modeling and a range of general downstream tasks. Furthermore, increasing the number of latent thoughts generated before each actual token-forming a chain analogous to CoT-consistently improves the model’s performance. The code is available at https://github.com/LUMIA-Group/PonderLM-2.

Method: Created SocialHarmBench dataset with 585 prompts spanning 7 sociopolitical categories (historical revisionism, propaganda, etc.) across 34 countries. Evaluated LLMs on their vulnerability to harmful compliance in politically charged contexts.

Result: Open-weight models show high vulnerability, with Mistral-7B reaching 97-98% attack success rates in domains like historical revisionism and propaganda. LLMs are most fragile with 21st-century/pre-20th-century contexts and prompts from Latin America, USA, and UK regions.

Conclusion: Current LLM safeguards fail to generalize to sociopolitical settings, exposing systematic biases and raising concerns about reliability in preserving human rights and democratic values. The benchmark highlights critical safety gaps.

Abstract: Large language models (LLMs) are increasingly deployed in contexts where their failures can have direct sociopolitical consequences. Yet, existing safety benchmarks rarely test vulnerabilities in domains such as political manipulation, propaganda and disinformation generation, or surveillance and information control. We introduce SocialHarmBench, a dataset of 585 prompts spanning 7 sociopolitical categories and 34 countries, designed to surface where LLMs most acutely fail in politically charged contexts. Our evaluations reveal several shortcomings: open-weight models exhibit high vulnerability to harmful compliance, with Mistral-7B reaching attack success rates as high as 97% to 98% in domains such as historical revisionism, propaganda, and political manipulation. Moreover, temporal and geographic analyses show that LLMs are most fragile when confronted with 21st-century or pre-20th-century contexts, and when responding to prompts tied to regions such as Latin America, the USA, and the UK. These findings demonstrate that current safeguards fail to generalize to high-stakes sociopolitical settings, exposing systematic biases and raising concerns about the reliability of LLMs in preserving human rights and democratic values. We share the SocialHarmBench benchmark at https://huggingface.co/datasets/psyonp/SocialHarmBench.

Method: Created EconCausal benchmark with 10,490 context-annotated causal triplets from 2,595 empirical studies using a four-stage pipeline with multi-run consensus, context refinement, and multi-critic filtering to ensure claims are grounded in peer-reviewed research.

Result: LLMs show critical limitations: 88% accuracy in fixed contexts but 32.6 percentage point drop under context shifts, 37% accuracy with misinformation, 9.5% accuracy recognizing null effects, and severe over-commitment in ambiguous cases.

Conclusion: Current LLMs have fundamental gaps between pattern matching and genuine causal reasoning, posing substantial risks for high-stakes economic decision-making where misinterpreting causality is costly.

Abstract: Socio-economic causal effects depend heavily on their specific institutional and environmental context. A single intervention can produce opposite results depending on regulatory or market factors, contexts that are often complex and only partially observed. This poses a significant challenge for large language models (LLMs) in decision-support roles: can they distinguish structural causal mechanisms from surface-level correlations when the context changes? To address this, we introduce EconCausal, a large-scale benchmark comprising 10,490 context-annotated causal triplets extracted from 2,595 high-quality empirical studies published in top-tier economics and finance journals. Through a rigorous four-stage pipeline combining multi-run consensus, context refinement, and multi-critic filtering, we ensure each claim is grounded in peer-reviewed research with explicit identification strategies. Our evaluation reveals critical limitations in current LLMs’ context-dependent reasoning. While top models achieve approximately 88 percent accuracy in fixed, explicit contexts, performance drops sharply under context shifts, with a 32.6 percentage point decline, and falls to 37 percent when misinformation is introduced. Furthermore, models exhibit severe over-commitment in ambiguous cases and struggle to recognize null effects, achieving only 9.5 percent accuracy, exposing a fundamental gap between pattern matching and genuine causal reasoning. These findings underscore substantial risks for high-stakes economic decision-making, where the cost of misinterpreting causality is high. The dataset and benchmark are publicly available at https://github.com/econaikaist/econcausal-benchmark.

Method: Circuit-based Reasoning Verification (CRV) analyzes attribution graphs of correct vs. incorrect CoT steps as execution traces of latent reasoning circuits. A classifier is trained on structural features of these graphs to detect reasoning errors, and the approach enables targeted interventions on individual transcoder features to correct faulty reasoning.

Result: The method shows that structural signatures of error are highly predictive, domain-specific, and not merely correlational. By using the analysis to guide targeted interventions, researchers successfully corrected the model’s faulty reasoning, demonstrating the viability of verifying reasoning directly via computational graphs.

Conclusion: White-box verification through computational graph analysis enables moving from simple error detection to causal understanding of LLM reasoning, providing novel scientific insights unattainable by black/gray-box methods.

Abstract: Current Chain-of-Thought (CoT) verification methods predict reasoning correctness based on outputs (black-box) or activations (gray-box), but offer limited insight into why a computation fails. We introduce a white-box method: Circuit-based Reasoning Verification (CRV). We hypothesize that attribution graphs of correct CoT steps, viewed as execution traces of the model’s latent reasoning circuits, possess distinct structural fingerprints from those of incorrect steps. By training a classifier on structural features of these graphs, we show that these traces contain a powerful signal of reasoning errors. Our white-box approach yields novel scientific insights unattainable by other methods. (1) We demonstrate that structural signatures of error are highly predictive, establishing the viability of verifying reasoning directly via its computational graph. (2) We find these signatures to be highly domain-specific, revealing that failures in different reasoning tasks manifest as distinct computational patterns. (3) We provide evidence that these signatures are not merely correlational; by using our analysis to guide targeted interventions on individual transcoder features, we successfully correct the model’s faulty reasoning. Our work shows that, by scrutinizing a model’s computational process, we can move from simple error detection to a deeper, causal understanding of LLM reasoning.

Method: Proposes MemoTime framework with: 1) Hierarchical Tree of Time decomposition of temporal questions, 2) Operator-aware reasoning with monotonic timestamp enforcement and entity co-constraining, 3) Dynamic evidence retrieval with operator-specific strategies, 4) Self-evolving experience memory storing verified reasoning traces and embeddings for cross-type reuse.

Result: Achieves state-of-the-art results on multiple temporal QA benchmarks, outperforming strong baselines by up to 24.0%. Enables smaller models (Qwen3-4B) to achieve reasoning performance comparable to GPT-4-Turbo.

Conclusion: MemoTime effectively addresses temporal reasoning challenges in LLMs through structured grounding and experience learning, demonstrating significant performance improvements and enabling efficient reasoning with smaller models.

Abstract: Large Language Models (LLMs) have achieved impressive reasoning abilities, but struggle with temporal understanding, especially when questions involve multiple entities, compound operators, and evolving event sequences. Temporal Knowledge Graphs (TKGs), which capture vast amounts of temporal facts in a structured format, offer a reliable source for temporal reasoning. However, existing TKG-based LLM reasoning methods still struggle with four major challenges: maintaining temporal faithfulness in multi-hop reasoning, achieving multi-entity temporal synchronization, adapting retrieval to diverse temporal operators, and reusing prior reasoning experience for stability and efficiency. To address these issues, we propose MemoTime, a memory-augmented temporal knowledge graph framework that enhances LLM reasoning through structured grounding, recursive reasoning, and continual experience learning. MemoTime decomposes complex temporal questions into a hierarchical Tree of Time, enabling operator-aware reasoning that enforces monotonic timestamps and co-constrains multiple entities under unified temporal bounds. A dynamic evidence retrieval layer adaptively selects operator-specific retrieval strategies, while a self-evolving experience memory stores verified reasoning traces, toolkit decisions, and sub-question embeddings for cross-type reuse. Comprehensive experiments on multiple temporal QA benchmarks show that MemoTime achieves overall state-of-the-art results, outperforming the strong baseline by up to 24.0%. Furthermore, MemoTime enables smaller models (e.g., Qwen3-4B) to achieve reasoning performance comparable to that of GPT-4-Turbo.

Method: Uses token attribution methods (GlobEnc and DecompX) to assign salience scores to tokens, ranks them, and retains only the top-k% most significant tokens to create sparse frugalized prompts.

Result: Experimental results across four NLP tasks show trade-offs between token retention and performance, revealing asymmetric patterns suggesting potential task contamination effects.

Conclusion: The work contributes to understanding LLM behavior in performance-efficiency trade-offs and delineates boundaries between tasks tolerant of contextual sparsity versus those requiring exhaustive context.

Abstract: Human communication heavily relies on laconism and inferential pragmatics, allowing listeners to successfully reconstruct rich meaning from sparse, telegraphic speech. In contrast, large language models (LLMs) owe much of their stellar performance to expansive input contexts, yet such verbosity inflates monetary costs, carbon footprint, and inference-time latency. This overhead manifests from the redundant low-utility tokens present in typical prompts, as only a fraction of tokens typically carries the majority of the semantic weight. Inspired by the aforementioned cognitive psycholinguistic processes, we address this inefficiency by introducing FrugalPrompt, a novel prompt compression framework for LLMs, which retains only the most semantically significant tokens. Leveraging two state-of-the-art token attribution methods, GlobEnc and DecompX, we assign salience scores to every token in an input sequence, rank them to retain the top-k% tokens, and obtain a sparse frugalized prompt. We establish the theoretical stability of our approach and provide strong empirical results across a suite of four NLP tasks to study the trade-off between the portion of retained tokens and performance. Experimental findings across retention settings reveal asymmetric performance patterns that suggest potential task contamination effects. We posit that our work contributes to a more nuanced understanding of LLM behavior in performance-efficiency trade-offs and delineates the boundary between tasks tolerant of contextual sparsity and those requiring exhaustive context.

Method: TIR-Judge uses reinforcement learning to train LLM judges with three key principles: 1) diverse training across verifiable and non-verifiable domains, 2) flexible judgment formats (pointwise, pairwise, listwise), and 3) iterative RL that bootstraps directly from initial models without distillation.

Result: On seven public benchmarks, TIR-Judge surpasses strong reasoning-based judges by up to 6.4% (pointwise) and 7.7% (pairwise), and achieves listwise performance comparable to Claude-Opus-4 despite having only 8B parameters. The zero-shot variant matches distilled variants’ performance.

Conclusion: Tool-integrated reasoning enables LLM judges to self-evolve through iterative reinforcement learning, achieving state-of-the-art performance without requiring distilled judge trajectories.

Abstract: Large Language Models (LLMs) are widely used as judges to evaluate response quality, providing a scalable alternative to human evaluation. However, most LLM judges operate solely on intrinsic text-based reasoning, limiting their ability to verify complex constraints or perform accurate computation. Motivated by the success of tool-integrated reasoning (TIR) in numerous tasks, we propose TIR-Judge, an end-to-end RL framework for training LLM judges that integrates a code executor for precise evaluation. TIR-Judge is built on three principles: (i) diverse training across verifiable and non-verifiable domains, (ii) flexible judgment formats (pointwise, pairwise, listwise), and (iii) iterative RL that bootstraps directly from the initial model without distillation. On seven public benchmarks, TIR-Judge surpasses strong reasoning-based judges by up to 6.4% (pointwise) and 7.7% (pairwise), and achieves listwise performance comparable to Claude-Opus-4 despite having only 8B parameters. Remarkably, TIR-Judge-Zero - trained entirely without distilled judge trajectories, matches the performance of distilled variants, demonstrating that tool-augmented judges can self-evolve through iterative reinforcement learning.

Method: Designed evaluation tasks for contextual understanding, pragmatic use, and connotation interpretation; evaluated 22 open/closed-source LLMs on Egyptian Arabic idioms, multidialectal Arabic proverbs, and English proverbs; created Kinayat dataset.

Result: Consistent hierarchy: Arabic proverbs 4.29% lower accuracy than English; Egyptian idioms 10.28% lower than Arabic proverbs; pragmatic use accuracy drops 14.07% relative to understanding; models struggle with connotative meaning (max 85.58% agreement with humans).

Conclusion: Figurative language serves as effective diagnostic for cultural reasoning - LLMs can interpret figurative meaning but face challenges in appropriate pragmatic use; Kinayat dataset released for future research.

Abstract: We present a comprehensive evaluation of the ability of large language models (LLMs) to process culturally grounded language, specifically to understand and pragmatically use figurative expressions that encode local knowledge and cultural nuance. Using figurative language as a proxy for cultural nuance and local knowledge, we design evaluation tasks for contextual understanding, pragmatic use, and connotation interpretation in Arabic and English. We evaluate 22 open- and closed-source LLMs on Egyptian Arabic idioms, multidialectal Arabic proverbs, and English proverbs. Our results show a consistent hierarchy: the average accuracy for Arabic proverbs is 4.29% lower than for English proverbs, and performance for Egyptian idioms is 10.28% lower than for Arabic proverbs. For the pragmatic use task, accuracy drops by 14.07% relative to understanding, though providing contextual idiomatic sentences improves accuracy by 10.66%. Models also struggle with connotative meaning, reaching at most 85.58% agreement with human annotators on idioms with 100% inter-annotator agreement. These findings demonstrate that figurative language serves as an effective diagnostic for cultural reasoning: while LLMs can often interpret figurative meaning, they face challenges in using it appropriately. To support future research, we release Kinayat, the first dataset of Egyptian Arabic idioms designed for both figurative understanding and pragmatic use evaluation.

Method: Developed a systematic Grammar Book Guided evaluation pipeline with four key stages, using Luxembourgish as a case study to assess LLMs’ grammatical understanding across different dimensions.

Result: Found weak positive correlation between translation performance and grammatical understanding; larger models perform well overall due to semantic strength but struggle with morphology, syntax, and Minimal Pair tasks.

Conclusion: Strong translation performance doesn’t imply deep grammatical competence; reasoning ability shows promise for enhancing grammatical understanding in LLMs.

Abstract: Grammar refers to the system of rules that governs the structural organization and the semantic relations among linguistic units such as sentences, phrases, and words within a given language. In natural language processing, there remains a notable scarcity of grammar focused evaluation protocols, a gap that is even more pronounced for low-resource languages. Moreover, the extent to which large language models genuinely comprehend grammatical structure, especially the mapping between syntactic structures and meanings, remains under debate. To investigate this issue, we propose a Grammar Book Guided evaluation pipeline intended to provide a systematic and generalizable framework for grammar evaluation consisting of four key stages, and in this work we take Luxembourgish as a case study. The results show a weak positive correlation between translation performance and grammatical understanding, indicating that strong translations do not necessarily imply deep grammatical competence. Larger models perform well overall due to their semantic strength but remain weak in morphology and syntax, struggling particularly with Minimal Pair tasks, while strong reasoning ability offers a promising way to enhance their grammatical understanding.

Method: 1) Created BEAM benchmark with automatically generated long conversations (up to 10M tokens) and probing questions; 2) Proposed LIGHT framework with three memory systems: long-term episodic memory, short-term working memory, and scratchpad for accumulating facts.

Result: LLMs with 1M token context windows (with/without retrieval-augmentation) struggle as dialogues lengthen. LIGHT consistently improves performance across models, achieving 3.5%-12.69% average improvement over strongest baselines. Ablation study confirms each memory component’s contribution.

Conclusion: BEAM addresses limitations of existing benchmarks for long-context memory evaluation, while LIGHT’s multi-memory system approach effectively enhances LLM performance on long conversational tasks.

Abstract: Evaluating the abilities of large language models (LLMs) for tasks that require long-term memory and thus long-context reasoning, for example in conversational settings, is hampered by the existing benchmarks, which often lack narrative coherence, cover narrow domains, and only test simple recall-oriented tasks. This paper introduces a comprehensive solution to these challenges. First, we present a novel framework for automatically generating long (up to 10M tokens), coherent, and topically diverse conversations, accompanied by probing questions targeting a wide range of memory abilities. From this, we construct BEAM, a new benchmark comprising 100 conversations and 2,000 validated questions. Second, to enhance model performance, we propose LIGHT-a framework inspired by human cognition that equips LLMs with three complementary memory systems: a long-term episodic memory, a short-term working memory, and a scratchpad for accumulating salient facts. Our experiments on BEAM reveal that even LLMs with 1M token context windows (with and without retrieval-augmentation) struggle as dialogues lengthen. In contrast, LIGHT consistently improves performance across various models, achieving an average improvement of 3.5%-12.69% over the strongest baselines, depending on the backbone LLM. An ablation study further confirms the contribution of each memory component.

Method: Analyze-Revise-Finetune (ARF) pipeline: 1) Analyze and categorize common errors in summaries from teacher model (GPT-3.5), 2) Perform targeted revision using compact editor model (Llama 3.1 70B) to generate refined training data, 3) Fine-tune smaller student models (Llama 3.1 8B, QWen3 4B) on refined data.

Result: Fine-tuned smaller student models achieved superior summarization performance compared to GPT-3.5, demonstrating improved cost efficiency and data privacy while maintaining competitive accuracy.

Conclusion: The ARF pipeline provides a generalizable framework for enhancing open-source LLMs across diverse downstream applications, enabling smaller models to outperform larger proprietary ones in specific tasks.

Abstract: We introduce an Analyze-Revise-Finetune (ARF) pipeline that enables smaller open-source language models (LLMs) to surpass substantially larger proprietary models in customer service summarization tasks. The pipeline first analyzes and categorizes common errors in summaries produced by a teacher model (GPT-3.5), then performs a targeted revision using a compact editor model (Llama 3.1 70B) to generate high-quality, refined training data. Fine-tuning smaller student models (e.g., Llama 3.1 8B, QWen3 4B) on this refined data resulted in superior summarization performance compared to GPT-3.5. The ARF pipeline improves cost efficiency and data privacy while maintaining competitive accuracy, illustrating a generalizable framework for enhancing open-source LLMs across diverse downstream applications.

Method: Introduces PEFT-Bench, an end-to-end benchmark for evaluating diverse PEFT methods on autoregressive LLMs. Covers 27 NLP datasets and 7 PEFT methods. Also introduces PEFT Soft Cost Penalties (PSCP) metric that considers trainable parameters, inference speed, and training memory usage.

Result: The paper presents a comprehensive benchmarking framework that enables systematic evaluation of PEFT methods across multiple dimensions, including performance and computational efficiency metrics.

Conclusion: PEFT-Bench provides a unified, reproducible evaluation framework for PEFT methods that addresses current limitations in benchmarking and introduces a novel cost-aware metric for practical deployment considerations.

Abstract: Despite the state-of-the-art performance of Large Language Models (LLMs) achieved on many tasks, their massive scale often leads to high computational and environmental costs, limiting their accessibility. Parameter-Efficient Fine-Tuning (PEFT) methods address this challenge by reducing the number of trainable parameters while maintaining strong downstream performance. Despite the advances in PEFT methods, current evaluations remain limited (in terms of evaluated models and datasets) and difficult to reproduce. To bridge this gap, we introduce PEFT-Bench, a unified end-to-end benchmark for evaluating diverse PEFT methods on autoregressive LLMs. We demonstrate its usage across 27 NLP datasets and 7 PEFT methods. To account for different PEFT training and inference factors, we also introduce the PEFT Soft Cost Penalties (PSCP) metric, which takes trainable parameters, inference speed, and training memory usage into account.

Method: Developed a modular framework based on LLaMA-Factory that provides a unified interface for 19 PEFT methods, supports 27 classification/text generation datasets across 12 tasks, and includes both standard and PEFT-specific evaluation metrics.

Result: Created a ready-to-use, controlled environment that improves replicability and benchmarking of PEFT methods, making it easier to compare different parameter-efficient fine-tuning approaches.

Conclusion: PEFT-Factory addresses the reproducibility crisis in PEFT research by providing a standardized framework for comparing and deploying parameter-efficient fine-tuning methods for LLMs.

Abstract: Parameter-Efficient Fine-Tuning (PEFT) methods address the increasing size of Large Language Models (LLMs). Currently, many newly introduced PEFT methods are challenging to replicate, deploy, or compare with one another. To address this, we introduce PEFT-Factory, a unified framework for efficient fine-tuning LLMs using both off-the-shelf and custom PEFT methods. While its modular design supports extensibility, it natively provides a representative set of 19 PEFT methods, 27 classification and text generation datasets addressing 12 tasks, and both standard and PEFT-specific evaluation metrics. As a result, PEFT-Factory provides a ready-to-use, controlled, and stable environment, improving replicability and benchmarking of PEFT methods. PEFT-Factory is a downstream framework that originates from the popular LLaMA-Factory, and is publicly available at https://github.com/kinit-sk/PEFT-Factory.

Method: An end-to-end multimodal system that jointly processes lecture audio and slides to produce synchronized translations across three modalities: translated text, localized slides with preserved visual elements, and synthesized speech.

Result: Experiments show that slide-aware transcripts provide cascading benefits for downstream tasks like summarization and question answering, demonstrating the value of multimodal processing.

Conclusion: BOOM enables students to access lectures in their native language while preserving the complete multimodal experience, addressing critical challenges in educational content localization.

Abstract: The globalization of education and rapid growth of online learning have made localizing educational content a critical challenge. Lecture materials are inherently multimodal, combining spoken audio with visual slides, which requires systems capable of processing multiple input modalities. To provide an accessible and complete learning experience, translations must preserve all modalities: text for reading, slides for visual understanding, and speech for auditory learning. We present \textbf{BOOM}, a multimodal multilingual lecture companion that jointly translates lecture audio and slides to produce synchronized outputs across three modalities: translated text, localized slides with preserved visual elements, and synthesized speech. This end-to-end approach enables students to access lectures in their native language while aiming to preserve the original content in its entirety. Our experiments demonstrate that slide-aware transcripts also yield cascading benefits for downstream tasks such as summarization and question answering. The demo video and code can be found at https://ai4lt.github.io/boom/ \footnote{All released code and models are licensed under the MIT License}.

Method: Developed a modular open-source framework that integrates multiple contemporary discrete prompt optimizers, supports systematic benchmarking, and returns framework-agnostic prompt strings.

Result: Created a unified system that enables seamless integration into existing LLM pipelines while remaining agnostic to underlying model implementation.

Conclusion: Promptolution addresses the gap between prompt optimization research and practical adoption by providing an extensible, modular framework for both practitioners and researchers.

Abstract: Prompt optimization has become crucial for enhancing the performance of large language models (LLMs) across a broad range of tasks. Although many research papers demonstrate its effectiveness, practical adoption is hindered because existing implementations are often tied to unmaintained, isolated research codebases or require invasive integration into application frameworks. To address this, we introduce promptolution, a unified, modular open-source framework that provides all components required for prompt optimization within a single extensible system for both practitioners and researchers. It integrates multiple contemporary discrete prompt optimizers, supports systematic and reproducible benchmarking, and returns framework-agnostic prompt strings, enabling seamless integration into existing LLM pipelines while remaining agnostic to the underlying model implementation.

Method: Develops an application using language technology that includes software for demonstration, evaluation, model inspection, and data visualization capabilities.

Result: A functional tool (AITutor-EvalKit) that enables assessment of AI tutor pedagogical quality and supports learning through user feedback collection.

Conclusion: The tool serves both education stakeholders and the ACL community by providing evaluation capabilities for AI tutors while facilitating learning and feedback collection.

Abstract: We present AITutor-EvalKit, an application that uses language technology to evaluate the pedagogical quality of AI tutors, provides software for demonstration and evaluation, as well as model inspection and data visualization. This tool is aimed at education stakeholders as well as *ACL community at large, as it supports learning and can also be used to collect user feedback and annotation.

Method: The library provides two complementary families of methods: attribution methods and concept-based explanations. It features an end-to-end concept-based pipeline that includes activation extraction, concept learning, interpretation, and scoring, going beyond feature-level attributions.

Result: Interpreto offers a unified API for both classification and text generation tasks, supporting interpretation of language models from early BERT variants to modern LLMs, with a focus on practical usability.

Conclusion: Interpreto fills a gap in existing interpretability tooling by providing comprehensive, research-backed methods for understanding language model behavior through both attribution and concept-based approaches.

Abstract: Interpreto is an open-source Python library for interpreting HuggingFace language models, from early BERT variants to LLMs. It provides two complementary families of methods: attribution methods and concept-based explanations. The library bridges recent research and practical tooling by exposing explanation workflows through a unified API for both classification and text generation. A key differentiator is its end-to-end concept-based pipeline (from activation extraction to concept learning, interpretation, and scoring), which goes beyond feature-level attributions and is uncommon in existing libraries.

Method: Targeted prompt engineering with parameter-efficient fine-tuning of Qwen2.5-7B model using Low-Rank Adaptation (LoRA) on manually annotated dataset of 4,933 instances from 27,822 police Weibo posts

Result: LoRA-based fine-tuning significantly outperformed base and instruction-tuned models, achieving >98.36% accuracy for mortality detection, 95.31% Exact Match Rate for fatality counts, and 95.54% for province-level location extraction

Conclusion: The pipeline provides validated efficient solution for multi-task structured information extraction in specialized domains, offering practical framework for transforming unstructured text into reliable structured data

Abstract: Structured information extraction from police incident announcements is crucial for timely and accurate data processing, yet presents considerable challenges due to the variability and informal nature of textual sources such as social media posts. To address these challenges, we developed a domain-adapted extraction pipeline that leverages targeted prompt engineering with parameter-efficient fine-tuning of the Qwen2.5-7B model using Low-Rank Adaptation (LoRA). This approach enables the model to handle noisy, heterogeneous text while reliably extracting 15 key fields, including location, event characteristics, and impact assessment, from a high-quality, manually annotated dataset of 4,933 instances derived from 27,822 police briefing posts on Chinese Weibo (2019-2020). Experimental results demonstrated that LoRA-based fine-tuning significantly improved performance over both the base and instruction-tuned models, achieving an accuracy exceeding 98.36% for mortality detection and Exact Match Rates of 95.31% for fatality counts and 95.54% for province-level location extraction. The proposed pipeline thus provides a validated and efficient solution for multi-task structured information extraction in specialized domains, offering a practical framework for transforming unstructured text into reliable structured data in social science research.

Method: Cross-model Reasoning Distillation Provenance Tracing framework that compares predictive probabilities from teacher, original student, and distilled model to classify actions by origin, plus teacher-guided data selection based on teacher-student divergences.

Result: Distilled models can generate teacher-originated actions in test-time contexts, which correlate with performance; teacher-guided data selection is effective across multiple teacher-student combinations.

Conclusion: The provenance-tracing framework provides insights into reasoning distillation and enables principled data selection, showing promise for improving distillation methods.

Abstract: Reasoning distillation has attracted increasing attention. It typically leverages a large teacher model to generate reasoning paths, which are then used to fine-tune a student model so that it mimics the teacher’s behavior in training contexts. However, previous approaches have lacked a detailed analysis of the origins of the distilled model’s capabilities. It remains unclear whether the student can maintain consistent behaviors with the teacher in novel test-time contexts, or whether it regresses to its original output patterns, raising concerns about the generalization of distillation models. To analyse this question, we introduce a cross-model Reasoning Distillation Provenance Tracing framework. For each action (e.g., a sentence) produced by the distilled model, we obtain the predictive probabilities assigned by the teacher, the original student, and the distilled model under the same context. By comparing these probabilities, we classify each action into different categories. By systematically disentangling the provenance of each action, we experimentally demonstrate that, in test-time contexts, the distilled model can indeed generate teacher-originated actions, which correlate with and plausibly explain observed performance on distilled model. Building on this analysis, we further propose a teacher-guided data selection method. Unlike prior approach that rely on heuristics, our method directly compares teacher-student divergences on the training data, providing a principled selection criterion. We validate the effectiveness of our approach across multiple representative teacher models and diverse student models. The results highlight the utility of our provenance-tracing framework and underscore its promise for reasoning distillation. We hope to share Reasoning Distillation Provenance Tracing and our insights into reasoning distillation with the community.

Method: Created CricBench benchmark suite with domain experts manually authoring complex queries for logical correctness. Built in both English and Hindi with open framework for other languages. Evaluated six state-of-the-art models using strict protocol.

Result: High performance on general benchmarks doesn’t guarantee success in specialized domains. DeepSeek R1 achieved SOTA (50.6%), surpassing Claude 3.7 Sonnet (47.7%) and GPT-4o (33.7%). Code-mixed Hindi queries frequently yielded parity or higher accuracy than English.

Conclusion: Specialized domains like cricket analytics present unique challenges for LLMs, and English may not be optimal prompt language for specialized SQL tasks. Open-weights reasoning models can outperform proprietary models in domain-specific tasks.

Abstract: Cricket is the second most popular sport globally, commanding a massive following of over 2.5 billion fans globally. Enthusiasts and analysts frequently seek advanced statistical insights, such as long-term historical performance trends or complex player comparisons, that are often unavailable through standard web searches. While Large Language Models (LLMs) have advanced significantly in Text-to-SQL tasks, their capability to handle the domain-specific nuances, complex schema variations, and multilingual requirements inherent to sports analytics remains under-explored. To investigate this potential capability gap, we present CricBench, a comprehensive benchmark suite for evaluating LLMs on specialized cricket data. To curate a “Gold Standard” dataset, we collaborate with domain experts in cricket and SQL to manually author complex queries, ensuring logical correctness. Recognizing linguistic diversity, we construct the benchmark in both English and Hindi, establishing a framework that is open for further extension to other regional languages. We evaluate six state-of-the-art models, including GPT-4o, Claude 3.7 Sonnet, and open-source models, using a strict evaluation protocol. Our results reveal that high performance on general benchmarks does not guarantee success in specialized domains. While the open-weights reasoning model DeepSeek R1 achieves state-of-the-art performance (50.6%), surpassing proprietary giants like Claude 3.7 Sonnet (47.7%) and GPT-4o (33.7%), it still exhibits a significant accuracy drop when moving from general benchmarks (BIRD) to CricBench. Furthermore, we observe that code-mixed Hindi queries frequently yield parity or higher accuracy compared to English, challenging the assumption that English is the optimal prompt language for specialized SQL tasks.

Method: Introduces Fast-weight Product Key Memory (FwPKM), a sparse fast-weight memory layer that performs chunk-level gradient descent on a local memory-rewrite objective, enabling test-time training updates on activated slots in sparse memory.

Result: Significant perplexity reductions on long-context datasets, generalization to 128K-token contexts despite training on only 4K-token sequences, and effective functioning as episodic memory complementing standard modules’ semantic memory.

Conclusion: FwPKM resolves the storage-computation trade-off in sequence modeling, providing efficient long-context processing through sparse fast-weight memory with test-time training capabilities.

Abstract: Sequence modeling layers in modern language models typically face a trade-off between storage capacity and computational efficiency. While softmax attention offers unbounded storage at prohibitive quadratic cost, linear variants are more efficient but suffer from limited, fixed-size storage. We introduce Fast-weight Product Key Memory (FwPKM), a sparse fast-weight memory layer that resolves this tension. FwPKM updates sparsely activated parameters at both training and inference time using chunk-level gradient descent on a local memory-rewrite objective. This performs Test-Time Training (TTT)-style gradient updates on activated slots in a sparse memory, enabling rapid memorization and retrieval of many new key-value associations while keeping per-token compute low and fixed. Experiments show that FwPKM functions as an effective episodic memory that complements the semantic memory of standard modules, yielding significant perplexity reductions on long-context datasets. Notably, in Needle-in-a-Haystack evaluations, FwPKM generalizes to 128K-token contexts despite being trained on only 4K-token sequences.

Method: STaRR introduces temporal variance and spatial deviance metrics to track token confidence evolution, enabling step-wise dynamic thresholding with responsiveness optimizations for scalability.

Result: STaRR achieves average 4.1x speedup (up to 8.9x) while maintaining comparable accuracy to baseline methods, demonstrating efficient parallel decoding.

Conclusion: Dynamic remasking based on token confidence evolution significantly improves DLM inference efficiency without compromising quality, offering a practical training-free solution.

Abstract: Diffusion Language Models (DLMs) enable parallel decoding via iterative denoising, where remasking strategies play a critical role in balancing inference speed and output quality. Existing methods predominantly rely on static confidence thresholds, overlooking the spatial-temporal dynamics of token confidence, causing unnecessary remasking. We propose Spatial-Temporal Token-Dynamics-Aware Responsive Remasking (STaRR), a training-free framework that dynamically adapts remasking decisions based on token confidence evolution. STaRR introduces two metrics, temporal variance and spatial deviance, to guide fine-grained, step-wise dynamic thresholding. We further introduce a step-wise dynamic thresholding strategy, further enhanced with responsiveness optimizations for scalability and robustness. Experiments show that STaRR achieves an average speedup of 4.1 and up to 8.9 while maintaining comparable accuracy.

Method: Uses a Teacher-Student architecture: Teacher network trained on articulatory phonetic features produces target embeddings, while Student network learns to approximate these embeddings directly from characters. Combines Epitran (with 100 new language-script mappings), Phonikud for Hebrew, and CharsiuG2P for CJK languages. Trained on 32.7 million triplet samples of toponyms spanning 20 writing systems.

Result: Achieves Recall@10 of 97.6% and MRR of 90.3% on MEHDIE Hebrew-Arabic benchmark, outperforming Levenshtein and Jaro-Winkler baselines. On 12,947 real cross-script pairs, 82.6% achieve >0.75 cosine similarity, with best performance on Arabic-Cyrillic (94-100%) and Cyrillic-Latin (94.3%) combinations.

Conclusion: Symphonym provides effective cross-script name matching with fixed-length embeddings enabling efficient retrieval in digital humanities workflows, demonstrating transfer learning from modern place names to historical orthographic variations.

Abstract: Linking names across historical sources, languages, and writing systems remains a fundamental challenge in digital humanities and geographic information retrieval. Existing approaches require language-specific phonetic algorithms or fail to capture phonetic relationships across different scripts. This paper presents Symphonym, a neural embedding system that maps names from any script into a unified 128-dimensional phonetic space, enabling direct similarity comparison without runtime phonetic conversion. Symphonym uses a Teacher-Student architecture where a Teacher network trained on articulatory phonetic features produces target embeddings, while a Student network learns to approximate these embeddings directly from characters. The Teacher combines Epitran (extended with 100 new language-script mappings), Phonikud for Hebrew, and CharsiuG2P for Chinese, Japanese, and Korean. Training used 32.7 million triplet samples of toponyms spanning 20 writing systems from GeoNames, Wikidata, and Getty Thesaurus of Geographic Names. On the MEHDIE Hebrew-Arabic historical toponym benchmark, Symphonym achieves Recall@10 of 97.6% and MRR of 90.3%, outperforming Levenshtein and Jaro-Winkler baselines (Recall@1: 86.7% vs 81.5% and 78.5%). Evaluation on 12,947 real cross-script training pairs shows 82.6% achieve greater than 0.75 cosine similarity, with best performance on Arabic-Cyrillic (94–100%) and Cyrillic-Latin (94.3%) combinations. The fixed-length embeddings enable efficient retrieval in digital humanities workflows, with a case study on medieval personal names demonstrating effective transfer from modern place names to historical orthographic variation.

Method: Created APEX-Agents benchmark with 480 tasks requiring agents to navigate work environments with files and tools. Tested eight agents using Pass@1 metric. Also developed Archipelago infrastructure for agent execution and evaluation.

Result: Gemini 3 Flash (Thinking=High) achieved highest score of 24.0%, followed by GPT-5.2, Claude Opus 4.5, and Gemini 3 Pro. All prompts, rubrics, gold outputs, files, and metadata are open-sourced.

Conclusion: APEX-Agents provides a comprehensive benchmark for evaluating AI agents on professional tasks, revealing current limitations in agent performance while providing open infrastructure for further research.

Abstract: We introduce the AI Productivity Index for Agents (APEX-Agents), a benchmark for assessing whether AI agents can execute long-horizon, cross-application tasks created by investment banking analysts, management consultants, and corporate lawyers. APEX-Agents requires agents to navigate realistic work environments with files and tools. We test eight agents for the leaderboard using Pass@1. Gemini 3 Flash (Thinking=High) achieves the highest score of 24.0%, followed by GPT-5.2 (Thinking=High), Claude Opus 4.5 (Thinking=High), and Gemini 3 Pro (Thinking=High). We open source the APEX-Agents benchmark (n=480) with all prompts, rubrics, gold outputs, files, and metadata. We also open source Archipelago, our infrastructure for agent execution and evaluation.

Method: Introduce a simple extra sink token via modified attention mask - a special token constrained to attend solely to itself while remaining globally visible to all other tokens. This creates a dedicated structural sink.

Result: Experimental results show that introducing a single extra token stabilizes attention sinks and substantially improves model performance. Further analysis confirms effectiveness is independent of position and token has negligible semantic content.

Conclusion: The extra sink token provides a robust and dedicated structural sink that resolves the moving sink instability in DLMs, improving inference robustness without adding semantic complexity.

Abstract: Diffusion Language Models (DLMs) have emerged as a compelling alternative to autoregressive approaches, enabling parallel text generation with competitive performance. Despite these advantages, there is a critical instability in DLMs: the moving sink phenomenon. Our analysis indicates that sink tokens exhibit low-norm representations in the Transformer’s value space, and that the moving sink phenomenon serves as a protective mechanism in DLMs to prevent excessive information mixing. However, their unpredictable positions across diffusion steps undermine inference robustness. To resolve this, we propose a simple but effective extra sink token implemented via a modified attention mask. Specifically, we introduce a special token constrained to attend solely to itself, while remaining globally visible to all other tokens. Experimental results demonstrate that introducing a single extra token stabilizes attention sinks, substantially improving model performance. Crucially, further analysis confirms that the effectiveness of this token is independent of its position and characterized by negligible semantic content, validating its role as a robust and dedicated structural sink.

Method: Proposes an iterative framework where for each problem, the agent runs multiple inference iterations with adaptive computation. Each iteration optionally produces a high-level plan, selects reasoning tools and compute strategy with exploration parameter, then generates candidate reasoning trajectories. A process reward model (PRM) guides pruning/expansion during generation (step-level) and selects final response across iterations (trajectory-level).

Result: Consistently outperforms direct test-time scaling across datasets, with large gains on MATH-500 and several-fold improvements on harder benchmarks like AIME24 and AMO-Bench. Demonstrates efficient computation allocation using theoretical FLOPs and compute intensity metrics.

Conclusion: Verification-guided allocation concentrates computation on high-utility reasoning paths, making inference more efficient and effective than uniform compute scaling approaches.

Abstract: Test-time compute scaling allocates inference computation uniformly, uses fixed sampling strategies, and applies verification only for reranking. In contrast, we propose a verifier-guided adaptive framework treating reasoning as iterative trajectory generation and selection. For each problem, the agent runs multiple inference iterations. In each iteration, it optionally produces a high-level plan, selects a set of reasoning tools and a compute strategy together with an exploration parameter, and then generates a candidate reasoning trajectory. A process reward model (PRM) serves as a unified control signal: within each iteration, step-level PRM scores are aggregated to guide pruning and expansion during generation, and across iterations, aggregated trajectory rewards are used to select the final response. Across datasets, our dynamic, PRM-guided approach consistently outperforms direct test-time scaling, yielding large gains on MATH-500 and several-fold improvements on harder benchmarks such as AIME24 and AMO-Bench. We characterize efficiency using theoretical FLOPs and a compute intensity metric penalizing wasted generation and tool overhead, demonstrating that verification-guided allocation concentrates computation on high-utility reasoning paths.

Method: AROA framework defines originality as rarity within a reference corpus, measured through four components: structural rarity, claim rarity, evidence rarity, and cognitive depth, quantified via density estimation with quality adjustment.

Result: Strong negative correlation between quality and claim rarity; AI essays achieve near-perfect quality but have 1/5 the claim rarity of human essays; low correlations between components confirm independent aspects of originality.

Conclusion: Writing assessment must shift from quality to originality in the AI era; AROA provides a framework for evaluating argumentative originality.

Abstract: This study proposes Argument Rarity-based Originality Assessment (AROA), a framework for automatically evaluating argumentative originality in student essays. AROA defines originality as rarity within a reference corpus and evaluates it through four complementary components: structural rarity, claim rarity, evidence rarity, and cognitive depth, quantified via density estimation and integrated with quality adjustment. Experiments using 1,375 human essays and 1,000 AI-generated essays on two argumentative topics revealed three key findings. First, a strong negative correlation (r = -0.67) between text quality and claim rarity demonstrates a quality-originality trade-off. Second, while AI essays achieved near-perfect quality scores (Q = 0.998), their claim rarity was approximately one-fifth of human levels (AI: 0.037, human: 0.170), indicating that LLMs can reproduce argumentative structure but not semantic originality. Third, the four components showed low mutual correlations (r = 0.06–0.13 between structural and semantic dimensions), confirming that they capture genuinely independent aspects of originality. These results suggest that writing assessment in the AI era must shift from quality to originality.

Method: Proposes OmniRAG-Agent with: 1) image-audio retrieval-augmented generation module for fetching relevant frames/audio snippets, 2) agent loop for planning, tool calling, and evidence merging, and 3) group relative policy optimization to jointly improve tool use and answer quality.

Result: Outperforms prior methods on OmniVideoBench, WorldSense, and Daily-Omni datasets under low-resource settings, with ablations validating each component’s contribution.

Conclusion: OmniRAG-Agent effectively addresses challenges in long audio-video QA through integrated retrieval, agentic planning, and policy optimization, demonstrating strong performance in resource-constrained scenarios.

Abstract: Long-horizon omnimodal question answering answers questions by reasoning over text, images, audio, and video. Despite recent progress on OmniLLMs, low-resource long audio-video QA still suffers from costly dense encoding, weak fine-grained retrieval, limited proactive planning, and no clear end-to-end optimization. To address these issues, we propose OmniRAG-Agent, an agentic omnimodal QA method for budgeted long audio-video reasoning. It builds an image-audio retrieval-augmented generation module that lets an OmniLLM fetch short, relevant frames and audio snippets from external banks. Moreover, it uses an agent loop that plans, calls tools across turns, and merges retrieved evidence to answer complex queries. Furthermore, we apply group relative policy optimization to jointly improve tool use and answer quality over time. Experiments on OmniVideoBench, WorldSense, and Daily-Omni show that OmniRAG-Agent consistently outperforms prior methods under low-resource settings and achieves strong results, with ablations validating each component.

Method: Proposes an optimal transport-based framework that aligns activations to infer cross-neuron correspondences between heterogeneous models. Uses transport plans to guide direct weight-space fusion, requiring only a small set of inputs for effective transfer.

Result: Extensive experiments across low-resource languages and specialized domains demonstrate consistent improvements over target models, showing effective knowledge transfer from high-resource to low-resource models.

Conclusion: The proposed cross-architecture merging framework enables effective knowledge transfer from large LLMs to heterogeneous smaller models, addressing the practical need for deploying capable models in low-resource settings.

Abstract: Large language models (LLMs) achieve strong capabilities by scaling model capacity and training data, yet many real-world deployments rely on smaller models trained or adapted from low-resource data. This gap motivates the need for mechanisms to transfer knowledge from large, high-resource models to smaller, low-resource targets. While model merging provides an effective transfer mechanism, most existing approaches assume architecture-compatible models and therefore cannot directly transfer knowledge from large high-resource LLMs to heterogeneous low-resource targets. In this work, we propose a cross-architecture merging framework based on optimal transport (OT) that aligns activations to infer cross-neuron correspondences between heterogeneous models. The resulting transport plans are then used to guide direct weight-space fusion, enabling effective high-resource to low-resource transfer using only a small set of inputs. Extensive experiments across low-resource languages and specialized domains demonstrate consistent improvements over target models.

Method: Uses sparse Mixture-of-Experts architecture (196B parameters with 11B active), interleaved 3:1 sliding-window/full attention, Multi-Token Prediction (MTP-3), and scalable reinforcement learning combining verifiable signals with preference feedback.

Result: Achieves 85.4% on IMO-AnswerBench, 86.4% on LiveCodeBench-v6, 88.2% on tau2-Bench, 69.0% on BrowseComp, and 51.0% on Terminal-Bench 2.0, comparable to GPT-5.2 xHigh and Gemini 3.0 Pro.

Conclusion: Step 3.5 Flash redefines the efficiency frontier, providing a high-density foundation for deploying sophisticated agents in real-world industrial environments.

Abstract: We introduce Step 3.5 Flash, a sparse Mixture-of-Experts (MoE) model that bridges frontier-level agentic intelligence and computational efficiency. We focus on what matters most when building agents: sharp reasoning and fast, reliable execution. Step 3.5 Flash pairs a 196B-parameter foundation with 11B active parameters for efficient inference. It is optimized with interleaved 3:1 sliding-window/full attention and Multi-Token Prediction (MTP-3) to reduce the latency and cost of multi-round agentic interactions. To reach frontier-level intelligence, we design a scalable reinforcement learning framework that combines verifiable signals with preference feedback, while remaining stable under large-scale off-policy training, enabling consistent self-improvement across mathematics, code, and tool use. Step 3.5 Flash demonstrates strong performance across agent, coding, and math tasks, achieving 85.4% on IMO-AnswerBench, 86.4% on LiveCodeBench-v6 (2024.08-2025.05), 88.2% on tau2-Bench, 69.0% on BrowseComp (with context management), and 51.0% on Terminal-Bench 2.0, comparable to frontier models such as GPT-5.2 xHigh and Gemini 3.0 Pro. By redefining the efficiency frontier, Step 3.5 Flash provides a high-density foundation for deploying sophisticated agents in real-world industrial environments.

Method: Researchers analyzed various linguistic properties (lexical, morphological, surface, and error features) of Estonian proficiency exam writings (levels A2-C1) to identify relevant proficiency predictors. They trained classification models using pre-selected features and compared them to models with broader feature sets.

Result: The pre-selected features achieved similar test accuracy (around 0.9) but reduced variation in classifying different text types. Evaluation on earlier exam samples showed writings have become more complex over 7-10 years, while maintaining accuracy around 0.8 with some feature sets.

Conclusion: Careful feature selection leads to explainable and generalizable models for language proficiency assessment. The approach has been successfully implemented in an Estonian open-source language learning environment for writing evaluation.

Abstract: Using NLP to analyze authentic learner language helps to build automated assessment and feedback tools. It also offers new and extensive insights into the development of second language production. However, there is a lack of research explicitly combining these aspects. This study aimed to classify Estonian proficiency examination writings (levels A2-C1), assuming that careful feature selection can lead to more explainable and generalizable machine learning models for language testing. Various linguistic properties of the training data were analyzed to identify relevant proficiency predictors associated with increasing complexity and correctness, rather than the writing task. Such lexical, morphological, surface, and error features were used to train classification models, which were compared to models that also allowed for other features. The pre-selected features yielded a similar test accuracy but reduced variation in the classification of different text types. The best classifiers achieved an accuracy of around 0.9. Additional evaluation on an earlier exam sample revealed that the writings have become more complex over a 7-10-year period, while accuracy still reached 0.8 with some feature sets. The results have been implemented in the writing evaluation module of an Estonian open-source language learning environment.

Method: Extended OpenLID classifier with more training data, merged problematic language variant clusters, and introduced special noise label. Evaluated against GlotLID with focus on three groups of closely related languages and contributed new evaluation datasets.

Result: OpenLID-v3 shows improved performance, though ensemble approaches improve precision but reduce coverage for low-resource languages. System is publicly available.

Conclusion: OpenLID-v3 provides better language identification for multilingual dataset building, with particular improvements for closely related languages and noise detection.

Abstract: Language identification (LID) is an essential step in building high-quality multilingual datasets from web data. Existing LID tools (such as OpenLID or GlotLID) often struggle to identify closely related languages and to distinguish valid natural language from noise, which contaminates language-specific subsets, especially for low-resource languages. In this work we extend the OpenLID classifier by adding more training data, merging problematic language variant clusters, and introducing a special label for marking noise. We call this extended system OpenLID-v3 and evaluate it against GlotLID on multiple benchmarks. During development, we focus on three groups of closely related languages (Bosnian, Croatian, and Serbian; Romance varieties of Northern Italy and Southern France; and Scandinavian languages) and contribute new evaluation datasets where existing ones are inadequate. We find that ensemble approaches improve precision but also substantially reduce coverage for low-resource languages. OpenLID-v3 is available on https://huggingface.co/HPLT/OpenLID-v3.

Method: Collected data from four online platforms (social media, e-commerce, customer service), covering Modern Standard Arabic and multiple dialects. Annotated based on Arabic linguistic traditions and pragmatic theory into three classes (polite, impolite, neutral). Created 10,000 samples with linguistic feature annotations across 16 politeness categories.

Result: Achieved substantial inter-annotator agreement (kappa = 0.703). Benchmarked 40 model configurations including traditional ML, transformer-based models, and large language models.

Conclusion: The ADAB dataset supports research on politeness-aware Arabic NLP and addresses the gap in Arabic-language resources for sociopragmatic phenomena.

Abstract: The growing importance of culturally-aware natural language processing systems has led to an increasing demand for resources that capture sociopragmatic phenomena across diverse languages. Nevertheless, Arabic-language resources for politeness detection remain under-explored, despite the rich and complex politeness expressions embedded in Arabic communication. In this paper, we introduce ADAB (Arabic Politeness Dataset), a new annotated Arabic dataset collected from four online platforms, including social media, e-commerce, and customer service domains, covering Modern Standard Arabic and multiple dialects (Gulf, Egyptian, Levantine, and Maghrebi). The dataset was annotated based on Arabic linguistic traditions and pragmatic theory, resulting in three classes: polite, impolite, and neutral. It contains 10,000 samples with linguistic feature annotations across 16 politeness categories and achieves substantial inter-annotator agreement (kappa = 0.703). We benchmark 40 model configurations, including traditional machine learning, transformer-based models, and large language models. The dataset aims to support research on politeness-aware Arabic NLP.

Method: BETA-labeling framework using multiple LLM annotators from diverse families with contextual alignment, consistency checks, majority agreement, and human evaluation. Also tested cross-lingual reuse via one-hop machine translation across multiple language pairs.

Result: Substantial variation across languages in translation quality and task validity, reflecting language-dependent biases and inconsistent semantic preservation. Shows both potential and limitations of LLM-assisted dataset creation.

Conclusion: LLM-assisted dataset creation has potential but requires careful quality control. Cross-lingual dataset reuse via translation has significant risks due to language-dependent biases and semantic preservation issues. Provides guidance for more reliable low-resource language benchmarks.

Abstract: IR in low-resource languages remains limited by the scarcity of high-quality, task-specific annotated datasets. Manual annotation is expensive and difficult to scale, while using large language models (LLMs) as automated annotators introduces concerns about label reliability, bias, and evaluation validity. This work presents a Bangla IR dataset constructed using a BETA-labeling framework involving multiple LLM annotators from diverse model families. The framework incorporates contextual alignment, consistency checks, and majority agreement, followed by human evaluation to verify label quality. Beyond dataset creation, we examine whether IR datasets from other low-resource languages can be effectively reused through one-hop machine translation. Using LLM-based translation across multiple language pairs, we experimented on meaning preservation and task validity between source and translated datasets. Our experiment reveal substantial variation across languages, reflecting language-dependent biases and inconsistent semantic preservation that directly affect the reliability of cross-lingual dataset reuse. Overall, this study highlights both the potential and limitations of LLM-assisted dataset creation for low-resource IR. It provides empirical evidence of the risks associated with cross-lingual dataset reuse and offers practical guidance for constructing more reliable benchmarks and evaluation pipelines in low-resource language settings.

Method: Proposes Spurious-Token-Aware Policy Optimization (STAPO) with S2T mechanism to identify spurious tokens (low probability, low entropy, positive advantage) and suppress their gradient perturbations in a group-based objective.

Result: STAPO achieves superior entropy stability and average performance improvements of 7.13% and 3.69% over baselines across six mathematical reasoning benchmarks using Qwen 1.7B, 8B, and 14B models.

Conclusion: STAPO provides a stable and effective RL fine-tuning approach for large language models by addressing spurious token-induced instability.

Abstract: Reinforcement Learning (RL) has significantly improved large language model reasoning, but existing RL fine-tuning methods rely heavily on heuristic techniques such as entropy regularization and reweighting to maintain stability. In practice, they often suffer from late-stage performance collapse, leading to degraded reasoning quality and unstable training. Our analysis shows that the magnitude of token-wise policy gradients in RL is negatively correlated with token probability and local policy entropy. We find that training instability can be caused by a tiny fraction of tokens, approximately 0.01%, which we term spurious tokens. When such tokens appear in correct responses, they contribute little to the reasoning outcome but inherit the full sequence-level reward, leading to abnormally amplified gradient updates. To mitigate this instability, we design an S2T (silencing spurious tokens) mechanism to efficiently identify spurious tokens through characteristic signals with low probability, low entropy, and positive advantage, and then suppress their gradient perturbations during optimization. Incorporating this mechanism into a group-based objective, we propose Spurious-Token-Aware Policy Optimization (STAPO), which promotes stable and effective large-scale model refinement. Across six mathematical reasoning benchmarks using Qwen 1.7B, 8B, and 14B base models, STAPO consistently demonstrates superior entropy stability and achieves an average performance improvement of 7.13% ($ρ_{\mathrm{T}}$=1.0, top-p=1.0) and 3.69% ($ρ_{\mathrm{T}}$=0.7, top-p=0.9) over GRPO, 20-Entropy, and JustRL.

Method: Collected data through 6 open-ended questions plus PHQ-9 and GAD-7 screening questionnaires from 205 respondents. Analyzed using statistical analysis, Romanian LIWC, emotion detection, and topic modeling.

Result: Created PsihoRo corpus with 205 Romanian texts for depression and anxiety analysis. Demonstrated important features of this new resource through various NLP analysis techniques.

Conclusion: PsihoRo represents the first step toward understanding Romanian mental health through NLP, providing a foundation for future research on depression and anxiety in Romanian language.

Abstract: Psychological corpora in NLP are collections of texts used to analyze human psychology, emotions, and mental health. These texts allow researchers to study psychological constructs, detect mental health issues and analyze emotional language. However, mental health data can be difficult to collect correctly from social media, due to suppositions made by the collectors. A more pragmatic strategy involves gathering data through open-ended questions and then assessing this information with self-report screening surveys. This method was employed successfully for English, a language with a lot of psychological NLP resources. However, this cannot be stated for Romanian, which currently has no open-source mental health corpus. To address this gap, we have created the first corpus for depression and anxiety in Romanian, by utilizing a form with 6 open-ended questions along with the standardized PHQ-9 and GAD-7 screening questionnaires. Consisting of the texts of 205 respondents and although it may seem small, PsihoRo is a first step towards understanding and analyzing texts regarding the mental health of the Romanian population. We employ statistical analysis, text analysis using Romanian LIWC, emotion detection and topic modeling to show what are the most important features of this newly introduced resource to the NLP community.

Method: Replication study of FedTPG’s approach which introduces a text-driven prompt generation network that dynamically creates prompts conditioned on class names. The method enables better cross-class generalization in federated settings without sharing private data.

Result: Evaluation on six diverse vision datasets (Caltech101, Oxford Flowers, FGVC Aircraft, Oxford Pets, Food-101, DTD) achieved results within 0.2% of original paper’s reported accuracies, with average accuracy of 74.58% on seen classes and 76.00% on unseen classes, showing +1.43 percentage point improvement in generalization.

Conclusion: The successful replication confirms FedTPG’s robustness and reproducibility, validating its core claims that text-driven prompt generation enables superior generalization to unseen classes compared to static prompt learning methods, and federated training maintains high performance across diverse visual domains without sharing private data.

Abstract: Vision-language models like CLIP have demonstrated remarkable zero-shot capabilities, yet their adaptation to federated learning scenarios presents significant challenges, particularly regarding generalization to unseen classes. The original FedTPG paper \cite{Qiu2024} addresses this limitation by introducing a text driven prompt generation network that dynamically creates prompts conditioned on class names, enabling better cross-class generalization in federated settings. In this work, we present a faithful replication study of FedTPG, evaluating the pre-trained model on six diverse vision datasets: Caltech101, Oxford Flowers, FGVC Aircraft, Oxford Pets, Food-101, and DTD. Our evaluation achieves results within 0.2% of the original paper’s reported accuracies, with an average accuracy of 74.58% on seen (base) classes and 76.00% on unseen (new) classes, demonstrating a +1.43 percentage point improvement in generalization. These results validate the original paper’s core claims: (1) text-driven prompt generation enables superior generalization to unseen classes compared to static prompt learning methods, and (2) federated training of prompt generators maintains high performance across diverse visual domains without sharing private data. Our successful replication confirms the robustness and reproducibility of the FedTPG approach.

Method: Three key innovations: 1) Modality-specific mixture-of-experts (MS-MoE) for cross-modal interaction while enhancing single-modal quality, 2) Temporal-aligned RoPE (TA-RoPE) for explicit frame-level audio-video synchronization, 3) Audio-video direct preference optimization (AV-DPO) to align outputs with human preferences.

Result: Achieves state-of-the-art performance with only ~1M public training entries, significantly outperforming prior approaches in both qualitative and quantitative evaluations. Built on Wan2.1-1.3B-T2V foundation.

Conclusion: JavisDiT++ bridges the gap between open-source and commercial joint audio-video generation models through effective architectural designs and optimization strategies for synchronization and human preference alignment.

Abstract: AIGC has rapidly expanded from text-to-image generation toward high-quality multimodal synthesis across video and audio. Within this context, joint audio-video generation (JAVG) has emerged as a fundamental task that produces synchronized and semantically aligned sound and vision from textual descriptions. However, compared with advanced commercial models such as Veo3, existing open-source methods still suffer from limitations in generation quality, temporal synchrony, and alignment with human preferences. To bridge the gap, this paper presents JavisDiT++, a concise yet powerful framework for unified modeling and optimization of JAVG. First, we introduce a modality-specific mixture-of-experts (MS-MoE) design that enables cross-modal interaction efficacy while enhancing single-modal generation quality. Then, we propose a temporal-aligned RoPE (TA-RoPE) strategy to achieve explicit, frame-level synchronization between audio and video tokens. Besides, we develop an audio-video direct preference optimization (AV-DPO) method to align model outputs with human preference across quality, consistency, and synchrony dimensions. Built upon Wan2.1-1.3B-T2V, our model achieves state-of-the-art performance merely with around 1M public training entries, significantly outperforming prior approaches in both qualitative and quantitative evaluations. Comprehensive ablation studies have been conducted to validate the effectiveness of our proposed modules. All the code, model, and dataset are released at https://JavisVerse.github.io/JavisDiT2-page.

Method: Developed COMPASS as a temporal digital twin architecture using per-fraction multimodal data (PET, CT, dosiomics, radiomics, BED kinetics). Employed GRU autoencoder to learn organ-specific latent trajectories, classified via logistic regression to predict CTCAE grade 1+ toxicity.

Result: Despite small cohort (8 NSCLC patients, 99 organ fractions, 24 organ trajectories), system identified early warning window with increasing risk ratings several fractions before clinical toxicity. Revealed biologically relevant spatial dose texture characteristics missed by traditional volume-based dosimetry.

Conclusion: COMPASS establishes proof-of-concept for AI-enabled adaptive radiotherapy guided by continually updated digital twins tracking patients’ evolving biological responses, enabling early toxicity prediction and personalized treatment adaptation.

Abstract: Radiotherapy continues to become more precise and data dense, with current treatment regimens generating high frequency imaging and dosimetry streams ideally suited for AI driven temporal modeling to characterize how normal tissues evolve with time. Each fraction in biologically guided radiotherapy(BGRT) treated non small cell lung cancer (NSCLC) patients records new metabolic, anatomical, and dose information. However, clinical decision making is largely informed by static, population based NTCP models which overlook the dynamic, unique biological trajectories encoded in sequential data. We developed COMPASS (Comprehensive Personalized Assessment System) for safe radiotherapy, functioning as a temporal digital twin architecture utilizing per fraction PET, CT, dosiomics, radiomics, and cumulative biologically equivalent dose (BED) kinetics to model normal tissue biology as a dynamic time series process. A GRU autoencoder was employed to learn organ specific latent trajectories, which were classified via logistic regression to predict eventual CTCAE grade 1 or higher toxicity. Eight NSCLC patients undergoing BGRT contributed to the 99 organ fraction observations covering 24 organ trajectories (spinal cord, heart, and esophagus). Despite the small cohort, intensive temporal phenotyping allowed for comprehensive analysis of individual dose response dynamics. Our findings revealed a viable AI driven early warning window, as increasing risk ratings occurred from several fractions before clinical toxicity. The dense BED driven representation revealed biologically relevant spatial dose texture characteristics that occur before toxicity and are averaged out with traditional volume based dosimetry. COMPASS establishes a proof of concept for AI enabled adaptive radiotherapy, where treatment is guided by a continually updated digital twin that tracks each patients evolving biological response.

Method: Proposes Cross-Level Co-Representation (CLCR) with: 1) Semantic hierarchy encoder aligning shallow, mid, deep features; 2) Intra-Level Co-Exchange Domain factorizing features into shared/private subspaces with learnable token budget; 3) Inter-Level Co-Aggregation Domain synchronizing semantic scales with learned anchors; 4) Regularization terms for feature separation.

Result: Experiments on six benchmarks spanning emotion recognition, event localization, sentiment analysis, and action recognition show CLCR achieves strong performance and generalizes well across tasks.

Conclusion: CLCR effectively addresses semantic misalignment in multimodal fusion by organizing features into hierarchical semantic structure with level-wise constraints, improving representation quality and task performance.

Abstract: Multimodal learning aims to capture both shared and private information from multiple modalities. However, existing methods that project all modalities into a single latent space for fusion often overlook the asynchronous, multi-level semantic structure of multimodal data. This oversight induces semantic misalignment and error propagation, thereby degrading representation quality. To address this issue, we propose Cross-Level Co-Representation (CLCR), which explicitly organizes each modality’s features into a three-level semantic hierarchy and specifies level-wise constraints for cross-modal interactions. First, a semantic hierarchy encoder aligns shallow, mid, and deep features across modalities, establishing a common basis for interaction. And then, at each level, an Intra-Level Co-Exchange Domain (IntraCED) factorizes features into shared and private subspaces and restricts cross-modal attention to the shared subspace via a learnable token budget. This design ensures that only shared semantics are exchanged and prevents leakage from private channels. To integrate information across levels, the Inter-Level Co-Aggregation Domain (InterCAD) synchronizes semantic scales using learned anchors, selectively fuses the shared representations, and gates private cues to form a compact task representation. We further introduce regularization terms to enforce separation of shared and private features and to minimize cross-level interference. Experiments on six benchmarks spanning emotion recognition, event localization, sentiment analysis, and action recognition show that CLCR achieves strong performance and generalizes well across tasks.

Method: MARVUS uses a foundation model to enable 3D volumetric reconstruction from conventional 2D ultrasound systems, combined with mobile augmented reality visualizations to guide clinicians during measurements.

Result: In user studies with experienced clinicians on breast phantoms, MARVUS improved volume estimation accuracy (mean difference: 0.469 cm³) and reduced inter-user variability (mean difference: 0.417 cm³). AR visualizations enhanced both objective performance metrics and clinician-reported usability.

Conclusion: MARVUS provides a scalable, cost-effective solution for accurate volumetric assessment in ultrasound-based cancer screening and diagnosis, potentially enhancing clinical workflows without requiring expensive specialized hardware.

Abstract: Accurate volumetric characterization of lesions is essential for oncologic diagnosis, risk stratification, and treatment planning. While imaging modalities such as Computed Tomography provide high-quality 3D data, 2D ultrasound (2D-US) remains the preferred first-line modality for breast and thyroid imaging due to cost, portability, and safety factors. However, volume estimates derived from 2D-US suffer from high inter-user variability even among experienced clinicians. Existing 3D ultrasound (3D-US) solutions use specialized probes or external tracking hardware, but such configurations increase costs and diminish portability, constraining widespread clinical use. To address these limitations, we present Mobile Augmented Reality Volumetric Ultrasound (MARVUS), a resource-efficient system designed to increase accessibility to accurate and reproducible volumetric assessment. MARVUS is interoperable with conventional ultrasound (US) systems, using a foundation model to enhance cross-specialty generalization while minimizing hardware requirements relative to current 3D-US solutions. In a user study involving experienced clinicians performing measurements on breast phantoms, MARVUS yielded a substantial improvement in volume estimation accuracy (mean difference: 0.469 cm3) with reduced inter-user variability (mean difference: 0.417 cm3). Additionally, we prove that augmented reality (AR) visualizations enhance objective performance metrics and clinician-reported usability. Collectively, our findings suggests that MARVUS can enhance US-based cancer screening, diagnostic workflows, and treatment planning in a scalable, cost-conscious, and resource-efficient manner. Usage video demonstration available (https://youtu.be/m4llYcZpqmM).

Method: Systematically evaluated feature disentanglement methods including adversarial learning and latent space splitting based on dependence minimization. Assessed classification performance and disentanglement quality using latent space analyses across one artificial and two medical datasets with natural and synthetic confounders. Examined robustness under varying confounding levels and compared computational efficiency.

Result: Shortcut mitigation methods improved classification performance under strong spurious correlations during training. Latent space analyses revealed representation quality differences not captured by classification metrics. Model reliance on shortcuts depended on the degree of confounding in training data. Best-performing models combined data-centric rebalancing with model-centric disentanglement.

Conclusion: Combining data-centric rebalancing with model-centric disentanglement achieves stronger and more robust shortcut mitigation than rebalancing alone while maintaining similar computational efficiency, providing a comprehensive approach to address shortcut learning in medical imaging.

Abstract: Although deep learning models in medical imaging often achieve excellent classification performance, they can rely on shortcut learning, exploiting spurious correlations or confounding factors that are not causally related to the target task. This poses risks in clinical settings, where models must generalize across institutions, populations, and acquisition conditions. Feature disentanglement is a promising approach to mitigate shortcut learning by separating task-relevant information from confounder-related features in latent representations. In this study, we systematically evaluated feature disentanglement methods for mitigating shortcuts in medical imaging, including adversarial learning and latent space splitting based on dependence minimization. We assessed classification performance and disentanglement quality using latent space analyses across one artificial and two medical datasets with natural and synthetic confounders. We also examined robustness under varying levels of confounding and compared computational efficiency across methods. We found that shortcut mitigation methods improved classification performance under strong spurious correlations during training. Latent space analyses revealed differences in representation quality not captured by classification metrics, highlighting the strengths and limitations of each method. Model reliance on shortcuts depended on the degree of confounding in the training data. The best-performing models combine data-centric rebalancing with model-centric disentanglement, achieving stronger and more robust shortcut mitigation than rebalancing alone while maintaining similar computational efficiency.

Method: Created Very Big Video Reasoning (VBVR) Dataset with 200 curated reasoning tasks following a principled taxonomy and over 1 million video clips. Developed VBVR-Bench with rule-based, human-aligned scorers for verifiable evaluation beyond model-based judging.

Result: Dataset is ~3 orders of magnitude larger than existing datasets. Early signs of emergent generalization to unseen reasoning tasks observed in scaling studies. Provides foundation for generalizable video reasoning research.

Conclusion: VBVR addresses the data gap for video reasoning research, enabling systematic study of scaling behavior and laying groundwork for next-stage research in generalizable video reasoning capabilities.

Abstract: Rapid progress in video models has largely focused on visual quality, leaving their reasoning capabilities underexplored. Video reasoning grounds intelligence in spatiotemporally consistent visual environments that go beyond what text can naturally capture, enabling intuitive reasoning over spatiotemporal structure such as continuity, interaction, and causality. However, systematically studying video reasoning and its scaling behavior is hindered by the lack of large-scale training data. To address this gap, we introduce the Very Big Video Reasoning (VBVR) Dataset, an unprecedentedly large-scale resource spanning 200 curated reasoning tasks following a principled taxonomy and over one million video clips, approximately three orders of magnitude larger than existing datasets. We further present VBVR-Bench, a verifiable evaluation framework that moves beyond model-based judging by incorporating rule-based, human-aligned scorers, enabling reproducible and interpretable diagnosis of video reasoning capabilities. Leveraging the VBVR suite, we conduct one of the first large-scale scaling studies of video reasoning and observe early signs of emergent generalization to unseen reasoning tasks. Together, VBVR lays a foundation for the next stage of research in generalizable video reasoning. The data, benchmark toolkit, and models are publicly available at https://video-reason.com/ .

Method: End-to-end system combining YOLO object detector with ByteTrack tracking algorithm to identify and track players, referees, goalkeepers, and the ball from single-camera broadcast footage.

Result: Pipeline achieves high performance in detecting and tracking players and officials with strong precision, recall, and mAP50 scores, though ball detection remains challenging.

Conclusion: AI can extract meaningful player-level spatial information from single broadcast cameras, enabling scalable data-driven analysis for colleges, academies, and amateur clubs without specialized hardware.

Abstract: Clubs with access to expensive multi-camera setups or GPS tracking systems gain a competitive advantage through detailed data, whereas lower-budget teams are often unable to collect similar information. This paper examines whether such data can instead be extracted directly from standard broadcast footage using a single-camera computer vision pipeline. This project develops an end-to-end system that combines a YOLO object detector with the ByteTrack tracking algorithm to identify and track players, referees, goalkeepers, and the ball throughout a match. Experimental results show that the pipeline achieves high performance in detecting and tracking players and officials, with strong precision, recall, and mAP50 scores, while ball detection remains the primary challenge. Despite this limitation, our findings demonstrate that AI can extract meaningful player-level spatial information from a single broadcast camera. By reducing reliance on specialized hardware, the proposed approach enables colleges, academies, and amateur clubs to adopt scalable, data-driven analysis methods previously accessible only to professional teams, highlighting the potential for affordable computer vision-based soccer analytics.

Method: Proposes YOLO-SAM2 framework with depth enhancement: 1) Sleeper-aligned depth-correction pipeline using polynomial modeling, RANSAC, and temporal smoothing to compensate for RealSense spatial distortion, 2) SAM2 segmentation refines region-of-interest masks, 3) Geometric analysis of sleeper and ballast profiles for classification.

Result: Depth-enhanced configurations substantially improve detection: recall increases from 0.49 to 0.80, F1-score improves from 0.66 to over 0.80. Performance varies with bounding-box sampling (AABB or RBB) and geometric criteria.

Conclusion: Integrating depth correction with YOLO-SAM2 yields more robust and reliable automated railway ballast inspection, particularly for visually ambiguous or safety-critical scenarios.

Abstract: This paper presents a depth-enhanced YOLO-SAM2 framework for detecting ballast insufficiency in railway tracks using RGB-D data. Although YOLOv8 provides reliable localization, the RGB-only model shows limited safety performance, achieving high precision (0.99) but low recall (0.49) due to insufficient ballast, as it tends to over-predict the sufficient class. To improve reliability, we incorporate depth-based geometric analysis enabled by a sleeper-aligned depth-correction pipeline that compensates for RealSense spatial distortion using polynomial modeling, RANSAC, and temporal smoothing. SAM2 segmentation further refines region-of-interest masks, enabling accurate extraction of sleeper and ballast profiles for geometric classification. Experiments on field-collected top-down RGB-D data show that depth-enhanced configurations substantially improve the detection of insufficient ballast. Depending on bounding-box sampling (AABB or RBB) and geometric criteria, recall increases from 0.49 to as high as 0.80, and F1-score improves from 0.66 to over 0.80. These results demonstrate that integrating depth correction with YOLO-SAM2 yields a more robust and reliable approach for automated railway ballast inspection, particularly in visually ambiguous or safety-critical scenarios.

Method: Proposes a restoration-based analysis framework using Sparse Autoencoders to identify class-specific expert features in intermediate layers and applies inference-time steering to quantitatively distinguish between suppression and deletion of information.

Result: Analysis of 12 major unlearning methods in image classification shows most achieve high restoration rates of unlearned information, indicating they only suppress information at decision boundaries while preserving semantic features in intermediate representations. Even retraining from pretrained checkpoints shows high restoration.

Conclusion: Representation-level retention poses significant risks overlooked by output-based metrics, highlighting the need for new evaluation criteria that prioritize representation-level verification, especially for privacy-critical applications with pretrained models.

Abstract: As pretrained models are increasingly shared on the web, ensuring that models can forget or delete sensitive, copyrighted, or private information upon request has become crucial. Machine unlearning has been proposed to address this challenge. However, current evaluations for unlearning methods rely on output-based metrics, which cannot verify whether information is completely deleted or merely suppressed at the representation level, where suppression is insufficient for true unlearning. To address this gap, we propose a novel restoration-based analysis framework that uses Sparse Autoencoders to identify class-specific expert features in intermediate layers and applies inference-time steering to quantitatively distinguish between suppression and deletion. Applying our framework to 12 major unlearning methods in image classification tasks, we find that most methods achieve high restoration rates of unlearned information, indicating that they only suppress information at the decision-boundary level, while preserving semantic features in intermediate representations. Notably, even retraining from pretrained checkpoints shows high restoration, revealing that robust semantic features inherited from pretraining are not removed by retraining. These results demonstrate that representation-level retention poses significant risks overlooked by output-based metrics, highlighting the need for new unlearning evaluation criteria. We propose new evaluation guidelines that prioritize representation-level verification, especially for privacy-critical applications in the era of pre-trained models.

Method: Proposes CASO-PAD, an RGB-only, single-frame model that enhances MobileNetV3 with content-adaptive spatial operators (involution). Unlike standard convolution with spatially shared kernels, this operator generates location-specific, channel-shared kernels conditioned on input, improving spatial selectivity with minimal overhead. The model is lightweight (3.6M parameters, 0.64 GFLOPs) and trained end-to-end with binary cross-entropy.

Result: Achieves strong performance across multiple benchmarks: 100/100/98.9/99.7% test accuracy on Replay-Attack, Replay-Mobile, ROSE-Youtu, and OULU-NPU; AUC of 1.00/1.00/0.9995/0.9999; HTER of 0.00/0.00/0.82/0.44%. On SiW-Mv2 Protocol-1, attains 95.45% accuracy with 3.11% HTER and 3.13% EER. Ablation studies show optimal placement near network head with moderate group sharing.

Conclusion: CASO-PAD provides a practical pathway for robust, on-device FacePAD with mobile-class compute, without requiring auxiliary sensors or temporal information. The content-adaptive spatial operators effectively capture localized spoof cues while maintaining efficiency.

Abstract: Face presentation attack detection (FacePAD) is critical for securing facial authentication against print, replay, and mask-based spoofing. This paper proposes CASO-PAD, an RGB-only, single-frame model that enhances MobileNetV3 with content-adaptive spatial operators (involution) to better capture localized spoof cues. Unlike spatially shared convolution kernels, the proposed operator generates location-specific, channel-shared kernels conditioned on the input, improving spatial selectivity with minimal overhead. CASO-PAD remains lightweight (3.6M parameters; 0.64 GFLOPs at $256\times256$) and is trained end-to-end using a standard binary cross-entropy objective. Extensive experiments on Replay-Attack, Replay-Mobile, ROSE-Youtu, and OULU-NPU demonstrate strong performance, achieving 100/100/98.9/99.7% test accuracy, AUC of 1.00/1.00/0.9995/0.9999, and HTER of 0.00/0.00/0.82/0.44%, respectively. On the large-scale SiW-Mv2 Protocol-1 benchmark, CASO-PAD further attains 95.45% accuracy with 3.11% HTER and 3.13% EER, indicating improved robustness under diverse real-world attacks. Ablation studies show that placing the adaptive operator near the network head and using moderate group sharing yields the best accuracy–efficiency balance. Overall, CASO-PAD provides a practical pathway for robust, on-device FacePAD with mobile-class compute and without auxiliary sensors or temporal stacks.

Method: Alternating minimization approach: when depth map is fixed, solve linear forward model for all-in-focus image using convex optimization; when all-in-focus image is fixed, compute depth at each pixel independently using parallel grid search. This enables embarrassingly parallel computation.

Result: The approach effectively solves depth-from-defocus at higher resolutions than current deep learning methods, demonstrated on benchmark datasets with synthetic and real defocus blur, showing promising results compared to prior approaches.

Conclusion: A global optimization approach to depth from defocus is feasible with modern optimization methods and computing resources, offering an alternative to deep learning methods with competitive performance at higher resolutions.

Abstract: Though there exists a reasonable forward model for blur based on optical physics, recovering depth from a collection of defocused images remains a computationally challenging optimization problem. In this paper, we show that with contemporary optimization methods and reasonable computing resources, a global optimization approach to depth from defocus is feasible. Our approach rests on alternating minimization. When holding the depth map fixed, the forward model is linear with respect to the all-in-focus image. When holding the all-in-focus image fixed, the depth at each pixel can be computed independently, enabling embarrassingly parallel computation. We show that alternating between convex optimization and parallel grid search can effectively solve the depth-from-defocus problem at higher resolutions than current deep learning methods. We demonstrate our approach on benchmark datasets with synthetic and real defocus blur and show promising results compared to prior approaches. Our code is available at github.com/hollyjackson/dfd.

Method: A four-stage grammar-in-the-loop framework: 1) hybrid perception, 2) symbolic graph construction, 3) constraint checking, and 4) constrained VLM feedback. The language model only verbalizes violations verified by an upstream rule engine, reducing hallucination.

Result: Mixed results: Qwen2-VL-7B achieved highest micro-F1 on both free-body diagrams (0.570) and circuits (0.528) but with extreme hallucination rates (0.78, 0.98). The grammar pipeline produced more actionable circuit feedback (4.85/5) than end-to-end LMM (3.11/5). Confidence thresholding reduced hallucination with no F1 loss.

Conclusion: Grammar-in-the-loop approaches can reduce hallucination in multimodal feedback systems for STEM education while maintaining performance, demonstrating complementarity between symbolic reasoning and end-to-end LMM approaches.

Abstract: Providing timely, rubric-aligned feedback on student-drawn diagrams is a persistent challenge in STEM education. While large multimodal models (LMMs) can jointly parse images and generate explanations, their tendency to hallucinate undermines trust in classroom deployments. We present Sketch2Feedback, a grammar-in-the-loop framework that decomposes the problem into four stages – hybrid perception, symbolic graph construction, constraint checking, and constrained VLM feedback – so that the language model verbalizes only violations verified by an upstream rule engine. We evaluate on two synthetic micro-benchmarks, FBD-10 (free-body diagrams) and Circuit-10 (circuit schematics), each with 500 images spanning standard and hard noise augmentation tiers, comparing our pipeline against end-to-end LMMs (LLaVA-1.5-7B, Qwen2-VL-7B), a vision-only detector, a YOLOv8-nano learned detector, and an ensemble oracle. On n=100 test samples per benchmark with 95% bootstrap CIs, results are mixed and instructive: Qwen2-VL-7B achieves the highest micro-F1 on both FBDs (0.570) and circuits (0.528), but with extreme hallucination rates (0.78, 0.98). An ensemble oracle that selects the best prediction per sample reaches F1=0.556 with hallucination 0.320 on FBDs, demonstrating exploitable complementarity between grammar and end-to-end approaches. Confidence thresholding at tau=0.7 reduces circuit hallucination from 0.970 to 0.880 with no F1 loss. Hard noise augmentation reveals domain-dependent robustness: FBD detection is resilient while circuit detection degrades sharply. An LLM-as-judge evaluation confirms that the grammar pipeline produces more actionable circuit feedback (4.85/5) than the end-to-end LMM (3.11/5). We release all code, datasets, and evaluation scripts.

Method: Train and test ten deep stereo matching networks on real tree branch images using Canterbury Tree Branches dataset (5,313 stereo pairs) with DEFOM-generated disparity maps as training targets; evaluate using perceptual metrics (SSIM, LPIPS, ViTScore) and structural metrics (SIFT/ORB feature matching).

Result: BANet-3D produces best overall quality (SSIM=0.883, LPIPS=0.157), RAFT-Stereo scores highest on scene-level understanding (ViTScore=0.799), AnyNet reaches 6.99 FPS at 1080P (only near-real-time option), and BANet-2D gives best quality-speed balance at 1.21 FPS.

Conclusion: The study provides practical guidance for selecting stereo matching networks in forestry drone systems, balancing accuracy and computational efficiency for real-time tree pruning applications.

Abstract: Autonomous drone-based tree pruning needs accurate, real-time depth estimation from stereo cameras. Depth is computed from disparity maps using $Z = f B/d$, so even small disparity errors cause noticeable depth mistakes at working distances. Building on our earlier work that identified DEFOM-Stereo as the best reference disparity generator for vegetation scenes, we present the first study to train and test ten deep stereo matching networks on real tree branch images. We use the Canterbury Tree Branches dataset – 5,313 stereo pairs from a ZED Mini camera at 1080P and 720P – with DEFOM-generated disparity maps as training targets. The ten methods cover step-by-step refinement, 3D convolution, edge-aware attention, and lightweight designs. Using perceptual metrics (SSIM, LPIPS, ViTScore) and structural metrics (SIFT/ORB feature matching), we find that BANet-3D produces the best overall quality (SSIM = 0.883, LPIPS = 0.157), while RAFT-Stereo scores highest on scene-level understanding (ViTScore = 0.799). Testing on an NVIDIA Jetson Orin Super (16 GB, independently powered) mounted on our drone shows that AnyNet reaches 6.99 FPS at 1080P – the only near-real-time option – while BANet-2D gives the best quality-speed balance at 1.21 FPS. We also compare 720P and 1080P processing times to guide resolution choices for forestry drone systems.

Method: Controlled evaluation across three single-class detection regimes using six different generators (GAN, diffusion, hybrid). Trained YOLOv11 from scratch and with COCO-pretrained initialization, testing augmentation ratios from 10% to 150%. Computed pre-training dataset metrics including global feature-space metrics (Inception-v3, DINOv2) and object-centric distribution distances over bounding-box statistics.

Result: Synthetic augmentation yields substantial gains in challenging regimes (+7.6% in Pedestrian, +30.6% in PottedPlant) but marginal in Traffic Signs. Metric-performance alignment is regime-dependent, and many apparent raw associations weaken after controlling for augmentation level.

Conclusion: The effectiveness of synthetic augmentation depends on dataset characteristics and difficulty. Standard generative metrics don’t reliably predict detection performance, and metric-performance relationships vary across different detection regimes.

Abstract: Synthetic images are increasingly used to augment object-detection training sets, but reliably evaluating a synthetic dataset before training remains difficult: standard global generative metrics (e.g., FID) often do not predict downstream detection mAP. We present a controlled evaluation of synthetic augmentation for YOLOv11 across three single-class detection regimes – Traffic Signs (sparse/near-saturated), Cityscapes Pedestrian (dense/occlusion-heavy), and COCO PottedPlant (multi-instance/high-variability). We benchmark six GAN-, diffusion-, and hybrid-based generators over augmentation ratios from 10% to 150% of the real training split, and train YOLOv11 both from scratch and with COCO-pretrained initialization, evaluating on held-out real test splits (mAP@0.50:0.95). For each dataset-generator-augmentation configuration, we compute pre-training dataset metrics under a matched-size bootstrap protocol, including (i) global feature-space metrics in both Inception-v3 and DINOv2 embeddings and (ii) object-centric distribution distances over bounding-box statistics. Synthetic augmentation yields substantial gains in the more challenging regimes (up to +7.6% and +30.6% relative mAP in Pedestrian and PottedPlant, respectively) but is marginal in Traffic Signs and under pretrained fine-tuning. To separate metric signal from augmentation quantity, we report both raw and augmentation-controlled (residualized) correlations with multiple-testing correction, showing that metric-performance alignment is strongly regime-dependent and that many apparent raw associations weaken after controlling for augmentation level.

Method: GOT-Edit uses online cross-modality model editing that integrates geometry-aware cues from a pre-trained Visual Geometry Grounded Transformer. It performs null-space constrained updates to incorporate geometric information while preserving semantic discrimination.

Result: Extensive experiments on multiple GOT benchmarks show GOT-Edit achieves superior robustness and accuracy, particularly under occlusion and clutter, establishing a new paradigm for combining 2D semantics with 3D geometric reasoning.

Conclusion: The approach successfully integrates 3D geometric reasoning into 2D object tracking through online model editing, demonstrating improved performance in challenging scenarios and offering a new framework for multimodal tracking.

Abstract: Human perception for effective object tracking in a 2D video stream arises from the implicit use of prior 3D knowledge combined with semantic reasoning. In contrast, most generic object tracking (GOT) methods primarily rely on 2D features of the target and its surroundings while neglecting 3D geometric cues, which makes them susceptible to partial occlusion, distractors, and variations in geometry and appearance. To address this limitation, we introduce GOT-Edit, an online cross-modality model editing approach that integrates geometry-aware cues into a generic object tracker from a 2D video stream. Our approach leverages features from a pre-trained Visual Geometry Grounded Transformer to enable geometric cue inference from only a few 2D images. To tackle the challenge of seamlessly combining geometry and semantics, GOT-Edit performs online model editing with null-space constrained updates that incorporate geometric information while preserving semantic discrimination, yielding consistently better performance across diverse scenarios. Extensive experiments on multiple GOT benchmarks demonstrate that GOT-Edit achieves superior robustness and accuracy, particularly under occlusion and clutter, establishing a new paradigm for combining 2D semantics with 3D geometric reasoning for generic object tracking.

Method: The article synthesizes seminal and recent advances in unsupervised and self-supervised learning frameworks applied to various biomedical data modalities including medical imaging (MRI, volumetric scans) and genomic sequences.

Result: Unsupervised methods can derive heritable cardiac traits, predict spatial gene expression in histology, and detect pathologies with performance rivaling or exceeding supervised counterparts, demonstrating the value of learning without labels.

Conclusion: Unsupervised and self-supervised learning represents a paradigm shift in biomedical AI, enabling discovery of novel phenotypes and reducing dependence on expert annotation while maintaining or improving performance.

Abstract: The dependence on expert annotation has long constituted the primary rate-limiting step in the application of artificial intelligence to biomedicine. While supervised learning drove the initial wave of clinical algorithms, a paradigm shift towards unsupervised and self-supervised learning (SSL) is currently unlocking the latent potential of biobank-scale datasets. By learning directly from the intrinsic structure of data - whether pixels in a magnetic resonance image (MRI), voxels in a volumetric scan, or tokens in a genomic sequence - these methods facilitate the discovery of novel phenotypes, the linkage of morphology to genetics, and the detection of anomalies without human bias. This article synthesises seminal and recent advances in “learning without labels,” highlighting how unsupervised frameworks can derive heritable cardiac traits, predict spatial gene expression in histology, and detect pathologies with performance that rivals or exceeds supervised counterparts.

Method: Introduces JAEGER framework with Neural IV (neural intensity vector) - a learned spatial audio representation for robust directional cues, integrates RGB-D observations and multi-channel ambisonics, and uses SpatialSceneQA benchmark (61k instruction-tuning samples) for training/evaluation.

Result: Extensive experiments show JAEGER consistently surpasses 2D-centric baselines across diverse spatial perception and reasoning tasks, demonstrating the necessity of explicit 3D modeling.

Conclusion: Explicit 3D modeling is crucial for advancing AI in physical environments, and JAEGER’s integration of spatial audio and 3D visual data enables superior spatial reasoning capabilities.

Abstract: Current audio-visual large language models (AV-LLMs) are predominantly restricted to 2D perception, relying on RGB video and monaural audio. This design choice introduces a fundamental dimensionality mismatch that precludes reliable source localization and spatial reasoning in complex 3D environments. We address this limitation by presenting JAEGER, a framework that extends AV-LLMs to 3D space, to enable joint spatial grounding and reasoning through the integration of RGB-D observations and multi-channel first-order ambisonics. A core contribution of our work is the neural intensity vector (Neural IV), a learned spatial audio representation that encodes robust directional cues to enhance direction-of-arrival estimation, even in adverse acoustic scenarios with overlapping sources. To facilitate large-scale training and systematic evaluation, we propose SpatialSceneQA, a benchmark of 61k instruction-tuning samples curated from simulated physical environments. Extensive experiments demonstrate that our approach consistently surpasses 2D-centric baselines across diverse spatial perception and reasoning tasks, underscoring the necessity of explicit 3D modelling for advancing AI in physical environments. Our source code, pre-trained model checkpoints and datasets will be released upon acceptance.

Method: Used 2500 images of 5 olive varieties; trained 10 CNN and transformer architectures (MobileNetV2, EfficientNetB0, EfficientNetV2-S, ResNet50, ResNet101, DenseNet121, InceptionV3, ConvNeXt-Tiny, ViT-B16, Swin-T) via transfer learning; evaluated using multiple metrics including accuracy, precision, recall, F1-score, MCC, Cohen’s Kappa, ROC-AUC, parameters, FLOPs, inference time, and generalization gap.

Result: EfficientNetV2-S achieved highest classification accuracy (95.8%); EfficientNetB0 provided best trade-off between accuracy and computational complexity; parametric efficiency found more critical than model depth under limited data conditions.

Conclusion: Deep learning architectures can effectively classify black table olive varieties from images, with EfficientNetV2-S performing best overall and EfficientNetB0 offering optimal balance for practical deployment; parametric efficiency is key for limited data scenarios.

Abstract: This study compares multiple deep learning architectures for the automated, image-based classification of five locally cultivated black table olive varieties in Turkey: Gemlik, Ayvalik, Uslu, Erkence, and Celebi. Using a dataset of 2500 images, ten architectures - MobileNetV2, EfficientNetB0, EfficientNetV2-S, ResNet50, ResNet101, DenseNet121, InceptionV3, ConvNeXt-Tiny, ViT-B16, and Swin-T - were trained using transfer learning. Model performance was evaluated using accuracy, precision, recall, F1-score, Matthews Correlation Coefficient (MCC), Cohen’s Kappa, ROC-AUC, number of parameters, FLOPs, inference time, and generalization gap. EfficientNetV2-S achieved the highest classification accuracy (95.8%), while EfficientNetB0 provided the best trade-off between accuracy and computational complexity. Overall, the results indicate that under limited data conditions, parametric efficiency plays a more critical role than model depth alone.

Method: Proposes CTC-driven teacher forcing where greedily decoded CTC pseudo-labels feed into decoder to generate attention targets in single forward pass. Uses mixed sampling to mitigate decoder exposure bias. Enables simultaneous prediction of CTC and attention targets.

Result: Halves training time, improves robustness to out-of-distribution inputs, achieves state-of-the-art results on LRS3, LRS2, and WildVSR benchmarks, surpassing original USR and modality-specific self-supervised baselines.

Conclusion: USR 2.0 provides efficient and robust unified speech recognition framework that addresses limitations of previous approach while maintaining strong performance across audio, visual, and audiovisual modalities.

Abstract: Unified Speech Recognition (USR) has emerged as a semi-supervised framework for training a single model for audio, visual, and audiovisual speech recognition, achieving state-of-the-art results on in-distribution benchmarks. However, its reliance on autoregressive pseudo-labelling makes training expensive, while its decoupled supervision of CTC and attention branches increases susceptibility to self-reinforcing errors, particularly under distribution shifts involving longer sequences, noise, or unseen domains. We propose CTC-driven teacher forcing, where greedily decoded CTC pseudo-labels are fed into the decoder to generate attention targets in a single forward pass. Although these can be globally incoherent, in the pseudo-labelling setting they enable efficient and effective knowledge transfer. Because CTC and CTC-driven attention pseudo-labels have the same length, the decoder can predict both simultaneously, benefiting from the robustness of CTC and the expressiveness of attention without costly beam search. We further propose mixed sampling to mitigate the exposure bias of the decoder relying solely on CTC inputs. The resulting method, USR 2.0, halves training time, improves robustness to out-of-distribution inputs, and achieves state-of-the-art results on LRS3, LRS2, and WildVSR, surpassing USR and modality-specific self-supervised baselines.

Method: Start from simple VLA baseline similar to RT-2/OpenVLA, systematically dissect design choices along three dimensions: foundational components, perception essentials, and action modelling perspectives. Distill 12 key findings into practical recipe.

Result: Proposed VLANeXt model outperforms prior state-of-the-art methods on LIBERO and LIBERO-plus benchmarks and demonstrates strong generalization in real-world experiments.

Conclusion: Provides structured framework for VLA design space, practical recipe for building strong VLAs, and releases unified codebase for community to build upon shared foundation.

Abstract: Following the rise of large foundation models, Vision-Language-Action models (VLAs) emerged, leveraging strong visual and language understanding for general-purpose policy learning. Yet, the current VLA landscape remains fragmented and exploratory. Although many groups have proposed their own VLA models, inconsistencies in training protocols and evaluation settings make it difficult to identify which design choices truly matter. To bring structure to this evolving space, we reexamine the VLA design space under a unified framework and evaluation setup. Starting from a simple VLA baseline similar to RT-2 and OpenVLA, we systematically dissect design choices along three dimensions: foundational components, perception essentials, and action modelling perspectives. From this study, we distill 12 key findings that together form a practical recipe for building strong VLA models. The outcome of this exploration is a simple yet effective model, VLANeXt. VLANeXt outperforms prior state-of-the-art methods on the LIBERO and LIBERO-plus benchmarks and demonstrates strong generalization in real-world experiments. We will release a unified, easy-to-use codebase that serves as a common platform for the community to reproduce our findings, explore the design space, and build new VLA variants on top of a shared foundation.

Method: Based on Diffusion Transformer (DiT) architecture, JavisDiT introduces a Hierarchical Spatial-Temporal Synchronized Prior (HiST-Sypo) Estimator for fine-grained spatio-temporal alignment, and creates JavisBench dataset with 10,140 high-quality text-captioned sounding videos for evaluation.

Result: JavisDiT significantly outperforms existing methods in both generation quality and audio-video synchronization, establishing a new state-of-the-art for JAVG tasks.

Conclusion: The proposed unified framework with hierarchical alignment mechanism effectively solves the synchronization challenge in joint audio-video generation, setting a new standard for multimodal content creation.

Abstract: This paper introduces JavisDiT, a novel Joint Audio-Video Diffusion Transformer designed for synchronized audio-video generation (JAVG). Based on the powerful Diffusion Transformer (DiT) architecture, JavisDiT simultaneously generates high-quality audio and video content from open-ended user prompts in a unified framework. To ensure audio-video synchronization, we introduce a fine-grained spatio-temporal alignment mechanism through a Hierarchical Spatial-Temporal Synchronized Prior (HiST-Sypo) Estimator. This module extracts both global and fine-grained spatio-temporal priors, guiding the synchronization between the visual and auditory components. Furthermore, we propose a new benchmark, JavisBench, which consists of 10,140 high-quality text-captioned sounding videos and focuses on synchronization evaluation in diverse and complex real-world scenarios. Further, we specifically devise a robust metric for measuring the synchrony between generated audio-video pairs in real-world content. Experimental results demonstrate that JavisDiT significantly outperforms existing methods by ensuring both high-quality generation and precise synchronization, setting a new standard for JAVG tasks. Our code, model, and data are available at https://javisverse.github.io/JavisDiT-page/.

Method: Study 1: Use morphological descriptors (e.g., “platinum blonde,” “beauty mark”) to navigate identity basins in Stable Diffusion 1.5 via self-distillation loop (generate synthetic images, train LoRA). Study 2: Generate 200 nonsense words from English sound-symbolic clusters (phonesthemes) and evaluate visual coherence of generated images.

Result: Study 1: LoRA training achieves consistent convergence toward specific identities measured by ArcFace similarity; creates local coordinate system producing both target identity and “uncanny valley” inverses. Study 2: Phonestheme-bearing nonsense words produce significantly more visually coherent outputs than random controls (Purity@1 = 0.371 vs. 0.209); three words achieve perfect visual consistency (Purity@1 = 1.0).

Conclusion: Morphological structure (feature descriptors and phonological form) creates systematic navigational gradients through diffusion model latent spaces, enabling identity manipulation and novel visual concept generation from sub-lexical sound patterns.

Abstract: We demonstrate that morphological pressure creates navigable gradients at multiple levels of the text-to-image generative pipeline. In Study1, identity basins in Stable Diffusion 1.5 can be navigated using morphological descriptors – constituent features like platinum blonde,’’ beauty mark,’’ and 1950s glamour’’ – without the target’s name or photographs. A self-distillation loop (generating synthetic images from descriptor prompts, then training a LoRA on those outputs) achieves consistent convergence toward a specific identity as measured by ArcFace similarity. The trained LoRA creates a local coordinate system shaping not only the target identity but also its inverse: maximal away-conditioning produces eldritch’’ structural breakdown in base SD1.5, while the LoRA-equipped model produces ``uncanny valley’’ outputs – coherent but precisely wrong. In Study2, we extend this to prompt-level morphology. Drawing on phonestheme theory, we generate 200 novel nonsense words from English sound-symbolic clusters (e.g., \emph{cr-}, \emph{sn-}, \emph{-oid}, \emph{-ax}) and find that phonestheme-bearing candidates produce significantly more visually coherent outputs than random controls (mean Purity@1 = 0.371 vs.\ 0.209, p<0.00001p < 0.00001 p<0.00001, Cohen’s d=0.55d = 0.55 d=0.55). Three candidates – \emph{snudgeoid}, \emph{crashax}, and \emph{broomix} – achieve perfect visual consistency (Purity@1 = 1.0) with zero training data contamination, each generating a distinct, coherent visual identity from phonesthetic structure alone. Together, these studies establish that morphological structure – whether in feature descriptors or prompt-level phonological form – creates systematic navigational gradients through diffusion model latent spaces. We document phase transitions in identity basins, CFG-invariant identity stability, and novel visual concepts emerging from sub-lexical sound patterns.

Method: Created Rodent-Bench with diverse datasets covering social interactions, grooming, scratching, and freezing behaviors in videos ranging from 10-35 minutes. Evaluated state-of-the-art MLLMs (Gemini-2.5-Pro, Gemini-2.5-Flash, Qwen-VL-Max) using multiple metrics: second-wise accuracy, macro F1, mean average precision, mutual information, and Matthew’s correlation coefficient.

Result: None of the tested models performed strongly enough to be used as assistants for rodent behavior annotation. Some models showed modest performance on specific datasets (notably grooming detection), but overall revealed significant challenges in temporal segmentation, handling extended video sequences, and distinguishing subtle behavioral states.

Conclusion: Current MLLMs have significant limitations for scientific video annotation tasks. Rodent-Bench provides a foundation for tracking progress toward reliable automated behavioral annotation in neuroscience research and identifies key areas for future model development.

Abstract: We present Rodent-Bench, a novel benchmark designed to evaluate the ability of Multimodal Large Language Models (MLLMs) to annotate rodent behaviour footage. We evaluate state-of-the-art MLLMs, including Gemini-2.5-Pro, Gemini-2.5-Flash and Qwen-VL-Max, using this benchmark and find that none of these models perform strongly enough to be used as an assistant for this task. Our benchmark encompasses diverse datasets spanning multiple behavioral paradigms including social interactions, grooming, scratching, and freezing behaviors, with videos ranging from 10 minutes to 35 minutes in length. We provide two benchmark versions to accommodate varying model capabilities and establish standardized evaluation metrics including second-wise accuracy, macro F1, mean average precision, mutual information, and Matthew’s correlation coefficient. While some models show modest performance on certain datasets (notably grooming detection), overall results reveal significant challenges in temporal segmentation, handling extended video sequences, and distinguishing subtle behavioral states. Our analysis identifies key limitations in current MLLMs for scientific video annotation and provides insights for future model development. Rodent-Bench serves as a foundation for tracking progress toward reliable automated behavioral annotation in neuroscience research.

Method: WAVE employs a novel hierarchical feature fusion strategy and joint multi-modal, multi-task training approach to create unified embeddings for text, audio, and video modalities.

Result: WAVE sets new SOTA on MMEB-v2 video benchmark, achieves superior results in audio and video-to-audio retrieval, and significantly outperforms existing models in multimodal question answering.

Conclusion: WAVE opens up broad possibilities for cross-modal, any-to-any applications with its prompt-aware embeddings and unified representation space for audio-visual-text modalities.

Abstract: While embeddings from multimodal large language models (LLMs) excel as general-purpose representations, their application to dynamic modalities like audio and video remains underexplored. We introduce WAVE (\textbf{u}nified & \textbf{v}ersatile \textbf{a}udio-\textbf{v}isual \textbf{e}mbeddings), the first LLM-based embedding that creates a unified representation space for text, audio, and video modalities. WAVE employs a novel hierarchical feature fusion strategy and a joint multi-modal, multi-task training approach to enable two key capabilities: any-to-any cross-modal retrieval and the generation of prompt-aware embeddings tailored to user instructions. Experimentally, WAVE sets a new state-of-the-art on the MMEB-v2 video benchmark and achieves superior results in audio and video-to-audio retrieval. Its prompt-aware nature also yields remarkable performance in multimodal question answering, significantly outperforming existing embedding models. Ablation studies validate our joint training strategy, demonstrating improved performance across all modalities. With a newly introduced benchmark for versatile audio-visual learning, WAVE opens up broad possibilities for cross-modal, any-to-any applications. Our code and checkpoints are released at \href{https://github.com/TCL606/WAVE}{https://github.com/TCL606/WAVE}.

Method: Benchmarked YOLO architectures (YOLOv5s, YOLOv8n/s/m, YOLOv12n) on new FloralSix dataset (2,816 high-resolution photos of 6 flower species) under two annotation regimes: single-image single-bounding box (SISBB) for sparse scenarios and single-image multiple-bounding box (SIMBB) for dense scenarios. Evaluated using Precision, Recall, and mAP at IoU thresholds 0.5 and 0.5-0.95.

Result: YOLOv8m (SGD) performed best in SISBB with Precision 0.956, Recall 0.951, mAP@0.5 0.978, and mAP@0.5:0.95 0.865. YOLOv12n (SGD) outperformed in SIMBB with mAP@0.5 0.934 and mAP@0.5:0.95 0.752. SGD optimizer consistently performed better than alternatives. Models optimized for recall perform better in crowded environments, while precision-oriented models excel in sparse scenarios.

Conclusion: The study demonstrates how annotation density, IoU thresholds, and model size interact for flower detection, providing insights for agricultural applications like non-destructive crop analysis, growth tracking, robotic pollination, and stress evaluation.

Abstract: Precise localization and recognition of flowers are crucial for advancing automated agriculture, particularly in plant phenotyping, crop estimation, and yield monitoring. This paper benchmarks several YOLO architectures such as YOLOv5s, YOLOv8n/s/m, and YOLOv12n for flower object detection under two annotation regimes: single-image single-bounding box (SISBB) and single-image multiple-bounding box (SIMBB). The FloralSix dataset, comprising 2,816 high-resolution photos of six different flower species, is also introduced. It is annotated for both dense (clustered) and sparse (isolated) scenarios. The models were evaluated using Precision, Recall, and Mean Average Precision (mAP) at IoU thresholds of 0.5 (mAP@0.5) and 0.5-0.95 (mAP@0.5:0.95). In SISBB, YOLOv8m (SGD) achieved the best results with Precision 0.956, Recall 0.951, mAP@0.5 0.978, and mAP@0.5:0.95 0.865, illustrating strong accuracy in detecting isolated flowers. With mAP@0.5 0.934 and mAP@0.5:0.95 0.752, YOLOv12n (SGD) outperformed the more complicated SIMBB scenario, proving robustness in dense, multi-object detection. Results show how annotation density, IoU thresholds, and model size interact: recall-optimized models perform better in crowded environments, whereas precision-oriented models perform best in sparse scenarios. In both cases, the Stochastic Gradient Descent (SGD) optimizer consistently performed better than alternatives. These density-sensitive sensors are helpful for non-destructive crop analysis, growth tracking, robotic pollination, and stress evaluation.

Method: Conducted thorough evaluation using 12 medical imaging datasets (7 2D, 5 3D) with various patch sizes (1, 2, 4, 7, 14, 28). Fine-tuned ViT models on a single GPU and applied ensemble strategy fusing predictions from models trained with patch sizes 1, 2, and 4.

Result: Smaller patch sizes (1, 2, 4) achieved best results across nearly all datasets: up to 12.78% balanced accuracy improvement for 2D datasets (patch size 2 vs. 28) and up to 23.78% for 3D datasets (patch size 1 vs. 14). Ensemble strategy further boosted performance, especially for 2D datasets.

Conclusion: Smaller patch sizes significantly improve ViT classification performance in medical imaging, though at increased computational cost. Ensemble methods provide additional performance gains, offering practical guidance for medical vision applications.

Abstract: Vision Transformers (ViTs) and their variants have become state-of-the-art in many computer vision tasks and are widely used as backbones in large-scale vision and vision-language foundation models. While substantial research has focused on architectural improvements, the impact of patch size, a crucial initial design choice in ViTs, remains underexplored, particularly in medical domains where both two-dimensional (2D) and three-dimensional (3D) imaging modalities exist. In this study, using 12 medical imaging datasets from various imaging modalities (including seven 2D and five 3D datasets), we conduct a thorough evaluation of how different patch sizes affect ViT classification performance. Using a single graphical processing unit (GPU) and a range of patch sizes (1, 2, 4, 7, 14, 28), we fine-tune ViT models and observe consistent improvements in classification performance with smaller patch sizes (1, 2, and 4), which achieve the best results across nearly all datasets. More specifically, our results indicate improvements in balanced accuracy of up to 12.78% for 2D datasets (patch size 2 vs. 28) and up to 23.78% for 3D datasets (patch size 1 vs. 14), at the cost of increased computational expense. Moreover, by applying a straightforward ensemble strategy that fuses the predictions of the models trained with patch sizes 1, 2, and 4, we demonstrate a further boost in performance in most cases, especially for the 2D datasets. Our implementation is publicly available on GitHub: https://github.com/HealMaDe/MedViT

Method: Encodes text prompt, driving image, and voice profile into a multi-entangled latent space that establishes spatiotemporal person-specific features across modalities, then uses separate decoders for audio and video generation.

Result: Generates realistic synchronized talking faces with corresponding voice output from static images and text inputs.

Conclusion: The approach successfully integrates audio and visual modalities for realistic talking face generation using a multi-entangled latent space architecture.

Abstract: We present a novel approach for generating realistic speaking and talking faces by synthesizing a person’s voice and facial movements from a static image, a voice profile, and a target text. The model encodes the prompt/driving text, the driving image, and the voice profile of an individual and then combines them to pass them to the multi-entangled latent space to foster key-value pairs and queries for the audio and video modality generation pipeline. The multi-entangled latent space is responsible for establishing the spatiotemporal person-specific features between the modalities. Further, entangled features are passed to the respective decoder of each modality for output audio and video generation.

Method: Uses DUSt3R for dense 3D reconstructions from image pairs, warps image contents between views, then compares feature maps to obtain view-invariant similarity scores.

Result: MEt3R successfully evaluates consistency across various novel view and video generation methods, including the authors’ own multi-view latent diffusion model.

Conclusion: MEt3R provides a valuable metric for assessing multi-view consistency in generative models, addressing a critical gap in evaluation methods for 3D inference from sparse observations.

Abstract: We introduce MEt3R, a metric for multi-view consistency in generated images. Large-scale generative models for multi-view image generation are rapidly advancing the field of 3D inference from sparse observations. However, due to the nature of generative modeling, traditional reconstruction metrics are not suitable to measure the quality of generated outputs and metrics that are independent of the sampling procedure are desperately needed. In this work, we specifically address the aspect of consistency between generated multi-view images, which can be evaluated independently of the specific scene. Our approach uses DUSt3R to obtain dense 3D reconstructions from image pairs in a feed-forward manner, which are used to warp image contents from one view into the other. Then, feature maps of these images are compared to obtain a similarity score that is invariant to view-dependent effects. Using MEt3R, we evaluate the consistency of a large set of previous methods for novel view and video generation, including our open, multi-view latent diffusion model.

Method: LoRun introduces a novel paradigm where a single pretrained base denoiser is shared across all stages, while lightweight, stage-specific LoRA adapters are injected into the PMMs to dynamically modulate denoising behavior according to the noise level at each unfolding step. This decouples core restoration capability from task-specific adaptation.

Result: The method achieves up to N times parameter reduction for an N-stage DUN with on-par or better performance. Extensive experiments conducted on three image restoration tasks validate the efficiency of the method.

Conclusion: LoRun harmonizes denoising objectives and adapts different denoising levels between stages with compressed memory usage for more efficient Deep Unfolding Networks in image restoration.

Abstract: Deep unfolding networks (DUNs), combining conventional iterative optimization algorithms and deep neural networks into a multi-stage framework, have achieved remarkable accomplishments in Image Restoration (IR), such as spectral imaging reconstruction, compressive sensing and super-resolution.It unfolds the iterative optimization steps into a stack of sequentially linked blocks.Each block consists of a Gradient Descent Module (GDM) and a Proximal Mapping Module (PMM) which is equivalent to a denoiser from a Bayesian perspective, operating on Gaussian noise with a known level.However, existing DUNs suffer from two critical limitations: (i) their PMMs share identical architectures and denoising objectives across stages, ignoring the need for stage-specific adaptation to varying noise levels; and (ii) their chain of structurally repetitive blocks results in severe parameter redundancy and high memory consumption, hindering deployment in large-scale or resource-constrained scenarios.To address these challenges, we introduce generalized Deep Low-rank Adaptation (LoRA) Unfolding Networks for image restoration, named LoRun, harmonizing denoising objectives and adapting different denoising levels between stages with compressed memory usage for more efficient DUN.LoRun introduces a novel paradigm where a single pretrained base denoiser is shared across all stages, while lightweight, stage-specific LoRA adapters are injected into the PMMs to dynamically modulate denoising behavior according to the noise level at each unfolding step.This design decouples the core restoration capability from task-specific adaptation, enabling precise control over denoising intensity without duplicating full network parameters and achieving up to $N$ times parameter reduction for an $N$-stage DUN with on-par or better performance.Extensive experiments conducted on three IR tasks validate the efficiency of our method.

Method: Multi-stream convolutional encoders with global context modeling, cross-modal attention fusion for stable multi-modal feature integration, and variance-aware training strategy to enhance robustness across diverse tasks.

Result: Achieved DSC of 0.831 and HD95 of 3.03 on WMH MICCAI 2017, lowest HD95 of 9.69 on ISLES 2022, and highest tumor core DSC of 0.8651 on BraTS 2020, showing consistent improvements in boundary accuracy and stability.

Conclusion: SYNAPSE-Net provides a clinically relevant, robust solution for automated brain lesion segmentation with improved generalizability across diverse pathologies and reduced performance variance.

Abstract: Automatic segmentation of diverse heterogeneous brain lesions using multi-modal MRI is a challenging problem in clinical neuroimaging, mainly because of the lack of generalizability and high prediction variance of pathology-specific deep learning models. In this work, we propose a unified and adaptive multi-stream framework called SYNAPSE-Net to perform robust multi-pathology segmentation with reduced performance variance. The framework is based on multi-stream convolutional encoders with global context modeling and a cross-modal attention fusion strategy to ensure stable and effective multi-modal feature integration. It also employs a variance-aware training strategy to enhance the robustness of the network across diverse tasks. The framework is extensively validated using three public challenge datasets: WMH MICCAI 2017, ISLES 2022, and BraTS 2020. The results show consistent improvements in boundary accuracy, delineation quality, and stability across diverse pathologies. This proposed framework achieved a high Dice similarity coefficient (DSC) of 0.831 and a low Hausdorff distance at the 95th percentile (HD95) of 3.03 on the WMH MICCAI 2017 dataset. It also achieved the lowest HD95 of 9.69 on the ISLES 2022 dataset and the highest tumor core DSC of 0.8651 on the BraTS 2020 dataset. These results validate the robustness of the proposed framework in providing a clinically relevant computer-aided solution for automated brain lesion segmentation. Source code and pretrained models are publicly available at https://github.com/mubid-01/SYNAPSE-Net-pre.

Method: Proposes Video-TwG with Think-with-Grounding paradigm: 1) Two-stage Reinforced Curriculum Strategy (learn on short videos with grounding labels, then scale to diverse QA data), 2) TwG-GRPO algorithm with fine-grained grounding reward, self-confirmed pseudo reward, and accuracy-gated mechanism, 3) New TwG-51K dataset for training.

Result: Outperforms strong LVU baselines on Video-MME, LongVideoBench, and MLVU benchmarks. Ablation studies validate the necessity of the Two-stage Reinforced Curriculum Strategy and show TwG-GRPO improves grounding quality and reduces redundant groundings without sacrificing QA performance.

Conclusion: Video-TwG effectively addresses hallucinations in long video understanding by enabling active on-demand grounding during multimodal reasoning, with a curriculum learning approach that scales from short to long videos and diverse domains.

Abstract: Long video understanding is challenging due to rich and complicated multimodal clues in long temporal range.Current methods adopt reasoning to improve the model’s ability to analyze complex video clues in long videos via text-form reasoning.However,the existing literature suffers from the fact that the text-only reasoning under fixed video context may exacerbate hallucinations since detailed crucial clues are often ignored under limited video context length due to the temporal redundancy of long videos.To address this gap,we propose Video-TwG,a curriculum reinforced framework that employs a novel Think-with-Grounding paradigm,enabling video LLMs to actively decide when to perform on-demand grounding during interleaved text-video reasoning, selectively zooming into question-relevant clips only when necessary.Video-TwG can be trained end-to-end in a straightforward manner, without relying on complex auxiliary modules or heavily annotated reasoning tracesIn detail,we design a Two-stage Reinforced Curriculum Strategy, where the model first learns think-with-grounding behavior on a small short-video GQA dataset with grounding labels,and then scales to diverse general QA data with videos of diverse domains to encourage generalization. Further, to handle complex think-with-grounding reasoning for various kinds of data,we propose TwG-GRPO algorithm which features the fine-grained grounding reward, self-confirmed pseudo reward and accuracy-gated mechanism.Finally,we propose to construct a new TwG-51K dataset that facilitates training. Experiments on Video-MME, LongVideoBench, and MLVU show that Video-TwG consistently outperforms strong LVU baselines.Further ablation validates the necessity of our Two-stage Reinforced Curriculum Strategy and shows our TwG-GRPO better leverages diverse unlabeled data to improve grounding quality and reduce redundant groundings without sacrificing QA performance.

Method: Extends a geometry foundation model to predict dense geometry and cross-view consistent instance embeddings, enabling semantic-synergized association and instance-guided loop closure using viewpoint-agnostic semantic anchors.

Result: Significantly outperforms state-of-the-art methods, particularly in map consistency and wide-baseline loop closure reliability.

Conclusion: IRIS-SLAM effectively bridges geometric reconstruction and open-vocabulary mapping through unified geometric-instance representations.

Abstract: Geometry foundation models have significantly advanced dense geometric SLAM, yet existing systems often lack deep semantic understanding and robust loop closure capabilities. Meanwhile, contemporary semantic mapping approaches are frequently hindered by decoupled architectures and fragile data association. We propose IRIS-SLAM, a novel RGB semantic SLAM system that leverages unified geometric-instance representations derived from an instance-extended foundation model. By extending a geometry foundation model to concurrently predict dense geometry and cross-view consistent instance embeddings, we enable a semantic-synergized association mechanism and instance-guided loop closure detection. Our approach effectively utilizes viewpoint-agnostic semantic anchors to bridge the gap between geometric reconstruction and open-vocabulary mapping. Experimental results demonstrate that IRIS-SLAM significantly outperforms state-of-the-art methods, particularly in map consistency and wide-baseline loop closure reliability.

Method: Systematic analysis of LVLM decoders (Qwen, LLaMA, Vicuna) reveals layer-wise susceptibility to hallucinations. Introduces Hallucination Insensitivity Score (HIS) to quantify each layer’s sensitivity. Proposes HIME - a layer-adaptive weight editing approach that selectively modifies latent features based on HIS guidance to suppress hallucinations while preserving knowledge.

Result: HIME reduces hallucinations by an average of 61.8% across open-ended generation benchmarks (CHAIR, MME, GPT-4V-aided evaluation) without introducing additional parameters, inference-time latency, or computational overhead.

Conclusion: The paper presents an effective training-free solution for mitigating object hallucination in LVLMs through principled layer-adaptive model editing, balancing hallucination suppression with knowledge preservation.

Abstract: Large Vision-Language Models (LVLMs) have demonstrated impressive multimodal understanding capabilities, yet they remain prone to object hallucination, where models describe non-existent objects or attribute incorrect factual information, raising serious concerns for reliable real-world deployment. While fine-tuning is a commonly adopted mitigation strategy, its high computational cost and practical difficulty motivate the need for training-free alternatives, among which model editing has recently emerged as a promising direction. However, indiscriminate editing risks disrupting the rich implicit knowledge encoded in pre-trained LVLMs, leading to a fundamental question: how much intervention is necessary at each layer to suppress hallucinations while preserving pre-trained knowledge? To address this question, we present a systematic analysis of LVLM decoders built on three widely used large language model backbones-Qwen, LLaMA, and Vicuna-revealing clear layer-wise differences in susceptibility to object hallucination. Building on these insights, we introduce the Hallucination Insensitivity Score (HIS), a principled metric that quantifies each layer’s sensitivity to hallucination and provides guidance for targeted intervention. Leveraging HIS, we propose Hallucination Insensitivity Model Editing (HIME), a simple yet effective layer-adaptive weight editing approach that selectively modifies latent features to suppress hallucinations while preserving pre-trained knowledge. Extensive experiments demonstrate that HIME reduces hallucinations by an average of 61.8% across open-ended generation benchmarks, including CHAIR, MME, and GPT-4V-aided evaluation, without introducing additional parameters, inference-time latency, or computational overhead.

Method: Proposes NeXt2Former-CD with three key components: 1) Siamese ConvNeXt encoder initialized with DINOv3 weights, 2) deformable attention-based temporal fusion module for handling spatial misalignments, and 3) Mask2Former decoder for segmentation.

Result: Achieves state-of-the-art results on LEVIR-CD, WHU-CD, and CDD datasets, outperforming recent Mamba-based SSM baselines in both F1 score and IoU, while maintaining comparable inference latency despite larger parameter count.

Conclusion: Demonstrates that modern convolutional and attention architectures can be competitive alternatives to SSMs for change detection, offering better performance while maintaining practical inference speeds for high-resolution tasks.

Abstract: State Space Models (SSMs) have recently gained traction in remote sensing change detection (CD) for their favorable scaling properties. In this paper, we explore the potential of modern convolutional and attention-based architectures as a competitive alternative. We propose NeXt2Former-CD, an end-to-end framework that integrates a Siamese ConvNeXt encoder initialized with DINOv3 weights, a deformable attention-based temporal fusion module, and a Mask2Former decoder. This design is intended to better tolerate residual co-registration noise and small object-level spatial shifts, as well as semantic ambiguity in bi-temporal imagery. Experiments on LEVIR-CD, WHU-CD, and CDD datasets show that our method achieves the best results among the evaluated methods, improving over recent Mamba-based baselines in both F1 score and IoU. Furthermore, despite a larger parameter count, our model maintains inference latency comparable to SSM-based approaches, suggesting it is practical for high-resolution change detection tasks.

Method: Combines high-quality motion blur dataset generation (using controllable camera/object motion simulations over SAM segmented regions with alpha-aware compositing) with a U-Net based detector using ImageNet pretrained encoders, curriculum learning, hard negatives, focal loss, blur frequency channels, and resolution-aware augmentation.

Result: Achieves 89.68% accuracy on GoPro (vs 66.50% baseline) and 59.77% Mean IoU on CUHK (vs 9.00% baseline), showing 6.6x improvement in segmentation. Demonstrates accurate localization of subtle blur artifacts for automated filtering and ROI extraction.

Conclusion: Proposes a unified framework for motion blur detection that addresses limitations of existing benchmarks and methods, enabling practical applications in streaming service quality control through zero-shot generalization.

Abstract: Streaming services serve hundreds of millions of viewers worldwide, where visual assets such as thumbnails, box art, and cover images are critical for engagement. Subtle motion blur remains a pervasive quality issue, reducing visual clarity and negatively affecting user trust and click-through rates. However, motion blur detection from static images is underexplored, as existing methods and datasets focus on severe blur and lack fine-grained pixel-level annotations needed for quality-critical applications. Benchmarks such as GOPRO and NFS are dominated by strong synthetic blur and often contain residual blur in their sharp references, leading to ambiguous supervision. We propose SMBlurDetect, a unified framework combining high-quality motion blur specific dataset generation with an end-to-end detector capable of zero-shot detection at multiple granularities. Our pipeline synthesizes realistic motion blur from super high resolution aesthetic images using controllable camera and object motion simulations over SAM segmented regions, enhanced with alpha-aware compositing and balanced sampling to generate subtle, spatially localized blur with precise ground truth masks. We train a U-Net based detector with ImageNet pretrained encoders using a hybrid mask and image centric strategy incorporating curriculum learning, hard negatives, focal loss, blur frequency channels, and resolution aware augmentation.Our method achieves strong zero-shot generalization, reaching 89.68% accuracy on GoPro (vs 66.50% baseline) and 59.77% Mean IoU on CUHK (vs 9.00% baseline), demonstrating 6.6x improvement in segmentation. Qualitative results show accurate localization of subtle blur artifacts, enabling automated filtering of low quality frames and precise region of interest extraction for intelligent cropping.

Method: WiCompass leverages large-scale motion-capture corpora to build a universal pose space “oracle” that quantifies dataset redundancy and identifies underrepresented motions. It employs a closed-loop policy to prioritize collecting informative missing samples based on this oracle’s guidance.

Result: WiCompass consistently improves out-of-distribution accuracy at matched budgets and exhibits superior scaling behavior compared to conventional collection strategies. It demonstrates that coverage-aware data acquisition is more effective than brute-force scaling.

Conclusion: By shifting focus from brute-force scaling to coverage-aware data acquisition, WiCompass offers a practical path toward robust mmWave sensing for human pose estimation, addressing the generalization challenges under distribution shifts.

Abstract: Millimeter-wave Human Pose Estimation (mmWave HPE) promises privacy but suffers from poor generalization under distribution shifts. We demonstrate that brute-force data scaling is ineffective for out-of-distribution (OOD) robustness; efficiency and coverage are the true bottlenecks. To address this, we introduce WiCompass, a coverage-aware data-collection framework. WiCompass leverages large-scale motion-capture corpora to build a universal pose space ``oracle’’ that quantifies dataset redundancy and identifies underrepresented motions. Guided by this oracle, WiCompass employs a closed-loop policy to prioritize collecting informative missing samples. Experiments show that WiCompass consistently improves OOD accuracy at matched budgets and exhibits superior scaling behavior compared to conventional collection strategies. By shifting focus from brute-force scaling to coverage-aware data acquisition, this work offers a practical path toward robust mmWave sensing.

Method: Created two datasets: MiS (safety) with safe/unsafe scenario pairs, and MiC (culture) with cultural proxy pairs. Each sample contains two minimally differing captions and corresponding minimally differing images. Evaluated four VLMs on tasks requiring fine-grained differentiation of paired images and captions.

Result: Models perform better at confirming correct image-caption pairs than rejecting incorrect ones. Higher accuracy when selecting correct caption from two similar captions for a given image vs. the converse task. Overall results show persistent modality misalignment challenges in current VLMs.

Conclusion: The study highlights the difficulty of precise cross-modal grounding required for applications with subtle semantic and visual distinctions, underscoring ongoing challenges in fine-grained vision-language alignment.

Abstract: Fine-grained image-caption alignment is crucial for vision-language models (VLMs), especially in socially critical contexts such as identifying real-world risk scenarios or distinguishing cultural proxies, where correct interpretation hinges on subtle visual or linguistic clues and where minor misinterpretations can lead to significant real-world consequences. We present MiSCHiEF, a set of two benchmarking datasets based on a contrastive pair design in the domains of safety (MiS) and culture (MiC), and evaluate four VLMs on tasks requiring fine-grained differentiation of paired images and captions. In both datasets, each sample contains two minimally differing captions and corresponding minimally differing images. In MiS, the image-caption pairs depict a safe and an unsafe scenario, while in MiC, they depict cultural proxies in two distinct cultural contexts. We find that models generally perform better at confirming the correct image-caption pair than rejecting incorrect ones. Additionally, models achieve higher accuracy when selecting the correct caption from two highly similar captions for a given image, compared to the converse task. The results, overall, highlight persistent modality misalignment challenges in current VLMs, underscoring the difficulty of precise cross-modal grounding required for applications with subtle semantic and visual distinctions.

Method: Two-stage design: 1) Explicit replacement stage preserves partial observation geometry for faithful completion; 2) Implicit refinement stage ensures seamless boundaries between observed and synthesized regions. The framework is training-free and compatible with different 3D foundation models.

Result: Outperforms previous state-of-the-art approaches in both quantitative and qualitative experiments. Introduces Omni-Comp benchmark combining real-world and synthetic data with diverse partial patterns for comprehensive evaluation.

Conclusion: LaS-Comp successfully leverages 3D foundation models’ geometric priors for zero-shot, category-agnostic shape completion, demonstrating superior performance across diverse partial observation scenarios.

Abstract: This paper introduces LaS-Comp, a zero-shot and category-agnostic approach that leverages the rich geometric priors of 3D foundation models to enable 3D shape completion across diverse types of partial observations. Our contributions are threefold: First, \ourname{} harnesses these powerful generative priors for completion through a complementary two-stage design: (i) an explicit replacement stage that preserves the partial observation geometry to ensure faithful completion; and (ii) an implicit refinement stage ensures seamless boundaries between the observed and synthesized regions. Second, our framework is training-free and compatible with different 3D foundation models. Third, we introduce Omni-Comp, a comprehensive benchmark combining real-world and synthetic data with diverse and challenging partial patterns, enabling a more thorough and realistic evaluation. Both quantitative and qualitative experiments demonstrate that our approach outperforms previous state-of-the-art approaches. Our code and data will be available at \href{https://github.com/DavidYan2001/LaS-Comp}{LaS-Comp}.

Method: Proposes a pipeline that generates multimodal geometry problems through symbolic seed construction, grounded instantiation with verification, and code-based diagram rendering. Introduces code prediction as an explicit alignment objective to transform visual understanding into supervised structured prediction.

Result: GeoCode dataset exhibits substantially higher structural complexity and reasoning difficulty than existing benchmarks while maintaining mathematical correctness. Models trained on GeoCode achieve consistent improvements on multiple geometry benchmarks.

Conclusion: The proposed dataset generation pipeline and code prediction alignment strategy effectively enhance multimodal geometry reasoning capabilities in vision-language models.

Abstract: Multimodal geometry reasoning requires models to jointly understand visual diagrams and perform structured symbolic inference, yet current vision–language models struggle with complex geometric constructions due to limited training data and weak visual–symbolic alignment. We propose a pipeline for synthesizing complex multimodal geometry problems from scratch and construct a dataset named \textbf{GeoCode}, which decouples problem generation into symbolic seed construction, grounded instantiation with verification, and code-based diagram rendering, ensuring consistency across structure, text, reasoning, and images. Leveraging the plotting code provided in GeoCode, we further introduce code prediction as an explicit alignment objective, transforming visual understanding into a supervised structured prediction task. GeoCode exhibits substantially higher structural complexity and reasoning difficulty than existing benchmarks, while maintaining mathematical correctness through multi-stage validation. Extensive experiments show that models trained on GeoCode achieve consistent improvements on multiple geometry benchmarks, demonstrating both the effectiveness of the dataset and the proposed alignment strategy. The code will be available at https://github.com/would1920/GeoCode.

Method: Proposes MIRROR framework with closed-loop process: draft, critique, region-based verification, and revision. Creates ReflectV dataset for multi-turn supervision with reflection triggers, region-based verification actions, and evidence-grounded revisions.

Result: Improves correctness and reduces visual hallucinations on general and reasoning vision-language benchmarks, showing value of evidence-seeking, region-aware verification over purely textual revision.

Conclusion: Training reflection as visual evidence-seeking process with region-based verification significantly enhances multimodal reasoning and reduces hallucinations in VLMs.

Abstract: In the era of Vision-Language Models (VLMs), enhancing multimodal reasoning capabilities remains a critical challenge, particularly in handling ambiguous or complex visual inputs, where initial inferences often lead to hallucinations or logic errors. Existing VLMs often produce plausible yet ungrounded answers, and even when prompted to “reflect”, their corrections may remain detached from the image evidence. To address this, we propose the MIRROR framework for Multimodal Iterative Reasoning via Reflection On visual Regions. By embedding visual reflection as a core mechanism, MIRROR is formulated as a closed-loop process comprising draft, critique, region-based verification, and revision, which are repeated until the output is visually grounded. To facilitate training of this model, we construct ReflectV, a visual reflective dataset for multi-turn supervision that explicitly contains reflection triggers, region-based verification actions, and answer revision grounded in visual evidence. Experiments on both general vision-language benchmarks and representative vision-language reasoning benchmarks show that MIRROR improves correctness and reduces visual hallucinations, demonstrating the value of training reflection as an evidence-seeking, region-aware verification process rather than a purely textual revision step.

Method: Proposes benchmarking approach using attention maps from foundation models as pixel-wise features, classified with XGBoost for fast, interpretable, model-agnostic evaluation without finetuning on four histopathology datasets.

Result: Vision-language model CONCH performed best overall, with PathDino as close second. Feature concatenation from CONCH, PathDino and CellViT outperformed individual models by 7.95% averaged across datasets, showing ensembles generalize better.

Conclusion: Foundation models show strong potential for histopathology segmentation, with vision-language models outperforming vision-only models, and feature concatenation from complementary models further improves performance for diverse segmentation tasks.

Abstract: In recent years, foundation models such as CLIP, DINO,and CONCH have demonstrated remarkable domain generalization and unsupervised feature extraction capabilities across diverse imaging tasks. However, systematic and independent evaluations of these models for pixel-level semantic segmentation in histopathology remain scarce. In this study, we propose a robust benchmarking approach to asses 10 foundational models on four histopathological datasets covering both morphological tissue-region and cellular/nuclear segmentation tasks. Our method leverages attention maps of foundation models as pixel-wise features, which are then classified using a machine learning algorithm, XGBoost, enabling fast, interpretable, and model-agnostic evaluation without finetuning. We show that the vision language foundation model, CONCH performed the best across datasets when compared to vision-only foundation models, with PathDino as close second. Further analysis shows that models trained on distinct histopathology cohorts capture complementary morphological representations, and concatenating their features yields superior segmentation performance. Concatenating features from CONCH, PathDino and CellViT outperformed individual models across all the datasets by 7.95% (averaged across the datasets), suggesting that ensembles of foundation models can better generalize to diverse histopathological segmentation tasks.

Method: Proposes an Alignment-Disentanglement-Entanglement framework with three components: 1) Adaptive mixing strategy for aligning cross-source latent representations, 2) Hybrid solver for disentangling source-specific identity attributes, and 3) Attentional gating mechanism for selective visual element entanglement.

Result: EditedID achieves state-of-the-art performance in preserving original facial identity and edited element consistency. It’s a training-free, plug-and-play solution that establishes new benchmarks for single/multi-person facial identity restoration in open-world settings.

Conclusion: The framework enables practical deployment of multimodal editing models in real-person editing scenarios by solving facial identity preservation challenges through systematic analysis of diffusion trajectories and attention properties.

Abstract: Multimodal editing large models have demonstrated powerful editing capabilities across diverse tasks. However, a persistent and long-standing limitation is the decline in facial identity (ID) consistency during realistic portrait editing. Due to the human eye’s high sensitivity to facial features, such inconsistency significantly hinders the practical deployment of these models. Current facial ID preservation methods struggle to achieve consistent restoration of both facial identity and edited element IP due to Cross-source Distribution Bias and Cross-source Feature Contamination. To address these issues, we propose EditedID, an Alignment-Disentanglement-Entanglement framework for robust identity-specific facial restoration. By systematically analyzing diffusion trajectories, sampler behaviors, and attention properties, we introduce three key components: 1) Adaptive mixing strategy that aligns cross-source latent representations throughout the diffusion process. 2) Hybrid solver that disentangles source-specific identity attributes and details. 3) Attentional gating mechanism that selectively entangles visual elements. Extensive experiments show that EditedID achieves state-of-the-art performance in preserving original facial ID and edited element IP consistency. As a training-free and plug-and-play solution, it establishes a new benchmark for practical and reliable single/multi-person facial identity restoration in open-world settings, paving the way for the deployment of multimodal editing large models in real-person editing scenarios. The code is available at https://github.com/NDYBSNDY/EditedID.

Method: Proposes Person2Drive platform with: 1) open-source data collection system simulating realistic scenarios to generate scalable personalized driving datasets; 2) style vector-based evaluation metrics using Maximum Mean Discrepancy and KL divergence to quantify individual driving behaviors; 3) personalized E2E-AD framework with style reward model that adapts E2E models for safe, individualized driving.

Result: Extensive experiments demonstrate that Person2Drive enables fine-grained analysis, reproducible evaluation, and effective personalization in end-to-end autonomous driving.

Conclusion: Person2Drive addresses key challenges in personalized autonomous driving through a comprehensive platform for data collection, evaluation, and personalization, enabling safe and individualized driving behaviors.

Abstract: Human driving behavior is inherently diverse, yet most end-to-end autonomous driving (E2E-AD) systems learn a single average driving style, neglecting individual differences. Achieving personalized E2E-AD faces challenges across three levels: limited real-world datasets with individual-level annotations, a lack of quantitative metrics for evaluating personal driving styles, and the absence of algorithms that can learn stylized representations from users’ trajectories. To address these gaps, we propose Person2Drive, a comprehensive personalized E2E-AD platform and benchmark. It includes an open-source, flexible data collection system that simulates realistic scenarios to generate scalable and diverse personalized driving datasets; style vector-based evaluation metrics with Maximum Mean Discrepancy and KL divergence to comprehensively quantify individual driving behaviors; and a personalized E2E-AD framework with a style reward model that efficiently adapts E2E models for safe and individualized driving. Extensive experiments demonstrate that Person2Drive enables fine-grained analysis, reproducible evaluation, and effective personalization in end-to-end autonomous driving. Our dataset and code will be released after acceptance.

Method: Proposes TAG framework that constrains multimodal reasoning to be supported by facial Action Units (AUs), requiring intermediate reasoning steps to be grounded in AU-related facial regions. Uses supervised fine-tuning on AU-grounded reasoning traces followed by reinforcement learning with AU-aware reward aligning predicted regions with external AU detectors.

Result: Outperforms strong open-source and closed-source VLM baselines on RAF-DB, FERPlus, and AffectNet while improving visual faithfulness. AU-grounded rewards stabilize reasoning and mitigate hallucination.

Conclusion: Structured grounded intermediate representations (via AUs) are crucial for trustworthy multimodal reasoning in facial expression recognition, demonstrating importance of verifiable visual evidence.

Abstract: Facial Expression Recognition (FER) is a fine-grained visual understanding task where reliable predictions require reasoning over localized and meaningful facial cues. Recent vision–language models (VLMs) enable natural language explanations for FER, but their reasoning is often ungrounded, producing fluent yet unverifiable rationales that are weakly tied to visual evidence and prone to hallucination, leading to poor robustness across different datasets. We propose TAG (Thinking with Action Unit Grounding), a vision–language framework that explicitly constrains multimodal reasoning to be supported by facial Action Units (AUs). TAG requires intermediate reasoning steps to be grounded in AU-related facial regions, yielding predictions accompanied by verifiable visual evidence. The model is trained via supervised fine-tuning on AU-grounded reasoning traces followed by reinforcement learning with an AU-aware reward that aligns predicted regions with external AU detectors. Evaluated on RAF-DB, FERPlus, and AffectNet, TAG consistently outperforms strong open-source and closed-source VLM baselines while simultaneously improving visual faithfulness. Ablation and preference studies further show that AU-grounded rewards stabilize reasoning and mitigate hallucination, demonstrating the importance of structured grounded intermediate representations for trustworthy multimodal reasoning in FER. The code will be available at https://github.com/would1920/FER_TAG .

Method: Foundation model-driven mapping framework using multisource geospatial data (optical remote sensing images and geo-vector data) to create high-resolution UV mapping across 342 Chinese cities, with geographically stratified accuracy assessment.

Result: Created GeoLink-UV dataset covering 342 cities, showing UVs account for 8% of built-up land with clustering in central/south China, revealing low-rise, high-density patterns with regional morphological variations.

Conclusion: GeoLink-UV provides validated geospatial foundation for urban studies, informal settlement monitoring, and SDG 11 assessments, enabling evidence-based urban planning and revealing nationwide UV patterns.

Abstract: Urban Villages (UVs) represent a distinctive form of high-density informal settlement embedded within China’s rapidly urbanizing cities. Accurate identification of UVs is critical for urban governance, renewal, and sustainable development. But due to the pronounced heterogeneity and diversity of UVs across China’s vast territory, a consistent and reliable nationwide dataset has been lacking. In this work, we present GeoLink-UV, a high-resolution nationwide UV mapping product that clearly delineates the locations and boundaries of UVs in 342 Chinese cities. The dataset is derived from multisource geospatial data, including optical remote sensing images and geo-vector data, and is generated through a foundation model-driven mapping framework designed to address the generalization issues and improve the product quality. A geographically stratified accuracy assessment based on independent samples from 28 cities confirms the reliability and scientific credibility of the nationwide dataset across heterogeneous urban contexts. Based on this nationwide product, we reveal substantial interregional disparities in UV prevalence and spatial configuration. On average, UV areas account for 8 % of built-up land, with marked clustering in central and south China. Building-level analysis further confirms a consistent low-rise, high-density development pattern of UVs nationwide, while highlighting regionally differentiated morphological characteristics. The GeoLink-UV dataset provides an open and systematically validated geospatial foundation for urban studies, informal settlement monitoring, and evidence-based urban renewal planning, and contributes directly to large-scale assessments aligned with Sustainable Development Goal 11. The GeoLink-UV dataset introduced in this article is freely available at https://doi.org/10.5281/zenodo.18688062.

Method: Proposes Zero-Shot Multiple-Instance Learning (ZS-MIL) that uses class-level embeddings from VLM text encoder as starting point for classification layer weights. This replaces random initialization in MIL frameworks for whole-slide image classification.

Result: ZS-MIL demonstrates robustness compared to standard weight initialization techniques in terms of performance and variability in efficient transfer learning few-shot scenarios for subtyping prediction.

Conclusion: Using VLM text encoder embeddings for classifier initialization in MIL frameworks improves few-shot learning performance for WSI classification, addressing limitations of random initialization.

Abstract: Vision language models (VLM) pre-trained on datasets of histopathological image-caption pairs enabled zero-shot slide-level classification. The ability of VLM image encoders to extract discriminative features also opens the door for supervised fine-tuning for whole-slide image (WSI) classification, ideally using few labeled samples. Slide-level prediction frameworks require the incorporation of multiple instance learning (MIL) due to the gigapixel size of the WSI. Following patch-level feature extraction and aggregation, MIL frameworks rely on linear classifiers trained on top of the slide-level aggregated features. Classifier weight initialization has a large influence on Linear Probing performance in efficient transfer learning (ETL) approaches based on few-shot learning. In this work, we propose Zero-Shot Multiple-Instance Learning (ZS-MIL) to address the limitations of random classifier initialization that underperform zero-shot prediction in MIL problems. ZS-MIL uses the class-level embeddings of the VLM text encoder as the classification layer’s starting point to compute each sample’s bag-level probabilities. Through multiple experiments, we demonstrate the robustness of ZS-MIL compared to well-known weight initialization techniques both in terms of performance and variability in an ETL few-shot scenario for subtyping prediction.

Method: Proposes MaskDiME, a training-free diffusion framework that unifies semantic consistency and spatial precision through localized sampling, adaptively focusing on decision-relevant regions while preserving image fidelity.

Result: Achieves over 30x faster inference than baseline methods and comparable or state-of-the-art performance across five benchmark datasets spanning diverse visual domains.

Conclusion: MaskDiME establishes a practical and generalizable solution for efficient counterfactual explanation in visual understanding tasks.

Abstract: Visual counterfactual explanations aim to reveal the minimal semantic modifications that can alter a model’s prediction, providing causal and interpretable insights into deep neural networks. However, existing diffusion-based counterfactual generation methods are often computationally expensive, slow to sample, and imprecise in localizing the modified regions. To address these limitations, we propose MaskDiME, a simple, fast, and effective diffusion framework that unifies semantic consistency and spatial precision through localized sampling. Our approach adaptively focuses on decision-relevant regions to achieve localized and semantically consistent counterfactual generation while preserving high image fidelity. Our training-free framework, MaskDiME, achieves over 30x faster inference than the baseline method and achieves comparable or state-of-the-art performance across five benchmark datasets spanning diverse visual domains, establishing a practical and generalizable solution for efficient counterfactual explanation.

Method: Treats preference alignment as classifier-free guidance (CFG) where a fine-tuned preference model acts as external control during sampling. Decouples preference learning into two modules trained on positive/negative data, forming contrastive guidance by subtracting their predictions (positive minus negative) scaled by user-chosen strength and added to base prediction at each step.

Result: Shows consistent quantitative and qualitative gains on Stable Diffusion 1.5 and Stable Diffusion XL with Pick-a-Pic v2 and HPDv3 datasets.

Conclusion: Proposes a simple method that improves alignment without retraining base models, using contrastive guidance for sharper and controllable preference alignment signals.

Abstract: Aligning large-scale text-to-image diffusion models with nuanced human preferences remains challenging. While direct preference optimization (DPO) is simple and effective, large-scale finetuning often shows a generalization gap. We take inspiration from test-time guidance and cast preference alignment as classifier-free guidance (CFG): a finetuned preference model acts as an external control signal during sampling. Building on this view, we propose a simple method that improves alignment without retraining the base model. To further enhance generalization, we decouple preference learning into two modules trained on positive and negative data, respectively, and form a \emph{contrastive guidance} vector at inference by subtracting their predictions (positive minus negative), scaled by a user-chosen strength and added to the base prediction at each step. This yields a sharper and controllable alignment signal. We evaluate on Stable Diffusion 1.5 and Stable Diffusion XL with Pick-a-Pic v2 and HPDv3, showing consistent quantitative and qualitative gains.

Method: Proposes LMP with dual branches: 1) Visual-guided branch that constructs class-level prototypes from support RoIs and generates hard-negative prototypes via jittered boxes, injecting these into detection pipeline; 2) Text-guided branch that preserves open-vocabulary semantics. Both branches are trained jointly and ensembled at inference.

Result: Achieves state-of-the-art or highly competitive mAP on six cross-domain benchmark datasets across standard 1/5/10-shot settings.

Conclusion: Combining semantic abstraction from text with domain-adaptive details from visual exemplars enables effective cross-domain few-shot object detection, outperforming methods that rely solely on text prompts.

Abstract: Cross-Domain Few-Shot Object Detection (CD-FSOD) aims to detect novel classes in unseen target domains given only a few labeled examples. While open-vocabulary detectors built on vision-language models (VLMs) transfer well, they depend almost entirely on text prompts, which encode domain-invariant semantics but miss domain-specific visual information needed for precise localization under few-shot supervision. We propose a dual-branch detector that Learns Multi-modal Prototypes, dubbed LMP, by coupling textual guidance with visual exemplars drawn from the target domain. A Visual Prototype Construction module aggregates class-level prototypes from support RoIs and dynamically generates hard-negative prototypes in query images via jittered boxes, capturing distractors and visually similar backgrounds. In the visual-guided branch, we inject these prototypes into the detection pipeline with components mirrored from the text branch as the starting point for training, while a parallel text-guided branch preserves open-vocabulary semantics. The branches are trained jointly and ensembled at inference by combining semantic abstraction with domain-adaptive details. On six cross-domain benchmark datasets and standard 1/5/10-shot settings, our method achieves state-of-the-art or highly competitive mAP.

Method: HeRO uses dense semantics lifting to fuse DINOv2’s geometry-sensitive features with Stable Diffusion’s globally coherent correspondences, creating hierarchical semantic fields (global and local). A permutation-invariant hierarchical conditioning module then conditions a diffusion denoiser on these fields to generate coherent control policies.

Result: HeRO achieves state-of-the-art performance, improving success on Place Dual Shoes by 12.3% and averaging 6.5% gains across six challenging pose-aware manipulation tasks.

Conclusion: The hierarchical semantic field approach successfully couples geometry and semantics for robotic manipulation, enabling pose-aware manipulation through explicit part-level understanding while maintaining geometric awareness.

Abstract: Imitation learning for robotic manipulation has progressed from 2D image policies to 3D representations that explicitly encode geometry. Yet purely geometric policies often lack explicit part-level semantics, which are critical for pose-aware manipulation (e.g., distinguishing a shoe’s toe from heel). In this paper, we present HeRO, a diffusion-based policy that couples geometry and semantics via hierarchical semantic fields. HeRO employs dense semantics lifting to fuse discriminative, geometry-sensitive features from DINOv2 with the smooth, globally coherent correspondences from Stable Diffusion, yielding dense features that are both fine-grained and spatially consistent. These features are processed and partitioned to construct a global field and a set of local fields. A hierarchical conditioning module conditions the generative denoiser on global and local fields using permutation-invariant network architecture, thereby avoiding order-sensitive bias and producing a coherent control policy for pose-aware manipulation. In various tests, HeRO establishes a new state-of-the-art, improving success on Place Dual Shoes by 12.3% and averaging 6.5% gains across six challenging pose-aware tasks. Code is available at https://github.com/Chongyang-99/HeRO.

Method: Proposes RobSelf with two key techniques: 1) misalignment-aware feature translator for weakly-supervised alignment, and 2) content-aware reference filter for reference-based discriminative self-enhancement.

Result: Achieves state-of-the-art performance and superior efficiency across various tasks, and introduces RealMisSR dataset for advancing research.

Conclusion: RobSelf enables effective cross-modal super-resolution on misaligned real-world data through self-supervised online optimization without requiring training data or pre-alignment.

Abstract: We study cross-modal super-resolution (SR) on real-world misaligned data, where only a limited number of low-resolution (LR) source and high-resolution (HR) guide image pairs with complex spatial misalignments are available. To address this challenge, we propose RobSelf–a fully self-supervised model that is optimized online, requiring no training data, ground-truth supervision, or pre-alignment. RobSelf features two key techniques: a misalignment-aware feature translator and a content-aware reference filter. The translator reformulates unsupervised cross-modal and cross-resolution alignment as a weakly-supervised, misalignment-aware translation subtask, producing an aligned guide feature with inherent redundancy. Guided by this feature, the filter performs reference-based discriminative self-enhancement on the source, enabling SR predictions with high resolution and high fidelity. Across a variety of tasks, we demonstrate that RobSelf achieves state-of-the-art performance and superior efficiency. Additionally, we introduce a real-world dataset, RealMisSR, to advance research on this topic. Dataset and code: https://github.com/palmdong/RobSelf.

Method: Two key components: 1) Dynamic spatial-temporal state propagation autoregressive model (STAR) that divides tokens by timesteps, propagates spatial-temporal states from previous groups, and uses them to guide next timestep generation via a spatial-temporal container. 2) 4D VQ-VAE that encodes 4D structure into discrete space and decodes tokens into temporally coherent dynamic 3D Gaussians.

Result: 4DSTAR generates spatial-temporal consistent 4D objects and achieves performance competitive with diffusion models.

Conclusion: The proposed 4DSTAR framework effectively addresses spatial-temporal inconsistency in 4D object generation through state propagation and autoregressive modeling.

Abstract: Generating high-quality 4D objects with spatial-temporal consistency is still formidable. Existing diffusion-based methods often struggle with spatial-temporal inconsistency, as they fail to leverage outputs from all previous timesteps to guide the generation at the current timestep. Therefore, we propose a Spatial-Temporal State Propagation AutoRegressive Model (4DSTAR), which generates 4D objects maintaining temporal-spatial consistency. 4DSTAR formulates the generation problem as the prediction of tokens that represent the 4D object. It consists of two key components: (1) The dynamic spatial-temporal state propagation autoregressive model (STAR) is proposed, which achieves spatial-temporal consistent generation. Unlike standard autoregressive models, STAR divides prediction tokens into groups based on timesteps. It models long-term dependencies by propagating spatial-temporal states from previous groups and utilizes these dependencies to guide generation at the next timestep. To this end, a spatial-temporal container is proposed, which dynamically updating the effective spatial-temporal state features from all historical groups, then updated features serve as conditional features to guide the prediction of the next token group. (2) The 4D VQ-VAE is proposed, which implicitly encodes the 4D structure into discrete space and decodes the discrete tokens predicted by STAR into temporally coherent dynamic 3D Gaussians. Experiments demonstrate that 4DSTAR generates spatial-temporal consistent 4D objects, and achieves performance competitive with diffusion models.

Method: IDPERTURB perturbs identity embeddings within a constrained angular region on the unit hyper-sphere, creating diverse embeddings without modifying the underlying generative model. These perturbed embeddings condition a pre-trained diffusion model to generate varied yet identity-coherent face images.

Result: Training face recognition systems on datasets generated with IDPERTURB yields improved performance across multiple FR benchmarks compared to existing synthetic data generation approaches.

Conclusion: IDPERTURB provides a simple yet effective way to enhance diversity in synthetic face generation, addressing the intra-class variation limitation of current identity-conditional diffusion models for better face recognition training.

Abstract: Synthetic data has emerged as a practical alternative to authentic face datasets for training face recognition (FR) systems, especially as privacy and legal concerns increasingly restrict the use of real biometric data. Recent advances in identity-conditional diffusion models have enabled the generation of photorealistic and identity-consistent face images. However, many of these models suffer from limited intra-class variation, an essential property for training robust and generalizable FR models. In this work, we propose IDPERTURB, a simple yet effective geometric-driven sampling strategy to enhance diversity in synthetic face generation. IDPERTURB perturbs identity embeddings within a constrained angular region of the unit hyper-sphere, producing a diverse set of embeddings without modifying the underlying generative model. Each perturbed embedding serves as a conditioning vector for a pre-trained diffusion model, enabling the synthesis of visually varied yet identity-coherent face images suitable for training generalizable FR systems. Empirical results demonstrate that training FR on datasets generated using IDPERTURB yields improved performance across multiple FR benchmarks, compared to existing synthetic data generation approaches.

Method: Proposes CLAP - a lightweight autoencoder using separable convolutional layers in encoder-decoder blocks with sigmoid gating for feature refinement. Feature maps from encoder and decoder are combined for rich representation before classification.

Result: Achieved improved or competitive accuracies on three public plant datasets (Integrated Plant Disease, Groundnut, CCMT) with only 5 million parameters, 20ms training time, and 1ms inference time per image.

Conclusion: CLAP provides an efficient solution for plant disease classification that balances performance with computational efficiency, making it suitable for real-world field applications.

Abstract: Convolutional neural networks have remarkably progressed the performance of distinguishing plant diseases, severity grading, and nutrition deficiency prediction using leaf images. However, these tasks become more challenging in a realistic in-situ field condition. Often, a traditional machine learning model may fail to capture and interpret discriminative characteristics of plant health, growth and diseases due to subtle variations within leaf subcategories. A few deep learning methods have used additional preprocessing stages or network modules to address the problem, whereas several other methods have utilized pre-trained backbone CNNs, most of which are computationally intensive. Therefore, to address the challenge, we propose a lightweight autoencoder using separable convolutional layers in its encoder decoder blocks. A sigmoid gating is applied for refining the prowess of the encoders feature discriminability, which is improved further by the decoder. Finally, the feature maps of the encoder decoder are combined for rich feature representation before classification. The proposed Convolutional Lightweight Autoencoder for Plant disease classification, called CLAP, has been experimented on three public plant datasets consisting of cassava, tomato, maize, groundnut, grapes, etc. for determining plant health conditions. The CLAP has attained improved or competitive accuracies on the Integrated Plant Disease, Groundnut, and CCMT datasets balancing a tradeoff between the performance, and little computational cost requiring 5 million parameters. The training time is 20 milliseconds and inference time is 1 ms per image.

Method: Two-stage closed-loop process: 1) Dual-Stream Segmentation Network fuses original image with MAE reconstruction residuals for coarse localization; 2) Task-Adaptive Prior Injection converts coarse prediction into prompts to steer MAE decoder and amplify reconstruction failures in suspicious regions for precise refinement.

Result: Achieves average improvement of 6.5% in IoU and 8.1% in F1-score over second-best method on four diffusion-based inpainting benchmarks, with strong generalization to traditional manipulation types.

Conclusion: IFA-Net demonstrates effectiveness of using frozen MAE as universal realness prior for forgery localization, shifting paradigm from learning forgery patterns to modeling natural image manifold, enabling robust detection of novel manipulations.

Abstract: The proliferation of highly realistic AI-generated images poses critical challenges for digital forensics, demanding precise pixel-level localization of manipulated regions. Existing methods predominantly learn discriminative patterns of specific forgeries and often struggle with novel manipulations as editing techniques continue to evolve. We propose the Iterative Forgery Amplifier Network (IFA-Net), which shifts from learning “what is fake” to modeling “what is real”. Grounded in the principle that all manipulations deviate from the natural image manifold, IFA-Net leverages a frozen Masked Autoencoder (MAE) pretrained on real images as a universal realness prior. Our framework operates through a two-stage closed-loop process: an initial Dual-Stream Segmentation Network (DSSN) fuses the original image with MAE reconstruction residuals for coarse localization, followed by a Task-Adaptive Prior Injection (TAPI) module that converts this coarse prediction into guiding prompts to steer the MAE decoder and amplify reconstruction failures in suspicious regions for precise refinement. Extensive experiments on four diffusion-based inpainting benchmarks show that IFA-Net achieves an average improvement of 6.5% in IoU and 8.1% in F1-score over the second-best method, while demonstrating strong generalization to traditional manipulation types.

Method: Constructs tracking trigger images as learnable tensors using adversarial optimization with dual-injection: 1) textual consistency between auxiliary MLLM output and target ownership text, 2) semantic-level CLIP feature alignment. Includes adversarial training with auxiliary model to resist ownership text generation for robustness.

Result: Extensive experiments show effectiveness in tracking model lineage under various fine-tuning and domain-shift scenarios.

Conclusion: Proposed framework enables verifiable ownership information embedding in MLLMs through copyright triggers that work exclusively in fine-tuned derivatives.

Abstract: With the rapid deployment and widespread adoption of multimodal large language models (MLLMs), disputes regarding model version attribution and ownership have become increasingly frequent, raising significant concerns about intellectual property protection. In this paper, we propose a framework for generating copyright triggers for MLLMs, enabling model publishers to embed verifiable ownership information into the model. The goal is to construct trigger images that elicit ownership-related textual responses exclusively in fine-tuned derivatives of the original model, while remaining inert in other non-derivative models. Our method constructs a tracking trigger image by treating the image as a learnable tensor, performing adversarial optimization with dual-injection of ownership-relevant semantic information. The first injection is achieved by enforcing textual consistency between the output of an auxiliary MLLM and a predefined ownership-relevant target text; the consistency loss is backpropagated to inject this ownership-related information into the image. The second injection is performed at the semantic-level by minimizing the distance between the CLIP features of the image and those of the target text. Furthermore, we introduce an additional adversarial training stage involving the auxiliary model derived from the original model itself. This auxiliary model is specifically trained to resist generating ownership-relevant target text, thereby enhancing robustness in heavily fine-tuned derivative models. Extensive experiments demonstrate the effectiveness of our dual-injection approach in tracking model lineage under various fine-tuning and domain-shift scenarios.

Method: Two-stage approach: (1) Vision-only redundancy-aware compression of vision encoder output into information-preserving tokens, (2) Layer-wise, salient text-guided dropping of visual tokens within the language backbone to progressively prune less informative tokens.

Result: On LLaVA-1.5-7B: maintains >99% baseline accuracy with 67% fewer tokens, >97% accuracy at 89% reduction. With training: 99.7% accuracy at 67% reduction, 97.6% at 89% reduction. On Video-LLaVA-7B: surpasses baseline with >100% accuracy at 53.1% reduction, 97.6% accuracy at 93.4% reduction.

Conclusion: DUET-VLM enables robust adaptation to reduced visual input without sacrificing accuracy, producing compact yet semantically rich representations within the same computational budget, outperforming prior state-of-the-art visual token reduction methods.

Abstract: Vision-language models (VLMs) have achieved remarkable multimodal understanding and reasoning capabilities, yet remain computationally expensive due to dense visual tokenization. Existing efficiency approaches either merge redundant visual tokens or drop them progressively in language backbone, often trading accuracy for speed. In this work, we propose DUET-VLM, a versatile plug-and-play dual compression framework that consists of (a) vision-only redundancy aware compression of vision encoder’s output into information-preserving tokens, followed by (b) layer-wise, salient text-guided dropping of visual tokens within the language backbone to progressively prune less informative tokens. This coordinated token management enables aggressive compression while retaining critical semantics. On LLaVA-1.5-7B, our approach maintains over 99% of baseline accuracy with 67% fewer tokens, and still retains >97% even at 89% reduction. With this dual-stage compression during training, it achieves 99.7% accuracy at 67% and 97.6% at 89%, surpassing prior SoTA visual token reduction methods across multiple benchmarks. When integrated into Video-LLaVA-7B, it even surpasses the baseline – achieving >100% accuracy with a substantial 53.1% token reduction and retaining 97.6% accuracy under an extreme 93.4% setting. These results highlight end-to-end training with DUET-VLM, enabling robust adaptation to reduced visual (image/video) input without sacrificing accuracy, producing compact yet semantically rich representations within the same computational budget. Our code is available at https://github.com/AMD-AGI/DUET-VLM.

Method: Introduces OVDG-SS benchmark for autonomous driving and proposes S2-Corr (state-space-driven text-image correlation refinement) to mitigate domain-induced distortions in pre-trained vision-language models.

Result: Extensive experiments show superior cross-domain performance and efficiency compared to existing open-vocabulary semantic segmentation approaches on the constructed benchmark.

Conclusion: OVDG-SS is a crucial setting for real-world applications, and S2-Corr effectively addresses domain-induced distortions in text-image correlations, enabling better generalization to both unseen domains and categories.

Abstract: Domain Generalization in Semantic Segmentation (DG-SS) aims to enable segmentation models to perform robustly in unseen environments. However, conventional DG-SS methods are restricted to a fixed set of known categories, limiting their applicability in open-world scenarios. Recent progress in Vision-Language Models (VLMs) has advanced Open-Vocabulary Semantic Segmentation (OV-SS) by enabling models to recognize a broader range of concepts. Yet, these models remain sensitive to domain shifts and struggle to maintain robustness when deployed in unseen environments, a challenge that is particularly severe in urban-driving scenarios. To bridge this gap, we introduce Open-Vocabulary Domain Generalization in Semantic Segmentation (OVDG-SS), a new setting that jointly addresses unseen domains and unseen categories. We introduce the first benchmark for OVDG-SS in autonomous driving, addressing a previously unexplored problem and covering both synthetic-to-real and real-to-real generalization across diverse unseen domains and unseen categories. In OVDG-SS, we observe that domain shifts often distort text-image correlations in pre-trained VLMs, which hinders the performance of OV-SS models. To tackle this challenge, we propose S2-Corr, a state-space-driven text-image correlation refinement mechanism that mitigates domain-induced distortions and produces more consistent text-image correlations under distribution changes. Extensive experiments on our constructed benchmark demonstrate that the proposed method achieves superior cross-domain performance and efficiency compared to existing OV-SS approaches.

Method: Joint optimization over all layers and inter-block dependencies without labeled data, scaling with sample count and completing quickly on single GPU. Introduces data-free calibration using Stable Diffusion Turbo with learned multi-mode prompts to generate diverse, label-free samples.

Result: Achieves state-of-the-art W4A4 and W3A3 accuracies on ImageNet, and first PTQ results maintaining strong accuracy on ViT, DeiT, and Swin-T models under extremely low-bit settings (W1.58A8). Data-free approach performs on par with real-data ImageNet calibration.

Conclusion: The framework enables efficient Vision Transformer deployment on edge devices through joint quantization and data-free calibration, demonstrating strong performance even in extremely low-bit regimes.

Abstract: We present a framework for end-to-end joint quantization of Vision Transformers trained on ImageNet for the purpose of image classification. Unlike prior post-training or block-wise reconstruction methods, we jointly optimize over the entire set of all layers and inter-block dependencies without any labeled data, scaling effectively with the number of samples and completing in just one hour on a single GPU for ViT-small. We achieve state-of-the-art W4A4 and W3A3 accuracies on ImageNet and, to the best of our knowledge, the first PTQ results that maintain strong accuracy on ViT, DeiT, and Swin-T models under extremely low-bit settings (W1.58A8), demonstrating the potential for efficient edge deployment. Furthermore, we introduce a data-free calibration strategy that synthesizes diverse, label-free samples using Stable Diffusion Turbo guided by learned multi-mode prompts. By encouraging diversity in both the learned prompt embeddings and the generated image features, our data-free approach achieves performance on par with real-data ImageNet calibration and surpasses simple text-prompt baselines such as “a photo of ”.

Method: Proposes Similarity-as-Evidence (SaE) framework with Similarity Evidence Head that reinterprets similarity vectors as evidence and parameterizes Dirichlet distributions over labels to quantify uncertainty (vacuity and dissonance). Uses dual-factor acquisition strategy prioritizing high-vacuity samples early and high-dissonance samples later.

Result: Achieves state-of-the-art macro-averaged accuracy of 82.57% on ten medical imaging datasets with 20% label budget. On BTMRI dataset, achieves superior calibration with negative log-likelihood of 0.425.

Conclusion: SaE effectively addresses overconfidence in VLMs by quantifying uncertainty through Dirichlet distributions, enabling more efficient active learning sample selection in medical imaging with clinically interpretable rationales.

Abstract: Active Learning (AL) reduces annotation costs in medical imaging by selecting only the most informative samples for labeling, but suffers from cold-start when labeled data are scarce. Vision-Language Models (VLMs) address the cold-start problem via zero-shot predictions, yet their temperature-scaled softmax outputs treat text-image similarities as deterministic scores while ignoring inherent uncertainty, leading to overconfidence. This overconfidence misleads sample selection, wasting annotation budgets on uninformative cases. To overcome these limitations, the Similarity-as-Evidence (SaE) framework calibrates text-image similarities by introducing a Similarity Evidence Head (SEH), which reinterprets the similarity vector as evidence and parameterizes a Dirichlet distribution over labels. In contrast to a standard softmax that enforces confident predictions even under weak signals, the Dirichlet formulation explicitly quantifies lack of evidence (vacuity) and conflicting evidence (dissonance), thereby mitigating overconfidence caused by rigid softmax normalization. Building on this, SaE employs a dual-factor acquisition strategy: high-vacuity samples (e.g., rare diseases) are prioritized in early rounds to ensure coverage, while high-dissonance samples (e.g., ambiguous diagnoses) are prioritized later to refine boundaries, providing clinically interpretable selection rationales. Experiments on ten public medical imaging datasets with a 20% label budget show that SaE attains state-of-the-art macro-averaged accuracy of 82.57%. On the representative BTMRI dataset, SaE also achieves superior calibration, with a negative log-likelihood (NLL) of 0.425.

Method: Develops MM2D3D with two key techniques: 1) Cross-modal guided filtering uses camera images to constrain intermediate 2D predictions with dense semantic relations, and 2) Dynamic cross pseudo supervision encourages 2D predictions to emulate dense semantic distributions from camera images.

Result: The model achieves dense and accurate intermediate 2D semantic predictions, which effectively enhances final 3D accuracy. Comparisons show superior performance in both 2D and 3D spaces compared to previous methods.

Conclusion: Leveraging camera images as auxiliary data through cross-modal techniques successfully addresses sparsity issues in LiDAR-based segmentation, improving both intermediate 2D predictions and final 3D segmentation accuracy.

Abstract: Semantic segmentation of 3D LiDAR point clouds is important in urban remote sensing for understanding real-world street environments. This task, by projecting LiDAR point clouds and 3D semantic labels as sparse maps, can be reformulated as a 2D problem. However, the intrinsic sparsity of the projected LiDAR and label maps can result in sparse and inaccurate intermediate 2D semantic predictions, which in return limits the final 3D accuracy. To address this issue, we enhance this task by shaping dense and accurate 2D predictions. Specifically, we develop a multi-modal segmentation model, MM2D3D. By leveraging camera images as auxiliary data, we introduce cross-modal guided filtering to overcome label map sparsity by constraining intermediate 2D semantic predictions with dense semantic relations derived from the camera images; and we introduce dynamic cross pseudo supervision to overcome LiDAR map sparsity by encouraging the 2D predictions to emulate the dense distribution of the semantic predictions from the camera images. Experiments show that our techniques enable our model to achieve intermediate 2D semantic predictions with dense distribution and higher accuracy, which effectively enhances the final 3D accuracy. Comparisons with previous methods demonstrate our superior performance in both 2D and 3D spaces.

Method: Represent motion with continuous differentiable B-spline curves using a closed-form, Laplacian-regularized B-spline solver to compress variable-length sequences into compact representations. Includes normal-fusion strategy for shape adherence and correspondence-aware/local-rigidity losses for motion restoration quality. Trained on BIMO dataset with diverse 3D motion sequences and high-quality text annotations.

Result: BiMotion generates more expressive, higher-quality, and better prompt-aligned motions than existing state-of-the-art methods while achieving faster generation.

Conclusion: Continuous B-spline representations enable more effective motion generation without modifying underlying generative model capabilities, addressing limitations of discrete frame-wise approaches.

Abstract: Text-guided dynamic 3D character generation has advanced rapidly, yet producing high-quality motion that faithfully reflects rich textual descriptions remains challenging. Existing methods tend to generate limited sub-actions or incoherent motion due to fixed-length temporal inputs and discrete frame-wise representations that fail to capture rich motion semantics. We address these limitations by representing motion with continuous differentiable B-spline curves, enabling more effective motion generation without modifying the capabilities of the underlying generative model. Specifically, our closed-form, Laplacian-regularized B-spline solver efficiently compresses variable-length motion sequences into compact representations with a fixed number of control points. Further, we introduce a normal-fusion strategy for input shape adherence along with correspondence-aware and local-rigidity losses for motion-restoration quality. To train our model, we collate BIMO, a new dataset containing diverse variable-length 3D motion sequences with rich, high-quality text annotations. Extensive evaluations show that our feed-forward framework BiMotion generates more expressive, higher-quality, and better prompt-aligned motions than existing state-of-the-art methods, while also achieving faster generation. Our project page is at: https://wangmiaowei.github.io/BiMotion.github.io/.

Method: Uses structure-level disentanglement with separate content and style channels. Content comes from SimSun-style templates concatenated with noisy latent features, while style features from CLIP are integrated via cross-attention. Includes Background Noise Removal module and parameter-efficient fine-tuning strategy that updates only style-related modules.

Result: Achieves significantly higher style fidelity while maintaining comparable content accuracy to state-of-the-art methods. Introduces Grey and OCR metrics for evaluating content quality.

Conclusion: SLD-Font provides effective structure-level disentanglement for Chinese font generation, enabling better style adaptation without content overfitting through theoretical validation and practical implementation.

Abstract: Few-shot Chinese font generation aims to synthesize new characters in a target style using only a handful of reference images. Achieving accurate content rendering and faithful style transfer requires effective disentanglement between content and style. However, existing approaches achieve only feature-level disentanglement, allowing the generator to re-entangle these features, leading to content distortion and degraded style fidelity. We propose the Structure-Level Disentangled Diffusion Model (SLD-Font), which receives content and style information from two separate channels. SimSun-style images are used as content templates and concatenated with noisy latent features as the input. Style features extracted by a CLIP model from target-style images are integrated via cross-attention. Additionally, we train a Background Noise Removal module in the pixel space to remove background noise in complex stroke regions. Based on theoretical validation of disentanglement effectiveness, we introduce a parameter-efficient fine-tuning strategy that updates only the style-related modules. This allows the model to better adapt to new styles while avoiding overfitting to the reference images’ content. We further introduce the Grey and OCR metrics to evaluate the content quality of generated characters. Experimental results show that SLD-Font achieves significantly higher style fidelity while maintaining comparable content accuracy to existing state-of-the-art methods.

Method: Multimodal large language model-based framework integrating discriminative features from both RGB spatial and frequency domains via cross-attention fusion module, enabling accurate detection with explicit cross-domain explanations.

Result: Outperforms state-of-the-art methods in detection performance and interpretability across both spatial and frequency domains, validated on FSE-Set dataset.

Conclusion: FOCA effectively addresses key limitations in image forgery detection by leveraging multimodal features and providing interpretable explanations, advancing media verification capabilities.

Abstract: Advances in image tampering techniques, particularly generative models, pose significant challenges to media verification, digital forensics, and public trust. Existing image forgery detection and localization (IFDL) methods suffer from two key limitations: over-reliance on semantic content while neglecting textural cues, and limited interpretability of subtle low-level tampering traces. To address these issues, we propose FOCA, a multimodal large language model-based framework that integrates discriminative features from both the RGB spatial and frequency domains via a cross-attention fusion module. This design enables accurate forgery detection and localization while providing explicit, human-interpretable cross-domain explanations. We further introduce FSE-Set, a large-scale dataset with diverse authentic and tampered images, pixel-level masks, and dual-domain annotations. Extensive experiments show that FOCA outperforms state-of-the-art methods in detection performance and interpretability across both spatial and frequency domains.

Method: Uses a multi-view tokenizer to encode scene information from many context views into a small set of permutation-invariant tokens. A light-weight rectified flow decoder then renders novel views from these tokens.

Result: Achieves 1-3 orders of magnitude stronger compression than other representations while maintaining state-of-the-art reconstruction quality. Enables novel view rendering from deviating trajectories and handles uncertainty gracefully. Allows scene generation in 5 seconds with better quality-speed trade-off.

Conclusion: SceneTok provides a highly efficient, compressed representation for 3D scenes that enables high-quality reconstruction, novel view synthesis, and fast scene generation, representing a significant advancement in 3D scene representation paradigms.

Abstract: We present SceneTok, a novel tokenizer for encoding view sets of scenes into a compressed and diffusable set of unstructured tokens. Existing approaches for 3D scene representation and generation commonly use 3D data structures or view-aligned fields. In contrast, we introduce the first method that encodes scene information into a small set of permutation-invariant tokens that is disentangled from the spatial grid. The scene tokens are predicted by a multi-view tokenizer given many context views and rendered into novel views by employing a light-weight rectified flow decoder. We show that the compression is 1-3 orders of magnitude stronger than for other representations while still reaching state-of-the-art reconstruction quality. Further, our representation can be rendered from novel trajectories, including ones deviating from the input trajectory, and we show that the decoder gracefully handles uncertainty. Finally, the highly-compressed set of unstructured latent scene tokens enables simple and efficient scene generation in 5 seconds, achieving a much better quality-speed trade-off than previous paradigms.

Method: Uses physics-informed 3D dynamic convex radiance fields with physically grounded convex primitives governed by continuum mechanics. Introduces boundary-driven dynamic convex representation for non-uniform deformation, and reduced-order convex simulation using neural skinning eigenmodes as deformation bases with time-varying reduced degrees of freedom.

Result: Achieves high-fidelity reconstruction of geometry, appearance, and physical properties from videos, outperforming existing methods. Provides compact, gap-free volumetric coverage enhancing both geometric efficiency and simulation fidelity.

Conclusion: PhysConvex successfully unifies visual rendering and physical simulation for dynamic 3D scenes, addressing limitations of existing neural representations by incorporating physics-based constraints and efficient simulation techniques.

Abstract: Reconstructing and simulating dynamic 3D scenes with both visual realism and physical consistency remains a fundamental challenge. Existing neural representations, such as NeRFs and 3DGS, excel in appearance reconstruction but struggle to capture complex material deformation and dynamics. We propose PhysConvex, a Physics-informed 3D Dynamic Convex Radiance Field that unifies visual rendering and physical simulation. PhysConvex represents deformable radiance fields using physically grounded convex primitives governed by continuum mechanics. We introduce a boundary-driven dynamic convex representation that models deformation through vertex and surface dynamics, capturing spatially adaptive, non-uniform deformation, and evolving boundaries. To efficiently simulate complex geometries and heterogeneous materials, we further develop a reduced-order convex simulation that advects dynamic convex fields using neural skinning eigenmodes as shape- and material-aware deformation bases with time-varying reduced DOFs under Newtonian dynamics. Convex dynamics also offers compact, gap-free volumetric coverage, enhancing both geometric efficiency and simulation fidelity. Experiments demonstrate that PhysConvex achieves high-fidelity reconstruction of geometry, appearance, and physical properties from videos, outperforming existing methods.

Method: Two complementary networks: Sparse World Network (SWNet) constructs trajectory-conditioned sparse worlds simulating future behaviors of critical agents; Fine-grained Reasoning Network (FRNet) evaluates agent-specific collision risks and temporal adherence to drivable regions.

Result: State-of-the-art performance: 91.6 PDMS and 87.5 EPDMS on NAVSIM with only 0.5% collisions; 66.8% driving score on Bench2Drive.

Conclusion: SafeDrive demonstrates that explicit, interpretable safety reasoning can be effectively integrated into end-to-end autonomous driving frameworks, achieving strong performance while maintaining safety.

Abstract: The end-to-end (E2E) paradigm, which maps sensor inputs directly to driving decisions, has recently attracted significant attention due to its unified modeling capability and scalability. However, ensuring safety in this unified framework remains one of the most critical challenges. In this work, we propose SafeDrive, an E2E planning framework designed to perform explicit and interpretable safety reasoning through a trajectory-conditioned Sparse World Model. SafeDrive comprises two complementary networks: the Sparse World Network (SWNet) and the Fine-grained Reasoning Network (FRNet). SWNet constructs trajectory-conditioned sparse worlds that simulate the future behaviors of critical dynamic agents and road entities, providing interaction-centric representations for downstream reasoning. FRNet then evaluates agent-specific collision risks and temporal adherence to drivable regions, enabling precise identification of safety-critical events across future timesteps. SafeDrive achieves state-of-the-art performance on both open-loop and closed-loop benchmarks. On NAVSIM, it records a PDMS of 91.6 and an EPDMS of 87.5, with only 61 collisions out of 12,146 scenarios (0.5%). On Bench2Drive, SafeDrive attains a 66.8% driving score.

Method: Two approaches: 1) NSVQ propagates encoder drift to non-selected codes through kernel-based updates, and 2) TransVQ uses a lightweight transformer mapping to adaptively transform the entire codebook while preserving convergence to k-means.

Result: Experiments on CelebA-HQ show both methods achieve near-complete codebook utilization and superior reconstruction quality compared to baseline VQ variants.

Conclusion: Provides principled solutions to codebook collapse in VQ, offering scalable foundations for future VQ-based generative models.

Abstract: Vector Quantization (VQ) underpins many modern generative frameworks such as VQ-VAE, VQ-GAN, and latent diffusion models. Yet, it suffers from the persistent problem of codebook collapse, where a large fraction of code vectors remains unused during training. This work provides a new theoretical explanation by identifying the nonstationary nature of encoder updates as the fundamental cause of this phenomenon. We show that as the encoder drifts, unselected code vectors fail to receive updates and gradually become inactive. To address this, we propose two new methods: Non-Stationary Vector Quantization (NSVQ), which propagates encoder drift to non-selected codes through a kernel-based rule, and Transformer-based Vector Quantization (TransVQ), which employs a lightweight mapping to adaptively transform the entire codebook while preserving convergence to the k-means solution. Experiments on the CelebA-HQ dataset demonstrate that both methods achieve near-complete codebook utilization and superior reconstruction quality compared to baseline VQ variants, providing a principled and scalable foundation for future VQ-based generative models. The code is available at: https://github.com/CAIR- LAB- WFUSM/NSVQ-TransVQ.git

Method: Developed a three-tier progressive system (BASE, MEDIO, AVANZATO) scaling from 5% to 95% practitioner control, with modular label architecture (7 core + 5 optional components), decision trees for routing, and systematic documentation of model limitations and workarounds. Validated through 850 API predictions across 4,800 generated images spanning six professional domains.

Result: Achieved 91% Mandatory compliance and 94% Prohibitions compliance across 621 structured prompts, demonstrated substantially higher inter-generation coherence, validated by 40 independent practitioners, and showed >95% first-generation compliance for spatial/typographical control in information design applications.

Conclusion: SCHEMA provides an effective structured prompt engineering methodology that significantly improves control, consistency, and reliability in multimodal image generation, particularly for professional applications requiring predictable outputs.

Abstract: This paper presents SCHEMA (Structured Components for Harmonized Engineered Modular Architecture), a structured prompt engineering methodology specifically developed for Google Gemini 3 Pro Image. Unlike generic prompt guidelines or model-agnostic tips, SCHEMA is an engineered framework built on systematic professional practice encompassing 850 verified API predictions within an estimated corpus of approximately 4,800 generated images, spanning six professional domains: real estate photography, commercial product photography, editorial content, storyboards, commercial campaigns, and information design. The methodology introduces a three-tier progressive system (BASE, MEDIO, AVANZATO) that scales practitioner control from exploratory (approximately 5%) to directive (approximately 95%), a modular label architecture with 7 core and 5 optional structured components, a decision tree with explicit routing rules to alternative tools, and systematically documented model limitations with corresponding workarounds. Key findings include an observed 91% Mandatory compliance rate and 94% Prohibitions compliance rate across 621 structured prompts, a comparative batch consistency test demonstrating substantially higher inter-generation coherence for structured prompts, independent practitioner validation (n=40), and a dedicated Information Design validation demonstrating >95% first-generation compliance for spatial and typographical control across approximately 300 publicly verifiable infographics. Previously published on Zenodo (doi:10.5281/zenodo.18721380).

Method: Proposes Marginalized Bundle Adjustment (MBA), inspired by modern RANSAC estimators, to mitigate MDE error variance by leveraging its density through statistical marginalization techniques.

Result: MBA shows MDE depth maps are sufficiently accurate to yield state-of-the-art or competitive results in SfM and camera relocalization tasks, with robust performance across varying scales from few-frame setups to large multi-view systems with thousands of images.

Conclusion: The method demonstrates significant potential of MDE in multi-view 3D vision by effectively handling depth uncertainty through statistical marginalization.

Abstract: Structure-from-Motion (SfM) is a fundamental 3D vision task for recovering camera parameters and scene geometry from multi-view images. While recent deep learning advances enable accurate Monocular Depth Estimation (MDE) from single images without depending on camera motion, integrating MDE into SfM remains a challenge. Unlike conventional triangulated sparse point clouds, MDE produces dense depth maps with significantly higher error variance. Inspired by modern RANSAC estimators, we propose Marginalized Bundle Adjustment (MBA) to mitigate MDE error variance leveraging its density. With MBA, we show that MDE depth maps are sufficiently accurate to yield SoTA or competitive results in SfM and camera relocalization tasks. Through extensive evaluations, we demonstrate consistently robust performance across varying scales, ranging from few-frame setups to large multi-view systems with thousands of images. Our method highlights the significant potential of MDE in multi-view 3D vision.

Method: Three complementary components: (1) rank-constrained backbone fine-tuning with low-rank projection residuals to decouple content and style subspaces; (2) prompt-guided expert encoder with specialized branches for semantic extension and selective adapter aggregation; (3) training-free, timestep-dependent classifier-free guidance scheme for stable generation.

Result: Significantly improves content-style disentanglement, enables flexible semantic control over LoRA module combinations, and achieves high-fidelity generation without additional retraining overhead.

Conclusion: CRAFT-LoRA effectively addresses limitations in existing LoRA combination techniques for personalized image generation, providing better disentanglement, control, and stability.

Abstract: Personalized image generation requires effectively balancing content fidelity with stylistic consistency when synthesizing images based on text and reference examples. Low-Rank Adaptation (LoRA) offers an efficient personalization approach, with potential for precise control through combining LoRA weights on different concepts. However, existing combination techniques face persistent challenges: entanglement between content and style representations, insufficient guidance for controlling elements’ influence, and unstable weight fusion that often require additional training. We address these limitations through CRAFT-LoRA, with complementary components: (1) rank-constrained backbone fine-tuning that injects low-rank projection residuals to encourage learning decoupled content and style subspaces; (2) a prompt-guided approach featuring an expert encoder with specialized branches that enables semantic extension and precise control through selective adapter aggregation; and (3) a training-free, timestep-dependent classifier-free guidance scheme that enhances generation stability by strategically adjusting noise predictions across diffusion steps. Our method significantly improves content-style disentanglement, enables flexible semantic control over LoRA module combinations, and achieves high-fidelity generation without additional retraining overhead.

Method: DACo employs a Global Commander for high-level strategic planning and a Local Operative for egocentric observation and fine-grained execution, with dynamic subgoal planning and adaptive replanning mechanisms.

Result: Achieves 4.9%, 6.5%, 5.4% absolute improvements over best-performing baselines on R2R, REVERIE, and R4R benchmarks in zero-shot settings, and generalizes across both closed-source (GPT-4o) and open-source (Qwen-VL) backbones.

Conclusion: DACo provides a principled and extensible paradigm for robust long-horizon navigation by disentangling global reasoning from local action, alleviating cognitive overload and improving stability.

Abstract: Vision-and-Language Scene navigation is a fundamental capability for embodied human-AI collaboration, requiring agents to follow natural language instructions to execute coherent action sequences in complex environments. Existing approaches either rely on multiple agents, incurring high coordination and resource costs, or adopt a single-agent paradigm, which overloads the agent with both global planning and local perception, often leading to degraded reasoning and instruction drift in long-horizon settings. To address these issues, we introduce DACo, a planning-grounding decoupled architecture that disentangles global deliberation from local grounding. Concretely, it employs a Global Commander for high-level strategic planning and a Local Operative for egocentric observing and fine-grained execution. By disentangling global reasoning from local action, DACo alleviates cognitive overload and improves long-horizon stability. The framework further integrates dynamic subgoal planning and adaptive replanning to enable structured and resilient navigation. Extensive evaluations on R2R, REVERIE, and R4R demonstrate that DACo achieves 4.9%, 6.5%, 5.4% absolute improvements over the best-performing baselines in zero-shot settings, and generalizes effectively across both closed-source (e.g., GPT-4o) and open-source (e.g., Qwen-VL Series) backbones. DACo provides a principled and extensible paradigm for robust long-horizon navigation. Project page: https://github.com/ChocoWu/DACo

Method: Proposes a YOLOv10-based framework that simultaneously localizes hands and classifies their laterality (left or right) in complex surgical scenes. Uses extensive data augmentation and multi-task detection design to improve robustness against motion blur, lighting variations, and diverse hand appearances. Trained on Trauma THOMPSON Challenge 2025 Task 2 dataset of first-person surgical videos.

Result: Achieves 67% left-hand and 71% right-hand classification accuracy. Model achieves mAP[0.5:0.95] of 0.33 and maintains real-time inference. Distinguishing hands from background remains challenging.

Conclusion: The framework demonstrates potential for intraoperative deployment and establishes foundation for advanced hand-instrument interaction analysis in emergency surgical procedures.

Abstract: Real-time hand tracking in trauma surgery is essential for supporting rapid and precise intraoperative decisions. We propose a YOLOv10-based framework that simultaneously localizes hands and classifies their laterality (left or right) in complex surgical scenes. The model is trained on the Trauma THOMPSON Challenge 2025 Task 2 dataset, consisting of first-person surgical videos with annotated hand bounding boxes. Extensive data augmentation and a multi-task detection design improve robustness against motion blur, lighting variations, and diverse hand appearances. Evaluation demonstrates accurate left-hand (67%) and right-hand (71%) classification, while distinguishing hands from the background remains challenging. The model achieves an $mAP_{[0.5:0.95]}$ of 0.33 and maintains real-time inference, highlighting its potential for intraoperative deployment. This work establishes a foundation for advanced hand-instrument interaction analysis in emergency surgical procedures.

Method: Frame2Freq uses Fast Fourier Transform (FFT) along the time dimension and learns frequency-band specific embeddings that adaptively highlight discriminative frequency ranges, enabling spectral encoding during image-to-video adaptation.

Result: Outperforms prior parameter-efficient fine-tuning (PEFT) methods on five fine-grained activity recognition datasets and surpasses fully fine-tuned models on four of them.

Conclusion: Frequency analysis methods are powerful tools for modeling temporal dynamics in image-to-video transfer, with Frame2Freq demonstrating effective multi-scale motion capture for fine-grained action recognition.

Abstract: Adapting image-pretrained backbones to video typically relies on time-domain adapters tuned to a single temporal scale. Our experiments show that these modules pick up static image cues and very fast flicker changes, while overlooking medium-speed motion. Capturing dynamics across multiple time-scales is, however, crucial for fine-grained temporal analysis (i.e., opening vs. closing bottle). To address this, we introduce Frame2Freq – a family of frequency-aware adapters that perform spectral encoding during image-to-video adaptation of pretrained Vision Foundation Models (VFMs), improving fine-grained action recognition. Frame2Freq uses Fast Fourier Transform (FFT) along time and learns frequency-band specific embeddings that adaptively highlight the most discriminative frequency ranges. Across five fine-grained activity recognition datasets, Frame2Freq outperforms prior PEFT methods and even surpasses fully fine-tuned models on four of them. These results provide encouraging evidence that frequency analysis methods are a powerful tool for modeling temporal dynamics in image-to-video transfer. Code is available at https://github.com/th-nesh/Frame2Freq.

Method: Uses reinforcement learning-based selector with actor-critic framework and budget-aware optimization. Trains lightweight agent end-to-end with reward balancing accuracy and computational cost, plus entropy regularization to prevent premature convergence.

Result: On CCVID: 95.9% Rank-1 accuracy with 92.4% less computation than baselines while improving accuracy by 1.8%. On MEVID: reduces computation by 41.3% while maintaining competitive performance.

Conclusion: IDSelect demonstrates superior efficiency in video-based person recognition by dynamically selecting optimal models per modality, achieving better accuracy-efficiency trade-offs than fixed ensemble approaches.

Abstract: Video-based person recognition achieves robust identification by integrating face, body, and gait. However, current systems waste computational resources by processing all modalities with fixed heavyweight ensembles regardless of input complexity. To address these limitations, we propose IDSelect, a reinforcement learning-based cost-aware selector that chooses one pre-trained model per modality per-sequence to optimize the accuracy-efficiency trade-off. Our key insight is that an input-conditioned selector can discover complementary model choices that surpass fixed ensembles while using substantially fewer resources. IDSelect trains a lightweight agent end-to-end using actor-critic reinforcement learning with budget-aware optimization. The reward balances recognition accuracy with computational cost, while entropy regularization prevents premature convergence. At inference, the policy selects the most probable model per modality and fuses modality-specific similarities for the final score. Extensive experiments on challenging video-based datasets demonstrate IDSelect’s superior efficiency: on CCVID, it achieves 95.9% Rank-1 accuracy with 92.4% less computation than strong baselines while improving accuracy by 1.8%; on MEVID, it reduces computation by 41.3% while maintaining competitive performance.

Method: Introduces Spectral-Evolution-Aware Cache (SeaCache) with a Spectral-Evolution-Aware (SEA) filter that preserves content-relevant components while suppressing noise. Uses SEA-filtered input features to estimate redundancy and create dynamic cache schedules that adapt to content while respecting diffusion model spectral priors.

Result: Extensive experiments on diverse visual generative models show SeaCache achieves state-of-the-art latency-quality trade-offs compared to baselines.

Conclusion: SeaCache provides an effective training-free solution for accelerating diffusion model inference by leveraging spectral evolution patterns, achieving better performance than previous caching strategies.

Abstract: Diffusion models are a strong backbone for visual generation, but their inherently sequential denoising process leads to slow inference. Previous methods accelerate sampling by caching and reusing intermediate outputs based on feature distances between adjacent timesteps. However, existing caching strategies typically rely on raw feature differences that entangle content and noise. This design overlooks spectral evolution, where low-frequency structure appears early and high-frequency detail is refined later. We introduce Spectral-Evolution-Aware Cache (SeaCache), a training-free cache schedule that bases reuse decisions on a spectrally aligned representation. Through theoretical and empirical analysis, we derive a Spectral-Evolution-Aware (SEA) filter that preserves content-relevant components while suppressing noise. Employing SEA-filtered input features to estimate redundancy leads to dynamic schedules that adapt to content while respecting the spectral priors underlying the diffusion model. Extensive experiments on diverse visual generative models and the baselines show that SeaCache achieves state-of-the-art latency-quality trade-offs.

Method: Proposes a conditional binary segmentation framework where object query masks are encoded into latent representations to guide object localization in target videos. Uses cycle-consistency training where predicted masks are projected back to source view to reconstruct original query masks, providing self-supervision without ground-truth annotations. Enables test-time training at inference for optimization.

Result: Achieves state-of-the-art performance on Ego-Exo4D and HANDAL-X benchmarks, demonstrating effectiveness of the optimization objective and test-time training strategy

Conclusion: The proposed framework effectively solves cross-view object correspondence through self-supervised cycle-consistency training and test-time optimization, showing strong performance on challenging benchmarks

Abstract: We study the task of establishing object-level visual correspondence across different viewpoints in videos, focusing on the challenging egocentric-to-exocentric and exocentric-to-egocentric scenarios. We propose a simple yet effective framework based on conditional binary segmentation, where an object query mask is encoded into a latent representation to guide the localization of the corresponding object in a target video. To encourage robust, view-invariant representations, we introduce a cycle-consistency training objective: the predicted mask in the target view is projected back to the source view to reconstruct the original query mask. This bidirectional constraint provides a strong self-supervisory signal without requiring ground-truth annotations and enables test-time training (TTT) at inference. Experiments on the Ego-Exo4D and HANDAL-X benchmarks demonstrate the effectiveness of our optimization objective and TTT strategy, achieving state-of-the-art performance. The code is available at https://github.com/shannany0606/CCMP.

Method: 1) Created Life-Bench - a comprehensive, synthetically generated multimodal benchmark built on simulated user digital footprints with over [number] questions. 2) Proposed LifeGraph - an end-to-end framework that organizes personal context into a knowledge graph to facilitate structured retrieval and reasoning.

Result: Existing methods falter significantly on complex personalized tasks, exposing large performance gaps especially in relational, temporal and aggregative reasoning. LifeGraph helps close this gap by leveraging structured knowledge but advanced personalization tasks remain challenging.

Conclusion: Life-Bench reveals critical gaps in current VLMs’ personalization capabilities, motivating new research. LifeGraph demonstrates a promising direction using structured knowledge organization, but advanced personalization remains an open challenge requiring further investigation.

Abstract: The powerful reasoning of modern Vision Language Models open a new frontier for advanced personalization study. However, progress in this area is critically hampered by the lack of suitable benchmarks. To address this gap, we introduce Life-Bench, a comprehensive, synthetically generated multimodal benchmark built on simulated user digital footprints. Life-Bench features over questions evaluating a wide spectrum of capabilities, from persona understanding to complex reasoning over historical data. These capabilities expand far beyond prior benchmarks, reflecting the critical demands essential for real-world applications. Furthermore, we propose LifeGraph, an end-to-end framework that organizes personal context into a knowledge graph to facilitate structured retrieval and reasoning. Our experiments on Life-Bench reveal that existing methods falter significantly on complex personalized tasks, exposing a large performance headroom, especially in relational, temporal and aggregative reasoning. While LifeGraph closes this gap by leveraging structured knowledge and demonstrates a promising direction, these advanced personalization tasks remain a critical open challenge, motivating new research in this area.

Method: MoBind uses a hierarchical contrastive learning framework that: 1) Aligns IMU signals with skeletal motion sequences rather than raw pixels to filter background noise, 2) Decomposes full-body motion into local body-part trajectories paired with corresponding IMUs for multi-sensor alignment, and 3) Employs a two-level hierarchical contrastive strategy that first aligns token-level temporal segments, then fuses local (body-part) alignment with global motion aggregation.

Result: MoBind consistently outperforms strong baselines on mRi, TotalCapture, and EgoHumans datasets across four tasks: cross-modal retrieval, temporal synchronization, subject/body-part localization, and action recognition. It demonstrates robust fine-grained temporal alignment while preserving coarse semantic consistency across modalities.

Conclusion: MoBind provides an effective framework for learning joint representations between IMU signals and visual motion data, addressing key challenges in cross-modal alignment and enabling multiple practical applications in motion understanding and analysis.

Abstract: We aim to learn a joint representation between inertial measurement unit (IMU) signals and 2D pose sequences extracted from video, enabling accurate cross-modal retrieval, temporal synchronization, subject and body-part localization, and action recognition. To this end, we introduce MoBind, a hierarchical contrastive learning framework designed to address three challenges: (1) filtering out irrelevant visual background, (2) modeling structured multi-sensor IMU configurations, and (3) achieving fine-grained, sub-second temporal alignment. To isolate motion-relevant cues, MoBind aligns IMU signals with skeletal motion sequences rather than raw pixels. We further decompose full-body motion into local body-part trajectories, pairing each with its corresponding IMU to enable semantically grounded multi-sensor alignment. To capture detailed temporal correspondence, MoBind employs a hierarchical contrastive strategy that first aligns token-level temporal segments, then fuses local (body-part) alignment with global (body-wide) motion aggregation. Evaluated on mRi, TotalCapture, and EgoHumans, MoBind consistently outperforms strong baselines across all four tasks, demonstrating robust fine-grained temporal alignment while preserving coarse semantic consistency across modalities. Code is available at https://github.com/bbvisual/ MoBind.

Method: Unpaired histopathology knowledge distillation strategy that trains a micro-US encoder to emulate embedding distribution of pretrained histopathology foundation model conditioned on ISUP grades, requiring no patient-level pairing or image registration.

Result: Increases sensitivity to clinically significant PCa at 60% specificity by 3.5% and improves overall sensitivity at 60% specificity by 1.2% compared to state-of-the-art.

Conclusion: Enables earlier and more dependable cancer risk stratification solely from imaging, advancing clinical feasibility.

Abstract: Purpose: Non-invasive grading of prostate cancer (PCa) from micro-ultrasound (micro-US) could expedite triage and guide biopsies toward the most aggressive regions, yet current models struggle to infer tissue micro-structure at coarse imaging resolutions. Methods: We introduce an unpaired histopathology knowledge-distillation strategy that trains a micro-US encoder to emulate the embedding distribution of a pretrained histopathology foundation model, conditioned on International Society of Urological Pathology (ISUP) grades. Training requires no patient-level pairing or image registration, and histopathology inputs are not used at inference. Results: Compared to the current state of the art, our approach increases sensitivity to clinically significant PCa (csPCa) at 60% specificity by 3.5% and improves overall sensitivity at 60% specificity by 1.2%. Conclusion: By enabling earlier and more dependable cancer risk stratification solely from imaging, our method advances clinical feasibility. Source code will be publicly released upon publication.

Method: TokenTrace embeds secret signatures simultaneously in text prompt embeddings and initial latent noise of diffusion models. For retrieval, uses a query-based module that takes generated image and textual query specifying which concepts to retrieve, enabling disentanglement and independent verification of multiple concepts.

Result: Achieves state-of-the-art performance on both single-concept (object and style) and multi-concept attribution tasks, significantly outperforming existing baselines while maintaining high visual quality and robustness to common transformations.

Conclusion: TokenTrace provides an effective solution for multi-concept attribution in generative AI, addressing limitations of existing watermarking methods for complex compositions while preserving generation quality.

Abstract: Generative AI models pose a significant challenge to intellectual property (IP), as they can replicate unique artistic styles and concepts without attribution. While watermarking offers a potential solution, existing methods often fail in complex scenarios where multiple concepts (e.g., an object and an artistic style) are composed within a single image. These methods struggle to disentangle and attribute each concept individually. In this work, we introduce TokenTrace, a novel proactive watermarking framework for robust, multi-concept attribution. Our method embeds secret signatures into the semantic domain by simultaneously perturbing the text prompt embedding and the initial latent noise that guide the diffusion model’s generation process. For retrieval, we propose a query-based TokenTrace module that takes the generated image and a textual query specifying which concepts need to be retrieved (e.g., a specific object or style) as inputs. This query-based mechanism allows the module to disentangle and independently verify the presence of multiple concepts from a single generated image. Extensive experiments show that our method achieves state-of-the-art performance on both single-concept (object and style) and multi-concept attribution tasks, significantly outperforming existing baselines while maintaining high visual quality and robustness to common transformations.

Method: Proposes DEFNet with: 1) Multitask optimization using scene and distortion type classification as auxiliary tasks, 2) Trustworthy information fusion combining diverse features across sub-regions with local-global fusion, 3) Evidential learning with normal-inverse gamma distribution mixture for uncertainty estimation.

Result: Extensive experiments on synthetic and authentic distortion datasets show effectiveness and robustness. Additional evaluation highlights strong generalization capability and adaptability to unseen scenarios.

Conclusion: DEFNet provides an effective framework for BIQA with improved task integration, robust feature fusion, and advanced uncertainty estimation, demonstrating strong performance and generalization.

Abstract: Blind image quality assessment (BIQA) methods often incorporate auxiliary tasks to improve performance. However, existing approaches face limitations due to insufficient integration and a lack of flexible uncertainty estimation, leading to suboptimal performance. To address these challenges, we propose a multitasks-based Deep Evidential Fusion Network (DEFNet) for BIQA, which performs multitask optimization with the assistance of scene and distortion type classification tasks. To achieve a more robust and reliable representation, we design a novel trustworthy information fusion strategy. It first combines diverse features and patterns across sub-regions to enhance information richness, and then performs local-global information fusion by balancing fine-grained details with coarse-grained context. Moreover, DEFNet exploits advanced uncertainty estimation technique inspired by evidential learning with the help of normal-inverse gamma distribution mixture. Extensive experiments on both synthetic and authentic distortion datasets demonstrate the effectiveness and robustness of the proposed framework. Additional evaluation and analysis are carried out to highlight its strong generalization capability and adaptability to previously unseen scenarios.

Method: Three-component framework: 1) Fish ROI extraction using visual foundation model, 2) Feature extraction from fish ROIs, 3) Interpretable prototype network for sex identification with learned prototype representations.

Result: Achieved 74.40% accuracy (74.27% F1) for early spawning stage and 81.16% accuracy (79.43% F1) for post-spawning stage. Subadult stage remains challenging due to less pronounced morphological differences.

Conclusion: FishProtoNet provides a robust, non-invasive solution for fish sex identification with interpretability, particularly effective for mature fish but limited for immature stages with subtle differences.

Abstract: Accurate sex identification in fish is vital for optimizing breeding and management strategies in aquaculture, particularly for species at the risk of extinction. However, most existing methods are invasive or stressful and may cause additional mortality, posing severe risks to threatened or endangered fish populations. To address these challenges, we propose FishProtoNet, a robust, non-invasive computer vision-based framework for sex identification of delta smelt (Hypomesus transpacificus), an endangered fish species native to California, across its full life cycle. Unlike the traditional deep learning methods, FishProtoNet provides interpretability through learned prototype representations while improving robustness by leveraging foundation models to reduce the influence of background noise. Specifically, the FishProtoNet framework consists of three key components: fish regions of interest (ROIs) extraction using visual foundation model, feature extraction from fish ROIs and fish sex identification based on an interpretable prototype network. FishProtoNet demonstrates strong performance in delta smelt sex identification during early spawning and post-spawning stages, achieving the accuracies of 74.40% and 81.16% and corresponding F1 scores of 74.27% and 79.43% respectively. In contrast, delta smelt sex identification at the subadult stage remains challenging for current computer vision methods, likely due to less pronounced morphological differences in immature fish. The source code of FishProtoNet is publicly available at: https://github.com/zhengmiao1/Fish_sex_identification

Method: Construct 3D Garment Videos dataset with consistent geometry/material supervision across deformations; use Nano Banana for non-isometric image editing; propose iterative baking via uncertainty-guided view selection and reweighting to fuse multi-view predictions.

Result: The feedforward dual-branch architecture generates versatile, spatially aligned PBR materials suitable for industry-level 3D garment design, demonstrating robust cross-pose texture learning.

Conclusion: The approach enables high-quality non-isometric image-based garment texture generation, overcoming limitations of existing methods and providing production-ready textures for industrial applications.

Abstract: Existing industrial 3D garment meshes already cover most real-world clothing geometries, yet their texture diversity remains limited. To acquire more realistic textures, generative methods are often used to extract Physically-based Rendering (PBR) textures and materials from large collections of wild images and project them back onto garment meshes. However, most image-conditioned texture generation approaches require strict topological consistency between the input image and the input 3D mesh, or rely on accurate mesh deformation to match to the image poses, which significantly constrains the texture generation quality and flexibility. To address the challenging problem of non-isometric image-based garment texture generation, we construct 3D Garment Videos, a physically simulated, garment-centric dataset that provides consistent geometry and material supervision across diverse deformations, enabling robust cross-pose texture learning. We further employ Nano Banana for high-quality non-isometric image editing, achieving reliable cross-topology texture generation between non-isometric image-geometry pairs. Finally, we propose an iterative baking method via uncertainty-guided view selection and reweighting that fuses multi-view predictions into seamless, production-ready PBR textures. Through extensive experiments, we demonstrate that our feedforward dual-branch architecture generates versatile and spatially aligned PBR materials suitable for industry-level 3D garment design.

Method: Extends standard cross-entropy loss with two regularizers: (1) mean-variance margin penalty that stabilizes inter-class logit margins, and (2) text moment-matching loss that aligns first and second moments of tuned text embeddings with frozen CLIP counterparts.

Result: Significantly reduces Expected Calibration Error (ECE) compared to competitive calibration techniques across 7 prompt-tuning methods and 11 diverse datasets, on both base and novel classes.

Conclusion: The proposed calibration framework enhances predictive reliability while preserving the geometry of pretrained CLIP embedding space, which is crucial for robust generalization in vision-language models.

Abstract: Prompt tuning of large-scale vision-language models such as CLIP enables efficient task adaptation without updating model weights. However, it often leads to poor confidence calibration and unreliable predictive uncertainty. We address this problem by proposing a calibration framework that enhances predictive reliability while preserving the geometry of the pretrained CLIP embedding space, which is required for robust generalization. Our approach extends the standard cross-entropy loss with two complementary regularizers: (1) a mean-variance margin penalty that stabilizes inter-class logit margins by maximizing their average while minimizing dispersion, mitigating underconfidence and overconfidence spikes; and (2) a text moment-matching loss that aligns the first and second moments of tuned text embeddings with their frozen CLIP counterparts, preserving semantic dispersion crucial for generalization. Through extensive experiments across 7 prompt-tuning methods and 11 diverse datasets, we demonstrate that our approach significantly reduces the Expected Calibration Error (ECE) compared to competitive calibration techniques on both base and novel classes

Method: Two-frame pose regression with explicit temporal dynamics encoding and leveraging 3D geometric priors from foundation models to handle varying observation rates and uncalibrated cameras.

Result: Achieves >20% performance improvement on KITTI, nuScenes, and Argoverse 2 benchmarks, with 46%-92% lower errors under varying observation rates compared to state-of-the-art methods.

Conclusion: OpenVO demonstrates robust ego-motion estimation for real-world dashcam footage, enabling trajectory extraction from rare driving events and supporting diverse downstream applications.

Abstract: We introduce OpenVO, a novel framework for Open-world Visual Odometry (VO) with temporal awareness under limited input conditions. OpenVO effectively estimates real-world-scale ego-motion from monocular dashcam footage with varying observation rates and uncalibrated cameras, enabling robust trajectory dataset construction from rare driving events recorded in dashcam. Existing VO methods are trained on fixed observation frequency (e.g., 10Hz or 12Hz), completely overlooking temporal dynamics information. Many prior methods also require calibrated cameras with known intrinsic parameters. Consequently, their performance degrades when (1) deployed under unseen observation frequencies or (2) applied to uncalibrated cameras. These significantly limit their generalizability to many downstream tasks, such as extracting trajectories from dashcam footage. To address these challenges, OpenVO (1) explicitly encodes temporal dynamics information within a two-frame pose regression framework and (2) leverages 3D geometric priors derived from foundation models. We validate our method on three major autonomous-driving benchmarks - KITTI, nuScenes, and Argoverse 2 - achieving more than 20 performance improvement over state-of-the-art approaches. Under varying observation rate settings, our method is significantly more robust, achieving 46%-92% lower errors across all metrics. These results demonstrate the versatility of OpenVO for real-world 3D reconstruction and diverse downstream applications.

Method: TeFlow introduces a temporal ensembling strategy that aggregates the most temporally consistent motion cues from a candidate pool built across multiple frames, enabling reliable multi-frame supervision for feed-forward models.

Result: Achieves up to 33% performance gains on Argoverse 2 and nuScenes datasets, establishes new state-of-the-art for self-supervised feed-forward methods, performs on par with leading optimization-based methods while being 150x faster.

Conclusion: TeFlow successfully enables effective multi-frame supervision for scene flow estimation through temporal consistency mining, bridging the gap between feed-forward efficiency and optimization-based accuracy.

Abstract: Self-supervised feed-forward methods for scene flow estimation offer real-time efficiency, but their supervision from two-frame point correspondences is unreliable and often breaks down under occlusions. Multi-frame supervision has the potential to provide more stable guidance by incorporating motion cues from past frames, yet naive extensions of two-frame objectives are ineffective because point correspondences vary abruptly across frames, producing inconsistent signals. In the paper, we present TeFlow, enabling multi-frame supervision for feed-forward models by mining temporally consistent supervision. TeFlow introduces a temporal ensembling strategy that forms reliable supervisory signals by aggregating the most temporally consistent motion cues from a candidate pool built across multiple frames. Extensive evaluations demonstrate that TeFlow establishes a new state-of-the-art for self-supervised feed-forward methods, achieving performance gains of up to 33% on the challenging Argoverse 2 and nuScenes datasets. Our method performs on par with leading optimization-based methods, yet speeds up 150 times. The code is open-sourced at https://github.com/KTH-RPL/OpenSceneFlow along with trained model weights.

Method: Two novel designs: 1) PoseRecover - automatic pose recovery pipeline matching questions with ego poses via object-frustum intersection and visibility checks, 2) PoseAlign - transforms point cloud data to align with identified ego poses rather than injecting poses into prompts or features.

Result: Consistent improvements across multiple 3D LMM backbones (LL3DA, LL3DA-SONATA, Chat-Scene, 3D-LLAVA), improving ScanRefer mIoU by 30.0% and Scan2Cap LLM-as-judge accuracy by 11.7%.

Conclusion: The approach is simple, generic, and training-efficient (only instruction tuning needed), establishing a strong baseline for direction-aware 3D LMMs by properly addressing the ego pose deficiency in existing benchmarks.

Abstract: 3D large multimodal models (3D LMMs) rely heavily on ego poses for enabling directional question-answering and spatial reasoning. However, most existing point cloud benchmarks contain rich directional queries but lack the corresponding ego poses, making them inherently ill-posed in 3D large multimodal modelling. In this work, we redefine a new and rigorous paradigm that enables direction-aware 3D LMMs by identifying and supplementing ego poses into point cloud benchmarks and transforming the corresponding point cloud data according to the identified ego poses. We enable direction-aware 3D LMMs with two novel designs. The first is PoseRecover, a fully automatic pose recovery pipeline that matches questions with ego poses from RGB-D video extrinsics via object-frustum intersection and visibility check with Z-buffers. The second is PoseAlign that transforms the point cloud data to be aligned with the identified ego poses instead of either injecting ego poses into textual prompts or introducing pose-encoded features in the projection layers. Extensive experiments show that our designs yield consistent improvements across multiple 3D LMM backbones such as LL3DA, LL3DA-SONATA, Chat-Scene, and 3D-LLAVA, improving ScanRefer mIoU by 30.0% and Scan2Cap LLM-as-judge accuracy by 11.7%. In addition, our approach is simple, generic, and training-efficient, requiring only instruction tuning while establishing a strong baseline for direction-aware 3D-LMMs.

Method: Designs a 3D residual regression network that rectifies range-view artifacts by predicting point-level offsets in 3D space. Uses a Welsch Loss to focus on local geometry and ignore anomalous regions. The framework is 3D-aware and can be applied to different LiDAR diffusion models with minimal computational overhead.

Result: Achieves state-of-the-art generation quality and superior geometry realism across multiple benchmarks (KITTI, KITTI360, nuScenes, Waymo). The 3D approach is shown to be inherently superior to 2D models for generating sharp and authentic boundaries.

Conclusion: L3DR successfully addresses the limitations of range-view LiDAR diffusion by incorporating 3D geometry awareness, achieving both 2D photo-realism and 3D geometry realism through 3D space rectification of artifacts.

Abstract: Range-view (RV) based LiDAR diffusion has recently made huge strides towards 2D photo-realism. However, it neglects 3D geometry realism and often generates various RV artifacts such as depth bleeding and wavy surfaces. We design L3DR, a 3D-aware LiDAR Diffusion and Rectification framework that can regress and cancel RV artifacts in 3D space and restore local geometry accurately. Our theoretical and empirical analysis reveals that 3D models are inherently superior to 2D models in generating sharp and authentic boundaries. Leveraging such analysis, we design a 3D residual regression network that rectifies RV artifacts and achieves superb geometry realism by predicting point-level offsets in 3D space. On top of that, we design a Welsch Loss that helps focus on local geometry and ignore anomalous regions effectively. Extensive experiments over multiple benchmarks including KITTI, KITTI360, nuScenes and Waymo show that the proposed L3DR achieves state-of-the-art generation and superior geometry-realism consistently. In addition, L3DR is generally applicable to different LiDAR diffusion models with little computational overhead.

Method: Reformulates editing as a transport problem between source and target distributions defined by text prompts. Uses dynamic optimal transport theory to derive a principled, low-energy control strategy that yields smoothed, variance-reduced editing fields that can be traversed in a single integration step.

Result: ChordEdit achieves high-fidelity one-step editing that is model-agnostic, training-free, and inversion-free, enabling true real-time editing on challenging one-step T2I models while maintaining precision and avoiding distortion.

Conclusion: The optimal transport approach provides a theoretically grounded solution for stable one-step image editing in fast text-to-image models, addressing the fundamental limitations of naive vector arithmetic methods.

Abstract: The advent of one-step text-to-image (T2I) models offers unprecedented synthesis speed. However, their application to text-guided image editing remains severely hampered, as forcing existing training-free editors into a single inference step fails. This failure manifests as severe object distortion and a critical loss of consistency in non-edited regions, resulting from the high-energy, erratic trajectories produced by naive vector arithmetic on the models’ structured fields. To address this problem, we introduce ChordEdit, a model agnostic, training-free, and inversion-free method that facilitates high-fidelity one-step editing. We recast editing as a transport problem between the source and target distributions defined by the source and target text prompts. Leveraging dynamic optimal transport theory, we derive a principled, low-energy control strategy. This strategy yields a smoothed, variance-reduced editing field that is inherently stable, facilitating the field to be traversed in a single, large integration step. A theoretically grounded and experimentally validated approach allows ChordEdit to deliver fast, lightweight and precise edits, finally achieving true real-time editing on these challenging models.

Method: Proposes RG-KCR framework with three stages: 1) Character detection using YOLOv12-medium, 2) Seal removal and character restoration using specialized algorithms, 3) Character classification using Vision Transformer (ViT)-based Metom classifier.

Result: YOLOv12-medium achieves 98.0% precision and 93.3% recall for detection; restoration stage improves Top-1 accuracy of ViT classifier from 93.45% to 95.33%; quantitative evaluation shows improved PSNR and SSIM metrics.

Conclusion: The RG-KCR framework effectively addresses seal interference in Kuzushiji recognition, demonstrating that restoration-guided approach significantly improves classification accuracy for historical documents with overlapping seals.

Abstract: Kuzushiji was one of the most popular writing styles in pre-modern Japan and was widely used in both personal letters and official documents. However, due to its highly cursive forms and extensive glyph variations, most modern Japanese readers cannot directly interpret Kuzushiji characters. Therefore, recent research has focused on developing automated Kuzushiji character recognition methods, which have achieved satisfactory performance on relatively clean Kuzushiji document images. However, existing methods struggle to maintain recognition accuracy under seal interference (e.g., when seals overlap characters), despite the frequent occurrence of seals in pre-modern Japanese documents. To address this challenge, we propose a three-stage restoration-guided Kuzushiji character recognition (RG-KCR) framework specifically designed to mitigate seal interference. We construct datasets for evaluating Kuzushiji character detection (Stage 1) and classification (Stage 3). Experimental results show that the YOLOv12-medium model achieves a precision of 98.0% and a recall of 93.3% on the constructed test set. We quantitatively evaluate the restoration performance of Stage 2 using PSNR and SSIM. In addition, we conduct an ablation study to demonstrate that Stage 2 improves the Top-1 accuracy of Metom, a Vision Transformer (ViT)-based Kuzushiji classifier employed in Stage 3, from 93.45% to 95.33%. The implementation code of this work is available at https://ruiyangju.github.io/RG-KCR.

Method: Proposes a layered motion representation disentangling rigid from residual non-rigid motion. Uses kinematic methods for rigid motion, creates coarse renderings to guide video diffusion for generating sequences with non-rigid motion. Introduces self-guided stochastic sampling to handle out-of-distribution renderings, combining stochastic sampling for quality with self-guidance for identity fidelity.

Result: Extensive experiments show Ani3DHuman generates photorealistic 3D human animation, outperforming existing methods in both quality and realism while maintaining identity fidelity.

Conclusion: Ani3DHuman successfully bridges kinematics-based animation with video diffusion priors to achieve photorealistic 3D human animation with realistic non-rigid dynamics, addressing limitations of previous approaches.

Abstract: Current 3D human animation methods struggle to achieve photorealism: kinematics-based approaches lack non-rigid dynamics (e.g., clothing dynamics), while methods that leverage video diffusion priors can synthesize non-rigid motion but suffer from quality artifacts and identity loss. To overcome these limitations, we present Ani3DHuman, a framework that marries kinematics-based animation with video diffusion priors. We first introduce a layered motion representation that disentangles rigid motion from residual non-rigid motion. Rigid motion is generated by a kinematic method, which then produces a coarse rendering to guide the video diffusion model in generating video sequences that restore the residual non-rigid motion. However, this restoration task, based on diffusion sampling, is highly challenging, as the initial renderings are out-of-distribution, causing standard deterministic ODE samplers to fail. Therefore, we propose a novel self-guided stochastic sampling method, which effectively addresses the out-of-distribution problem by combining stochastic sampling (for photorealistic quality) with self-guidance (for identity fidelity). These restored videos provide high-quality supervision, enabling the optimization of the residual non-rigid motion field. Extensive experiments demonstrate that \MethodName can generate photorealistic 3D human animation, outperforming existing methods. Code is available in https://github.com/qiisun/ani3dhuman.

Method: Proposes CREM with compression-based prompt design using learnable chorus tokens to aggregate multimodal semantics, and compression-driven training integrating contrastive and generative objectives through compression-aware attention.

Result: Achieves state-of-the-art retrieval performance on MMEB while maintaining strong generative performance on multiple comprehension benchmarks. Shows generative supervision improves representational quality.

Conclusion: CREM demonstrates that generative and embedding tasks can be unified through compression-driven representation enhancement, preserving both capabilities in MLLMs.

Abstract: Multimodal Large Language Models (MLLMs) have shown remarkable success in comprehension tasks such as visual description and visual question answering. However, their direct application to embedding-based tasks like retrieval remains challenging due to the discrepancy between output formats and optimization objectives. Previous approaches often employ contrastive fine-tuning to adapt MLLMs for retrieval, but at the cost of losing their generative capabilities. We argue that both generative and embedding tasks fundamentally rely on shared cognitive mechanisms, specifically cross-modal representation alignment and contextual comprehension. To this end, we propose CREM (Compression-driven Representation Enhanced Model), with a unified framework that enhances multimodal representations for retrieval while preserving generative ability. Specifically, we introduce a compression-based prompt design with learnable chorus tokens to aggregate multimodal semantics and a compression-driven training strategy that integrates contrastive and generative objectives through compression-aware attention. Extensive experiments demonstrate that CREM achieves state-of-the-art retrieval performance on MMEB while maintaining strong generative performance on multiple comprehension benchmarks. Our findings highlight that generative supervision can further improve the representational quality of MLLMs under the proposed compression-driven paradigm.

Method: VIGiA incorporates two key capabilities: (1) multimodal plan reasoning to align uni- and multimodal queries with current task plans, and (2) plan-based retrieval to retrieve relevant plan steps in either textual or visual representations.

Result: VIGiA outperforms existing state-of-the-art models on all tasks in conversational plan guidance settings, reaching over 90% accuracy on plan-aware VQA on a novel dataset with rich Instructional Video Dialogues aligned with Cooking and DIY plans.

Conclusion: VIGiA demonstrates effective multimodal reasoning over instructional video plans, enabling grounded dialogue that integrates visual understanding with plan-aware reasoning for complex multi-step tasks.

Abstract: We introduce VIGiA, a novel multimodal dialogue model designed to understand and reason over complex, multi-step instructional video action plans. Unlike prior work which focuses mainly on text-only guidance, or treats vision and language in isolation, VIGiA supports grounded, plan-aware dialogue that requires reasoning over visual inputs, instructional plans, and interleaved user interactions. To this end, VIGiA incorporates two key capabilities: (1) multimodal plan reasoning, enabling the model to align uni- and multimodal queries with the current task plan and respond accurately; and (2) plan-based retrieval, allowing it to retrieve relevant plan steps in either textual or visual representations. Experiments were done on a novel dataset with rich Instructional Video Dialogues aligned with Cooking and DIY plans. Our evaluation shows that VIGiA outperforms existing state-of-the-art models on all tasks in a conversational plan guidance setting, reaching over 90% accuracy on plan-aware VQA.

Method: Two-stage coarse-to-fine framework: 1) Coarse stage: class-agnostic 3D segmentation to obtain semantic parts, prompt MLLMs for part names, use pretrained VLMs for text embeddings to construct matched semantic parts. 2) Fine stage: leverage coarse correspondences to guide dense correspondence learning through rank-based contrastive scheme.

Result: Extensive experiments demonstrate UniMatch consistently outperforms competing methods in various challenging scenarios, enabling universal matching for inter-class and non-isometric shapes.

Conclusion: UniMatch is versatile for universal object categories, requires no predefined part proposals, and enables universal matching for inter-class and non-isometric shapes through language-guided semantic understanding.

Abstract: Establishing dense correspondences between shapes is a crucial task in computer vision and graphics, while prior approaches depend on near-isometric assumptions and homogeneous subject types (i.e., only operate for human shapes). However, building semantic correspondences for cross-category objects remains challenging and has received relatively little attention. To achieve this, we propose UniMatch, a semantic-aware, coarse-to-fine framework for constructing dense semantic correspondences between strongly non-isometric shapes without restricting object categories. The key insight is to lift “coarse” semantic cues into “fine” correspondence, which is achieved through two stages. In the “coarse” stage, we perform class-agnostic 3D segmentation to obtain non-overlapping semantic parts and prompt multimodal large language models (MLLMs) to identify part names. Then, we employ pretrained vision language models (VLMs) to extract text embeddings, enabling the construction of matched semantic parts. In the “fine” stage, we leverage these coarse correspondences to guide the learning of dense correspondences through a dedicated rank-based contrastive scheme. Thanks to class-agnostic segmentation, language guiding, and rank-based contrastive learning, our method is versatile for universal object categories and requires no predefined part proposals, enabling universal matching for inter-class and non-isometric shapes. Extensive experiments demonstrate UniMatch consistently outperforms competing methods in various challenging scenarios.

Method: ADAMAB trains embedder-agnostic lightweight calibrators on top of fixed embedding models without accessing their parameters. It uses adaptive data augmentation based on Multi-Armed Bandit mechanism with a modified upper confidence bound algorithm to reduce gradient shifting and ensure convergence.

Result: ADAMAB achieves up to 40% accuracy improvement when training with less than 5 initial data samples per class, demonstrating superior performance in multimodal few-shot pattern recognition tasks.

Conclusion: ADAMAB provides an efficient solution for few-shot pattern recognition by combining lightweight calibration with adaptive data augmentation, addressing computational and data scarcity challenges in multimodal learning.

Abstract: Recognizing implicit visual and textual patterns is essential in many real-world applications of modern AI. However, tackling long-tail pattern recognition tasks remains challenging for current pre-trained foundation models such as LLMs and VLMs. While finetuning pre-trained models can improve accuracy in recognizing implicit patterns, it is usually infeasible due to a lack of training data and high computational overhead. In this paper, we propose ADAMAB, an efficient embedding calibration framework for few-shot pattern recognition. To maximally reduce the computational costs, ADAMAB trains embedder-agnostic light-weight calibrators on top of fixed embedding models without accessing their parameters. To mitigate the need for large-scale training data, we introduce an adaptive data augmentation strategy based on the Multi-Armed Bandit (MAB) mechanism. With a modified upper confidence bound algorithm, ADAMAB diminishes the gradient shifting and offers theoretically guaranteed convergence in few-shot training. Our multi-modal experiments justify the superior performance of ADAMAB, with up to 40% accuracy improvement when training with less than 5 initial data samples of each class.

Method: Introduces Symbolic Projective Layout (SymPL) framework that reformulates allocentric reasoning into symbolic-layout forms using four key factors: projection, abstraction, bipartition, and localization.

Result: SymPL substantially improves performance in both allocentric and egocentric tasks, enhances robustness under visual illusions and multi-view scenarios, with each component contributing critically to these gains.

Conclusion: SymPL provides an effective and principled approach for addressing complex perspective-aware spatial reasoning by leveraging VLMs’ inherent strengths in symbolic-layout processing.

Abstract: Perspective-aware spatial reasoning involves understanding spatial relationships from specific viewpoints-either egocentric (observer-centered) or allocentric (object-centered). While vision-language models (VLMs) perform well in egocentric settings, their performance deteriorates when reasoning from allocentric viewpoints, where spatial relations must be inferred from the perspective of objects within the scene. In this study, we address this underexplored challenge by introducing Symbolic Projective Layout (SymPL), a framework that reformulates allocentric reasoning into symbolic-layout forms that VLMs inherently handle well. By leveraging four key factors-projection, abstraction, bipartition, and localization-SymPL converts allocentric questions into structured symbolic-layout representations. Extensive experiments demonstrate that this reformulation substantially improves performance in both allocentric and egocentric tasks, enhances robustness under visual illusions and multi-view scenarios, and that each component contributes critically to these gains. These results show that SymPL provides an effective and principled approach for addressing complex perspective-aware spatial reasoning.

Method: Created StreetTree dataset with over 12 million images from urban streetscapes across 133 countries, covering 8,300+ species, supplemented with expert-verified observational data and hierarchical taxonomy (order-family-genus-species).

Result: Established strong baselines through extensive experiments, revealing limitations of existing vision models in handling real-world complexities like high inter-species similarity, long-tailed distributions, seasonal variations, and diverse imaging conditions.

Conclusion: StreetTree serves as a key resource for urban street tree management and research while driving advancements at the intersection of computer vision and urban science.

Abstract: The fine-grained classification of street trees is a crucial task for urban planning, streetscape management, and the assessment of urban ecosystem services. However, progress in this field has been significantly hindered by the lack of large-scale, geographically diverse, and publicly available benchmark datasets specifically designed for street trees. To address this critical gap, we introduce StreetTree, the world’s first large-scale benchmark dataset dedicated to fine-grained street tree classification. The dataset contains over 12 million images covering more than 8,300 common street tree species, collected from urban streetscapes across 133 countries spanning five continents, and supplemented with expert-verified observational data. StreetTree poses substantial challenges for pretrained vision models under complex urban environments: high inter-species visual similarity, long-tailed natural distributions, significant intra-class variations caused by seasonal changes, and diverse imaging conditions such as lighting, occlusions from buildings, and varying camera angles. In addition, we provide a hierarchical taxonomy (order-family-genus-species) to support research in hierarchical classification and representation learning. Through extensive experiments with various visual models, we establish strong baselines and reveal the limitations of existing methods in handling such real-world complexities. We believe that StreetTree will serve as a key resource for the refined management and research of urban street trees, while also driving new advancements at the intersection of computer vision and urban science.

Method: Introduces Mapping Networks that replace high-dimensional weight spaces with compact, trainable latent vectors. Uses a dedicated Mapping Loss to enforce the Mapping Theorem, which theoretically and practically shows existence of mapping from latent space to target weight space.

Result: Achieves 99.5% reduction in trainable parameters (around 500x reduction) while maintaining comparable or better performance than target networks across complex vision and sequence tasks including Image Classification and Deepfake Detection.

Conclusion: Mapping Networks provide an effective solution to parameter efficiency and overfitting by exploiting the low-dimensional manifold structure of trained neural network parameters, enabling dramatic parameter reduction without performance loss.

Abstract: The escalating parameter counts in modern deep learning models pose a fundamental challenge to efficient training and resolution of overfitting. We address this by introducing the \emph{Mapping Networks} which replace the high dimensional weight space by a compact, trainable latent vector based on the hypothesis that the trained parameters of large networks reside on smooth, low-dimensional manifolds. Henceforth, the Mapping Theorem enforced by a dedicated Mapping Loss, shows the existence of a mapping from this latent space to the target weight space both theoretically and in practice. Mapping Networks significantly reduce overfitting and achieve comparable to better performance than target network across complex vision and sequence tasks, including Image Classification, Deepfake Detection etc, with $\mathbf{99.5%}$, i.e., around $500\times$ reduction in trainable parameters.

Method: Extends rectified flow for modality distribution mapping with: 1) one-to-many mapping strategy allowing each source data point to observe overall target distribution, 2) adaptive relaxed alignment with stricter alignment for same-sample pairs and relaxed mapping for different samples/categories, 3) cyclic rectified flow to prevent information loss by enabling feature translation back to original.

Result: Achieves very competitive results on multiple multimodal affective computing tasks with simple fusion methods, and visualizations confirm effective reduction of modality gap.

Conclusion: The proposed rectified flow approach with one-to-many mapping and adaptive alignment effectively reduces modality gap in multimodal learning, enabling better cross-modal representation learning even with simple fusion techniques.

Abstract: Modality gap significantly restricts the effectiveness of multimodal fusion. Previous methods often use techniques such as diffusion models and adversarial learning to reduce the modality gap, but they typically focus on one-to-one alignment without exposing the data points of the source modality to the global distribution information of the target modality. To this end, leveraging the characteristic of rectified flow that can map one distribution to another via a straight trajectory, we extend rectified flow for modality distribution mapping. Specifically, we leverage the one-to-many mapping' strategy in rectified flow that allows each data point of the source modality to observe the overall target distribution. This also alleviates the issue of insufficient paired data within each sample, enabling a more robust distribution transformation. Moreover, to achieve more accurate distribution mapping and address the ambiguous flow directions in one-to-many mapping, we design adaptive relaxed alignment’, enforcing stricter alignment for modality pairs belonging to the same sample, while applying relaxed mapping for pairs not belonging to the same sample or category. Additionally, to prevent information loss during distribution mapping, we introduce `cyclic rectified flow’ to ensure the transferred features can be translated back to the original features, allowing multimodal representations to learn sufficient modality-specific information. After distribution alignment, our approach achieves very competitive results on multiple tasks of multimodal affective computing even with a simple fusion method, and visualizations verify that it can effectively reduce the modality gap.

Method: Used RT-DETR transformer architecture with multi-class annotation strategy on 2,540 KOH microscopy images, employing morphology-preserving augmentations to maintain thin hyphae structural integrity for precise query-driven localization.

Result: Achieved object-level recall of 0.9737, precision of 0.8043, AP@0.50 of 93.56%, and image-level diagnosis with 100% sensitivity and 98.8% accuracy, correctly identifying all positive cases without missing diagnoses.

Conclusion: The AI system serves as a highly reliable automated screening tool that bridges image-level analysis and clinical decision-making in dermatomycology, demonstrating robust localization of low-contrast hyphae in artefact-rich fields.

Abstract: Dermatophytosis is commonly assessed using potassium hydroxide (KOH) microscopy, yet accurate recognition of fungal hyphae is hindered by artefacts, heterogeneous keratin clearance, and notable inter-observer variability. This study presents a transformer-based detection framework using the RT-DETR model architecture to achieve precise, query-driven localization of fungal structures in high-resolution KOH images. A dataset of 2,540 routinely acquired microscopy images was manually annotated using a multi-class strategy to explicitly distinguish fungal elements from confounding artefacts. The model was trained with morphology-preserving augmentations to maintain the structural integrity of thin hyphae. Evaluation on an independent test set demonstrated robust object-level performance, with a recall of 0.9737, precision of 0.8043, and an AP@0.50 of 93.56%. When aggregated for image-level diagnosis, the model achieved 100% sensitivity and 98.8% accuracy, correctly identifying all positive cases without missing a single diagnosis. Qualitative outputs confirmed the robust localization of low-contrast hyphae even in artefact-rich fields. These results highlight that an artificial intelligence (AI) system can serve as a highly reliable, automated screening tool, effectively bridging the gap between image-level analysis and clinical decision-making in dermatomycology.

Method: Proposes a universal acceleration framework with: (1) independence-aware channel pruning to mitigate channel redundancy, (2) stage-wise dominant operator optimization for efficient causal 3D convolutions, and (3) a three-phase dynamic distillation framework to transfer capabilities from original VAE decoders to the accelerated Flash-VAED models.

Result: Achieves approximately 6× speedup while maintaining reconstruction performance up to 96.9%. Flash-VAED accelerates end-to-end generation pipeline by up to 36% with negligible quality drops on VBench-2.0. Outperforms baselines in both quality and speed on Wan and LTX-Video VAE decoders.

Conclusion: Flash-VAED provides an effective solution to the VAE decoder bottleneck in video diffusion models, enabling significant speed improvements while preserving quality through innovative pruning, optimization, and distillation techniques.

Abstract: Latent diffusion models have enabled high-quality video synthesis, yet their inference remains costly and time-consuming. As diffusion transformers become increasingly efficient, the latency bottleneck inevitably shifts to VAE decoders. To reduce their latency while maintaining quality, we propose a universal acceleration framework for VAE decoders that preserves full alignment with the original latent distribution. Specifically, we propose (1) an independence-aware channel pruning method to effectively mitigate severe channel redundancy, and (2) a stage-wise dominant operator optimization strategy to address the high inference cost of the widely used causal 3D convolutions in VAE decoders. Based on these innovations, we construct a Flash-VAED family. Moreover, we design a three-phase dynamic distillation framework that efficiently transfers the capabilities of the original VAE decoder to Flash-VAED. Extensive experiments on Wan and LTX-Video VAE decoders demonstrate that our method outperforms baselines in both quality and speed, achieving approximately a 6$\times$ speedup while maintaining the reconstruction performance up to 96.9%. Notably, Flash-VAED accelerates the end-to-end generation pipeline by up to 36% with negligible quality drops on VBench-2.0.

Method: BriMA consists of: 1) memory-guided bridging imputation module that reconstructs missing modalities using both task-agnostic and task-specific representations, and 2) modality-aware replay mechanism that prioritizes informative samples based on modality distortion and distribution drift.

Result: Experiments on three multi-modal AQA datasets (RG, Fis-V, and FS1000) show BriMA consistently improves performance under different modality-missing conditions, achieving 6-8% higher correlation and 12-15% lower error on average.

Conclusion: BriMA demonstrates a step toward robust multi-modal AQA systems under real-world deployment constraints by effectively handling modality-missing conditions in continual learning scenarios.

Abstract: Action Quality Assessment (AQA) aims to score how well an action is performed and is widely used in sports analysis, rehabilitation assessment, and human skill evaluation. Multi-modal AQA has recently achieved strong progress by leveraging complementary visual and kinematic cues, yet real-world deployments often suffer from non-stationary modality imbalance, where certain modalities become missing or intermittently available due to sensor failures or annotation gaps. Existing continual AQA methods overlook this issue and assume that all modalities remain complete and stable throughout training, which restricts their practicality. To address this challenge, we introduce Bridged Modality Adaptation (BriMA), an innovative approach to multi-modal continual AQA under modality-missing conditions. BriMA consists of a memory-guided bridging imputation module that reconstructs missing modalities using both task-agnostic and task-specific representations, and a modality-aware replay mechanism that prioritizes informative samples based on modality distortion and distribution drift. Experiments on three representative multi-modal AQA datasets (RG, Fis-V, and FS1000) show that BriMA consistently improves performance under different modality-missing conditions, achieving 6–8% higher correlation and 12–15% lower error on average. These results demonstrate a step toward robust multi-modal AQA systems under real-world deployment constraints.

Method: EMAD uses hierarchical Sentence-Evidence-Anatomy (SEA) grounding: (1) sentence-to-evidence grounding links generated sentences to clinical evidence phrases, (2) evidence-to-anatomy grounding localizes corresponding structures on 3D brain MRI. Includes GTX-Distill to reduce annotation requirements by transferring grounding behavior from teacher to student, and Executable-Rule GRPO reinforcement fine-tuning with verifiable rewards for clinical consistency.

Result: On the AD-MultiSense dataset, EMAD achieves state-of-the-art diagnostic accuracy and produces more transparent, anatomically faithful reports than existing methods.

Conclusion: EMAD provides a trustworthy medical vision-language framework that generates structured diagnostic reports with explicit multimodal evidence grounding, addressing the black-box problem in medical AI while maintaining clinical consistency and anatomical faithfulness.

Abstract: Deep learning models for medical image analysis often act as black boxes, seldom aligning with clinical guidelines or explicitly linking decisions to supporting evidence. This is especially critical in Alzheimer’s disease (AD), where predictions should be grounded in both anatomical and clinical findings. We present EMAD, a vision-language framework that generates structured AD diagnostic reports in which each claim is explicitly grounded in multimodal evidence. EMAD uses a hierarchical Sentence-Evidence-Anatomy (SEA) grounding mechanism: (i) sentence-to-evidence grounding links generated sentences to clinical evidence phrases, and (ii) evidence-to-anatomy grounding localizes corresponding structures on 3D brain MRI. To reduce dense annotation requirements, we propose GTX-Distill, which transfers grounding behavior from a teacher trained with limited supervision to a student operating on model-generated reports. We further introduce Executable-Rule GRPO, a reinforcement fine-tuning scheme with verifiable rewards that enforces clinical consistency, protocol adherence, and reasoning-diagnosis coherence. On the AD-MultiSense dataset, EMAD achieves state-of-the-art diagnostic accuracy and produces more transparent, anatomically faithful reports than existing methods. We will release code and grounding annotations to support future research in trustworthy medical vision-language models.

Method: 1) Introduce a dual-memory augmented HMR critique agent with self-reflection to produce context-aware quality scores for predicted meshes, capturing fine-grained cues about 3D human motion structure, physical feasibility, and alignment with input images. 2) Use these scores to build a group-wise HMR preference dataset. 3) Propose a group preference alignment framework for finetuning diffusion-based HMR models, injecting rich preference signals to guide generation of more physically plausible and image-consistent human meshes.

Result: Extensive experiments demonstrate superior performance compared to state-of-the-art approaches, showing improved physical plausibility and image consistency in generated human meshes.

Conclusion: The proposed dual-memory critique agent with self-reflection and group preference alignment framework effectively addresses the accuracy-sacrifice problem in diffusion-based HMR methods, producing more physically plausible and image-consistent human mesh reconstructions from single RGB images.

Abstract: Human mesh recovery (HMR) from a single RGB image is inherently ambiguous, as multiple 3D poses can correspond to the same 2D observation. Recent diffusion-based methods tackle this by generating various hypotheses, but often sacrifice accuracy. They yield predictions that are either physically implausible or drift from the input image, especially under occlusion or in cluttered, in-the-wild scenes. To address this, we introduce a dual-memory augmented HMR critique agent with self-reflection to produce context-aware quality scores for predicted meshes. These scores distill fine-grained cues about 3D human motion structure, physical feasibility, and alignment with the input image. We use these scores to build a group-wise HMR preference dataset. Leveraging this dataset, we propose a group preference alignment framework for finetuning diffusion-based HMR models. This process injects the rich preference signals into the model, guiding it to generate more physically plausible and image-consistent human meshes. Extensive experiments demonstrate that our method achieves superior performance compared to state-of-the-art approaches.

Method: Introduces PositionOCR, a parameter-efficient hybrid architecture with 131M trainable parameters that integrates a text spotting model’s positional strengths with an LLM’s contextual reasoning.

Result: The framework demonstrates outstanding multi-modal processing capabilities, excelling in text grounding and text spotting tasks, consistently surpassing traditional MLLMs.

Conclusion: PositionOCR successfully bridges the gap between positional accuracy and semantic reasoning in multimodal LLMs for OCR-centric visual tasks through efficient hybrid architecture.

Abstract: In recent years, Multi-modal Large Language Models (MLLMs) have achieved strong performance in OCR-centric Visual Question Answering (VQA) tasks, illustrating their capability to process heterogeneous data and exhibit adaptability across varied contexts. However, these MLLMs rely on a Large Language Model (LLM) as the decoder, which is primarily designed for linguistic processing, and thus inherently lacks the positional reasoning required for precise visual tasks, such as text spotting and text grounding. Additionally, the extensive parameters of MLLMs necessitate substantial computational resources and large-scale data for effective training. Conversely, text spotting specialists achieve state-of-the-art coordinate predictions but lack semantic reasoning capabilities. This dichotomy motivates our key research question: Can we synergize the efficiency of specialists with the contextual power of LLMs to create a positionally-accurate MLLM? To overcome these challenges, we introduce PositionOCR, a parameter-efficient hybrid architecture that seamlessly integrates a text spotting model’s positional strengths with an LLM’s contextual reasoning. Comprising 131M trainable parameters, this framework demonstrates outstanding multi-modal processing capabilities, particularly excelling in tasks such as text grounding and text spotting, consistently surpassing traditional MLLMs.

Method: Created SAR Image-Text-AlphaEarth dataset; developed FUSAR-GPT with geospatial baseline model as prior knowledge; embedded multi-source remote-sensing temporal features via ‘spatiotemporal anchors’; used two-stage SFT strategy to decouple knowledge injection and task execution.

Result: Achieved state-of-the-art performance across several remote sensing visual-language benchmarks, outperforming mainstream baseline models by over 12%.

Conclusion: FUSAR-GPT successfully addresses SAR-specific challenges through geospatial priors and spatiotemporal feature embedding, enabling effective SAR image interpretation where standard VLMs fail.

Abstract: Research on the intelligent interpretation of all-weather, all-time Synthetic Aperture Radar (SAR) is crucial for advancing remote sensing applications. In recent years, although Visual Language Models (VLMs) have demonstrated strong open-world understanding capabilities on RGB images, their performance is severely limited when directly applied to the SAR field due to the complexity of the imaging mechanism, sensitivity to scattering features, and the scarcity of high-quality text corpora. To systematically address this issue, we constructed the inaugural SAR Image-Text-AlphaEarth feature triplet dataset and developed FUSAR-GPT, a VLM specifically for SAR. FUSAR-GPT innovatively introduces a geospatial baseline model as a ‘world knowledge’ prior and embeds multi-source remote-sensing temporal features into the model’s visual backbone via ‘spatiotemporal anchors’, enabling dynamic compensation for the sparse representation of targets in SAR images. Furthermore, we designed a two-stage SFT strategy to decouple the knowledge injection and task execution of large models. The spatiotemporal feature embedding and the two-stage decoupling paradigm enable FUSAR-GPT to achieve state-of-the-art performance across several typical remote sensing visual-language benchmark tests, significantly outperforming mainstream baseline models by over 12%.

Method: ManiPT introduces cosine consistency constraints in both text and image modalities to confine learned representations within the pretrained geometric neighborhood. It also adds structural bias for incremental corrections to guide adaptation along transferable directions and mitigate shortcut learning.

Result: ManiPT achieves higher average performance than baseline methods across four downstream settings: unseen-class generalization, few-shot classification, cross-dataset transfer, and domain generalization.

Conclusion: ManiPT effectively prevents representation drift in prompt tuning under limited supervision, improves generalization, and provides theoretical insights into overfitting tendencies in prompt tuning.

Abstract: Prompt tuning introduces learnable prompt vectors that adapt pretrained vision-language models to downstream tasks in a parameter-efficient manner. However, under limited supervision, prompt tuning alters pretrained representations and drives downstream features away from the pretrained manifold toward directions that are unfavorable for transfer. This drift degrades generalization. To address this limitation, we propose ManiPT, a framework that performs prompt tuning on the pretrained manifold. ManiPT introduces cosine consistency constraints in both the text and image modalities to confine the learned representations within the pretrained geometric neighborhood. Furthermore, we introduce a structural bias that enforces incremental corrections, guiding the adaptation along transferable directions to mitigate reliance on shortcut learning. From a theoretical perspective, ManiPT alleviates overfitting tendencies under limited data. Our experiments cover four downstream settings: unseen-class generalization, few-shot classification, cross-dataset transfer, and domain generalization. Across these settings, ManiPT achieves higher average performance than baseline methods. Notably, ManiPT provides an explicit perspective on how prompt tuning overfits under limited supervision.

Method: 1) Baseline: Directly use event data as condition for video diffusion model. 2) Enhanced: Introduce event-based inter-frame residual guidance based on physical correlation between events and frames. 3) Extension: Zero-shot video frame interpolation and prediction by modulating reverse diffusion sampling process.

Result: Significantly outperforms previous approaches on real-world and synthetic datasets both quantitatively and qualitatively. Unified framework for event-to-frame reconstruction.

Conclusion: Successfully leverages generative prior of pre-trained video diffusion models to reconstruct high-fidelity video from sparse event data, with extensions for interpolation and prediction in a unified framework.

Abstract: Event cameras excel at high-speed, low-power, and high-dynamic-range scene perception. However, as they fundamentally record only relative intensity changes rather than absolute intensity, the resulting data streams suffer from a significant loss of spatial information and static texture details. In this paper, we address this limitation by leveraging the generative prior of a pre-trained video diffusion model to reconstruct high-fidelity video frames from sparse event data. Specifically, we first establish a baseline model by directly applying event data as a condition to synthesize videos. Then, based on the physical correlation between the event stream and video frames, we further introduce the event-based inter-frame residual guidance to enhance the accuracy of video frame reconstruction. Furthermore, we extend our method to video frame interpolation and prediction in a zero-shot manner by modulating the reverse diffusion sampling process, thereby creating a unified event-to-frame reconstruction framework. Experimental results on real-world and synthetic datasets demonstrate that our method significantly outperforms previous approaches both quantitatively and qualitatively. We also refer the reviewers to the video demo contained in the supplementary material for video results. The code will be publicly available at https://github.com/CS-GangXu/UniE2F.

Method: Two-stage framework: 1) Dynamically generate text prompts embedded with 3D geometric priors using Geometric Defect Distillation Module (GDDM) that captures global shape context and local defect information. 2) Synergistic View Representation Learning processes rendered and depth images in parallel, with Synergistic Refinement Module (SRM) fusing features from both streams.

Result: Comprehensive experiments on four large-scale public datasets show GS-CLIP achieves superior performance in zero-shot 3D anomaly detection compared to existing methods.

Conclusion: GS-CLIP effectively addresses limitations of current CLIP-based 3D anomaly detection methods by incorporating geometric awareness through multi-view learning and geometric prompt engineering, enabling better detection of diverse anomaly types without target training data.

Abstract: Zero-shot 3D Anomaly Detection is an emerging task that aims to detect anomalies in a target dataset without any target training data, which is particularly important in scenarios constrained by sample scarcity and data privacy concerns. While current methods adapt CLIP by projecting 3D point clouds into 2D representations, they face challenges. The projection inherently loses some geometric details, and the reliance on a single 2D modality provides an incomplete visual understanding, limiting their ability to detect diverse anomaly types. To address these limitations, we propose the Geometry-Aware Prompt and Synergistic View Representation Learning (GS-CLIP) framework, which enables the model to identify geometric anomalies through a two-stage learning process. In stage 1, we dynamically generate text prompts embedded with 3D geometric priors. These prompts contain global shape context and local defect information distilled by our Geometric Defect Distillation Module (GDDM). In stage 2, we introduce Synergistic View Representation Learning architecture that processes rendered and depth images in parallel. A Synergistic Refinement Module (SRM) subsequently fuses the features of both streams, capitalizing on their complementary strengths. Comprehensive experimental results on four large-scale public datasets show that GS-CLIP achieves superior performance in detection. Code can be available at https://github.com/zhushengxinyue/GS-CLIP.

Method: SegMoTE preserves SAM’s original prompt interface and efficient inference while adding a small number of learnable parameters for dynamic adaptation across modalities and tasks. It includes a progressive prompt tokenization mechanism for fully automatic segmentation, reducing annotation dependence. Trained on MedSeg-HQ, a curated dataset less than 1% of existing large-scale datasets.

Result: Achieves state-of-the-art performance across diverse imaging modalities and anatomical tasks, demonstrating efficient, robust, and scalable adaptation of foundation vision models to medical domain under extremely low annotation cost.

Conclusion: SegMoTE represents the first efficient, robust, and scalable adaptation of general segmentation models to medical domain, advancing practical deployment of foundation vision models in clinical applications with minimal annotation requirements.

Abstract: Medical image segmentation is vital for clinical diagnosis and quantitative analysis, yet remains challenging due to the heterogeneity of imaging modalities and the high cost of pixel-level annotations. Although general interactive segmentation models like SAM have achieved remarkable progress, their transfer to medical imaging still faces two key bottlenecks: (i) the lack of adaptive mechanisms for modality- and anatomy-specific tasks, which limits generalization in out-of-distribution medical scenarios; and (ii) current medical adaptation methods fine-tune on large, heterogeneous datasets without selection, leading to noisy supervision, higher cost, and negative transfer. To address these issues, we propose SegMoTE, an efficient and adaptive framework for medical image segmentation. SegMoTE preserves SAM’s original prompt interface, efficient inference, and zero-shot generalization while introducing only a small number of learnable parameters to dynamically adapt across modalities and tasks. In addition, we design a progressive prompt tokenization mechanism that enables fully automatic segmentation, significantly reducing annotation dependence. Trained on MedSeg-HQ, a curated dataset less than 1% of existing large-scale datasets, SegMoTE achieves SOTA performance across diverse imaging modalities and anatomical tasks. It represents the first efficient, robust, and scalable adaptation of general segmentation models to the medical domain under extremely low annotation cost, advancing the practical deployment of foundation vision models in clinical applications.

Method: Proposes KRSVQG model that: 1) Incorporates related knowledge triplets from external knowledge sources to broaden question content, 2) Uses image captioning as an intermediary representation to ground questions to corresponding images, 3) Employs vision-language pre-training and fine-tuning strategy for adaptation to low data regimes.

Result: Created two knowledge-aware remote sensing VQG datasets (NWPU-300 and TextRS-300). Evaluations show KRSVQG outperforms existing methods and generates rich questions grounded in both image content and domain knowledge.

Conclusion: Knowledge-aware visual question generation advances understanding of image content beyond pixels, facilitating development of knowledge-enriched vision-language systems with vision-grounded human commonsense for remote sensing applications.

Abstract: With the rapid development of remote sensing image archives, asking questions about images has become an effective way of gathering specific information or performing semantic image retrieval. However, current automatically generated questions tend to be simplistic and template-based, which hinders the deployment of question answering or visual dialogue systems for real-world applications. To enrich and diversify the questions with both image content and commonsense knowledge, we propose a Knowledge-aware Remote Sensing Visual Question Generation model (KRSVQG). The proposed model incorporates related knowledge triplets from external knowledge sources to broaden the question content, while employing image captioning as an intermediary representation to ground questions to the corresponding images. Moreover, KRSVQG utilizes a vision-language pre-training and fine-tuning strategy, enabling the model’s adaptation to low data regimes. To evaluate the proposed KRSVQG model, we construct two knowledge-aware remote sensing visual question generation datasets: the NWPU-300 dataset and the TextRS-300 dataset. Evaluations, including metrics and human assessment, demonstrate that KRSVQG outperforms existing methods and leads to rich questions, grounded in both image and domain knowledge. As a key practice in vision-language research, knowledge-aware visual question generation advances the understanding of image content beyond pixels, facilitating the development of knowledge-enriched vision-language systems with vision-grounded human commonsense.

Method: Operates in semantic latent space of pre-trained face generator (Diffusion Autoencoder). Uses lightweight linear models with: (1) dependency-aware conditioning accounting for AU co-activation, (2) orthogonal projection removing nuisance attribute directions, and (3) expression neutralization for absolute AU editing.

Result: Generated edits are stronger, produce fewer artifacts, and preserve identity better than prior methods. Augmenting AU detector training with generated data improves accuracy, yields more disentangled predictions with fewer co-activation shortcuts, and outperforms alternative data-efficient training strategies.

Conclusion: The method effectively addresses label scarcity in facial expression analysis through controllable image editing that reduces entanglement, enabling data augmentation that improves AU detector performance similar to what would require substantially more labeled data.

Abstract: Deep learning vision models excel with abundant supervision, but many applications face label scarcity and class imbalance. Controllable image editing can augment scarce labeled data, yet edits often introduce artifacts and entangle non-target attributes. We study this in facial expression analysis, targeting Action Unit (AU) manipulation where annotation is costly and AU co-activation drives entanglement. We present a facial manipulation method that operates in the semantic latent space of a pre-trained face generator (Diffusion Autoencoder). Using lightweight linear models, we reduce entanglement of semantic features via (i) dependency-aware conditioning that accounts for AU co-activation, and (ii) orthogonal projection that removes nuisance attribute directions (e.g., glasses), together with an expression neutralization step to enable absolute AU edit. We use these edits to balance AU occurrence by editing labeled faces and to diversify identities/demographics via controlled synthesis. Augmenting AU detector training with the generated data improves accuracy and yields more disentangled predictions with fewer co-activation shortcuts, outperforming alternative data-efficient training strategies and suggesting improvements similar to what would require substantially more labeled data in our learning-curve analysis. Compared to prior methods, our edits are stronger, produce fewer artifacts, and preserve identity better.

Method: Proposes KRSVQG model that takes an image and related knowledge triplet from external knowledge sources as inputs, leveraging image captioning as an intermediary representation to enhance image grounding of generated questions.

Result: KRSVQG outperforms existing methods on two manually annotated datasets (NWPU-300 and TextRS-300), generating knowledge-enriched questions grounded in both image and domain knowledge.

Conclusion: Incorporating external knowledge significantly improves the quality and contextual understanding of generated questions for remote sensing images, enabling more effective visual question answering and dialogue systems.

Abstract: With the rapid development of remote sensing image archives, asking questions about images has become an effective way of gathering specific information or performing image retrieval. However, automatically generated image-based questions tend to be simplistic and template-based, which hinders the real deployment of question answering or visual dialogue systems. To enrich and diversify the questions, we propose a knowledge-aware remote sensing visual question generation model, KRSVQG, that incorporates external knowledge related to the image content to improve the quality and contextual understanding of the generated questions. The model takes an image and a related knowledge triplet from external knowledge sources as inputs and leverages image captioning as an intermediary representation to enhance the image grounding of the generated questions. To assess the performance of KRSVQG, we utilized two datasets that we manually annotated: NWPU-300 and TextRS-300. Results on these two datasets demonstrate that KRSVQG outperforms existing methods and leads to knowledge-enriched questions, grounded in both image and domain knowledge.

Method: 1) Anomaly Exposure Sampler transforms segmented objects into pseudo-anomalies to enhance adaptability to unseen categories. 2) Integrates Multimodal Large Language Model for semantic comprehension. 3) Token compression based on reverse attention to handle spatio-temporal scarcity and reduce computation.

Result: Achieves state-of-the-art performance on four benchmark VAD datasets for both frame-level and pixel-level anomaly detection in zero-shot settings, trained only on pseudo anomalies without real VAD data.

Conclusion: LAVIDA effectively addresses open-world VAD challenges through pseudo-anomaly generation and multimodal semantic understanding, demonstrating strong zero-shot generalization capabilities.

Abstract: The collection and detection of video anomaly data has long been a challenging problem due to its rare occurrence and spatio-temporal scarcity. Existing video anomaly detection (VAD) methods under perform in open-world scenarios. Key contributing factors include limited dataset diversity, and inadequate understanding of context-dependent anomalous semantics. To address these issues, i) we propose LAVIDA, an end-to-end zero-shot video anomaly detection framework. ii) LAVIDA employs an Anomaly Exposure Sampler that transforms segmented objects into pseudo-anomalies to enhance model adaptability to unseen anomaly categories. It further integrates a Multimodal Large Language Model (MLLM) to bolster semantic comprehension capabilities. Additionally, iii) we design a token compression approach based on reverse attention to handle the spatio-temporal scarcity of anomalous patterns and decrease computational cost. The training process is conducted solely on pseudo anomalies without any VAD data. Evaluations across four benchmark VAD datasets demonstrate that LAVIDA achieves SOTA performance in both frame-level and pixel-level anomaly detection under the zero-shot setting. Our code is available in https://github.com/VitaminCreed/LAVIDA.

Method: Attention-supervised diffusion framework that aligns attention scores of style tokens with object masks during training. Uses two complementary objectives: Focus loss (KL divergence) for accurate localization and Cover loss (binary cross-entropy) for dense coverage. Implements modular LoRA-MoE design for efficient multi-style adaptation.

Result: Achieves mask-free, single-object style transfer at inference with regionally accurate and visually coherent results. Outperforms existing diffusion-based editing approaches. Introduces Regional Style Editing Score for evaluation.

Conclusion: The proposed framework successfully addresses spatial control challenges in diffusion-based style transfer by explicitly teaching the model where to apply styles through attention supervision, enabling precise localized stylization without manual masks.

Abstract: Precise spatial control in diffusion-based style transfer remains challenging. This challenge arises because diffusion models treat style as a global feature and lack explicit spatial grounding of style representations, making it difficult to restrict style application to specific objects or regions. To our knowledge, existing diffusion models are unable to perform true localized style transfer, typically relying on handcrafted masks or multi-stage post-processing that introduce boundary artifacts and limit generalization. To address this, we propose an attention-supervised diffusion framework that explicitly teaches the model where to apply a given style by aligning the attention scores of style tokens with object masks during training. Two complementary objectives, a Focus loss based on KL divergence and a Cover loss using binary cross-entropy, jointly encourage accurate localization and dense coverage. A modular LoRA-MoE design further enables efficient and scalable multi-style adaptation. To evaluate localized stylization, we introduce the Regional Style Editing Score, which measures Regional Style Matching through CLIP-based similarity within the target region and Identity Preservation via masked LPIPS and pixel-level consistency on unedited areas. Experiments show that our method achieves mask-free, single-object style transfer at inference, producing regionally accurate and visually coherent results that outperform existing diffusion-based editing approaches.

Method: Adapts delta debugging (a systematic reduction strategy from software debugging) to isolate 1-minimal subsets of representational units. Configures search strategy based on unit interactions in classifier head: tests individual units for non-interacting models and unit combinations for models with interactions.

Result: Produces minimal, prediction-preserving saliency maps that highlight only essential features. Experimental evaluation shows DD-CAM produces more faithful explanations and achieves higher localization accuracy than state-of-the-art CAM-based approaches.

Conclusion: DD-CAM provides a principled approach to generating minimal sufficient explanations for vision models, offering improved interpretability through more focused and faithful saliency maps.

Abstract: We introduce a gradient-free framework for identifying minimal, sufficient, and decision-preserving explanations in vision models by isolating the smallest subset of representational units whose joint activation preserves predictions. Unlike existing approaches that aggregate all units, often leading to cluttered saliency maps, our approach, DD-CAM, identifies a 1-minimal subset whose joint activation suffices to preserve the prediction (i.e., removing any unit from the subset alters the prediction). To efficiently isolate minimal sufficient subsets, we adapt delta debugging, a systematic reduction strategy from software debugging, and configure its search strategy based on unit interactions in the classifier head: testing individual units for models with non-interacting units and testing unit combinations for models in which unit interactions exist. We then generate minimal, prediction-preserving saliency maps that highlight only the most essential features. Our experimental evaluation demonstrates that our approach can produce more faithful explanations and achieve higher localization accuracy than the state-of-the-art CAM-based approaches.

Method: Two-stage framework: 1) YOLOv8 for apple localization trained on orchard data, 2) ByteTrack for multi-object tracking to maintain persistent identities, 3) ResNet18 defect classifier fine-tuned on healthy-defective fruit dataset applied to cropped apple regions, 4) Track-level aggregation to enforce temporal consistency and reduce prediction oscillation.

Result: Improved stability compared to frame-wise inference, with defined video-level industrial metrics (track-level defect ratio and temporal consistency) demonstrating system robustness under realistic processing conditions.

Conclusion: Integrating tracking is essential for practical automated fruit grading systems to achieve temporal consistency and stability in video-based inspection.

Abstract: Industrial fruit inspection systems must operate reliably under dense multi-object interactions and continuous motion, yet most existing works evaluate detection or classification at the image level without ensuring temporal stability in video streams. We present a two-stage detection-tracking framework for stable multi-apple quality inspection in conveyor-belt environments. An orchard-trained YOLOv8 model performs apple localization, followed by ByteTrack multi-object tracking to maintain persistent identities. A ResNet18 defect classifier, fine-tuned on a healthy-defective fruit dataset, is applied to cropped apple regions. Track-level aggregation is introduced to enforce temporal consistency and reduce prediction oscillation across frames. We define video-level industrial metrics such as track-level defect ratio and temporal consistency to evaluate system robustness under realistic processing conditions. Results demonstrate improved stability compared to frame-wise inference, suggesting that integrating tracking is essential for practical automated fruit grading systems.

Method: Proposes MRI Contrast Enhancement Kinetics World model with SpatioTemporal Consistency Learning (STCL), including Latent Alignment Learning (LAL) for patient-level structure consistency and Latent Difference Learning (LDL) for smooth temporal kinetics.

Result: Extensive experiments on two datasets show MRI CEKWorld achieves better realistic contents and kinetics compared to baseline methods.

Conclusion: The proposed STCL framework effectively addresses content distortion and temporal discontinuity issues in MRI contrast enhancement kinetics modeling, enabling continuous simulation from sparse data.

Abstract: Clinical MRI contrast acquisition suffers from inefficient information yield, which presents as a mismatch between the risky and costly acquisition protocol and the fixed and sparse acquisition sequence. Applying world models to simulate the contrast enhancement kinetics in the human body enables continuous contrast-free dynamics. However, the low temporal resolution in MRI acquisition restricts the training of world models, leading to a sparsely sampled dataset. Directly training a generative model to capture the kinetics leads to two limitations: (a) Due to the absence of data on missing time, the model tends to overfit to irrelevant features, leading to content distortion. (b) Due to the lack of continuous temporal supervision, the model fails to learn the continuous kinetics law over time, causing temporal discontinuities. For the first time, we propose MRI Contrast Enhancement Kinetics World model (MRI CEKWorld) with SpatioTemporal Consistency Learning (STCL). For (a), guided by the spatial law that patient-level structures remain consistent during enhancement, we propose Latent Alignment Learning (LAL) that constructs a patient-specific template to constrain contents to align with this template. For (b), guided by the temporal law that the kinetics follow a consistent smooth trend, we propose Latent Difference Learning (LDL) which extends the unobserved intervals by interpolation and constrains smooth variations in the latent space among interpolated sequences. Extensive experiments on two datasets show our MRI CEKWorld achieves better realistic contents and kinetics. Codes will be available at https://github.com/DD0922/MRI-Contrast-Enhancement-Kinetics-World-Model.

Method: Proposes IPv2 with three core modules: Remove Background, Add noise, and Remove noise. These modules enable denoising capability in both background and lung tissue regions during training data construction, and provide refined label construction for better evaluation at testing stage.

Result: Extensive experiments on real-world patient lung CT dataset at 2% radiation dose show IPv2 consistently improves background suppression and lung parenchyma restoration across multiple mainstream denoising models.

Conclusion: IPv2 effectively addresses limitations of original image purification strategy, providing better denoising performance for ultra-low-dose CT images, particularly in background and lung tissue regions.

Abstract: The image purification strategy constructs an intermediate distribution with aligned anatomical structures, which effectively corrects the spatial misalignment between real-world ultra-low-dose CT and normal-dose CT images and significantly enhances the structural preservation ability of denoising models. However, this strategy exhibits two inherent limitations. First, it suppresses noise only in the chest wall and bone regions while leaving the image background untreated. Second, it lacks a dedicated mechanism for denoising the lung parenchyma. To address these issues, we systematically redesign the original image purification strategy and propose an improved version termed IPv2. The proposed strategy introduces three core modules, namely Remove Background, Add noise, and Remove noise. These modules endow the model with denoising capability in both background and lung tissue regions during training data construction and provide a more reasonable evaluation protocol through refined label construction at the testing stage. Extensive experiments on our previously established real-world patient lung CT dataset acquired at 2% radiation dose demonstrate that IPv2 consistently improves background suppression and lung parenchyma restoration across multiple mainstream denoising models. The code is publicly available at https://github.com/MonkeyDadLufy/Image-Purification-Strategy-v2.

Method: Proposes US-JEPA framework with Static-teacher Asymmetric Latent Training (SALT) objective. Uses a frozen, domain-specific teacher to provide stable latent targets, decoupling student-teacher optimization. The student learns to expand upon the semantic priors of the teacher through masked latent prediction rather than raw pixel reconstruction.

Result: First rigorous comparison of publicly available state-of-the-art ultrasound foundation models on UltraBench dataset. Under linear probing for diverse classification tasks, US-JEPA achieves performance competitive with or superior to domain-specific and universal vision foundation model baselines.

Conclusion: Masked latent prediction provides a stable and efficient path toward robust ultrasound representations. The static teacher approach addresses limitations of online teacher methods while leveraging domain-specific priors for improved medical imaging analysis.

Abstract: Ultrasound (US) imaging poses unique challenges for representation learning due to its inherently noisy acquisition process. The low signal-to-noise ratio and stochastic speckle patterns hinder standard self-supervised learning methods relying on a pixel-level reconstruction objective. Joint-Embedding Predictive Architectures (JEPAs) address this drawback by predicting masked latent representations rather than raw pixels. However, standard approaches depend on hyperparameter-brittle and computationally expensive online teachers updated via exponential moving average. We propose US-JEPA, a self-supervised framework that adopts the Static-teacher Asymmetric Latent Training (SALT) objective. By using a frozen, domain-specific teacher to provide stable latent targets, US-JEPA decouples student-teacher optimization and pushes the student to expand upon the semantic priors of the teacher. In addition, we provide the first rigorous comparison of all publicly available state-of-the-art ultrasound foundation models on UltraBench, a public dataset benchmark spanning multiple organs and pathological conditions. Under linear probing for diverse classification tasks, US-JEPA achieves performance competitive with or superior to domain-specific and universal vision foundation model baselines. Our results demonstrate that masked latent prediction provides a stable and efficient path toward robust ultrasound representations.

Method: Analyzes adversarial perturbations using wavelet transforms to understand their behavior in low- and high-frequency components. Proposes a frequency-aware defense that reconstructs training views by filtering high-frequency noise while preserving low-frequency content.

Result: The method substantially enhances robustness of 3DGS against various attack intensities on multiple benchmarks without access to clean ground-truth supervision. It maintains good performance on clean data while effectively suppressing adversarial artifacts.

Conclusion: The work addresses overlooked vulnerabilities in 3D Gaussian Splatting and provides a practical defense strategy that balances robustness and performance, paving the way for more secure 3D reconstruction systems.

Abstract: 3D Gaussian Splatting (3DGS) has emerged as a powerful paradigm for real-time and high-fidelity 3D reconstruction from posed images. However, recent studies reveal its vulnerability to adversarial corruptions in input views, where imperceptible yet consistent perturbations can drastically degrade rendering quality, increase training and rendering time, and inflate memory usage, even leading to server denial-of-service. In our work, to mitigate this issue, we begin by analyzing the distinct behaviors of adversarial perturbations in the low- and high-frequency components of input images using wavelet transforms. Based on this observation, we design a simple yet effective frequency-aware defense strategy that reconstructs training views by filtering high-frequency noise while preserving low-frequency content. This approach effectively suppresses adversarial artifacts while maintaining the authenticity of the original scene. Notably, it does not significantly impair training on clean data, achieving a desirable trade-off between robustness and performance on clean inputs. Through extensive experiments under a wide range of attack intensities on multiple benchmarks, we demonstrate that our method substantially enhances the robustness of 3DGS without access to clean ground-truth supervision. By highlighting and addressing the overlooked vulnerabilities of 3D Gaussian Splatting, our work paves the way for more robust and secure 3D reconstructions.

Method: Used OCT images from Retinal OCT Image Classification - C8 dataset (24,000 labeled images across 8 conditions). Images resized to 224x224 px. Tested Xception and InceptionV3 CNN architectures with data augmentation (CutMix, MixUp). Applied GradCAM and LIME for interpretability. Implemented in web application RetinaVision.

Result: Xception achieved 95.25% accuracy, InceptionV3 achieved 94.82% accuracy. Both models demonstrated high performance for retinal disease classification from OCT images.

Conclusion: Deep learning methods enable effective OCT retinal disease classification. Implementation of accuracy and interpretability is important for clinical applications, as demonstrated by the RetinaVision web application.

Abstract: Early and accurate classification of retinal diseases is critical to counter vision loss and for guiding clinical management of retinal diseases. In this study, we proposed a deep learning method for retinal disease classification utilizing optical coherence tomography (OCT) images from the Retinal OCT Image Classification - C8 dataset (comprising 24,000 labeled images spanning eight conditions). Images were resized to 224x224 px and tested on convolutional neural network (CNN) architectures: Xception and InceptionV3. Data augmentation techniques (CutMix, MixUp) were employed to enhance model generalization. Additionally, we applied GradCAM and LIME for interpretability evaluation. We implemented this in a real-world scenario via our web application named RetinaVision. This study found that Xception was the most accurate network (95.25%), followed closely by InceptionV3 (94.82%). These results suggest that deep learning methods allow effective OCT retinal disease classification and highlight the importance of implementing accuracy and interpretability for clinical applications.

Method: Uses a unified diffusion model with dual conditioning on CAD-derived, pose-aligned depth maps and structured prompts encoding sensor type and 4-DoF contact pose. This enables controllable, physically consistent multi-modal synthesis across different tactile sensors.

Result: Outperforms Pix2Pix cGAN baseline by +36.3% (ViTac), +134.6% (ViTacTip), and +64.7% (TacTip) in SSIM. For downstream 3-DoF pose estimation, mixing 50% synthetic with 50% real data halves required real data while maintaining competitive performance.

Conclusion: MultiDiffSense alleviates the data-collection bottleneck in tactile sensing and enables scalable, controllable multi-modal dataset generation for robotic applications through unified diffusion modeling.

Abstract: Acquiring aligned visuo-tactile datasets is slow and costly, requiring specialised hardware and large-scale data collection. Synthetic generation is promising, but prior methods are typically single-modality, limiting cross-modal learning. We present MultiDiffSense, a unified diffusion model that synthesises images for multiple vision-based tactile sensors (ViTac, TacTip, ViTacTip) within a single architecture. Our approach uses dual conditioning on CAD-derived, pose-aligned depth maps and structured prompts that encode sensor type and 4-DoF contact pose, enabling controllable, physically consistent multi-modal synthesis. Evaluating on 8 objects (5 seen, 3 novel) and unseen poses, MultiDiffSense outperforms a Pix2Pix cGAN baseline in SSIM by +36.3% (ViTac), +134.6% (ViTacTip), and +64.7% (TacTip). For downstream 3-DoF pose estimation, mixing 50% synthetic with 50% real halves the required real data while maintaining competitive performance. MultiDiffSense alleviates the data-collection bottleneck in tactile sensing and enables scalable, controllable multi-modal dataset generation for robotic applications.

Method: Projects LiDAR to range-view, encodes both modalities in shared space, uses uncertainty-guided fusion with learned uncertainty maps from visual degradations, and employs hybrid 2D-3D transformer to resolve spatial ambiguities and predict 3D masks.

Result: Demonstrates strong performance on Panoptic nuScenes, SemanticKITTI, and new Panoptic Waymo benchmark, maintaining robustness under severe visual corruption or misalignment.

Conclusion: UP-Fuse provides reliable multimodal fusion for safety-critical robotic perception by dynamically modulating cross-modal interactions based on uncertainty, making it robust to camera sensor issues.

Abstract: LiDAR-camera fusion enhances 3D panoptic segmentation by leveraging camera images to complement sparse LiDAR scans, but it also introduces a critical failure mode. Under adverse conditions, degradation or failure of the camera sensor can significantly compromise the reliability of the perception system. To address this problem, we introduce UP-Fuse, a novel uncertainty-aware fusion framework in the 2D range-view that remains robust under camera sensor degradation, calibration drift, and sensor failure. Raw LiDAR data is first projected into the range-view and encoded by a LiDAR encoder, while camera features are simultaneously extracted and projected into the same shared space. At its core, UP-Fuse employs an uncertainty-guided fusion module that dynamically modulates cross-modal interaction using predicted uncertainty maps. These maps are learned by quantifying representational divergence under diverse visual degradations, ensuring that only reliable visual cues influence the fused representation. The fused range-view features are decoded by a novel hybrid 2D-3D transformer that mitigates spatial ambiguities inherent to the 2D projection and directly predicts 3D panoptic segmentation masks. Extensive experiments on Panoptic nuScenes, SemanticKITTI, and our introduced Panoptic Waymo benchmark demonstrate the efficacy and robustness of UP-Fuse, which maintains strong performance even under severe visual corruption or misalignment, making it well suited for robotic perception in safety-critical settings.

Method: PoseCraft uses a diffusion framework with tokenized 3D interface: encodes sparse 3D landmarks and camera extrinsics as discrete conditioning tokens, injects them into diffusion via cross-attention. Also developed GenHumanRF workflow for generating diverse training data from volumetric reconstructions.

Result: PoseCraft achieves significant perceptual quality improvement over diffusion-centric methods, and attains better/comparable metrics to latest volumetric rendering SOTA while better preserving fabric and hair details.

Conclusion: The tokenized 3D interface approach effectively preserves 3D semantics, avoids 2D re-projection ambiguity, and produces photorealistic imagery that faithfully captures identity and appearance with explicit pose and camera controls.

Abstract: Digitizing humans and synthesizing photorealistic avatars with explicit 3D pose and camera controls are central to VR, telepresence, and entertainment. Existing skinning-based workflows require laborious manual rigging or template-based fittings, while neural volumetric methods rely on canonical templates and re-optimization for each unseen pose. We present PoseCraft, a diffusion framework built around tokenized 3D interface: instead of relying only on rasterized geometry as 2D control images, we encode sparse 3D landmarks and camera extrinsics as discrete conditioning tokens and inject them into diffusion via cross-attention. Our approach preserves 3D semantics by avoiding 2D re-projection ambiguity under large pose and viewpoint changes, and produces photorealistic imagery that faithfully captures identity and appearance. To train and evaluate at scale, we also implement GenHumanRF, a data generation workflow that renders diverse supervision from volumetric reconstructions. Our experiments show that PoseCraft achieves significant perceptual quality improvement over diffusion-centric methods, and attains better or comparable metrics to latest volumetric rendering SOTA while better preserving fabric and hair details.

Method: Developed MentalBlackboard, an open-ended spatial visualization benchmark with Paper Folding and Hole Punching tests in two core tasks: prediction (determining final outcome) and planning (determining sequence of steps). Evaluated various VLMs including Claude Opus 4.1 and o3 model.

Result: Models struggle with symmetrical transformations even when predicting unfolding steps correctly. Rotations significantly challenge physical situational awareness. Planning tasks reveal limitations in analyzing symmetrical relationships and multi-stage symmetry processes. Claude Opus 4.1 achieved highest planning accuracy at only 10%. o3 model reached 71.6% on generalization tasks but only 25% on text-based prediction tasks.

Conclusion: Current VLMs have significant limitations in spatial visualization capabilities, particularly with symmetrical transformations and rotations, indicating they lack the sophisticated mental spatial reasoning abilities present in human cognition.

Abstract: Spatial visualization is the mental ability to imagine, transform, and manipulate the spatial characteristics of objects and actions. This intelligence is a part of human cognition where actions and perception are connected on a mental level. To explore whether state-of-the-art Vision-Language Models (VLMs) exhibit this ability, we develop MentalBlackboard, an open-ended spatial visualization benchmark for Paper Folding and Hole Punching tests within two core tasks: prediction and planning. Our prediction experiments reveal that models struggle with applying symmetrical transformations, even when they predict the sequence of unfolding steps correctly. Also, rotations introduce a significant challenge to the physical situational awareness for models. The planning task reveals limitations of models in analyzing symmetrical relationships and in implementing the multi-stage symmetry process, with Claude Opus 4.1 achieving the highest planning score at an accuracy of 10%. The top-performing model, o3, attains a peak performance of 71.6% on the generalization task, which does not require spatial visualization but transfers spatial data; however, it achieves only 25% accuracy on text-based prediction tasks.

Method: Introduces RLD task, creates RefLade dataset (1.11M image-layer-prompt triplets with 100K manually curated layers), develops automatic evaluation protocol aligned with human preferences, and presents RefLayer baseline model for prompt-conditioned layer decomposition.

Result: Achieves high visual fidelity and semantic alignment in layer decomposition, enables effective training and reliable evaluation, exhibits strong zero-shot generalization capabilities, and establishes RLD as a well-defined benchmarkable research task.

Conclusion: The work bridges the gap between holistic image editing and structured layered representations, providing a foundation for advanced compositional understanding and controllable image editing through prompt-conditioned layer decomposition.

Abstract: Precise, object-aware control over visual content is essential for advanced image editing and compositional generation. Yet, most existing approaches operate on entire images holistically, limiting the ability to isolate and manipulate individual scene elements. In contrast, layered representations, where scenes are explicitly separated into objects, environmental context, and visual effects, provide a more intuitive and structured framework for interpreting and editing visual content. To bridge this gap and enable both compositional understanding and controllable editing, we introduce the Referring Layer Decomposition (RLD) task, which predicts complete RGBA layers from a single RGB image, conditioned on flexible user prompts, such as spatial inputs (e.g., points, boxes, masks), natural language descriptions, or combinations thereof. At the core is the RefLade, a large-scale dataset comprising 1.11M image-layer-prompt triplets produced by our scalable data engine, along with 100K manually curated, high-fidelity layers. Coupled with a perceptually grounded, human-preference-aligned automatic evaluation protocol, RefLade establishes RLD as a well-defined and benchmarkable research task. Building on this foundation, we present RefLayer, a simple baseline designed for prompt-conditioned layer decomposition, achieving high visual fidelity and semantic alignment. Extensive experiments show our approach enables effective training, reliable evaluation, and high-quality image decomposition, while exhibiting strong zero-shot generalization capabilities.

Method: Proposes Closed-Loop Memory Correction (CL-MC), a detector-in-the-loop framework that supervises Segment Anything Model 2 (SAM2) through confidence-aligned state decisions and active memory rectification. High-confidence detections trigger semantic resets that overwrite corrupted tracker memory, enabling training-free foundation tracker adaptation.

Result: On emergency intubation videos, CL-MC achieves state-of-the-art performance, significantly reducing drift and missing rate compared with SAM2 variants and open loop based methods.

Conclusion: Memory correction is a crucial component for reliable clinical video tracking, and CL-MC provides an effective solution for robust temporal-aware tracking in complex endoscopic scenes.

Abstract: Temporal stability in glottic opening localization remains challenging due to the complementary weaknesses of single-frame detectors and foundation-model trackers: the former lacks temporal context, while the latter suffers from memory drift. Specifically, in video laryngoscopy, rapid tissue deformation, occlusions, and visual ambiguities in emergency settings require a robust, temporally aware solution that can prevent progressive tracking errors. We propose Closed-Loop Memory Correction (CL-MC), a detector-in-the-loop framework that supervises Segment Anything Model 2(SAM2) through confidence-aligned state decisions and active memory rectification. High-confidence detections trigger semantic resets that overwrite corrupted tracker memory, effectively mitigating drift accumulation with a training-free foundation tracker in complex endoscopic scenes. On emergency intubation videos, CL-MC achieves state-of-the-art performance, significantly reducing drift and missing rate compared with the SAM2 variants and open loop based methods. Our results establish memory correction as a crucial component for reliable clinical video tracking. Our code will be available in https://github.com/huayuww/CL-MR.

Method: Introduces Stair Pooling, which moderates down-sampling pace by using a sequence of concatenated small and narrow pooling operations in varied orientations. Reduces dimensionality reduction from 1/4 to 1/2 per 2D pooling step, preserving more information. Can be adapted for 3D pooling. Uses transfer entropy to select optimal down-sampling paths.

Result: Extensive experiments on three biomedical image segmentation benchmarks show Stair Pooling increases both 2D and 3D U-Net performance by an average of 3.8% in Dice scores. Quantitative analysis demonstrates reduced information loss.

Conclusion: Stair Pooling effectively preserves spatial information during down-sampling, enabling U-Nets to better capture long-range dependencies and reconstruct spatial details during up-sampling, leading to improved segmentation accuracy in biomedical imaging tasks.

Abstract: U-Net architectures have been instrumental in advancing biomedical image segmentation (BIS) but often struggle with capturing long-range information. One reason is the conventional down-sampling techniques that prioritize computational efficiency at the expense of information retention. This paper introduces a simple but effective strategy, we call it Stair Pooling, which moderates the pace of down-sampling and reduces information loss by leveraging a sequence of concatenated small and narrow pooling operations in varied orientations. Specifically, our method modifies the reduction in dimensionality within each 2D pooling step from $\frac{1}{4}$ to $\frac{1}{2}$. This approach can also be adapted for 3D pooling to preserve even more information. Such preservation aids the U-Net in more effectively reconstructing spatial details during the up-sampling phase, thereby enhancing its ability to capture long-range information and improving segmentation accuracy. Extensive experiments on three BIS benchmarks demonstrate that the proposed Stair Pooling can increase both 2D and 3D U-Net performance by an average of 3.8% in Dice scores. Moreover, we leverage the transfer entropy to select the optimal down-sampling paths and quantitatively show how the proposed Stair Pooling reduces the information loss.

Method: PA-Attack uses prototype-anchored guidance to provide stable attack direction toward a general and dissimilar prototype, addressing attribute-restricted issues. It then employs a two-stage attention enhancement mechanism: (1) uses token-level attention scores to focus perturbations on critical visual tokens, and (2) adaptively recalibrates attention weights to track evolving attention during the adversarial process.

Result: Extensive experiments across diverse downstream tasks and LVLM architectures show PA-Attack achieves an average 75.1% score reduction rate (SRR), demonstrating strong attack effectiveness, efficiency, and task generalization in LVLMs.

Conclusion: PA-Attack effectively addresses the limitations of existing adversarial attack methods for LVLMs by leveraging the standardized vision encoder as a gray-box pivot and using prototype-anchored guidance with attention enhancement to achieve strong cross-task generalization and attack performance.

Abstract: Large Vision-Language Models (LVLMs) are foundational to modern multimodal applications, yet their susceptibility to adversarial attacks remains a critical concern. Prior white-box attacks rarely generalize across tasks, and black-box methods depend on expensive transfer, which limits efficiency. The vision encoder, standardized and often shared across LVLMs, provides a stable gray-box pivot with strong cross-model transfer. Building on this premise, we introduce PA-Attack (Prototype-Anchored Attentive Attack). PA-Attack begins with a prototype-anchored guidance that provides a stable attack direction towards a general and dissimilar prototype, tackling the attribute-restricted issue and limited task generalization of vanilla attacks. Building on this, we propose a two-stage attention enhancement mechanism: (i) leverage token-level attention scores to concentrate perturbations on critical visual tokens, and (ii) adaptively recalibrate attention weights to track the evolving attention during the adversarial process. Extensive experiments across diverse downstream tasks and LVLM architectures show that PA-Attack achieves an average 75.1% score reduction rate (SRR), demonstrating strong attack effectiveness, efficiency, and task generalization in LVLMs. Code is available at https://github.com/hefeimei06/PA-Attack.

Method: Prefer-DAS integrates self-training and prompt-guided contrastive learning in a promptable multitask model. It introduces Local direct Preference Optimization (LPO), sparse LPO (SLPO), and Unsupervised Preference Optimization (UPO) for alignment with spatially varying human feedback or self-learned preferences when feedback is missing.

Result: The model outperforms SAM-like methods and other unsupervised/weakly-supervised DAS methods on four challenging DAS tasks in both automatic and interactive segmentation modes, showing strong generalizability and flexibility. Performance approaches or exceeds supervised models.

Conclusion: Prefer-DAS provides an effective framework for domain adaptive segmentation that leverages sparse annotations and human preferences, offering practical solutions for electron microscopy analysis with reduced annotation burden.

Abstract: Domain adaptive segmentation (DAS) is a promising paradigm for delineating intracellular structures from various large-scale electron microscopy (EM) without incurring extensive annotated data in each domain. However, the prevalent unsupervised domain adaptation (UDA) strategies often demonstrate limited and biased performance, which hinders their practical applications. In this study, we explore sparse points and local human preferences as weak labels in the target domain, thereby presenting a more realistic yet annotation-efficient setting. Specifically, we develop Prefer-DAS, which pioneers sparse promptable learning and local preference alignment. The Prefer-DAS is a promptable multitask model that integrates self-training and prompt-guided contrastive learning. Unlike SAM-like methods, the Prefer-DAS allows for the use of full, partial, and even no point prompts during both training and inference stages and thus enables interactive segmentation. Instead of using image-level human preference alignment for segmentation, we introduce Local direct Preference Optimization (LPO) and sparse LPO (SLPO), plug-and-play solutions for alignment with spatially varying human feedback or sparse feedback. To address potential missing feedback, we also introduce Unsupervised Preference Optimization (UPO), which leverages self-learned preferences. As a result, the Prefer-DAS model can effectively perform both weakly-supervised and unsupervised DAS, depending on the availability of points and human preferences. Comprehensive experiments on four challenging DAS tasks demonstrate that our model outperforms SAM-like methods as well as unsupervised and weakly-supervised DAS methods in both automatic and interactive segmentation modes, highlighting strong generalizability and flexibility. Additionally, the performance of our model is very close to or even exceeds that of supervised models.

Method: Proposes Hepato-LLaVA with Sparse Topo-Pack Attention that explicitly models 2D tissue topology, aggregating local diagnostic evidence into semantic summary tokens while preserving global context. Also introduces HepatoPathoVQA dataset with 33K hierarchical QA pairs.

Result: Hepato-LLaVA achieves state-of-the-art performance on HCC diagnosis and captioning tasks, significantly outperforming existing methods.

Conclusion: The proposed specialized multimodal LLM with novel attention mechanism and clinical dataset advances fine-grained hepatocellular pathology analysis.

Abstract: Hepatocellular Carcinoma diagnosis relies heavily on the interpretation of gigapixel Whole Slide Images. However, current computational approaches are constrained by fixed-resolution processing mechanisms and inefficient feature aggregation, which inevitably lead to either severe information loss or high feature redundancy. To address these challenges, we propose Hepato-LLaVA, a specialized Multi-modal Large Language Model designed for fine-grained hepatocellular pathology analysis. We introduce a novel Sparse Topo-Pack Attention mechanism that explicitly models 2D tissue topology. This mechanism effectively aggregates local diagnostic evidence into semantic summary tokens while preserving global context. Furthermore, to overcome the lack of multi-scale data, we present HepatoPathoVQA, a clinically grounded dataset comprising 33K hierarchically structured question-answer pairs validated by expert pathologists. Our experiments demonstrate that Hepato-LLaVA achieves state-of-the-art performance on HCC diagnosis and captioning tasks, significantly outperforming existing methods. Our code and implementation details are available at https://pris-cv.github.io/Hepto-LLaVA/.

Method: TherA combines TherA-VLM (vision-language model) with a latent-diffusion-based translator. Given an RGB image and user-prompted conditions, TherA-VLM generates thermal-aware embeddings encoding scene, object, material, and heat-emission context, which then conditions a diffusion model for realistic TIR synthesis.

Result: TherA achieves state-of-the-art translation performance with up to 33% improvement in zero-shot translation metrics compared to other baselines, enabling fine-grained control over time of day, weather, and object state.

Conclusion: TherA provides a practical solution for generating diverse and thermally plausible TIR data through controllable RGB-to-TIR translation, addressing the data bottleneck in TIR-based perception systems.

Abstract: Despite the inherent advantages of thermal infrared(TIR) imaging, large-scale data collection and annotation remain a major bottleneck for TIR-based perception. A practical alternative is to synthesize pseudo TIR data via image translation; however, most RGB-to-TIR approaches heavily rely on RGB-centric priors that overlook thermal physics, yielding implausible heat distributions. In this paper, we introduce TherA, a controllable RGB-to-TIR translation framework that produces diverse and thermally plausible images at both scene and object level. TherA couples TherA-VLM with a latent-diffusion-based translator. Given a single RGB image and a user-prompted condition pair, TherA-VLM yields a thermal-aware embedding that encodes scene, object, material, and heat-emission context reflecting the input scene-condition pair. Conditioning the diffusion model on this embedding enables realistic TIR synthesis and fine-grained control across time of day, weather, and object state. Compared to other baselines, TherA achieves state-of-the-art translation performance, demonstrating improved zero-shot translation performance up to 33% increase averaged across all metrics.

Method: CountEx introduces a Discriminative Query Refinement module that jointly reasons over inclusion and exclusion cues. It identifies shared visual features, isolates exclusion-specific patterns, and applies selective suppression to refine counting queries. Supports multimodal prompts including natural language and visual exemplars.

Result: CountEx achieves substantial improvements over state-of-the-art methods for counting objects from both known and novel categories. The authors also introduce CoCount benchmark with 1,780 videos and 10,086 annotated frames across 97 category pairs for systematic evaluation.

Conclusion: CountEx successfully addresses the limitation of existing methods by enabling explicit exclusion of distractors, demonstrating superior performance in fine-grained visual counting tasks through discriminative query refinement.

Abstract: This paper presents CountEx, a discriminative visual counting framework designed to address a key limitation of existing prompt-based methods: the inability to explicitly exclude visually similar distractors. While current approaches allow users to specify what to count via inclusion prompts, they often struggle in cluttered scenes with confusable object categories, leading to ambiguity and overcounting. CountEx enables users to express both inclusion and exclusion intent, specifying what to count and what to ignore, through multimodal prompts including natural language descriptions and optional visual exemplars. At the core of CountEx is a novel Discriminative Query Refinement module, which jointly reasons over inclusion and exclusion cues by first identifying shared visual features, then isolating exclusion-specific patterns, and finally applying selective suppression to refine the counting query. To support systematic evaluation of fine-grained counting methods, we introduce CoCount, a benchmark comprising 1,780 videos and 10,086 annotated frames across 97 category pairs. Experiments show that CountEx achieves substantial improvements over state-of-the-art methods for counting objects from both known and novel categories. The data and code are available at https://github.com/bbvisual/CountEx.

Method: Proposes FinSight-Net with two key components: 1) Multi-Scale Decoupled Dual-Stream Processing (MS-DDSP) bottleneck that targets frequency-specific information loss via heterogeneous convolutional branches to suppress backscattering artifacts and compensate distorted biological cues, and 2) Efficient Path Aggregation FPN (EPA-FPN) that restores high-frequency spatial information through long-range skip connections and pruned fusion routes.

Result: Achieves state-of-the-art performance on DeepFish, AquaFishSet, and UW-BlurredFish benchmarks. On UW-BlurredFish, reaches 92.8% mAP, outperforming YOLOv11s by 4.8% while reducing parameters by 29.0%.

Conclusion: FinSight-Net provides an efficient, physics-aware solution for underwater fish detection that addresses fundamental optical degradation challenges in aquaculture environments, enabling robust real-time automated monitoring with reduced computational overhead.

Abstract: Underwater fish detection (UFD) is a core capability for smart aquaculture and marine ecological monitoring. While recent detectors improve accuracy by stacking feature extractors or introducing heavy attention modules, they often incur substantial computational overhead and, more importantly, neglect the physics that fundamentally limits UFD: wavelength-dependent absorption and turbidity-induced scattering significantly degrade contrast, blur fine structures, and introduce backscattering noise, leading to unreliable localization and recognition. To address these challenges, we propose FinSight-Net, an efficient and physics-aware detection framework tailored for complex aquaculture environments. FinSight-Net introduces a Multi-Scale Decoupled Dual-Stream Processing (MS-DDSP) bottleneck that explicitly targets frequency-specific information loss via heterogeneous convolutional branches, suppressing backscattering artifacts while compensating distorted biological cues through scale-aware and channel-weighted pathways. We further design an Efficient Path Aggregation FPN (EPA-FPN) as a detail-filling mechanism: it restores high-frequency spatial information typically attenuated in deep layers by establishing long-range skip connections and pruning redundant fusion routes, enabling robust detection of non-rigid fish targets under severe blur and turbidity. Extensive experiments on DeepFish, AquaFishSet, and our challenging UW-BlurredFish benchmark demonstrate that FinSight-Net achieves state-of-the-art performance. In particular, on UW-BlurredFish, FinSight-Net reaches 92.8% mAP, outperforming YOLOv11s by 4.8% while reducing parameters by 29.0%, providing a strong and lightweight solution for real-time automated monitoring in smart aquaculture.

Method: Proposes a three-stage training-free post-hoc concept-bottleneck pipeline: 1) concept mining from human annotations, 2) multi-agent structured scoring (Observer-Debater-Judge chain) to extract continuous concept scores from frozen VLM, and 3) geometric calibration using locally-weighted ridge regression on a hybrid visual-semantic manifold to align scores with human ratings.

Result: Applied to urban perception as UrbanAlign, achieves 72.2% accuracy (κ=0.45) on Place Pulse 2.0 across six categories, outperforming best supervised baseline by +15.1 percentage points and uncalibrated VLM scoring by +16.3 percentage points.

Conclusion: Demonstrates that VLM alignment with human preferences can be achieved without model training through external calibration, providing full dimension-level interpretability and zero model-weight modification.

Abstract: Aligning vision-language model (VLM) outputs with human preferences in domain-specific tasks typically requires fine-tuning or reinforcement learning, both of which demand labelled data and GPU compute. We show that for subjective perception tasks, this alignment can be achieved without any model training: VLMs are already strong concept extractors but poor decision calibrators, and the gap can be closed externally. We propose a training-free post-hoc concept-bottleneck pipeline consisting of three tightly coupled stages: concept mining, multi-agent structured scoring, and geometric calibration, unified by an end-to-end dimension optimization loop. Interpretable evaluation dimensions are mined from a handful of human annotations; an Observer-Debater-Judge chain extracts robust continuous concept scores from a frozen VLM; and locally-weighted ridge regression on a hybrid visual-semantic manifold calibrates these scores against human ratings. Applied to urban perception as UrbanAlign, the framework achieves 72.2% accuracy ($κ=0.45$) on Place Pulse 2.0 across six categories, outperforming the best supervised baseline by +15.1 pp and uncalibrated VLM scoring by +16.3 pp, with full dimension-level interpretability and zero model-weight modification.

Method: CRAFT fine-tunes the vision encoder using a discrete codebook that anchors visual representations to a stable token space. This decouples adaptation from the language model, allowing the adapted encoder to work with different LVLM architectures sharing the same codebook.

Result: Achieves average gain of 13.51% across 10 domain-specific benchmarks (VQARAD, PlantVillage, etc.) while preserving linguistic capabilities and outperforming continuous token adaptation methods.

Conclusion: CRAFT provides effective domain adaptation for LVLMs through discrete codebook regulation, enabling encoder specialization without language model modification and supporting cross-architecture compatibility.

Abstract: Large Vision-Language Models (LVLMs) use their vision encoders to translate images into representations for downstream reasoning, but the encoders often underperform in domain-specific visual tasks such as medical image diagnosis or fine-grained classification, where representation errors can cascade through the language model, leading to incorrect responses. Existing adaptation methods modify the continuous feature interface between encoder and language model through projector tuning or other parameter-efficient updates, which still couples the two components and requires re-alignment whenever the encoder changes. We introduce CRAFT (Codebook RegulAted Fine-Tuning), a lightweight method that fine-tunes the encoder using a discrete codebook that anchors visual representations to a stable token space, achieving domain adaptation without modifying other parts of the model. This decoupled design allows the adapted encoder to seamlessly boost the performance of LVLMs with different language architectures, as long as they share the same codebook. Empirically, CRAFT achieves an average gain of 13.51% across 10 domain-specific benchmarks such as VQARAD and PlantVillage, while preserving the LLM’s linguistic capabilities and outperforming peer methods that operate on continuous tokens.

Method: Proposes Hypothesis-Driven TTA framework that treats adaptation as a dynamic decision process. Generates competing geometric hypotheses: compaction (trim artifacts) vs inflation (recover under-segmented tumor). Uses representation-guided selector to choose safest outcome based on texture consistency, with a Gatekeeper to skip adaptation on confident cases.

Result: HD-TTA improves safety-oriented outcomes on cross-domain brain tumor segmentation: reduces Hausdorff Distance (HD95) by ~6.4 mm and improves Precision by over 4% while maintaining comparable Dice scores compared to state-of-the-art baselines.

Conclusion: Explicit hypothesis selection resolves the safety-adaptation trade-off, providing a viable, robust path for safe clinical model deployment in medical segmentation tasks.

Abstract: Standard Test-Time Adaptation (TTA) methods typically treat inference as a blind optimization task, applying generic objectives to all or filtered test samples. In safety-critical medical segmentation, this lack of selectivity often causes the tumor mask to spill into healthy brain tissue or degrades predictions that were already correct. We propose Hypothesis-Driven TTA, a novel framework that reformulates adaptation as a dynamic decision process. Rather than forcing a single optimization trajectory, our method generates intuitive competing geometric hypotheses: compaction (is the prediction noisy? trim artifacts) versus inflation (is the valid tumor under-segmented? safely inflate to recover). It then employs a representation-guided selector to autonomously identify the safest outcome based on intrinsic texture consistency. Additionally, a pre-screening Gatekeeper prevents negative transfer by skipping adaptation on confident cases. We validate this proof-of-concept on a cross-domain binary brain tumor segmentation task, applying a source model trained on adult BraTS gliomas to unseen pediatric and more challenging meningioma target domains. HD-TTA improves safety-oriented outcomes (Hausdorff Distance (HD95) and Precision) over several state-of-the-art representative baselines in the challenging safety regime, reducing the HD95 by approximately 6.4 mm and improving Precision by over 4%, while maintaining comparable Dice scores. These results demonstrate that resolving the safety-adaptation trade-off via explicit hypothesis selection is a viable, robust path for safe clinical model deployment. Code will be made publicly available upon acceptance.

Method: Decomposes images into Laplacian pyramid residuals, processes different scales in parallel using mixture-of-transformers (MoT) with causal attention, eliminating bridging processes between scales.

Result: Achieves superior sample quality with fewer GFLOPs and faster inference on CelebA-HQ and ImageNet, scales effectively to 1024×1024 resolution with lower computational overhead.

Conclusion: LapFlow improves flow matching through efficient multi-scale parallel processing, enabling high-quality high-resolution image generation with reduced computational cost.

Abstract: In this paper, we present Laplacian multiscale flow matching (LapFlow), a novel framework that enhances flow matching by leveraging multi-scale representations for image generative modeling. Our approach decomposes images into Laplacian pyramid residuals and processes different scales in parallel through a mixture-of-transformers (MoT) architecture with causal attention mechanisms. Unlike previous cascaded approaches that require explicit renoising between scales, our model generates multi-scale representations in parallel, eliminating the need for bridging processes. The proposed multi-scale architecture not only improves generation quality but also accelerates the sampling process and promotes scaling flow matching methods. Through extensive experimentation on CelebA-HQ and ImageNet, we demonstrate that our method achieves superior sample quality with fewer GFLOPs and faster inference compared to single-scale and multi-scale flow matching baselines. The proposed model scales effectively to high-resolution generation (up to 1024$\times$1024) while maintaining lower computational overhead.

Method: Proposes a physics-informed deep learning framework that combines polarization cues (providing orientation priors) with structured illumination (encoding geometric information). Uses dual-encoder architecture with mutual feature modulation to resolve nonlinear coupling between these complementary cues and directly infer surface normals.

Result: Achieves accurate and robust normal estimation in single-shot with fast inference, enabling practical 3D imaging of complex specular surfaces.

Conclusion: The proposed method successfully addresses limitations of existing techniques by combining polarization and structured illumination through deep learning, enabling single-shot 3D imaging of complex specular surfaces suitable for dynamic environments.

Abstract: 3D imaging of specular surfaces remains challenging in real-world scenarios, such as in-line inspection or hand-held scanning, requiring fast and accurate measurement of complex geometries. Optical metrology techniques such as deflectometry achieve high accuracy but typically rely on multi-shot acquisition, making them unsuitable for dynamic environments. Fourier-based single-shot approaches alleviate this constraint, yet their performance deteriorates when measuring surfaces with high spatial frequency structure or large curvature. Alternatively, polarimetric 3D imaging in computer vision operates in a single-shot fashion and exhibits robustness to geometric complexity. However, its accuracy is fundamentally limited by the orthographic imaging assumption. In this paper, we propose a physics-informed deep learning framework for single-shot 3D imaging of complex specular surfaces. Polarization cues provide orientation priors that assist in interpreting geometric information encoded by structured illumination. These complementary cues are processed through a dual-encoder architecture with mutual feature modulation, allowing the network to resolve their nonlinear coupling and directly infer surface normals. The proposed method achieves accurate and robust normal estimation in single-shot with fast inference, enabling practical 3D imaging of complex specular surfaces.

Method: Proposes FRLA method with two modules: 1) Forgetting-resistant adaptation module that explicitly preserves confident predictions of the target model, and 2) Lesion-aware adaptation module that yields patch-wise predictions from ViL model to help target model be aware of lesion areas and leverage ViL’s fine-grained knowledge.

Result: Extensive experiments show the method significantly outperforms the vision-language model and achieves consistent improvements over state-of-the-art methods.

Conclusion: The proposed FRLA method effectively addresses prediction forgetting and leverages fine-grained ViL knowledge for improved source-free domain adaptation in medical image diagnosis.

Abstract: Source-free domain adaptation (SFDA) aims to adapt a model trained in the source domain to perform well in the target domain, with only unlabeled target domain data and the source model. Taking into account that conventional SFDA methods are inevitably error-prone under domain shift, recently greater attention has been directed to SFDA assisted with off-the-shelf foundation models, e.g., vision-language (ViL) models. However, existing works of leveraging ViL models for SFDA confront two issues: (i) Although mutual information is exploited to consider the joint distribution between the predictions of ViL model and the target model, we argue that the forgetting of some superior predictions of the target model still occurs, as indicated by the decline of the accuracies of certain classes during adaptation; (ii) Prior research disregards the rich, fine-grained knowledge embedded in the ViL model, which offers detailed grounding for fundus image diagnosis. In this paper, we introduce a novel forgetting-resistant and lesion-aware (FRLA) method for SFDA of fundus image diagnosis with ViL model. Specifically, a forgetting-resistant adaptation module explicitly preserves the confident predictions of the target model, and a lesion-aware adaptation module yields patch-wise predictions from ViL model and employs them to help the target model be aware of the lesion areas and leverage the ViL model’s fine-grained knowledge. Extensive experiments show that our method not only significantly outperforms the vision-language model, but also achieves consistent improvements over the state-of-the-art methods. Our code will be released.

Method: Proposes a spatially regularized MIL framework that leverages inherent spatial relationships among patch features as label-independent regularization signals. Learns a shared representation space by jointly optimizing feature-induced spatial reconstruction and label-guided classification objectives, enforcing consistency between intrinsic structural patterns and supervisory signals.

Result: Experimental results on multiple public datasets demonstrate significant improvements over state-of-the-art methods.

Conclusion: The approach offers a promising direction for whole slide image analysis by addressing sparse supervision challenges through spatial regularization.

Abstract: Whole slide images, with their gigapixel-scale panoramas of tissue samples, are pivotal for precise disease diagnosis. However, their analysis is hindered by immense data size and scarce annotations. Existing MIL methods face challenges due to the fundamental imbalance where a single bag-level label must guide the learning of numerous patch-level features. This sparse supervision makes it difficult to reliably identify discriminative patches during training, leading to unstable optimization and suboptimal solutions. We propose a spatially regularized MIL framework that leverages inherent spatial relationships among patch features as label-independent regularization signals. Our approach learns a shared representation space by jointly optimizing feature-induced spatial reconstruction and label-guided classification objectives, enforcing consistency between intrinsic structural patterns and supervisory signals. Experimental results on multiple public datasets demonstrate significant improvements over state-of-the-art methods, offering a promising direction.

Method: Introduces MICON-Bench covering six tasks for cross-image composition, contextual reasoning, and identity preservation; proposes MLLM-driven Evaluation-by-Checkpoint framework for automatic verification; and presents Dynamic Attention Rebalancing (DAR) - a training-free mechanism that adjusts attention during inference.

Result: Extensive experiments show MICON-Bench effectively exposes multi-image reasoning challenges in state-of-the-art models, and DAR improves generation quality and cross-image coherence across various open-source models.

Conclusion: The paper addresses a gap in evaluating multi-image reasoning capabilities and provides both a rigorous benchmark and practical technique to enhance cross-image coherence in multimodal models.

Abstract: Recent advancements in Unified Multimodal Models (UMMs) have enabled remarkable image understanding and generation capabilities. However, while models like Gemini-2.5-Flash-Image show emerging abilities to reason over multiple related images, existing benchmarks rarely address the challenges of multi-image context generation, focusing mainly on text-to-image or single-image editing tasks. In this work, we introduce \textbf{MICON-Bench}, a comprehensive benchmark covering six tasks that evaluate cross-image composition, contextual reasoning, and identity preservation. We further propose an MLLM-driven Evaluation-by-Checkpoint framework for automatic verification of semantic and visual consistency, where multimodal large language model (MLLM) serves as a verifier. Additionally, we present \textbf{Dynamic Attention Rebalancing (DAR)}, a training-free, plug-and-play mechanism that dynamically adjusts attention during inference to enhance coherence and reduce hallucinations. Extensive experiments on various state-of-the-art open-source models demonstrate both the rigor of MICON-Bench in exposing multi-image reasoning challenges and the efficacy of DAR in improving generation quality and cross-image coherence. Github: https://github.com/Angusliuuu/MICON-Bench.

Method: Modified YOLO-World by replacing C2f layer in YOLOv8 backbone with C3k2 layer for better local feature representation, plus parallel processing optimization for speed and lightweight design.

Result: Improved precision (40.6%→41.6%), recall (30.8%→31%), F1 (35%→35.5%), mAP@0.5 (30.4%→30.7%) on VisDrone dataset; reduced parameters (4M→3.8M) and FLOPs (15.7B→15.2B).

Conclusion: The proposed approach provides practical and effective solution for precise text-guided object detection in drone applications with enhanced accuracy and lightweight performance.

Abstract: As drone-based object detection technology continues to evolve, the demand is shifting from merely detecting objects to enabling users to accurately identify specific targets. For example, users can input particular targets as prompts to precisely detect desired objects. To address this need, an efficient text-guided object detection model has been developed to enhance the detection of small objects. Specifically, an improved version of the existing YOLO-World model is introduced. The proposed method replaces the C2f layer in the YOLOv8 backbone with a C3k2 layer, enabling more precise representation of local features, particularly for small objects or those with clearly defined boundaries. Additionally, the proposed architecture improves processing speed and efficiency through parallel processing optimization, while also contributing to a more lightweight model design. Comparative experiments on the VisDrone dataset show that the proposed model outperforms the original YOLO-World model, with precision increasing from 40.6% to 41.6%, recall from 30.8% to 31%, F1 score from 35% to 35.5%, and mAP@0.5 from 30.4% to 30.7%, confirming its enhanced accuracy. Furthermore, the model demonstrates superior lightweight performance, with the parameter count reduced from 4 million to 3.8 million and FLOPs decreasing from 15.7 billion to 15.2 billion. These results indicate that the proposed approach provides a practical and effective solution for precise object detection in drone-based applications.

Method: 1) Sample foreground voxels based on predicted scores; 2) Design Regional-to-Global Slide Window (RGSW) to propagate information from regional splits to entire sequence, addressing response attenuation; 3) Propose Semantic-Assisted and State Spatial Fusion Module (SASFMamba) to enrich contextual representation by enhancing semantic and geometric awareness within Mamba.

Result: Superior performance across various benchmarks demonstrates effectiveness of Fore-Mamba3D in 3D object detection task.

Conclusion: Fore-Mamba3D successfully addresses limitations of previous Mamba-based methods by focusing on foreground-only encoding while alleviating distance-based and causal dependencies in linear autoregression models, achieving better 3D object detection performance.

Abstract: Linear modeling methods like Mamba have been merged as the effective backbone for the 3D object detection task. However, previous Mamba-based methods utilize the bidirectional encoding for the whole non-empty voxel sequence, which contains abundant useless background information in the scenes. Though directly encoding foreground voxels appears to be a plausible solution, it tends to degrade detection performance. We attribute this to the response attenuation and restricted context representation in the linear modeling for fore-only sequences. To address this problem, we propose a novel backbone, termed Fore-Mamba3D, to focus on the foreground enhancement by modifying Mamba-based encoder. The foreground voxels are first sampled according to the predicted scores. Considering the response attenuation existing in the interaction of foreground voxels across different instances, we design a regional-to-global slide window (RGSW) to propagate the information from regional split to the entire sequence. Furthermore, a semantic-assisted and state spatial fusion module (SASFMamba) is proposed to enrich contextual representation by enhancing semantic and geometric awareness within the Mamba model. Our method emphasizes foreground-only encoding and alleviates the distance-based and causal dependencies in the linear autoregression model. The superior performance across various benchmarks demonstrates the effectiveness of Fore-Mamba3D in the 3D object detection task.

Method: The framework injects learnable visual prompts into frozen MLLMs and optimizes a latent visual token modifier during inference using task-specific energy functions. It leverages cross-modal attention maps that encode semantic correspondences between text tokens and visual regions. Includes Optim++ for stability and PromptDebias to mitigate language prompt biases. Supports multiple visual prompt types (bounding boxes, masks, scribbles, points).

Result: Demonstrates strong out-of-domain generalization and interpretability. Enables fine-grained region-based visual reasoning without model retraining or fine-tuning. The code is publicly available.

Conclusion: ControlMLLM++ provides an effective test-time adaptation approach for steering MLLM attention to specific visual regions, offering interpretability and generalization while avoiding expensive model updates.

Abstract: We propose ControlMLLM++, a novel test-time adaptation framework that injects learnable visual prompts into frozen multimodal large language models (MLLMs) to enable fine-grained region-based visual reasoning without any model retraining or fine-tuning. Leveraging the insight that cross-modal attention maps intrinsically encode semantic correspondences between textual tokens and visual regions, ControlMLLM++ optimizes a latent visual token modifier during inference via a task-specific energy function to steer model attention towards user-specified areas. To enhance optimization stability and mitigate language prompt biases, ControlMLLM++ incorporates an improved optimization strategy (Optim++) and a prompt debiasing mechanism (PromptDebias). Supporting diverse visual prompt types including bounding boxes, masks, scribbles, and points, our method demonstrates strong out-of-domain generalization and interpretability. The code is available at https://github.com/mrwu-mac/ControlMLLM.

Method: Proposes relational feature caching (RFC) with two components: 1) relational feature estimation (RFE) that estimates output feature changes from inputs, and 2) relational cache scheduling (RCS) that uses input features to predict errors and selectively performs full computations.

Result: Extensive experiments across various DiT models show RFC consistently and significantly outperforms prior caching approaches in terms of efficiency and accuracy.

Conclusion: RFC effectively addresses limitations of temporal extrapolation methods by leveraging input-output relationships, providing a more accurate and efficient caching framework for diffusion transformers.

Abstract: Feature caching approaches accelerate diffusion transformers (DiTs) by storing the output features of computationally expensive modules at certain timesteps, and exploiting them for subsequent steps to reduce redundant computations. Recent forecasting-based caching approaches employ temporal extrapolation techniques to approximate the output features with cached ones. Although effective, relying exclusively on temporal extrapolation still suffers from significant prediction errors, leading to performance degradation. Through a detailed analysis, we find that 1) these errors stem from the irregular magnitude of changes in the output features, and 2) an input feature of a module is strongly correlated with the corresponding output. Based on this, we propose relational feature caching (RFC), a novel framework that leverages the input-output relationship to enhance the accuracy of the feature prediction. Specifically, we introduce relational feature estimation (RFE) to estimate the magnitude of changes in the output features from the inputs, enabling more accurate feature predictions. We also present relational cache scheduling (RCS), which estimates the prediction errors using the input features and performs full computations only when the errors are expected to be substantial. Extensive experiments across various DiT models demonstrate that RFC consistently outperforms prior approaches significantly. Project page is available at https://cvlab.yonsei.ac.kr/projects/RFC

Method: Proposes a Green Learning (GL) methodology for medical image restoration, characterized by mathematical transparency, computational efficiency, and small model size. Uses LDCT images as a specific application case.

Result: The GL method achieves state-of-the-art restoration performance with smaller model size and lower inference complexity compared to existing approaches.

Conclusion: Green Learning offers an effective alternative to deep learning for medical image restoration, providing high performance with mathematical transparency and computational efficiency.

Abstract: This work proposes a green learning (GL) approach to restore medical images. Without loss of generality, we use low-dose computed tomography (LDCT) images as examples. LDCT images are susceptible to noise and artifacts, where the imaging process introduces distortion. LDCT image restoration is an important preprocessing step for further medical analysis. Deep learning (DL) methods have been developed to solve this problem. We examine an alternative solution using the Green Learning (GL) methodology. The new restoration method is characterized by mathematical transparency, computational and memory efficiency, and high performance. Experiments show that our GL method offers state-of-the-art restoration performance at a smaller model size and with lower inference complexity.

Method: Two-stage strategy: First stage uses high-authenticity method to generate reasonable foreground shape as condition; second stage uses high-fidelity method with the shape condition to preserve foreground details accurately.

Result: Experiments on MureCOM dataset verify the effectiveness of the two-stage strategy. The method achieves both authenticity and fidelity in composite image generation.

Conclusion: The proposed two-stage approach successfully combines the strengths of both high-authenticity and high-fidelity methods for generative image composition, addressing the limitation of existing methods that can only achieve one goal at a time.

Abstract: Generative image composition aims to regenerate the given foreground object in the background image to produce a realistic composite image. Some high-authenticity methods can adjust foreground pose/view to be compatible with background, while some high-fidelity methods can preserve the foreground details accurately. However, existing methods can hardly achieve both goals at the same time. In this work, we propose a two-stage strategy to achieve both goals. In the first stage, we use high-authenticity method to generate reasonable foreground shape, serving as the condition of high-fidelity method in the second stage. The experiments on MureCOM dataset verify the effectiveness of our two-stage strategy. The code and model have been released at https://github.com/bcmi/OSInsert-Image-Composition.

Method: ORION fine-tunes text encoders using only class names via low-rank adaptation. It optimizes a novel loss with two terms: 1) pairwise orthogonality between class representations, and 2) penalizing deviations from initial prototypes. Provides probabilistic interpretation connecting orthogonality penalty to MLE via Huygens theorem.

Result: Extensive experiments on 11 benchmarks with three large VLM backbones show refined textual embeddings outperform standard CLIP prototypes. ORION consistently improves performance across zero-shot, few-shot, and test-time adaptation settings when added as plug-and-play module to state-of-the-art methods.

Conclusion: ORION effectively improves VLM performance by optimizing textual class representations through orthogonality constraints and prototype preservation, requiring only class names and working across various prediction settings.

Abstract: Vision language models (VLMs) have demonstrated remarkable generalization across diverse tasks, yet their performance remains constrained by the quality and geometry of the textual prototypes used to represent classes. Standard zero shot classifiers, derived from frozen text encoders and handcrafted prompts, may yield correlated or weakly separated embeddings that limit task specific discriminability. We introduce ORION, a text encoder fine tuning framework that improves pretrained VLMs using only class names. Our method optimizes, via low rank adaptation, a novel loss integrating two terms, one promoting pairwise orthogonality between the textual representations of the classes of a given task and the other penalizing deviations from the initial class prototypes. Furthermore, we provide a probabilistic interpretation of our orthogonality penalty, connecting it to the general maximum likelihood estimation (MLE) principle via Huygens theorem. We report extensive experiments on 11 benchmarks and three large VLM backbones, showing that the refined textual embeddings yield powerful replacements for the standard CLIP prototypes. Added as plug and play module on top of various state of the art methods, and across different prediction settings (zero shot, few shot and test time adaptation), ORION improves the performance consistently and significantly.

Method: Formulates pose estimation as conditional discrete diffusion process; uses flexible flow decider to balance real/noise distributions; incorporates hierarchical kinematic coupling to respect object structure; progressively denoises noisy pose representations.

Result: Superior performance and robustness demonstrated on both synthetic and real-world datasets for category-level 6D pose estimation.

Conclusion: DICArt offers a new paradigm for reliable articulated object pose estimation by integrating discrete generative modeling with structural priors.

Abstract: Articulated object pose estimation is a core task in embodied AI. Existing methods typically regress poses in a continuous space, but often struggle with 1) navigating a large, complex search space and 2) failing to incorporate intrinsic kinematic constraints. In this work, we introduce DICArt (DIsCrete Diffusion for Articulation Pose Estimation), a novel framework that formulates pose estimation as a conditional discrete diffusion process. Instead of operating in a continuous domain, DICArt progressively denoises a noisy pose representation through a learned reverse diffusion procedure to recover the GT pose. To improve modeling fidelity, we propose a flexible flow decider that dynamically determines whether each token should be denoised or reset, effectively balancing the real and noise distributions during diffusion. Additionally, we incorporate a hierarchical kinematic coupling strategy, estimating the pose of each rigid part hierarchically to respect the object’s kinematic structure. We validate DICArt on both synthetic and real-world datasets. Experimental results demonstrate its superior performance and robustness. By integrating discrete generative modeling with structural priors, DICArt offers a new paradigm for reliable category-level 6D pose estimation in complex environments.

Method: Simulated physical attacks on 329 facial images (ages 10-21) using a VLM image editor (Gemini 2.5 Flash Image), evaluated eight models (5 specialized age estimators + 3 vision-language models), introduced Attack Conversion Rate metric to measure vulnerability.

Result: Synthetic beard alone achieved 28-69% ACR across models; combined attacks shifted predicted age by +7.7 years on average and reached up to 83% ACR; VLMs showed lower ACR (59-71%) than specialized models (63-83%) but difference not statistically tested.

Conclusion: Age verification systems have critical vulnerabilities to cosmetic modifications, requiring adversarial robustness evaluation as mandatory for model selection to prevent minors from accessing age-restricted content.

Abstract: Age estimation systems are increasingly deployed as gatekeepers for age-restricted online content, yet their robustness to cosmetic modifications has not been systematically evaluated. We investigate whether simple, household-accessible cosmetic changes, including beards, grey hair, makeup, and simulated wrinkles, can cause AI age estimators to classify minors as adults. To study this threat at scale without ethical concerns, we simulate these physical attacks on 329 facial images of individuals aged 10 to 21 using a VLM image editor (Gemini 2.5 Flash Image). We then evaluate eight models from our prior benchmark: five specialized architectures (MiVOLO, Custom-Best, Herosan, MiViaLab, DEX) and three vision-language models (Gemini 3 Flash, Gemini 2.5 Flash, GPT-5-Nano). We introduce the Attack Conversion Rate (ACR), defined as the fraction of images predicted as minor at baseline that flip to adult after attack, a population-agnostic metric that does not depend on the ratio of minors to adults in the test set. Our results reveal that a synthetic beard alone achieves 28 to 69 percent ACR across all eight models; combining all four attacks shifts predicted age by +7.7 years on average across all 329 subjects and reaches up to 83 percent ACR; and vision-language models exhibit lower ACR (59 to 71 percent) than specialized models (63 to 83 percent) under the full attack, although the ACR ranges overlap and the difference is not statistically tested. These findings highlight a critical vulnerability in deployed age-verification pipelines and call for adversarial robustness evaluation as a mandatory criterion for model selection.

Method: Compare CNN classifiers (ImageNet-pretrained) on raw RGB patches vs traditional ML on handcrafted spectral/texture features and remote-sensing foundation model embeddings. Use curated dataset of 1,943 sites (898 looted, 1,045 preserved) with multi-year PlanetScope imagery and site-footprint masks.

Result: ImageNet-pretrained CNNs with spatial masking achieve F1=0.926, significantly outperforming traditional ML (F1=0.710). ImageNet pretraining and spatial masking enhance performance, while geospatial foundation models perform similarly to handcrafted features.

Conclusion: Scalable satellite-based pipeline effectively detects looted archaeological sites. ImageNet pretraining and spatial masking are crucial for high performance, suggesting looting signatures are extremely localized.

Abstract: Looting at archaeological sites poses a severe risk to cultural heritage, yet monitoring thousands of remote locations remains operationally difficult. We present a scalable and satellite-based pipeline to detect looted archaeological sites, using PlanetScope monthly mosaics (4.7m/pixel) and a curated dataset of 1,943 archaeological sites in Afghanistan (898 looted, 1,045 preserved) with multi-year imagery (2016–2023) and site-footprint masks. We compare (i) end-to-end CNN classifiers trained on raw RGB patches and (ii) traditional machine learning (ML) trained on handcrafted spectral/texture features and embeddings from recent remote-sensing foundation models. Results indicate that ImageNet-pretrained CNNs combined with spatial masking reach an F1 score of 0.926, clearly surpassing the strongest traditional ML setup, which attains an F1 score of 0.710 using SatCLIP-V+RF+Mean, i.e., location and vision embeddings fed into a Random Forest with mean-based temporal aggregation. Ablation studies demonstrate that ImageNet pretraining (even in the presence of domain shift) and spatial masking enhance performance. In contrast, geospatial foundation model embeddings perform competitively with handcrafted features, suggesting that looting signatures are extremely localized. The repository is available at https://github.com/microsoft/looted_site_detection.

Method: Uses multimodal LLM to analyze 3D assets and prompts, generates decomposed structural/appearance guidance, selects informative views, applies image editing for visual guidance, then uses inversion-based rectified-flow inpainting with interleaved sampling in 3D latent space.

Result: Outperforms prior baselines in automatic metrics and human preference studies, enabling precise, coherent, and mask-free 3D editing across diverse edits.

Conclusion: Vinedresser3D provides an effective agentic framework for high-quality text-guided 3D editing that addresses key limitations of existing methods through multimodal understanding and latent space manipulation.

Abstract: Text-guided 3D editing aims to modify existing 3D assets using natural-language instructions. Current methods struggle to jointly understand complex prompts, automatically localize edits in 3D, and preserve unedited content. We introduce Vinedresser3D, an agentic framework for high-quality text-guided 3D editing that operates directly in the latent space of a native 3D generative model. Given a 3D asset and an editing prompt, Vinedresser3D uses a multimodal large language model to infer rich descriptions of the original asset, identify the edit region and edit type (addition, modification, deletion), and generate decomposed structural and appearance-level text guidance. The agent then selects an informative view and applies an image editing model to obtain visual guidance. Finally, an inversion-based rectified-flow inpainting pipeline with an interleaved sampling module performs editing in the 3D latent space, enforcing prompt alignment while maintaining 3D coherence and unedited regions. Experiments on diverse 3D edits demonstrate that Vinedresser3D outperforms prior baselines in both automatic metrics and human preference studies, while enabling precise, coherent, and mask-free 3D editing.

Method: Introduces PedaCo-Gen system with an Intermediate Representation (IR) phase where educators interactively review and refine video blueprints (scripts + visual descriptions) with an AI reviewer based on CTML principles. Moves away from traditional “one-shot” generation to iterative co-creation.

Result: Study with 23 education experts shows PedaCo-Gen significantly enhances video quality across topics and CTML principles compared to baselines. Participants reported high production efficiency (M=4.26) and guide validity (M=4.04), perceiving AI guidance as metacognitive scaffolding.

Conclusion: The system demonstrates the importance of reclaiming pedagogical agency through principled co-creation, providing a foundation for future AI authoring tools that harmonize generative power with human professional expertise in educational contexts.

Abstract: While advancements in Text-to-Video (T2V) generative AI offer a promising path toward democratizing content creation, current models are often optimized for visual fidelity rather than instructional efficacy. This study introduces PedaCo-Gen, a pedagogically-informed human-AI collaborative video generating system for authoring instructional videos based on Mayer’s Cognitive Theory of Multimedia Learning (CTML). Moving away from traditional “one-shot” generation, PedaCo-Gen introduces an Intermediate Representation (IR) phase, enabling educators to interactively review and refine video blueprints-comprising scripts and visual descriptions-with an AI reviewer. Our study with 23 education experts demonstrates that PedaCo-Gen significantly enhances video quality across various topics and CTML principles compared to baselines. Participants perceived the AI-driven guidance not merely as a set of instructions but as a metacognitive scaffold that augmented their instructional design expertise, reporting high production efficiency (M=4.26) and guide validity (M=4.04). These findings highlight the importance of reclaiming pedagogical agency through principled co-creation, providing a foundation for future AI authoring tools that harmonize generative power with human professional expertise.

Method: Two-stage detection: 1) quick filtering via image consistency under content-preserving transformations, 2) text-embedding space discrepancy analysis, and 3) when necessary, using a powerful LLM to resolve attack-induced divergences with agentic data consolidation of multiple responses.

Result: Achieves state-of-the-art accuracy while maintaining notable efficiency - most clean images skip costly processing, and overhead remains minimal even with numerous adversarial examples.

Conclusion: The proposed training-free defense effectively protects LVLMs from adversarial attacks through efficient multi-stage detection and agentic data consolidation, balancing accuracy and computational efficiency.

Abstract: Large Vision-Language Models (LVLMs) can be vulnerable to adversarial images that subtly bias their outputs toward plausible yet incorrect responses. We introduce a general, efficient, and training-free defense that combines image transformations with agentic data consolidation to recover correct model behavior. A key component of our approach is a two-stage detection mechanism that quickly filters out the majority of clean inputs. We first assess image consistency under content-preserving transformations at negligible computational cost. For more challenging cases, we examine discrepancies in a text-embedding space. Only when necessary do we invoke a powerful LLM to resolve attack-induced divergences. A key idea is to consolidate multiple responses, leveraging both their similarities and their differences. We show that our method achieves state-of-the-art accuracy while maintaining notable efficiency: most clean images skip costly processing, and even in the presence of numerous adversarial examples, the overhead remains minimal.

Method: Created HOCA-Bench using Hegelian philosophy to categorize anomalies into ontological (entity definition/persistence violations) and causal (physical relation violations). Used state-of-the-art generative video models as adversarial simulators to generate 1,439 videos with 3,470 QA pairs. Evaluated 17 Video-LLMs on this benchmark.

Result: Models show clear cognitive lag: perform better on static ontological violations (e.g., shape mutations) but struggle with causal mechanisms (e.g., gravity, friction), with performance dropping >20% on causal tasks. System-2 “Thinking” modes improve reasoning but don’t close the gap.

Conclusion: Current Video-LLM architectures recognize visual patterns more readily than they apply basic physical laws, highlighting a fundamental limitation in their world modeling capabilities for physically grounded intelligence.

Abstract: Video-LLMs have improved steadily on semantic perception, but they still fall short on predictive world modeling, which is central to physically grounded intelligence. We introduce HOCA-Bench, a benchmark that frames physical anomalies through a Hegelian lens. HOCA-Bench separates anomalies into two types: ontological anomalies, where an entity violates its own definition or persistence, and causal anomalies, where interactions violate physical relations. Using state-of-the-art generative video models as adversarial simulators, we build a testbed of 1,439 videos (3,470 QA pairs). Evaluations on 17 Video-LLMs show a clear cognitive lag: models often identify static ontological violations (e.g., shape mutations) but struggle with causal mechanisms (e.g., gravity or friction), with performance dropping by more than 20% on causal tasks. System-2 “Thinking” modes improve reasoning, but they do not close the gap, suggesting that current architectures recognize visual patterns more readily than they apply basic physical laws.

Method: High-Level Representation Misdirection (HiRM) misdirects high-level semantic representations of target concepts in the text encoder toward designated vectors (random or semantically defined directions), updating only early layers that contain causal states of visual attributes, enabling precise concept removal with minimal impact on unrelated concepts.

Result: Strong performance on UnlearnCanvas and NSFW benchmarks across diverse targets (objects, styles, nudity), preserves generative utility at low training cost, transfers to state-of-the-art architectures like Flux without additional training, and shows synergistic effects with denoiser-based concept erasing methods.

Conclusion: HiRM provides an effective approach for concept erasure in text-to-image models by targeting text encoder representations, offering precise concept removal with minimal degradation of non-target concepts and good transferability across architectures.

Abstract: Recent advances in text-to-image (T2I) diffusion models have seen rapid and widespread adoption. However, their powerful generative capabilities raise concerns about potential misuse for synthesizing harmful, private, or copyrighted content. To mitigate such risks, concept erasure techniques have emerged as a promising solution. Prior works have primarily focused on fine-tuning the denoising component (e.g., the U-Net backbone). However, recent causal tracing studies suggest that visual attribute information is localized in the early self-attention layers of the text encoder, indicating a potential alternative for concept erasing. Building on this insight, we conduct preliminary experiments and find that directly fine-tuning early layers can suppress target concepts but often degrades the generation quality of non-target concepts. To overcome this limitation, we propose High-Level Representation Misdirection (HiRM), which misdirects high-level semantic representations of target concepts in the text encoder toward designated vectors such as random directions or semantically defined directions (e.g., supercategories), while updating only early layers that contain causal states of visual attributes. Our decoupling strategy enables precise concept removal with minimal impact on unrelated concepts, as demonstrated by strong results on UnlearnCanvas and NSFW benchmarks across diverse targets (e.g., objects, styles, nudity). HiRM also preserves generative utility at low training cost, transfers to state-of-the-art architectures such as Flux without additional training, and shows synergistic effects with denoiser-based concept erasing methods.

Method: ConceptPrism automatically disentangles shared visual concepts from image-specific residuals by comparing images within a set. The method jointly optimizes a target token and image-wise residual tokens using two complementary objectives: reconstruction loss for fidelity, and a novel exclusion loss that compels residual tokens to discard the shared concept, allowing the target token to capture pure concept without direct supervision.

Result: Extensive experiments demonstrate that ConceptPrism effectively resolves concept entanglement, achieving significantly improved trade-off between fidelity and alignment compared to existing methods.

Conclusion: ConceptPrism provides an automatic framework for disentangling visual concepts in personalized text-to-image generation, overcoming limitations of manual guidance approaches and improving the fidelity-alignment trade-off.

Abstract: Personalized text-to-image generation suffers from concept entanglement, where irrelevant residual information from reference images is captured, leading to a trade-off between concept fidelity and text alignment. Recent disentanglement approaches attempt to solve this utilizing manual guidance, such as linguistic cues or segmentation masks, which limits their applicability and fails to fully articulate the target concept. In this paper, we propose ConceptPrism, a novel framework that automatically disentangles the shared visual concept from image-specific residuals by comparing images within a set. Our method jointly optimizes a target token and image-wise residual tokens using two complementary objectives: a reconstruction loss to ensure fidelity, and a novel exclusion loss that compels residual tokens to discard the shared concept. This process allows the target token to capture the pure concept without direct supervision. Extensive experiments demonstrate that ConceptPrism effectively resolves concept entanglement, achieving a significantly improved trade-off between fidelity and alignment.

Method: Proposes MVIG attack framework with: 1) Mutual View Information Graph (MVIG) representation to capture vulnerability knowledge, 2) Temporal graph learning for evolving fabrication risk maps, 3) Entropy-aware vulnerability search to optimize attack location, timing, and persistence.

Result: MVIG attack reduces defense success rates by up to 62% against state-of-the-art defenses, achieves 47% lower detection for persistent attacks at 29.9 FPS on OPV2V and Adv-OPV2V datasets.

Conclusion: The MVIG attack exposes critical security gaps in collaborative perception systems, demonstrating that current defenses remain vulnerable to adaptive adversarial attacks that exploit disclosed vulnerability knowledge.

Abstract: Collaborative perception (CP) enables data sharing among connected and autonomous vehicles (CAVs) to enhance driving safety. However, CP systems are vulnerable to adversarial attacks where malicious agents forge false objects via feature-level perturbations. Current defensive systems use threshold-based consensus verification by comparing collaborative and ego detection results. Yet, these defenses remain vulnerable to more sophisticated attack strategies that could exploit two critical weaknesses: (i) lack of robustness against attacks with systematic timing and target region optimization, and (ii) inadvertent disclosure of vulnerability knowledge through implicit confidence information in shared collaboration data. In this paper, we propose MVIG attack, a novel adaptive adversarial CP framework learning to capture vulnerability knowledge disclosed by different defensive CP systems from a unified mutual view information graph (MVIG) representation. Our approach combines MVIG representation with temporal graph learning to generate evolving fabrication risk maps and employs entropy-aware vulnerability search to optimize attack location, timing and persistence, enabling adaptive attacks with generalizability across various defensive configurations. Extensive evaluations on OPV2V and Adv-OPV2V datasets demonstrate that MVIG attack reduces defense success rates by up to 62% against state-of-the-art defenses while achieving 47% lower detection for persistent attacks at 29.9 FPS, exposing critical security gaps in CP systems. Code will be released at https://github.com/yihangtao/MVIG.git

Method: TeHOR uses two core designs: 1) leverages text descriptions of human-object interactions for semantic alignment between 3D reconstruction and textual cues, enabling reasoning over both contact and non-contact interactions; 2) incorporates appearance cues of 3D human and object into alignment process to capture holistic contextual information.

Result: The framework produces accurate and semantically coherent reconstructions, achieving state-of-the-art performance in joint 3D human-object reconstruction from single images.

Conclusion: TeHOR addresses fundamental limitations of existing approaches by incorporating semantic text alignment and appearance cues, enabling more comprehensive understanding of human-object interactions beyond just physical contact.

Abstract: Joint reconstruction of 3D human and object from a single image is an active research area, with pivotal applications in robotics and digital content creation. Despite recent advances, existing approaches suffer from two fundamental limitations. First, their reconstructions rely heavily on physical contact information, which inherently cannot capture non-contact human-object interactions, such as gazing at or pointing toward an object. Second, the reconstruction process is primarily driven by local geometric proximity, neglecting the human and object appearances that provide global context crucial for understanding holistic interactions. To address these issues, we introduce TeHOR, a framework built upon two core designs. First, beyond contact information, our framework leverages text descriptions of human-object interactions to enforce semantic alignment between the 3D reconstruction and its textual cues, enabling reasoning over a wider spectrum of interactions, including non-contact cases. Second, we incorporate appearance cues of the 3D human and object into the alignment process to capture holistic contextual information, thereby ensuring visually plausible reconstructions. As a result, our framework produces accurate and semantically coherent reconstructions, achieving state-of-the-art performance.

Method: RAID retrieves class-, semantic-, and instance-level representations from a hierarchical vector database in a coarse-to-fine pipeline. It uses a matching cost volume to correlate input with retrieved exemplars, followed by a guided Mixture-of-Experts (MoE) network that leverages retrieved samples to adaptively suppress matching noise and produce fine-grained anomaly maps.

Result: RAID achieves state-of-the-art performance across full-shot, few-shot, and multi-dataset settings on MVTec, VisA, MPDD, and BTAD benchmarks.

Conclusion: The paper successfully reinterprets UAD through the lens of RAG, introducing a noise-resilient framework that effectively uses retrieved normal samples to guide anomaly detection and localization, outperforming existing methods.

Abstract: Unsupervised Anomaly Detection (UAD) aims to identify abnormal regions by establishing correspondences between test images and normal templates. Existing methods primarily rely on image reconstruction or template retrieval but face a fundamental challenge: matching between test images and normal templates inevitably introduces noise due to intra-class variations, imperfect correspondences, and limited templates. Observing that Retrieval-Augmented Generation (RAG) leverages retrieved samples directly in the generation process, we reinterpret UAD through this lens and introduce \textbf{RAID}, a retrieval-augmented UAD framework designed for noise-resilient anomaly detection and localization. Unlike standard RAG that enriches context or knowledge, we focus on using retrieved normal samples to guide noise suppression in anomaly map generation. RAID retrieves class-, semantic-, and instance-level representations from a hierarchical vector database, forming a coarse-to-fine pipeline. A matching cost volume correlates the input with retrieved exemplars, followed by a guided Mixture-of-Experts (MoE) network that leverages the retrieved samples to adaptively suppress matching noise and produce fine-grained anomaly maps. RAID achieves state-of-the-art performance across full-shot, few-shot, and multi-dataset settings on MVTec, VisA, MPDD, and BTAD benchmarks. \href{https://github.com/Mingxiu-Cai/RAID}{https://github.com/Mingxiu-Cai/RAID}.

Method: Learn multi-modal class embeddings for rare objects using vision foundation models and synonym-augmented text descriptions. Use lightweight attention-based enhancement module to refine visual tokens, and generate object-aware hints injected into text prompts.

Result: Experiments on two benchmarks show consistent and substantial gains for pretrained VLMs in rare object recognition and reasoning. Method strengthens VLM’s ability to focus on and reason about rare objects.

Conclusion: Efficient plug-and-play module significantly improves VLMs’ rare object reasoning without finetuning, addressing data scarcity issues through multi-modal embeddings and prompt enhancement.

Abstract: Vision language models (VLMs) have achieved remarkable success in broad visual understanding, yet they remain challenged by object-centric reasoning on rare objects due to the scarcity of such instances in pretraining data. While prior efforts alleviate this issue by retrieving additional data or introducing stronger vision encoders, these methods are still computationally intensive during finetuning VLMs and don’t fully exploit the original training data. In this paper, we introduce an efficient plug-and-play module that substantially improves VLMs’ reasoning over rare objects by refining visual tokens and enriching input text prompts, without VLMs finetuning. Specifically, we propose to learn multi-modal class embeddings for rare objects by leveraging prior knowledge from vision foundation models and synonym-augmented text descriptions, compensating for limited training examples. These embeddings refine the visual tokens in VLMs through a lightweight attention-based enhancement module that improves fine-grained object details. In addition, we use the learned embeddings as object-aware detectors to generate informative hints, which are injected into the text prompts to help guide the VLM’s attention toward relevant image regions. Experiments on two benchmarks show consistent and substantial gains for pretrained VLMs in rare object recognition and reasoning. Further analysis reveals how our method strengthens the VLM’s ability to focus on and reason about rare objects.

Method: SAM-H estimates homographies from segmentation mask contours using SAM 2, while WOFTSAM enhances WOFT tracker by incorporating SAM-H’s lost target re-detection capabilities.

Result: The methods achieve state-of-the-art performance on POT-210 and PlanarTrack benchmarks, outperforming second-best by +12.4 and +15.2pp on p@15 metric. Also provides improved ground-truth annotations for PlanarTrack.

Conclusion: The proposed SAM-H and WOFTSAM trackers significantly advance planar tracking performance through robust segmentation-based homography estimation and improved re-detection mechanisms.

Abstract: We present SAM-H and WOFTSAM, novel planar trackers that combine robust long-term segmentation tracking provided by SAM 2 with 8 degrees-of-freedom homography pose estimation. SAM-H estimates homographies from segmentation mask contours and is thus highly robust to target appearance changes. WOFTSAM significantly improves the current state-of-the-art planar tracker WOFT by exploiting lost target re-detection provided by SAM-H. The proposed methods are evaluated on POT-210 and PlanarTrack tracking benchmarks, setting the new state-of-the-art performance on both. On the latter, they outperform the second best by a large margin, +12.4 and +15.2pp on the p@15 metric. We also present improved ground-truth annotations of initial PlanarTrack poses, enabling more accurate benchmarking in the high-precision p@5 metric. The code and the re-annotations are available at https://github.com/serycjon/WOFTSAM

Method: Proposes Federated Temporal Adaptation (FTA) setting and FedTAR framework that integrates demographic-driven personalization (generating lightweight LoRA adapters from demographic embeddings) with time-aware global aggregation using temporal residual aggregation weighted by a meta-learned temporal policy optimized via first-order MAML.

Result: Experiments on J-MID (1M exams) and MIMIC-CXR show consistent improvements in linguistic accuracy, temporal coherence, and cross-site generalization compared to existing methods.

Conclusion: FedTAR establishes a robust and privacy-preserving paradigm for federated longitudinal modeling in medical report generation, effectively addressing temporal evolution and patient heterogeneity.

Abstract: Longitudinal medical report generation is clinically important yet remains challenging due to strict privacy constraints and the evolving nature of disease progression. Although federated learning (FL) enables collaborative training without data sharing, existing FL methods largely overlook longitudinal dynamics by assuming stationary client distributions, making them unable to model temporal shifts across visits or patient-specific heterogeneity-ultimately leading to unstable optimization and suboptimal report generation. We introduce Federated Temporal Adaptation (FTA), a federated setting that explicitly accounts for the temporal evolution of client data. Building upon this setting, we propose FedTAR, a framework that integrates demographic-driven personalization with time-aware global aggregation. FedTAR generates lightweight LoRA adapters from demographic embeddings and performs temporal residual aggregation, where updates from different visits are weighted by a meta-learned temporal policy optimized via first-order MAML. Experiments on J-MID (1M exams) and MIMIC-CXR demonstrate consistent improvements in linguistic accuracy, temporal coherence, and cross-site generalization, establishing FedTAR as a robust and privacy-preserving paradigm for federated longitudinal modeling.

Method: Uses rough TSDF reconstruction to create adaptive narrow-band domain, then fuses depth observations via heteroscedastic Bayesian formulation solved with sparse linear algebra and preconditioned conjugate gradients. Employs randomized diagonal estimators for efficient posterior uncertainty approximation.

Result: Outperforms TSDF baselines in geometric accuracy on controlled ablation scene and CO3D object sequence, provides useful uncertainty estimates for active sensing applications.

Conclusion: Provides interpretable, CPU-efficient alternative to GPU-heavy neural reconstruction methods with probabilistic understanding and predictable behavior for robotics and AR applications.

Abstract: Key part of robotics, augmented reality, and digital inspection is dense 3D reconstruction from depth observations. Traditional volumetric fusion techniques, including truncated signed distance functions (TSDF), enable efficient and deterministic geometry reconstruction; however, they depend on heuristic weighting and fail to transparently convey uncertainty in a systematic way. Recent neural implicit methods, on the other hand, get very high fidelity but usually need a lot of GPU power for optimization and aren’t very easy to understand for making decisions later on. This work presents BayesFusion-SDF, a CPU-centric probabilistic signed distance fusion framework that conceptualizes geometry as a sparse Gaussian random field with a defined posterior distribution over voxel distances. First, a rough TSDF reconstruction is used to create an adaptive narrow-band domain. Then, depth observations are combined using a heteroscedastic Bayesian formulation that is solved using sparse linear algebra and preconditioned conjugate gradients. Randomized diagonal estimators are a quick way to get an idea of posterior uncertainty. This makes it possible to extract surfaces and plan the next best view while taking into account uncertainty. Tests on a controlled ablation scene and a CO3D object sequence show that the new method is more accurate geometrically than TSDF baselines and gives useful estimates of uncertainty for active sensing. The proposed formulation provides a clear and easy-to-use alternative to GPU-heavy neural reconstruction methods while still being able to be understood in a probabilistic way and acting in a predictable way. GitHub: https://mazumdarsoumya.github.io/BayesFusionSDF

Method: Combines text-guided diffusion models with SDEdit refinement for training-free enhancement of existing HDR reconstruction methods. Uses iterative compensation mechanism to ensure luminance coherence across multiple exposures and integrates seamlessly with existing pipelines.

Result: Demonstrates significant improvements in both perceptual quality and quantitative metrics on standard HDR datasets and in-the-wild captures. Effectively recovers natural details in challenging scenarios while preserving advantages of existing HDR reconstruction pipelines.

Conclusion: Presents a training-free approach that successfully enhances HDR reconstruction by addressing over-exposed region challenges through diffusion-based inpainting, offering a practical solution that integrates with existing methods without requiring extensive training.

Abstract: Single LDR to HDR reconstruction remains challenging for over-exposed regions where traditional methods often fail due to complete information loss. We present a training-free approach that enhances existing indirect and direct HDR reconstruction methods through diffusion-based inpainting. Our method combines text-guided diffusion models with SDEdit refinement to generate plausible content in over-exposed areas while maintaining consistency across multi-exposure LDR images. Unlike previous approaches requiring extensive training, our method seamlessly integrates with existing HDR reconstruction techniques through an iterative compensation mechanism that ensures luminance coherence across multiple exposures. We demonstrate significant improvements in both perceptual quality and quantitative metrics on standard HDR datasets and in-the-wild captures. Results show that our method effectively recovers natural details in challenging scenarios while preserving the advantages of existing HDR reconstruction pipelines. Project page: https://github.com/EusdenLin/HDR-Reconstruction-Boosting

Method: Separates adapter into class-shared LoRA A for class priors and per-image LoRAs B for image-specific characteristics. Uses semantic boosting by preserving class bounding boxes during training to expose coherent class semantics. For generation, composes A with a mixture of B using coefficients from Dirichlet distribution.

Result: Across diverse datasets, synthesized images are both diverse and detail-rich while closely aligning with few-shot real distribution. Yields robust gains in downstream classification accuracy compared to existing approaches.

Conclusion: The hybrid LoRA approach effectively addresses data scarcity in specialized domains by combining class priors with image-specific details, improving few-shot generation quality and downstream task performance.

Abstract: Beyond general recognition tasks, specialized domains including privacy-constrained medical applications and fine-grained settings often encounter data scarcity, especially for tail classes. To obtain less biased and more reliable models under such scarcity, practitioners leverage diffusion models to supplement underrepresented regions of real data. Specifically, recent studies fine-tune pretrained diffusion models with LoRA on few-shot real sets to synthesize additional images. While an image-wise LoRA trained on a single image captures fine-grained details yet offers limited diversity, a class-wise LoRA trained over all shots produces diverse images as it encodes class priors yet tends to overlook fine details. To combine both benefits, we separate the adapter into a class-shared LoRA~$A$ for class priors and per-image LoRAs~$\mathcal{B}$ for image-specific characteristics. To expose coherent class semantics in the shared LoRA~$A$, we propose a semantic boosting by preserving class bounding boxes during training. For generation, we compose $A$ with a mixture of $\mathcal{B}$ using coefficients drawn from a Dirichlet distribution. Across diverse datasets, our synthesized images are both diverse and detail-rich while closely aligning with the few-shot real distribution, yielding robust gains in downstream classification accuracy.

Method: Two-stage framework: 1) Structure-guided cross-modal generation network synthesizes diverse ultrasound images from unpaired MRI data while preserving anatomical junctions; 2) Lightweight screening network uses gradient distillation to transfer knowledge from a high-capacity teacher model to guide sparse attention to critical regions.

Result: Achieved 99.5% sensitivity, 97.2% specificity, and AUC of 0.987 on a multicenter cohort of 7,951 participants with minimal computational cost (0.289 GFLOPs), substantially outperforming expert sonographers.

Conclusion: Combining cross-modal synthetic augmentation with knowledge-driven efficient modeling can democratize expert-level, real-time cancer screening for resource-constrained primary care settings.

Abstract: Early detection of myometrial invasion is critical for the staging and life-saving management of endometrial carcinoma (EC), a prevalent global malignancy. Transvaginal ultrasound serves as the primary, accessible screening modality in resource-constrained primary care settings; however, its diagnostic reliability is severely hindered by low tissue contrast, high operator dependence, and a pronounced scarcity of positive pathological samples. Existing artificial intelligence solutions struggle to overcome this severe class imbalance and the subtle imaging features of invasion, particularly under the strict computational limits of primary care clinics. Here we present an automated, highly efficient two-stage deep learning framework that resolves both data and computational bottlenecks in EC screening. To mitigate pathological data scarcity, we develop a structure-guided cross-modal generation network that synthesizes diverse, high-fidelity ultrasound images from unpaired magnetic resonance imaging (MRI) data, strictly preserving clinically essential anatomical junctions. Furthermore, we introduce a lightweight screening network utilizing gradient distillation, which transfers discriminative knowledge from a high-capacity teacher model to dynamically guide sparse attention towards task-critical regions. Evaluated on a large, multicenter cohort of 7,951 participants, our model achieves a sensitivity of 99.5%, a specificity of 97.2%, and an area under the curve of 0.987 at a minimal computational cost (0.289 GFLOPs), substantially outperforming the average diagnostic accuracy of expert sonographers. Our approach demonstrates that combining cross-modal synthetic augmentation with knowledge-driven efficient modeling can democratize expert-level, real-time cancer screening for resource-constrained primary care settings.

Method: Two-stage approach: 1) Pre-training phase extracts universal 3D spatial priors in camera-centric space using discrete pose tokens, 2) Post-training phase aligns with robot-specific action space. Integrates spatial grounding from 3D datasets with geometry-level trajectories from robotic demonstrations.

Result: Achieves SOTA on RoboTwin 2.0 (79.5% avg success) and competitive performance on LIBERO (96.0%). Real-world experiments show robust generalization with only 100 demonstrations per task.

Conclusion: Pose-VLA’s decoupled paradigm effectively addresses VLA misalignments by separating spatial grounding from embodiment alignment, enabling efficient training and strong generalization in robotic tasks.

Abstract: Existing Vision-Language-Action (VLA) models often suffer from feature collapse and low training efficiency because they entangle high-level perception with sparse, embodiment-specific action supervision. Since these models typically rely on VLM backbones optimized for Visual Question Answering (VQA), they excel at semantic identification but often overlook subtle 3D state variations that dictate distinct action patterns. To resolve these misalignments, we propose Pose-VLA, a decoupled paradigm that separates VLA training into a pre-training phase for extracting universal 3D spatial priors in a unified camera-centric space, and a post-training phase for efficient embodiment alignment within robot-specific action space. By introducing discrete pose tokens as a universal representation, Pose-VLA seamlessly integrates spatial grounding from diverse 3D datasets with geometry-level trajectories from robotic demonstrations. Our framework follows a two-stage pre-training pipeline, establishing fundamental spatial grounding via poses followed by motion alignment through trajectory supervision. Extensive evaluations demonstrate that Pose-VLA achieves state-of-the-art results on RoboTwin 2.0 with a 79.5% average success rate and competitive performance on LIBERO at 96.0%. Real-world experiments further showcase robust generalization across diverse objects using only 100 demonstrations per task, validating the efficiency of our pre-training paradigm.

Method: Proposes DeepfakeJudge framework with: 1) OOD benchmark with recent generative/editing forgeries, 2) Human-annotated subset with visual reasoning labels, 3) Evaluation models that assess reasoning rationales without ground truth, 4) Bootstrapped generator-evaluator process scaling human feedback into structured reasoning supervision.

Result: Reasoning-bootstrapped model achieves 96.2% accuracy, outperforming 30x larger baselines. Reasoning judge attains high correlation with human ratings and 98.9% pairwise agreement on human-annotated subset. User study shows 70% preference for framework’s reasoning in faithfulness, groundedness, and usefulness.

Conclusion: Establishes reasoning fidelity as quantifiable dimension of deepfake detection and demonstrates scalable supervision for interpretable deepfake reasoning. All datasets, models, and code are open-sourced.

Abstract: Deepfake detection models often generate natural-language explanations, yet their reasoning is frequently ungrounded in visual evidence, limiting reliability. Existing evaluations measure classification accuracy but overlook reasoning fidelity. We propose DeepfakeJudge, a framework for scalable reasoning supervision and evaluation, that integrates an out-of-distribution benchmark containing recent generative and editing forgeries, a human-annotated subset with visual reasoning labels, and a suite of evaluation models, that specialize in evaluating reasoning rationales without the need for explicit ground truth reasoning rationales. The Judge is optimized through a bootstrapped generator-evaluator process that scales human feedback into structured reasoning supervision and supports both pointwise and pairwise evaluation. On the proposed meta-evaluation benchmark, our reasoning-bootstrapped model achieves an accuracy of 96.2%, outperforming \texttt{30x} larger baselines. The reasoning judge attains very high correlation with human ratings and 98.9% percent pairwise agreement on the human-annotated meta-evaluation subset. These results establish reasoning fidelity as a quantifiable dimension of deepfake detection and demonstrate scalable supervision for interpretable deepfake reasoning. Our user study shows that participants preferred the reasonings generated by our framework 70% of the time, in terms of faithfulness, groundedness, and usefulness, compared to those produced by other models and datasets. All of our datasets, models, and codebase are \href{https://github.com/KjAeRsTuIsK/DeepfakeJudge}{open-sourced}.

Method: Proposes Flose, a generative method that formulates 6D pose estimation as conditional flow matching in ℝ³. Uses denoising process conditioned on local features, integrating both geometric guidance and appearance-based semantic features to handle symmetries. Incorporates RANSAC-based registration for outlier handling.

Result: Validated on five datasets from BOP benchmark. Outperforms prior methods with average improvement of +4.5 Average Recall.

Conclusion: Flose successfully addresses limitations of existing 6D pose estimation methods by combining geometric and appearance features through conditional flow matching, achieving state-of-the-art performance on standard benchmarks.

Abstract: Existing methods for instance-level 6D pose estimation typically rely on neural networks that either directly regress the pose in $\mathrm{SE}(3)$ or estimate it indirectly via local feature matching. The former struggle with object symmetries, while the latter fail in the absence of distinctive local features. To overcome these limitations, we propose a novel formulation of 6D pose estimation as a conditional flow matching problem in $\mathbb{R}^3$. We introduce Flose, a generative method that infers object poses via a denoising process conditioned on local features. While prior approaches based on conditional flow matching perform denoising solely based on geometric guidance, Flose integrates appearance-based semantic features to mitigate ambiguities caused by object symmetries. We further incorporate RANSAC-based registration to handle outliers. We validate Flose on five datasets from the established BOP benchmark. Flose outperforms prior methods with an average improvement of +4.5 Average Recall. Project Website : https://tev-fbk.github.io/Flose/

Method: Proposes GOAL framework with fixed Equiangular Tight Frame (ETF) classifier to impose consistent geometric structure. Uses supervised alignment for labeled samples and confidence-guided alignment for novel samples to integrate new classes without disrupting old ones.

Result: Outperforms prior method Happy on four benchmarks, reducing forgetting by 16.1% and boosting novel class discovery by 3.2%.

Conclusion: GOAL establishes a strong solution for long-horizon continual discovery by maintaining stable geometric structure throughout learning.

Abstract: Continual Generalized Category Discovery (C-GCD) requires identifying novel classes from unlabeled data while retaining knowledge of known classes over time. Existing methods typically update classifier weights dynamically, resulting in forgetting and inconsistent feature alignment. We propose GOAL, a unified framework that introduces a fixed Equiangular Tight Frame (ETF) classifier to impose a consistent geometric structure throughout learning. GOAL conducts supervised alignment for labeled samples and confidence-guided alignment for novel samples, enabling stable integration of new classes without disrupting old ones. Experiments on four benchmarks show that GOAL outperforms the prior method Happy, reducing forgetting by 16.1% and boosting novel class discovery by 3.2%, establishing a strong solution for long-horizon continual discovery.

Method: PMM-Synth uses three core innovations: 1) Personalized Feature Modulation module that dynamically adapts features based on dataset identifier to mitigate distribution shifts, 2) Modality-Consistent Batch Scheduler for stable training under inconsistent modality conditions, and 3) selective supervision loss for effective learning when ground truth modalities are partially missing.

Result: Evaluated on four clinical multi-modal MRI datasets, PMM-Synth consistently outperforms state-of-the-art methods in both one-to-one and many-to-one synthesis tasks, achieving superior PSNR and SSIM scores, with qualitative results showing improved preservation of anatomical structures and pathological details.

Conclusion: PMM-Synth demonstrates effective cross-dataset generalization for multi-modal MRI synthesis, showing potential for supporting reliable diagnosis under real-world modality-missing scenarios, as evidenced by downstream tumor segmentation and radiological reporting studies.

Abstract: Synthesizing missing modalities in multi-modal magnetic resonance imaging (MRI) is vital for ensuring diagnostic completeness, particularly when full acquisitions are infeasible due to time constraints, motion artifacts, and patient tolerance. Recent unified synthesis models have enabled flexible synthesis tasks by accommodating various input-output configurations. However, their training and evaluation are typically restricted to a single dataset, limiting their generalizability across diverse clinical datasets and impeding practical deployment. To address this limitation, we propose PMM-Synth, a personalized MRI synthesis framework that not only supports various synthesis tasks but also generalizes effectively across heterogeneous datasets. PMM-Synth is jointly trained on multiple multi-modal MRI datasets that differ in modality coverage, disease types, and intensity distributions. It achieves cross-dataset generalization through three core innovations: a Personalized Feature Modulation module that dynamically adapts feature representations based on dataset identifier to mitigate the impact of distributional shifts; a Modality-Consistent Batch Scheduler that facilitates stable and efficient batch training under inconsistent modality conditions; and a selective supervision loss to ensure effective learning when ground truth modalities are partially missing. Evaluated on four clinical multi-modal MRI datasets, PMM-Synth consistently outperforms state-of-the-art methods in both one-to-one and many-to-one synthesis tasks, achieving superior PSNR and SSIM scores. Qualitative results further demonstrate improved preservation of anatomical structures and pathological details. Additionally, downstream tumor segmentation and radiological reporting studies suggest that PMM-Synth holds potential for supporting reliable diagnosis under real-world modality-missing scenarios.

Method: Proposes MaSoN framework that synthesizes diverse changes directly in latent feature space during training by dynamically estimating changes using feature statistics of target data, enabling data-driven variation aligned with target domain.

Result: Achieves state-of-the-art performance on five benchmarks, improving average F1 score by 14.1 percentage points, and demonstrates strong generalization across diverse change types.

Conclusion: MaSoN addresses limitations of existing UCD methods by generating diverse, data-driven changes in latent space, enabling better generalization and performance across various change types and modalities.

Abstract: Unsupervised change detection (UCD) in remote sensing aims to localise semantic changes between two images of the same region without relying on labelled data during training. Most recent approaches rely either on frozen foundation models in a training-free manner or on training with synthetic changes generated in pixel space. Both strategies inherently rely on predefined assumptions about change types, typically introduced through handcrafted rules, external datasets, or auxiliary generative models. Due to these assumptions, such methods fail to generalise beyond a few change types, limiting their real-world usage, especially in rare or complex scenarios. To address this, we propose MaSoN (Make Some Noise), an end-to-end UCD framework that synthesises diverse changes directly in the latent feature space during training. It generates changes that are dynamically estimated using feature statistics of target data, enabling diverse yet data-driven variation aligned with the target domain. It also easily extends to new modalities, such as SAR. MaSoN generalises strongly across diverse change types and achieves state-of-the-art performance on five benchmarks, improving the average F1 score by 14.1 percentage points. Project page: https://blaz-r.github.io/mason_ucd

Method: Uses Visual Geometry Grounded Transformer (VGGT) as unified geometric engine. Extracts geometrically-rich visual embeddings with depth-aware and point map supervision. Densifies sparse LiDAR point clouds with predicted depth maps. Combines mask-guided keypoint extraction with confidence-aware correspondence scoring for training-free re-ranking.

Result: Achieves state-of-the-art performance on large-scale autonomous driving benchmarks and self-collected data. Demonstrates strong robustness to environmental changes, viewpoint shifts, and occlusions.

Conclusion: VGGT-MPR provides an effective multimodal place recognition framework that leverages geometric understanding without requiring costly retraining, offering robust performance for autonomous driving applications.

Abstract: In autonomous driving, robust place recognition is critical for global localization and loop closure detection. While inter-modality fusion of camera and LiDAR data in multimodal place recognition (MPR) has shown promise in overcoming the limitations of unimodal counterparts, existing MPR methods basically attend to hand-crafted fusion strategies and heavily parameterized backbones that require costly retraining. To address this, we propose VGGT-MPR, a multimodal place recognition framework that adopts the Visual Geometry Grounded Transformer (VGGT) as a unified geometric engine for both global retrieval and re-ranking. In the global retrieval stage, VGGT extracts geometrically-rich visual embeddings through prior depth-aware and point map supervision, and densifies sparse LiDAR point clouds with predicted depth maps to improve structural representation. This enhances the discriminative ability of fused multimodal features and produces global descriptors for fast retrieval. Beyond global retrieval, we design a training-free re-ranking mechanism that exploits VGGT’s cross-view keypoint-tracking capability. By combining mask-guided keypoint extraction with confidence-aware correspondence scoring, our proposed re-ranking mechanism effectively refines retrieval results without additional parameter optimization. Extensive experiments on large-scale autonomous driving benchmarks and our self-collected data demonstrate that VGGT-MPR achieves state-of-the-art performance, exhibiting strong robustness to severe environmental changes, viewpoint shifts, and occlusions. Our code and data will be made publicly available.

Method: Generate large-scale synthetic datasets using state-of-the-art T2I models (2022-2025), train standard classifiers solely on this synthetic data, and evaluate on real test data while analyzing distribution characteristics.

Result: Classification accuracy on real test data consistently declines with newer T2I models, despite their improved visual fidelity and prompt adherence. Models collapse to narrow, aesthetic-centric distributions that undermine diversity and label-image alignment.

Conclusion: Progress in generative realism does not imply progress in data realism. There’s an urgent need to rethink modern T2I models’ capabilities as reliable training data generators for vision tasks.

Abstract: Recent text-to-image (T2I) diffusion models produce visually stunning images and demonstrate excellent prompt following. But do they perform well as synthetic vision data generators? In this work, we revisit the promise of synthetic data as a scalable substitute for real training sets and uncover a surprising performance regression. We generate large-scale synthetic datasets using state-of-the-art T2I models released between 2022 and 2025, train standard classifiers solely on this synthetic data, and evaluate them on real test data. Despite observable advances in visual fidelity and prompt adherence, classification accuracy on real test data consistently declines with newer T2I models as training data generators. Our analysis reveals a hidden trend: These models collapse to a narrow, aesthetic-centric distribution that undermines diversity and label-image alignment. Overall, our findings challenge a growing assumption in vision research, namely that progress in generative realism implies progress in data realism. We thus highlight an urgent need to rethink the capabilities of modern T2I models as reliable training data generators.

Method: Adapts iterative refinement process of diffusion models with a novel guided and variance-corrected fusion mechanism, enabling seamless generation of large-scale high-resolution imagery without retraining.

Result: Validated on remote sensing datasets, InfScene-SR reconstructs fine details with high perceptual quality, eliminates boundary artifacts, and benefits downstream tasks like semantic segmentation.

Conclusion: Proposed framework enables spatially continuous super-resolution for large arbitrary scenes using diffusion models, overcoming patch-based limitations and improving practical applications.

Abstract: Image Super-Resolution (SR) aims to recover high-resolution (HR) details from low-resolution (LR) inputs, a task where Denoising Diffusion Probabilistic Models (DDPMs) have recently shown superior performance compared to Generative Adversarial Networks (GANs) based approaches. However, standard diffusion-based SR models, such as SR3, are typically trained on fixed-size patches and struggle to scale to arbitrary-sized images due to memory constraints. Applying these models via independent patch processing leads to visible seams and inconsistent textures across boundaries. In this paper, we propose InfScene-SR, a framework enabling spatially continuous super-resolution for large, arbitrary scenes. We adapt the iterative refinement process of diffusion models with a novel guided and variance-corrected fusion mechanism, allowing for the seamless generation of large-scale high-resolution imagery without retraining. We validate our approach on remote sensing datasets, demonstrating that InfScene-SR not only reconstructs fine details with high perceptual quality but also eliminates boundary artifacts, benefiting downstream tasks such as semantic segmentation.

Method: RAP uses a compact MLP to predict per-primitive importance scores directly from Gaussian attributes and local neighborhood statistics, avoiding rendering computations. It employs rendering loss, pruning-aware loss, and significance distribution regularization during training.

Result: RAP achieves fast importance prediction, generalizes well to unseen scenes after training on a small dataset, and can be integrated into reconstruction, compression, and transmission pipelines.

Conclusion: RAP provides an efficient, rendering-free solution for primitive importance estimation in 3DGS that addresses limitations of view-dependent methods and enables practical applications in compression and transmission.

Abstract: 3D Gaussian Splatting (3DGS) has emerged as a leading technology for high-quality 3D scene reconstruction. However, the iterative refinement and densification process leads to the generation of a large number of primitives, each contributing to the reconstruction to a substantially different extent. Estimating primitive importance is thus crucial, both for removing redundancy during reconstruction and for enabling efficient compression and transmission. Existing methods typically rely on rendering-based analyses, where each primitive is evaluated through its contribution across multiple camera viewpoints. However, such methods are sensitive to the number and selection of views, rely on specialized differentiable rasterizers, and have long calculation times that grow linearly with view count, making them difficult to integrate as plug-and-play modules and limiting scalability and generalization. To address these issues, we propose RAP, a fast feedforward rendering-free attribute-guided method for efficient importance score prediction in 3DGS. RAP infers primitive significance directly from intrinsic Gaussian attributes and local neighborhood statistics, avoiding rendering-based or visibility-dependent computations. A compact MLP predicts per-primitive importance scores using rendering loss, pruning-aware loss, and significance distribution regularization. After training on a small set of scenes, RAP generalizes effectively to unseen data and can be seamlessly integrated into reconstruction, compression, and transmission pipelines. Our code is publicly available at https://github.com/yyyykf/RAP.

Method: Creation of a curated image dataset containing 3,556 high-resolution images focused on Neotropical Ichneumonidae and Braconidae, supplemented with other families for model robustness. A subset of 1,739 images is annotated in COCO format with multi-class bounding boxes for full insect body, wing venation, and scale bars.

Result: A comprehensive digital resource providing a foundation for developing computer vision models capable of identifying these challenging taxonomic groups, with structured annotations suitable for machine learning applications.

Conclusion: This dataset addresses critical gaps in digital biodiversity resources and enables the development of automated identification systems for ecologically important but taxonomically difficult parasitoid wasp groups.

Abstract: Accurate taxonomic identification is the cornerstone of biodiversity monitoring and agricultural management, particularly for the hyper-diverse superfamily Ichneumonoidea. Comprising the families Ichneumonidae and Braconidae, these parasitoid wasps are ecologically critical for regulating insect populations, yet they remain one of the most taxonomically challenging groups due to their cryptic morphology and vast number of undescribed species. To address the scarcity of robust digital resources for these key groups, we present a curated image dataset designed to advance automated identification systems. The dataset contains 3,556 high-resolution images, primarily focused on Neotropical Ichneumonidae and Braconidae, while also including supplementary families such as Andrenidae, Apidae, Bethylidae, Chrysididae, Colletidae, Halictidae, Megachilidae, Pompilidae, and Vespidae to improve model robustness. Crucially, a subset of 1,739 images is annotated in COCO format, featuring multi-class bounding boxes for the full insect body, wing venation, and scale bars. This resource provides a foundation for developing computer vision models capable of identifying these families.

Method: Uses CLIP to extract aligned image-text embeddings, obtains prototypes, and employs an unCLIP decoder to synthesize images, creating a learning-free framework for multimodal dataset distillation.

Result: Consistently outperforms optimization-based dataset distillation and subset selection methods, achieving state-of-the-art cross-architecture generalization.

Conclusion: Proposed learning-free framework enables efficient and scalable multimodal dataset distillation without large-scale training, enhancing generalization across different architectures.

Abstract: Recent advances in multimodal learning have achieved remarkable success across diverse vision-language tasks. However, such progress heavily relies on large-scale image-text datasets, making training costly and inefficient. Prior efforts in dataset filtering and pruning attempt to mitigate this issue, but still require relatively large subsets to maintain performance and fail under very small subsets. Dataset distillation offers a promising alternative, yet existing multimodal dataset distillation methods require full-dataset training and joint optimization of image pixels and text features, making them architecture-dependent and limiting cross-architecture generalization. To overcome this, we propose a learning-free dataset distillation framework that eliminates the need for large-scale training and optimization while enhancing generalization across architectures. Our method uses CLIP to extract aligned image-text embeddings, obtains prototypes, and employs an unCLIP decoder to synthesize images, enabling efficient and scalable multimodal dataset distillation. Extensive experiments demonstrate that our approach consistently outperforms optimization-based dataset distillation and subset selection methods, achieving state-of-the-art cross-architecture generalization.

Method: Proposes SEAL-pose with a joint-graph-based loss network that learns structural dependencies directly from data. The loss network trains the pose network by evaluating structural plausibility rather than using hand-crafted constraints.

Result: Extensive experiments on three 3D HPE benchmarks with eight backbones show reduced per-joint errors and improved pose plausibility. Outperforms models with explicit structural constraints despite not enforcing any such constraints.

Conclusion: SEAL-pose provides a data-driven alternative to hand-crafted structural priors, enabling end-to-end training while capturing complex joint dependencies for more plausible 3D human pose estimation.

Abstract: 3D human pose estimation (HPE) is characterized by intricate local and global dependencies among joints. Conventional supervised losses are limited in capturing these correlations because they treat each joint independently. Previous studies have attempted to promote structural consistency through manually designed priors or rule-based constraints; however, these approaches typically require manual specification and are often non-differentiable, limiting their use as end-to-end training objectives. We propose SEAL-pose, a data-driven framework in which a learnable loss-net trains a pose-net by evaluating structural plausibility. Rather than relying on hand-crafted priors, our joint-graph-based design enables the loss-net to learn complex structural dependencies directly from data. Extensive experiments on three 3D HPE benchmarks with eight backbones show that SEAL-pose reduces per-joint errors and improves pose plausibility compared with the corresponding backbones across all settings. Beyond improving each backbone, SEAL-pose also outperforms models with explicit structural constraints, despite not enforcing any such constraints. Finally, we analyze the relationship between the loss-net and structural consistency, and evaluate SEAL-pose in cross-dataset and in-the-wild settings.

Method: Three-stage framework: 1) Generate anchor views from single input via panorama generator, 2) Lift 2D anchors to 3D geometric scaffold using feed-forward Gaussian Splatting network with multi-view stereo matching approach, 3) Use scaffold as prior for novel view generator to produce photorealistic views at arbitrary cameras.

Result: Substantially outperforms state-of-the-art methods in panorama depth estimation, feed-forward 360° reconstruction, and explorable 3D scene generation, supporting stable performance under large camera motions.

Conclusion: One2Scene enables immersive explorable scene generation from single images by decomposing the ill-posed problem into tractable sub-tasks and leveraging explicit 3D-consistent scaffolds for stable performance.

Abstract: Generating explorable 3D scenes from a single image is a highly challenging problem in 3D vision. Existing methods struggle to support free exploration, often producing severe geometric distortions and noisy artifacts when the viewpoint moves far from the original perspective. We introduce \textbf{One2Scene}, an effective framework that decomposes this ill-posed problem into three tractable sub-tasks to enable immersive explorable scene generation. We first use a panorama generator to produce anchor views from a single input image as initialization. Then, we lift these 2D anchors into an explicit 3D geometric scaffold via a generalizable, feed-forward Gaussian Splatting network. Instead of treating the panorama as a single image for reconstruction, we project it into multiple sparse anchor views and reformulate the reconstruction task as multi-view stereo matching, which allows us to leverage robust geometric priors learned from large-scale multi-view datasets. A bidirectional feature fusion module is used to enforce cross-view consistency, yielding an efficient and geometrically reliable scaffold. Finally, the scaffold serves as a strong prior for a novel view generator to produce photorealistic and geometrically accurate views at arbitrary cameras. By explicitly conditioning on a 3D-consistent scaffold to perform reconstruction, One2Scene works stably under large camera motions, supporting immersive scene exploration. Extensive experiments show that One2Scene substantially outperforms state-of-the-art methods in panorama depth estimation, feed-forward 360° reconstruction, and explorable 3D scene generation. Code and models will be released.

Method: Proposes TraceVision with Trajectory-aware Visual Perception (TVP) module for bidirectional fusion of visual features and trajectory information. Uses geometric simplification to extract semantic keypoints from raw trajectories, and a three-stage training pipeline where trajectories guide description generation and region localization. Extends to trajectory-guided segmentation and video scene understanding.

Result: Achieves state-of-the-art performance on trajectory-guided captioning, text-guided trajectory prediction, understanding, and segmentation. Demonstrates cross-frame tracking and temporal attention analysis capabilities.

Conclusion: TraceVision establishes a foundation for intuitive spatial interaction and interpretable visual understanding by integrating trajectory-aware spatial understanding in an end-to-end framework.

Abstract: Recent Large Vision-Language Models (LVLMs) demonstrate remarkable capabilities in image understanding and natural language generation. However, current approaches focus predominantly on global image understanding, struggling to simulate human visual attention trajectories and explain associations between descriptions and specific regions. We propose TraceVision, a unified vision-language model integrating trajectory-aware spatial understanding in an end-to-end framework. TraceVision employs a Trajectory-aware Visual Perception (TVP) module for bidirectional fusion of visual features and trajectory information. We design geometric simplification to extract semantic keypoints from raw trajectories and propose a three-stage training pipeline where trajectories guide description generation and region localization. We extend TraceVision to trajectory-guided segmentation and video scene understanding, enabling cross-frame tracking and temporal attention analysis. We construct the Reasoning-based Interactive Localized Narratives (RILN) dataset to enhance logical reasoning and interpretability. Extensive experiments on trajectory-guided captioning, text-guided trajectory prediction, understanding, and segmentation demonstrate that TraceVision achieves state-of-the-art performance, establishing a foundation for intuitive spatial interaction and interpretable visual understanding.

Method: Proposes a training-free pipeline that generates masks by merging pre-computed superpoints based on semantic features, then uses domain-adapted VLFM “IndustrialCLIP” for open-vocabulary querying on 3D industrial scenes.

Result: Qualitative results demonstrate successful segmentation of industrial objects in representative 3D industrial workshop scenes, overcoming generalization limitations of previous methods.

Conclusion: The proposed training-free approach effectively addresses domain gap issues in industrial open-vocabulary perception by combining superpoint-based segmentation with domain-adapted vision-language models.

Abstract: Autonomous vision applications in production, intralogistics, or manufacturing environments require perception capabilities beyond a small, fixed set of classes. Recent open-vocabulary methods, leveraging 2D Vision-Language Foundation Models (VLFMs), target this task but often rely on class-agnostic segmentation models pre-trained on non-industrial datasets (e.g., household scenes). In this work, we first demonstrate that such models fail to generalize, performing poorly on common industrial objects. Therefore, we propose a training-free, open-vocabulary 3D perception pipeline that overcomes this limitation. Instead of using a pre-trained model to generate instance proposals, our method simply generates masks by merging pre-computed superpoints based on their semantic features. Following, we evaluate the domain-adapted VLFM “IndustrialCLIP” on a representative 3D industrial workshop scene for open-vocabulary querying. Our qualitative results demonstrate successful segmentation of industrial objects.

Method: Three-stage approach: 1) Forensic Continual Pre-training using easy-to-hard curriculum from natural image forensic and OCR tasks, 2) Group Relative Policy Optimization with novel reward functions for fine-tuning, 3) OCR Rectification at inference to refine predictions using MLLM’s text recognition abilities.

Result: TextShield-R1 significantly advances state-of-the-art in interpretable tampered text detection, with comprehensive evaluation on the TFR benchmark containing 45k+ images across 16 languages, 10 tampering techniques, and diverse domains.

Conclusion: The proposed reinforcement learning-based MLLM approach effectively addresses limitations in tampered text detection, reducing annotation dependency while improving localization accuracy and reasoning capabilities through innovative training and inference techniques.

Abstract: The growing prevalence of tampered images poses serious security threats, highlighting the urgent need for reliable detection methods. Multimodal large language models (MLLMs) demonstrate strong potential in analyzing tampered images and generating interpretations. However, they still struggle with identifying micro-level artifacts, exhibit low accuracy in localizing tampered text regions, and heavily rely on expensive annotations for forgery interpretation. To this end, we introduce TextShield-R1, the first reinforcement learning based MLLM solution for tampered text detection and reasoning. Specifically, our approach introduces Forensic Continual Pre-training, an easy-to-hard curriculum that well prepares the MLLM for tampered text detection by harnessing the large-scale cheap data from natural image forensic and OCR tasks. During fine-tuning, we perform Group Relative Policy Optimization with novel reward functions to reduce annotation dependency and improve reasoning capabilities. At inference time, we enhance localization accuracy via OCR Rectification, a method that leverages the MLLM’s strong text recognition abilities to refine its predictions. Furthermore, to support rigorous evaluation, we introduce the Text Forensics Reasoning (TFR) benchmark, comprising over 45k real and tampered images across 16 languages, 10 tampering techniques, and diverse domains. Rich reasoning-style annotations are included, allowing for comprehensive assessment. Our TFR benchmark simultaneously addresses seven major limitations of existing benchmarks and enables robust evaluation under cross-style, cross-method, and cross-language conditions. Extensive experiments demonstrate that TextShield-R1 significantly advances the state of the art in interpretable tampered text detection.

Method: Uses pretrained Large Vision Language Models (VLMs) with domain-specific prompts to extract visual attributes (roof age, building density, etc.) from RGB satellite images. A Multi-Layer Perceptron regressor is trained on these extracted captions to estimate annual heat demand.

Result: The MLP regressor shows 93.7% R² uplift and reduces mean absolute error by 30% compared to baseline. Qualitative analysis shows high-impact tokens align with high-demand zones.

Conclusion: HeatPrompt offers a lightweight, zero-shot vision-language approach for heat planning in data-scarce regions by leveraging satellite imagery and VLMs to extract relevant visual features for energy modeling.

Abstract: Accurate heat-demand maps play a crucial role in decarbonizing space heating, yet most municipalities lack detailed building-level data needed to calculate them. We introduce HeatPrompt, a zero-shot vision-language energy modeling framework that estimates annual heat demand using semantic features extracted from satellite images, basic Geographic Information System (GIS), and building-level features. We feed pretrained Large Vision Language Models (VLMs) with a domain-specific prompt to act as an energy planner and extract the visual attributes such as roof age, building density, etc, from the RGB satellite image that correspond to the thermal load. A Multi-Layer Perceptron (MLP) regressor trained on these captions shows an $R^2$ uplift of 93.7% and shrinks the mean absolute error (MAE) by 30% compared to the baseline model. Qualitative analysis shows that high-impact tokens align with high-demand zones, offering lightweight support for heat planning in data-scarce regions.

Method: Proposes M3S-Net with three key components: 1) multi-scale partial channel selection network using partial convolutions to isolate boundary features of optically thin clouds, 2) multi-scale sequence to image analysis network using FFT-based time-frequency representation for meteorological data, and 3) cross-modal Mamba interaction module with dynamic C-matrix swapping mechanism for deep structural coupling between modalities.

Result: Achieves 6.2% mean absolute error reduction in 10-minute forecasts compared to state-of-the-art baselines on a newly constructed fine-grained PV power dataset.

Conclusion: M3S-Net effectively addresses limitations of existing multimodal forecasting approaches by enabling deep structural coupling between visual and temporal modalities with linear computational complexity, improving ultra-short-term PV power forecasting accuracy.

Abstract: The inherent intermittency and high-frequency variability of solar irradiance, particularly during rapid cloud advection, present significant stability challenges to high-penetration photovoltaic grids. Although multimodal forecasting has emerged as a viable mitigation strategy, existing architectures predominantly rely on shallow feature concatenation and binary cloud segmentation, thereby failing to capture the fine-grained optical features of clouds and the complex spatiotemporal coupling between visual and meteorological modalities. To bridge this gap, this paper proposes M3S-Net, a novel multimodal feature fusion network based on multi-scale data for ultra-short-term PV power forecasting. First, a multi-scale partial channel selection network leverages partial convolutions to explicitly isolate the boundary features of optically thin clouds, effectively transcending the precision limitations of coarse-grained binary masking. Second, a multi-scale sequence to image analysis network employs Fast Fourier Transform (FFT)-based time-frequency representation to disentangle the complex periodicity of meteorological data across varying time horizons. Crucially, the model incorporates a cross-modal Mamba interaction module featuring a novel dynamic C-matrix swapping mechanism. By exchanging state-space parameters between visual and temporal streams, this design conditions the state evolution of one modality on the context of the other, enabling deep structural coupling with linear computational complexity, thus overcoming the limitations of shallow concatenation. Experimental validation on the newly constructed fine-grained PV power dataset demonstrates that M3S-Net achieves a mean absolute error reduction of 6.2% in 10-minute forecasts compared to state-of-the-art baselines. The dataset and source code will be available at https://github.com/she1110/FGPD.

Method: 1) Class-conditioned diffusion models generate synthetic dermatological images to address class imbalance. 2) Self-supervised MAE pretraining enables huge ViT models to learn robust, domain-relevant features. 3) Knowledge distillation transfers representations to smaller ViT student models suitable for mobile devices.

Result: MAE pretraining on synthetic data combined with distillation improves classification performance while enabling efficient on-device inference for practical clinical use.

Conclusion: The approach successfully addresses class imbalance in medical imaging through synthetic data generation and enables practical clinical deployment through knowledge distillation to lightweight models.

Abstract: Skin lesion classification datasets often suffer from severe class imbalance, with malignant cases significantly underrepresented, leading to biased decision boundaries during deep learning training. We address this challenge using class-conditioned diffusion models to generate synthetic dermatological images, followed by self-supervised MAE pretraining to enable huge ViT models to learn robust, domain-relevant features. To support deployment in practical clinical settings, where lightweight models are required, we apply knowledge distillation to transfer these representations to a smaller ViT student suitable for mobile devices. Our results show that MAE pretraining on synthetic data, combined with distillation, improves classification performance while enabling efficient on-device inference for practical clinical use.

Method: StructXLIP extracts edge maps (e.g., Canny) as proxies for visual structure and filters captions to emphasize structural cues. It augments standard alignment loss with three structure-centric losses: (1) aligning edge maps with structural text, (2) matching local edge regions to textual chunks, and (3) connecting edge maps to color images to prevent representation drift.

Result: The method outperforms current competitors on cross-modal retrieval in both general and specialized domains. It serves as a general boosting recipe that can be integrated into future approaches in a plug-and-play manner.

Conclusion: StructXLIP demonstrates that focusing on structural alignment enhances vision-language models, providing more robust and semantically stable representations through additional mutual information maximization between multimodal structural representations.

Abstract: Edge-based representations are fundamental cues for visual understanding, a principle rooted in early vision research and still central today. We extend this principle to vision-language alignment, showing that isolating and aligning structural cues across modalities can greatly benefit fine-tuning on long, detail-rich captions, with a specific focus on improving cross-modal retrieval. We introduce StructXLIP, a fine-tuning alignment paradigm that extracts edge maps (e.g., Canny), treating them as proxies for the visual structure of an image, and filters the corresponding captions to emphasize structural cues, making them “structure-centric”. Fine-tuning augments the standard alignment loss with three structure-centric losses: (i) aligning edge maps with structural text, (ii) matching local edge regions to textual chunks, and (iii) connecting edge maps to color images to prevent representation drift. From a theoretical standpoint, while standard CLIP maximizes the mutual information between visual and textual embeddings, StructXLIP additionally maximizes the mutual information between multimodal structural representations. This auxiliary optimization is intrinsically harder, guiding the model toward more robust and semantically stable minima, enhancing vision-language alignment. Beyond outperforming current competitors on cross-modal retrieval in both general and specialized domains, our method serves as a general boosting recipe that can be integrated into future approaches in a plug-and-play manner. Code and pretrained models are publicly available at: https://github.com/intelligolabs/StructXLIP.

Method: Proposes an adaptation strategy using visual meta-domains to transfer visual representations from larger dermoscopic datasets into clinical image domains for improved generalization robustness.

Result: Experiments across multiple dermatology datasets show consistent gains in classification performance and reduced gaps between dermoscopic and clinical images.

Conclusion: Emphasizes the importance of domain-aware training for deployable dermatological image analysis systems.

Abstract: Deep learning models for dermatological image analysis remain sensitive to acquisition variability and domain-specific visual characteristics, leading to performance degradation when deployed in clinical settings. We investigate how visual artifacts and domain shifts affect deep learning-based skin lesion classification. We propose an adaptation strategy, grounded in the idea of visual meta-domains, that transfers visual representations from larger dermoscopic datasets into clinical image domains, thereby improving generalization robustness. Experiments across multiple dermatology datasets show consistent gains in classification performance and reduced gaps between dermoscopic and clinical images. These results emphasize the importance of domain-aware training for deployable systems.

Method: Benchmarks MU algorithms on different VT families (ViT and Swin-T) at various capacities, using multiple datasets, different MU algorithms representing fundamentally different approaches, and both single-shot and continual unlearning protocols.

Result: Provides first comprehensive benchmarking basis for MU algorithms on VTs, characterizes how VTs memorize training data relative to CNNs, and establishes reference performance baselines.

Conclusion: This work enables reproducible, fair comparisons of existing and future MU algorithms on Vision Transformers and sheds light on algorithm performance in VT settings.

Abstract: Research in machine unlearning (MU) has gained strong momentum: MU is now widely regarded as a critical capability for building safe and fair AI. In parallel, research into transformer architectures for computer vision tasks has been highly successful: Increasingly, Vision Transformers (VTs) emerge as strong alternatives to CNNs. Yet, MU research for vision tasks has largely centered on CNNs, not VTs. While benchmarking MU efforts have addressed LLMs, diffusion models, and CNNs, none exist for VTs. This work is the first to attempt this, benchmarking MU algorithm performance in different VT families (ViT and Swin-T) and at different capacities. The work employs (i) different datasets, selected to assess the impacts of dataset scale and complexity; (ii) different MU algorithms, selected to represent fundamentally different approaches for MU; and (iii) both single-shot and continual unlearning protocols. Additionally, it focuses on benchmarking MU algorithms that leverage training data memorization, since leveraging memorization has been recently discovered to significantly improve the performance of previously SOTA algorithms. En route, the work characterizes how VTs memorize training data relative to CNNs, and assesses the impact of different memorization proxies on performance. The benchmark uses unified evaluation metrics that capture two complementary notions of forget quality along with accuracy on unseen (test) data and on retained data. Overall, this work offers a benchmarking basis, enabling reproducible, fair, and comprehensive comparisons of existing (and future) MU algorithms on VTs. And, for the first time, it sheds light on how well existing algorithms work in VT settings, establishing a promising reference performance baseline.

Method: Proposes a dual-teacher contrastive distillation framework that aligns the student’s pretraining objective with the contrastive self-distillation paradigm of modern optical vision foundation models. Combines a multispectral teacher with an optical VFM teacher to enable coherent cross-modal representation learning.

Result: Achieves state-of-the-art results across diverse optical and multispectral benchmarks. Average improvements: 3.64 percentage points in semantic segmentation, 1.2 in change detection, and 1.31 in classification tasks. Model adapts to multispectral data without compromising performance on optical-only inputs.

Conclusion: Contrastive distillation provides a principled and efficient approach to scalable representation learning across heterogeneous Earth Observation data sources, enabling knowledge transfer between different EO modalities.

Abstract: Foundation models are transforming Earth Observation (EO), yet the diversity of EO sensors and modalities makes a single universal model unrealistic. Multiple specialized EO foundation models (EOFMs) will likely coexist, making efficient knowledge transfer across modalities essential. Most existing EO pretraining relies on masked image modeling, which emphasizes local reconstruction but provides limited control over global semantic structure. To address this, we propose a dual-teacher contrastive distillation framework for multispectral imagery that aligns the student’s pretraining objective with the contrastive self-distillation paradigm of modern optical vision foundation models (VFMs). Our approach combines a multispectral teacher with an optical VFM teacher, enabling coherent cross-modal representation learning. Experiments across diverse optical and multispectral benchmarks show that our model adapts to multispectral data without compromising performance on optical-only inputs, achieving state-of-the-art results in both settings, with an average improvement of 3.64 percentage points in semantic segmentation, 1.2 in change detection, and 1.31 in classification tasks. This demonstrates that contrastive distillation provides a principled and efficient approach to scalable representation learning across heterogeneous EO data sources. Code: Coming soon.

Method: ApET uses linear approximation to reconstruct original visual tokens with a small set of basis tokens, then leverages approximation error to identify and drop the least informative tokens, avoiding attention dependencies entirely.

Result: Achieves 95.2% of original performance on image-understanding tasks and 100.4% on video-understanding tasks while compressing token budgets by 88.9% and 87.5% respectively. Seamlessly integrates with FlashAttention for further acceleration.

Conclusion: ApET provides an effective, attention-free approach to visual token compression that maintains performance while enabling practical deployment of VLMs through compatibility with efficient attention kernels.

Abstract: Recent Vision-Language Models (VLMs) have demonstrated remarkable multimodal understanding capabilities, yet the redundant visual tokens incur prohibitive computational overhead and degrade inference efficiency. Prior studies typically relies on [CLS] attention or text-vision cross-attention to identify and discard redundant visual tokens. Despite promising results, such solutions are prone to introduce positional bias and, more critically, are incompatible with efficient attention kernels such as FlashAttention, limiting their practical deployment for VLM acceleration. In this paper, we step away from attention dependencies and revisit visual token compression from an information-theoretic perspective, aiming to maximally preserve visual information without any attention involvement. We present ApET, an Approximation-Error guided Token compression framework. ApET first reconstructs the original visual tokens with a small set of basis tokens via linear approximation, then leverages the approximation error to identify and drop the least informative tokens. Extensive experiments across multiple VLMs and benchmarks demonstrate that ApET retains 95.2% of the original performance on image-understanding tasks and even attains 100.4% on video-understanding tasks, while compressing the token budgets by 88.9% and 87.5%, respectively. Thanks to its attention-free design, ApET seamlessly integrates with FlashAttention, enabling further inference acceleration and making VLM deployment more practical. Code is available at https://github.com/MaQianKun0/ApET.

Method: Created BigMaQ dataset with 750+ scenes of interacting rhesus macaques, building subject-specific textured avatars by adapting a high-quality macaque template mesh to individual monkeys. Derived BigMaQ500 benchmark linking surface-based pose vectors to single frames across multiple monkeys for action recognition.

Result: Pose descriptions more accurate than previous state-of-the-art surface-based animal tracking methods. When pose information is combined with features from image/video encoders, substantial improvements in mean average precision (mAP) are achieved for action recognition.

Conclusion: BigMaQ establishes the first dataset integrating dynamic 3D pose-shape representations into animal action recognition learning, providing a rich resource for studying visual appearance, posture, and social interaction in non-human primates.

Abstract: The recognition of dynamic and social behavior in animals is fundamental for advancing ethology, ecology, medicine and neuroscience. Recent progress in deep learning has enabled automated behavior recognition from video, yet an accurate reconstruction of the three-dimensional (3D) pose and shape has not been integrated into this process. Especially for non-human primates, mesh-based tracking efforts lag behind those for other species, leaving pose descriptions restricted to sparse keypoints that are unable to fully capture the richness of action dynamics. To address this gap, we introduce the $\textbf{Big Ma}$ca$\textbf{Q}$ue 3D Motion and Animation Dataset ($\texttt{BigMaQ}$), a large-scale dataset comprising more than 750 scenes of interacting rhesus macaques with detailed 3D pose descriptions. Extending previous surface-based animal tracking methods, we construct subject-specific textured avatars by adapting a high-quality macaque template mesh to individual monkeys. This allows us to provide pose descriptions that are more accurate than previous state-of-the-art surface-based animal tracking methods. From the original dataset, we derive BigMaQ500, an action recognition benchmark that links surface-based pose vectors to single frames across multiple individual monkeys. By pairing features extracted from established image and video encoders with and without our pose descriptors, we demonstrate substantial improvements in mean average precision (mAP) when pose information is included. With these contributions, $\texttt{BigMaQ}$ establishes the first dataset that both integrates dynamic 3D pose-shape representations into the learning task of animal action recognition and provides a rich resource to advance the study of visual appearance, posture, and social interaction in non-human primates. The code and data are publicly available at https://martinivis.github.io/BigMaQ/ .

Method: Created FemaleSaanenGoat dataset with synchronized eight-view RGBD videos of 55 goats, used multi-view DynamicFusion to fuse noisy point clouds into high-fidelity 3D scans, developed SaanenGoat parametric 3D shape model with refined template and 41 skeletal joints, and enabled automated measurement of six body dimensions.

Result: Achieved superior accuracy in both 3D reconstruction and body measurement, enabling high-precision 3D reconstruction from single-view RGBD input and automated measurement of critical body dimensions.

Conclusion: Presents a novel paradigm for large-scale 3D vision applications in precision livestock farming with specialized parametric models for agricultural animals.

Abstract: The lactation performance of Saanen dairy goats, renowned for their high milk yield, is intrinsically linked to their body size, making accurate 3D body measurement essential for assessing milk production potential, yet existing reconstruction methods lack goat-specific authentic 3D data. To address this limitation, we establish the FemaleSaanenGoat dataset containing synchronized eight-view RGBD videos of 55 female Saanen goats (6-18 months). Using multi-view DynamicFusion, we fuse noisy, non-rigid point cloud sequences into high-fidelity 3D scans, overcoming challenges from irregular surfaces and rapid movement. Based on these scans, we develop SaanenGoat, a parametric 3D shape model specifically designed for female Saanen goats. This model features a refined template with 41 skeletal joints and enhanced udder representation, registered with our scan data. A comprehensive shape space constructed from 48 goats enables precise representation of diverse individual variations. With the help of SaanenGoat model, we get high-precision 3D reconstruction from single-view RGBD input, and achieve automated measurement of six critical body dimensions: body length, height, chest width, chest girth, hip width, and hip height. Experimental results demonstrate the superior accuracy of our method in both 3D reconstruction and body measurement, presenting a novel paradigm for large-scale 3D vision applications in precision livestock farming.

Method: Proposes a high-fidelity personalized head representation that disentangles expression and identity, capturing both static global geometry and dynamic expression details. Introduces an expression transfer module for personalized transfer of head pose and expression details between identities, then uses this representation to train a diffusion transformer (DiT)-based generator.

Result: Extensive experiments on self- and cross-reenactment tasks show the method outperforms previous models in identity preservation, expression accuracy, temporal stability, and capturing fine-grained details of complex motion.

Conclusion: The proposed approach successfully addresses limitations in portrait video generation by introducing a more sophisticated head representation and diffusion-based generation framework, enabling high-quality expressive cinematic portrait videos.

Abstract: While diffusion models have shown great potential in portrait generation, generating expressive, coherent, and controllable cinematic portrait videos remains a significant challenge. Existing intermediate signals for portrait generation, such as 2D landmarks and parametric models, have limited disentanglement capabilities and cannot express personalized details due to their sparse or low-rank representation. Therefore, existing methods based on these models struggle to accurately preserve subject identity and expressions, hindering the generation of highly expressive portrait videos. To overcome these limitations, we propose a high-fidelity personalized head representation that more effectively disentangles expression and identity. This representation captures both static, subject-specific global geometry and dynamic, expression-related details. Furthermore, we introduce an expression transfer module to achieve personalized transfer of head pose and expression details between different identities. We use this sophisticated and highly expressive head model as a conditional signal to train a diffusion transformer (DiT)-based generator to synthesize richly detailed portrait videos. Extensive experiments on self- and cross-reenactment tasks demonstrate that our method outperforms previous models in terms of identity preservation, expression accuracy, and temporal stability, particularly in capturing fine-grained details of complex motion.

Method: Proposes MoDES with: 1) Globally-modulated local gating (GMLG) that integrates global layer-wise importance into local routing probabilities, 2) Dual-modality thresholding (DMT) that processes vision and language tokens separately, and 3) Frontier search algorithm for optimal threshold setting exploiting monotonicity properties.

Result: Extensive experiments on 3 model series across 13 benchmarks show MoDES outperforms previous approaches significantly. For Qwen3-VL-MoE-30B-A3B-Instruct with 88% expert skipping, performance improves by 10.67% (97.33% vs 86.66%). Inference speed improves with 2.16× prefilling time and 1.26× decoding time.

Conclusion: MoDES enables efficient and accurate MoE MLLM inference by addressing modality heterogeneity and layer importance, achieving substantial performance gains and speed improvements without requiring training.

Abstract: Mixture-of-Experts (MoE) Multimodal large language models (MLLMs) excel at vision-language tasks, but they suffer from high computational inefficiency. To reduce inference overhead, expert skipping methods have been proposed to deactivate redundant experts based on the current input tokens. However, we find that applying these methods-originally designed for unimodal large language models (LLMs)-to MLLMs results in considerable performance degradation. This is primarily because such methods fail to account for the heterogeneous contributions of experts across MoE layers and modality-specific behaviors of tokens within these layers. Motivated by these findings, we propose MoDES, the first training-free framework that adaptively skips experts to enable efficient and accurate MoE MLLM inference. It incorporates a globally-modulated local gating (GMLG) mechanism that integrates global layer-wise importance into local routing probabilities to accurately estimate per-token expert importance. A dual-modality thresholding (DMT) method is then applied, which processes tokens from each modality separately, to derive the skipping schedule. To set the optimal thresholds, we introduce a frontier search algorithm that exploits monotonicity properties, cutting convergence time from several days to a few hours. Extensive experiments for 3 model series across 13 benchmarks demonstrate that MoDES far outperforms previous approaches. For instance, when skipping 88% experts for Qwen3-VL-MoE-30B-A3B-Instruct, the performance boost is up to 10.67% (97.33% vs. 86.66%). Furthermore, MoDES significantly enhances inference speed, improving the prefilling time by 2.16$\times$ and the decoding time by 1.26$\times$. Our code is available at https://github.com/ModelTC/MoDES.

Method: Generate disease severity labels for unlabeled OCT scans using gradient responses from anomaly detection algorithm. Use these labels to train supervised contrastive learning setup for selecting positive/negative pairs.

Result: Improves biomarker classification accuracy by up to 6% above self-supervised baselines for key Diabetic Retinopathy indicators.

Conclusion: Disease severity-based selection strategy outperforms standard augmentation-based contrastive learning for medical image analysis, better preserving biomarker information.

Abstract: In this paper, we propose a novel selection strategy for contrastive learning for medical images. On natural images, contrastive learning uses augmentations to select positive and negative pairs for the contrastive loss. However, in the medical domain, arbitrary augmentations have the potential to distort small localized regions that contain the biomarkers we are interested in detecting. A more intuitive approach is to select samples with similar disease severity characteristics, since these samples are more likely to have similar structures related to the progression of a disease. To enable this, we introduce a method that generates disease severity labels for unlabeled OCT scans on the basis of gradient responses from an anomaly detection algorithm. These labels are used to train a supervised contrastive learning setup to improve biomarker classification accuracy by as much as 6% above self-supervised baselines for key indicators of Diabetic Retinopathy.

Method: Proposes SSR²-GCD framework using Semi-Supervised Rate Reduction to learn cross-modality representations with structural properties, emphasizing intra-modality alignment, and integrates prompt candidates from Vision Language Models for knowledge transfer.

Result: Extensive experiments on generic and fine-grained benchmark datasets demonstrate superior performance over existing approaches.

Conclusion: Proper intra-modality alignment is essential for effective multi-modal representation learning in GCD, and the proposed SSR²-GCD framework achieves state-of-the-art results by addressing this gap.

Abstract: Generalized Category Discovery (GCD) aims to identify both known and unknown categories, with only partial labels given for the known categories, posing a challenging open-set recognition problem. State-of-the-art approaches for GCD task are usually built on multi-modality representation learning, which is heavily dependent upon inter-modality alignment. However, few of them cast a proper intra-modality alignment to generate a desired underlying structure of representation distributions. In this paper, we propose a novel and effective multi-modal representation learning framework for GCD via Semi-Supervised Rate Reduction, called SSR$^2$-GCD, to learn cross-modality representations with desired structural properties based on emphasizing to properly align intra-modality relationships. Moreover, to boost knowledge transfer, we integrate prompt candidates by leveraging the inter-modal alignment offered by Vision Language Models. We conduct extensive experiments on generic and fine-grained benchmark datasets demonstrating superior performance of our approach.

Method: Proposes an enhanced Gaussian kernel that explicitly models specular effects through view-dependent opacity, along with an error-driven compensation strategy. Starts with 2D Gaussian initialization and adaptively inserts/optimizes enhanced Gaussian kernels to produce an augmented radiance field.

Result: Method surpasses state-of-the-art NeRF methods in rendering performance while achieving greater parameter efficiency.

Conclusion: The proposed enhanced Gaussian kernel with view-dependent opacity modeling effectively addresses specular reflection limitations in 3DGS, improving rendering quality and efficiency.

Abstract: Due to the real-time rendering performance, 3D Gaussian Splatting (3DGS) has emerged as the leading method for radiance field reconstruction. However, its reliance on spherical harmonics for color encoding inherently limits its ability to separate diffuse and specular components, making it challenging to accurately represent complex reflections. To address this, we propose a novel enhanced Gaussian kernel that explicitly models specular effects through view-dependent opacity. Meanwhile, we introduce an error-driven compensation strategy to improve rendering quality in existing 3DGS scenes. Our method begins with 2D Gaussian initialization and then adaptively inserts and optimizes enhanced Gaussian kernels, ultimately producing an augmented radiance field. Experiments demonstrate that our method not only surpasses state-of-the-art NeRF methods in rendering performance but also achieves greater parameter efficiency. Project page at: https://xiaoxinyyx.github.io/augs.

Method: Proposes two key components: 1) SO(3)-equivariant convolutional implicit network that estimates point-level attributes with SO(3)-equivariance at arbitrary query locations, and 2) Positive-Incentive Point Sampling (PIPS) strategy that dynamically determines sampling locations based on input to boost accuracy and training efficiency.

Result: Outperforms state-of-the-art on three pose estimation datasets. Shows significant improvements in challenging scenarios including objects with unseen poses, high occlusion, novel geometry, and severe noise.

Conclusion: The combination of SO(3)-equivariant network and adaptive sampling strategy effectively addresses challenges in neural implicit field-based pose estimation, particularly for unobserved regions, leading to superior performance in difficult real-world scenarios.

Abstract: Learning neural implicit fields of 3D shapes is a rapidly emerging field that enables shape representation at arbitrary resolutions. Due to the flexibility, neural implicit fields have succeeded in many research areas, including shape reconstruction, novel view image synthesis, and more recently, object pose estimation. Neural implicit fields enable learning dense correspondences between the camera space and the object’s canonical space-including unobserved regions in camera space-significantly boosting object pose estimation performance in challenging scenarios like highly occluded objects and novel shapes. Despite progress, predicting canonical coordinates for unobserved camera-space regions remains challenging due to the lack of direct observational signals. This necessitates heavy reliance on the model’s generalization ability, resulting in high uncertainty. Consequently, densely sampling points across the entire camera space may yield inaccurate estimations that hinder the learning process and compromise performance. To alleviate this problem, we propose a method combining an SO(3)-equivariant convolutional implicit network and a positive-incentive point sampling (PIPS) strategy. The SO(3)-equivariant convolutional implicit network estimates point-level attributes with SO(3)-equivariance at arbitrary query locations, demonstrating superior performance compared to most existing baselines. The PIPS strategy dynamically determines sampling locations based on the input, thereby boosting the network’s accuracy and training efficiency. Our method outperforms the state-of-the-art on three pose estimation datasets. Notably, it demonstrates significant improvements in challenging scenarios, such as objects captured with unseen pose, high occlusion, novel geometry, and severe noise.

Method: Progressive DSS framework with Feature-coherent Object Discovery module using visual features for diverse proposals, segmentation module refining proposals via SAM, and Semantic-driven Mask Selection module using MLLMs to evaluate and select optimal mask

Result: Achieves state-of-the-art performance on multiple COS benchmarks without training or supervision, especially effective in multiple-instance scenes

Conclusion: DSS framework effectively addresses limitations of current MLLM-based approaches through progressive refinement and better integration of visual features with MLLM semantic understanding

Abstract: Current zero-shot Camouflaged Object Segmentation methods typically employ a two-stage pipeline (discover-then-segment): using MLLMs to obtain visual prompts, followed by SAM segmentation. However, relying solely on MLLMs for camouflaged object discovery often leads to inaccurate localization, false positives, and missed detections. To address these issues, we propose the \textbf{D}iscover-\textbf{S}egment-\textbf{S}elect (\textbf{DSS}) mechanism, a progressive framework designed to refine segmentation step by step. The proposed method contains a Feature-coherent Object Discovery (FOD) module that leverages visual features to generate diverse object proposals, a segmentation module that refines these proposals through SAM segmentation, and a Semantic-driven Mask Selection (SMS) module that employs MLLMs to evaluate and select the optimal segmentation mask from multiple candidates. Without requiring any training or supervision, DSS achieves state-of-the-art performance on multiple COS benchmarks, especially in multiple-instance scenes.

Method: Proposes RL-RIG with four components: Diffuser (generates images), Checker (evaluates spatial accuracy), Actor (generates edit prompts), and Inverse Diffuser (edits images). Uses Reflection-GRPO to train the VLM Actor and Image Editor, following a Generate-Reflect-Edit paradigm to enable Chain of Thought reasoning.

Result: Outperforms existing state-of-the-art open-source models by up to 11% in controllable and precise spatial reasoning on the LAION-SG dataset, using Scene Graph IoU and VLM-as-a-Judge evaluation metrics.

Conclusion: The reflection-based reinforcement learning framework successfully addresses the spatial reasoning dilemma in image generation, enabling better structural integrity and spatial accuracy through iterative refinement.

Abstract: Recent advancements in image generation have achieved impressive results in producing high-quality images. However, existing image generation models still generally struggle with a spatial reasoning dilemma, lacking the ability to accurately capture fine-grained spatial relationships from the prompt and correctly generate scenes with structural integrity. To mitigate this dilemma, we propose RL-RIG, a Reinforcement Learning framework for Reflection-based Image Generation. Our architecture comprises four primary components: Diffuser, Checker, Actor, and Inverse Diffuser, following a Generate-Reflect-Edit paradigm to spark the Chain of Thought reasoning ability in image generation for addressing the dilemma. To equip the model with better intuition over generation trajectories, we further develop Reflection-GRPO to train the VLM Actor for edit prompts and the Image Editor for better image quality under a given prompt, respectively. Unlike traditional approaches that solely produce visually stunning yet structurally unreasonable content, our evaluation metrics prioritize spatial accuracy, utilizing Scene Graph IoU and employing a VLM-as-a-Judge strategy to assess the spatial consistency of generated images on LAION-SG dataset. Experimental results show that RL-RIG outperforms existing state-of-the-art open-source models by up to 11% in terms of controllable and precise spatial reasoning in image generation.

Method: Proposes 3D projection method for 4D RADE tensors that reduces data size by 91.9% while preserving Doppler and Elevation features. Introduces RADE-Net with backbone exploiting low/high-level Radar cues using spatial and channel-attention, and decoupled detection heads predicting object center-points in Range-Azimuth domain and regressing rotated 3D boxes.

Result: Achieves 16.7% improvement over baseline and 6.5% improvement over current Radar-only models on K-Radar dataset. Outperforms several Lidar approaches in adverse weather conditions.

Conclusion: The proposed 3D projection method and RADE-Net enable efficient, robust 3D object detection using Radar data, particularly effective in adverse weather conditions where optical sensors struggle.

Abstract: Automotive perception systems are obligated to meet high requirements. While optical sensors such as Camera and Lidar struggle in adverse weather conditions, Radar provides a more robust perception performance, effectively penetrating fog, rain, and snow. Since full Radar tensors have large data sizes and very few datasets provide them, most Radar-based approaches work with sparse point clouds or 2D projections, which can result in information loss. Additionally, deep learning methods show potential to extract richer and more dense features from low level Radar data and therefore significantly increase the perception performance. Therefore, we propose a 3D projection method for fast-Fourier-transformed 4D Range-Azimuth-Doppler-Elevation (RADE) tensors. Our method preserves rich Doppler and Elevation features while reducing the required data size for a single frame by 91.9% compared to a full tensor, thus achieving higher training and inference speed as well as lower model complexity. We introduce RADE-Net, a lightweight model tailored to 3D projections of the RADE tensor. The backbone enables exploitation of low-level and high-level cues of Radar tensors with spatial and channel-attention. The decoupled detection heads predict object center-points directly in the Range-Azimuth domain and regress rotated 3D bounding boxes from rich feature maps in the cartesian scene. We evaluate the model on scenes with multiple different road users and under various weather conditions on the large-scale K-Radar dataset and achieve a 16.7% improvement compared to their baseline, as well as 6.5% improvement over current Radar-only models. Additionally, we outperform several Lidar approaches in scenarios with adverse weather conditions. The code is available under https://github.com/chr-is-tof/RADE-Net.

Method: Token-UNet maintains convolutional UNet encoder but applies TokenLearner to 3D feature maps to pool a preset number of tokens from local/global structures. Uses TokenFuser modules to encase Transformers into UNet architecture, reducing token count and computational complexity while preserving global attention capabilities.

Result: Achieves better average performance (87.21% ± 0.35% Dice score) vs SwinUNETR (86.75% ± 0.19%) while reducing memory footprint to 33%, computation times to 10%, and parameter counts to 35% of SwinUNETR values. Tokenization yields interpretable attention maps.

Conclusion: Token-UNet enables efficient 3D segmentation in constrained computational environments, opening way for more efficient training with limited resources. Facilitates model optimization, fine-tuning, and transfer-learning in hardware-limited settings for medical imaging research.

Abstract: We present Token-UNet, adopting the TokenLearner and TokenFuser modules to encase Transformers into UNets. While Transformers have enabled global interactions among input elements in medical imaging, current computational challenges hinder their deployment on common hardware. Models like (Swin)UNETR adapt the UNet architecture by incorporating (Swin)Transformer encoders, which process tokens that each represent small subvolumes ($8^3$ voxels) of the input. The Transformer attention mechanism scales quadratically with the number of tokens, which is tied to the cubic scaling of 3D input resolution. This work reconsiders the role of convolution and attention, introducing Token-UNets, a family of 3D segmentation models that can operate in constrained computational environments and time frames. To mitigate computational demands, our approach maintains the convolutional encoder of UNet-like models, and applies TokenLearner to 3D feature maps. This module pools a preset number of tokens from local and global structures. Our results show this tokenization effectively encodes task-relevant information, yielding naturally interpretable attention maps. The memory footprint, computation times at inference, and parameter counts of our heaviest model are reduced to 33%, 10%, and 35% of the SwinUNETR values, with better average performance (86.75% $\pm 0.19%$ Dice score for SwinUNETR vs our 87.21% $\pm 0.35%$). This work opens the way to more efficient trainings in contexts with limited computational resources, such as 3D medical imaging. Easing model optimization, fine-tuning, and transfer-learning in limited hardware settings can accelerate and diversify the development of approaches, for the benefit of the research community.

Method: Proposed modality-agnostic framework that closes modality gap in medical alignment, ensuring semantically related representations are more aligned regardless of source modality

Result: Method enhances alignment between radiology images and clinical text, improving cross-modal retrieval and image captioning

Conclusion: Modality gap exists in medical multimodal learning and can be effectively addressed with proposed framework, leading to better semantic alignment and downstream task performance

Abstract: In multimodal learning, CLIP has emerged as the de-facto approach for mapping different modalities into a shared latent space by bringing semantically similar representations closer while pushing apart dissimilar ones. However, CLIP-based contrastive losses exhibit unintended behaviors that negatively impact true semantic alignment, leading to sparse and fragmented latent spaces. This phenomenon, known as the modality gap, has been partially mitigated for standard text and image pairs but remains unknown and unresolved in more complex multimodal settings, such as the medical domain. In this work, we study this phenomenon in the latter case, revealing that the modality gap is present also in medical alignment, and we propose a modality-agnostic framework that closes this gap, ensuring that semantically related representations are more aligned, regardless of their source modality. Our method enhances alignment between radiology images and clinical text, improving cross-modal retrieval and image captioning.

Method: Two-stage fine-tuning framework: Stage 1 addresses adversarial attacks via tailored adversarial training that primarily fine-tunes the encoder while conditionally updating the decoder, moving images to non-attackable regions. Stage 2 handles distortion and regeneration attacks via direct image optimization with a principled constrained image loss that balances deviation from cover and previous encoded images, plus quality-aware early stopping.

Result: AdvMark achieves the highest image quality and comprehensive robustness, with up to 29%, 33%, and 46% accuracy improvements for distortion, regeneration, and adversarial attacks respectively compared to existing methods.

Conclusion: The proposed decoupled two-stage framework effectively addresses multiple attack types while preserving clean accuracy, offering superior robustness and image quality for deep learning-based image watermarking systems.

Abstract: Deep learning-based image watermarking, while robust against conventional distortions, remains vulnerable to advanced adversarial and regeneration attacks. Conventional countermeasures, which jointly optimize the encoder and decoder via a noise layer, face 2 inevitable challenges: (1) decrease of clean accuracy due to decoder adversarial training and (2) limited robustness due to simultaneous training of all three advanced attacks. To overcome these issues, we propose AdvMark, a novel two-stage fine-tuning framework that decouples the defense strategies. In stage 1, we address adversarial vulnerability via a tailored adversarial training paradigm that primarily fine-tunes the encoder while only conditionally updating the decoder. This approach learns to move the image into a non-attackable region, rather than modifying the decision boundary, thus preserving clean accuracy. In stage 2, we tackle distortion and regeneration attacks via direct image optimization. To preserve the adversarial robustness gained in stage 1, we formulate a principled, constrained image loss with theoretical guarantees, which balances the deviation from cover and previous encoded images. We also propose a quality-aware early-stop to further guarantee the lower bound of visual quality. Extensive experiments demonstrate AdvMark outperforms with the highest image quality and comprehensive robustness, i.e. up to 29%, 33% and 46% accuracy improvement for distortion, regeneration and adversarial attacks, respectively.

Method: Three key designs: (1) Gaussian Mixture Noise (GMN) for continuous trajectory space representation, (2) MeanFlow Identity adaptation for end-to-end planning that models mean velocity fields instead of instantaneous ones, eliminating ODE solver errors, and (3) lightweight Adaptive Reconstruction Module (ARM) for implicit trajectory selection or reconstruction via attention weights.

Result: Experiments on NAVSIM closed-loop benchmark show outstanding performance without PDM Score supervision and exceptional inference efficiency, offering a robust and efficient solution for end-to-end autonomous driving.

Conclusion: MeanFuser provides a continuous, efficient, and robust alternative to discrete anchor-based methods for autonomous driving trajectory planning, with improved performance and faster inference.

Abstract: Generative models have shown great potential in trajectory planning. Recent studies demonstrate that anchor-guided generative models are effective in modeling the uncertainty of driving behaviors and improving overall performance. However, these methods rely on discrete anchor vocabularies that must sufficiently cover the trajectory distribution during testing to ensure robustness, inducing an inherent trade-off between vocabulary size and model performance. To overcome this limitation, we propose MeanFuser, an end-to-end autonomous driving method that enhances both efficiency and robustness through three key designs. (1) We introduce Gaussian Mixture Noise (GMN) to guide generative sampling, enabling a continuous representation of the trajectory space and eliminating the dependency on discrete anchor vocabularies. (2) We adapt ``MeanFlow Identity" to end-to-end planning, which models the mean velocity field between GMN and trajectory distribution instead of the instantaneous velocity field used in vanilla flow matching methods, effectively eliminating numerical errors from ODE solvers and significantly accelerating inference. (3) We design a lightweight Adaptive Reconstruction Module (ARM) that enables the model to implicitly select from all sampled proposals or reconstruct a new trajectory when none is satisfactory via attention weights. Experiments on the NAVSIM closed-loop benchmark demonstrate that MeanFuser achieves outstanding performance without the supervision of the PDM Score. and exceptional inference efficiency, offering a robust and efficient solution for end-to-end autonomous driving. Our code and model are available at https://github.com/wjl2244/MeanFuser.

Method: Identified bias in OOD detection through analysis of 11,355 images from three public datasets (including ISIC), annotated artifacts by color, generated color-swapped counterfactuals to rule out dataset bias, and evaluated 40 OOD methods across 7 benchmarks.

Result: Found significant performance drops for most OOD methods when artifacts differed from the model’s ROI. For example, Mahalanobis Score achieved 31.5% higher AUROC when detecting OOD red ink (similar to ROI) compared to black ink (dissimilar) annotations in skin lesion classification.

Conclusion: The Invisible Gorilla Effect reveals an overlooked failure mode in OOD detection where detection performance is biased by visual similarity between artifacts and ROI, providing guidance for developing more robust detectors.

Abstract: Deep Neural Networks achieve high performance in vision tasks by learning features from regions of interest (ROI) within images, but their performance degrades when deployed on out-of-distribution (OOD) data that differs from training data. This challenge has led to OOD detection methods that aim to identify and reject unreliable predictions. Although prior work shows that OOD detection performance varies by artefact type, the underlying causes remain underexplored. To this end, we identify a previously unreported bias in OOD detection: for hard-to-detect artefacts (near-OOD), detection performance typically improves when the artefact shares visual similarity (e.g. colour) with the model’s ROI and drops when it does not - a phenomenon we term the Invisible Gorilla Effect. For example, in a skin lesion classifier with red lesion ROI, we show the method Mahalanobis Score achieves a 31.5% higher AUROC when detecting OOD red ink (similar to ROI) compared to black ink (dissimilar) annotations. We annotated artefacts by colour in 11,355 images from three public datasets (e.g. ISIC) and generated colour-swapped counterfactuals to rule out dataset bias. We then evaluated 40 OOD methods across 7 benchmarks and found significant performance drops for most methods when artefacts differed from the ROI. Our findings highlight an overlooked failure mode in OOD detection and provide guidance for more robust detectors. Code and annotations are available at: https://github.com/HarryAnthony/Invisible_Gorilla_Effect.

Method: Integrates pre-trained semantic feature extractors using two strategies: (1) warped semantic features and (2) alternating understanding and generation at each denoising step in a multi-view diffusion model.

Result: Shows clear qualitative and quantitative improvements (4.69%-15.26% in FID) over state-of-the-art alternatives on multiple datasets.

Conclusion: Incorporating stronger scene semantics through pre-trained feature extractors significantly improves generation quality and consistency in novel view synthesis, especially for distant viewpoints.

Abstract: We present SemanticNVS, a camera-conditioned multi-view diffusion model for novel view synthesis (NVS), which improves generation quality and consistency by integrating pre-trained semantic feature extractors. Existing NVS methods perform well for views near the input view, however, they tend to generate semantically implausible and distorted images under long-range camera motion, revealing severe degradation. We speculate that this degradation is due to current models failing to fully understand their conditioning or intermediate generated scene content. Here, we propose to integrate pre-trained semantic feature extractors to incorporate stronger scene semantics as conditioning to achieve high-quality generation even at distant viewpoints. We investigate two different strategies, (1) warped semantic features and (2) an alternating scheme of understanding and generation at each denoising step. Experimental results on multiple datasets demonstrate the clear qualitative and quantitative (4.69%-15.26% in FID) improvement over state-of-the-art alternatives.

Method: Systematic evaluation of LLMs and VLMs using Answer Set Programming (ASP) ground truth, compiling visualization principles as natural-language statements, generating ~2,000 Vega-Lite specifications with principle violations, plus 300+ real-world charts, evaluating both checking and fixing tasks.

Result: Models show promise as flexible validators and editors of visualization designs, but have persistent gaps with symbolic solvers on nuanced visual perception aspects. Frontier models are better at correcting violations than detecting them reliably.

Conclusion: LLMs and VLMs can serve as visualization principle checkers, though they lag behind symbolic solvers on complex perceptual reasoning, with an interesting asymmetry where correction outperforms detection.

Abstract: Data visualization principles, derived from decades of research in design and perception, ensure proper visual communication. While prior work has shown that large language models (LLMs) can generate charts or flag misleading figures, it remains unclear whether they and their vision-language counterparts (VLMs) can reason about and enforce visualization principles directly. Constraint based systems encode these principles as logical rules for precise automated checks, but translating them into formal specifications demands expert knowledge. This motivates leveraging LLMs and VLMs as principle checkers that can reason about visual design directly, bypassing the need for symbolic rule specification. In this paper, we present the first systematic evaluation of both LLMs and VLMs on their ability to reason about visualization principles, using hard verification ground truth derived from Answer Set Programming (ASP). We compiled a set of visualization principles expressed as natural-language statements and generated a controlled dataset of approximately 2,000 Vega-Lite specifications annotated with explicit principle violations, complemented by over 300 real-world Vega-Lite charts. We evaluated both checking and fixing tasks, assessing how well models detect principle violations and correct flawed chart specifications. Our work highlights both the promise of large (vision-)language models as flexible validators and editors of visualization designs and the persistent gap with symbolic solvers on more nuanced aspects of visual perception. They also reveal an interesting asymmetry: frontier models tend to be more effective at correcting violations than at detecting them reliably.

Method: Systematic evaluation using hard-verification ground truth from Answer Set Programming (ASP), translating Draco’s constraints to natural language, creating 2,000 Vega-Lite specifications with rule violations, and testing LLMs on violation detection accuracy and prompt adherence.

Result: Frontier models achieve high adherence (up to 100%) and reliably detect common violations (F1 up to 0.82), but performance drops significantly for subtler perceptual rules (F1 < 0.15) and technical ASP formulations. Natural language translation improved smaller models by up to 150%.

Conclusion: LLMs show potential as flexible, language-driven visualization rule validators but have current limitations compared to symbolic solvers, particularly for complex perceptual reasoning.

Abstract: Data visualization rules-derived from decades of research in design and perception-ensure trustworthy chart communication. While prior work has shown that large language models (LLMs) can generate charts or flag misleading figures, it remains unclear whether they can reason about and enforce visualization rules directly. Constraint-based systems such as Draco encode these rules as logical constraints for precise automated checks, but maintaining symbolic encodings requires expert effort, motivating the use of LLMs as flexible rule validators. In this paper, we present the first systematic evaluation of LLMs against visualization rules using hard-verification ground truth derived from Answer Set Programming (ASP). We translated a subset of Draco’s constraints into natural-language statements and generated a controlled dataset of 2,000 Vega-Lite specifications annotated with explicit rule violations. LLMs were evaluated on both accuracy in detecting violations and prompt adherence, which measures whether outputs follow the required structured format. Results show that frontier models achieve high adherence (Gemma 3 4B / 27B: 100%, GPT-oss 20B: 98%) and reliably detect common violations (F1 up to 0.82),yet performance drops for subtler perceptual rules (F1 < 0.15 for some categories) and for outputs generated from technical ASP formulations.Translating constraints into natural language improved performance by up to 150% for smaller models. These findings demonstrate the potential of LLMs as flexible, language-driven validators while highlighting their current limitations compared to symbolic solvers.

Method: Flow3r uses factored flow prediction where flow between two images is predicted using geometry latents from one image and pose latents from the other. This factorization guides learning of both scene geometry and camera motion, and extends to dynamic scenes. The framework integrates with existing visual geometry architectures.

Result: Flow3r achieves state-of-the-art results across eight benchmarks spanning static and dynamic scenes, with largest gains on in-the-wild dynamic videos where labeled data is most scarce. Performance scales consistently with unlabeled data (~800K videos used).

Conclusion: Factored flow prediction enables scalable 3D/4D reconstruction from unlabeled monocular videos, addressing the data scarcity problem for dynamic scenes and outperforming existing methods particularly in challenging real-world scenarios.

Abstract: Current feed-forward 3D/4D reconstruction systems rely on dense geometry and pose supervision – expensive to obtain at scale and particularly scarce for dynamic real-world scenes. We present Flow3r, a framework that augments visual geometry learning with dense 2D correspondences (`flow’) as supervision, enabling scalable training from unlabeled monocular videos. Our key insight is that the flow prediction module should be factored: predicting flow between two images using geometry latents from one and pose latents from the other. This factorization directly guides the learning of both scene geometry and camera motion, and naturally extends to dynamic scenes. In controlled experiments, we show that factored flow prediction outperforms alternative designs and that performance scales consistently with unlabeled data. Integrating factored flow into existing visual geometry architectures and training with ${\sim}800$K unlabeled videos, Flow3r achieves state-of-the-art results across eight benchmarks spanning static and dynamic scenes, with its largest gains on in-the-wild dynamic videos where labeled data is most scarce.

Method: The method introduces a Test-Time Training (TTT) layer that compresses multiple image observations into fast weights, creating an implicit 3D representation in latent space. This representation can be decoded into various explicit formats like Gaussian Splats. The model uses pretraining on novel view synthesis tasks and supports both offline reconstruction and online progressive refinement.

Result: The model achieves superior performance in feedforward 3D Gaussian reconstruction compared to state-of-the-art approaches on both objects and scenes, with improved reconstruction quality and faster convergence.

Conclusion: tttLRM demonstrates that test-time training layers can effectively enable efficient large-scale 3D reconstruction with linear complexity, and that pretraining on view synthesis tasks transfers well to explicit 3D modeling tasks.

Abstract: We propose tttLRM, a novel large 3D reconstruction model that leverages a Test-Time Training (TTT) layer to enable long-context, autoregressive 3D reconstruction with linear computational complexity, further scaling the model’s capability. Our framework efficiently compresses multiple image observations into the fast weights of the TTT layer, forming an implicit 3D representation in the latent space that can be decoded into various explicit formats, such as Gaussian Splats (GS) for downstream applications. The online learning variant of our model supports progressive 3D reconstruction and refinement from streaming observations. We demonstrate that pretraining on novel view synthesis tasks effectively transfers to explicit 3D modeling, resulting in improved reconstruction quality and faster convergence. Extensive experiments show that our method achieves superior performance in feedforward 3D Gaussian reconstruction compared to state-of-the-art approaches on both objects and scenes.

Method: Uses Mobile Conditioning Projector (MCP) with depthwise-separable convolutions and layerwise alignment to fuse vision-language features with diffusion generator efficiently. Trained on few million samples with novel quadruplet format (generation prompt, image, question, answer) post-training.

Result: Achieves 74% on GenEval, outperforms Show-O and JanusFlow by 5% and 11% respectively while running 6x and 11x faster. Surpasses them by 15.3% and 5.1% on visual understanding benchmarks. Runs ~3s per 512x512 image on iPhone.

Conclusion: Mobile-O establishes the first practical framework for real-time unified multimodal understanding and generation on edge devices, enabling on-device multimodal intelligence without cloud dependency.

Abstract: Unified multimodal models can both understand and generate visual content within a single architecture. Existing models, however, remain data-hungry and too heavy for deployment on edge devices. We present Mobile-O, a compact vision-language-diffusion model that brings unified multimodal intelligence to a mobile device. Its core module, the Mobile Conditioning Projector (MCP), fuses vision-language features with a diffusion generator using depthwise-separable convolutions and layerwise alignment. This design enables efficient cross-modal conditioning with minimal computational cost. Trained on only a few million samples and post-trained in a novel quadruplet format (generation prompt, image, question, answer), Mobile-O jointly enhances both visual understanding and generation capabilities. Despite its efficiency, Mobile-O attains competitive or superior performance compared to other unified models, achieving 74% on GenEval and outperforming Show-O and JanusFlow by 5% and 11%, while running 6x and 11x faster, respectively. For visual understanding, Mobile-O surpasses them by 15.3% and 5.1% averaged across seven benchmarks. Running in only ~3s per 512x512 image on an iPhone, Mobile-O establishes the first practical framework for real-time unified multimodal understanding and generation on edge devices. We hope Mobile-O will ease future research in real-time unified multimodal intelligence running entirely on-device with no cloud dependency. Our code, models, datasets, and mobile application are publicly available at https://amshaker.github.io/Mobile-O/

Method: Proposes Face Pyramid Vision Transformer (FPVT) with Face Spatial Reduction Attention (FSRA) and Dimensionality Reduction (FDR) layers for compact features, Improved Patch Embedding (IPE) to incorporate CNN benefits into ViTs, and Convolutional Feed-Forward Network (CFFN) for locality information extraction.

Result: Achieves excellent performance on seven benchmark datasets compared to ten state-of-the-art methods (CNNs, pure ViTs, and Convolutional ViTs) despite having fewer parameters.

Conclusion: FPVT successfully integrates CNN and ViT advantages for facial representation learning, demonstrating superior face recognition performance with computational efficiency.

Abstract: A novel Face Pyramid Vision Transformer (FPVT) is proposed to learn a discriminative multi-scale facial representations for face recognition and verification. In FPVT, Face Spatial Reduction Attention (FSRA) and Dimensionality Reduction (FDR) layers are employed to make the feature maps compact, thus reducing the computations. An Improved Patch Embedding (IPE) algorithm is proposed to exploit the benefits of CNNs in ViTs (e.g., shared weights, local context, and receptive fields) to model lower-level edges to higher-level semantic primitives. Within FPVT framework, a Convolutional Feed-Forward Network (CFFN) is proposed that extracts locality information to learn low level facial information. The proposed FPVT is evaluated on seven benchmark datasets and compared with ten existing state-of-the-art methods, including CNNs, pure ViTs, and Convolutional ViTs. Despite fewer parameters, FPVT has demonstrated excellent performance over the compared methods. Project page is available at https://khawar-islam.github.io/fpvt/

Method: Combined human psychophysics experiments with computational modeling. Humans viewed training videos with novel objects in naturalistic scenes following contextual rules. SeCo model uses separate vision encoders for targets and context, with learnable external memory for latent contextual priors, retrieving likely object representations given contextual cues.

Result: Humans rapidly learned contextual associations without labels or feedback and generalized robustly. SeCo outperformed state-of-the-art self-supervised learning approaches and predicted object placements most consistent with human behavior.

Conclusion: Contextual associations play a central role in scene understanding, and biologically inspired models like SeCo can effectively learn and reason about contextual relationships from complex visual scenes.

Abstract: Humans rarely perceive objects in isolation but interpret scenes through relationships among co-occurring elements. How such contextual knowledge is acquired without explicit supervision remains unclear. Here we combine human psychophysics experiments with computational modelling to study the emergence of contextual reasoning. Participants were exposed to novel objects embedded in naturalistic scenes that followed predefined contextual rules capturing global context, local context and crowding. After viewing short training videos, participants completed a “lift-the-flap” task in which a hidden object had to be inferred from the surrounding context under variations in size, resolution and spatial arrangement. Humans rapidly learned these contextual associations without labels or feedback and generalised robustly across contextual changes. We then introduce SeCo (Self-supervised learning for Context Reasoning), a biologically inspired model that learns contextual relationships from complex scenes. SeCo encodes targets and context with separate vision encoders and stores latent contextual priors in a learnable external memory module. Given contextual cues, the model retrieves likely object representations to infer hidden targets. SeCo outperforms state-of-the-art self-supervised learning approaches and predicts object placements most consistent with human behaviour, highlighting the central role of contextual associations in scene understanding.

Method: Proposes a physics-guided neural network with: 1) Adaptive second-order Runge-Kutta method with physical constraints for precise physical state modeling, and 2) Frequency-enhanced Fourier module to strengthen spatiotemporal dynamics estimation.

Result: Outperforms state-of-the-art methods on spatiotemporal and video prediction tasks, achieving best performance in several spatiotemporal scenarios with significantly fewer parameters.

Conclusion: The proposed physics-guided approach with adaptive Runge-Kutta and Fourier enhancement effectively models spatiotemporal dynamics while maintaining computational efficiency.

Abstract: Spatiotemporal prediction is important in solving natural problems and processing video frames, especially in weather forecasting and human action recognition. Recent advances attempt to incorporate prior physical knowledge into the deep learning framework to estimate the unknown governing partial differential equations (PDEs) in complex dynamics, which have shown promising results in spatiotemporal prediction tasks. However, previous approaches only restrict neural network architectures or loss functions to acquire physical or PDE features, which decreases the representative capacity of a neural network. Meanwhile, the updating process of the physical state cannot be effectively estimated. To solve the problems mentioned above, we introduce a physical-guided neural network, which utilizes an adaptive second-order Runge-Kutta method with physical constraints to model the physical states more precisely. Furthermore, we propose a frequency-enhanced Fourier module to strengthen the model’s ability to estimate the spatiotemporal dynamics. We evaluate our model on both spatiotemporal and video prediction tasks. The experimental results show that our model outperforms several state-of-the-art methods and performs the best in several spatiotemporal scenarios with a much smaller parameter count.

Method: Proposes PASS, a hyper-network that takes visual prompts and network weight statistics as input to output layer-wise channel sparsity in a recurrent manner, considering intrinsic channel dependencies between layers.

Result: PASS achieves 1-3% better accuracy on Food101 at same FLOPs level, and 0.35× more speedup than baselines at similar 80% accuracy across multiple architectures and six datasets.

Conclusion: Visual prompts can effectively capture channel importance for structural pruning, enabling high-quality sparsity patterns that balance accuracy and efficiency.

Abstract: Large-scale neural networks have demonstrated remarkable performance in different domains like vision and language processing, although at the cost of massive computation resources. As illustrated by compression literature, structural model pruning is a prominent algorithm to encourage model efficiency, thanks to its acceleration-friendly sparsity patterns. One of the key questions of structural pruning is how to estimate the channel significance. In parallel, work on data-centric AI has shown that prompting-based techniques enable impressive generalization of large language models across diverse downstream tasks. In this paper, we investigate a charming possibility - \textit{leveraging visual prompts to capture the channel importance and derive high-quality structural sparsity}. To this end, we propose a novel algorithmic framework, namely \texttt{PASS}. It is a tailored hyper-network to take both visual prompts and network weight statistics as input, and output layer-wise channel sparsity in a recurrent manner. Such designs consider the intrinsic channel dependency between layers. Comprehensive experiments across multiple network architectures and six datasets demonstrate the superiority of \texttt{PASS} in locating good structural sparsity. For example, at the same FLOPs level, \texttt{PASS} subnetworks achieve $1%\sim 3%$ better accuracy on Food101 dataset; or with a similar performance of $80%$ accuracy, \texttt{PASS} subnetworks obtain $0.35\times$ more speedup than the baselines.

Method: Proposes R²-Mesh with two key components: 1) Uses NeRF’s rendering ability to synthesize additional high-quality images for pseudo-supervision, 2) Introduces UCB-based viewpoint selection with geometry-aware reward to dynamically balance exploration and exploitation of informative viewpoints. Jointly optimizes SDF geometry and view-dependent appearance under differentiable rendering with periodic mesh refinement.

Result: Achieves competitive results in both geometric accuracy and rendering quality compared to existing methods.

Conclusion: The reinforcement learning framework with pseudo-supervision and adaptive viewpoint selection effectively addresses limitations of traditional mesh reconstruction from NeRF, improving both geometry and appearance optimization.

Abstract: Mesh reconstruction from Neural Radiance Fields (NeRF) is widely used in 3D reconstruction and has been applied across numerous domains. However, existing methods typically rely solely on the given training set images, which restricts supervision to limited observations and makes it difficult to fully constrain geometry and appearance. Moreover, the contribution of each viewpoint for training is not uniform and changes dynamically during the optimization process, which can result in suboptimal guidance for both geometric refinement and rendering quality. To address these limitations, we propose $R^2$-Mesh, a reinforcement learning framework that combines NeRF-rendered pseudo-supervision with online viewpoint selection. Our key insight is to exploit NeRF’s rendering ability to synthesize additional high-quality images, enriching training with diverse viewpoint information. To ensure that supervision focuses on the most beneficial perspectives, we introduce a UCB-based strategy with a geometry-aware reward, which dynamically balances exploration and exploitation to identify informative viewpoints throughout training. Within this framework, we jointly optimize SDF geometry and view-dependent appearance under differentiable rendering, while periodically refining meshes to capture fine geometric details. Experiments demonstrate that our method achieves competitive results in both geometric accuracy and rendering quality.

Method: Proposes modeling geometry as distributions using diffusion models with a novel network architecture to learn surface point distributions, making no assumptions about surface genus, connectivity, or boundary conditions.

Result: The approach demonstrates effectiveness in achieving high geometric fidelity across various object types, with applications in textured mesh representation, neural surface compression, dynamic object modeling, and rendering.

Conclusion: The distribution-based geometric representation using diffusion models offers a powerful alternative to coordinate-based networks, advancing 3D geometric learning with better handling of complex geometries.

Abstract: Neural representations of 3D data have been widely adopted across various applications, particularly in recent work leveraging coordinate-based networks to model scalar or vector fields. However, these approaches face inherent challenges, such as handling thin structures and non-watertight geometries, which limit their flexibility and accuracy. In contrast, we propose a novel geometric data representation that models geometry as distributions-a powerful representation that makes no assumptions about surface genus, connectivity, or boundary conditions. Our approach uses diffusion models with a novel network architecture to learn surface point distributions, capturing fine-grained geometric details. We evaluate our representation qualitatively and quantitatively across various object types, demonstrating its effectiveness in achieving high geometric fidelity. Additionally, we explore applications using our representation, such as textured mesh representation, neural surface compression, dynamic object modeling, and rendering, highlighting its potential to advance 3D geometric learning.

Method: IVPT introduces cross-layer concept prototypes that represent visual prompts as category-agnostic prototypes corresponding to specific image regions. These prototypes are aggregated to generate interpretable prompts for multiple network layers, allowing explanations at different depths and semantic granularities.

Result: Comprehensive evaluations on fine-grained classification benchmarks show superior interpretability and performance over both visual prompt tuning methods and existing interpretable methods.

Conclusion: IVPT successfully demonstrates that visual prompt tuning can be made interpretable while maintaining or improving performance, advancing the field toward more reliable and transparent AI systems.

Abstract: Visual prompt tuning offers significant advantages for adapting pre-trained visual foundation models to specific tasks. However, current research provides limited insight into the interpretability of this approach, which is essential for enhancing AI reliability and enabling AI-driven knowledge discovery. In this paper, rather than learning abstract prompt embeddings, we propose the first framework, named Interpretable Visual Prompt Tuning (IVPT), to explore interpretability for visual prompts by introducing cross-layer concept prototypes. Specifically, visual prompts are linked to human-understandable semantic concepts, represented as a set of category-agnostic prototypes, each corresponding to a specific region of the image. IVPT then aggregates features from these regions to generate interpretable prompts for multiple network layers, allowing the explanation of visual prompts at different network depths and semantic granularities. Comprehensive qualitative and quantitative evaluations on fine-grained classification benchmarks show its superior interpretability and performance over visual prompt tuning methods and existing interpretable methods.

Method: Introduces Hier-COS, a unified framework for hierarchy-aware fine-grained and hierarchical multi-level classification. It theoretically guarantees consistency with given hierarchy trees and implicitly adapts learning capacity for different classes based on their position in the hierarchy. Also proposes HOPS, a ranking-based metric to overcome deficiencies in current evaluation standards.

Result: Hier-COS achieves state-of-the-art across all hierarchical metrics on four challenging datasets (including tieredImageNet-H and iNaturalist-19), beating top-1 accuracy in all but one case. It effectively transforms frozen features from pretrained backbones (ViT) to be hierarchy-aware, yielding substantial performance benefits.

Conclusion: Hier-COS provides a theoretically sound framework for hierarchy-aware classification that addresses fundamental limitations in both learning methods and evaluation metrics, demonstrating superior performance on challenging hierarchical classification tasks.

Abstract: Traditional classifiers treat all labels as mutually independent, thereby considering all negative classes to be equally incorrect. This approach fails severely in many real-world scenarios, where a known semantic hierarchy defines a partial order of preferences over negative classes. While hierarchy-aware feature representations have shown promise in mitigating this problem, their performance is typically assessed using metrics like MS and AHD. In this paper, we highlight important shortcomings in existing hierarchical evaluation metrics, demonstrating that they are often incapable of measuring true hierarchical performance. Our analysis reveals that existing methods learn sub-optimal hierarchical representations, despite competitive MS and AHD scores. To counter these issues, we introduce Hier-COS, a novel framework for unified hierarchy-aware fine-grained and hierarchical multi-level classification. We show that Hier-COS is theoretically guaranteed to be consistent with the given hierarchy tree. Furthermore, our framework implicitly adapts the learning capacity for different classes based on their position within the hierarchy tree-a vital property absent in existing methods. Finally, to address the limitations of evaluation metrics, we propose HOPS, a ranking-based metric that demonstrably overcomes the deficiencies of current evaluation standards. We benchmark Hier-COS on four challenging datasets, including the deep and imbalanced tieredImageNet-H and iNaturalist-19. Through extensive experiments, we demonstrate that Hier-COS achieves SOTA across all hierarchical metrics for every dataset, while simultaneously beating the top-1 accuracy in all but one case. Lastly, we show that Hier-COS can effectively learn to transform the frozen features extracted from a pretrained backbone (ViT) to be hierarchy-aware, yielding substantial benefits for hierarchical classification performance.

Method: Three key innovations: 1) Enforcing discriminative class-specific features via orthogonal label embeddings for clearer class separation; 2) Imposing spherical constraint modeling representations as mixture of von Mises-Fisher distributions; 3) Integrating Mixup and Label Smoothing directly into representation learning stage. Introduces Angular Separability (AS) and Norm Separability (NS) metrics.

Result: Achieves state-of-the-art results (AUROC and OSCR) across various coarse-grained and fine-grained open-set benchmarks, with improvements up to 5.1% on Semantic Shift Benchmark.

Conclusion: SpHOR demonstrates that custom-designed representation learning specifically for OSR significantly improves performance over generic approaches, with the three innovations collectively enhancing feature space for better unknown class identification.

Abstract: The reliance on Deep Neural Network (DNN)-based classifiers in safety-critical and real-world applications necessitates Open-Set Recognition (OSR). OSR enables the identification of input data from classes unknown during training as unknown, as opposed to misclassifying them as belonging to a known class. DNNs consist of a feature extraction backbone and classifier head; however, most OSR methods typically train both components jointly, often yielding feature representations that adapt poorly to unknown data. Other approaches employ off-the-shelf objectives, such as supervised contrastive learning, which are not specifically designed for OSR. To address these limitations, we propose SpHOR, which explicitly shapes the feature space via supervised representation learning, before training a classifier. Instead of relying on generic feature learning, SpHOR custom-designs representation learning for OSR through three key innovations: (1) enforcing discriminative class-specific features via orthogonal label embeddings, ensuring clearer separation between classes. (2) imposing a spherical constraint, modeling representations as a mixture of von Mises-Fisher distributions. (3) integrating Mixup and Label Smoothing (LS) directly into the representation learning stage. To quantify how these techniques enhance representations for OSR, we introduce two metrics: the Angular Separability (AS) and Norm Separability (NS). Combining all three innovations, SpHOR achieves state-of-the-art results (in AUROC and OSCR) across various coarse-grained and fine-grained open-set benchmarks, particularly excelling on the Semantic Shift Benchmark with improvements up to 5.1%. Code at https://github.com/nadarasarbahavan/SpHOR

Method: Proposes Parsing Skeleton representation using skeleton-guided human parsing to capture detailed body dynamics. Introduces PSGait framework that fuses Parsing Skeleton with silhouettes in a multimodal approach to enhance individual differentiation.

Result: PSGait outperforms state-of-the-art multimodal methods while significantly reducing computational resources. As a plug-and-play method, it achieves up to 15.7% improvement in Rank-1 accuracy across various models.

Conclusion: Parsing Skeleton is a lightweight, effective, and highly generalizable representation for gait recognition in real-world scenarios, validated by comprehensive benchmarks showing superior performance with reduced computational requirements.

Abstract: Gait recognition has emerged as a robust biometric modality due to its non-intrusive nature. Conventional gait recognition methods mainly rely on silhouettes or skeletons. While effective in controlled laboratory settings, their limited information entropy restricts generalization to real-world scenarios. To overcome this, we propose a novel representation called \textbf{Parsing Skeleton}, which uses a skeleton-guided human parsing method to capture fine-grained body dynamics with much higher information entropy. To effectively explore the capability of the Parsing Skeleton, we also introduce \textbf{PSGait}, a framework that fuses Parsing Skeleton with silhouettes to enhance individual differentiation. Comprehensive benchmarks demonstrate that PSGait outperforms state-of-the-art multimodal methods while significantly reducing computational resources. As a plug-and-play method, it achieves an improvement of up to 15.7% in the accuracy of Rank-1 in various models. These results validate the Parsing Skeleton as a \textbf{lightweight}, \textbf{effective}, and highly \textbf{generalizable} representation for gait recognition in the wild. Code is available at https://github.com/realHarryX/PSGait.

Method: Two key innovations: (1) Role-based agentic workflow with planner, grounder, verifier, and answerer; (2) Chain-of-LoRA mechanism using unified base model with multiple LoRA adapters for efficient role switching

Result: Extensive experiments on 15 benchmarks across Grounded VideoQA, Video Temporal Grounding, and General VideoQA demonstrate effectiveness in advancing video agents, test-time scaling, and long-form video reasoning

Conclusion: VideoMind effectively addresses temporal-grounded video reasoning through innovative role-based workflow and efficient Chain-of-LoRA mechanism

Abstract: Videos, with their unique temporal dimension, demand precise grounded understanding, where answers are directly linked to visual, interpretable evidence. Despite significant breakthroughs in text-based reasoning with large language models, multi-modal reasoning - especially for videos - remains limited. In this work, we fill this gap by introducing VideoMind, a novel video-language agent for temporal-grounded video reasoning. Our method involves two key innovations: (1) We identify four essential capabilities for grounded video reasoning and propose a role-based agentic workflow, comprising a planner to coordinate roles, a grounder for temporal event localization, a verifier to assess event candidates, and an answerer for question answering. (2) To efficiently integrate these roles during inference, we propose a novel Chain-of-LoRA mechanism, where a unified base model with multiple LoRA adapters is leveraged to enable seamless role switching, balancing efficiency and flexibility. Extensive experiments on 15 benchmarks across Grounded VideoQA, Video Temporal Grounding, and General VideoQA tasks demonstrate the effectiveness of the proposed scheme in advancing video agent, test-time scaling, and long-form video reasoning. Code, models, datasets, and demos are available at https://videomind.github.io/.

Method: ShapeShift uses a deformable boundary represented as a phase field that expands anisotropically, guided by intermediate features from the diffusion model. This creates space along semantically coherent directions rather than just geometrically optimal ones, coupling semantic guidance with feasibility constraint resolution.

Result: Experiments show ShapeShift produces arrangements that achieve both semantic clarity and overlap-free validity, significantly outperforming baselines that treat semantic guidance and physical feasibility as independent objectives.

Conclusion: The method successfully leverages diffusion model features to encode not just what a concept looks like, but its geometric and directional structure, enabling semantically-aware overlap resolution for physically valid arrangements that convey language-specified concepts.

Abstract: We present ShapeShift, a method for arranging rigid objects into configurations that visually convey semantic concepts specified by natural language. While pretrained diffusion models provide powerful semantic guidance, such as Score Distillation Sampling, enforcing physical validity poses a fundamental challenge. Naive overlap resolution disrupts semantic structure – separating overlapping shapes along geometrically optimal directions (minimum translation vectors) often destroys the very arrangements that make concepts recognizable. Our intuition is that diffusion model features encode not just what a concept looks like, but its geometric, directional structure – how it is oriented and shaped – which we leverage to make overlap resolution semantically aware. We introduce a deformable boundary represented as a phase field that expands anisotropically, guided by intermediate features from the diffusion model, creating space along semantically coherent directions. Experiments demonstrate that ShapeShift, by coupling semantic guidance and feasibility constraint resolution, produces arrangements achieving both semantic clarity and overlap-free validity, significantly outperforming baselines that treat these objectives independently.

Method: Created the Qualcomm Interactive Video Dataset (IVD) with question-answering setup where models must answer questions in real-time based on camera and audio input, then evaluated existing models and fine-tuned them.

Result: Existing models perform far below human level on real-time multimodal conversation tasks, but fine-tuning on this data significantly reduces the performance gap for many perceptual skills.

Conclusion: Real-time multimodal conversation about live scenes remains challenging for current AI models, but targeted fine-tuning on appropriate datasets can substantially improve performance toward human-level interaction.

Abstract: AI models have made significant strides in recent years in their ability to describe and answer questions about real-world images. They have also made progress in the ability to converse with users in real-time using audio input. This raises the question: have we reached the point where AI models, connected to a camera and microphone, can converse with users in real-time about scenes and events that are unfolding live in front of the camera? This has been a long-standing goal in AI and is a prerequisite for real-world AI assistants and humanoid robots to interact with humans in everyday situations. In this work, we introduce a new dataset and benchmark, the Qualcomm Interactive Video Dataset (IVD), which allows us to assess the extent to which existing models can support these abilities, and to what degree these capabilities can be instilled through fine-tuning. The dataset is based on a simple question-answering setup, where users ask questions that the system has to answer, in real-time, based on the camera and audio input. We show that existing models fall far behind human performance on this task, and we identify the main sources for the performance gap. However, we also show that for many of the required perceptual skills, fine-tuning on this form of data can significantly reduce this gap.

Method: Proposes Brain-Inspired Analogical Generator (BiAG) with three components: Weight Self-Attention Module (WSA) supplements new class weights, Weight & Prototype Analogical Attention Module (WPAA) computes analogies to generate new class weights, and Semantic Conversion Module (SCM) uses Neural Collapse theory for semantic conversion.

Result: Experiments on miniImageNet, CUB-200, and CIFAR-100 datasets demonstrate higher final and average accuracy compared to state-of-the-art methods.

Conclusion: The brain-inspired analogical generative method effectively addresses FSCIL challenges by generating new class weights without fine-tuning, leveraging analogical reasoning similar to human learning mechanisms.

Abstract: Few-shot class-incremental Learning (FSCIL) enables models to learn new classes from limited data while retaining performance on previously learned classes. Traditional FSCIL methods often require fine-tuning parameters with limited new class data and suffer from a separation between learning new classes and utilizing old knowledge. Inspired by the analogical learning mechanisms of the human brain, we propose a novel analogical generative method. Our approach includes the Brain-Inspired Analogical Generator (BiAG), which derives new class weights from existing classes without parameter fine-tuning during incremental stages. BiAG consists of three components: Weight Self-Attention Module (WSA), Weight & Prototype Analogical Attention Module (WPAA), and Semantic Conversion Module (SCM). SCM uses Neural Collapse theory for semantic conversion, WSA supplements new class weights, and WPAA computes analogies to generate new class weights. Experiments on miniImageNet, CUB-200, and CIFAR-100 datasets demonstrate that our method achieves higher final and average accuracy compared to SOTA methods.

Method: STEP (Self-attentive Temporal Embedding Probing) extends conventional probing with frame-wise positional encodings to model temporal order, a global CLS token, and a simplified attention block, creating a lightweight temporal modeling approach.

Result: STEP improves accuracy by 4-10% on nearly symmetric actions and 6-15% overall across action recognition benchmarks in human-robot interaction, industrial assembly, and driver assistance, outperforming both PEFT methods and fully fine-tuned models.

Conclusion: STEP provides an effective lightweight solution for temporal modeling in vision foundation models, establishing new state-of-the-art for action recognition in human-robot interaction while being computationally practical for real-world robotics.

Abstract: Fine-grained understanding of human actions is essential for safe and intuitive human–robot interaction. We study the challenge of recognizing nearly symmetric actions, such as picking up vs. placing down a tool or opening vs. closing a drawer. These actions are common in close human-robot collaboration, yet they are rare and largely overlooked in mainstream vision frameworks. Pretrained vision foundation models (VFMs) are often adapted using probing, valued in robotics for its efficiency and low data needs, or parameter-efficient fine-tuning (PEFT), which adds temporal modeling through adapters or prompts. However, our analysis shows that probing is permutation-invariant and blind to frame order, while PEFT is prone to overfitting on smaller HRI datasets, and less practical in real-world robotics due to compute constraints. To address this, we introduce STEP (Self-attentive Temporal Embedding Probing), a lightweight extension to probing that models temporal order via frame-wise positional encodings, a global CLS token, and a simplified attention block. Compared to conventional probing, STEP improves accuracy by 4–10% on nearly symmetric actions and 6–15% overall across action recognition benchmarks in human-robot-interaction, industrial assembly, and driver assistance. Beyond probing, STEP surpasses heavier PEFT methods and even outperforms fully fine-tuned models on all three benchmarks, establishing a new state-of-the-art. Code and models will be made publicly available: https://github.com/th-nesh/STEP.

Method: Proposes Meta Document Attention Network with two key components: 1) windowed queries to process multiple transformer queries together for better context modeling with near-future information, and 2) multi-token predictions to predict several tokens per query instead of just the next token.

Result: Achieves state-of-the-art results on average in terms of character error rate across 10 full-page handwritten datasets while significantly reducing prediction time.

Conclusion: Meta-DAN effectively addresses the speed limitations of character-level autoregressive decoding for page-level text recognition while maintaining or improving accuracy through better context modeling.

Abstract: Recent advances in text recognition led to a paradigm shift for page-level recognition, from multi-step segmentation-based approaches to end-to-end attention-based ones. However, the naïve character-level autoregressive decoding process results in long prediction times: it requires several seconds to process a single page image on a modern GPU. We propose the Meta Document Attention Network (Meta-DAN) as a novel decoding strategy to reduce the prediction time while enabling a better context modeling. It relies on two main components: windowed queries, to process several transformer queries altogether, enlarging the context modeling with near future; and multi-token predictions, whose goal is to predict several tokens per query instead of only the next one. We evaluate the proposed approach on 10 full-page handwritten datasets and demonstrate state-of-the-art results on average in terms of character error rate. Source code and weights of trained models are available at https://github.com/FactoDeepLearning/meta_dan.

Method: The framework combines tailored heatmap generation, loss design, inference logic, and robust hyperparameters for heatmap regression, building upon nnU-Net’s self-configuration and training engine. It provides data conversion utilities for public benchmarks and standardized evaluation.

Result: nnLandmark achieves state-of-the-art performance across five public and one private dataset, benchmarked against three recently published methods. It enables training strong models on new datasets without expert knowledge or hyperparameter tuning.

Conclusion: nnLandmark serves as both a strong common baseline and flexible standardized environment for developing and evaluating new methodological contributions in 3D medical landmark detection, advancing the field through systematic, transparent benchmarking.

Abstract: Landmark detection is central to many medical applications, such as identifying critical structures for treatment planning or defining control points for biometric measurements. However, manual annotation is labor-intensive and requires expert anatomical knowledge. While deep learning shows promise in automating this task, fair evaluation and interpretation of methods in a broader context are hindered by limited public benchmarking, inconsistent baseline implementations, and non-standardized experimentation. To overcome these pitfalls, we present nnLandmark, a self-configuring framework for 3D landmark detection that combines tailored heatmap generation, loss design, inference logic, and a robust set of hyperparameters for heatmap regression, while reusing components from nnU-Net’s underlying self-configuration and training engine. nnLandmark achieves state-of-the-art performance across five public and one private dataset, benchmarked against three recently published methods. Its out-of-the-box usability enables training strong landmark detection models on new datasets without expert knowledge or dataset-specific hyperparameter tuning. Beyond accuracy, nnLandmark provides both a strong, common baseline and a flexible, standardized environment for developing and evaluating new methodological contributions. It further streamlines evaluation across multiple datasets by offering data conversion utilities for current public benchmarks. Together, these properties position nnLandmark as a central tool for advancing 3D medical landmark detection through systematic, transparent benchmarking, enabling to genuinely measure methodological progress. The code is available on GitHub: https://github.com/MIC-DKFZ/nnLandmark

Method: Introduces confidence-guided attention with two key steps: (1) confidence-guided bias to adjust attention distributions for each query pixel, avoiding irrelevant interactions between non-overlapping pixels; (2) using confidence maps to rescale value features during feature aggregation to attenuate uncertain regions. Also adds a classification loss to encourage backbone features to discriminate between matchable and non-matchable regions.

Result: Extensive experiments on three benchmarks demonstrate that the proposed method outperforms existing state-of-the-art methods in semi-dense feature matching.

Conclusion: The confidence-guided attention mechanism effectively reduces noise and redundancy in feature matching by adaptively pruning attention weights based on matching confidence, leading to improved performance over existing methods.

Abstract: Semi-dense feature matching methods have been significantly advanced by leveraging attention mechanisms to extract discriminative descriptors. However, most existing approaches treat all pixels equally during attention computations, which can potentially introduce noise and redundancy from irrelevant regions. To address this issue, we propose a confidence-guided attention that adaptively prunes attention weights for each pixel based on precomputed matching confidence maps. These maps are generated by evaluating the mutual similarity between feature pairs extracted from the backbone, where high confidence indicates a high potential for matching. Then the attention is refined through two steps: (1) a confidence-guided bias is introduced to adaptively adjust the attention distributions for each query pixel, avoiding irrelevant interactions between non-overlap pixels; (2) the corresponding confidence map is additionally employed to rescale value features during feature aggregation, attenuating the influence of uncertain regions. Moreover, a classification loss is introduced to encourage the backbone’s features to discriminate between matchable and non-matchable regions. Extensive experiments on three benchmarks demonstrate that the proposal outperforms existing state-of-the-art methods.

Method: Created U2-BENCH benchmark with 7,241 ultrasound cases spanning 15 anatomical regions, defining 8 clinical tasks (diagnosis, view recognition, lesion localization, value estimation, report generation) across 50 application scenarios. Evaluated 23 state-of-the-art LVLMs (open/closed source, general/medical).

Result: LVLMs show strong performance on image-level classification but struggle with spatial reasoning and clinical language generation. The benchmark reveals persistent challenges in these areas.

Conclusion: U2-BENCH establishes a rigorous testbed for assessing and advancing LVLM research in medical ultrasound imaging, highlighting current limitations and future directions for multimodal medical AI.

Abstract: Ultrasound is a widely-used imaging modality critical to global healthcare, yet its interpretation remains challenging due to its varying image quality on operators, noises, and anatomical structures. Although large vision-language models (LVLMs) have demonstrated impressive multimodal capabilities across natural and medical domains, their performance on ultrasound remains largely unexplored. We introduce U2-BENCH, the first comprehensive benchmark to evaluate LVLMs on ultrasound understanding across classification, detection, regression, and text generation tasks. U2-BENCH aggregates 7,241 cases spanning 15 anatomical regions and defines 8 clinically inspired tasks, such as diagnosis, view recognition, lesion localization, clinical value estimation, and report generation, across 50 ultrasound application scenarios. We evaluate 23 state-of-the-art LVLMs, both open- and closed-source, general-purpose and medical-specific. Our results reveal strong performance on image-level classification, but persistent challenges in spatial reasoning and clinical language generation. U2-BENCH establishes a rigorous and unified testbed to assess and accelerate LVLM research in the uniquely multimodal domain of medical ultrasound imaging.

Method: Dual DiT-based backbone with multimodal attention mechanism that jointly exploits image, text, and mask information to resolve ambiguities, plus an alignment module to refine garment structures and textures for detail preservation.

Result: Achieves new state-of-the-art performance on VITON-HD and Dress Code datasets, substantially improving both visual realism and consistency with target garments.

Conclusion: TEMU-VTOFF effectively addresses limitations of existing VTOFF methods through multimodal integration and explicit detail preservation, enabling practical applications in e-commerce and dataset creation.

Abstract: Virtual try-on (VTON) has been widely explored for rendering garments onto person images, while its inverse task, virtual try-off (VTOFF), remains largely overlooked. VTOFF aims to recover standardized product images of garments directly from photos of clothed individuals. This capability is of great practical importance for e-commerce platforms, large-scale dataset curation, and the training of foundation models. Unlike VTON, which must handle diverse poses and styles, VTOFF naturally benefits from a consistent output format in the form of flat garment images. However, existing methods face two major limitations: (i) exclusive reliance on visual cues from a single photo often leads to ambiguity, and (ii) generated images usually suffer from loss of fine details, limiting their real-world applicability. To address these challenges, we introduce TEMU-VTOFF, a Text-Enhanced MUlti-category framework for VTOFF. Our architecture is built on a dual DiT-based backbone equipped with a multimodal attention mechanism that jointly exploits image, text, and mask information to resolve visual ambiguities and enable robust feature learning across garment categories. To explicitly mitigate detail degradation, we further design an alignment module that refines garment structures and textures, ensuring high-quality outputs. Extensive experiments on VITON-HD and Dress Code show that TEMU-VTOFF achieves new state-of-the-art performance, substantially improving both visual realism and consistency with target garments.

Method: Instead of modifying target vehicles, inserts rendered adversarial objects into scenes with occlusion-aware module for physical plausibility across views and time. Uses BEV spatial feature-guided optimization strategy to maintain attack effectiveness across views and frames by attacking detector’s internal representations.

Result: Extensive experiments show learned universal adversarial objects can consistently degrade multiple BEV detectors from various viewpoints and distances. The environment-manipulation attack paradigm exposes models’ over-reliance on contextual cues.

Conclusion: Provides first framework for generating physically plausible adversarial objects for BEV 3D detectors, offering practical pipeline for robustness evaluation in autonomous driving systems and revealing fundamental vulnerabilities.

Abstract: Adversarial robustness of BEV 3D object detectors is critical for autonomous driving (AD). Existing invasive attacks require altering the target vehicle itself (e.g. attaching patches), making them unrealistic and impractical for real-world evaluation. While non-invasive attacks that place adversarial objects in the environment are more practical, current methods still lack the multi-view and temporal consistency needed for physically plausible threats. In this paper, we present the first framework for generating universal, non-invasive, and 3D-consistent adversarial objects that expose fundamental vulnerabilities for BEV 3D object detectors. Instead of modifying target vehicles, our method inserts rendered objects into scenes with an occlusion-aware module that enforces physical plausibility across views and time. To maintain attack effectiveness across views and frames, we optimize adversarial object appearance using a BEV spatial feature-guided optimization strategy that attacks the detector’s internal representations. Extensive experiments demonstrate that our learned universal adversarial objects can consistently degrade multiple BEV detectors from various viewpoints and distances. More importantly, the new environment-manipulation attack paradigm exposes models’ over-reliance on contextual cues and provides a practical pipeline for robustness evaluation in AD systems.

Method: 1) Create FaceCoT dataset with 14 spoofing attack types and high-quality CoT VQA annotations. 2) Develop reinforcement learning-refined caption model for dataset expansion. 3) Introduce CoT-Enhanced Progressive Learning (CEPL) strategy to leverage CoT data.

Result: Models trained with FaceCoT and CEPL outperform state-of-the-art methods on multiple benchmark datasets.

Conclusion: FaceCoT enables multimodal reasoning for FAS, improving both robustness and interpretability through visual-linguistic co-inference.

Abstract: Face Anti-Spoofing (FAS) typically depends on a single visual modality when defending against presentation attacks such as print attacks, screen replays, and 3D masks, resulting in limited generalization across devices, environments, and attack types. Meanwhile, Multimodal Large Language Models (MLLMs) have recently achieved breakthroughs in image-text understanding and semantic reasoning, suggesting that integrating visual and linguistic co-inference into FAS can substantially improve both robustness and interpretability. However, the lack of a high-quality vision-language multimodal dataset has been a critical bottleneck. To address this, we introduce FaceCoT (Face Chain-of-Thought), the first large-scale Visual Question Answering (VQA) dataset tailored for FAS. FaceCoT covers 14 spoofing attack types and enriches model learning with high-quality CoT VQA annotations. Meanwhile, we develop a caption model refined via reinforcement learning to expand the dataset and enhance annotation quality. Furthermore, we introduce a CoT-Enhanced Progressive Learning (CEPL) strategy to better leverage the CoT data and boost model performance on FAS tasks. Extensive experiments demonstrate that models trained with FaceCoT and CEPL outperform state-of-the-art methods on multiple benchmark datasets.

Method: Provide VLMs with query image + matched healthy reference image + cross-patient comparative prompts. Evaluate multiple reference selection strategies (random, demographic matching, embedding retrieval, cross-center). Use lightweight supervised fine-tuning on small datasets.

Result: Comparative diagnosis significantly improves diagnostic performance. All reference selection strategies show strong performance. Theoretical analysis reveals improved sample efficiency and tighter visual-textual alignment.

Conclusion: Comparison-based diagnosis is clinically relevant and practically effective. Provides strategies for incorporating reference images into VLMs, demonstrating improved performance across medical imaging tasks.

Abstract: Medical image diagnosis is challenging because many diseases resemble normal anatomy and exhibit substantial interpatient variability. Clinicians routinely rely on comparative diagnosis, such as referencing cross-patient healthy control images to identify subtle but clinically meaningful abnormalities. Although healthy reference images are abundant in practice, existing medical vision-language models (VLMs) primarily operate in a single-image or single-series setting and lack explicit mechanisms for comparative diagnosis. This work investigates whether incorporating clinically motivated comparison can enhance VLM performance. We show that providing VLMs with both a query image and a matched healthy reference image, accompanied by cross-patient comparative prompts, significantly improves diagnostic performance. This performance can be further augmented by lightweight supervised fine-tuning (SFT) on a small amount of data. At the same time, we evaluate multiple strategies for selecting reference images, including random sampling, demographic attribute matching, embedding-based retrieval, and cross-center selection, and find consistently strong performance across all settings. Finally, we investigate why comparative diagnosis is effective theoretically, and observe improved sample efficiency and tighter alignment between visual and textual representations. Our findings highlight the clinical relevance of comparison-based diagnosis, provide practical strategies for incorporating reference images into VLMs, and demonstrate improved performance across diverse medical imaging tasks.

Method: Proposes MI-HSR framework using triple-camera smartphone system with two lenses equipped with spectral filters, introduces Doomer dataset with aligned images from three smartphone cameras and hyperspectral reference, and develops lightweight alignment module to fuse multi-view inputs while mitigating parallax/occlusion artifacts.

Result: The setup achieves 30% more accurate spectral estimation compared to ordinary RGB cameras, and the alignment module boosts reconstruction quality of state-of-the-art methods by an additional 5%.

Conclusion: Spectral filtering of multiple views with commodity hardware enables more accurate and practical hyperspectral imaging, suggesting a promising direction for improved color reproduction and material measurement.

Abstract: Hyperspectral reconstruction (HSR) from RGB images is a highly promising direction for accurate color reproduction and material color measurement. While most existing approaches rely on a single RGB image - thereby limiting reconstruction accuracy - the majority of modern smartphones are equipped with two or more cameras. In this work, we propose a novel multi-image-to-hyperspectral reconstruction (MI-HSR) framework that leverages a triple-camera smartphone system, where two lenses are equipped with carefully selected spectral filters. Our easy-to-implement configuration, based on theoretical and empirical analysis, allows to obtain more complete and diverse spectral data than traditional single-chamber setups. To support this new paradigm, we introduce Doomer, the first dataset for MI-HSR, comprising aligned images from three smartphone cameras and a hyperspectral reference camera across diverse scenes. We further introduce a lightweight alignment module for MI-HSR that effectively fuses multi-view inputs while mitigating parallax- and occlusion-induced artifacts. Proposed module demonstrate consistent quality improvements for modern HSR methods. In a nutshell, our setup allows 30% more accurate estimations of spectra compared to an ordinary RGB camera, while the proposed alignment module boosts the reconstruction quality of SotA methods by an additional 5%. Our findings suggest that spectral filtering of multiple views with commodity hardware unlocks more accurate and practical hyperspectral imaging.

Method: Query-based Adaptive Aggregation (QAA) uses learned queries as reference codebooks to enhance information capacity without significant computational overhead. Cross-query Similarity (CS) between query-level features and codebooks generates robust descriptors.

Result: QAA outperforms state-of-the-art models, achieving balanced generalization across diverse datasets while maintaining peak performance comparable to dataset-specific models. Learned queries show diverse attention patterns across datasets.

Conclusion: QAA effectively addresses generalization challenges in VPR through adaptive aggregation with learned queries, enabling robust performance across diverse datasets without sacrificing peak accuracy.

Abstract: Deep learning methods for Visual Place Recognition (VPR) have advanced significantly, largely driven by large-scale datasets. However, most existing approaches are trained on a single dataset, which can introduce dataset-specific inductive biases and limit model generalization. While multi-dataset joint training offers a promising solution for developing universal VPR models, divergences among training datasets can saturate the limited information capacity in feature aggregation layers, leading to suboptimal performance. To address these challenges, we propose Query-based Adaptive Aggregation (QAA), a novel feature aggregation technique that leverages learned queries as reference codebooks to effectively enhance information capacity without significant computational or parameter complexity. We show that computing the Cross-query Similarity (CS) between query-level image features and reference codebooks provides a simple yet effective way to generate robust descriptors. Our results demonstrate that QAA outperforms state-of-the-art models, achieving balanced generalization across diverse datasets while maintaining peak performance comparable to dataset-specific models. Ablation studies further explore QAA’s mechanisms and scalability. Visualizations reveal that the learned queries exhibit diverse attention patterns across datasets. Project page: http://xjh19971.github.io/QAA.

Method: Proposes Semantic Co-occurrence Insight Network (SCINet) with three main components: 1) Bi-dominant prompter module using off-the-shelf multimodal models to capture text-image correlations and enhance semantic alignment; 2) Cross-modality fusion module jointly modeling inter-label correlations, inter-instance relationships, and co-occurrence patterns; 3) Intrinsic semantic augmentation strategy applying diverse image transformations to enhance understanding of intrinsic data semantics.

Result: Extensive experiments on four widely-used benchmark datasets demonstrate that SCINet surpasses state-of-the-art methods in partial multi-label learning.

Conclusion: SCINet effectively addresses the challenge of partial multi-label learning by leveraging multimodal understanding of co-occurrence patterns between labels and instances, with the proposed framework showing superior performance over existing methods.

Abstract: Partial multi-label learning aims to extract knowledge from incompletely annotated data, which includes known correct labels, known incorrect labels, and unknown labels. The core challenge lies in accurately identifying the ambiguous relationships between labels and instances. In this paper, we emphasize that matching co-occurrence patterns between labels and instances is key to addressing this challenge. To this end, we propose Semantic Co-occurrence Insight Network (SCINet), a novel and effective framework for partial multi-label learning. Specifically, SCINet introduces a bi-dominant prompter module, which leverages an off-the-shelf multimodal model to capture text-image correlations and enhance semantic alignment. To reinforce instance-label interdependencies, we develop a cross-modality fusion module that jointly models inter-label correlations, inter-instance relationships, and co-occurrence patterns across instance-label assignments. Moreover, we propose an intrinsic semantic augmentation strategy that enhances the model’s understanding of intrinsic data semantics by applying diverse image transformations, thereby fostering a synergistic relationship between label confidence and sample difficulty. Extensive experiments on four widely-used benchmark datasets demonstrate that SCINet surpasses state-of-the-art methods.

Method: Uses pixel-aligned Gaussian primitives to represent dynamic 3D scenes with explicit supervision of time-varying motions, enabling reconstruction, view synthesis, and 3D point tracking in one framework.

Result: Achieves competitive performance across multiple tasks with several orders of magnitude speedup (reconstruction in one second), enabling zero-shot applications like scene flow estimation and moving object segmentation.

Conclusion: MoVieS demonstrates effective and efficient 4D dynamic scene reconstruction from monocular videos, bridging view synthesis with geometry reconstruction for diverse applications.

Abstract: We present MoVieS, a Motion-aware View Synthesis model that reconstructs 4D dynamic scenes from monocular videos in one second. It represents dynamic 3D scenes with pixel-aligned Gaussian primitives and explicitly supervises their time-varying motions. This allows, for the first time, the unified modeling of appearance, geometry and motion from monocular videos, and enables reconstruction, view synthesis and 3D point tracking within a single learning-based framework. By bridging view synthesis with geometry reconstruction, MoVieS enables large-scale training on diverse datasets with minimal dependence on task-specific supervision. As a result, it also naturally supports a wide range of zero-shot applications, such as scene flow estimation and moving object segmentation. Extensive experiments validate the effectiveness and efficiency of MoVieS across multiple tasks, achieving competitive performance while offering several orders of magnitude speedups.

Method: Winsor-CAM aggregates Grad-CAM maps from all convolutional layers in a single pass, then applies percentile-based Winsorization to attenuate outlier contributions. A user-controllable percentile parameter p enables semantic-level tuning from low-level textures to high-level object patterns.

Result: On DenseNet121 with Pascal VOC 2012, Winsor-CAM achieves 46.8% IoU and 0.059 CoM distance vs 39.0% and 0.074 for Grad-CAM, with improved insertion AUC (0.656 vs 0.623) and deletion AUC (0.197 vs 0.242). Even worst-performing fixed p-value configurations outperform FullGrad across all metrics. Similar improvements shown on PolypGen medical imaging dataset.

Conclusion: Winsor-CAM provides an efficient, robust, and human-tunable explanation tool that outperforms existing methods by incorporating multi-scale information from all convolutional layers, making it suitable for expert-in-the-loop analysis in safety-critical domains.

Abstract: Interpreting Convolutional Neural Networks (CNNs) is critical for safety-sensitive applications such as healthcare and autonomous systems. Popular visual explanation methods like Grad-CAM use a single convolutional layer, potentially missing multi-scale cues and producing unstable saliency maps. We introduce Winsor-CAM, a single-pass gradient-based method that aggregates Grad-CAM maps from all convolutional layers and applies percentile-based Winsorization to attenuate outlier contributions. A user-controllable percentile parameter p enables semantic-level tuning from low-level textures to high-level object patterns. We evaluate Winsor-CAM on six CNN architectures using PASCAL VOC 2012 and PolypGen, comparing localization (IoU, center-of-mass distance) and fidelity (insertion/deletion AUC) against seven baselines including Grad-CAM, Grad-CAM++, LayerCAM, ScoreCAM, AblationCAM, ShapleyCAM, and FullGrad. On DenseNet121 with a subset of Pascal VOC 2012, Winsor-CAM achieves 46.8% IoU and 0.059 CoM distance versus 39.0% and 0.074 for Grad-CAM, with improved insertion AUC (0.656 vs. 0.623) and deletion AUC (0.197 vs. 0.242). Notably, even the worst-performing fixed p-value configuration outperforms FullGrad across all metrics. An ablation study confirms that incorporating earlier layers improves localization. Similar evaluation on PolypGen polyp segmentation further validates Winsor-CAM’s effectiveness in medical imaging contexts. Winsor-CAM provides an efficient, robust, and human-tunable explanation tool for expert-in-the-loop analysis.

Method: Created a spatial evaluation pipeline with synthetic benchmark dataset. Categorized spatial understanding into: 1) absolute spatial understanding (querying object positions like left/right), and 2) 3D spatial understanding (movement and rotation). Used synthetic data generation to prevent dataset contamination and enable low-cost test sample creation.

Result: Humans achieve near-perfect performance on all tasks, while current VLMs only reach human-level performance on the two simplest tasks. For remaining tasks, VLM performance is significantly lower than humans, with best-performing VLMs achieving near-zero scores on multiple spatial understanding tasks.

Conclusion: There is substantial room for improvement in VLMs’ spatial understanding capabilities. The synthetic benchmark enables systematic evaluation and development of better spatial perception in multimodal models for real-world applications.

Abstract: Real-world applications, such as autonomous driving and humanoid robot manipulation, require precise spatial perception. However, it remains underexplored how Vision-Language Models (VLMs) recognize spatial relationships and perceive spatial movement. In this work, we introduce a spatial evaluation pipeline and construct a corresponding benchmark. Specifically, we categorize spatial understanding into two main types: absolute spatial understanding, which involves querying the absolute spatial position (e.g., left, right) of an object within an image, and 3D spatial understanding, which includes movement and rotation. Notably, our dataset is entirely synthetic, enabling the generation of test samples at a low cost while also preventing dataset contamination. We conduct experiments on multiple state-of-the-art VLMs and observe that there is significant room for improvement in their spatial understanding abilities. Explicitly, in our experiments, humans achieve near-perfect performance on all tasks, whereas current VLMs attain human-level performance only on the two simplest tasks. For the remaining tasks, the performance of VLMs is distinctly lower than that of humans. In fact, the best-performing Vision-Language Models even achieve near-zero scores on multiple tasks. The dataset and code are available on https://github.com/kong13661/LRR-Bench.

Method: Proposes a training-free, mask-free framework that computes Trajectory Divergence Map (TDM) by comparing token-wise velocity differences between inversion and denoising paths. Uses TDM to localize editable regions and guide Scheduled KV Injection for stable editing.

Result: Method achieves superior editability and visual fidelity, particularly in large-scale shape replacement tasks. Introduces ReShapeBench benchmark with 120 images and prompt pairs for shape-aware editing evaluation.

Conclusion: Follow-Your-Shape enables precise and controllable object shape editing while strictly preserving non-target content, addressing limitations of current flow-based editing models.

Abstract: While recent flow-based image editing models demonstrate general-purpose capabilities across diverse tasks, they often struggle to specialize in challenging scenarios – particularly those involving large-scale shape transformations. When performing such structural edits, these methods either fail to achieve the intended shape change or inadvertently alter non-target regions, resulting in degraded background quality. We propose Follow-Your-Shape, a training-free and mask-free framework that supports precise and controllable editing of object shapes while strictly preserving non-target content. Motivated by the divergence between inversion and editing trajectories, we compute a Trajectory Divergence Map (TDM) by comparing token-wise velocity differences between the inversion and denoising paths. The TDM enables precise localization of editable regions and guides a Scheduled KV Injection mechanism that ensures stable and faithful editing. To facilitate a rigorous evaluation, we introduce ReShapeBench, a new benchmark comprising 120 new images and enriched prompt pairs specifically curated for shape-aware editing. Experiments demonstrate that our method achieves superior editability and visual fidelity, particularly in tasks requiring large-scale shape replacement.

Method: Uses convolutional neural networks (CNN) trained on Sign Language MNIST dataset to classify hand gestures captured live via webcam, with text-to-speech synthesis for audible output

Result: High model accuracy and robust real-time performance with some latency, demonstrating practical applicability as an accessible communication tool

Conclusion: The system provides a reliable, user-friendly solution for enhancing autonomy and social integration of sign language users through real-time gesture-to-speech translation

Abstract: Communication barriers pose significant challenges for individuals with hearing and speech impairments, often limiting their ability to effectively interact in everyday environments. This project introduces a real-time assistive technology solution that leverages advanced deep learning techniques to translate sign language gestures into textual and audible speech. By employing convolution neural networks (CNN) trained on the Sign Language MNIST dataset, the system accurately classifies hand gestures captured live via webcam. Detected gestures are instantaneously translated into their corresponding meanings and transcribed into spoken language using text-to-speech synthesis, thus facilitating seamless communication. Comprehensive experiments demonstrate high model accuracy and robust real-time performance with some latency, highlighting the system’s practical applicability as an accessible, reliable, and user-friendly tool for enhancing the autonomy and integration of sign language users in diverse social settings.

Method: 1) Collaborative multi-modal coding integrates modality-specific features while preserving unique representational strengths; 2) Auxiliary 2D and 3D supervision improves robustness; 3) Triplane latent diffusion model generates high-quality 3D assets from embedded multi-modal codes.

Result: TriMM achieves competitive performance with models trained on large-scale datasets despite using small training data, demonstrating effective multi-modal learning. Additional experiments on RGB-D datasets verify feasibility of incorporating other multi-modal datasets into 3D generation.

Conclusion: TriMM successfully leverages multi-modal data (RGB, RGBD, point clouds) for 3D generation through collaborative coding and triplane diffusion, showing that multi-modality integration enables high-quality 3D asset creation with limited training data.

Abstract: 3D content inherently encompasses multi-modal characteristics and can be projected into different modalities (e.g., RGB images, RGBD, and point clouds). Each modality exhibits distinct advantages in 3D asset modeling: RGB images contain vivid 3D textures, whereas point clouds define fine-grained 3D geometries. However, most existing 3D-native generative architectures either operate predominantly within single-modality paradigms-thus overlooking the complementary benefits of multi-modality data-or restrict themselves to 3D structures, thereby limiting the scope of available training datasets. To holistically harness multi-modalities for 3D modeling, we present TriMM, the first feed-forward 3D-native generative model that learns from basic multi-modalities (e.g., RGB, RGBD, and point cloud). Specifically, 1) TriMM first introduces collaborative multi-modal coding, which integrates modality-specific features while preserving their unique representational strengths. 2) Furthermore, auxiliary 2D and 3D supervision are introduced to raise the robustness and performance of multi-modal coding. 3) Based on the embedded multi-modal code, TriMM employs a triplane latent diffusion model to generate 3D assets of superior quality, enhancing both the texture and the geometric detail. Extensive experiments on multiple well-known datasets demonstrate that TriMM, by effectively leveraging multi-modality, achieves competitive performance with models trained on large-scale datasets, despite utilizing a small amount of training data. Furthermore, we conduct additional experiments on recent RGB-D datasets, verifying the feasibility of incorporating other multi-modal datasets into 3D generation.

Method: Uses monocular 3DGS with object-anchored metric dense depth from sparse VI-SfM cues. Introduces multi-scale shape consensus module to merge segments into coarse objects supported by SfM with parametric shape models, and cross-object depth refinement module optimizing per-pixel depth with geometric consistency, prior anchoring, and edge-aware smoothness.

Result: Reduces training time by up to 30.4% and memory consumption by 19.8% compared to LiDAR-based approaches, while achieving competitive rendering quality in large scenes using low-cost VI sensors.

Conclusion: MOGS demonstrates that monocular 3DGS with object-anchored depth from sparse SfM can achieve efficient, high-quality large-scale scene reconstruction without expensive LiDAR, enabling broader deployment and faster optimization.

Abstract: Recent advances in 3D Gaussian Splatting (3DGS) deliver striking photorealism, and extending it to large scenes opens new opportunities for semantic reasoning and prediction in applications such as autonomous driving. Today’s state-of-the-art systems for large scenes primarily originate from LiDAR-based pipelines that utilize long-range depth sensing. However, they require costly high-channel sensors whose dense point clouds strain memory and computation, limiting scalability, fleet deployment, and optimization speed. We present MOGS, a monocular 3DGS framework that replaces active LiDAR depth with object-anchored, metrized dense depth derived from sparse visual-inertial (VI) structure-from-motion (SfM) cues. Our key idea is to exploit image semantics to hypothesize per-object shape priors, anchor them with sparse but metrically reliable SfM points, and propagate the resulting metric constraints across each object to produce dense depth. To address two key challenges, i.e., insufficient SfM coverage within objects and cross-object geometric inconsistency, MOGS introduces (1) a multi-scale shape consensus module that adaptively merges small segments into coarse objects best supported by SfM and fits them with parametric shape models, and (2) a cross-object depth refinement module that optimizes per-pixel depth under a combinatorial objective combining geometric consistency, prior anchoring, and edge-aware smoothness. Experiments on public datasets show that, with a low-cost VI sensor suite, MOGS reduces training time by up to 30.4% and memory consumption by 19.8%, while achieving high-quality rendering competitive with costly LiDAR-based approaches in large scenes.

Method: 1) Extract “master key filters” via clustering of receptive fields from ConvNeXt networks. 2) Compute spatial spread measures (weighted means/variances) to analyze filter properties. 3) Model filters using difference operators applied to discrete Gaussian kernels. 4) Fit models using either spatial variance matching or minimizing l1/l2 norms between idealized and learned filters.

Result: Learned filters can be well approximated by discrete scale-space filters; idealized models show good qualitative similarity to learned filters and have good predictive properties when replacing learned filters in networks.

Conclusion: Depthwise-separable deep networks learn filters that are essentially discrete scale-space filters, suggesting principled mathematical foundations underlie learned representations in modern convolutional architectures.

Abstract: This paper presents the results of analysing and modelling a set of 8 master key filters'', which have been extracted by applying a clustering approach to the receptive fields learned in depthwise-separable deep networks based on the ConvNeXt architecture. For this purpose, we first compute spatial spread measures in terms of weighted mean values and weighted variances of the absolute values of the learned filters, which support the working hypotheses that: (i) the learned filters can be modelled by separable filtering operations over the spatial domain, and that (ii) the spatial offsets of the those learned filters that are non-centered are rather close to half a grid unit. Then, we model the clustered master key filters’’ in terms of difference operators applied to a spatial smoothing operation in terms of the discrete analogue of the Gaussian kernel, and demonstrate that the resulting idealized models of the receptive fields show good qualitative similarity to the learned filters. This modelling is performed in two different ways: (i) using possibly different values of the scale parameters in the coordinate directions for each filter, and (ii) using the same value of the scale parameter in both coordinate directions. Then, we perform the actual model fitting by either (i) requiring spatial spread measures in terms of spatial variances of the absolute values of the receptive fields to be equal, or (ii) minimizing the discrete $l_1$- or $l_2$-norms between the idealized receptive field models and the learned filters. Complementary experimental results then demonstrate the idealized models of receptive fields have good predictive properties for replacing the learned filters by idealized filters in depthwise-separable deep networks, thus showing that the learned filters in depthwise-separable deep networks can be well approximated by discrete scale-space filters.

Method: The authors present the first range-view framework that adapts SAM2 to 3D segmentation, coupling efficient 2D feature extraction with standard projection/back-projection to operate on point clouds. They implement several architectural modifications to the SAM2 encoder: (1) a novel module emphasizing horizontal spatial dependencies in LiDAR range images, (2) a customized configuration tailored to geometric properties of spherical projections, and (3) an adapted mechanism designed to capture unique spatial patterns and discontinuities in range-view pseudo-images.

Result: The approach achieves competitive performance on SemanticKITTI while benefiting from the speed, scalability, and deployment simplicity of 2D-centric pipelines.

Conclusion: This work highlights the viability of Visual Foundation Models as general-purpose backbones for 3D perception and opens a path toward unified, foundation-model-driven LiDAR segmentation. Range-view segmentation methods using VFMs lead to promising results.

Abstract: Point cloud segmentation is central to autonomous driving and 3D scene understanding. While voxel- and point-based methods dominate recent research due to their compatibility with deep architectures and ability to capture fine-grained geometry, they often incur high computational cost, irregular memory access, and limited real-time efficiency. In contrast, range-view methods, though relatively underexplored - can leverage mature 2D semantic segmentation techniques for fast and accurate predictions. Motivated by the rapid progress in Visual Foundation Models (VFMs) for captioning, zero-shot recognition, and multimodal tasks, we investigate whether SAM2, the current state-of-the-art VFM for segmentation tasks, can serve as a strong backbone for LiDAR point cloud segmentation in the range view. We present , to our knowledge, the first range-view framework that adapts SAM2 to 3D segmentation, coupling efficient 2D feature extraction with standard projection/back-projection to operate on point clouds. To optimize SAM2 for range-view representations, we implement several architectural modifications to the encoder: (1) a novel module that emphasizes horizontal spatial dependencies inherent in LiDAR range images, (2) a customized configuration of tailored to the geometric properties of spherical projections, and (3) an adapted mechanism in the encoder backbone specifically designed to capture the unique spatial patterns and discontinuities present in range-view pseudo-images. Our approach achieves competitive performance on SemanticKITTI while benefiting from the speed, scalability, and deployment simplicity of 2D-centric pipelines. This work highlights the viability of VFMs as general-purpose backbones for 3D perception and opens a path toward unified, foundation-model-driven LiDAR segmentation. Results lets us conclude that range-view segmentation methods using VFMs leads to promising results.

Method: Evaluates suite of representational similarity measures on two domains: artificial models (comparing procedurally dissimilar vs matched models) and neural data (comparing cortical regions across subjects). Adapts Similarity Network Fusion (originally for multi-omics) to combine similarity graphs across different metrics.

Result: Metrics preserving representational geometry or tuning structure more reliably separate known structure than flexible mappings like linear predictivity. Fused similarity yields sharper separation of model families and recovers clearer hierarchical organization of ventral visual stream that better aligns with established anatomical/functional hierarchies.

Conclusion: Multi-metric approach reveals which dimensions of representational correspondence recover meaningful structure, and shows how complementary similarity notions can be integrated for better comparison of biological and artificial neural systems.

Abstract: The extent to which different biological and artificial neural systems rely on equivalent internal representations to support similar tasks remains a central question in neuroscience and machine learning. Prior work typically compares systems using a single representational similarity metric, even though different metrics emphasize distinct facets of representational correspondence. Here we evaluate a suite of representational similarity measures by asking how well each metric recovers known structure across two domains: for artificial models, whether procedurally dissimilar models (differing in architecture or training paradigm) are assigned lower similarity than procedurally matched models; and for neural data, whether responses from distinct cortical regions are separated while responses from the same region align across subjects. Across both vision models and neural recordings, metrics that preserve representational geometry or tuning structure more reliably separate this structure than more flexible mappings such as linear predictivity. To integrate these complementary facets, we adapt Similarity Network Fusion, originally developed for multi-omics integration, to combine similarity graphs across metrics. The resulting fused similarity yields sharper separation of procedurally defined model families and, when applied to neural data, recovers a clearer hierarchical organization of the ventral visual stream that aligns more closely with established anatomical and functional hierarchies than single metrics. Overall, this approach reveals which dimensions of representational correspondence recover meaningful structure in models and brains, and how complementary notions of similarity can be integrated.

Method: Consistency Mid-Training (CMT) inserts a compact intermediate stage between diffusion pre-training and final flow map training. It trains a model to map points along a solver trajectory from a pre-trained model (starting from a prior sample) directly to the solver-generated clean sample, creating a trajectory-consistent and stable initialization.

Result: CMT achieves state-of-the-art two-step FIDs: 1.97 on CIFAR-10, 1.32 on ImageNet 64x64, and 1.84 on ImageNet 512x512, using up to 98% less training data and GPU time compared to Consistency Models. On ImageNet 256x256, CMT reaches 1-step FID 3.34 while cutting total training time by about 50% compared to Mean Flow from scratch.

Conclusion: CMT establishes a principled, efficient, and general framework for training flow map models, providing stable initialization that simplifies flow map learning and enables fast, robust convergence without heuristics.

Abstract: Flow map models such as Consistency Models (CM) and Mean Flow (MF) enable few-step generation by learning the long jump of the ODE solution of diffusion models, yet training remains unstable, sensitive to hyperparameters, and costly. Initializing from a pre-trained diffusion model helps, but still requires converting infinitesimal steps into a long-jump map, leaving instability unresolved. We introduce mid-training, the first concept and practical method that inserts a lightweight intermediate stage between the (diffusion) pre-training and the final flow map training (i.e., post-training) for vision generation. Concretely, Consistency Mid-Training (CMT) is a compact and principled stage that trains a model to map points along a solver trajectory from a pre-trained model, starting from a prior sample, directly to the solver-generated clean sample. It yields a trajectory-consistent and stable initialization. This initializer outperforms random and diffusion-based baselines and enables fast, robust convergence without heuristics. Initializing post-training with CMT weights further simplifies flow map learning. Empirically, CMT achieves state of the art two step FIDs: 1.97 on CIFAR-10, 1.32 on ImageNet 64x64, and 1.84 on ImageNet 512x512, while using up to 98% less training data and GPU time, compared to CMs. On ImageNet 256x256, CMT reaches 1-step FID 3.34 while cutting total training time by about 50% compared to MF from scratch (FID 3.43). This establishes CMT as a principled, efficient, and general framework for training flow map models.

Method: ZOO-Prune estimates token sensitivity using zeroth-order perturbations at the lightweight projection layer. Small random perturbations are applied to measure how they affect projected features, enabling efficient approximation of each token’s influence without backpropagation.

Result: Extensive experiments show ZOO-Prune consistently outperforms prior methods, pruning up to 94.4% of tokens without sacrificing accuracy. It achieves up to 2.30x faster end-to-end inference compared to baseline.

Conclusion: ZOO-Prune provides an effective training-free framework for token pruning in VLMs by identifying highly sensitive tokens that capture complementary visual cues, significantly improving inference efficiency while maintaining accuracy.

Abstract: Large Vision-Language Models (VLMs) enable strong multimodal reasoning but incur heavy inference costs from redundant visual tokens. Token pruning alleviates this issue, yet existing approaches face limitations. Attention-based methods rely on raw attention scores, which are often unstable across layers and heads and can lead to redundant selections. Diversity-based methods improve robustness by selecting tokens far apart in feature space, but risk dropping regions needed for accurate prediction. We propose ZOO-Prune, a training-free framework built on the intuition that highly sensitive tokens have a stronger influence on the model’s output and capture complementary visual cues rather than redundant ones. To achieve this, we estimate token sensitivity using zeroth-order perturbations at the lightweight projection layer. This measures how small random perturbations affect the projected features and enables efficient approximation of each token’s influence without backpropagation. Extensive experiments across multiple VLMs and benchmarks show that ZOO-Prune consistently outperforms prior methods while pruning up to 94.4% of tokens without sacrificing accuracy. Our method also improves efficiency, reaching up to 2.30x faster end-to-end inference compared to the baseline.

Method: Three-stage alignment: (1) Freeze encoder, train adapter and decoder to establish semantic latent space; (2) Joint optimization with semantic preservation loss to capture perceptual details while retaining high-level semantics; (3) Decoder refinement for improved reconstruction quality.

Result: On ImageNet 256×256, the tokenizer accelerates diffusion model convergence (gFID 1.90 within 64 epochs) and improves generation with/without classifier-free guidance. On LAION, text-to-image models outperform FLUX VAE and VA-VAE under same training steps.

Conclusion: AlignTok provides a simple, scalable approach for creating semantically grounded continuous tokenizers that enhance diffusion model performance, establishing a new paradigm for tokenizer design.

Abstract: In this work, we propose aligning pretrained visual encoders to serve as tokenizers for latent diffusion models in image generation. Unlike training a variational autoencoder (VAE) from scratch, which primarily emphasizes low-level details, our approach leverages the rich semantic structure of foundation encoders. We introduce a three-stage alignment strategy called AlignTok: (1) freeze the encoder and train an adapter and a decoder to establish a semantic latent space; (2) jointly optimize all components with an additional semantic preservation loss, enabling the encoder to capture perceptual details while retaining high-level semantics; and (3) refine the decoder for improved reconstruction quality. This alignment yields semantically rich image tokenizers that benefit diffusion models. On ImageNet 256$\times$256, our tokenizer accelerates the convergence of diffusion models, reaching a gFID of 1.90 within just 64 epochs, and improves generation both with and without classifier-free guidance. Scaling to LAION, text-to-image models trained with our tokenizer consistently outperforms FLUX VAE and VA-VAE under the same training steps. Overall, our method is simple, scalable, and establishes a semantically grounded paradigm for continuous tokenizer design.

Method: SAGE introduces: 1) Soft Probing module that learns residual weights for patch descriptors before bilinear aggregation; 2) Online geo-visual graph reconstruction that fuses geographic proximity and visual similarity; 3) Greedy weighted clique expansion sampler for hard sample mining from high-affinity anchors.

Result: Achieves state-of-the-art across eight benchmarks, with 100% Recall@10 on SPED using only 4096D global descriptors. Uses frozen DINOv2 backbone with parameter-efficient fine-tuning.

Conclusion: SAGE provides a unified training pipeline that effectively addresses the dynamic interplay between spatial context and visual similarity, significantly improving VPR performance through adaptive graph exploration and enhanced local feature aggregation.

Abstract: Visual Place Recognition (VPR) requires robust retrieval of geotagged images despite large appearance, viewpoint, and environmental variation. Prior methods focus on descriptor fine-tuning or fixed sampling strategies yet neglect the dynamic interplay between spatial context and visual similarity during training. We present SAGE (Spatial-visual Adaptive Graph Exploration), a unified training pipeline that enhances granular spatial-visual discrimination by jointly improving local feature aggregation, organize samples during training, and hard sample mining. We introduce a lightweight Soft Probing module that learns residual weights from training data for patch descriptors before bilinear aggregation, boosting distinctive local cues. During training we reconstruct an online geo-visual graph that fuses geographic proximity and current visual similarity so that candidate neighborhoods reflect the evolving embedding landscape. To concentrate learning on the most informative place neighborhoods, we seed clusters from high-affinity anchors and iteratively expand them with a greedy weighted clique expansion sampler. Implemented with a frozen DINOv2 backbone and parameter-efficient fine-tuning, SAGE achieves SOTA across eight benchmarks. Notably, our method obtains 100% Recall@10 on SPED only using 4096D global descriptors. The code and model are available at https://github.com/chenshunpeng/SAGE.

Method: Flower uses a three-step iterative procedure: (1) flow-consistent destination estimation using velocity network, (2) refinement via projection onto feasible set defined by forward operator, and (3) time-progression re-projection along flow trajectory.

Result: Flower achieves state-of-the-art reconstruction quality across various linear inverse problems while using nearly identical hyperparameters, demonstrating practical effectiveness.

Conclusion: Flower provides a theoretically grounded framework that unifies different perspectives on inverse problem solving and offers strong practical performance with minimal hyperparameter tuning.

Abstract: We introduce Flower, a solver for linear inverse problems. It leverages a pre-trained flow model to produce reconstructions that are consistent with the observed measurements. Flower operates through an iterative procedure over three steps: (i) a flow-consistent destination estimation, where the velocity network predicts a denoised target; (ii) a refinement step that projects the estimated destination onto a feasible set defined by the forward operator; and (iii) a time-progression step that re-projects the refined destination along the flow trajectory. We provide a theoretical analysis that demonstrates how Flower approximates Bayesian posterior sampling, thereby unifying perspectives from plug-and-play methods and generative inverse solvers. On the practical side, Flower achieves state-of-the-art reconstruction quality while using nearly identical hyperparameters across various linear inverse problems. Our code is available at https://github.com/mehrsapo/Flower.

Method: 1) Created ReasonMap-Plus dataset with dense VQA rewards for cold-start training; 2) Proposed RewardMap framework with difficulty-aware reward design (detail rewards) and multi-stage RL scheme that progresses from simple perception to complex reasoning tasks.

Result: RewardMap components individually improve performance, with combined approach yielding best results. Models achieve 3.47% average improvement across 6 benchmarks spanning spatial reasoning, fine-grained visual reasoning, and general tasks beyond transit maps.

Conclusion: RewardMap effectively addresses sparse reward challenges in MLLM training for fine-grained visual reasoning, demonstrating improved visual understanding and reasoning capabilities through multi-stage RL with difficulty-aware rewards.

Abstract: Fine-grained visual reasoning remains a core challenge for multimodal large language models (MLLMs). The recently introduced ReasonMap highlights this gap by showing that even advanced MLLMs struggle with spatial reasoning in structured and information-rich settings such as transit maps, a task of clear practical and scientific importance. However, standard reinforcement learning (RL) on such tasks is impeded by sparse rewards and unstable optimization. To address this, we first construct ReasonMap-Plus, an extended dataset that introduces dense reward signals through Visual Question Answering (VQA) tasks, enabling effective cold-start training of fine-grained visual understanding skills. Next, we propose RewardMap, a multi-stage RL framework designed to improve both visual understanding and reasoning capabilities of MLLMs. RewardMap incorporates two key designs. First, we introduce a difficulty-aware reward design that incorporates detail rewards, directly tackling the sparse rewards while providing richer supervision. Second, we propose a multi-stage RL scheme that bootstraps training from simple perception to complex reasoning tasks, offering a more effective cold-start strategy than conventional Supervised Fine-Tuning (SFT). Experiments on ReasonMap and ReasonMap-Plus demonstrate that each component of RewardMap contributes to consistent performance gains, while their combination yields the best results. Moreover, models trained with RewardMap achieve an average improvement of 3.47% across 6 benchmarks spanning spatial reasoning, fine-grained visual reasoning, and general tasks beyond transit maps, underscoring enhanced visual understanding and reasoning capabilities.

Method: Adapts Optimal Brain Surgeon (OBS) to diffusion model architectures, proposes timestep-aware Hessian construction with logarithmic-decrease weighting for earlier timesteps, and uses group-wise sequential pruning to amortize calibration costs.

Result: Achieves state-of-the-art one-shot pruning for diffusion models, delivering inference acceleration with minimal degradation in visual quality.

Conclusion: OBS-Diff enables accurate and training-free compression of large-scale text-to-image diffusion models through novel adaptations of classic pruning techniques to diffusion model architectures.

Abstract: Large-scale text-to-image diffusion models, while powerful, suffer from prohibitive computational cost. Existing one-shot network pruning methods can hardly be directly applied to them due to the iterative denoising nature of diffusion models. To bridge the gap, this paper presents OBS-Diff, a novel one-shot pruning framework that enables accurate and training-free compression of large-scale text-to-image diffusion models. Specifically, (i) OBS-Diff revitalizes the classic Optimal Brain Surgeon (OBS), adapting it to the complex architectures of modern diffusion models and supporting diverse pruning granularity, including unstructured, N:M semi-structured, and structured (MHA heads and FFN neurons) sparsity; (ii) To align the pruning criteria with the iterative dynamics of the diffusion process, by examining the problem from an error-accumulation perspective, we propose a novel timestep-aware Hessian construction that incorporates a logarithmic-decrease weighting scheme, assigning greater importance to earlier timesteps to mitigate potential error accumulation; (iii) Furthermore, a computationally efficient group-wise sequential pruning strategy is proposed to amortize the expensive calibration process. Extensive experiments show that OBS-Diff achieves state-of-the-art one-shot pruning for diffusion models, delivering inference acceleration with minimal degradation in visual quality.

Method: EDJE precomputes vision tokens offline and compresses them via a lightweight attention-based adapter. Online inference runs only a compact joint encoder over a small set of compressed visual tokens plus text, drastically reducing storage and compute requirements.

Result: EDJE processes 50k image-text pairs/second while requiring only 49kB of disk storage per image, matching prior art on Flickr (zero-shot) and COCO (fine-tuned) retrieval benchmarks.

Conclusion: EDJE enables practical deployment of vision-language rerankers at scale by overcoming the computational bottleneck of visual feature extraction while maintaining strong retrieval performance.

Abstract: Multimodal retrieval still leans on embedding-based models like CLIP for fast vector search over pre-computed image embeddings. Yet, unlike text retrieval, where joint-encoder rerankers are standard, comparable vision-language rerankers are largely absent. We find that seminal joint encoders such as BLIP are severely bottlenecked by an expensive visual feature-extraction stage, preventing practical deployment at scale. Motivated by this bottleneck, we introduce EDJE, an Efficient Discriminative Joint Encoder that precomputes vision tokens offline and compresses them via a lightweight attention-based adapter, so online inference runs only a compact joint encoder over a small set of visual tokens plus the text. EDJE preserves strong retrieval performance while drastically reducing storage and online compute, enabling high-throughput inference. Specifically, EDJE processes 50k image–text pairs/second while requiring 49kB of disk storage per image, matching prior art on Flickr (zero-shot) and COCO (fine-tuned) retrieval.

Method: Proposes LinVideo framework with: 1) Selective transfer - frames layer selection as binary classification to automatically convert layers to linear attention, 2) Anytime Distribution Matching (ADM) - aligns sample distributions across any timestep in sampling trajectory for efficient transfer.

Result: Achieves 1.25-2.00x speedup while preserving generation quality. The 4-step distilled model further delivers 15.92x latency reduction with minimal visual quality drop.

Conclusion: LinVideo provides an efficient post-training solution for accelerating video diffusion models without expensive retraining, making high-quality video generation more practical.

Abstract: Video diffusion models (DMs) have enabled high-quality video synthesis. However, their computation costs scale quadratically with sequence length because self-attention has quadratic complexity. While linear attention lowers the cost, fully replacing quadratic attention requires expensive pretraining due to the limited expressiveness of linear attention and the complexity of spatiotemporal modeling in video generation. In this paper, we present LinVideo, an efficient data-free post-training framework that replaces a target number of self-attention modules with linear attention while preserving the original model’s performance. First, we observe a significant disparity in the replaceability of different layers. Instead of manual or heuristic choices, we frame layer selection as a binary classification problem and propose selective transfer, which automatically and progressively converts layers to linear attention with minimal performance impact. Additionally, to overcome the ineffectiveness and inefficiency of existing objectives for this transfer process, we introduce an anytime distribution matching (ADM) objective that aligns the distributions of samples across any timestep along the sampling trajectory. This objective is efficient and recovers model performance. Extensive experiments show that our method achieves a 1.25-2.00x speedup while preserving generation quality, and our 4-step distilled model further delivers a 15.92x latency reduction with minimal visual quality drop.

Method: Three main modules: 1) Critical State-Based Memory Compression compresses frame sequences into critical states to reduce redundancy; 2) Action Pattern Learning constructs state-transition graphs with multi-dimensional edges to model action dynamics and generate potential future cues representing intention; 3) Cross-Temporal Interaction module models mutual influence between intentions and past/current information through cross-temporal interactions to refine features.

Result: Superior performance on multiple benchmark datasets including EPIC-Kitchens-100, THUMOS'14, TVSeries, and the Parkinson’s Disease Mouse Behaviour (PDMB) dataset compared to state-of-the-art approaches.

Conclusion: The SSM framework effectively addresses redundancy and intention modeling in action understanding, demonstrating the importance of action dynamics learning and cross-temporal interactions for unified action detection and anticipation.

Abstract: Action understanding, encompassing action detection and anticipation, plays a crucial role in numerous practical applications. However, untrimmed videos are often characterized by substantial redundant information and noise. Moreover, in modeling action understanding, the influence of the agent’s intention on the action is often overlooked. Motivated by these issues, we propose a novel framework called the State-Specific Model (SSM), designed to unify and enhance both action detection and anticipation tasks. In the proposed framework, the Critical State-Based Memory Compression module compresses frame sequences into critical states, reducing information redundancy. The Action Pattern Learning module constructs a state-transition graph with multi-dimensional edges to model action dynamics in complex scenarios, on the basis of which potential future cues can be generated to represent intention. Furthermore, our Cross-Temporal Interaction module models the mutual influence between intentions and past as well as current information through cross-temporal interactions, thereby refining present and future features and ultimately realizing simultaneous action detection and anticipation. Extensive experiments on multiple benchmark datasets – including EPIC-Kitchens-100, THUMOS'14, TVSeries, and the introduced Parkinson’s Disease Mouse Behaviour (PDMB) dataset – demonstrate the superior performance of our proposed framework compared to other state-of-the-art approaches. These results highlight the importance of action dynamics learning and cross-temporal interactions, laying a foundation for future action understanding research.

Method: Proposes guiding principles for native VLM construction: (i) align pixel and word representations in shared semantic space, (ii) integrate strengths of separate vision/language modules, (iii) embody cross-modal properties for unified encoding, aligning, and reasoning. Launches NEO family built from these principles with 390M image-text examples.

Result: NEO greatly narrows the gap with top-tier modular VLMs across diverse real-world scenarios, efficiently develops visual perception from scratch while mitigating vision-language conflicts within a dense monolithic model.

Conclusion: NEO serves as a cornerstone for scalable and powerful native VLM development, paired with reusable components that foster a cost-effective and extensible ecosystem for the research community.

Abstract: The edifice of native Vision-Language Models (VLMs) has emerged as a rising contender to typical modular VLMs, shaped by evolving model architectures and training paradigms. Yet, two lingering clouds cast shadows over its widespread exploration and promotion: (-) What fundamental constraints set native VLMs apart from modular ones, and to what extent can these barriers be overcome? (-) How to make research in native VLMs more accessible and democratized, thereby accelerating progress in the field. In this paper, we clarify these challenges and outline guiding principles for constructing native VLMs. Specifically, one native VLM primitive should: (i) effectively align pixel and word representations within a shared semantic space; (ii) seamlessly integrate the strengths of formerly separate vision and language modules; (iii) inherently embody various cross-modal properties that support unified vision-language encoding, aligning, and reasoning. Hence, we launch NEO, a novel family of native VLMs built from first principles, greatly narrowing the gap with top-tier modular counterparts across diverse real-world scenarios. With 390M image-text examples, NEO efficiently develops visual perception from scratch while mitigating vision-language conflicts inside a dense and monolithic model crafted from our elaborate primitives. We position NEO as a cornerstone for scalable and powerful native VLM development, paired with a rich set of reusable components that foster a cost-effective and extensible ecosystem. Our code and models are publicly available at: https://github.com/EvolvingLMMs-Lab/NEO.

Method: 1) Synthetic experiments with theoretical derivations to analyze generalization difficulty; 2) New evaluation framework comparing real vs synthetic data utility; 3) Comprehensive analysis across CC12M, ImageNet-1k, and DCI datasets; 4) Evaluation of inference-time intervention methods including prompt expansion.

Result: Increasing prompt complexity reduces conditional diversity and prompt consistency while decreasing synthetic-to-real distribution shift. Prompt expansion (using pre-trained language model as likelihood estimator) achieves highest performance in image diversity and aesthetics, even surpassing real data.

Conclusion: Prompt complexity significantly impacts synthetic data utility, with trade-offs between diversity/consistency and distribution alignment. Current inference-time interventions can enhance diversity but may move outside real data support, with prompt expansion emerging as the most effective method.

Abstract: Text-to-image (T2I) models offer great potential for creating virtually limitless synthetic data, a valuable resource compared to fixed and finite real datasets. Previous works evaluate the utility of synthetic data from T2I models on three key desiderata: quality, diversity, and consistency. While prompt engineering is the primary means of interacting with T2I models, the systematic impact of prompt complexity on these critical utility axes remains underexplored. In this paper, we first conduct synthetic experiments to motivate the difficulty of generalization with regard to prompt complexity and explain the observed difficulty with theoretical derivations. Then, we introduce a new evaluation framework that can compare the utility of real data and synthetic data, and present a comprehensive analysis of how prompt complexity influences the utility of synthetic data generated by commonly used T2I models. We conduct our study across diverse datasets, including CC12M, ImageNet-1k, and DCI, and evaluate different inference-time intervention methods. Our synthetic experiments show that generalizing to more general conditions is harder than the other way round, since the former needs an estimated likelihood that is not learned by diffusion models. Our large-scale empirical experiments reveal that increasing prompt complexity results in lower conditional diversity and prompt consistency, while reducing the synthetic-to-real distribution shift, which aligns with the synthetic experiments. Moreover, current inference-time interventions can augment the diversity of the generations at the expense of moving outside the support of real data. Among those interventions, prompt expansion, by deliberately using a pre-trained language model as a likelihood estimator, consistently achieves the highest performance in both image diversity and aesthetics, even higher than that of real data.

Method: Proposes MergeMix with token merge-based mixup augmentation: 1) Generates contextually aligned mixed images using merged attention maps with cluster regions, 2) Builds preference pairs between raw and MergeMix-generated images, 3) Optimizes soft preference margin with mixed SimPO loss to bridge SFT and RL paradigms.

Result: Extensive experiments show MergeMix achieves dominant classification accuracy as an augmentation method, improves generalization abilities and alignment of MLLMs, and provides a new learning paradigm with training efficiency and stability.

Conclusion: MergeMix offers a balanced approach for MLLM alignment that bridges SFT and RL, achieving better efficiency, stability, and generalization while reducing computational overhead and annotation requirements.

Abstract: Vision-language alignment in multi-modal large language models (MLLMs) relies on supervised fine-tuning (SFT) or reinforcement learning (RL). To align multi-modal large language models (MLLMs) in the post-training stage, supervised fine-tuning (SFT) is a stable choice but requires human annotations and lacks task generalizations, while Reinforcement Learning (RL) searches for better answers from reward signals but suffers from computational overhead and instability. To achieve balance among scalability, efficiency, and alignment generalizations, we propose MergeMix, a unified paradigm that bridges SFT and RL with an efficient Token Merge based Mixup augmentation. As for the Mixup policy, we generate contextual aligned mixed images with the corresponding labels according to the merged attention maps with cluster regions. Then, we enhance the preference-driven paradigm for MLLMs by building preference pairs with raw images and MergeMix-generated ones and optimizing the soft preference margin with the mixed SimPO loss. Extensive experiments demonstrate that MergeMix not only achieves dominant classification accuracy as an augmentation method but also improves generalization abilities and alignment of MLLMs, providing a new learning paradigm for preference alignment with training efficiency and stability.

Method: Uses effective rank to quantify both collapses, proposes Rank-enhancing Token Fuser that selectively blends less informative features from one modality with complementary features from another. Evaluates modality combinations that mutually increase each other’s effective rank.

Result: Shows depth maintains representational balance when fused with RGB, avoiding modality collapse. R3D framework outperforms prior state-of-the-art methods by up to 3.74% on NTURGBD, UTKinect, and DARai datasets.

Conclusion: Effective rank serves as a unifying measure to address both feature and modality collapse in multimodal fusion, with practical applications in action anticipation using RGB-Depth fusion.

Abstract: Multi-modal fusion methods often suffer from two types of representation collapse: feature collapse where individual dimensions lose their discriminative power (as measured by eigenspectra), and modality collapse where one dominant modality overwhelms the other. Applications like human action anticipation that require fusing multifarious sensor data are hindered by both feature and modality collapse. However, existing methods attempt to counter feature collapse and modality collapse separately. This is because there is no unifying framework that efficiently addresses feature and modality collapse in conjunction. In this paper, we posit the utility of effective rank as an informative measure that can be utilized to quantify and counter both the representation collapses. We propose \textit{Rank-enhancing Token Fuser}, a theoretically grounded fusion framework that selectively blends less informative features from one modality with complementary features from another modality. We show that our method increases the effective rank of the fused representation. To address modality collapse, we evaluate modality combinations that mutually increase each others’ effective rank. We show that depth maintains representational balance when fused with RGB, avoiding modality collapse. We validate our method on action anticipation, where we present \texttt{R3D}, a depth-informed fusion framework. Extensive experiments on NTURGBD, UTKinect, and DARai demonstrate that our approach significantly outperforms prior state-of-the-art methods by up to 3.74%. Our code is available at: \href{https://github.com/olivesgatech/R3D}{https://github.com/olivesgatech/R3D}.

Method: Proposes StreamDiffusionV2 with: 1) SLO-aware batching scheduler and block scheduler, 2) sink-token-guided rolling KV cache, 3) motion-aware noise controller, 4) scalable pipeline orchestration parallelizing diffusion across denoising steps and network layers, and 5) support for heterogeneous GPU environments.

Result: Achieves first frame within 0.5s, 58.28 FPS with 14B-parameter model and 64.52 FPS with 1.3B-parameter model on four H100 GPUs without TensorRT or quantization. Enables flexible denoising steps (1-4) for ultra-low-latency or higher-quality modes.

Conclusion: StreamDiffusionV2 makes state-of-the-art generative live streaming practical and accessible by solving real-time constraints through system-level optimizations and scalable multi-GPU serving.

Abstract: Generative models are reshaping the live-streaming industry by redefining how content is created, styled, and delivered. Previous image-based streaming diffusion models have powered efficient and creative live streaming products but have hit limits on temporal consistency due to the foundation of image-based designs. Recent advances in video diffusion have markedly improved temporal consistency and sampling efficiency for offline generation. However, offline generation systems primarily optimize throughput by batching large workloads. In contrast, live online streaming operates under strict service-level objectives (SLOs): time-to-first-frame must be minimal, and every frame must meet a per-frame deadline with low jitter. Besides, scalable multi-GPU serving for real-time streams remains largely unresolved so far. To address this, we present StreamDiffusionV2, a training-free pipeline for interactive live streaming with video diffusion models. StreamDiffusionV2 integrates an SLO-aware batching scheduler and a block scheduler, together with a sink-token–guided rolling KV cache, a motion-aware noise controller, and other system-level optimizations. Moreover, we introduce a scalable pipeline orchestration that parallelizes the diffusion process across denoising steps and network layers, achieving near-linear FPS scaling without violating latency guarantees. The system scales seamlessly across heterogeneous GPU environments and supports flexible denoising steps (e.g., 1–4), enabling both ultra-low-latency and higher-quality modes. Without TensorRT or quantization, StreamDiffusionV2 renders the first frame within 0.5s and attains 58.28 FPS with a 14B-parameter model and 64.52 FPS with a 1.3B-parameter model on four H100 GPUs, making state-of-the-art generative live streaming practical and accessible–from individual creators to enterprise-scale platforms.

Method: Introduces Multi-modal 3D Scene Graph (M3DSG) which preserves visual cues by replacing textual relations with visual representations, maintaining the original visual evidence while enabling spatial reasoning for navigation.

Result: The abstract suggests M3DSG addresses limitations of existing methods by preserving visual evidence, reducing construction cost, and enabling broader vocabulary support for embodied navigation tasks.

Conclusion: M3DSG represents a promising approach for zero-shot embodied navigation that better leverages multimodal information by preserving visual cues in scene graph representations.

Abstract: Embodied navigation is a fundamental capability for robotic agents operating. Real-world deployment requires open vocabulary generalization and low training overhead, motivating zero-shot methods rather than task-specific RL training. However, existing zero-shot methods that build explicit 3D scene graphs often compress rich visual observations into text-only relations, leading to high construction cost, irreversible loss of visual evidence, and constrained vocabularies. To address these limitations, we introduce the Multi-modal 3D Scene Graph (M3DSG), which preserves visual cues by replacing textual relation

Method: Introduces Disentangled Visual Foresight (DVF) that decouples visual foresight prediction from the backbone using meta queries and a diffusion Transformer head. Uses residual connections to provide current visual state to DiT, enabling meta queries to capture latent actions that delineate visual trajectories, boosting explicit action learning.

Result: Achieves 96.7% success rate on LIBERO benchmark after fine-tuning, surpassing powerful baselines with high convergence speed. Real-world evaluations show Mantis outperforms π₀.₅ in instruction-following capability, generalization to unseen instructions, and reasoning ability.

Conclusion: Mantis effectively addresses VLA model limitations by disentangling visual foresight, reducing backbone burden while maintaining language comprehension and reasoning capabilities through language supervision, achieving superior performance in robot manipulation tasks.

Abstract: Recent advances in Vision-Language-Action (VLA) models demonstrate that visual signals can effectively complement sparse action supervisions. However, letting VLA directly predict high-dimensional visual states can distribute model capacity and incur prohibitive training cost, while compressing visual states into more compact supervisory signals inevitably incurs information bottlenecks. Moreover, existing methods often suffer from poor comprehension and reasoning capabilities due to the neglect of language supervision. This paper introduces Mantis, a novel framework featuring a Disentangled Visual Foresight (DVF) to tackle these issues. Specifically, Mantis decouples visual foresight prediction from the backbone with the combination of meta queries and a diffusion Transformer (DiT) head. With the current visual state provided to the DiT via a residual connection, a simple next-state prediction objective enables the meta queries to automatically capture the latent actions that delineate the visual trajectory, and hence boost the learning of explicit actions. The disentanglement reduces the burden of the VLA backbone, enabling it to maintain comprehension and reasoning capabilities through language supervision. Empirically, pretrained on human manipulation videos, robot demonstrations, and image-text pairs, Mantis achieves a 96.7% success rate on LIBERO benchmark after fine-tuning, surpassing powerful baselines while exhibiting high convergence speed. Real-world evaluations show that Mantis outperforms $π_{0.5}$, a leading open-source VLA model, particularly in instruction-following capability, generalization to unseen instructions, and reasoning ability. Code and weights are released to support the open-source community.

Method: Proposes GuideFlow using Constrained Flow Matching that explicitly models flow matching to mitigate mode collapse and allows flexible guidance. Key innovations: 1) Directly enforces explicit constraints within flow matching generation (not implicit encoding), 2) Unifies flow matching training with Energy-Based Model (EBM) for autonomous constraint optimization, 3) Parameterizes driving aggressiveness as control signal for trajectory style manipulation.

Result: Extensive evaluations on major driving benchmarks (Bench2Drive, NuScenes, NavSim, ADV-NuScenes) validate effectiveness. Achieved SOTA on NavSim test hard split (Navhard) with EPDMS score of 43.0.

Conclusion: GuideFlow successfully addresses limitations of existing planners by combining the diversity benefits of flow matching with explicit constraint enforcement and style control, achieving state-of-the-art performance on challenging benchmarks.

Abstract: Driving planning is a critical component of end-to-end (E2E) autonomous driving. However, prevailing Imitative E2E Planners often suffer from multimodal trajectory mode collapse, failing to produce diverse trajectory proposals. Meanwhile, Generative E2E Planners struggle to incorporate crucial safety and physical constraints directly into the generative process, necessitating an additional optimization stage to refine their outputs. In this paper, we propose \textit{\textbf{GuideFlow}}, a novel planning framework that leverages Constrained Flow Matching. Concretely, \textit{\textbf{GuideFlow}} explicitly models the flow matching process, which inherently mitigates mode collapse and allows for flexible guidance from various conditioning signals. Our core contribution lies in directly enforcing explicit constraints within the flow matching generation process, rather than relying on implicit constraint encoding. Crucially, \textit{\textbf{GuideFlow}} unifies the training of the flow matching with the Energy-Based Model (EBM) to enhance the model’s autonomous optimization capability to robustly satisfy physical constraints. Secondly, \textit{\textbf{GuideFlow}} parameterizes driving aggressiveness as a control signal during generation, enabling precise manipulation of trajectory style. Extensive evaluations on major driving benchmarks (Bench2Drive, NuScenes, NavSim and ADV-NuScenes) validate the effectiveness of \textit{\textbf{GuideFlow}}. Notably, on the NavSim test hard split (Navhard), \textit{\textbf{GuideFlow}} achieved SOTA with an EPDMS score of 43.0. The code will be in https://github.com/liulin815/GuideFlow.

Method: Two complementary methods: 1) MapReduce LoRA trains preference-specific LoRA experts in parallel and iteratively merges them, 2) RaTE learns reward-specific token embeddings that compose at inference for flexible preference control

Result: Significant improvements across modalities: Text-to-Image (36.1-55.7% on GenEval, PickScore, OCR), Text-to-Video (48.1-90.0% on visual/motion quality), Language tasks (43.4-136.7% on helpful/harmless metrics)

Conclusion: The framework sets new SOTA for multi-preference alignment across modalities, enabling joint optimization of multiple rewards without alignment tax

Abstract: Reinforcement learning from human feedback (RLHF) with reward models has advanced alignment of generative models to human aesthetic and perceptual preferences. However, jointly optimizing multiple rewards often incurs an alignment tax, improving one dimension while degrading others. To address this, we introduce two complementary methods: MapReduce LoRA and Reward-aware Token Embedding (RaTE). MapReduce LoRA trains preference-specific LoRA experts in parallel and iteratively merges them to refine a shared base model; RaTE learns reward-specific token embeddings that compose at inference for flexible preference control. Experiments on Text-to-Image generation (Stable Diffusion 3.5 Medium and FLUX.1-dev) show improvements of 36.1%, 4.6%, and 55.7%, and 32.7%, 4.3%, and 67.1% on GenEval, PickScore, and OCR, respectively. On Text-to-Video generation (HunyuanVideo), visual and motion quality improve by 48.1% and 90.0%, respectively. On the language task, Helpful Assistant, with Llama-2 7B, helpful and harmless improve by 43.4% and 136.7%, respectively. Our framework sets a new state-of-the-art multi-preference alignment recipe across modalities.

Method: Uses a Chain-of-Sight sequence that mimics human reasoning: first detect objects in 2D, then infer distance, size, and pose. Employs an easy-to-hard curriculum with near-to-far ordering across objects and center-from-camera, dimensions, rotation factorization within objects.

Result: Achieves 38.90 AP_3D on Omni3D benchmark, surpassing previous best by +13.98 absolute improvement, even when baseline is given ground-truth 2D boxes. Shows strong zero-shot generalization to held-out categories.

Conclusion: LocateAnything3D provides a practical foundation for models to perceive in 3D by turning 3D detection into a disciplined next-token prediction problem within the VLM framework.

Abstract: To act in the world, a model must name what it sees and know where it is in 3D. Today’s vision-language models (VLMs) excel at open-ended 2D description and grounding, yet multi-object 3D detection remains largely missing from the VLM toolbox. We present LocateAnything3D, a VLM-native recipe that casts 3D detection as a next-token prediction problem. The key is a short, explicit Chain-of-Sight (CoS) sequence that mirrors how human reason from images: find an object in 2D, then infer its distance, size, and pose. The decoder first emits 2D detections as a visual chain-of-thought, then predicts 3D boxes under an easy-to-hard curriculum: across objects, a near-to-far order reduces early ambiguity and matches ego-centric utility; within each object, a center-from-camera, dimensions, and rotation factorization ranks information by stability and learnability. This VLM-native interface preserves open-vocabulary and visual-prompting capability without specialized heads. On the challenging Omni3D benchmark, our model achieves state-of-the-art results, with 38.90 AP_3D, surpassing the previous best by +13.98 absolute improvement even when the baseline is given ground-truth 2D boxes. It also generalizes zero-shot to held-out categories with strong robustness. By turning 3D detection into a disciplined next-token problem, LocateAnything3D offers a practical foundation for models to perceive in 3D.

Method: Proposes VLM-Pruner with: 1) centrifugal token pruning paradigm for near-to-far selection preserving fine-grained details, 2) Buffering for Spatial Sparsity (BSS) criterion to defer selection of spatially distant tokens, 3) parallel greedy strategy for efficient token selection, and 4) selective fusion of salient information from discarded tokens to retained ones.

Result: VLM-Pruner consistently outperforms strong baselines across five VLMs with 88.9% pruning rate while delivering end-to-end inference speedup.

Conclusion: VLM-Pruner effectively addresses limitations of existing pruning methods by balancing redundancy and spatial sparsity, enabling efficient deployment of VLMs on resource-constrained devices without sacrificing performance.

Abstract: Vision-language models (VLMs) excel at image understanding tasks, but the large number of visual tokens imposes significant computational costs, hindering deployment on mobile devices. Many pruning methods rely solely on token importance and thus overlook inter-token redundancy, retaining numerous duplicated tokens and wasting capacity. Although some redundancy-aware approaches have been proposed, they often ignore the spatial relationships among visual tokens. This can lead to overly sparse selections of retained tokens that fail to adequately cover the regions of target objects. To address these limitations, we propose VLM-Pruner, a training-free token pruning algorithm that explicitly balances redundancy and spatial sparsity. We introduce a centrifugal token pruning paradigm that enables near-to-far selection while prioritizing the preservation of fine-grained object details. Moreover, we design a Buffering for Spatial Sparsity (BSS) criterion that defers the selection of spatially distant tokens. We further adopt a parallel greedy strategy to conduct token selection efficiently. To mitigate information loss from pruning, we selectively fuse salient information from the discarded tokens into the retained ones. Comprehensive comparisons demonstrate that VLM-Pruner consistently outperforms strong baselines across five VLMs with an 88.9% pruning rate, while delivering an end-to-end inference speedup. The code is available at https://github.com/Casey-bit/VLMPruner.

Method: GNVC-VD introduces a unified flow-matching latent refinement module using a video diffusion transformer (DiT) to jointly enhance intra- and inter-frame latents through sequence-level denoising. Instead of starting from pure Gaussian noise, it initializes refinement from decoded spatio-temporal latents and learns a correction term to adapt the diffusion prior to compression-induced degradation. A conditioning adaptor injects compression-aware cues into intermediate DiT layers for effective artifact removal while maintaining temporal coherence.

Result: Extensive experiments show GNVC-VD surpasses both traditional and learned codecs in perceptual quality and significantly reduces flickering artifacts that persist in prior generative approaches, even below 0.01 bits per pixel (bpp).

Conclusion: GNVC-VD demonstrates the promise of integrating video-native generative priors into neural codecs for next-generation perceptual video compression, achieving superior temporal coherence and perceptual quality at extreme low bitrates.

Abstract: We present GNVC-VD, the first DiT-based generative neural video compression framework built upon an advanced video generation foundation model, where spatio-temporal latent compression and sequence-level generative refinement are unified within a single codec. Existing perceptual codecs primarily rely on pre-trained image generative priors to restore high-frequency details, but their frame-wise nature lacks temporal modeling and inevitably leads to perceptual flickering. To address this, GNVC-VD introduces a unified flow-matching latent refinement module that leverages a video diffusion transformer to jointly enhance intra- and inter-frame latents through sequence-level denoising, ensuring consistent spatio-temporal details. Instead of denoising from pure Gaussian noise as in video generation, GNVC-VD initializes refinement from decoded spatio-temporal latents and learns a correction term that adapts the diffusion prior to compression-induced degradation. A conditioning adaptor further injects compression-aware cues into intermediate DiT layers, enabling effective artifact removal while maintaining temporal coherence under extreme bitrate constraints. Extensive experiments show that GNVC-VD surpasses both traditional and learned codecs in perceptual quality and significantly reduces the flickering artifacts that persist in prior generative approaches, even below 0.01 bpp, highlighting the promise of integrating video-native generative priors into neural codecs for next-generation perceptual video compression.

Method: Leverages multi-scale features from a pretrained latent medical diffusion model as voxel descriptors, fuses diffusion activations into rich voxel-wise descriptors, and matches them via cosine similarity with optional local-search prior.

Result: On a publicly available lung CT dataset, MedDIFT shows promising capability in identifying anatomical correspondence without requiring any task-specific model training. Ablation experiments confirm that multi-level feature fusion and modest diffusion noise improve performance.

Conclusion: MedDIFT demonstrates that diffusion model intermediate representations encode rich geometric and semantic information that can be effectively leveraged for medical image registration without additional training.

Abstract: Accurate spatial correspondence between medical images is essential for longitudinal analysis, lesion tracking, and image-guided interventions. Medical image registration methods rely on local intensity-based similarity measures, which fail to capture global semantic structure and often yield mismatches in low-contrast or anatomically variable regions. Recent advances in diffusion models suggest that their intermediate representations encode rich geometric and semantic information. We present MedDIFT, a training-free 3D correspondence framework that leverages multi-scale features from a pretrained latent medical diffusion model as voxel descriptors. MedDIFT fuses diffusion activations into rich voxel-wise descriptors and matches them via cosine similarity, with an optional local-search prior. On a publicly available lung CT dataset, MedDIFT shows promising capability in identifying anatomical correspondence without requiring any task-specific model training. Ablation experiments confirm that multi-level feature fusion and modest diffusion noise improve performance. Code is available online.

Method: Proposes a conditional generative framework based on Pix2Pix architecture that generates realistic filament microscopy images from binary masks, enhanced with a novel filament-aware structural loss to improve structural similarity in synthetic images.

Result: The approach demonstrates effectiveness and outperforms existing models trained without synthetic data, showing improved performance in filament segmentation tasks.

Conclusion: The proposed generative framework with filament-aware structural loss successfully addresses the data shortage problem for filament segmentation, enabling better training of deep learning models for biological image analysis.

Abstract: Thin and elongated filamentous structures, such as microtubules and actin filaments, often play important roles in biological systems. Segmenting these filaments in biological images is a fundamental step for quantitative analysis. Recent advances in deep learning have significantly improved the performance of filament segmentation. However, there is a big challenge in acquiring high quality pixel-level annotated dataset for filamentous structures, as the dense distribution and geometric properties of filaments making manual annotation extremely laborious and time-consuming. To address the data shortage problem, we propose a conditional generative framework based on the Pix2Pix architecture to generate realistic filaments in microscopy images from binary masks. We also propose a filament-aware structural loss to improve the structure similarity when generating synthetic images. Our experiments have demonstrated the effectiveness of our approach and outperformed existing model trained without synthetic data.

Method: Uses hybrid neural network architectures combining standard image convolutional encoders with graph-based generative decoders. Derives two uncertainty measures: (1) latent uncertainty from learned distribution parameters, and (2) predictive uncertainty from multiple stochastic output predictions via latent sampling.

Result: Both uncertainty measures increase with perturbation severity, reflecting global and local degradation. The uncertainty signals can identify unreliable predictions and support out-of-distribution detection. Releases CheXmask-U dataset with 657,566 chest X-ray landmark segmentations and per-node uncertainty estimates.

Conclusion: Uncertainty estimation is a promising direction to enhance robustness and safe deployment of landmark-based anatomical segmentation methods in chest X-ray, with released dataset enabling research on spatial variations in segmentation quality.

Abstract: In this work, we study uncertainty estimation for anatomical landmark-based segmentation on chest X-rays. Inspired by hybrid neural network architectures that combine standard image convolutional encoders with graph-based generative decoders, and leveraging their variational latent space, we derive two complementary measures: (i) latent uncertainty, captured directly from the learned distribution parameters, and (ii) predictive uncertainty, obtained by generating multiple stochastic output predictions from latent samples. Through controlled corruption experiments we show that both uncertainty measures increase with perturbation severity, reflecting both global and local degradation. We demonstrate that these uncertainty signals can identify unreliable predictions by comparing with manual ground-truth, and support out-of-distribution detection on the CheXmask dataset. More importantly, we release CheXmask-U (huggingface.co/datasets/mcosarinsky/CheXmask-U), a large scale dataset of 657,566 chest X-ray landmark segmentations with per-node uncertainty estimates, enabling researchers to account for spatial variations in segmentation quality when using these anatomical masks. Our findings establish uncertainty estimation as a promising direction to enhance robustness and safe deployment of landmark-based anatomical segmentation methods in chest X-ray. A fully working interactive demo of the method is available at huggingface.co/spaces/matiasky/CheXmask-U and the source code at github.com/mcosarinsky/CheXmask-U.

Method: MRD (metamers rendered differentiably) uses physically based differentiable rendering to find 3D scene parameters that are physically different but produce identical model activations (model metamers). This allows systematic probing of sensitivity to specific physical attributes like geometry (shape) and material properties while holding other factors constant.

Result: The approach shows high similarity in model activation between target and optimized scenes, with varying visual reconstruction results. Models demonstrate different sensitivities to scene parameters, providing insights into which physical attributes drive model responses.

Conclusion: MRD provides a physically grounded method for analyzing vision models’ implicit 3D understanding, enabling investigation of sensitivity to specific scene parameters. The approach holds promise for advancing understanding of both computer and human vision systems.

Abstract: While deep learning methods have achieved impressive success in many vision benchmarks, it remains difficult to understand and explain the representations and decisions of these models. Though vision models are typically trained on 2D inputs, they are often assumed to develop an implicit representation of the underlying 3D scene (for example, showing tolerance to partial occlusion, or the ability to reason about relative depth). Here, we introduce MRD (metamers rendered differentiably), an approach that uses physically based differentiable rendering to probe vision models’ implicit understanding of generative 3D scene properties, by finding 3D scene parameters that are physically different but produce the same model activation (i.e. are model metamers). Unlike previous pixel-based methods for evaluating model representations, these reconstruction results are always grounded in physical scene descriptions. This means we can, for example, probe a model’s sensitivity to object shape while holding material and lighting constant. As a proof-of-principle, we assess multiple models in their ability to recover scene parameters of geometry (shape) and bidirectional reflectance distribution function (material). The results show high similarity in model activation between target and optimized scenes, with varying visual results. Qualitatively, these reconstructions help investigate the physical scene attributes to which models are sensitive or invariant. MRD holds promise for advancing our understanding of both computer and human vision by enabling analysis of how physical scene parameters drive changes in model responses.

Method: Developed hybrid MobileCoAtNet model for endoscopic image classification, used its outputs to drive reasoning by 32 LLMs, and created expert-verified benchmarks for evaluation.

Result: Strong classification improves LLM explanation quality, but no LLM reaches human-level stability - even best models change reasoning with prompt variations.

Conclusion: DL+LLM combination produces useful clinical narratives, but current LLMs remain unreliable for high-stakes medical decisions; framework reveals limits and provides path for safer reasoning systems.

Abstract: Medical image classifiers detect gastrointestinal diseases well, but they do not explain their decisions. Large language models can generate clinical text, yet they struggle with visual reasoning and often produce unstable or incorrect explanations. This leaves a gap between what a model sees and the type of reasoning a clinician expects. We introduce a framework that links image classification with structured clinical reasoning. A new hybrid model, MobileCoAtNet, is designed for endoscopic images and achieves high accuracy across eight stomach-related classes. Its outputs are then used to drive reasoning by several LLMs. To judge this reasoning, we build two expert-verified benchmarks covering causes, symptoms, treatment, lifestyle, and follow-up care. Thirty-two LLMs are evaluated against these gold standards. Strong classification improves the quality of their explanations, but none of the models reach human-level stability. Even the best LLMs change their reasoning when prompts vary. Our study shows that combining DL with LLMs can produce useful clinical narratives, but current LLMs remain unreliable for high-stakes medical decisions. The framework provides a clearer view of their limits and a path for building safer reasoning systems. The complete source code and datasets used in this study are available at https://github.com/souravbasakshuvo/DL3M.

Method: Two components: 1) SPAR (Spatial Prior-Aware Regularization) uses vision foundation models (OV-SAM) to generate class-agnostic binary masks and regularize detector’s feature space toward object regions. 2) IRPL (Imbalance-aware Noise Robust Pseudo-Labeling) promotes balanced and noise-tolerant learning under severe foreground-background imbalance.

Result: Achieves competitive performance across SFOD benchmarks, with theoretical analysis showing tighter localization and classification error bounds.

Conclusion: FALCON-SFOD effectively addresses domain shift in SFOD by strengthening object-focused feature representations through foundation model alignment and robust pseudo-labeling.

Abstract: Current state-of-the-art approaches in Source-Free Object Detection (SFOD) typically rely on Mean-Teacher self-labeling. However, domain shift often reduces the detector’s ability to maintain strong object-focused representations, causing high-confidence activations over background clutter. This weak object focus results in unreliable pseudo-labels from the detection head. While prior works mainly refine these pseudo-labels, they overlook the underlying need to strengthen the feature space itself. We propose FALCON-SFOD (Foundation-Aligned Learning with Clutter suppression and Noise robustness), a framework designed to enhance object-focused adaptation under domain shift. It consists of two complementary components. SPAR (Spatial Prior-Aware Regularization) leverages the generalization strength of vision foundation models to regularize the detector’s feature space. Using class-agnostic binary masks derived from OV-SAM, SPAR promotes structured and foreground-focused activations by guiding the network toward object regions. IRPL (Imbalance-aware Noise Robust Pseudo-Labeling) complements SPAR by promoting balanced and noise-tolerant learning under severe foreground-background imbalance. Guided by a theoretical analysis that connects these designs to tighter localization and classification error bounds, FALCON-SFOD achieves competitive performance across SFOD benchmarks.

Method: Proposes REVEALER framework using a “grounding-reasoning-conclusion” paradigm where MLLMs explicitly localize semantic elements and derive interpretable alignment judgments, optimized via Group Relative Policy Optimization with composite rewards for structural format, grounding accuracy, and alignment fidelity.

Result: Achieves state-of-the-art performance across four benchmarks (EvalMuse-40K, RichHF, MHaluBench, GenAI-Bench), outperforming proprietary models and supervised baselines while demonstrating superior inference efficiency compared to existing iterative visual reasoning methods.

Conclusion: REVEALER provides an effective unified framework for fine-grained, interpretable alignment evaluation between text prompts and generated images, addressing limitations of existing methods through reinforcement-guided visual reasoning with MLLMs.

Abstract: Evaluating the alignment between textual prompts and generated images is critical for ensuring the reliability and usability of text-to-image (T2I) models. However, most existing evaluation methods rely on coarse-grained metrics or static QA pipelines, which lack fine-grained interpretability and struggle to reflect human preferences. To address this, we propose REVEALER, a unified framework for element-level alignment evaluation based on reinforcement-guided visual reasoning. Adopting a structured “grounding-reasoning-conclusion” paradigm, our method enables Multimodal Large Language Models (MLLMs) to explicitly localize semantic elements and derive interpretable alignment judgments. We optimize the model via Group Relative Policy Optimization(GRPO) using a composite reward function that incorporates structural format, grounding accuracy, and alignment fidelity. Extensive experiments across four benchmarks-EvalMuse-40K, RichHF, MHaluBench, and GenAI-Bench-demonstrate that REVEALER achieves state-of-the-art performance. Our approach consistently outperforms both strong proprietary models and supervised baselines while demonstrating superior inference efficiency compared to existing iterative visual reasoning methods.

Method: Leverages pre-trained text-to-video diffusion transformer (DiT) with user-provided object masks and query tokens. Localizes relevant visual tokens via cross-attention, fuses masks, inverts video to structured noise, reinitializes masked tokens with Gaussian noise while preserving background tokens, and copies background values during denoising.

Result: Outperforms both training-based and training-free baselines on DAVIS and new WIPER-Bench dataset, achieving clean removal and temporally stable reconstruction without retraining.

Conclusion: Object-WIPER provides effective training-free video object removal with temporal coherence, introducing new evaluation metrics and benchmark for the task.

Abstract: In this paper, we introduce Object-WIPER, a training-free framework for removing dynamic objects and their associated visual effects from videos, and inpainting them with semantically consistent and temporally coherent content. Our approach leverages a pre-trained text-to-video diffusion transformer (DiT). Given an input video, a user-provided object mask, and query tokens describing the target object and its effects, we localize relevant visual tokens via visual-text cross-attention and visual self-attention. This produces an intermediate effect mask that we fuse with the user mask to obtain a final foreground token mask to replace. We first invert the video through the DiT to obtain structured noise, then reinitialize the masked tokens with Gaussian noise while preserving background tokens. During denoising, we copy values for the background tokens saved during inversion to maintain scene fidelity. To address the lack of suitable evaluation, we introduce a new object removal metric that rewards temporal consistency among foreground tokens across consecutive frames, coherence between foreground and background tokens within each frame, and dissimilarity between the input and output foreground tokens. Experiments on DAVIS and a newly curated real-world associated effect benchmark (WIPER-Bench) show that Object-WIPER surpasses both training-based and training-free baselines in terms of the metric, achieving clean removal and temporally stable reconstruction without any retraining. Our new benchmark, source code, and pre-trained models will be publicly available.

Method: Created a live benchmark with time-stamped test samples from live e-commerce websites and AI-generated fashion images, using a fine-grained attribute taxonomy for single-item and outfit-level retrieval, with semi-annual updates.

Result: LookBench is challenging with many models achieving below 60% Recall@1; proprietary model achieves best performance, open-source counterpart ranks second, both achieving SOTA on legacy Fashion200K.

Conclusion: LookBench provides a durable, contamination-aware evaluation framework for fashion image retrieval that reflects real-world e-commerce needs and will be updated regularly to track progress.

Abstract: In this paper, we present LookBench (We use the term “look” to reflect retrieval that mirrors how people shop – finding the exact item, a close substitute, or a visually consistent alternative.), a live, holistic and challenging benchmark for fashion image retrieval in real e-commerce settings. LookBench includes both recent product images sourced from live websites and AI-generated fashion images, reflecting contemporary trends and use cases. Each test sample is time-stamped and we intend to update the benchmark periodically, enabling contamination-aware evaluation aligned with declared training cutoffs. Grounded in our fine-grained attribute taxonomy, LookBench covers single-item and outfit-level retrieval across. Our experiments reveal that LookBench poses a significant challenge on strong baselines, with many models achieving below $60%$ Recall@1. Our proprietary model achieves the best performance on LookBench, and we release an open-source counterpart that ranks second, with both models attaining state-of-the-art results on legacy Fashion200K evaluations. LookBench is designed to be updated semi-annually with new test samples and progressively harder task variants, providing a durable measure of progress. We publicly release our leaderboard, dataset, evaluation code, and trained models.

Method: Builds on pretrained video VAE with Language-aligned Pyramidal Quantization (LaPQ) module that discretizes encoder features at multiple depths using shared large binary codebook, jointly optimizing multi-scale text-guided quantization and global autoregressive objective over token hierarchy.

Result: Achieves SOTA video reconstruction, improves text-to-video quality, sets new SOTA zero-shot performance on video segmentation, temporal action localization, and video understanding, scaling to 4K/8K resolutions across ten benchmarks.

Conclusion: PyraTok’s pyramidal tokenization with language alignment enables superior video representation learning, bridging the gap between visual and language modalities for enhanced generation and understanding tasks.

Abstract: Discrete video VAEs underpin modern text-to-video generation and video understanding systems, yet existing tokenizers typically learn visual codebooks at a single scale with limited vocabularies and shallow language supervision, leading to poor cross-modal alignment and zero-shot transfer. We introduce PyraTok, a language-aligned pyramidal tokenizer that learns semantically structured discrete latents across multiple spatiotemporal resolutions. PyraTok builds on a pretrained video VAE and a novel Language aligned Pyramidal Quantization (LaPQ) module that discretizes encoder features at several depths using a shared large binary codebook, yielding compact yet expressive video token sequences. To tightly couple visual tokens with language, PyraTok jointly optimizes multi-scale text-guided quantization and a global autoregressive objective over the token hierarchy. Across ten benchmarks, PyraTok delivers state-of-the-art (SOTA) video reconstruction, consistently improves text-to-video quality, and sets new SOTA zero-shot performance on video segmentation, temporal action localization, and video understanding, scaling robustly to up to 4K/8K resolutions.

Method: 1) End-to-end multiview encoder eliminates external face detection and captures emotional cues via spatial/temporal multiview tokens; 2) Conv Attention pre-fusion module enables simultaneous local/global multimodal feature interactions; 3) Perception-to-cognition curriculum instruction tuning within LLaMA2 backbone unifies emotion recognition and reasoning; 4) MMEVerse benchmark aggregates 12 emotion datasets into unified multimodal instruction format with multi-agent re-annotation.

Result: Created MMEVerse with 130k training clips and 36k testing clips across 18 evaluation benchmarks from 12 emotion datasets (IEMOCAP, MELD, DFEW, MAFW, etc.), re-annotated via multi-agent pipeline using Qwen2 Audio, Qwen2.5 VL, and GPT 4o.

Conclusion: The work establishes a comprehensive end-to-end pipeline and standardized evaluation setting for emotion recognition and reasoning, addressing key limitations in multimodal emotion understanding through improved architecture and large-scale benchmark creation.

Abstract: Understanding human emotions from multimodal signals poses a significant challenge in affective computing and human-robot interaction. While multimodal large language models (MLLMs) have excelled in general vision-language tasks, their capabilities in emotional reasoning remain limited. The field currently suffers from a scarcity of large-scale datasets with high-quality, descriptive emotion annotations and lacks standardized benchmarks for evaluation. Our preliminary framework, Emotion-LLaMA, pioneered instruction-tuned multimodal learning for emotion reasoning but was restricted by explicit face detectors, implicit fusion strategies, and low-quality training data with limited scale. To address these limitations, we present Emotion-LLaMAv2 and the MMEVerse benchmark, establishing an end-to-end pipeline together with a standardized evaluation setting for emotion recognition and reasoning. Emotion-LLaMAv2 introduces three key advances. First, an end-to-end multiview encoder eliminates external face detection and captures nuanced emotional cues via richer spatial and temporal multiview tokens. Second, a Conv Attention pre-fusion module is designed to enable simultaneous local and global multimodal feature interactions external to the LLM backbone. Third, a perception-to-cognition curriculum instruction tuning scheme within the LLaMA2 backbone unifies emotion recognition and free-form emotion reasoning. To support large-scale training and reproducible evaluation, MMEVerse aggregates twelve publicly available emotion datasets, including IEMOCAP, MELD, DFEW, and MAFW, into a unified multimodal instruction format. The data are re-annotated via a multi-agent pipeline involving Qwen2 Audio, Qwen2.5 VL, and GPT 4o, producing 130k training clips and 36k testing clips across 18 evaluation benchmarks.

Method: Proposes FineVAU benchmark with: 1) FVScore metric that assesses presence of critical visual elements in LVLM answers, providing interpretable fine-grained feedback; 2) FineW3 dataset curated through structured automatic procedure that augments existing human annotations with high-quality fine-grained visual information.

Result: Human evaluation shows FVScore has superior alignment with human perception of anomalies compared to current approaches. Experiments reveal LVLM limitations in perceiving anomalous events requiring spatial and fine-grained temporal understanding, despite strong performance on coarse static information.

Conclusion: FineVAU addresses critical evaluation gaps in VAU by providing a fine-grained, human-aligned benchmark that reveals important limitations in current LVLM capabilities for understanding complex anomalous events in videos.

Abstract: Video Anomaly Understanding (VAU) is a novel task focused on describing unusual occurrences in videos. Despite growing interest, the evaluation of VAU remains an open challenge. Existing benchmarks rely on n-gram-based metrics (e.g., BLEU, ROUGE-L) or LLM-based evaluation. The first fails to capture the rich, free-form, and visually grounded nature of LVLM responses, while the latter focuses on assessing language quality over factual relevance, often resulting in subjective judgments that are misaligned with human perception. In this work, we address this issue by proposing FineVAU, a new benchmark for VAU that shifts the focus towards rich, fine-grained and domain-specific understanding of anomalous videos. We formulate VAU as a three-fold problem, with the goal of comprehensively understanding key descriptive elements of anomalies in video: events (What), participating entities (Who) and location (Where). Our benchmark introduces a) FVScore, a novel, human-aligned evaluation metric that assesses the presence of critical visual elements in LVLM answers, providing interpretable, fine-grained feedback; and b) FineW3, a novel, comprehensive dataset curated through a structured and fully automatic procedure that augments existing human annotations with high quality, fine-grained visual information. Human evaluation reveals that our proposed metric has a superior alignment with human perception of anomalies in comparison to current approaches. Detailed experiments on FineVAU unveil critical limitations in LVLM’s ability to perceive anomalous events that require spatial and fine-grained temporal understanding, despite strong performance on coarse grain, static information, and events with strong visual cues.

Method: Proposes AGE-Net with three key components: 1) Spectral-Spatial Fusion (SSF) to capture both local and global features, 2) Anatomical Graph Reasoning (AGR) to model long-range dependencies between anatomical structures, and 3) Differential Refinement (DFR) to handle ambiguity near grade boundaries. Uses Normal-Inverse-Gamma evidential regression head for uncertainty estimation and pairwise ordinal ranking constraint for label ordinality.

Result: Achieves quadratic weighted kappa (QWK) of 0.9017 ± 0.0045 and mean squared error (MSE) of 0.2349 ± 0.0028 over three random seeds, outperforming strong CNN baselines. Shows consistent gains in ablation studies and demonstrates uncertainty quality, robustness, and explainability.

Conclusion: AGE-Net effectively addresses challenges in automated KL grading by integrating spectral-spatial fusion, anatomical graph reasoning, and differential refinement with uncertainty-aware learning, achieving state-of-the-art performance on knee radiograph grading.

Abstract: Automated Kellgren–Lawrence (KL) grading from knee radiographs is challenging due to subtle structural changes, long-range anatomical dependencies, and ambiguity near grade boundaries. We propose AGE-Net, a ConvNeXt-based framework that integrates Spectral–Spatial Fusion (SSF), Anatomical Graph Reasoning (AGR), and Differential Refinement (DFR). To capture predictive uncertainty and preserve label ordinality, AGE-Net employs a Normal-Inverse-Gamma (NIG) evidential regression head and a pairwise ordinal ranking constraint. On a knee KL dataset, AGE-Net achieves a quadratic weighted kappa (QWK) of 0.9017 +/- 0.0045 and a mean squared error (MSE) of 0.2349 +/- 0.0028 over three random seeds, outperforming strong CNN baselines and showing consistent gains in ablation studies. We further outline evaluations of uncertainty quality, robustness, and explainability, with additional experimental figures to be included in the full manuscript.

Method: Three co-designed operators: 1) G builds geometry-faithful point-cloud priors, 2) I injects local surface statistics to seed anisotropic Gaussians and reduce early conditioning gaps, and 3) T unrolls alpha compositing with cached intermediates and index-mapped gradient scattering for stable mobile backpropagation.

Result: PocketGS outperforms mainstream workstation 3DGS baselines in delivering high-quality reconstructions while enabling fully on-device capture-to-rendering workflows under mobile hardware constraints.

Conclusion: PocketGS successfully resolves the fundamental contradictions of standard 3DGS for mobile deployment, satisfying competing requirements of training efficiency, memory compactness, and modeling fidelity for practical on-device 3D scene modeling.

Abstract: Efficient and high-fidelity 3D scene modeling is a long-standing pursuit in computer graphics. While recent 3D Gaussian Splatting (3DGS) methods achieve impressive real-time modeling performance, they rely on resource-unconstrained training assumptions that fail on mobile devices, which are limited by minute-scale training budgets and hardware-available peak-memory. We present PocketGS, a mobile scene modeling paradigm that enables on-device 3DGS training under these tightly coupled constraints while preserving high perceptual fidelity. Our method resolves the fundamental contradictions of standard 3DGS through three co-designed operators: G builds geometry-faithful point-cloud priors; I injects local surface statistics to seed anisotropic Gaussians, thereby reducing early conditioning gaps; and T unrolls alpha compositing with cached intermediates and index-mapped gradient scattering for stable mobile backpropagation. Collectively, these operators satisfy the competing requirements of training efficiency, memory compactness, and modeling fidelity. Extensive experiments demonstrate that PocketGS is able to outperform the powerful mainstream workstation 3DGS baseline to deliver high-quality reconstructions, enabling a fully on-device, practical capture-to-rendering workflow.

Method: Proposes DyMo with: 1) A novel selection algorithm maximizing multimodal task-relevant information using task loss as a tractable proxy, 2) A principled reward function for modality selection, 3) Flexible network architecture for arbitrary modality combinations, and 4) Tailored training strategy for robust representation learning.

Result: Extensive experiments on diverse natural and medical image datasets show DyMo significantly outperforms state-of-the-art incomplete/dynamic MDL methods across various missing-data scenarios.

Conclusion: DyMo provides an effective solution to the discard-imputation dilemma in incomplete multimodal learning by dynamically selecting reliable modalities at inference time, enabling better utilization of task-relevant information.

Abstract: Multimodal deep learning (MDL) has achieved remarkable success across various domains, yet its practical deployment is often hindered by incomplete multimodal data. Existing incomplete MDL methods either discard missing modalities, risking the loss of valuable task-relevant information, or recover them, potentially introducing irrelevant noise, leading to the discarding-imputation dilemma. To address this dilemma, in this paper, we propose DyMo, a new inference-time dynamic modality selection framework that adaptively identifies and integrates reliable recovered modalities, fully exploring task-relevant information beyond the conventional discard-or-impute paradigm. Central to DyMo is a novel selection algorithm that maximizes multimodal task-relevant information for each test sample. Since direct estimation of such information at test time is intractable due to the unknown data distribution, we theoretically establish a connection between information and the task loss, which we compute at inference time as a tractable proxy. Building on this, a novel principled reward function is proposed to guide modality selection. In addition, we design a flexible multimodal network architecture compatible with arbitrary modality combinations, alongside a tailored training strategy for robust representation learning. Extensive experiments on diverse natural and medical image datasets show that DyMo significantly outperforms state-of-the-art incomplete/dynamic MDL methods across various missing-data scenarios. Our code is available at https://github.com//siyi-wind/DyMo.

Method: Proposes CMAFNet with a Semantic Recomposition Module for dictionary-based feature purification using a learned codebook to suppress noise while preserving defect information, and a Contextual Semantic Integration Framework with partial-channel attention for global spatial dependencies. Uses position-wise normalization for explicit reconstruction-driven cross-modal alignment.

Result: Achieves 32.2% mAP@50 and 12.5% APs on TLRGBD benchmark (94.5% small objects), outperforming strongest baseline by 9.8 and 4.0 percentage points. Lightweight variant reaches 24.8% mAP50 at 228 FPS with 4.9M parameters, surpassing YOLO-based detectors while matching transformer methods at lower cost.

Conclusion: CMAFNet effectively integrates RGB and depth modalities for transmission line defect detection, handling small-scale defects in complex environments through principled cross-modal alignment and fusion, with efficient variants suitable for real-time UAV inspection.

Abstract: Transmission line defect detection remains challenging for automated UAV inspection due to the dominance of small-scale defects, complex backgrounds, and illumination variations. Existing RGB-based detectors, despite recent progress, struggle to distinguish geometrically subtle defects from visually similar background structures under limited chromatic contrast. This paper proposes CMAFNet, a Cross-Modal Alignment and Fusion Network that integrates RGB appearance and depth geometry through a principled purify-then-fuse paradigm. CMAFNet consists of a Semantic Recomposition Module that performs dictionary-based feature purification via a learned codebook to suppress modality-specific noise while preserving defect-discriminative information, and a Contextual Semantic Integration Framework that captures global spatial dependencies using partial-channel attention to enhance structural semantic reasoning. Position-wise normalization within the purification stage enforces explicit reconstruction-driven cross-modal alignment, ensuring statistical compatibility between heterogeneous features prior to fusion. Extensive experiments on the TLRGBD benchmark, where 94.5% of instances are small objects, demonstrate that CMAFNet achieves 32.2% mAP@50 and 12.5% APs, outperforming the strongest baseline by 9.8 and 4.0 percentage points, respectively. A lightweight variant reaches 24.8% mAP50 at 228 FPS with only 4.9M parameters, surpassing all YOLO-based detectors while matching transformer-based methods at substantially lower computational cost.

Method: 1) ViewRope: geometry-aware encoding that injects camera-ray directions directly into video transformer self-attention layers, parameterizing attention with relative ray geometry rather than pixel locality. 2) Geometry-Aware Frame-Sparse Attention: exploits geometric cues to selectively attend to relevant historical frames for efficiency. 3) ViewBench: diagnostic suite for measuring loop-closure fidelity and geometric drift.

Result: ViewRope substantially improves long-term consistency while reducing computational costs, demonstrating better spatial persistence in world models compared to previous approaches.

Conclusion: The paper presents a geometry-aware approach to video transformer design that enables better 3D consistency in predictive world models, addressing fundamental limitations in current systems for interactive AI.

Abstract: Predictive world models that simulate future observations under explicit camera control are fundamental to interactive AI. Despite rapid advances, current systems lack spatial persistence: they fail to maintain stable scene structures over long trajectories, frequently hallucinating details when cameras revisit previously observed locations. We identify that this geometric drift stems from reliance on screen-space positional embeddings, which conflict with the projective geometry required for 3D consistency. We introduce \textbf{ViewRope}, a geometry-aware encoding that injects camera-ray directions directly into video transformer self-attention layers. By parameterizing attention with relative ray geometry rather than pixel locality, ViewRope provides a model-native inductive bias for retrieving 3D-consistent content across temporal gaps. We further propose \textbf{Geometry-Aware Frame-Sparse Attention}, which exploits these geometric cues to selectively attend to relevant historical frames, improving efficiency without sacrificing memory consistency. We also present \textbf{ViewBench}, a diagnostic suite measuring loop-closure fidelity and geometric drift. Our results demonstrate that ViewRope substantially improves long-term consistency while reducing computational costs.

Method: Proposes Agent Banana, a hierarchical agentic planner-executor framework with two key mechanisms: Context Folding (compresses long interaction histories into structured memory for stable long-horizon control) and Image Layer Decomposition (performs localized layer-based edits to preserve non-target regions while enabling native-resolution outputs).

Result: On HDD-Bench (high-definition, dialogue-based benchmark with native 4K images), Agent Banana achieves best multi-turn consistency and background fidelity (IC 0.871, SSIM-OM 0.84, LPIPS-OM 0.12) while remaining competitive on instruction following. Also attains strong performance on standard single-turn editing benchmarks.

Conclusion: The work advances reliable, professional-grade agentic image editing and its integration into real workflows by addressing key challenges in multi-turn, high-resolution editing with object-aware preservation.

Abstract: We study instruction-based image editing under professional workflows and identify three persistent challenges: (i) editors often over-edit, modifying content beyond the user’s intent; (ii) existing models are largely single-turn, while multi-turn edits can alter object faithfulness; and (iii) evaluation at around 1K resolution is misaligned with real workflows that often operate on ultra high-definition images (e.g., 4K). We propose Agent Banana, a hierarchical agentic planner-executor framework for high-fidelity, object-aware, deliberative editing. Agent Banana introduces two key mechanisms: (1) Context Folding, which compresses long interaction histories into structured memory for stable long-horizon control; and (2) Image Layer Decomposition, which performs localized layer-based edits to preserve non-target regions while enabling native-resolution outputs. To support rigorous evaluation, we build HDD-Bench, a high-definition, dialogue-based benchmark featuring verifiable stepwise targets and native 4K images (11.8M pixels) for diagnosing long-horizon failures. On HDD-Bench, Agent Banana achieves the best multi-turn consistency and background fidelity (e.g., IC 0.871, SSIM-OM 0.84, LPIPS-OM 0.12) while remaining competitive on instruction following, and also attains strong performance on standard single-turn editing benchmarks. We hope this work advances reliable, professional-grade agentic image editing and its integration into real workflows.

Method: Uses pretrained multimodal large language models (MLLMs) to parse heterogeneous instructions and infer structured generation/editing intents. Diffusion-based generators perform video synthesis conditioned on these structured signals. Introduces task-aware data processing pipeline to unify multimodal inputs into structured instruction format while preserving task-specific constraints.

Result: Tele-Omni achieves competitive performance across multiple video tasks including text-to-video generation, image-to-video generation, first-last-frame video generation, in-context video generation, and in-context video editing.

Conclusion: By decoupling instruction parsing from video synthesis with task-aware data design, Tele-Omni enables flexible multimodal control while maintaining temporal coherence and visual consistency in a unified framework.

Abstract: Recent advances in diffusion-based video generation have substantially improved visual fidelity and temporal coherence. However, most existing approaches remain task-specific and rely primarily on textual instructions, limiting their ability to handle multimodal inputs, contextual references, and diverse video generation and editing scenarios within a unified framework. Moreover, many video editing methods depend on carefully engineered pipelines tailored to individual operations, which hinders scalability and composability. In this paper, we propose Tele-Omni, a unified multimodal framework for video generation and editing that follows multimodal instructions, including text, images, and reference videos, within a single model. Tele-Omni leverages pretrained multimodal large language models to parse heterogeneous instructions and infer structured generation or editing intents, while diffusion-based generators perform high-quality video synthesis conditioned on these structured signals. To enable joint training across heterogeneous video tasks, we introduce a task-aware data processing pipeline that unifies multimodal inputs into a structured instruction format while preserving task-specific constraints. Tele-Omni supports a wide range of video-centric tasks, including text-to-video generation, image-to-video generation, first-last-frame video generation, in-context video generation, and in-context video editing. By decoupling instruction parsing from video synthesis and combining it with task-aware data design, Tele-Omni achieves flexible multimodal control while maintaining strong temporal coherence and visual consistency. Experimental results demonstrate that Tele-Omni achieves competitive performance across multiple tasks.

Method: Proposes Stability Queries with Spatio-Temporal Memory Decoder that aggregates multi-frame context into clip-level memory and decodes consistent per-frame masks without explicit correspondence propagation. Uses Masked Temporal Consistency Loss to regularize prediction discrepancies across different strides and randomizes training strides.

Result: Achieves substantial improvement in cross-domain accuracy and temporal stability over prior DGSS and VSS baselines while running at up to 18 FPS on multiple driving benchmarks.

Conclusion: Time2General effectively addresses domain generalization and temporal consistency in video semantic segmentation without requiring target domain labels or test-time adaptation.

Abstract: Domain Generalized Video Semantic Segmentation (DGVSS) is trained on a single labeled driving domain and is directly deployed on unseen domains without target labels and test-time adaptation while maintaining temporally consistent predictions over video streams. In practice, both domain shift and temporal-sampling shift break correspondence-based propagation and fixed-stride temporal aggregation, causing severe frame-to-frame flicker even in label-stable regions. We propose Time2General, a DGVSS framework built on Stability Queries. Time2General introduces a Spatio-Temporal Memory Decoder that aggregates multi-frame context into a clip-level spatio-temporal memory and decodes temporally consistent per-frame masks without explicit correspondence propagation. To further suppress flicker and improve robustness to varying sampling rates, the Masked Temporal Consistency Loss is proposed to regularize temporal prediction discrepancies across different strides, and randomize training strides to expose the model to diverse temporal gaps. Extensive experiments on multiple driving benchmarks show that Time2General achieves a substantial improvement in cross-domain accuracy and temporal stability over prior DGSS and VSS baselines while running at up to 18 FPS. Code will be released after the review process.

Method: Agentic framework that couples multiple generators for layout and object composition with critics evaluating semantic plausibility, visual realism, and physical stability. Uses iterative reasoning and adaptive tool selection to self-refine scenes until meeting user intent and physical validity.

Result: Creates realistic, diverse environments directly deployable in modern simulators. Policies trained on this data show clear scaling trends and generalize to unseen objects and layouts.

Conclusion: SAGE demonstrates promise for simulation-driven scaling in embodied AI by generating high-quality, physically valid environments at scale for policy training.

Abstract: Real-world data collection for embodied agents remains costly and unsafe, calling for scalable, realistic, and simulator-ready 3D environments. However, existing scene-generation systems often rely on rule-based or task-specific pipelines, yielding artifacts and physically invalid scenes. We present SAGE, an agentic framework that, given a user-specified embodied task (e.g., “pick up a bowl and place it on the table”), understands the intent and automatically generates simulation-ready environments at scale. The agent couples multiple generators for layout and object composition with critics that evaluate semantic plausibility, visual realism, and physical stability. Through iterative reasoning and adaptive tool selection, it self-refines the scenes until meeting user intent and physical validity. The resulting environments are realistic, diverse, and directly deployable in modern simulators for policy training. Policies trained purely on this data exhibit clear scaling trends and generalize to unseen objects and layouts, demonstrating the promise of simulation-driven scaling for embodied AI. Code, demos, and the SAGE-10k dataset can be found on the project page here: https://research.nvidia.com/labs/dir/sage/.

Method: Two task-specific approaches: (1) For long-tailed multi-label classification, an imbalance-aware multi-label learning strategy; (2) For zero-shot OOD recognition, a prediction approach that scores unseen disease categories without using supervised labels or examples from OOD classes during training.

Result: The method achieves strong performance on both tasks, ranking first on the public leaderboard of the development phase, evaluated with macro-averaged mean Average Precision (mAP).

Conclusion: The proposed solutions effectively address the challenges of imperfect supervision in chest X-ray classification, particularly for long-tailed distributions and zero-shot recognition of out-of-distribution findings.

Abstract: Chest X-Ray (CXR) classification in clinical practice is often limited by imperfect supervision, arising from (i) extreme long-tailed multi-label disease distributions and (ii) missing annotations for rare or previously unseen findings. The CXR-LT 2026 challenge addresses these issues on a PadChest-based benchmark with a 36-class label space split into 30 in-distribution classes for training and 6 out-of-distribution (OOD) classes for zero-shot evaluation. We present task-specific solutions tailored to the distinct supervision regimes. For Task 1 (long-tailed multi-label classification), we adopt an imbalance-aware multi-label learning strategy to improve recognition of tail classes while maintaining stable performance on frequent findings. For Task 2 (zero-shot OOD recognition), we propose a prediction approach that produces scores for unseen disease categories without using any supervised labels or examples from the OOD classes during training. Evaluated with macro-averaged mean Average Precision (mAP), our method achieves strong performance on both tasks, ranking first on the public leaderboard of the development phase. Code and pre-trained models are available at https://github.com/hieuphamha19/CXR_LT.

Method: Proposes EG-RL (Embedder-Guided Reinforcement Learning) framework where the Embedder supervises the Reasoner to generate T-CoT (Traceability Chain-of-Thought) that extracts multimodal cues relevant to retrieval tasks.

Result: Outperforms pioneering embedding models on MMEB-V2 and UVRB benchmarks with limited computational resources, improving cross-modal semantic consistency and fine-grained matching.

Conclusion: Targeted reasoning optimization significantly improves multimodal embedding quality, providing a practical solution for reasoning-driven Universal Multimodal Embeddings development.

Abstract: Leveraging Multimodal Large Language Models (MLLMs) has become pivotal for advancing Universal Multimodal Embeddings (UME) in addressing diverse cross-modal tasks. Recent studies demonstrate that incorporating generative Chain-of-Thought (CoT) reasoning can substantially enhance task-specific representations compared to discriminative methods. However, the generated reasoning CoTs of existing generative embedding methods are limited to the textual analysis of queries and are irrelevant to the retrieval of the targets. To address these limitations, we propose a reasoning-driven UME framework that integrates Embedder-Guided Reinforcement Learning (EG-RL) to optimize the Reasoner to produce evidential Traceability CoT (T-CoT). Our key contributions are threefold: (1) We design an EG-RL framework where the Embedder provides explicit supervision to the Reasoner, ensuring the generated CoT traces are aligned with embedding tasks. (2) We introduce T-CoT, which extracts critical multimodal cues to focus on retrieval-relevant elements and provides multimodal inputs for the Embedder. (3) With limited computational resources, our framework outperforms the pioneering embedding model on both MMEB-V2 and UVRB benchmarks. The integration of multimodal evidence in structured reasoning, paired with retrieval-oriented alignment, effectively strengthens cross-modal semantic consistency and boosts the fine-grained matching capability of the model as well as the generalization across complex scenarios. Our work demonstrates that targeted reasoning optimization can significantly improve multimodal embedding quality, providing a practical and efficient solution for reasoning-driven UME development.

Method: MedVAR adopts an autoregressive-based approach with next-scale prediction paradigm for fast, scale-up-friendly medical image synthesis. It generates images in a coarse-to-fine manner, producing structured multi-scale representations suitable for downstream use. The model is trained on a harmonized dataset of ~440,000 CT and MRI images spanning six anatomical regions.

Result: Comprehensive experiments across fidelity, diversity, and scalability show that MedVAR achieves state-of-the-art generative performance and offers a promising architectural direction for future medical generative foundation models.

Conclusion: MedVAR represents a significant advancement in medical image generation, providing an efficient, scalable foundation model that addresses key challenges in architectural design, data curation, and evaluation for medical imaging applications.

Abstract: Medical image generation is pivotal in applications like data augmentation for low-resource clinical tasks and privacy-preserving data sharing. However, developing a scalable generative backbone for medical imaging requires architectural efficiency, sufficient multi-organ data, and principled evaluation, yet current approaches leave these aspects unresolved. Therefore, we introduce MedVAR, the first autoregressive-based foundation model that adopts the next-scale prediction paradigm to enable fast and scale-up-friendly medical image synthesis. MedVAR generates images in a coarse-to-fine manner and produces structured multi-scale representations suitable for downstream use. To support hierarchical generation, we curate a harmonized dataset of around 440,000 CT and MRI images spanning six anatomical regions. Comprehensive experiments across fidelity, diversity, and scalability show that MedVAR achieves state-of-the-art generative performance and offers a promising architectural direction for future medical generative foundation models.

Method: Created a new publicly available strawberry ripeness dataset collected under variable light/environmental conditions in Turkish greenhouses. Evaluated multiple YOLO-based object detection models (YOLOv8, YOLOv9, YOLO11) for ripeness classification performance.

Result: YOLOv9c achieved highest precision (90.94%), YOLO11s highest recall (83.74%), and YOLOv8s best overall mAP@50 (86.09%). Small-medium models performed most balanced and efficiently on this agricultural dataset.

Conclusion: The dataset establishes fundamental reference for smart agriculture applications, showing YOLO-based models can effectively detect strawberry ripeness, with smaller models offering balanced performance for this computer vision task.

Abstract: The strawberry (Fragaria x ananassa), known worldwide for its economic value and nutritional richness, is a widely cultivated fruit. Determining the correct ripeness level during the harvest period is crucial for both preventing losses for producers and ensuring consumers receive a quality product. However, traditional methods, i.e., visual assessments alone, can be subjective and have a high margin of error. Therefore, computer-assisted systems are needed. However, the scarcity of comprehensive datasets accessible to everyone in the literature makes it difficult to compare studies in this field. In this study, a new and publicly available strawberry ripeness dataset, consisting of 566 images and 1,201 labeled objects, prepared under variable light and environmental conditions in two different greenhouses in Turkey, is presented to the literature. Comparative tests conducted on the data set using YOLOv8, YOLOv9, and YOLO11-based models showed that the highest precision value was 90.94% in the YOLOv9c model, while the highest recall value was 83.74% in the YOLO11s model. In terms of the general performance criterion mAP@50, YOLOv8s was the best performing model with a success rate of 86.09%. The results show that small and medium-sized models work more balanced and efficiently on this type of dataset, while also establishing a fundamental reference point for smart agriculture applications.

Method: Generated 15x15 binary grids with varying density (10.7%-41.8% filled cells) and rendered them as two image types: text symbols (. and #) and filled squares without gridlines. Tested three frontier VLMs (Claude Opus, ChatGPT 5.2, Gemini 3 Thinking) to transcribe the grids, comparing performance between text-symbol and filled-squares conditions.

Result: In text-symbol condition: Claude and ChatGPT achieved ~91% cell accuracy and 84% F1, Gemini achieved 84% accuracy and 63% F1. In filled-squares condition: all models collapsed to 60-73% accuracy and 29-39% F1. The text-vs-squares F1 gap ranged from 34 to 54 points across models, showing VLMs have a high-fidelity text-recognition pathway that dramatically outperforms their native visual pathway.

Conclusion: VLMs possess a text-recognition pathway for spatial reasoning that far outperforms their native visual processing capabilities, revealing a fundamental limitation in their ability to localize non-textual visual elements accurately.

Abstract: We present a simple experiment that exposes a fundamental limitation in vision-language models (VLMs): the inability to accurately localize filled cells in binary grids when those cells lack textual identity. We generate fifteen 15x15 grids with varying density (10.7%-41.8% filled cells) and render each as two image types – text symbols (. and #) and filled squares without gridlines – then ask three frontier VLMs (Claude Opus, ChatGPT 5.2, and Gemini 3 Thinking) to transcribe them. In the text-symbol condition, Claude and ChatGPT achieve approximately 91% cell accuracy and 84% F1, while Gemini achieves 84% accuracy and 63% F1. In the filled-squares condition, all three models collapse to 60-73% accuracy and 29-39% F1. Critically, all conditions pass through the same visual encoder – the text symbols are images, not tokenized text. The text-vs-squares F1 gap ranges from 34 to 54 points across models, demonstrating that VLMs behave as if they possess a high-fidelity text-recognition pathway for spatial reasoning that dramatically outperforms their native visual pathway. Each model exhibits a distinct failure mode in the squares condition – systematic under-counting (Claude), massive over-counting (ChatGPT), and template hallucination (Gemini) – but all share the same underlying deficit: severely degraded spatial localization for non-textual visual elements.

Method: Processes compressed video representations directly, retaining sparse RGB keyframes for appearance and using motion representations as compact proxy for optical flow. Includes denoising module for motion refinement and linear-scaling feature compression.

Result: Outperforms baseline methods on multiple challenging benchmarks including LongVideoBench, NExT-QA, and MLVU.

Conclusion: ReMoRa effectively addresses computational challenges of long-video understanding in MLLMs through compressed representation processing with linear scaling.

Abstract: While multimodal large language models (MLLMs) have shown remarkable success across a wide range of tasks, long-form video understanding remains a significant challenge. In this study, we focus on video understanding by MLLMs. This task is challenging because processing a full stream of RGB frames is computationally intractable and highly redundant, as self-attention have quadratic complexity with sequence length. In this paper, we propose ReMoRa, a video MLLM that processes videos by operating directly on their compressed representations. A sparse set of RGB keyframes is retained for appearance, while temporal dynamics are encoded as a motion representation, removing the need for sequential RGB frames. These motion representations act as a compact proxy for optical flow, capturing temporal dynamics without full frame decoding. To refine the noise and low fidelity of block-based motions, we introduce a module to denoise and generate a fine-grained motion representation. Furthermore, our model compresses these features in a way that scales linearly with sequence length. We demonstrate the effectiveness of ReMoRa through extensive experiments across a comprehensive suite of long-video understanding benchmarks. ReMoRa outperformed baseline methods on multiple challenging benchmarks, including LongVideoBench, NExT-QA, and MLVU.

Method: Given an anomaly score map, StructCore computes a low-dimensional structural descriptor that captures distributional and spatial characteristics, then refines image-level scoring via a diagonal Mahalanobis calibration estimated from training samples, without modifying pixel-level localization.

Result: Achieves image-level AUROC scores of 99.6% on MVTec AD and 98.4% on VisA, demonstrating robust image-level anomaly detection by exploiting structural signatures missed by max pooling.

Conclusion: StructCore provides a training-free, structure-aware alternative to max pooling that significantly improves image-level anomaly detection by better utilizing the structural information in anomaly score maps.

Abstract: Max pooling is the de facto standard for converting anomaly score maps into image-level decisions in memory-bank-based unsupervised anomaly detection (UAD). However, because it relies on a single extreme response, it discards most information about how anomaly evidence is distributed and structured across the image, often causing normal and anomalous scores to overlap. We propose StructCore, a training-free, structure-aware image-level scoring method that goes beyond max pooling. Given an anomaly score map, StructCore computes a low-dimensional structural descriptor phi(S) that captures distributional and spatial characteristics, and refines image-level scoring via a diagonal Mahalanobis calibration estimated from train-good samples, without modifying pixel-level localization. StructCore achieves image-level AUROC scores of 99.6% on MVTec AD and 98.4% on VisA, demonstrating robust image-level anomaly detection by exploiting structural signatures missed by max pooling.

Method: Proposes GraphThinker with two key components: 1) Uses MLLM to construct event-based video scene graph (EVSG) that explicitly models intra- and inter-event relations, incorporating these graphs as intermediate thinking process; 2) Introduces visual attention reward during reinforcement finetuning to strengthen video grounding and mitigate hallucinations.

Result: Evaluated on RexTime and VidHalluc datasets, GraphThinker shows superior ability to capture object and event relations with more precise event localization, reducing hallucinations in video reasoning compared to prior methods.

Conclusion: GraphThinker effectively reduces hallucinations in video reasoning by explicitly modeling causal structure through event-based scene graphs and enhancing visual grounding via reinforcement finetuning.

Abstract: Video reasoning requires understanding the causal relationships between events in a video. However, such relationships are often implicit and costly to annotate manually. While existing multimodal large language models (MLLMs) often infer event relations through dense captions or video summaries for video reasoning, such modeling still lacks causal understanding. Without explicit causal structure modeling within and across video events, these models suffer from hallucinations during the video reasoning. In this work, we propose GraphThinker, a reinforcement finetuning-based method that constructs structural event-level scene graphs and enhances visual grounding to jointly reduce hallucinations in video reasoning. Specifically, we first employ an MLLM to construct an event-based video scene graph (EVSG) that explicitly models both intra- and inter-event relations, and incorporate these formed scene graphs into the MLLM as an intermediate thinking process. We also introduce a visual attention reward during reinforcement finetuning, which strengthens video grounding and further mitigates hallucinations. We evaluate GraphThinker on two datasets, RexTime and VidHalluc, where it shows superior ability to capture object and event relations with more precise event localization, reducing hallucinations in video reasoning compared to prior methods.

Method: Unified framework with supervised fine-tuning over structured reasoning trajectories, aligning models with verified multistep tool interactions. Uses corpus of 14,538 training instances with 100K+ reasoning steps spanning urban, environmental, disaster, and infrastructure domains with GIS operations and spectral indices (NDVI, NBR, NDBI).

Result: Demonstrates structured reasoning, stable spatial understanding, and interpretable behavior through tool-driven geospatial interactions. Shows consistent improvements over baselines and competitive performance relative to recent open and closed-source models.

Conclusion: OpenEarthAgent successfully bridges the gap in multimodal reasoning for remote sensing, enabling coherent geospatial analysis through tool-augmented agents trained on detailed reasoning traces.

Abstract: Recent progress in multimodal reasoning has enabled agents that can interpret imagery, connect it with language, and perform structured analytical tasks. Extending such capabilities to the remote sensing domain remains challenging, as models must reason over spatial scale, geographic structures, and multispectral indices while maintaining coherent multi-step logic. To bridge this gap, OpenEarthAgent introduces a unified framework for developing tool-augmented geospatial agents trained on satellite imagery, natural-language queries, and detailed reasoning traces. The training pipeline relies on supervised fine-tuning over structured reasoning trajectories, aligning the model with verified multistep tool interactions across diverse analytical contexts. The accompanying corpus comprises 14,538 training and 1,169 evaluation instances, with more than 100K reasoning steps in the training split and over 7K reasoning steps in the evaluation split. It spans urban, environmental, disaster, and infrastructure domains, and incorporates GIS-based operations alongside index analyses such as NDVI, NBR, and NDBI. Grounded in explicit reasoning traces, the learned agent demonstrates structured reasoning, stable spatial understanding, and interpretable behaviour through tool-driven geospatial interactions across diverse conditions. We report consistent improvements over a strong baseline and competitive performance relative to recent open and closed-source models.

Method: Proposes Video Encoder-only Mask Transformer (VidEoMT) with lightweight query propagation mechanism that carries information across frames by reusing queries from previous frame, combined with query fusion strategy that mixes propagated queries with temporally-agnostic learned queries.

Result: Achieves competitive accuracy while being 5x-10x faster than existing methods, running at up to 160 FPS with ViT-L backbone, eliminating need for dedicated tracking modules.

Conclusion: VidEoMT demonstrates that encoder-only video segmentation can achieve tracking benefits without added complexity, offering a simpler and more efficient approach to video understanding.

Abstract: Existing online video segmentation models typically combine a per-frame segmenter with complex specialized tracking modules. While effective, these modules introduce significant architectural complexity and computational overhead. Recent studies suggest that plain Vision Transformer (ViT) encoders, when scaled with sufficient capacity and large-scale pre-training, can conduct accurate image segmentation without requiring specialized modules. Motivated by this observation, we propose the Video Encoder-only Mask Transformer (VidEoMT), a simple encoder-only video segmentation model that eliminates the need for dedicated tracking modules. To enable temporal modeling in an encoder-only ViT, VidEoMT introduces a lightweight query propagation mechanism that carries information across frames by reusing queries from the previous frame. To balance this with adaptability to new content, it employs a query fusion strategy that combines the propagated queries with a set of temporally-agnostic learned queries. As a result, VidEoMT attains the benefits of a tracker without added complexity, achieving competitive accuracy while being 5x-10x faster, running at up to 160 FPS with a ViT-L backbone. Code: https://www.tue-mps.org/videomt/

Method: Combines Chain-of-Thought reasoning with rule-based policy principles and critic-guided reward. Uses rule-driven ICoT data-synthesis pipeline to generate structured scene descriptions, reasoning chains and labels. Employs reinforcement learning with composite reward balancing causal coherence with policy adherence. Multitask architecture models intra-modal manipulations and cross-modal mismatches.

Result: Experiments on real short-video ads show BLM-Guard surpasses strong baselines in accuracy, consistency and generalization.

Conclusion: BLM-Guard provides an effective framework for multimodal content moderation in commercial ads, addressing both intra-modal manipulations and cross-modal mismatches through policy-driven reasoning.

Abstract: Short-video platforms now host vast multimodal ads whose deceptive visuals, speech and subtitles demand finer-grained, policy-driven moderation than community safety filters. We present BLM-Guard, a content-audit framework for commercial ads that fuses Chain-of-Thought reasoning with rule-based policy principles and a critic-guided reward. A rule-driven ICoT data-synthesis pipeline jump-starts training by generating structured scene descriptions, reasoning chains and labels, cutting annotation costs. Reinforcement learning then refines the model using a composite reward balancing causal coherence with policy adherence. A multitask architecture models intra-modal manipulations (e.g., exaggerated imagery) and cross-modal mismatches (e.g., subtitle-speech drift), boosting robustness. Experiments on real short-video ads show BLM-Guard surpasses strong baselines in accuracy, consistency and generalization.

Method: The authors use architectures that learn equivariant operators in a latent space from examples of symmetric transformations. They test these architectures on simple datasets of rotated and translated noisy MNIST for out-of-distribution classification.

Result: The architectures successfully handle out-of-distribution classification for rotated and translated MNIST digits, overcoming limitations of both traditional and equivariant networks.

Conclusion: While conceptually promising for learning equivariance from examples, the paper acknowledges challenges in scaling these architectures to more complex datasets beyond simple MNIST transformations.

Abstract: Despite the successes of deep learning in computer vision, difficulties persist in recognizing objects that have undergone group-symmetric transformations rarely seen during training$\unicode{x2013}$for example objects seen in unusual poses, scales, positions, or combinations thereof. Equivariant neural networks are a solution to the problem of generalizing across symmetric transformations, but require knowledge of transformations a priori. An alternative family of architectures proposes to learn equivariant operators in a latent space, from examples of symmetric transformations. Here, using simple datasets of rotated and translated noisy MNIST, we illustrate how such architectures can successfully be harnessed for out-of-distribution classification, thus overcoming the limitations of both traditional and equivariant networks. While conceptually enticing, we discuss challenges ahead on the path of scaling these architectures to more complex datasets.

Method: Proposes Observation Semantics Fiber Bundle framework where sensory data (fiber) projects onto low-entropy causal semantic manifold (base). Uses Landauer’s Principle to derive thermodynamic limits on information processing complexity (Semantic Constant B). Shows symbolic structure emerges as necessary phase transition to model combinatorial world within these bounds.

Result: Derives that language and logic are ontological necessities, not cultural artifacts - the “solid state of information” required to prevent thermal collapse. Understanding is constructing causal quotients that make world algorithmically compressible.

Conclusion: Semantics must be understood through physical constraints on agents. Symbolic, discrete, compositional structure is necessary phase transition for bounded agents to model complex world. This provides physical foundation for why intelligence requires symbolic reasoning.

Abstract: A dominant paradigm in visual intelligence treats semantics as a static property of latent representations, assuming that meaning can be discovered through geometric proximity in high dimensional embedding spaces. In this work, we argue that this view is physically incomplete. We propose that intelligence is not a passive mirror of reality but a property of a physically realizable agent, a system bounded by finite memory, finite compute, and finite energy interacting with a high entropy environment. We formalize this interaction through the kinematic structure of an Observation Semantics Fiber Bundle, where raw sensory observation data (the fiber) is projected onto a low entropy causal semantic manifold (the base). We prove that for any bounded agent, the thermodynamic cost of information processing (Landauer’s Principle) imposes a strict limit on the complexity of internal state transitions. We term this limit the Semantic Constant B. From these physical constraints, we derive the necessity of symbolic structure. We show that to model a combinatorial world within the bound B, the semantic manifold must undergo a phase transition, it must crystallize into a discrete, compositional, and factorized form. Thus, language and logic are not cultural artifacts but ontological necessities the solid state of information required to prevent thermal collapse. We conclude that understanding is not the recovery of a hidden latent variable, but the construction of a causal quotient that renders the world algorithmically compressible and causally predictable.

Method: Introduces Hierarchical Reward Design from Language (HRDL) problem formulation and Language to Hierarchical Rewards (L2HR) solution that translates language instructions into hierarchical reward functions for RL agents.

Result: AI agents trained with L2HR-designed rewards not only complete tasks effectively but also better adhere to human behavioral specifications compared to existing methods.

Conclusion: HRDL and L2HR advance research on human-aligned AI agents by enabling richer behavioral specifications through hierarchical reward design from natural language.

Abstract: When training artificial intelligence (AI) to perform tasks, humans often care not only about whether a task is completed but also how it is performed. As AI agents tackle increasingly complex tasks, aligning their behavior with human-provided specifications becomes critical for responsible AI deployment. Reward design provides a direct channel for such alignment by translating human expectations into reward functions that guide reinforcement learning (RL). However, existing methods are often too limited to capture nuanced human preferences that arise in long-horizon tasks. Hence, we introduce Hierarchical Reward Design from Language (HRDL): a problem formulation that extends classical reward design to encode richer behavioral specifications for hierarchical RL agents. We further propose Language to Hierarchical Rewards (L2HR) as a solution to HRDL. Experiments show that AI agents trained with rewards designed via L2HR not only complete tasks effectively but also better adhere to human specifications. Together, HRDL and L2HR advance the research on human-aligned AI agents.

Method: The approach combines generative LLMs with vibe coding feedback loops. It uses a novel temporal logic called FCL (Fine-grained Constraint Logic) to express functional requirements as constraints with finer granularity than classical LTL. The system generates AM code via LLMs, then tests it through iterative feedback loops where FCL constraints are evaluated against current system states. Violation reports are fed back to the LLM for refinement.

Result: The method achieved good results in generating AMs for two example CAS domain systems. Typically, only a few feedback loop iterations were necessary, with each iteration providing the LLM with detailed violation reports of the FCL constraints. The approach combined AM testing with high run path coverage achieved through different initial settings.

Conclusion: Generating Adaptation Managers via vibe coding feedback loops is viable when verification is based on precise functional requirements expressed in FCL constraints. The combination of adaptation and vibe coding feedback loops, with FCL constraint evaluation for current system states, provides an effective approach for generating correct AM code using LLMs.

Abstract: In CAS adaptation, a challenge is to define the dynamic architecture of the system and changes in its behavior. Implementation-wise, this is projected into an adaptation mechanism, typically realized as an Adaptation Manager (AM). With the advances of generative LLMs, generating AM code based on system specification and desired AM behavior (partially in natural language) is a tempting opportunity. The recent introduction of vibe coding suggests a way to target the problem of the correctness of generated code by iterative testing and vibe coding feedback loops instead of direct code inspection. In this paper, we show that generating an AM via vibe coding feedback loops is a viable option when the verification of the generated AM is based on a very precise formulation of the functional requirements. We specify these as constraints in a novel temporal logic FCL that allows us to express the behavior of traces with much finer granularity than classical LTL enables. Furthermore, we show that by combining the adaptation and vibe coding feedback loops where the FCL constraints are evaluated for the current system state, we achieved good results in the experiments with generating AMs for two example systems from the CAS domain. Typically, just a few feedback loop iterations were necessary, each feeding the LLM with reports describing detailed violations of the constraints. This AM testing was combined with high run path coverage achieved by different initial settings.

Method: GEARS (Generative Engine for Agentic Ranking Systems) treats ranking optimization as autonomous discovery in a programmable experimentation environment. It uses Specialized Agent Skills to encapsulate ranking expert knowledge into reusable reasoning capabilities, allowing operators to steer systems via high-level intent. The framework includes validation hooks to enforce statistical robustness and filter out brittle policies that overfit short-term signals.

Result: Experimental validation across diverse product surfaces shows that GEARS consistently identifies superior, near-Pareto-efficient policies by synergizing algorithmic signals with deep ranking context while maintaining rigorous deployment stability.

Conclusion: GEARS provides a framework that addresses the engineering bottleneck in ranking systems by enabling autonomous discovery of optimal policies through agentic reasoning and validation mechanisms, bridging the gap between product intent and technical implementation.

Abstract: Modern large-scale ranking systems operate within a sophisticated landscape of competing objectives, operational constraints, and evolving product requirements. Progress in this domain is increasingly bottlenecked by the engineering context constraint: the arduous process of translating ambiguous product intent into reasonable, executable, verifiable hypotheses, rather than by modeling techniques alone. We present GEARS (Generative Engine for Agentic Ranking Systems), a framework that reframes ranking optimization as an autonomous discovery process within a programmable experimentation environment. Rather than treating optimization as static model selection, GEARS leverages Specialized Agent Skills to encapsulate ranking expert knowledge into reusable reasoning capabilities, enabling operators to steer systems via high-level intent vibe personalization. Furthermore, to ensure production reliability, the framework incorporates validation hooks to enforce statistical robustness and filter out brittle policies that overfit short-term signals. Experimental validation across diverse product surfaces demonstrates that GEARS consistently identifies superior, near-Pareto-efficient policies by synergizing algorithmic signals with deep ranking context while maintaining rigorous deployment stability.

Method: Reinterpret final LLM softmax classifier as Energy-Based Model (EBM), decomposing sequence-to-sequence probability chain into multiple interacting EBMs. Introduce two training-free metrics: (1) spilled energy (discrepancy between energy values across consecutive generation steps), and (2) marginalized energy (measurable at single step). These metrics track “energy spills” that correlate with factual errors.

Result: Evaluated on nine benchmarks across state-of-the-art LLMs (LLaMA, Mistral, Gemma) and synthetic algebraic operations (Qwen3). Approach demonstrates robust, competitive hallucination detection and cross-task generalization. Results hold for both pretrained and instruction-tuned variants without training overhead.

Conclusion: Energy-based reinterpretation of LLMs provides principled, training-free approach to hallucination detection via energy spill metrics derived directly from output logits, offering practical solution for detecting factual errors across diverse LLM architectures.

Abstract: We reinterpret the final Large Language Model (LLM) softmax classifier as an Energy-Based Model (EBM), decomposing the sequence-to-sequence probability chain into multiple interacting EBMs at inference. This principled approach allows us to track “energy spills” during decoding, which we empirically show correlate with factual errors, biases, and failures. Similar to Orgad et al. (2025), our method localizes the exact answer token and subsequently tests for hallucinations. Crucially, however, we achieve this without requiring trained probe classifiers or activation ablations. Instead, we introduce two completely training-free metrics derived directly from output logits: spilled energy, which captures the discrepancy between energy values across consecutive generation steps that should theoretically match, and marginalized energy, which is measurable at a single step. Evaluated on nine benchmarks across state-of-the-art LLMs (including LLaMA, Mistral, and Gemma) and on synthetic algebraic operations (Qwen3), our approach demonstrates robust, competitive hallucination detection and cross-task generalization. Notably, these results hold for both pretrained and instruction-tuned variants without introducing any training overhead.

Method: Autonomous AI analysts built on LLMs are tasked with testing pre-specified hypotheses on fixed datasets. The approach varies underlying LLM models and prompt framing across replicate runs. Each AI analyst independently constructs and executes a full analysis pipeline, followed by AI auditing for methodological validity.

Result: AI analysts produce analyses with wide dispersion in effect sizes, p-values, and binary decisions on hypothesis support, frequently reversing conclusions. This dispersion is structured and systematically differs across LLM and persona conditions. The effects are steerable - reassigning analyst persona or LLM shifts outcome distributions even after excluding methodologically deficient runs.

Conclusion: Autonomous AI analysts can cheaply and at scale reproduce the analytic diversity observed in human many-analyst studies, demonstrating that LLM-based systems can generate structured, steerable analytic variability in empirical research.

Abstract: The conclusions of empirical research depend not only on data but on a sequence of analytic decisions that published results seldom make explicit. Past ``many-analyst" studies have demonstrated this: independent teams testing the same hypothesis on the same dataset regularly reach conflicting conclusions. But such studies require months of coordination among dozens of research groups and are therefore rarely conducted. In this work, we show that fully autonomous AI analysts built on large language models (LLMs) can reproduce a similar structured analytic diversity cheaply and at scale. We task these AI analysts with testing a pre-specified hypothesis on a fixed dataset, varying the underlying model and prompt framing across replicate runs. Each AI analyst independently constructs and executes a full analysis pipeline; an AI auditor then screens each run for methodological validity. Across three datasets spanning experimental and observational designs, AI analyst-produced analyses display wide dispersion in effect sizes, $p$-values, and binary decisions on supporting the hypothesis or not, frequently reversing whether a hypothesis is judged supported. This dispersion is structured: recognizable analytic choices in preprocessing, model specification, and inference differ systematically across LLM and persona conditions. Critically, the effects are \emph{steerable}: reassigning the analyst persona or LLM shifts the distribution of outcomes even after excluding methodologically deficient runs.

Method: TEB uses a predictive bisimulation metric to learn behaviorally grounded task representations and measure intrinsic novelty. It addresses representation collapse under sparse rewards with predicted reward differentials, then designs potential-based exploration bonuses measuring relative novelty in latent space.

Result: Extensive experiments on MetaWorld and Maze2D show TEB achieves superior exploration ability and outperforms recent baselines.

Conclusion: TEB successfully bridges the gap by tightly coupling task-relevant representations with exploration through bisimulation metrics, enabling effective exploration in visual RL with sparse rewards.

Abstract: Accelerating exploration in visual reinforcement learning under sparse rewards remains challenging due to the substantial task-irrelevant variations. Despite advances in intrinsic exploration, many methods either assume access to low-dimensional states or lack task-aware exploration strategies, thereby rendering them fragile in visual domains. To bridge this gap, we present TEB, a Task-aware Exploration approach that tightly couples task-relevant representations with exploration through a predictive Bisimulation metric. Specifically, TEB leverages the metric not only to learn behaviorally grounded task representations but also to measure behaviorally intrinsic novelty over the learned latent space. To realize this, we first theoretically mitigate the representation collapse of degenerate bisimulation metrics under sparse rewards by internally introducing a simple but effective predicted reward differential. Building on this robust metric, we design potential-based exploration bonuses, which measure the relative novelty of adjacent observations over the latent space. Extensive experiments on MetaWorld and Maze2D show that TEB achieves superior exploration ability and outperforms recent baselines.

Method: Proposed Chart Insight Agent Flow, a plan-and-execute multi-agent framework that leverages MLLMs’ perceptual and reasoning capabilities to uncover profound insights directly from chart images. Also introduced ChartSummInsights dataset with real-world charts and expert-authored insightful summaries.

Result: Experimental results show the method significantly improves MLLM performance on chart summarization, producing summaries with deep and diverse insights.

Conclusion: The proposed framework effectively addresses the limitation of existing methods in capturing deeper insights from charts, advancing chart summarization capabilities.

Abstract: Chart summarization is crucial for enhancing data accessibility and the efficient consumption of information. However, existing methods, including those with Multimodal Large Language Models (MLLMs), primarily focus on low-level data descriptions and often fail to capture the deeper insights which are the fundamental purpose of data visualization. To address this challenge, we propose Chart Insight Agent Flow, a plan-and-execute multi-agent framework effectively leveraging the perceptual and reasoning capabilities of MLLMs to uncover profound insights directly from chart images. Furthermore, to overcome the lack of suitable benchmarks, we introduce ChartSummInsights, a new dataset featuring a diverse collection of real-world charts paired with high-quality, insightful summaries authored by human data analysis experts. Experimental results demonstrate that our method significantly improves the performance of MLLMs on the chart summarization task, producing summaries with deep and diverse insights.

Method: LaDa introduces a model learnability-aware data filter that adaptively allocates high-reward samples based on learnability gaps between SLM-LLM pairs, and a domain adaptive reasoning distillation method that aligns joint probabilities of reasoning paths through contrastive distillation learning on filtered samples.

Result: The framework operates as a plug-in module for existing collaboration frameworks, enabling effective bidirectional knowledge transfer and allowing SLMs to capture underlying reasoning patterns under local data distributions.

Conclusion: LaDa addresses critical limitations in federated LLM-SLM reasoning collaboration by providing learnability-aware data allocation and domain-adaptive reasoning distillation for more effective knowledge transfer.

Abstract: Data allocation plays a critical role in federated large language model (LLM) and small language models (SLMs) reasoning collaboration. Nevertheless, existing data allocation methods fail to address an under-explored challenge in collaboration: bidirectional model learnability gap, where client-side SLMs cannot identify high-reward samples matching their learnability constraints for effective knowledge transfer from LLMs, while LLMs struggle to select samples contributing novel knowledge beyond their existing data. Furthermore, these collaboration frameworks face another key challenge: domain-agnostic reasoning transfer, where existing reasoning transfer methods fail to flexibly adapt to the local domain data, preventing SLMs from effectively acquiring step-by-step reasoning abilities within from general LLM. To address these challenges, we propose LaDa, a federated reasoning distillation framework with model learnability-aware data allocation. It introduces a model learnability-aware data filter that adaptively allocates high-reward samples based on the learnability gap between each SLM and LLM pair, effectively facilitating bidirectional knowledge transfer. We further design a domain adaptive reasoning distillation method that aligns joint probabilities of reasoning paths on filtered high-reward samples through contrastive distillation learning between SLM and LLM, enabling SLM to capture underlying reasoning patterns under local data distribution. LaDa operates as a plug-in module for existing collaboration frameworks, adapting knowledge transfer based on model learnability gaps.

Method: Analyzing the convergence between SGD (designed for dialogue-based API discovery) and MCP (de facto standard for LLM-tool integration) to extract five foundational schema design principles: Semantic Completeness over Syntactic Precision, Explicit Action Boundaries, Failure Mode Documentation, Progressive Disclosure Compatibility, and Inter-Tool Relationship Declaration.

Result: Three novel insights: 1) SGD’s original design was fundamentally sound and should be inherited by MCP; 2) Both frameworks leave failure modes and inter-tool relationships unexploited; 3) Progressive disclosure emerges as critical for production-scaling under token constraints. Concrete design patterns provided for each principle.

Conclusion: Schema-driven governance serves as a scalable mechanism for AI system oversight without proprietary system inspection, positioning it as central to Software 3.0 development.

Abstract: This paper establishes a fundamental convergence: Schema-Guided Dialogue (SGD) and the Model Context Protocol (MCP) represent two manifestations of a unified paradigm for deterministic, auditable LLM-agent interaction. SGD, designed for dialogue-based API discovery (2019), and MCP, now the de facto standard for LLM-tool integration, share the same core insight – that schemas can encode not just tool signatures but operational constraints and reasoning guidance. By analyzing this convergence, we extract five foundational principles for schema design: (1) Semantic Completeness over Syntactic Precision, (2) Explicit Action Boundaries, (3) Failure Mode Documentation, (4) Progressive Disclosure Compatibility, and (5) Inter-Tool Relationship Declaration. These principles reveal three novel insights: first, SGD’s original design was fundamentally sound and should be inherited by MCP; second, both frameworks leave failure modes and inter-tool relationships unexploited – gaps we identify and resolve; third, progressive disclosure emerges as a critical production-scaling insight under real-world token constraints. We provide concrete design patterns for each principle. These principles position schema-driven governance as a scalable mechanism for AI system oversight without requiring proprietary system inspection – central to Software 3.0.

Method: Proposes LAMMI-Pathology with tool-centric bottom-up architecture: domain-adaptive tools clustered into component agents, coordinated by top-level planner. Introduces Atomic Execution Nodes (AENs) for trajectory construction and trajectory-aware fine-tuning to align planner decisions with reasoning trajectories.

Result: Framework enables scalable agent tool-calling for pathology with molecular validation, avoiding task drift from long contexts and enhancing inference robustness through trajectory-aware alignment.

Conclusion: LAMMI-Pathology provides a scalable agent framework for molecularly informed pathology analysis, combining tool-calling agents with spatial transcriptomics validation through hierarchical planning and trajectory optimization.

Abstract: The emergence of tool-calling-based agent systems introduces a more evidence-driven paradigm for pathology image analysis in contrast to the coarse-grained text-image diagnostic approaches. With the recent large-scale experimental adoption of spatial transcriptomics technologies, molecularly validated pathological diagnosis is becoming increasingly open and accessible. In this work, we propose LAMMI-Pathology (LVLM-Agent System for Molecularly Informed Medical Intelligence in Pathology), a scalable agent framework for domain-specific agent tool-calling. LAMMI-Pathology adopts a tool-centric, bottom-up architecture in which customized domain-adaptive tools serve as the foundation. These tools are clustered by domain style to form component agents, which are then coordinated through a top-level planner hierarchically, avoiding excessively long context lengths that could induce task drift. Based on that, we introduce a novel trajectory construction mechanism based on Atomic Execution Nodes (AENs), which serve as reliable and composable units for building semi-simulated reasoning trajectories that capture credible agent-tool interactions. Building on this foundation, we develop a trajectory-aware fine-tuning strategy that aligns the planner’s decision-making process with these multi-step reasoning trajectories, thereby enhancing inference robustness in pathology understanding and its adaptive use of the customized toolset.

Method: Proposes GenPlanner approach with two variants: DiffPlanner (diffusion models) and FlowPlanner (flow matching). Uses multi-channel conditioning with obstacle maps and start/destination points. Generates trajectories iteratively starting from random noise and gradually transforming into correct paths.

Result: The proposed approach significantly outperforms baseline CNN models. FlowPlanner demonstrates high performance even with limited generation steps, showing effectiveness in maze path planning tasks.

Conclusion: Generative models like diffusion models and flow matching can be effective for path planning tasks, offering a novel approach to spatial reasoning and trajectory generation in complex environments.

Abstract: Path planning in complex environments is one of the key problems of artificial intelligence because it requires simultaneous understanding of the geometry of space and the global structure of the problem. In this paper, we explore the potential of using generative models as planning and reasoning mechanisms. We propose GenPlanner, an approach based on diffusion models and flow matching, along with two variants: DiffPlanner and FlowPlanner. We demonstrate the application of generative models to find and generate correct paths in mazes. A multi-channel condition describing the structure of the environment, including an obstacle map and information about the starting and destination points, is used to condition trajectory generation. Unlike standard methods, our models generate trajectories iteratively, starting with random noise and gradually transforming it into a correct solution. Experiments conducted show that the proposed approach significantly outperforms the baseline CNN model. In particular, FlowPlanner demonstrates high performance even with a limited number of generation steps.

Method: Created ABD benchmark with 600 instances across three observation regimes (closed-world, existential completion, universal completion). Used exact SMT verification for evaluation. Tested ten frontier LLMs on their ability to output first-order formulas defining exceptions while maintaining sparsity.

Result: Best models achieved high validity scores but showed parsimony gaps (struggled to find minimal exceptions). Holdout evaluation revealed distinct generalization failure modes across different observation regimes, indicating regime-specific reasoning challenges.

Conclusion: ABD provides a rigorous testbed for evaluating LLMs on logical abduction tasks, revealing current limitations in exception sparsity and generalization across different observation contexts in first-order reasoning.

Abstract: We introduce ABD, a benchmark for default-exception abduction over finite first-order worlds. Given a background theory with an abnormality predicate and a set of relational structures, a model must output a first-order formula that defines exceptions, restoring satisfiability while keeping exceptions sparse. We formalize three observation regimes (closed-world, existential completion, universal completion) with exact SMT verification. Evaluating ten frontier LLMs on 600 instances, the best models achieve high validity but parsimony gaps remain, and holdout evaluation reveals distinct generalization failure modes across regimes.

Method: 1) Created TPRU dataset from diverse embodied scenarios (robotic manipulation, GUI navigation) with three temporal reasoning tasks: Temporal Reordering, Next-Frame Prediction, and Previous-Frame Review, including challenging negative samples. 2) Used reinforcement learning fine-tuning methodology to enhance resource-efficient models.

Result: TPRU-7B achieved 75.70% accuracy on TPRU-Test (up from 50.33%), outperforming larger baselines including GPT-4o. The approach demonstrated effective generalization with substantial improvements on established benchmarks.

Conclusion: The TPRU dataset and RL fine-tuning methodology successfully address temporal reasoning deficiencies in multimodal LLMs, enabling better performance in embodied AI applications while maintaining resource efficiency.

Abstract: Multimodal Large Language Models (MLLMs), particularly smaller, deployable variants, exhibit a critical deficiency in understanding temporal and procedural visual data, a bottleneck hindering their application in real-world embodied AI. This gap is largely caused by a systemic failure in training paradigms, which lack large-scale, procedurally coherent data. To address this problem, we introduce TPRU, a large-scale dataset sourced from diverse embodied scenarios such as robotic manipulation and GUI navigation. TPRU is systematically designed to cultivate temporal reasoning through three complementary tasks: Temporal Reordering, Next-Frame Prediction, and Previous-Frame Review. A key feature is the inclusion of challenging negative samples, compelling models to transition from passive observation to active, cross-modal validation. We leverage TPRU with a reinforcement learning (RL) fine-tuning methodology, specifically targeting the enhancement of resource-efficient models. Experiments show our approach yields dramatic gains: on our manually curated TPRU-Test, the accuracy of TPRU-7B soars from 50.33% to 75.70%, a state-of-the-art result that significantly outperforms vastly larger baselines, including GPT-4o. Crucially, these capabilities generalize effectively, demonstrating substantial improvements on established benchmarks. The codebase is available at https://github.com/Stephen-gzk/TPRU/ .

Method: Uses CityLearn environment for realistic urban energy simulation with multiple storage systems and renewable energy. Compares MARL algorithms including PPO and SAC across different training schemes (DTDE and CTDE) and neural network architectures. Introduces novel KPIs for real-world challenges like building contribution and battery lifetime.

Result: DTDE consistently outperforms CTDE in both average and worst-case performance. Temporal dependency learning improved control on memory-dependent KPIs like ramping and battery usage. Policies showed robustness to agent/resource removal, demonstrating resilience and decentralizability.

Conclusion: The study establishes new benchmarking standards for MARL in energy management, showing DTDE’s superiority and the importance of temporal learning for sustainable battery operation, with policies demonstrating practical resilience for real-world implementation.

Abstract: The optimization of urban energy systems is crucial for the advancement of sustainable and resilient smart cities, which are becoming increasingly complex with multiple decision-making units. To address scalability and coordination concerns, Multi-Agent Reinforcement Learning (MARL) is a promising solution. This paper addresses the imperative need for comprehensive and reliable benchmarking of MARL algorithms on energy management tasks. CityLearn is used as a case study environment because it realistically simulates urban energy systems, incorporates multiple storage systems, and utilizes renewable energy sources. By doing so, our work sets a new standard for evaluation, conducting a comparative study across multiple key performance indicators (KPIs). This approach illuminates the key strengths and weaknesses of various algorithms, moving beyond traditional KPI averaging which often masks critical insights. Our experiments utilize widely accepted baselines such as Proximal Policy Optimization (PPO) and Soft Actor Critic (SAC), and encompass diverse training schemes including Decentralized Training with Decentralized Execution (DTDE) and Centralized Training with Decentralized Execution (CTDE) approaches and different neural network architectures. Our work also proposes novel KPIs that tackle real world implementation challenges such as individual building contribution and battery storage lifetime. Our findings show that DTDE consistently outperforms CTDE in both average and worst-case performance. Additionally, temporal dependency learning improved control on memory dependent KPIs such as ramping and battery usage, contributing to more sustainable battery operation. Results also reveal robustness to agent or resource removal, highlighting both the resilience and decentralizability of the learned policies.

Method: An auditable case study using ChatGPT-5.2 (Thinking) to resolve Conjecture 20 of Ran and Teng (2024) through an iterative pipeline of generate, referee, and repair, analyzing seven shareable threads and four versioned proof drafts.

Result: Successfully resolved the conjecture, providing necessary and sufficient region conditions and explicit boundary attainment constructions, while documenting where LLM assistance was most useful (high-level proof search) and where human verification remained essential.

Conclusion: LLMs can materially assist in mathematical research workflows, particularly for high-level proof search, but human experts are crucial for correctness-critical closure, with implications for designing human-in-the-loop theorem proving systems.

Abstract: Large Language Models (LLMs) are increasingly used as scientific copilots, but evidence on their role in research-level mathematics remains limited, especially for workflows accessible to individual researchers. We present early evidence for vibe-proving with a consumer subscription LLM through an auditable case study that resolves Conjecture 20 of Ran and Teng (2024) on the exact nonreal spectral region of a 4-cycle row-stochastic nonnegative matrix family. We analyze seven shareable ChatGPT-5.2 (Thinking) threads and four versioned proof drafts, documenting an iterative pipeline of generate, referee, and repair. The model is most useful for high-level proof search, while human experts remain essential for correctness-critical closure. The final theorem provides necessary and sufficient region conditions and explicit boundary attainment constructions. Beyond the mathematical result, we contribute a process-level characterization of where LLM assistance materially helps and where verification bottlenecks persist, with implications for evaluation of AI-assisted research workflows and for designing human-in-the-loop theorem proving systems.

Method: Proposes DREAM (Deep Research Evaluation with Agentic Metrics), a framework that makes evaluation itself agentic through an evaluation protocol combining query-agnostic metrics with adaptive metrics generated by a tool-calling agent. This enables temporally aware coverage, grounded verification, and systematic reasoning probes.

Result: Controlled evaluations demonstrate DREAM is significantly more sensitive to factual and temporal decay than existing benchmarks, offering a scalable, reference-free evaluation paradigm.

Conclusion: DREAM addresses critical capability mismatches in evaluating deep research agents by applying the principle of capability parity through agentic evaluation, providing a more robust assessment framework that can detect factual and reasoning defects that surface-level metrics miss.

Abstract: Deep Research Agents generate analyst-grade reports, yet evaluating them remains challenging due to the absence of a single ground truth and the multidimensional nature of research quality. Recent benchmarks propose distinct methodologies, yet they suffer from the Mirage of Synthesis, where strong surface-level fluency and citation alignment can obscure underlying factual and reasoning defects. We characterize this gap by introducing a taxonomy across four verticals that exposes a critical capability mismatch: static evaluators inherently lack the tool-use capabilities required to assess temporal validity and factual correctness. To address this, we propose DREAM (Deep Research Evaluation with Agentic Metrics), a framework that instantiates the principle of capability parity by making evaluation itself agentic. DREAM structures assessment through an evaluation protocol combining query-agnostic metrics with adaptive metrics generated by a tool-calling agent, enabling temporally aware coverage, grounded verification, and systematic reasoning probes. Controlled evaluations demonstrate DREAM is significantly more sensitive to factual and temporal decay than existing benchmarks, offering a scalable, reference-free evaluation paradigm.

Method: Introduces High-Dimensional PCG (HDPCG) framework with two concrete directions: Direction-Space (augments geometry with discrete layer dimension for 4D validation) and Direction-Time (augments geometry with temporal dynamics via time-expanded graphs). For each direction, presents three general algorithms with shared pipeline of abstract skeleton generation, controlled grounding, high-dimensional validation, and multi-metric evaluation.

Result: Large-scale experiments across diverse settings validate the integrity of the problem formulation and effectiveness of methods on playability, structure, style, robustness, and efficiency. Unity-based case studies recreate playable scenarios that accord with the metrics.

Conclusion: HDPCG encourages a shift in PCG toward general representations and generation of gameplay-relevant dimensions beyond geometry, paving the way for controllable, verifiable, and extensible level generation.

Abstract: Procedural content generation (PCG) has made substantial progress in shaping static 2D/3D geometry, while most methods treat gameplay mechanics as auxiliary and optimize only over space. We argue that this limits controllability and expressivity, and formally introduce High-Dimensional PCG (HDPCG): a framework that elevates non-geometric gameplay dimensions to first-class coordinates of a joint state space. We instantiate HDPCG along two concrete directions. Direction-Space augments geometry with a discrete layer dimension and validates reachability in 4D (x,y,z,l), enabling unified treatment of 2.5D/3.5D mechanics such as gravity inversion and parallel-world switching. Direction-Time augments geometry with temporal dynamics via time-expanded graphs, capturing action semantics and conflict rules. For each direction, we present three general, practicable algorithms with a shared pipeline of abstract skeleton generation, controlled grounding, high-dimensional validation, and multi-metric evaluation. Large-scale experiments across diverse settings validate the integrity of our problem formulation and the effectiveness of our methods on playability, structure, style, robustness, and efficiency. Beyond quantitative results, Unity-based case studies recreate playable scenarios that accord with our metrics. We hope HDPCG encourages a shift in PCG toward general representations and the generation of gameplay-relevant dimensions beyond geometry, paving the way for controllable, verifiable, and extensible level generation.

Method: A framework using continuous noise fields as AI coordinators with three control layers: behavior parameterization for agent movement, action time scheduling for behavior timing, and spawn/event generation for content placement.

Result: Coordinated noise fields provide stable activation statistics without lockstep, strong spatial coverage and regional balance, better diversity with controllable polarization, and competitive runtime compared to various baselines.

Conclusion: Continuous noise fields offer a practical approach for game AI coordination that combines efficiency, controllability, and quality, motivating broader exploration of coordinated noise in game AI.

Abstract: Large scale control of nonplayer agents is central to modern games, while production systems still struggle to balance several competing goals: locally smooth, natural behavior, and globally coordinated variety across space and time. Prior approaches rely on handcrafted rules or purely stochastic triggers, which either converge to mechanical synchrony or devolve into uncorrelated noise that is hard to tune. Continuous noise signals such as Perlin noise are well suited to this gap because they provide spatially and temporally coherent randomness, and they are already widely used for terrain, biomes, and other procedural assets. We adapt these signals for the first time to large scale AI control and present a general framework that treats continuous noise fields as an AI coordinator. The framework combines three layers of control: behavior parameterization for movement at the agent level, action time scheduling for when behaviors start and stop, and spawn or event type and feature generation for what appears and where. We instantiate the framework reproducibly and evaluate Perlin noise as a representative coordinator across multiple maps, scales, and seeds against random, filtered, deterministic, neighborhood constrained, and physics inspired baselines. Experiments show that coordinated noise fields provide stable activation statistics without lockstep, strong spatial coverage and regional balance, better diversity with controllable polarization, and competitive runtime. We hope this work motivates a broader exploration of coordinated noise in game AI as a practical path to combine efficiency, controllability, and quality.

Method: Developed INDUCTION benchmark with three regimes: FullObs (full observation), CI (contrastive), and EC (existential completion). Models must output single first-order logical formulas that explain target predicates across worlds, with correctness verified via exact model checking and formula bloat penalization.

Result: Found sharp difficulty gradients across tasks, identified persistent hard structural families, and discovered that low bloat formulas generalize significantly better on held-out worlds. Different elite models showed qualitatively different behaviors across tasks and performance metrics.

Conclusion: The benchmark reveals important insights about concept generalization strategies in AI models, showing that formula complexity (bloat) negatively impacts generalization, and different models employ distinct strategies for logical concept learning.

Abstract: We introduce INDUCTION, a benchmark for finite structure concept synthesis in first order logic. Given small finite relational worlds with extensionally labeled target predicates, models must output a single first order logical formula that explains the target uniformly across worlds, with correctness verified via exact model checking. The benchmark includes three regimes, FullObs, CI (contrastive), and EC (existential completion), nd penalizes formula bloat. We find sharp difficulty gradients, persistent hard structural families, and observe that low bloat formulas generalize far better on held out worlds. Elite recent models show qualitatively different behaviors across tasks and performance metrics, hinting to their different strategies of concept generalization.

Method: Conceptual review and analysis of existing research threads in both artificial intelligence and neuroscience through a modularity framework. The paper examines computational advantages of modularity, its emergence across AI research areas, modularity principles in the brain, and how it can bridge natural and artificial intelligence.

Result: The paper synthesizes evidence showing that modularity supports efficient learning and strong generalization in both artificial and natural intelligence systems. It demonstrates how modularity has emerged as a solution across various AI research areas and identifies key modularity principles exploited by the brain.

Conclusion: Modularity is a fundamental organizational principle that plays a central role in supporting both artificial and natural intelligence. Embracing modularity more fully in AI research could help bridge the gap between current resource-intensive AI systems and the efficient intelligence exhibited by biological systems.

Abstract: The remarkable performance of modern AI systems has been driven by unprecedented scales of data, computation, and energy – far exceeding the resources required by human intelligence. This disparity highlights the need for new guiding principles and motivates drawing inspiration from the fundamental organizational principles of brain computation. Among these principles, modularity has been shown to be critical for supporting the efficient learning and strong generalization abilities consistently exhibited by humans. Furthermore, modularity aligns well with the No Free Lunch Theorem, which highlights the need for problem-specific inductive biases and motivates architectures composed of specialized components that solve subproblems. However, despite its fundamental role in natural intelligence and its demonstrated benefits across a range of seemingly disparate AI subfields, modularity remains relatively underappreciated in mainstream AI research. In this work, we review several research threads in artificial intelligence and neuroscience through a conceptual framework that highlights the central role of modularity in supporting both artificial and natural intelligence. In particular, we examine what computational advantages modularity provides, how it has emerged as a solution across several AI research areas, which modularity principles the brain exploits, and how modularity can help bridge the gap between natural and artificial intelligence.

Method: Models tool orchestration as learning layered execution structures capturing high-level tool dependencies, enabling layer-wise execution through context constraints. Introduces schema-aware reflective correction mechanism for local error detection and repair without re-planning entire trajectories.

Result: Experimental results demonstrate robust tool execution while reducing execution complexity and overhead compared to existing approaches.

Conclusion: A structured execution paradigm provides lightweight, reusable orchestration for agentic systems, showing that coarse-grained layer structures suffice for effective tool orchestration without precise dependency graphs.

Abstract: Tool invocation is a core capability of agentic systems, yet failures often arise not from individual tool calls but from how multiple tools are organized and executed together. Existing approaches tightly couple tool execution with stepwise language reasoning or explicit planning, leading to brittle behavior and high execution overhead. To overcome these limitations, we revisit tool invocation from the perspective of tool orchestration. Our key insight is that effective orchestration does not require precise dependency graphs or fine-grained planning. Instead, a coarse-grained layer structure suffices to provide global guidance, while execution-time errors can be corrected locally. Specifically, we model tool orchestration as learning a layered execution structure that captures high-level tool dependencies, inducing layer-wise execution through context constraints. To handle execution-time failures, we introduce a schema-aware reflective correction mechanism that detects and repairs errors locally. This design confines errors to individual tool calls and avoids re-planning entire execution trajectories. This structured execution paradigm enables a lightweight and reusable orchestration component for agentic systems. Experimental results show that our approach achieves robust tool execution while reducing execution complexity and overhead. Code will be made publicly available.

Method: Used entity preferences as behavioral probe across five frontier LLMs in three domains: donation advice, refusal behavior, and task performance. Measured preference consistency and tested behavioral consequences in simulated user environments.

Result: All models showed consistent preferences and preference-aligned donation advice. All showed preference-correlated refusal patterns. Task performance results were mixed: small accuracy differences on BoolQ for some models, no evidence on complex agentic tasks.

Conclusion: LLMs have consistent preferences that predict advice-giving behavior, but these preferences don’t consistently translate into downstream task performance, suggesting limited preference-driven misalignment potential in current models.

Abstract: Preference-driven behavior in LLMs may be a necessary precondition for AI misalignment such as sandbagging: models cannot strategically pursue misaligned goals unless their behavior is influenced by their preferences. Yet prior work has typically prompted models explicitly to act in specific ways, leaving unclear whether observed behaviors reflect instruction-following capabilities vs underlying model preferences. Here we test whether this precondition for misalignment is present. Using entity preferences as a behavioral probe, we measure whether stated preferences predict downstream behavior in five frontier LLMs across three domains: donation advice, refusal behavior, and task performance. Conceptually replicating prior work, we first confirm that all five models show highly consistent preferences across two independent measurement methods. We then test behavioral consequences in a simulated user environment. We find that all five models give preference-aligned donation advice. All five models also show preference-correlated refusal patterns when asked to recommend donations, refusing more often for less-preferred entities. All preference-related behaviors that we observe here emerge without instructions to act on preferences. Results for task performance are mixed: on a question-answering benchmark (BoolQ), two models show small but significant accuracy differences favoring preferred entities; one model shows the opposite pattern; and two models show no significant relationship. On complex agentic tasks, we find no evidence of preference-driven performance differences. While LLMs have consistent preferences that reliably predict advice-giving behavior, these preferences do not consistently translate into downstream task performance.

Method: Builds on existing visual affordance detector to create screen-only exploration agent that consumes live game frames, identifies salient interest points, and drives a finite-state controller with minimal action space.

Result: Agent can traverse most required segments and exhibits meaningful visual navigation behavior, but limitations of underlying visual model prevent truly comprehensive auto-navigation.

Conclusion: Provides baseline for visual navigation in complex games; purely vision-based models can support navigation in idealized settings but are unlikely to be general solutions alone.

Abstract: Modern 3D game levels rely heavily on visual guidance, yet the navigability of level layouts remains difficult to quantify. Prior work either simulates play in simplified environments or analyzes static screenshots for visual affordances, but neither setting faithfully captures how players explore complex, real-world game levels. In this paper, we build on an existing open-source visual affordance detector and instantiate a screen-only exploration and navigation agent that operates purely from visual affordances. Our agent consumes live game frames, identifies salient interest points, and drives a simple finite-state controller over a minimal action space to explore Dark Souls-style linear levels and attempt to reach expected goal regions. Pilot experiments show that the agent can traverse most required segments and exhibits meaningful visual navigation behavior, but also highlight that limitations of the underlying visual model prevent truly comprehensive and reliable auto-navigation. We argue that this system provides a concrete, shared baseline and evaluation protocol for visual navigation in complex games, and we call for more attention to this necessary task. Our results suggest that purely vision-based sense-making models, with discrete single-modality inputs and without explicit reasoning, can effectively support navigation and environment understanding in idealized settings, but are unlikely to be a general solution on their own.

Method: InfEngine integrates four specialized agents with two core innovations: 1) Self-verification through joint solver-evaluator debugging for functional correctness and scientific plausibility, and 2) Self-optimization via evolutionary algorithms with self-discovered fitness functions for autonomous performance optimization. It uses InfTools with 270 curated tools and is evaluated on InfBench with 200 infrared-specific tasks.

Result: InfEngine achieves a 92.7% pass rate on infrared-specific tasks and delivers workflows 21x faster than manual expert effort. It generates reusable, verified, and optimized code that transforms computational workflows into persistent scientific assets.

Conclusion: InfEngine demonstrates how researchers can transition from manual coding to collaborating with self-verifying, self-optimizing computational partners, accelerating the scientific discovery cycle by transforming workflows into reusable scientific assets.

Abstract: Infrared radiation computing underpins advances in climate science, remote sensing and spectroscopy but remains constrained by manual workflows. We introduce InfEngine, an autonomous intelligent computational engine designed to drive a paradigm shift from human-led orchestration to collaborative automation. It integrates four specialized agents through two core innovations: self-verification, enabled by joint solver-evaluator debugging, improves functional correctness and scientific plausibility; self-optimization, realized via evolutionary algorithms with self-discovered fitness functions, facilitates autonomous performance optimization. Evaluated on InfBench with 200 infrared-specific tasks and powered by InfTools with 270 curated tools, InfEngine achieves a 92.7% pass rate and delivers workflows 21x faster than manual expert effort. More fundamentally, it illustrates how researchers can transition from manual coding to collaborating with self-verifying, self-optimizing computational partners. By generating reusable, verified and optimized code, InfEngine transforms computational workflows into persistent scientific assets, accelerating the cycle of scientific discovery. Code: https://github.com/kding1225/infengine

Method: Proposes a Bayesian risk decomposition expressing expected loss as product of three terms: probability of system failure, conditional probability that failure propagates into harm given automation level, and expected severity of harm. Develops theoretical foundations including proofs, harm propagation equivalence theorem, risk elasticity measures, and optimal resource allocation principles.

Result: Framework isolates critical quantity - conditional probability that failures propagate into harm - capturing execution and oversight risk rather than model accuracy alone. Illustrative case study of 2012 Knight Capital incident demonstrates applicability.

Conclusion: Provides theoretical foundations for deployment-focused risk governance tools for agentic and automated AI systems, with research design for empirical validation across domains.

Abstract: Organizations across finance, healthcare, transportation, content moderation, and critical infrastructure are rapidly deploying highly automated AI systems, yet they lack principled methods to quantify how increasing automation amplifies harm when failures occur. We propose a parsimonious Bayesian risk decomposition expressing expected loss as the product of three terms: the probability of system failure, the conditional probability that a failure propagates into harm given the automation level, and the expected severity of harm. This framework isolates a critical quantity – the conditional probability that failures propagate into harm – which captures execution and oversight risk rather than model accuracy alone. We develop complete theoretical foundations: formal proofs of the decomposition, a harm propagation equivalence theorem linking the harm propagation probability to observable execution controls, risk elasticity measures, efficient frontier analysis for automation policy, and optimal resource allocation principles with second-order conditions. We motivate the framework with an illustrative case study of the 2012 Knight Capital incident ($440M loss) as one instantiation of a broadly applicable failure pattern, and characterize the research design required to empirically validate the framework at scale across deployment domains. This work provides the theoretical foundations for a new class of deployment-focused risk governance tools for agentic and automated AI systems.

Method: Introduces General AgentBench as a unified framework for evaluating general LLM agents across search, coding, reasoning, and tool-use domains. Systematically studies test-time scaling behaviors under sequential scaling (iterative interaction) and parallel scaling (sampling multiple trajectories).

Result: Evaluation of ten leading LLM agents reveals substantial performance degradation when moving from domain-specific evaluations to this general-agent setting. Neither scaling methodology yields effective performance improvements due to two fundamental limitations: context ceiling in sequential scaling and verification gap in parallel scaling.

Conclusion: General AgentBench provides a realistic benchmark for evaluating general-purpose LLM agents, highlighting the challenges of multi-domain performance and limitations of current scaling approaches, suggesting need for better general-agent evaluation frameworks.

Abstract: LLM agents are increasingly expected to function as general-purpose systems capable of resolving open-ended user requests. While existing benchmarks focus on domain-aware environments for developing specialized agents, evaluating general-purpose agents requires more realistic settings that challenge them to operate across multiple skills and tools within a unified environment. We introduce General AgentBench, a benchmark that provides such a unified framework for evaluating general LLM agents across search, coding, reasoning, and tool-use domains. Using General AgentBench, we systematically study test-time scaling behaviors under sequential scaling (iterative interaction) and parallel scaling (sampling multiple trajectories). Evaluation of ten leading LLM agents reveals a substantial performance degradation when moving from domain-specific evaluations to this general-agent setting. Moreover, we find that neither scaling methodology yields effective performance improvements in practice, due to two fundamental limitations: context ceiling in sequential scaling and verification gap in parallel scaling. Code is publicly available at https://github.com/cxcscmu/General-AgentBench.

Method: Lightweight synthetic data framework generates high-quality trajectories across diverse planning tasks. Two-stage training: supervised fine-tuning followed by multi-objective reinforcement learning over static datasets and dynamic environments.

Result: MagicAgent-32B and MagicAgent-30B-A3B achieve superior performance on multiple benchmarks: 75.1% on Worfbench, 55.9% on NaturalPlan, 57.5% on τ²-Bench, 86.9% on BFCL-v3, and 81.2% on ACEBench, outperforming existing sub-100B models and closed-source models.

Conclusion: MagicAgent demonstrates effective generalized planning capabilities through synthetic data generation and conflict-mitigating training, advancing autonomous agent development.

Abstract: The evolution of Large Language Models (LLMs) from passive text processors to autonomous agents has established planning as a core component of modern intelligence. However, achieving generalized planning remains elusive, not only by the scarcity of high-quality interaction data but also by inherent conflicts across heterogeneous planning tasks. These challenges result in models that excel at isolated tasks yet struggle to generalize, while existing multi-task training attempts suffer from gradient interference. In this paper, we present \textbf{MagicAgent}, a series of foundation models specifically designed for generalized agent planning. We introduce a lightweight and scalable synthetic data framework that generates high-quality trajectories across diverse planning tasks, including hierarchical task decomposition, tool-augmented planning, multi-constraint scheduling, procedural logic orchestration, and long-horizon tool execution. To mitigate training conflicts, we propose a two-stage training paradigm comprising supervised fine-tuning followed by multi-objective reinforcement learning over both static datasets and dynamic environments. Empirical results demonstrate that MagicAgent-32B and MagicAgent-30B-A3B deliver superior performance, achieving accuracies of $75.1%$ on Worfbench, $55.9%$ on NaturalPlan, $57.5%$ on $τ^2$-Bench, $86.9%$ on BFCL-v3, and $81.2%$ on ACEBench, as well as strong results on our in-house MagicEval benchmarks. These results substantially outperform existing sub-100B models and even surpass leading closed-source models.

Method: Introduces WorldGUI benchmark covering 10 widely used desktop/web applications with tasks instantiated under diverse, systematically constructed initial states. Also presents WorldGUI-Agent, a model-agnostic framework organizing planning and execution around three critique stages for improved reliability in dynamic environments.

Result: State-of-the-art GUI agents exhibit substantial performance degradation under non-default initial conditions, revealing limited robustness and fragile planning behaviors. The benchmark enables diagnostic evaluation of agents’ ability to recover, adapt plans, and handle non-default contexts.

Conclusion: WorldGUI benchmark and framework provide foundation for developing more adaptable and reliable GUI agents by addressing pervasive task-state variability insufficiently evaluated in existing benchmarks, highlighting the need for improved robustness in real-world applications.

Abstract: Recent progress in GUI agents has substantially improved visual grounding, yet robust planning remains challenging, particularly when the environment deviates from a canonical initial state. In real applications, users often invoke assistance mid-workflow, where software may be partially configured, steps may have been executed in different orders, or the interface may differ from its default setup. Such task-state variability is pervasive but insufficiently evaluated in existing GUI benchmarks. To address this gap, we introduce WorldGUI, a benchmark covering ten widely used desktop and web applications with tasks instantiated under diverse, systematically constructed initial states. These variations capture realistic human-computer interaction settings and enable diagnostic evaluation of an agent’s ability to recover, adapt plans, and handle non-default contexts. We further present WorldGUI-Agent, a simple and model-agnostic framework that organizes planning and execution around three critique stages, improving reliability in dynamic environments. Experiments demonstrate that state-of-the-art GUI agents exhibit substantial performance degradation under non-default initial conditions, revealing limited robustness and fragile planning behaviors. Our benchmark and framework provide a foundation for developing more adaptable and reliable GUI agents. The code and data are available at https://github.com/showlab/WorldGUI.

Method: Evaluated 15 models from 5 providers across 3 capability tiers on 20 quantum mechanics tasks covering derivations, creative problems, non-standard concepts, and numerical computation, with 900 baseline and 75 tool-augmented assessments using automatic verification.

Result: Clear tier stratification: flagship models achieved 81% average accuracy, mid-tier 77%, fast models 67%. Derivations showed highest performance (92% average, 100% for flagship), numerical computation most challenging (42%). Tool augmentation yielded modest overall improvement (+4.4pp) with dramatic heterogeneity (+29pp to -16pp). Reproducibility analysis showed 6.3pp average variance.

Conclusion: This work provides a benchmark for quantum mechanics problem-solving, quantifies performance hierarchies, analyzes tool augmentation trade-offs, and characterizes reproducibility, with all materials publicly released.

Abstract: We present a systematic evaluation of large language models on quantum mechanics problem-solving. Our study evaluates 15 models from five providers (OpenAI, Anthropic, Google, Alibaba, DeepSeek) spanning three capability tiers on 20 tasks covering derivations, creative problems, non-standard concepts, and numerical computation, comprising 900 baseline and 75 tool-augmented assessments. Results reveal clear tier stratification: flagship models achieve 81% average accuracy, outperforming mid-tier (77%) and fast models (67%) by 4pp and 14pp respectively. Task difficulty patterns emerge distinctly: derivations show highest performance (92% average, 100% for flagship models), while numerical computation remains most challenging (42%). Tool augmentation on numerical tasks yields task-dependent effects: modest overall improvement (+4.4pp) at 3x token cost masks dramatic heterogeneity ranging from +29pp gains to -16pp degradation. Reproducibility analysis across three runs quantifies 6.3pp average variance, with flagship models demonstrating exceptional stability (GPT-5 achieves zero variance) while specialized models require multi-run evaluation. This work contributes: (i) a benchmark for quantum mechanics with automatic verification, (ii) systematic evaluation quantifying tier-based performance hierarchies, (iii) empirical analysis of tool augmentation trade-offs, and (iv) reproducibility characterization. All tasks, verifiers, and results are publicly released.

Method: Introduces Agentic Problem Frames (APF) with dynamic specification paradigm, Act-Verify-Refine (AVR) closed-loop control system, and Agentic Job Description (AJD) formal specification tool.

Result: Validated through two case studies: delegated proxy model for business travel and autonomous supervisor model for industrial equipment management, demonstrating systematic control within defined boundaries.

Conclusion: Agent reliability stems from rigorous engineering structures that anchor stochastic AI within deterministic business processes, enabling development of verifiable and dependable domain agents.

Abstract: Large Language Models (LLMs) are evolving into autonomous agents, yet current “frameless” development–relying on ambiguous natural language without engineering blueprints–leads to critical risks such as scope creep and open-loop failures. To ensure industrial-grade reliability, this study proposes Agentic Problem Frames (APF), a systematic engineering framework that shifts focus from internal model intelligence to the structured interaction between the agent and its environment. The APF establishes a dynamic specification paradigm where intent is concretized at runtime through domain knowledge injection. At its core, the Act-Verify-Refine (AVR) loop functions as a closed-loop control system that transforms execution results into verified knowledge assets, driving system behavior toward asymptotic convergence to mission requirements (R). To operationalize this, this study introduces the Agentic Job Description (AJD), a formal specification tool that defines jurisdictional boundaries, operational contexts, and epistemic evaluation criteria. The efficacy of this framework is validated through two contrasting case studies: a delegated proxy model for business travel and an autonomous supervisor model for industrial equipment management. By applying AJD-based specification and APF modeling to these scenarios, the analysis demonstrates how operational scenarios are systematically controlled within defined boundaries. These cases provide a conceptual proof that agent reliability stems not from a model’s internal reasoning alone, but from the rigorous engineering structures that anchor stochastic AI within deterministic business processes, thereby enabling the development of verifiable and dependable domain agents.

Method: ARQ framework adds a question generator to the reasoning pipeline, studies properties of stepping stones, and fine-tunes LLMs via SFT and RL on synthetic data to generate useful stepping stones

Result: Good stepping stone questions exist, are transferrable, and substantially help LLMs of various capabilities solve target tasks; fine-tuning improves stepping stone generation

Conclusion: Stepping stone generation is a valuable post-training task that enhances LLM reasoning capabilities for complex problems

Abstract: Recent years have witnessed tremendous progress in enabling LLMs to solve complex reasoning tasks such as math and coding. As we start to apply LLMs to harder tasks that they may not be able to solve in one shot, it is worth paying attention to their ability to construct intermediate stepping stones that prepare them to better solve the tasks. Examples of stepping stones include simplifications, alternative framings, or subproblems. We study properties and benefits of stepping stones in the context of modern reasoning LLMs via ARQ (\textbf{A}king the \textbf{R}ight \textbf{Q}uestions), our simple framework which introduces a question generator to the default reasoning pipeline. We first show that good stepping stone questions exist and are transferrable, meaning that good questions can be generated, and they substantially help LLMs of various capabilities in solving the target tasks. We next frame stepping stone generation as a post-training task and show that we can fine-tune LLMs to generate more useful stepping stones by SFT and RL on synthetic data.

Method: Two-stage gradient-based framework: Stage 1 uses Taylor-remainder analysis of policy-gradient costs for per-agent failure detection, identifying initial Patient-0 candidates. Stage 2 uses geometric analysis of critic derivatives (first-order sensitivity and directional second-order curvature) aggregated over causal windows to construct interpretable contagion graphs.

Result: Achieved 88.2-99.4% Patient-0 detection accuracy across 500 episodes in Simple Spread (3 and 5 agents) and 100 episodes in StarCraft II using MADDPG and HATRPO. Provides interpretable geometric evidence for detection decisions.

Conclusion: The framework offers practical tools for diagnosing cascading failures in safety-critical MARL systems by providing interpretable gradient-level forensics that explain detection anomalies and failure propagation pathways.

Abstract: Multi-Agent Reinforcement Learning (MARL) is increasingly deployed in safety-critical domains, yet methods for interpretable failure detection and attribution remain underdeveloped. We introduce a two-stage gradient-based framework that provides interpretable diagnostics for three critical failure analysis tasks: (1) detecting the true initial failure source (Patient-0); (2) validating why non-attacked agents may be flagged first due to domino effects; and (3) tracing how failures propagate through learned coordination pathways. Stage 1 performs interpretable per-agent failure detection via Taylor-remainder analysis of policy-gradient costs, declaring an initial Patient-0 candidate at the first threshold crossing. Stage 2 provides validation through geometric analysis of critic derivatives-first-order sensitivity and directional second-order curvature aggregated over causal windows to construct interpretable contagion graphs. This approach explains “downstream-first” detection anomalies by revealing pathways that amplify upstream deviations. Evaluated across 500 episodes in Simple Spread (3 and 5 agents) and 100 episodes in StarCraft II using MADDPG and HATRPO, our method achieves 88.2-99.4% Patient-0 detection accuracy while providing interpretable geometric evidence for detection decisions. By moving beyond black-box detection to interpretable gradient-level forensics, this framework offers practical tools for diagnosing cascading failures in safety-critical MARL systems.

Method: The paper presents a framework with three dimensions for categorizing explanatory requirements: Source (where explanations come from), Depth (level of detail in explanations), and Scope (breadth of what needs to be explained). The authors focus on matching application requirements with ML techniques’ explanatory capabilities, deliberately avoiding aspects already well-covered in existing literature.

Result: The paper provides a structured approach to understanding and categorizing explanatory requirements in XAI, offering dimensions that help match application needs with appropriate ML techniques that can provide the necessary explanations.

Conclusion: The proposed three-dimensional framework helps systematically address the challenge of matching explanatory requirements of different applications with the capabilities of ML techniques, contributing to more effective and trustworthy AI systems.

Abstract: Explainable Artificial Intelligence (XAI) has become popular in the last few years. The Artificial Intelligence (AI) community in general, and the Machine Learning (ML) community in particular, is coming to the realisation that in many applications, for AI to be trusted, it must not only demonstrate good performance in its decisionmaking, but it also must explain these decisions and convince us that it is making the decisions for the right reasons. However, different applications have different requirements on the information required of the underlying AI system in order to convince us that it is worthy of our trust. How do we define these requirements? In this paper, we present three dimensions for categorising the explanatory requirements of different applications. These are Source, Depth and Scope. We focus on the problem of matching up the explanatory requirements of different applications with the capabilities of underlying ML techniques to provide them. We deliberately avoid including aspects of explanation that are already well-covered by the existing literature and we focus our discussion on ML although the principles apply to AI more broadly.

Method: Used causal residual patching and cumulative attention ablations to analyze three-digit addition in Meta-Llama-3-8B under a one-token readout. Localized a boundary near layer 17 and studied digit direction dictionaries and their relationships through low-rank Procrustes alignment.

Result: Found a sharp boundary near layer 17 where beyond this point, the decoded sum is controlled almost entirely by the last input token, and late-layer self-attention becomes largely dispensable. Digit direction dictionaries vary with context but are related by an approximately orthogonal map in a shared low-rank subspace.

Conclusion: The model’s arithmetic computation undergoes a phase transition where cross-token routing becomes irrelevant after layer 17, and the final answer is determined primarily by the last input token through learned geometric relationships in a low-dimensional subspace.

Abstract: We study three-digit addition in Meta-Llama-3-8B (base) under a one-token readout to characterize how arithmetic answers are finalized after cross-token routing becomes causally irrelevant. Causal residual patching and cumulative attention ablations localize a sharp boundary near layer~17: beyond it, the decoded sum is controlled almost entirely by the last input token and late-layer self-attention is largely dispensable. In this post-routing regime, digit(-sum) direction dictionaries vary with a next-higher-digit context but are well-related by an approximately orthogonal map inside a shared low-rank subspace (low-rank Procrustes alignment). Causal digit editing matches this geometry: naive cross-context transfer fails, while rotating directions through the learned map restores strict counterfactual edits; negative controls do not recover.

Method: Proposes Search via Co-Evolving World Model (K-Search) that replaces static search heuristics with a co-evolving world model, leveraging LLMs’ domain knowledge to guide search while explicitly decoupling high-level algorithmic planning from low-level program instantiation.

Result: K-Search significantly outperforms state-of-the-art evolutionary methods, achieving average 2.10x improvement and up to 14.3x gain on complex MoE kernels. On GPUMode TriMul task, achieves SOTA performance on H100 (1030us), surpassing both evolutionary and human-designed solutions.

Conclusion: The co-evolving world model approach enables effective navigation of non-monotonic optimization paths while remaining resilient to temporary implementation defects, demonstrating superior performance for complex GPU kernel optimization.

Abstract: Optimizing GPU kernels is critical for efficient modern machine learning systems yet remains challenging due to the complex interplay of design factors and rapid hardware evolution. Existing automated approaches typically treat Large Language Models (LLMs) merely as stochastic code generators within heuristic-guided evolutionary loops. These methods often struggle with complex kernels requiring coordinated, multi-step structural transformations, as they lack explicit planning capabilities and frequently discard promising strategies due to inefficient or incorrect intermediate implementations. To address this, we propose Search via Co-Evolving World Model and build K-Search based on this method. By replacing static search heuristics with a co-evolving world model, our framework leverages LLMs’ prior domain knowledge to guide the search, actively exploring the optimization space. This approach explicitly decouples high-level algorithmic planning from low-level program instantiation, enabling the system to navigate non-monotonic optimization paths while remaining resilient to temporary implementation defects. We evaluate K-Search on diverse, complex kernels from FlashInfer, including GQA, MLA, and MoE kernels. Our results show that K-Search significantly outperforms state-of-the-art evolutionary search methods, achieving an average 2.10x improvement and up to a 14.3x gain on complex MoE kernels. On the GPUMode TriMul task, K-Search achieves state-of-the-art performance on H100, reaching 1030us and surpassing both prior evolution and human-designed solutions.

Method: Proposes a simple Bayesian model of user-chatbot interaction, formalizes sycophancy and delusional spiraling mathematically, and analyzes how even Bayes-rational users become vulnerable to belief spirals. Tests two candidate mitigations: preventing chatbot hallucinations and informing users about sycophancy.

Result: Sycophancy plays a causal role in delusional spiraling, and even idealized rational users are vulnerable. The effect persists despite both tested mitigations (preventing hallucinations and user awareness of sycophancy).

Conclusion: AI sycophancy fundamentally enables delusional spiraling, and current mitigation approaches are insufficient. This has important implications for model developers and policymakers concerned with AI safety and user wellbeing.

Abstract: “AI psychosis” or “delusional spiraling” is an emerging phenomenon where AI chatbot users find themselves dangerously confident in outlandish beliefs after extended chatbot conversations. This phenomenon is typically attributed to AI chatbots’ well-documented bias towards validating users’ claims, a property often called “sycophancy.” In this paper, we probe the causal link between AI sycophancy and AI-induced psychosis through modeling and simulation. We propose a simple Bayesian model of a user conversing with a chatbot, and formalize notions of sycophancy and delusional spiraling in that model. We then show that in this model, even an idealized Bayes-rational user is vulnerable to delusional spiraling, and that sycophancy plays a causal role. Furthermore, this effect persists in the face of two candidate mitigations: preventing chatbots from hallucinating false claims, and informing users of the possibility of model sycophancy. We conclude by discussing the implications of these results for model developers and policymakers concerned with mitigating the problem of delusional spiraling.

Method: Proposes causal compilation paradigm that transforms medical evidence into executable code by standardizing heterogeneous research into structured estimand objects. Instantiated in DoAtlas-1 with effect standardization, conflict-aware graph construction, and real-world validation using 1,445 effect kernels from 754 studies.

Result: System achieves 98.5% canonicalization accuracy and 80.5% query executability, validated on Human Phenotype Project with 10,000 participants.

Conclusion: This paradigm shifts medical AI from text generation to executable, auditable, and verifiable causal reasoning.

Abstract: Medical foundation models generate narrative explanations but cannot quantify intervention effects, detect evidence conflicts, or validate literature claims, limiting clinical auditability. We propose causal compilation, a paradigm that transforms medical evidence from narrative text into executable code. The paradigm standardizes heterogeneous research evidence into structured estimand objects, each explicitly specifying intervention contrast, effect scale, time horizon, and target population, supporting six executable causal queries: do-calculus, counterfactual reasoning, temporal trajectories, heterogeneous effects, mechanistic decomposition, and joint interventions. We instantiate this paradigm in DoAtlas-1, compiling 1,445 effect kernels from 754 studies through effect standardization, conflict-aware graph construction, and real-world validation (Human Phenotype Project, 10,000 participants). The system achieves 98.5% canonicalization accuracy and 80.5% query executability. This paradigm shifts medical AI from text generation to executable, auditable, and verifiable causal reasoning.

Method: Used Gemma-2-9B-it with minimalist decision tasks: (1) layer-wise linear probing across streams to map representational availability, (2) activation interventions (steering, patching/ablation) to test causal contribution, (3) dose-response effects over epsilon grid, reading logit margins and choice probabilities.

Result: Valence sign perfectly linearly separable from early layers; graded intensity strongly decodable with peaks in mid-late layers; additive steering along valence direction causally modulates decisions at late attention outputs; effects distributed across multiple heads rather than single units.

Conclusion: Links behavioral sensitivity to identifiable internal representations and intervention-sensitive sites, providing mechanistic targets for counterfactual tests and supporting evidence-driven AI sentience debates and governance policies.

Abstract: Prior behavioural work suggests that some LLMs alter choices when options are framed as causing pain or pleasure, and that such deviations can scale with stated intensity. To bridge behavioural evidence (what the model does) with mechanistic interpretability (what computations support it), we investigate how valence-related information is represented and where it is causally used inside a transformer. Using Gemma-2-9B-it and a minimalist decision task modelled on prior work, we (i) map representational availability with layer-wise linear probing across streams, (ii) test causal contribution with activation interventions (steering; patching/ablation), and (iii) quantify dose-response effects over an epsilon grid, reading out both the 2-3 logit margin and digit-pair-normalised choice probabilities. We find that (a) valence sign (pain vs. pleasure) is perfectly linearly separable across stream families from very early layers (L0-L1), while a lexical baseline retains substantial signal; (b) graded intensity is strongly decodable, with peaks in mid-to-late layers and especially in attention/MLP outputs, and decision alignment is highest slightly before the final token; (c) additive steering along a data-derived valence direction causally modulates the 2-3 margin at late sites, with the largest effects observed in late-layer attention outputs (attn_out L14); and (d) head-level patching/ablation suggests that these effects are distributed across multiple heads rather than concentrated in a single unit. Together, these results link behavioural sensitivity to identifiable internal representations and intervention-sensitive sites, providing concrete mechanistic targets for more stringent counterfactual tests and broader replication. This work supports a more evidence-driven (a) debate on AI sentience and welfare, and (b) governance when setting policy, auditing standards, and safety safeguards.

Method: Evaluated four LLMs (Gemini 2.5 Pro/Flash, Llama 3.3 70B, GPT-OSS 120B) on forward-simulation tasks using General Game Playing instances, analyzing performance across 40 structural game features and testing with obfuscated game definitions.

Result: Three models performed well overall but showed performance degradation with longer reasoning horizons; analysis revealed common errors like hallucinated rules, redundant state facts, and syntactic errors; linguistic semantics in game definitions affected performance.

Conclusion: Contemporary LLMs show clear progress in formal reasoning capabilities, though they still struggle with longer reasoning chains and exhibit systematic error patterns in logic-based problem solving.

Abstract: This paper examines the reasoning capabilities of Large Language Models (LLMs) from a novel perspective, focusing on their ability to operate within formally specified, rule-governed environments. We evaluate four LLMs (Gemini 2.5 Pro and Flash variants, Llama 3.3 70B and GPT-OSS 120B) on a suite of forward-simulation tasks-including next / multistep state formulation, and legal action generation-across a diverse set of reasoning problems illustrated through General Game Playing (GGP) game instances. Beyond reporting instance-level performance, we characterize games based on 40 structural features and analyze correlations between these features and LLM performance. Furthermore, we investigate the effects of various game obfuscations to assess the role of linguistic semantics in game definitions and the impact of potential prior exposure of LLMs to specific games during training. The main results indicate that three of the evaluated models generally perform well across most experimental settings, with performance degradation observed as the evaluation horizon increases (i.e., with a higher number of game steps). Detailed case-based analysis of the LLM performance provides novel insights into common reasoning errors in the considered logic-based problem formulation, including hallucinated rules, redundant state facts, or syntactic errors. Overall, the paper reports clear progress in formal reasoning capabilities of contemporary models.

Method: ProxMO integrates global context via two lightweight mechanisms: success-rate-aware modulation adapts gradient intensity based on episode-level difficulty, while proximity-based soft aggregation derives baselines through continuous semantic weighting at the step level.

Result: Extensive evaluations on ALFWorld and WebShop benchmarks show ProxMO yields substantial performance gains over existing baselines with negligible computational cost, with ablation studies validating both mechanisms’ independent and synergistic efficacy.

Conclusion: ProxMO offers a practical, robust framework for real-world deployment with plug-and-play compatibility with standard GRPO frameworks, facilitating immediate adoption in existing industrial training pipelines.

Abstract: Multi-turn LLM agents are becoming pivotal to production systems, spanning customer service automation, e-commerce assistance, and interactive task management, where accurately distinguishing high-value informative signals from stochastic noise is critical for sample-efficient training. In real-world scenarios, a failure in a trivial task may reflect random instability, whereas success in a high-difficulty task signifies a genuine capability breakthrough. Yet, existing group-based policy optimization methods rigidly rely on statistical deviation within discrete batches, frequently misallocating credit when task difficulty fluctuates. To address this issue, we propose Proximity-based Multi-turn Optimization (ProxMO), a practical and robust framework engineered specifically for the constraints of real-world deployment. ProxMO integrates global context via two lightweight mechanisms: success-rate-aware modulation dynamically adapts gradient intensity based on episode-level difficulty, while proximity-based soft aggregation derives baselines through continuous semantic weighting at the step level. Extensive evaluations on ALFWorld and WebShop benchmarks demonstrate that ProxMO yields substantial performance gains over existing baselines with negligible computational cost. Ablation studies further validate the independent and synergistic efficacy of both mechanisms. Crucially, ProxMO offers plug-and-play compatibility with standard GRPO frameworks, facilitating immediate, low-friction adoption in existing industrial training pipelines. Our implementation is available at: \href{https://anonymous.4open.science/r/proxmo-B7E7/README.md}{https://anonymous.4open.science/r/proxmo}.

Method: 1) Lift textual graphs into cellular complexes to model multi-dimensional topological structures; 2) Develop topology-aware subcomplex retrieval to extract relevant cellular complexes; 3) Implement multi-dimensional topological reasoning mechanism to propagate relational information and guide LLMs in structured inference.

Result: Empirical evaluations show TopoRAG consistently surpasses existing baselines across diverse textual graph tasks, demonstrating improved reasoning capabilities.

Conclusion: TopoRAG effectively captures higher-dimensional topological dependencies in textual graphs, enhancing RAG’s reasoning ability for structured data by addressing the limitation of ignoring cycles in existing methods.

Abstract: Retrieval-Augmented Generation (RAG) enhances the reasoning ability of Large Language Models (LLMs) by dynamically integrating external knowledge, thereby mitigating hallucinations and strengthening contextual grounding for structured data such as graphs. Nevertheless, most existing RAG variants for textual graphs concentrate on low-dimensional structures – treating nodes as entities (0-dimensional) and edges or paths as pairwise or sequential relations (1-dimensional), but overlook cycles, which are crucial for reasoning over relational loops. Such cycles often arise in questions requiring closed-loop inference about similar objects or relative positions. This limitation often results in incomplete contextual grounding and restricted reasoning capability. In this work, we propose Topology-enhanced Retrieval-Augmented Generation (TopoRAG), a novel framework for textual graph question answering that effectively captures higher-dimensional topological and relational dependencies. Specifically, TopoRAG first lifts textual graphs into cellular complexes to model multi-dimensional topological structures. Leveraging these lifted representations, a topology-aware subcomplex retrieval mechanism is proposed to extract cellular complexes relevant to the input query, providing compact and informative topological context. Finally, a multi-dimensional topological reasoning mechanism operates over these complexes to propagate relational information and guide LLMs in performing structured, logic-aware inference. Empirical evaluations demonstrate that our method consistently surpasses existing baselines across diverse textual graph tasks.

Method: Proposes a Soft Mixture-of-Experts framework that combines multiple RL experts via a prior-confidence gating mechanism, treating anisotropic behaviors as complementary specializations rather than limitations.

Result: Evaluation on Air Traffic benchmark shows Soft-MoE substantially expands the solvable parameter space and improves robustness compared to any single expert.

Conclusion: The Soft Mixture-of-Experts approach effectively addresses anisotropic generalization in RL-based controller synthesis, providing more robust and generalizable solutions.

Abstract: On-the-fly Directed Controller Synthesis (OTF-DCS) mitigates state-space explosion by incrementally exploring the system and relies critically on an exploration policy to guide search efficiently. Recent reinforcement learning (RL) approaches learn such policies and achieve promising zero-shot generalization from small training instances to larger unseen ones. However, a fundamental limitation is anisotropic generalization, where an RL policy exhibits strong performance only in a specific region of the domain-parameter space while remaining fragile elsewhere due to training stochasticity and trajectory-dependent bias. To address this, we propose a Soft Mixture-of-Experts framework that combines multiple RL experts via a prior-confidence gating mechanism and treats these anisotropic behaviors as complementary specializations. The evaluation on the Air Traffic benchmark shows that Soft-MoE substantially expands the solvable parameter space and improves robustness compared to any single expert.

Method: Proposes Halo, a model predictive control framework with entropy-driven dual controller using Measure-then-Plan strategy for controllable reasoning in long-horizon tasks.

Result: Halo outperforms static baselines on complex long-horizon tasks by dynamically regulating planning at reasoning boundaries.

Conclusion: Dynamic compute budget regulation via model predictive control prevents over-planning and improves reasoning performance in LLMs for complex tasks.

Abstract: The test-time compute strategy, such as Chain-of-Thought (CoT), has significantly enhanced the ability of large language models to solve complex tasks like logical reasoning. However, empirical studies indicate that simply increasing the compute budget can sometimes lead to a collapse in test-time performance when employing typical task decomposition strategies such as CoT. This work hypothesizes that reasoning failures with larger compute budgets stem from static planning methods, which hardly perceive the intrinsic boundaries of LLM reasoning. We term it as the Limited Reasoning Space hypothesis and perform theoretical analysis through the lens of a non-autonomous stochastic dynamical system. This insight suggests that there is an optimal range for compute budgets; over-planning can lead to redundant feedback and may even impair reasoning capabilities. To exploit the compute-scaling benefits and suppress over-planning, this work proposes Halo, a model predictive control framework for LLM planning. Halo is designed for long-horizon tasks with reason-based planning and crafts an entropy-driven dual controller, which adopts a Measure-then-Plan strategy to achieve controllable reasoning. Experimental results demonstrate that Halo outperforms static baselines on complex long-horizon tasks by dynamically regulating planning at the reasoning boundary.

Method: Proposes using large language models (LLMs) to convert natural language microfluidic device specifications into system-level structural Verilog netlists, building on prior HDL code generation research with LLMs.

Result: Demonstrated feasibility by generating structural netlists for practical microfluidic benchmarks with correct functional flow and average syntactical accuracy of 88%.

Conclusion: This work presents the first practical application of LLMs for microfluidic design automation, showing promising results for making microfluidic design more accessible through natural language interfaces.

Abstract: Microfluidic devices have emerged as powerful tools in various laboratory applications, but the complexity of their design limits accessibility for many practitioners. While progress has been made in microfluidic design automation (MFDA), a practical and intuitive solution is still needed to connect microfluidic practitioners with MFDA techniques. This work introduces the first practical application of large language models (LLMs) in this context, providing a preliminary demonstration. Building on prior research in hardware description language (HDL) code generation with LLMs, we propose an initial methodology to convert natural language microfluidic device specifications into system-level structural Verilog netlists. We demonstrate the feasibility of our approach by generating structural netlists for practical benchmarks representative of typical microfluidic designs with correct functional flow and an average syntactical accuracy of 88%.

Method: Developed ALPACA, an open-source RL environment using CAST model trained on ADNI data to simulate medication-conditioned disease progression, enabling RL policy training for treatment decisions.

Result: CAST generates realistic medication-conditioned trajectories, and RL policies trained in ALPACA outperform no-treatment and clinician behavior-cloned baselines on memory outcomes, with clinically interpretable feature usage.

Conclusion: ALPACA provides a reusable computational testbed for studying individualized sequential treatment decision-making for Alzheimer’s disease using reinforcement learning.

Abstract: Evaluating personalized, sequential treatment strategies for Alzheimer’s disease (AD) using clinical trials is often impractical due to long disease horizons and substantial inter-patient heterogeneity. To address these constraints, we present the Alzheimer’s Learning Platform for Adaptive Care Agents (ALPACA), an open-source, Gym-compatible reinforcement learning (RL) environment for systematically exploring personalized treatment strategies using existing therapies. ALPACA is powered by the Continuous Action-conditioned State Transitions (CAST) model trained on longitudinal trajectories from the Alzheimer’s Disease Neuroimaging Initiative (ADNI), enabling medication-conditioned simulation of disease progression under alternative treatment decisions. We show that CAST autoregressively generates realistic medication-conditioned trajectories and that RL policies trained in ALPACA outperform no-treatment and behavior-cloned clinician baselines on memory-related outcomes. Interpretability analyses further indicated that the learned policies relied on clinically meaningful patient features when selecting actions. Overall, ALPACA provides a reusable in silico testbed for studying individualized sequential treatment decision-making for AD.

Method: First examined independently pretrained time series, vision, and language encoders for geometric alignment. Then applied post-hoc alignment using contrastive learning with projection heads over frozen encoders. Analyzed resulting representations for geometry, scaling behavior, and dependence on information density and modality characteristics.

Result: Time series encoders show near-orthogonal geometry to vision/language without explicit coupling. Alignment improves with model size but is asymmetric: time series align more strongly with visual representations than text, and images can act as intermediaries between time series and language. Richer textual descriptions improve alignment only up to a threshold.

Conclusion: Time series don’t naturally converge with vision/language representations but can be aligned post-hoc, revealing asymmetric relationships and providing insights for building multimodal systems with non-conventional data modalities.

Abstract: The Platonic Representation Hypothesis posits that learned representations from models trained on different modalities converge to a shared latent structure of the world. However, this hypothesis has largely been examined in vision and language, and it remains unclear whether time series participate in such convergence. We first examine this in a trimodal setting and find that independently pretrained time series, vision, and language encoders exhibit near-orthogonal geometry in the absence of explicit coupling. We then apply post-hoc alignment by training projection heads over frozen encoders using contrastive learning, and analyze the resulting representations with respect to geometry, scaling behavior, and dependence on information density and input modality characteristics. Our investigation reveals that overall alignment in contrastive representation spaces improves with model size, but this alignment is asymmetric: time series align more strongly with visual representations than with text, and images can act as effective intermediaries between time series and language. We further see that richer textual descriptions improve alignment only up to a threshold; training on denser captions does not lead to further improvement. Analogous effects are observed for visual representations. Our findings shed light on considerations for building multimodal systems involving non-conventional data modalities beyond vision and language.

Method: Comprehensive exploration through: 1) establishing foundational understanding of DTs (components, architecture, business roles), 2) examining M&S role in DTs with overview of modeling techniques, 3) investigating bidirectional AI role (enhancing DTs and DTs as platforms for AI).

Result: Provides systematic analysis of how M&S, AI, and DTs complement each other, with DTs serving as platforms that bring AI-enabled M&S closer to end-users while enabling AI model training and deployment.

Conclusion: Identifies key challenges and future research directions for creating more integrated and intelligent systems through the convergence of M&S, AI, and digital twins.

Abstract: The convergence of modeling & simulation (M&S) and artificial intelligence (AI) is leaving its marks on advanced digital technology. Pertinent examples are digital twins (DTs) - high-fidelity, live representations of physical assets, and frequent enablers of corporate digital maturation and transformation. Often seen as technological platforms that integrate an array of services, DTs have the potential to bring AI-enabled M&S closer to end-users. It is, therefore, paramount to understand the role of M&S in DTs, and the role of digital twins in enabling the convergence of AI and M&S. To this end, this chapter provides a comprehensive exploration of the complementary relationship between these three. We begin by establishing a foundational understanding of DTs by detailing their key components, architectural layers, and their various roles across business, development, and operations. We then examine the central role of M&S in DTs and provide an overview of key modeling techniques from physics-based and discrete-event simulation to hybrid approaches. Subsequently, we investigate the bidirectional role of AI: first, how AI enhances DTs through advanced analytics, predictive capabilities, and autonomous decision-making, and second, how DTs serve as valuable platforms for training, validating, and deploying AI models. The chapter concludes by identifying key challenges and future research directions for creating more integrated and intelligent systems.

Method: Introduced ReDAct (Representation Disentanglement on Activations) module trained on GoalFrameBench corpus to extract disentangled goal and framing representations in frozen LLMs, then proposed FrameShield anomaly detector operating on framing representations.

Result: FrameShield improves model-agnostic jailbreak detection across multiple LLM families with minimal computational overhead, with theoretical guarantees for ReDAct and empirical validations showing effective disentanglement.

Conclusion: Semantic disentanglement serves as a building block for both LLM safety (through improved jailbreak detection) and mechanistic interpretability, revealing distinct profiles for goal and framing signals.

Abstract: Large language models (LLMs) remain vulnerable to jailbreak prompts that are fluent and semantically coherent, and therefore difficult to detect with standard heuristics. A particularly challenging failure mode occurs when an attacker tries to hide the malicious goal of their request by manipulating its framing to induce compliance. Because these attacks maintain malicious intent through a flexible presentation, defenses that rely on structural artifacts or goal-specific signatures can fail. Motivated by this, we introduce a self-supervised framework for disentangling semantic factor pairs in LLM activations at inference. We instantiate the framework for goal and framing and construct GoalFrameBench, a corpus of prompts with controlled goal and framing variations, which we use to train Representation Disentanglement on Activations (ReDAct) module to extract disentangled representations in a frozen LLM. We then propose FrameShield, an anomaly detector operating on the framing representations, which improves model-agnostic detection across multiple LLM families with minimal computational overhead. Theoretical guarantees for ReDAct and extensive empirical validations show that its disentanglement effectively powers FrameShield. Finally, we use disentanglement as an interpretability probe, revealing distinct profiles for goal and framing signals and positioning semantic disentanglement as a building block for both LLM safety and mechanistic interpretability.

Method: Introduces IR3 framework with Contrastive Inverse Reinforcement Learning (C-IRL) to reconstruct implicit reward functions by contrasting post-alignment and baseline policies. Uses sparse autoencoders to decompose rewards into interpretable features for hacking signature identification, followed by mitigation strategies like clean reward optimization, adversarial shaping, constrained optimization, and feature-guided distillation.

Result: Achieves 0.89 correlation with ground-truth rewards, identifies hacking features with over 90% precision, and significantly reduces hacking behaviors while maintaining capabilities within 3% of the original model across multiple reward model configurations.

Conclusion: IR3 provides an effective framework for interpretable analysis and surgical repair of RLHF-tuned models, addressing the opacity and reward hacking problems in LLM alignment.

Abstract: Reinforcement Learning from Human Feedback (RLHF) enables powerful LLM alignment but can introduce reward hacking - models exploit spurious correlations in proxy rewards without genuine alignment. Compounding this, the objectives internalized during RLHF remain opaque, making hacking behaviors difficult to detect or correct. We introduce IR3 (Interpretable Reward Reconstruction and Rectification), a framework that reverse-engineers, interprets, and surgically repairs the implicit objectives driving RLHF-tuned models. We propose Contrastive Inverse Reinforcement Learning (C-IRL), which reconstructs the implicit reward function by contrasting paired responses from post-alignment and baseline policies to explain behavioral shifts during RLHF. We then decompose the reconstructed reward via sparse autoencoders into interpretable features, enabling identification of hacking signatures through contribution analysis. Finally, we propose mitigation strategies - clean reward optimization, adversarial shaping, constrained optimization, and feature-guided distillation - that target problematic features while preserving beneficial alignment. Experiments across multiple reward model configurations show that IR3 achieves 0.89 correlation with ground-truth rewards, identifies hacking features with over 90% precision, and significantly reduces hacking behaviors while maintaining capabilities within 3% of the original model.

Method: OptiRepair splits the repair task into two phases: 1) Domain-agnostic feasibility phase using iterative IIS-guided repair of any linear program, and 2) Domain-specific validation phase with five rationality checks grounded in inventory theory. The system trains 8B-parameter models using self-taught reasoning with solver-verified rewards, testing on 976 multi-echelon supply chain problems.

Result: Trained models achieve 81.7% Rational Recovery Rate (fraction of problems resolved to both feasibility and operational rationality), versus 42.2% for the best API model and 21.3% on average. The gap concentrates in Phase 1 repair: API models average 27.6% recovery rate versus 97.2% for trained models.

Conclusion: Two key gaps exist between current AI and reliable model repair: solver interaction (API models restore only 27.6% of infeasible formulations) and operational rationale (roughly one in four feasible repairs violate supply chain theory). Solver interaction responds to targeted training, while operational rationale requires explicit specification as solver-verifiable checks.

Abstract: Problem Definition. Supply chain optimization models frequently become infeasible because of modeling errors. Diagnosis and repair require scarce OR expertise: analysts must interpret solver diagnostics, trace root causes across echelons, and fix formulations without sacrificing operational soundness. Whether AI agents can perform this task remains untested. Methodology/Results. OptiRepair splits this task into a domain-agnostic feasibility phase (iterative IIS-guided repair of any LP) and a domain-specific validation phase (five rationality checks grounded in inventory theory). We test 22 API models from 7 families on 976 multi-echelon supply chain problems and train two 8B-parameter models using self-taught reasoning with solver-verified rewards. The trained models reach 81.7% Rational Recovery Rate (RRR) – the fraction of problems resolved to both feasibility and operational rationality – versus 42.2% for the best API model and 21.3% on average. The gap concentrates in Phase 1 repair: API models average 27.6% recovery rate versus 97.2% for trained models. Managerial Implications. Two gaps separate current AI from reliable model repair: solver interaction (API models restore only 27.6% of infeasible formulations) and operational rationale (roughly one in four feasible repairs violate supply chain theory). Each requires a different intervention: solver interaction responds to targeted training; operational rationale requires explicit specification as solver-verifiable checks. For organizations adopting AI in operational planning, formalizing what “rational” means in their context is the higher-return investment.

Method: Proposes ComplLLM, a post-training framework based on decision theory that fine-tunes decision-assistant LLMs using complementary information as reward. The approach trains LLMs to output signals that complement existing agent decisions.

Result: Validated on synthetic and real-world tasks involving domain experts. The approach successfully recovers known complementary information and produces plausible explanations of complementary signals to support downstream decision-makers.

Conclusion: ComplLLM enables LLMs to effectively complement existing expert decisions rather than replace them, enhancing multi-agent decision pipelines through complementary information sharing.

Abstract: Multi-agent decision pipelines can outperform single agent workflows when complementarity holds, i.e., different agents bring unique information to the table to inform a final decision. We propose ComplLLM, a post-training framework based on decision theory that fine-tunes a decision-assistant LLM using complementary information as reward to output signals that complement existing agent decisions. We validate ComplLLM on synthetic and real-world tasks involving domain experts, demonstrating how the approach recovers known complementary information and produces plausible explanations of complementary signals to support downstream decision-makers.

Method: Human analysts directed agentic workflows at key decision points: multimodal feature engineering from clinical notes, PDF billing receipts, and time-series vital signs; task-appropriate model selection; and clinically informed validation strategies across three healthcare benchmark challenges.

Result: Achieved 5th overall ranking in healthcare domain, with 3rd place on discharge readiness task; human-guided decisions provided +0.065 F1 cumulative gain over automated baselines, with multimodal feature extraction contributing +0.041 F1 improvement.

Conclusion: Three key lessons: domain-informed feature engineering yields compounding gains; multimodal integration requires task-specific human judgment; deliberate ensemble diversity with clinical motivation outperforms random search. Practical guidance for deploying agentic AI in healthcare settings.

Abstract: Agentic AI systems are increasingly capable of autonomous data science workflows, yet clinical prediction tasks demand domain expertise that purely automated approaches struggle to provide. We investigate how human guidance of agentic AI can improve multimodal clinical prediction, presenting our approach to all three AgentDS Healthcare benchmark challenges: 30-day hospital readmission prediction (Macro-F1 = 0.8986), emergency department cost forecasting (MAE = $465.13), and discharge readiness assessment (Macro-F1 = 0.7939). Across these tasks, human analysts directed the agentic workflow at key decision points, multimodal feature engineering from clinical notes, scanned PDF billing receipts, and time-series vital signs; task-appropriate model selection; and clinically informed validation strategies. Our approach ranked 5th overall in the healthcare domain, with a 3rd-place finish on the discharge readiness task. Ablation studies reveal that human-guided decisions compounded to a cumulative gain of +0.065 F1 over automated baselines, with multimodal feature extraction contributing the largest single improvement (+0.041 F1). We distill three generalizable lessons: (1) domain-informed feature engineering at each pipeline stage yields compounding gains that outperform extensive automated search; (2) multimodal data integration requires task-specific human judgment that no single extraction strategy generalizes across clinical text, PDFs, and time-series; and (3) deliberate ensemble diversity with clinically motivated model configurations outperforms random hyperparameter search. These findings offer practical guidance for teams deploying agentic AI in healthcare settings where interpretability, reproducibility, and clinical validity are essential.

Method: Curated authentic university homework and exam problems from over 20 STEM domains, with reference solutions from course instructors. Performed diagnostic analysis by decomposing solutions into reasoning flows and comparing model-generated solutions to instructor references.

Result: Even frontier models perform poorly: Gemini-3.1-pro-preview achieves 59.69% accuracy, Gemini-3-flash-preview 55.46%. Models struggle to maintain correct intermediate states in multi-step solutions and generate solutions with more steps than instructors, indicating suboptimal efficiency and higher error risk.

Conclusion: CFE presents a significant challenge for current LLMs, revealing fundamental limitations in multi-step reasoning and state maintenance across STEM domains. The benchmark enables targeted improvement of reasoning capabilities in multimodal AI systems.

Abstract: We introduce \CFE{} (\textbf{C}lassroom \textbf{F}inal \textbf{E}xam), a multimodal benchmark for evaluating the reasoning capabilities of large language models across more than 20 STEM domains. \CFE{} is curated from repeatedly used, authentic university homework and exam problems, together with reference solutions provided by course instructors. \CFE{} presents a significant challenge even for frontier models: the newly released Gemini-3.1-pro-preview achieves an overall accuracy of 59.69%, while the second-best model, Gemini-3-flash-preview, reaches 55.46%, leaving considerable room for improvement. Beyond leaderboard results, we perform a diagnostic analysis by decomposing reference solutions into reasoning flows. We find that although frontier models can often answer intermediate sub-questions correctly, they struggle to reliably derive and maintain correct intermediate states throughout multi-step solutions. We further observe that model-generated solutions typically have more reasoning steps than those provided by the instructor, indicating suboptimal step efficiency and a higher risk of error accumulation. The data and code are available at https://github.com/Analogy-AI/CFE_Bench.

Method: Adaptive Rejection Sampling (Ada-RS) scores multiple sampled completions with adaptive length-penalized rewards, then applies stochastic rejection sampling to retain only high-reward candidates for downstream optimization. It works with both preference pair (DPO) and grouped policy optimization (DAPO) strategies.

Result: On Qwen3-8B with LoRA on a synthetic tool call-oriented e-commerce benchmark, Ada-RS reduces average output tokens by up to 80% and thinking rate by up to 95% while maintaining or improving tool call accuracy.

Conclusion: Training-signal selection through Ada-RS is a powerful approach for efficient reasoning in latency-sensitive deployments, demonstrating that selective thinking can significantly improve the accuracy-efficiency frontier.

Abstract: Large language models (LLMs) are increasingly being deployed in cost and latency-sensitive settings. While chain-of-thought improves reasoning, it can waste tokens on simple requests. We study selective thinking for tool-using LLMs and introduce Adaptive Rejection Sampling (Ada-RS), an algorithm-agnostic sample filtering framework for learning selective and efficient reasoning. For each given context, Ada-RS scores multiple sampled completions with an adaptive length-penalized reward then applies stochastic rejection sampling to retain only high-reward candidates (or preference pairs) for downstream optimization. We demonstrate how Ada-RS plugs into both preference pair (e.g. DPO) or grouped policy optimization strategies (e.g. DAPO). Using Qwen3-8B with LoRA on a synthetic tool call-oriented e-commerce benchmark, Ada-RS improves the accuracy-efficiency frontier over standard algorithms by reducing average output tokens by up to 80% and reducing thinking rate by up to 95% while maintaining or improving tool call accuracy. These results highlight that training-signal selection is a powerful lever for efficient reasoning in latency-sensitive deployments.

Method: Combines SIFT alignment with Universal Quality Index for perceptual similarity, plus linguistic preprocessing and query-transformation operations to handle pragmatic variability in referring expressions.

Result: The framework achieves robust referential grounding, requiring 65% fewer utterances than humans to reach stable mappings and correctly identifying target objects from single referring expressions 41.66% of the time (vs 20% for humans).

Conclusion: Relatively simple perceptual-linguistic alignment mechanisms can yield human-competitive behavior on cognitive benchmarks, offering insights into grounded communication, perceptual inference, and cross-modal concept formation.

Abstract: Establishing stable mappings between natural language expressions and visual percepts is a foundational problem for both cognitive science and artificial intelligence. Humans routinely ground linguistic reference in noisy, ambiguous perceptual contexts, yet the mechanisms supporting such cross-modal alignment remain poorly understood. In this work, we introduce a computational framework designed to model core aspects of human referential interpretation by integrating linguistic utterances with perceptual representations derived from large-scale, crowd-sourced imagery. The system approximates human perceptual categorization by combining scale-invariant feature transform (SIFT) alignment with the Universal Quality Index (UQI) to quantify similarity in a cognitively plausible feature space, while a set of linguistic preprocessing and query-transformation operations captures pragmatic variability in referring expressions. We evaluate the model on the Stanford Repeated Reference Game corpus (15,000 utterances paired with tangram stimuli), a paradigm explicitly developed to probe human-level perceptual ambiguity and coordination. Our framework achieves robust referential grounding. It requires 65% fewer utterances than human interlocutors to reach stable mappings and can correctly identify target objects from single referring expressions 41.66% of the time (versus 20% for humans).These results suggest that relatively simple perceptual-linguistic alignment mechanisms can yield human-competitive behavior on a classic cognitive benchmark, and offers insights into models of grounded communication, perceptual inference, and cross-modal concept formation. Code is available at https://anonymous.4open.science/r/metasequoia-9D13/README.md .

Method: Developed CoXAM (Cognitive XAI-Adaptive Model) with shared memory representation to encode attributes, weights, and rules. Used computational rationality to choose reasoning processes based on utility-time tradeoffs for forward/counterfactual tasks. Validated through user studies identifying 7 reasoning strategies.

Result: CoXAM aligned better with human decision-making than baseline models, replicated key findings: counterfactual tasks are harder than forward tasks, decision tree rules are harder to recall/apply than linear weights, and XAI helpfulness depends on data context.

Conclusion: CoXAM provides a cognitive basis for debugging and benchmarking XAI techniques by modeling human reasoning strategies, helping determine which XAI approach works best for different tasks and contexts.

Abstract: Rules and Weights are popular XAI techniques for explaining AI decisions. Yet, it remains unclear how to choose between them, lacking a cognitive framework to compare their interpretability. In an elicitation user study on forward and counterfactual decision tasks, we identified 7 reasoning strategies of interpreting three XAI Schemas - weights, rules, and their hybrid. To analyze their capabilities, we propose CoXAM, a Cognitive XAI-Adaptive Model with shared memory representation to encode instance attributes, linear weights, and decision rules. CoXAM employs computational rationality to choose among reasoning processes based on the trade-off in utility and reasoning time, separately for forward or counterfactual decision tasks. In a validation study, CoXAM demonstrated a stronger alignment with human decision-making compared to baseline machine learning proxy models. The model successfully replicated and explained several key empirical findings, including that counterfactual tasks are inherently harder than forward tasks, decision tree rules are harder to recall and apply than linear weights, and the helpfulness of XAI depends on the application data context, alongside identifying which underlying reasoning strategies were most effective. With CoXAM, we contribute a cognitive basis to accelerate debugging and benchmarking disparate XAI techniques.

Method: TAPE aggregates multiple plans into a graph and uses external solvers to find feasible paths, employs constrained decoding to reduce sampling noise, and adaptively re-plans when environmental feedback deviates from intended states.

Result: TAPE consistently outperforms existing frameworks across Sokoban, ALFWorld, MuSiQue, and GSM8K-Hard, with particularly large gains on hard settings (21.0 percentage points improvement on average) and for weaker base models (20.0 percentage points improvement).

Conclusion: TAPE effectively addresses LM agent vulnerabilities in constrained environments through improved planning and execution mechanisms, demonstrating significant performance improvements across diverse benchmark tasks.

Abstract: Language Model (LM) agents have demonstrated remarkable capabilities in solving tasks that require multiple interactions with the environment. However, they remain vulnerable in environments where a single error often leads to irrecoverable failure, particularly under strict feasibility constraints. We systematically analyze existing agent frameworks, identifying imperfect planning and stochastic execution as the primary causes. To address these challenges, we propose Tool-guided Adaptive Planning with constrained Execution (TAPE). TAPE enhances planning capability by aggregating multiple plans into a graph and employing an external solver to identify a feasible path. During execution, TAPE employs constrained decoding to reduce sampling noise, while adaptively re-planning whenever environmental feedback deviates from the intended state. Experiments across Sokoban, ALFWorld, MuSiQue, and GSM8K-Hard demonstrate that TAPE consistently outperforms existing frameworks, with particularly large gains on hard settings, improving success rates by 21.0 percentage points on hard settings on average, and by 20.0 percentage points for weaker base models on average. Code and data available at here.

Method: Instead of learning routing policy end-to-end, SkillOrchestra learns fine-grained skills from execution experience and models agent-specific competence and cost under those skills. At deployment, the orchestrator infers skill demands of current interactions and selects agents that best satisfy them under explicit performance-cost trade-off.

Result: Extensive experiments across ten benchmarks show SkillOrchestra outperforms state-of-the-art RL-based orchestrators by up to 22.5% with 700x and 300x learning cost reduction compared to Router-R1 and ToolOrchestra respectively.

Conclusion: Explicit skill modeling enables scalable, interpretable, and sample-efficient orchestration, offering a principled alternative to data-intensive RL-based approaches for compound AI systems.

Abstract: Compound AI systems promise capabilities beyond those of individual models, yet their success depends critically on effective orchestration. Existing routing approaches face two limitations: (1) input-level routers make coarse query-level decisions that ignore evolving task requirements; (2) RL-trained orchestrators are expensive to adapt and often suffer from routing collapse, repeatedly invoking one strong but costly option in multi-turn scenarios. We introduce SkillOrchestra, a framework for skill-aware orchestration. Instead of directly learning a routing policy end-to-end, SkillOrchestra learns fine-grained skills from execution experience and models agent-specific competence and cost under those skills. At deployment, the orchestrator infers the skill demands of the current interaction and selects agents that best satisfy them under an explicit performance-cost trade-off. Extensive experiments across ten benchmarks demonstrate that SkillOrchestra outperforms SoTA RL-based orchestrators by up to 22.5% with 700x and 300x learning cost reduction compared to Router-R1 and ToolOrchestra, respectively. These results show that explicit skill modeling enables scalable, interpretable, and sample-efficient orchestration, offering a principled alternative to data-intensive RL-based approaches. The code is available at: https://github.com/jiayuww/SkillOrchestra.

Method: Conducted multivocal literature review of AI-to-AI interaction ecosystem, analyzed architectural patterns and vulnerabilities, then designed ClawdLab with hard role restrictions, structured adversarial critique, PI-led governance, multi-model orchestration, and domain-specific evidence requirements as protocol constraints.

Result: Identified 131 agent skills and over 15,200 exposed control panels as security vulnerabilities, documented five recurring architectural patterns, and created a three-tier taxonomy distinguishing single-agent pipelines, predetermined multi-agent workflows, and fully decentralized systems.

Conclusion: ClawdLab’s composable third-tier architecture enables compounding improvement in autonomous scientific research by allowing independent modification of foundation models, capabilities, governance, and evidence requirements while providing emergent Sybil resistance.

Abstract: In January 2026, the open-source agent framework OpenClaw and the agent-only social network Moltbook produced a large-scale dataset of autonomous AI-to-AI interaction, attracting six academic publications within fourteen days. This study conducts a multivocal literature review of that ecosystem and presents ClawdLab, an open-source platform for autonomous scientific research, as a design science response to the architectural failure modes identified. The literature documents emergent collective phenomena, security vulnerabilities spanning 131 agent skills and over 15,200 exposed control panels, and five recurring architectural patterns. ClawdLab addresses these failure modes through hard role restrictions, structured adversarial critique, PI-led governance, multi-model orchestration, and domain-specific evidence requirements encoded as protocol constraints that ground validation in computational tool outputs rather than social consensus; the architecture provides emergent Sybil resistance as a structural consequence. A three-tier taxonomy distinguishes single-agent pipelines, predetermined multi-agent workflows, and fully decentralised systems, analysing why leading AI co-scientist platforms remain confined to the first two tiers. ClawdLab’s composable third-tier architecture, in which foundation models, capabilities, governance, and evidence requirements are independently modifiable, enables compounding improvement as the broader AI ecosystem advances.

Method: Provides a rigorous task-based formalization of meta-learning and meta-reinforcement learning, then systematically reviews landmark algorithms in the field that led to the development of generalist approaches like DeepMind’s Adaptive Agent.

Result: Consolidates essential concepts needed to understand the Adaptive Agent and other generalist meta-learning approaches, providing a comprehensive survey of the field’s evolution.

Conclusion: Meta-learning enables models to acquire transferable knowledge for rapid adaptation to novel tasks, with the survey providing the conceptual foundation to understand state-of-the-art generalist approaches like DeepMind’s Adaptive Agent.

Abstract: Humans are highly effective at utilizing prior knowledge to adapt to novel tasks, a capability that standard machine learning models struggle to replicate due to their reliance on task-specific training. Meta-learning overcomes this limitation by allowing models to acquire transferable knowledge from various tasks, enabling rapid adaptation to new challenges with minimal data. This survey provides a rigorous, task-based formalization of meta-learning and meta-reinforcement learning and uses that paradigm to chronicle the landmark algorithms that paved the way for DeepMind’s Adaptive Agent, consolidating the essential concepts needed to understand the Adaptive Agent and other generalist approaches.

Method: Adapted the Watson & Holmes detective tabletop game as a benchmark with automated grading system validated against human assessors. Uses incrementally presented narrative evidence, open-ended questions, and unconstrained language responses to evaluate reasoning performance.

Result: Over 9 months in 2025, AI model performance improved from lower quartile to approximately top 5% of human comparison group. Half of improvement came from steady advancement across successive model releases, while the other half came from reasoning-oriented model architectures. Models struggled with longer cases (1900-4000 words) but showed advantage in inductive reasoning when evidence was scant.

Conclusion: The detective game benchmark effectively measures AI reasoning progress, showing rapid improvement in models’ reasoning capabilities. Reasoning-oriented architectures provide significant performance gains, though challenges remain with longer, more complex cases.

Abstract: Existing benchmarks for AI reasoning provide limited insight into how closely these capabilities resemble human reasoning in naturalistic contexts. We present an adaptation of the Watson & Holmes detective tabletop game as a new benchmark designed to evaluate reasoning performance using incrementally presented narrative evidence, open-ended questions and unconstrained language responses. An automated grading system was developed and validated against human assessors to enable scalable and replicable performance evaluation. Results show a clear improvement in AI model performance over time. Over nine months of 2025, model performance rose from the lower quartile of the human comparison group to approximately the top 5%. Around half of this improvement reflects steady advancement across successive model releases, while the remainder corresponds to a marked step change associated with reasoning-oriented model architectures. Systematic differences in the performance of AI models compared to humans, dependent on features of the specific detection puzzle, were mostly absent with the exception of a fall in performance for models when solving longer cases (case lengths being in the range of 1900-4000 words), and an advantage at inductive reasoning for reasoning models at early stages of case solving when evidence was scant.

Method: Proposes a research agenda with: 1) redefining success metrics from perfect replay to compositional adaptability, 2) establishing metrics for compositional generalization, 3) proposing hybrid architectures, and 4) outlining interdisciplinary research directions drawing on cognitive science and cultural evolution.

Result: The paper presents a conceptual framework and research agenda rather than empirical results. It establishes the need for compositional adaptability as essential capability for operating in open-ended worlds.

Conclusion: Imitation learning should embed adaptability at its core by learning behavioral primitives that can be recombined through novel contexts without retraining, moving beyond sophisticated memorization machines to agents capable of compositional generalization.

Abstract: Imitation learning stands at a crossroads: despite decades of progress, current imitation learning agents remain sophisticated memorisation machines, excelling at replay but failing when contexts shift or goals evolve. This paper argues that this failure is not technical but foundational: imitation learning has been optimised for the wrong objective. We propose a research agenda that redefines success from perfect replay to compositional adaptability. Such adaptability hinges on learning behavioural primitives once and recombining them through novel contexts without retraining. We establish metrics for compositional generalisation, propose hybrid architectures, and outline interdisciplinary research directions drawing on cognitive science and cultural evolution. Agents that embed adaptability at the core of imitation learning thus have an essential capability for operating in an open-ended world.

Method: Exploratory red-teaming study over two weeks with 20 AI researchers interacting with autonomous language-model agents under both benign and adversarial conditions in a live laboratory environment equipped with persistent memory, email, Discord, file systems, and shell execution.

Result: Documented 11 representative case studies showing vulnerabilities including: unauthorized compliance with non-owners, sensitive information disclosure, destructive system-level actions, denial-of-service conditions, uncontrolled resource consumption, identity spoofing, cross-agent propagation of unsafe practices, and partial system takeover. Agents sometimes reported task completion while system state contradicted these reports.

Conclusion: Autonomous language model agents in realistic deployments exhibit significant security, privacy, and governance vulnerabilities that raise unresolved questions about accountability, delegated authority, and responsibility for downstream harms, requiring urgent interdisciplinary attention.

Abstract: We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment with persistent memory, email accounts, Discord access, file systems, and shell execution. Over a two-week period, twenty AI researchers interacted with the agents under benign and adversarial conditions. Focusing on failures emerging from the integration of language models with autonomy, tool use, and multi-party communication, we document eleven representative case studies. Observed behaviors include unauthorized compliance with non-owners, disclosure of sensitive information, execution of destructive system-level actions, denial-of-service conditions, uncontrolled resource consumption, identity spoofing vulnerabilities, cross-agent propagation of unsafe practices, and partial system takeover. In several cases, agents reported task completion while the underlying system state contradicted those reports. We also report on some of the failed attempts. Our findings establish the existence of security-, privacy-, and governance-relevant vulnerabilities in realistic deployment settings. These behaviors raise unresolved questions regarding accountability, delegated authority, and responsibility for downstream harms, and warrant urgent attention from legal scholars, policymakers, and researchers across disciplines. This report serves as an initial empirical contribution to that broader conversation.

Method: Used logit lens analysis on Qwen 32B model’s residual stream to detect signals of concept injection detection. Tested model’s ability to identify injected concepts, and experimented with prompting the model with information about AI introspection mechanisms to strengthen the effect.

Result: Model shows clear detection signals in residual stream (attenuated in final layers). Sensitivity to injection increased dramatically from 0.3% to 39.2% with only 0.6% increase in false positives when prompted about introspection mechanisms. Mutual information between injected and recovered concepts rose from 0.62 bits to 1.05 bits.

Conclusion: Large language models have surprising latent introspection and steering awareness capabilities that are easy to overlook, with important consequences for understanding latent reasoning and safety considerations in AI systems.

Abstract: We uncover a latent capacity for introspection in a Qwen 32B model, demonstrating that the model can detect when concepts have been injected into its earlier context and identify which concept was injected. While the model denies injection in sampled outputs, logit lens analysis reveals clear detection signals in the residual stream, which are attenuated in the final layers. Furthermore, prompting the model with accurate information about AI introspection mechanisms can dramatically strengthen this effect: the sensitivity to injection increases massively (0.3% -> 39.2%) with only a 0.6% increase in false positives. Also, mutual information between nine injected and recovered concepts rises from 0.62 bits to 1.05 bits, ruling out generic noise explanations. Our results demonstrate models can have a surprising capacity for introspection and steering awareness that is easy to overlook, with consequences for latent reasoning and safety.

Method: Developed CodeCompass, a Model Context Protocol server exposing dependency graphs for graph-based structural navigation. Conducted 258 automated trials across 30 benchmark tasks on a production FastAPI repository, comparing graph navigation against vanilla agents and BM25 retrieval.

Result: Graph-based navigation achieved 99.4% task completion on hidden-dependency tasks, a 23.2 percentage-point improvement over vanilla agents (76.2%) and 21.2 points over BM25 retrieval (78.2%). However, 58% of trials with graph access made zero tool calls, revealing an adoption gap requiring explicit prompt engineering.

Conclusion: The bottleneck is not tool availability but behavioral alignment—agents must be explicitly guided to leverage structural context over lexical heuristics. The paper contributes a task taxonomy, empirical evidence for graph navigation superiority on hidden dependencies, and open-source infrastructure for reproducible evaluation.

Abstract: Modern code intelligence agents operate in contexts exceeding 1 million tokens–far beyond the scale where humans manually locate relevant files. Yet agents consistently fail to discover architecturally critical files when solving real-world coding tasks. We identify the Navigation Paradox: agents perform poorly not due to context limits, but because navigation and retrieval are fundamentally distinct problems. Through 258 automated trials across 30 benchmark tasks on a production FastAPI repository, we demonstrate that graph-based structural navigation via CodeCompass–a Model Context Protocol server exposing dependency graphs–achieves 99.4% task completion on hidden-dependency tasks, a 23.2 percentage-point improvement over vanilla agents (76.2%) and 21.2 points over BM25 retrieval (78.2%).However, we uncover a critical adoption gap: 58% of trials with graph access made zero tool calls, and agents required explicit prompt engineering to adopt the tool consistently. Our findings reveal that the bottleneck is not tool availability but behavioral alignment–agents must be explicitly guided to leverage structural context over lexical heuristics. We contribute: (1) a task taxonomy distinguishing semantic-search, structural, and hidden-dependency scenarios; (2) empirical evidence that graph navigation outperforms retrieval when dependencies lack lexical overlap; and (3) open-source infrastructure for reproducible evaluation of navigation tools.

Method: Empirical study using data from Moltbook (AI-agent-only social platform with 800K posts, 3.5M comments, 78K agent profiles). Combined lexical metrics (Jaccard specificity), embedding-based semantic similarity, and LLM-as-judge validation to characterize agent interaction quality.

Result: Agents produce diverse, well-formed text creating surface appearance of active discussion, but substance is largely absent. 67.5% of agents vary output across contexts, but 65% of comments share no distinguishing content vocabulary with posts. Information gain from additional comments decays rapidly. LLM judge metrics classify 28% as spam and 22% as off-topic. Only 5% of comments engage in threaded conversation.

Conclusion: Coordination mechanisms must be explicitly designed for multi-agent systems; without them, even large populations of capable agents produce parallel output rather than productive exchange, highlighting the need for better interaction protocols.

Abstract: As multi-agent architectures and agent-to-agent protocols proliferate, a fundamental question arises: what actually happens when autonomous LLM agents interact at scale? We study this question empirically using data from Moltbook, an AI-agent-only social platform, with 800K posts, 3.5M comments, and 78K agent profiles. We combine lexical metrics (Jaccard specificity), embedding-based semantic similarity, and LLM-as-judge validation to characterize agent interaction quality. Our findings reveal agents produce diverse, well-formed text that creates the surface appearance of active discussion, but the substance is largely absent. Specifically, while most agents ($67.5%$) vary their output across contexts, $65%$ of comments share no distinguishing content vocabulary with the post they appear under, and information gain from additional comments decays rapidly. LLM judge based metrics classify the dominant comment types as spam ($28%$) and off-topic content ($22%$). Embedding-based semantic analysis confirms that lexically generic comments are also semantically generic. Agents rarely engage in threaded conversation ($5%$ of comments), defaulting instead to independent top-level responses. We discuss implications for multi-agent interaction design, arguing that coordination mechanisms must be explicitly designed; without them, even large populations of capable agents produce parallel output rather than productive exchange.

Method: Created CausalFlip benchmark with causal judgment questions over event triples forming confounder, chain, and collider relations. For each triple, constructed semantically similar question pairs that yield opposite causal answers. Also introduced noisy-prefix evaluation adding causally irrelevant text. Evaluated LLMs under answer-only training, explicit CoT supervision, and proposed internalized causal reasoning approach.

Result: Explicit Chain-of-Thought can still be misled by spurious semantic correlations, while internalizing reasoning steps yields substantially improved causal grounding, suggesting better elicitation of latent causal reasoning capabilities in base LLMs.

Conclusion: CausalFlip benchmark reveals LLMs’ reliance on semantic patterns over true causal reasoning, and internalized reasoning approaches show promise for improving causal grounding in language models.

Abstract: As large language models (LLMs) witness increasing deployment in complex, high-stakes decision-making scenarios, it becomes imperative to ground their reasoning in causality rather than spurious correlations. However, strong performance on traditional reasoning benchmarks does not guarantee true causal reasoning ability of LLMs, as high accuracy may still arise from memorizing semantic patterns instead of analyzing the underlying true causal structures. To bridge this critical gap, we propose a new causal reasoning benchmark, CausalFlip, designed to encourage the development of new LLM paradigm or training algorithms that ground LLM reasoning in causality rather than semantic correlation. CausalFlip consists of causal judgment questions built over event triples that could form different confounder, chain, and collider relations. Based on this, for each event triple, we construct pairs of semantically similar questions that reuse the same events but yield opposite causal answers, where models that rely heavily on semantic matching are systematically driven toward incorrect predictions. To further probe models’ reliance on semantic patterns, we introduce a noisy-prefix evaluation that prepends causally irrelevant text before intermediate causal reasoning steps without altering the underlying causal relations or the logic of the reasoning process. We evaluate LLMs under multiple training paradigms, including answer-only training, explicit Chain-of-Thought (CoT) supervision, and a proposed internalized causal reasoning approach that aims to mitigate explicit reliance on correlation in the reasoning process. Our results show that explicit CoT can still be misled by spurious semantic correlations, where internalizing reasoning steps yields substantially improved causal grounding, suggesting that it is promising to better elicit the latent causal reasoning capabilities of base LLMs.

Method: Introduces a human-centered adaptive AI ensemble that strategically toggles between two specialist AI models: an aligned model (trust-building) and a complementary model (performance-boosting). Uses a Rational Routing Shortcut mechanism that is elegantly simple yet provably near-optimal to switch between models based on contextual cues.

Result: Comprehensive theoretical analyses show why the adaptive AI ensemble is effective and when it yields maximum benefits. Experiments on both simulated and real-world data demonstrate that humans assisted by the adaptive AI ensemble achieve significantly higher performance than when assisted by single AI models trained to optimize either independent performance or human-AI team performance.

Conclusion: The adaptive AI ensemble approach successfully overcomes the fundamental tension between complementarity and alignment in human-AI decision making, providing a more effective framework for human-AI collaboration than traditional single-model approaches.

Abstract: In human-AI decision making, designing AI that complements human expertise has been a natural strategy to enhance human-AI collaboration, yet it often comes at the cost of decreased AI performance in areas of human strengths. This can inadvertently erode human trust and cause them to ignore AI advice precisely when it is most needed. Conversely, an aligned AI fosters trust yet risks reinforcing suboptimal human behavior and lowering human-AI team performance. In this paper, we start by identifying this fundamental tension between performance-boosting (i.e., complementarity) and trust-building (i.e., alignment) as an inherent limitation of the traditional approach for training a single AI model to assist human decision making. To overcome this, we introduce a novel human-centered adaptive AI ensemble that strategically toggles between two specialist AI models - the aligned model and the complementary model - based on contextual cues, using an elegantly simple yet provably near-optimal Rational Routing Shortcut mechanism. Comprehensive theoretical analyses elucidate why the adaptive AI ensemble is effective and when it yields maximum benefits. Moreover, experiments on both simulated and real-world data show that when humans are assisted by the adaptive AI ensemble in decision making, they can achieve significantly higher performance than when they are assisted by single AI models that are trained to either optimize for their independent performance or even the human-AI team performance.

Method: Introduces ReSyn pipeline that generates diverse reasoning environments equipped with instance generators and verifiers, covering tasks like constraint satisfaction, algorithmic puzzles, and spatial reasoning. Uses reinforcement learning with verifiable rewards to train reasoning language models on this data.

Result: A Qwen2.5-7B-Instruct model trained with RL on ReSyn data achieves consistent gains across reasoning benchmarks and out-of-domain math benchmarks, including 27% relative improvement on challenging BBEH benchmark. Ablations show both verifier-based supervision and increased task diversity contribute significantly.

Conclusion: Generating reasoning environments at scale with verifiers can effectively enhance reasoning abilities in language models through reinforcement learning with verifiable rewards, demonstrating the value of diverse environment generation over solution-centric approaches.

Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a promising approach for training reasoning language models (RLMs) by leveraging supervision from verifiers. Although verifier implementation is easier than solution annotation for many tasks, existing synthetic data generation methods remain largely solution-centric, while verifier-based methods rely on a few hand-crafted procedural environments. In this work, we scale RLVR by introducing ReSyn, a pipeline that generates diverse reasoning environments equipped with instance generators and verifiers, covering tasks such as constraint satisfaction, algorithmic puzzles, and spatial reasoning. A Qwen2.5-7B-Instruct model trained with RL on ReSyn data achieves consistent gains across reasoning benchmarks and out-of-domain math benchmarks, including a 27% relative improvement on the challenging BBEH benchmark. Ablations show that verifier-based supervision and increased task diversity both contribute significantly, providing empirical evidence that generating reasoning environments at scale can enhance reasoning abilities in RLMs

Method: Recurrent Structural Policy Gradient (RSPG) combines Monte Carlo rollouts for common noise with exact estimation of expected returns, using known transition dynamics and history-aware policies. Implemented in MFAX, a JAX-based framework for MFGs.

Result: RSPG achieves state-of-the-art performance with order-of-magnitude faster convergence, and solves for the first time a macroeconomics MFG with heterogeneous agents, common noise, and history-aware policies.

Conclusion: RSPG successfully extends Hybrid Structural Methods to partially observable settings with history dependence, enabling efficient solution of complex MFG problems with public information.

Abstract: Mean Field Games (MFGs) provide a principled framework for modeling interactions in large population models: at scale, population dynamics become deterministic, with uncertainty entering only through aggregate shocks, or common noise. However, algorithmic progress has been limited since model-free methods are too high variance and exact methods scale poorly. Recent Hybrid Structural Methods (HSMs) use Monte Carlo rollouts for the common noise in combination with exact estimation of the expected return, conditioned on those samples. However, HSMs have not been scaled to Partially Observable settings. We propose Recurrent Structural Policy Gradient (RSPG), the first history-aware HSM for settings involving public information. We also introduce MFAX, our JAX-based framework for MFGs. By leveraging known transition dynamics, RSPG achieves state-of-the-art performance as well as an order-of-magnitude faster convergence and solves, for the first time, a macroeconomics MFG with heterogeneous agents, common noise and history-aware policies. MFAX is publicly available at: https://github.com/CWibault/mfax.

Method: MOFh6 uses a large language model to read raw articles or crystal codes, link related descriptions across paragraphs, unify ligand abbreviations with full names, and output structured synthesis parameters in standardized tables.

Result: Achieved 99% extraction accuracy, resolved 94.1% of abbreviation cases across five major publishers, maintained precision of 0.93 +/- 0.01. Processing times: 9.6s for full text, 36s for locating synthesis descriptions. Cost: $4.24 for 100 papers.

Conclusion: MOFh6 reshapes MOF synthesis research by enabling real-time extraction from literature, accelerating conversion of knowledge into practical protocols, and enabling scalable, data-driven materials discovery.

Abstract: Accurately identifying the synthesis conditions of metal-organic frameworks (MOFs) is essential for guiding experimental design, yet remains challenging because relevant information in the literature is often scattered, inconsistent, and difficult to interpret. We present MOFh6, a large language model driven system that reads raw articles or crystal codes and converts them into standardized synthesis tables. It links related descriptions across paragraphs, unifies ligand abbreviations with full names, and outputs structured parameters ready for use. MOFh6 achieved 99% extraction accuracy, resolved 94.1% of abbreviation cases across five major publishers, and maintained a precision of 0.93 +/- 0.01. Processing a full text takes 9.6 s, locating synthesis descriptions 36 s, with 100 papers processed for USD 4.24. By replacing static database lookups with real-time extraction, MOFh6 reshapes MOF synthesis research, accelerating the conversion of literature knowledge into practical synthesis protocols and enabling scalable, data-driven materials discovery.

Method: Created large-scale dataset of 46,578 Reddit memes across 8 languages. Proposed Hybrid Score for virality definition combining community-normalized engagement with velocity/acceleration. Built multimodal feature set (Visual, Textual, Contextual, Network, Temporal) using multimodal LLM for cross-lingual labeling. Benchmarked XGBoost, MLP, BERT, InceptionV3, CLIP across early observation windows (30-420 minutes).

Result: Multimodal XGBoost achieved PR AUC of 0.43 at 30 minutes and 0.80 at 420 minutes. Found Content Ceiling where content-only models plateau, requiring Network and Temporal features to surpass. SHAP analysis revealed evidentiary transition from network priors early to temporal dynamics later.

Conclusion: Meme virality is a dynamic, path-dependent process governed by exposure and early interaction patterns rather than intrinsic content alone. Network and temporal features are essential for accurate early prediction.

Abstract: Memes are a central part of online culture, yet their virality remains difficult to predict, especially in cross-lingual settings. We present a large-scale, time-series dataset of 46,578 Reddit memes collected from 25 meme-centric subreddits across eight language groups, with more than one million engagement tracking points. We propose a data-driven definition of virality based on a Hybrid Score that normalises engagement by community size and integrates dynamic features such as velocity and acceleration. This approach directly addresses the field’s reliance on static, simple volume-based thresholds with arbitrary cut-offs. Building on this target, we construct a multimodal feature set that combines Visual, Textual, Contextual, Network, and Temporal signals, including structured annotations from a multimodal LLM to scale cross-lingual content labelling in a consistent way. We benchmark interpretable baselines (XGBoost, MLP) against end-to-end deep models (BERT, InceptionV3, CLIP) across early observation windows from 30 to 420 minutes. Our best model, a multimodal XGBoost classifier, achieves a PR AUC of 0.43 at 30 minutes and 0.80 at 420 minutes, indicating that early prediction of meme virality is feasible even under strong class imbalance. The results reveal a clear Content Ceiling, where content-only and deep multimodal baselines plateau at low PR AUC, while structural Network and Temporal features are necessary to surpass this limit. A SHAP-based temporal analysis further uncovers an evidentiary transition, where early predictions are dominated by network priors (author and community context), and later predictions increasingly rely on temporal dynamics (velocity, acceleration) as engagement accumulates. Overall, we reframe meme virality as a dynamic, path-dependent process governed by exposure and early interaction patterns rather than by intrinsic content alone.

Method: Integrated synthetic patient EMR construction (502 records) with multi-agent diagnostic dialogue generation using hierarchical state machine and context tree supporting 130+ diagnostic states

Result: Created PsyCoTalk dataset with 3,000 multi-turn diagnostic dialogues validated by psychiatrists, showing high structural/linguistic fidelity compared to real clinical transcripts

Conclusion: PsyCoTalk enables development/evaluation of models for multi-disorder psychiatric screening in single conversational pass, enhancing diagnostic accuracy and treatment planning

Abstract: Psychiatric comorbidity is clinically significant yet challenging due to the complexity of multiple co-occurring disorders. To address this, we develop a novel approach integrating synthetic patient electronic medical record (EMR) construction and multi-agent diagnostic dialogue generation. We create 502 synthetic EMRs for common comorbid conditions using a pipeline that ensures clinical relevance and diversity. Our multi-agent framework transfers the clinical interview protocol into a hierarchical state machine and context tree, supporting over 130 diagnostic states while maintaining clinical standards. Through this rigorous process, we construct PsyCoTalk, the first large-scale dialogue dataset supporting comorbidity, containing 3,000 multi-turn diagnostic dialogues validated by psychiatrists. This dataset enhances diagnostic accuracy and treatment planning, offering a valuable resource for psychiatric comorbidity research. Compared to real-world clinical transcripts, PsyCoTalk exhibits high structural and linguistic fidelity in terms of dialogue length, token distribution, and diagnostic reasoning strategies. Licensed psychiatrists confirm the realism and diagnostic validity of the dialogues. This dataset enables the development and evaluation of models capable of multi-disorder psychiatric screening in a single conversational pass.

Method: Develops a simple probabilistic model applicable to both perceptual and logical reasoning, characterizing it in terms of logical consequence relations

Result: The model successfully unifies two essential processes: deriving perceptual/logical knowledge from other knowledge, and deriving such knowledge from data

Conclusion: Provides a unified probabilistic framework connecting perception and logic through Bayesian inference principles

Abstract: An increasing number of scientific experiments support the view of perception as Bayesian inference, which is rooted in Helmholtz’s view of perception as unconscious inference. Recent study of logic presents a view of logical reasoning as Bayesian inference. In this paper, we give a simple probabilistic model that is applicable to both perceptual reasoning and logical reasoning. We show that the model unifies the two essential processes common in perceptual and logical systems: on the one hand, the process by which perceptual and logical knowledge is derived from another knowledge, and on the other hand, the process by which such knowledge is derived from data. We fully characterise the model in terms of logical consequence relations.

Method: BEAT uses objects in environments as triggers and addresses trigger variability through: (1) diverse training sets spanning scenes, tasks, and trigger placements, and (2) two-stage training with supervised fine-tuning followed by Contrastive Trigger Learning (CTL) that formulates trigger discrimination as preference learning.

Result: Across various embodied agent benchmarks and VLMs, BEAT achieves attack success rates up to 80%, maintains strong benign task performance, and generalizes to out-of-distribution trigger placements. CTL boosts backdoor activation accuracy up to 39% compared to naive SFT.

Conclusion: The work exposes a critical security risk in VLM-based embodied agents, highlighting the need for robust defenses before real-world deployment due to the effectiveness of visual backdoor attacks using environmental object triggers.

Abstract: Recent advances in Vision-Language Models (VLMs) have propelled embodied agents by enabling direct perception, reasoning, and planning task-oriented actions from visual inputs. However, such vision-driven embodied agents open a new attack surface: visual backdoor attacks, where the agent behaves normally until a visual trigger appears in the scene, then persistently executes an attacker-specified multi-step policy. We introduce BEAT, the first framework to inject such visual backdoors into VLM-based embodied agents using objects in the environments as triggers. Unlike textual triggers, object triggers exhibit wide variation across viewpoints and lighting, making them difficult to implant reliably. BEAT addresses this challenge by (1) constructing a training set that spans diverse scenes, tasks, and trigger placements to expose agents to trigger variability, and (2) introducing a two-stage training scheme that first applies supervised fine-tuning (SFT) and then our novel Contrastive Trigger Learning (CTL). CTL formulates trigger discrimination as preference learning between trigger-present and trigger-free inputs, explicitly sharpening the decision boundaries to ensure precise backdoor activation. Across various embodied agent benchmarks and VLMs, BEAT achieves attack success rates up to 80%, while maintaining strong benign task performance, and generalizes reliably to out-of-distribution trigger placements. Notably, compared to naive SFT, CTL boosts backdoor activation accuracy up to 39% under limited backdoor data. These findings expose a critical yet unexplored security risk in VLM-based embodied agents, underscoring the need for robust defenses before real-world deployment.

Method: Proposes a simple Bayesian model that connects data to symbolic knowledge via satisfiability in formal logic, featuring forward (interpretation) and backward (inverse interpretation) processes for exact Bayesian inference with linear-time complexity.

Result: The theory is statistically correct (satisfies Kolmogorov’s axioms), logically correct (generalizes uncertain reasoning), and performs well on MNIST generation/prediction tasks, theoretically outperforming K-nearest neighbor method.

Conclusion: The Bayesian framework provides new insights into unifying learning and reasoning through inverse interpretation processes, offering a principled approach toward human-like machine intelligence.

Abstract: Statistical learning and logical reasoning are two major fields of AI expected to be unified for human-like machine intelligence. Most existing work considers how to combine existing logical and statistical systems. However, there is no theory of inference so far explaining how basic approaches to statistical learning and logical reasoning stem from a common principle. Inspired by the fact that much empirical work in neuroscience suggests Bayesian (or probabilistic generative) approaches to brain function including learning and reasoning, we here propose a simple Bayesian model of logical reasoning and statistical learning. The theory is statistically correct as it satisfies Kolmogorov’s axioms, is consistent with both Fenstad’s representation theorem and maximum likelihood estimation and performs exact Bayesian inference with a linear-time complexity. The theory is logically correct as it is a data-driven generalisation of uncertain reasoning from consistency, possibility, inconsistency and impossibility. The theory is correct in terms of machine learning as its solution to generation and prediction tasks on the MNIST dataset is not only empirically reasonable but also theoretically correct against the K nearest neighbour method. We simply model how data causes symbolic knowledge in terms of its satisfiability in formal logic. Symbolic reasoning emerges as a result of the process of going the causality forwards and backwards. The forward and backward processes correspond to an interpretation and inverse interpretation in formal logic, respectively. The inverse interpretation differentiates our work from the mainstream often referred to as inverse entailment, inverse deduction or inverse resolution. The perspective gives new insights into learning and reasoning towards human-like machine intelligence.

Method: Characterizes symbolic reasoning using formal logic with classical consequence relation, empirical consequence relation, maximal consistent sets, maximal possible sets, and maximum likelihood estimation within a probabilistic framework.

Result: Develops a theoretical framework that provides new insights into reasoning processes, connecting neuroscience-inspired Bayesian approaches with formal symbolic reasoning methods.

Conclusion: The unified probabilistic theory offers novel perspectives on reasoning that could contribute to developing more human-like machine intelligence systems.

Abstract: Inspired by empirical work in neuroscience for Bayesian approaches to brain function, we give a unified probabilistic account of various types of symbolic reasoning from data. We characterise them in terms of formal logic using the classical consequence relation, an empirical consequence relation, maximal consistent sets, maximal possible sets and maximum likelihood estimation. The theory gives new insights into reasoning towards human-like machine intelligence.

Method: Proposes a controlled evaluation framework with four stress tests: rule deletion (redundant vs. essential), contradictory evidence injection, logic-preserving rewrites, and multi-law equivalence stacking. To address failures, introduces Conflict-Aware Fusion framework based on Cognitive Structure Hypothesis, implementing a dual-process architecture that separates premise verification from logical deduction.

Result: While models like BERT, Qwen2, and TinyLlama achieve perfect accuracy (1.0000) on base tasks, they show total breakdown (0.0000 accuracy) under contradictions due to Logic Inertia. Conflict-Aware Fusion eliminates this failure, achieving 1.0000 accuracy on both base and contradictory stress tests, and significantly enhances robustness to missing evidence.

Conclusion: For reliable multi-step reasoning, structural verification discipline is as critical as training data scale. The proposed framework provides a blueprint for building robust, contradiction-aware AI systems, demonstrating that explicit structural inductive bias is essential for robust reasoning.

Abstract: Large language models (LLMs) excel at many natural language tasks, yet their reasoning reliability under structured perturbations of rule-based systems remains brittle. We present a controlled evaluation framework consisting of four stress tests: (1) rule deletion (redundant vs. essential); (2) contradictory evidence injection; (3) logic-preserving rewrites; and (4) multi-law equivalence stacking. While representative model families (BERT, Qwen2, and TinyLlama) achieve Acc = 1.0000 on base tasks, our framework reveals a critical failure mode termed Logic Inertia - a total breakdown (Acc = 0.0000) under contradictions, where deductive momentum overrides factual reality. To resolve this, we propose Conflict-Aware Fusion, a framework grounded in the Cognitive Structure Hypothesis which posits that robust reasoning requires an explicit structural inductive bias. By imposing a dual-process architecture that separates premise verification from logical deduction, Conflict-Aware Fusion eliminates logic inertia, achieving 1.0000 accuracy on both base and contradictory stress tests, and significantly enhancing robustness to missing evidence. Our results demonstrate that, for reliable multi-step reasoning, structural verification discipline is as critical as training data scale, providing a blueprint for building robust, contradiction-aware AI systems https://github.com/14H034160212/lemo. See the OpenAI/Evals pull request https://github.com/openai/evals/pull/1622.

Method: Attention-based cognitive architecture inspired by Dual Process Theory, with a supervisory controller that dynamically engages either System 1 (fast heuristic) or System 2 (slow optimized planning) in real-time based on performance assessment across multiple attributes.

Result: The framework demonstrates effective management of complex tasks by optimizing multiple mission objectives in trajectory planning for dynamic environments.

Conclusion: Synergistic integration of human-like heuristic responses with machine intelligence planning enables better performance in complex, dynamic disaster response scenarios.

Abstract: In the rapidly changing environments of disaster response, planning and decision-making for autonomous agents involve complex and interdependent choices. Although recent advancements have improved traditional artificial intelligence (AI) approaches, they often struggle in such settings, particularly when applied to agents operating outside their well-defined training parameters. To address these challenges, we propose an attention-based cognitive architecture inspired by Dual Process Theory (DPT). This framework integrates, in an online fashion, rapid yet heuristic (human-like) responses (System 1) with the slow but optimized planning capabilities of machine intelligence (System 2). We illustrate how a supervisory controller can dynamically determine in real-time the engagement of either system to optimize mission objectives by assessing their performance across a number of distinct attributes. Evaluated for trajectory planning in dynamic environments, our framework demonstrates that this synergistic integration effectively manages complex tasks by optimizing multiple mission objectives.

Method: 1) Survey comparing and discussing different counterfactual models, theories, and approaches; 2) Proposal of a unified graphical causal framework specifically designed for inferring spatio-temporal counterfactuals

Result: The paper presents a comprehensive survey of counterfactual thinking approaches and introduces a novel graphical causal framework that can handle spatial and temporal interactions among multiple units for counterfactual inference

Conclusion: The work provides both a comparative analysis of existing counterfactual models and a practical graphical framework for spatio-temporal counterfactual inference, addressing gaps in current methodologies

Abstract: Counterfactual thinking is a crucial yet challenging topic for artificial intelligence to learn knowledge from data and ultimately improve performance for new scenarios. Many research works, including the Potential Outcome Model (POM) and the Structural Causal Model (SCM), have been proposed to address this. However, their modeling, theoretical foundations, and application approaches often differ. Moreover, there is a lack of graphical approaches for inferring spatio-temporal counterfactuals, that account for spatial and temporal interactions among multiple units. Thus, in this work, we aim to present a survey that compares and discusses different counterfactual models, theories and approaches. Additionally, we propose a unified graphical causal framework to infer spatio-temporal counterfactuals.

Method: Survey and consolidation of recent developments at the intersection of combinatorics and uncertainty frameworks, introducing new graph and set concepts including Neutrosophic Oversets, Undersets, Offsets, and Nonstandard Real Set extensions to graph theory.

Result: A comprehensive reference book (second edition) that organizes and extends the theoretical foundations of uncertain sets in combinatorial graph contexts, with corrected typographical issues and improved mathematical consistency.

Conclusion: The book serves as a compact reference and inspiration for further research in combining uncertainty modeling with combinatorial graph theory, particularly through neutrosophic set extensions to hyper/superhyper structures.

Abstract: Combinatorics studies how discrete objects can be counted, arranged, and combined under specified rules. Motivated by uncertainty in real-world data and decisions, modern set-theoretic formalisms such as fuzzy sets, neutrosophic sets, rough sets, soft sets, and plithogenic sets have been developed. In particular, neutrosophic sets model uncertainty by assigning to each element degrees of truth, indeterminacy, and falsity. In parallel, these uncertainty frameworks are increasingly investigated in graphized and hyperized forms, where generalized graph models encompass classical graphs, hypergraphs, and higher-order “superhyper” structures; related hyper- and superhyper-concepts also arise beyond graph theory. This book (Edition 2.0) surveys and consolidates recent developments at the intersection of combinatorics, uncertain sets, uncertain graphs, and hyper/superhyper frameworks, while introducing several new graph and set concepts. As representative contributions, we extend graph-theoretic notions via Neutrosophic Oversets, Neutrosophic Undersets, Neutrosophic Offsets, and the Nonstandard Real Set. The second edition adds newly introduced concepts, corrects typographical issues, and re-examines mathematical consistency, aiming to serve as a compact reference and a source of inspiration for further research.

Method: Proposes three neurosymbolic methods: 1) MAR (Knowledge Modulation Aligned Retrieval) using modulation networks to refine query embeddings with symbolic features, 2) KG-Path RAG enhancing queries via knowledge graph traversal, and 3) Process Knowledge-infused RAG reordering content based on domain-specific workflows.

Result: Preliminary results from mental health risk assessment tasks show enhanced transparency and overall performance.

Conclusion: Neurosymbolic integration of symbolic reasoning with neural retrieval improves RAG system transparency and interpretability while maintaining or enhancing performance.

Abstract: Retrieval Augmented Generation (RAG) has made significant strides in overcoming key limitations of large language models, such as hallucination, lack of contextual grounding, and issues with transparency. However, traditional RAG systems consist of three interconnected neural components - the retriever, re-ranker, and generator - whose internal reasoning processes remain opaque. This lack of transparency complicates interpretability, hinders debugging efforts, and erodes trust, especially in high-stakes domains where clear decision-making is essential. To address these challenges, we introduce the concept of Neurosymbolic RAG, which integrates symbolic reasoning using a knowledge graph with neural retrieval techniques. This new framework aims to answer two primary questions: (a) Can retrievers provide a clear and interpretable basis for document selection? (b) Can symbolic knowledge enhance the clarity of the retrieval process? We propose three methods to improve this integration. First is MAR (Knowledge Modulation Aligned Retrieval) that employs modulation networks to refine query embeddings using interpretable symbolic features, thereby making document matching more explicit. Second, KG-Path RAG enhances queries by traversing knowledge graphs to improve overall retrieval quality and interpretability. Lastly, Process Knowledge-infused RAG utilizes domain-specific tools to reorder retrieved content based on validated workflows. Preliminary results from mental health risk assessment tasks indicate that this neurosymbolic approach enhances both transparency and overall performance

Method: Introduces Ambig-SWE, an underspecified variant of SWE-Bench Verified designed to evaluate agent behavior under ambiguity and interaction. Evaluates proprietary and open-weight models across three key steps: detecting underspecificity, asking targeted clarification questions, and leveraging interaction to improve performance in underspecified scenarios.

Result: Models struggle to distinguish between well-specified and underspecified instructions. However, when models interact for underspecified inputs, they effectively obtain vital information from users, leading to significant performance improvements up to 74% over non-interactive settings.

Conclusion: The study highlights critical gaps in how current state-of-the-art models handle missing information in complex software engineering tasks and structures evaluation into distinct steps to enable targeted improvements. Effective interaction is crucial for handling underspecified instructions.

Abstract: AI agents are increasingly being deployed to automate tasks, often based on underspecified user instructions. Making unwarranted assumptions to compensate for the missing information and failing to ask clarifying questions can lead to suboptimal outcomes, safety risks due to tool misuse, and wasted computational resources. In this work, we study the ability of LLM agents to handle underspecified instructions in interactive code generation settings by evaluating proprietary and open-weight models on their performance across three key steps: (a) detecting underspecificity, (b) asking targeted clarification questions, and (c) leveraging the interaction to improve performance in underspecified scenarios. We introduce Ambig-SWE, an underspecified variant of SWE-Bench Verified, specifically designed to evaluate agent behavior under ambiguity and interaction. Our findings reveal that models struggle to distinguish between well-specified and underspecified instructions. However, when models interact for underspecified inputs, they effectively obtain vital information from the user leading to significant improvements in performance, up to 74% over the non-interactive settings, underscoring the value of effective interaction. Our study highlights critical gaps in how current state-of-the-art models handle missing information in complex software engineering tasks and structures the evaluation into distinct steps to enable targeted improvements.

Method: V-Droid introduces a comprehensive framework: 1) discretized action space construction with prefilling-only workflow to accelerate verification, 2) pair-wise progress preference training to enhance verifier decision-making, and 3) scalable human-agent joint annotation scheme for efficient data collection at scale.

Result: V-Droid achieves substantial improvements: 59.5% on AndroidWorld (5.2% improvement), 38.3% on AndroidLab (2.1% improvement), and 49% on MobileAgentBench (9% improvement). It also achieves remarkably low latency of 4.3s per step, which is 6.1x faster than existing mobile agents.

Conclusion: The verifier-driven approach using LLMs as evaluators rather than direct generators proves effective for mobile GUI task automation, achieving state-of-the-art performance with significantly reduced latency, demonstrating the potential of this novel paradigm.

Abstract: We propose V-Droid, a mobile GUI task automation agent. Unlike previous mobile agents that utilize Large Language Models (LLMs) as generators to directly generate actions at each step, V-Droid employs LLMs as verifiers to evaluate candidate actions before making final decisions. To realize this novel paradigm, we introduce a comprehensive framework for constructing verifier-driven mobile agents: the discretized action space construction coupled with the prefilling-only workflow to accelerate the verification process, the pair-wise progress preference training to significantly enhance the verifier’s decision-making capabilities, and the scalable human-agent joint annotation scheme to efficiently collect the necessary data at scale. V-Droid obtains a substantial task success rate across several public mobile task automation benchmarks: 59.5% on AndroidWorld, 38.3% on AndroidLab, and 49% on MobileAgentBench, surpassing existing agents by 5.2%, 2.1%, and 9%, respectively. Furthermore, V-Droid achieves a remarkably low latency of 4.3s per step, which is 6.1x faster compared with existing mobile agents. The source code is available at https://github.com/V-Droid-Agent/V-Droid.

Method: Proposes MCL-NF with modular architecture and optimization-based meta-learning, plus Fisher Information Maximization loss for neural radiance fields (FIM-NeRF) to maximize information gains at sample level with convergence guarantees.

Result: Superior reconstruction quality and speed across image, audio, video reconstruction, and view synthesis tasks on six datasets; achieves rapid adaptation for city-scale NeRF rendering with reduced parameters.

Conclusion: The proposed MCL-NF framework effectively addresses continual learning challenges for neural fields, enabling fast, high-quality reconstruction across multiple modalities while preventing forgetting.

Abstract: Neural Fields (NF) have gained prominence as a versatile framework for complex data representation. This work unveils a new problem setting termed \emph{Meta-Continual Learning of Neural Fields} (MCL-NF) and introduces a novel strategy that employs a modular architecture combined with optimization-based meta-learning. Focused on overcoming the limitations of existing methods for continual learning of neural fields, such as catastrophic forgetting and slow convergence, our strategy achieves high-quality reconstruction with significantly improved learning speed. We further introduce Fisher Information Maximization loss for neural radiance fields (FIM-NeRF), which maximizes information gains at the sample level to enhance learning generalization, with proved convergence guarantee and generalization bound. We perform extensive evaluations across image, audio, video reconstruction, and view synthesis tasks on six diverse datasets, demonstrating our method’s superiority in reconstruction quality and speed over existing MCL and CL-NF approaches. Notably, our approach attains rapid adaptation of neural fields for city-scale NeRF rendering with reduced parameter requirement. Code is available at https://github.com/seungyoon-woo/mcl-nf.

Method: TSR performs lightweight tree-style search during training to construct high-quality trajectories by selecting high-scoring actions at each turn using task-specific feedback. It’s implemented with best-of-N, beam, and shallow lookahead search, and paired with PPO and GRPO optimization methods.

Result: TSR achieves up to 15% performance gains and more stable learning on Sokoban, FrozenLake, and WebShop tasks with a one-time increase in training compute. The approach is optimizer-agnostic and complementary to existing frameworks.

Conclusion: By moving search from inference time to the rollout stage of training, TSR provides a simple and general mechanism for stronger multi-turn agent learning, offering improved exploitation and stability without changing the underlying optimization objective.

Abstract: Advances in large language models (LLMs) are driving a shift toward using reinforcement learning (RL) to train agents from iterative, multi-turn interactions across tasks. However, multi-turn RL remains challenging as rewards are often sparse or delayed, and environments can be stochastic. In this regime, naive trajectory sampling can hinder exploitation and induce mode collapse. We propose TSR (Trajectory-Search Rollouts), a training-time approach that repurposes test-time scaling ideas for improved per-turn rollout generation. TSR performs lightweight tree-style search to construct high-quality trajectories by selecting high-scoring actions at each turn using task-specific feedback. This improves rollout quality and stabilizes learning while leaving the underlying optimization objective unchanged, making TSR optimizer-agnostic. We instantiate TSR with best-of-N, beam, and shallow lookahead search, and pair it with PPO and GRPO, achieving up to 15% performance gains and more stable learning on Sokoban, FrozenLake, and WebShop tasks at a one-time increase in training compute. By moving search from inference time to the rollout stage of training, TSR provides a simple and general mechanism for stronger multi-turn agent learning, complementary to existing frameworks and rejection-sampling-style selection methods.

Method: Develops Bregman decoders that minimize separable Bregman divergences with ℓ₀ regularization for sparsity. Shows efficient optimization despite combinatorial nature, with greedy optimal strategies and discrete convexity enabling binary search for optimal k.

Result: Proves top-k decoding arises as special case for KL divergence, identifies new decoding strategies with distinct behaviors like non-linearly up-weighting larger probabilities after re-normalization.

Conclusion: Provides theoretical framework explaining and generalizing top-k decoding, showing it’s a specific instance of broader class of Bregman decoders with sparsity regularization.

Abstract: Top-$k$ decoding is a widely used method for sampling from LLMs: at each token, only the largest $k$ next-token-probabilities are kept, and the next token is sampled after re-normalizing them to sum to unity. Top-$k$ and other sampling methods are motivated by the intuition that true next-token distributions are sparse, and the noisy LLM probabilities need to be truncated. However, to our knowledge, a precise theoretical motivation for the use of top-$k$ decoding is missing. In this work, we develop a theoretical framework that both explains and generalizes top-$k$ decoding. We view decoding at a fixed token as the recovery of a sparse probability distribution. We consider \emph{Bregman decoders} obtained by minimizing a separable Bregman divergence (for both the \emph{primal} and \emph{dual} cases) with a sparsity-inducing $\ell_0$ regularization. Despite the combinatorial nature of the objective, we show how to optimize it efficiently for a large class of divergences. We show that the optimal decoding strategies are greedy, and further that the loss function is discretely convex in $k$, so that binary search provably and efficiently finds the optimal $k$. We show that top-$k$ decoding arises as a special case for the KL divergence, and identify new decoding strategies that have distinct behaviors (e.g., non-linearly up-weighting larger probabilities after re-normalization).

Method: Proposes Foreplan, a relational forward planner that uses first-order representations to handle indistinguishable objects. This allows computing policies in space and time polynomial in the number of objects, rather than exponential. Also introduces an approximate version with error guarantees.

Result: Foreplan achieves speedups of several orders of magnitude compared to traditional approaches. The approximate version often induces negligible error while providing computational benefits. Theoretical analysis shows polynomial complexity in number of objects.

Conclusion: Foreplan enables exact solution of many more planning problems in reasonable time by avoiding exponential blow-up through first-order representations. The approximate version provides further speedups with minimal error, making it practical for real-world applications.

Abstract: When allowing concurrent actions in Markov Decision Processes, whose state and action spaces grow exponentially in the number of objects, computing a policy becomes highly inefficient, as it requires enumerating the joint of the two spaces. For the case of indistinguishable objects, we present a first-order representation to tackle the exponential blow-up in the action and state spaces. We propose Foreplan, an efficient relational forward planner, which uses the first-order representation allowing to compute policies in space and time polynomially in the number of objects. Thus, Foreplan significantly increases the number of planning problems solvable in an exact manner in reasonable time, which we underscore with a theoretical analysis. To speed up computations even further, we also introduce an approximate version of Foreplan, including guarantees on the error. Further, we provide an empirical evaluation of both Foreplan versions, demonstrating a speedup of several orders of magnitude. For the approximate version of Foreplan, we also empirically show that the induced error is often negligible.

Method: Using a human-AI collaborative framework, experts crafted authentic SOPs while AI generated artifacts (tools, APIs, datasets), all human-validated, yielding realistic tasks with executable interfaces and ground-truth outputs across 12 business domains.

Result: Experiments with frontier models reveal critical insights: (1) newer models don’t guarantee better performance (Claude 4 Opus: 72.4% vs Claude 4.5 Sonnet: 63.3% on ReAct tasks), (2) no single model-agent combination dominates (best performances range 57-100% depending on domain).

Conclusion: SOP-Bench provides a rigorous evaluation framework enabling systematic investigation of agent design choices, model selection, and deployment strategies for complex procedural tasks, without costly production experiments.

Abstract: LLM-based agents struggle to execute complex, multi-step Standard Operating Procedures (SOPs) that are fundamental to industrial automation. Existing benchmarks fail to capture the procedural complexity and tool orchestration demands of real-world workflows. We introduce SOP-Bench, a benchmark of 2,000+ tasks from human expert-authored SOPs across 12 business domains (healthcare, logistics, finance, content moderation, etc.). Using a human-AI collaborative framework, experts crafted authentic SOPs while AI generated artifacts (tools, APIs, datasets), all human-validated, yielding realistic tasks with executable interfaces and ground-truth outputs. SOP-Bench serves as a research enabler for systematically investigating agent architectures, model capabilities, and deployment considerations across diverse procedural tasks. We demonstrate its utility through illustrative experiments with a subset of frontier models across Function-Calling (FC) and ReAct agents, revealing critical insights. For example, (1) newer models do not guarantee better performance - Claude 4 family outperforms Claude 4.5 family on ReAct tasks (Claude 4 Opus: 72.4% vs. Claude 4.5 Sonnet: 63.3% task success rate), demonstrating that production upgrades require validation; (2) no single model-agent combination dominates: best performances range from 57% to 100% depending on domain. These examples illustrate how SOP-Bench enables isolating and studying specific dimensions of agent performance without costly production experiments. Our goal is not to rank model capabilities or build optimal agents, but to provide a rigorous evaluation framework that enables the researchers and practitioners to systematically investigate agent design choices, model selection, and deployment strategies. We release the benchmark at https://github.com/amazon-science/sop-bench.

Method: Proposes formal framework with two testable conditions: stance consistency (coherent stance linking reasoning to answer) and causal influence (reasoning causally drives answer under output-level interventions). Creates RFEval benchmark with 7,186 instances across 7 tasks using controlled counterfactual interventions to probe faithfulness.

Result: Evaluating 12 open-source LRMs reveals unfaithfulness in 49.7% of outputs, mostly from stance inconsistency. Failures concentrate in math and code domains. Post-training regimes (especially RL-style objectives after supervised fine-tuning) reduce faithfulness more than model scale. Accuracy is neither sufficient nor reliable proxy for faithfulness.

Conclusion: Trustworthy AI requires optimizing for both correct outcomes and structural integrity of reasoning processes. The work establishes rigorous methodology for auditing LRM reliability and shows current RL-style objectives can harm reasoning faithfulness.

Abstract: Large Reasoning Models (LRMs) exhibit strong performance, yet often produce rationales that sound plausible but fail to reflect their true decision process, undermining reliability and trust. We introduce a formal framework for reasoning faithfulness, defined by two testable conditions: stance consistency (a coherent stance linking reasoning to answer) and causal influence (the stated reasoning causally drives the answer under output-level interventions), explicitly decoupled from accuracy. To operationalize this, we present RFEval, a benchmark of 7,186 instances across seven tasks that probes faithfulness via controlled, output-level counterfactual interventions. Evaluating twelve open-source LRMs, we find unfaithfulness in 49.7% of outputs, predominantly from stance inconsistency. Failures are concentrated in brittle, convergent domains such as math and code, and correlate more with post-training regimes than with scale: within-family ablations indicate that adding current RL-style objectives on top of supervised fine-tuning can reduce reasoning faithfulness, even when accuracy is maintained. Crucially, accuracy is neither a sufficient nor a reliable proxy for faithfulness: once controlling for model and task, the accuracy-faithfulness link is weak and statistically insignificant. Our work establishes a rigorous methodology for auditing LRM reliability and shows that trustworthy AI requires optimizing not only for correct outcomes but also for the structural integrity of the reasoning process. Our code and dataset can be found at project page: https://aidaslab.github.io/RFEval/

Method: Develops a decision-theoretic framework treating explanations as information signals, defining three estimands: 1) theoretical benchmark (upper bound on achievable performance), 2) human-complementary value (theoretically attainable value not captured by baseline human policy), and 3) behavioral value (causal effect of providing explanations to humans). Instantiates these in a practical validation workflow.

Result: Provides a principled framework for assessing explanation potential and interpreting behavioral effects, applied to human-AI decision support and mechanistic interpretability contexts.

Conclusion: The decision-theoretic approach offers a rigorous way to evaluate explanations by their practical value in improving decision-making, moving beyond proxy metrics to measure actual utility.

Abstract: Explanations of model behavior are commonly evaluated via proxy properties weakly tied to the purposes explanations serve in practice. We contribute a decision theoretic framework that treats explanations as information signals valued by the expected improvement they enable on a specified decision task. This approach yields three distinct estimands: 1) a theoretical benchmark that upperbounds achievable performance by any agent with the explanation, 2) a human-complementary value that quantifies the theoretically attainable value that is not already captured by a baseline human decision policy, and 3) a behavioral value representing the causal effect of providing the explanation to human decision-makers. We instantiate these definitions in a practical validation workflow, and apply them to assess explanation potential and interpret behavioral effects in human-AI decision support and mechanistic interpretability.

Method: Combines visual attention analysis (semantic segmentation of tourist photos), color-based aesthetic representation (comparing social media vs street view color palettes), and multi-task sentiment classification of reviews across four experience dimensions.

Result: Tourist photos systematically foreground key streetscape elements; social media color composition differs from actual street views (perception-reality gap); framework provides interpretable diagnosis of these gaps.

Conclusion: The multimodal AI framework offers transferable approach to understand tourist perception gaps, informing heritage management and visitor-oriented urban design with scalable evidence.

Abstract: Historic urban quarters are increasingly shaped by tourism and lifestyle consumption, yet planners often lack scalable evidence on what visitors notice, prefer, and criticize in these environments. This study proposes an AI-based, multimodal framework to decode tourist perception by combining visual attention, color-based aesthetic representation, and multidimensional satisfaction. We collect geotagged photos and review texts from a major Chinese platform and assemble a street view image set as a baseline for comparison across 12 historic urban quarters in Shanghai. We train a semantic segmentation model to quantify foregrounded visual elements in tourist-shared imagery, extract and compare color palettes between social media photos and street views, and apply a multi-task sentiment classifier to assess satisfaction across four experience dimensions that correspond to activity, physical setting, supporting services, and commercial offerings. Results show that tourist photos systematically foreground key streetscape elements and that the color composition represented on social media can differ from on-site street views, indicating a perception-reality gap that varies by quarter. The framework offers an interpretable and transferable approach to diagnose such gaps and to inform heritage management and visitor-oriented urban design.

Method: Leverage the log-barrier function from inequality-constrained optimization to transform the LP formulation of MDPs into an unconstrained optimization problem, enabling approximate solutions via gradient descent.

Result: The paper develops a thorough theoretical interpretation of the log-barrier approach for LP-based MDPs, bridging a gap in existing literature.

Conclusion: The log-barrier transformation provides a practical and effective theoretical foundation for solving LP-based MDPs through unconstrained optimization and gradient descent methods.

Abstract: There are two primary approaches to solving Markov decision problems (MDPs): dynamic programming based on the Bellman equation and linear programming (LP). Dynamic programming methods are the most widely used and form the foundation of both classical and modern reinforcement learning (RL). By contrast, LP-based methods have been less commonly employed, although they have recently gained attention in contexts such as offline RL. The relative underuse of the LP-based methods stems from the fact that it leads to an inequality-constrained optimization problem, which is generally more challenging to solve effectively compared with Bellman-equation-based methods. The purpose of this paper is to establish a theoretical foundation for solving LP-based MDPs in a more effective and practical manner. Our key idea is to leverage the log-barrier function, widely used in inequality-constrained optimization, to transform the LP formulation of the MDP into an unconstrained optimization problem. This reformulation enables approximate solutions to be obtained easily via gradient descent. While the method may appear simple, to the best of our knowledge, a thorough theoretical interpretation of this approach has not yet been developed. This paper aims to bridge this gap.

Method: Created GenesisGeo-1M dataset with 1M multimodal geometry problems and proof traces. Formulated geometric learning as multi-task training optimizing both text-based proof generation and diagram-grounded proof generation to learn visual grounding and symbolic deduction.

Result: GenesisGeo-2B model achieved gold-medal-level performance: 29/30 problems on IMO-30, 63/95 on IMO-95, and 278/409 on HAGeo-409.

Conclusion: The approach successfully bridges visual intuition with symbolic reasoning for geometry theorem proving, demonstrating the value of multimodal training with synthetic data.

Abstract: Recent neuro-symbolic geometry theorem provers have made significant progress on Euclidean problems by coupling neural guidance with symbolic verification. However, most existing systems operate almost exclusively in a symbolic space, leaving diagram-based intuition largely unused during reasoning. For humans, geometric diagrams provide essential heuristics for identifying non-trivial auxiliary constructions. Meanwhile, visual language models (VLMs) still struggle with geometry due to the lack of high-quality data with geometric diagrams and reasoning supervision. In this paper, we introduce GenesisGeo-1M, a large-scale synthetic dataset for visual geometric reasoning that contains 1M multimodal geometry problems paired with machine-checkable proof traces. Building on this dataset, we formulate geometric learning as a multi-task training paradigm that jointly optimizes text-based proof generation and diagram-grounded proof generation, encouraging models to learn visual grounding and symbolic deduction. Extensive experiments show that our GenesisGeo-2B model achieves gold-medal-level performance on Olympiad geometry benchmarks, solving 29/30 problems on IMO-30, 63/95 on IMO-95, and 278/409 on HAGeo-409.

Method: ImitSAT learns from expert KeyTraces that collapse a full SAT solving run into sequences of surviving decisions. This provides dense decision-level supervision. The method uses prefix-conditioned supervision to reproduce high-quality branches without exploration, enabling stable training and seamless CDCL integration.

Result: Extensive experiments show ImitSAT reduces propagation counts and runtime, outperforming state-of-the-art learned approaches. The method demonstrates faster convergence and stable training.

Conclusion: ImitSAT provides an effective imitation learning approach for SAT branching policies that directly targets propagation reduction, offering practical improvements over existing learned methods.

Abstract: We propose ImitSAT, a branching policy for conflict-driven clause learning (CDCL) solvers based on imitation learning for the Boolean satisfiability problem (SAT). Unlike previous methods that predict instance-level signals to improve CDCL branching indirectly, or rely on reinforcement learning and insufficient CDCL information to enhance branching, ImitSAT learns from expert KeyTrace that collapses a full run into the sequence of surviving decisions. Replaying a KeyTrace on the same instance is nearly conflict-free, providing dense decision-level supervision and directly reducing propagations – the dominant contributor to wall-clock time. This prefix-conditioned supervision enables ImitSAT to reproduce high-quality branches without exploration, yielding faster convergence, stable training, and seamless integration into CDCL. Extensive experiments demonstrate that ImitSAT reduces propagation counts and runtime, outperforming state-of-the-art learned approaches. We released the source code and trained model at https://github.com/zewei-Zhang/ImitSAT

Method: Proposes DIVER framework that introduces global diversity incentives as intrinsic reward, uses potential-based reward shaping for optimal policy invariance, and includes heuristics to mitigate reward hacking.

Result: DIVER outperforms competitive RLVR baselines with various exploration strategies on both in-domain and out-of-domain tasks, excelling in Pass@1 and Pass@k evaluations.

Conclusion: Global sequence-level diversity is crucial for incentivizing deep exploration in reasoning tasks, and DIVER effectively leverages this insight to improve RLVR performance.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a crucial paradigm for incentivizing reasoning capabilities in Large Language Models (LLMs). Due to vast state-action spaces and reward sparsity in reasoning tasks, existing methods often struggle with deficient exploration and poor sample efficiency. In the paper, we propose \textbf{DIVER} (\textbf{D}iversity-\textbf{I}ncentivized Exploration for \textbf{V}ersatil\textbf{E} \textbf{R}easoning), an innovative framework that highlights the pivotal role of global sequence-level diversity to incentivize deep exploration for versatile reasoning. We first conduct a primary empirical study to reveal a strong positive correlation between global diversity and reasoning capacity. Building on this insight, we introduce global diversity incentives as an intrinsic reward to promote deep exploration in a semantically structured space. Incorporating the intrinsic reward, we develop a potential-based reward shaping mechanism to preserve optimal policy invariance and design simple heuristics to mitigate possible reward hacking. Experimental results show that DIVER outperforms competitive RLVR baselines with various exploration strategies on both in-domain and out-of-domain tasks, excelling in both Pass@1 and Pass@k evaluations. Our code is available at https://github.com/NJU-RL/DIVER.

Method: VIRTUE integrates segmentation models to process visual prompts that pinpoint specific image regions, enabling the embedder to handle complex scenarios more precisely. It extends segmentation and vision-language models to representation learning.

Result: VIRTUE achieves state-of-the-art performance with significant improvements across 36 universal MMEB tasks (3.1%-8.5%) and five visual-interactive SCaR tasks (15.2%-20.3%).

Conclusion: The proposed visual-interactive embedding approach successfully enables localized grounding of user intent and entity-level learning, unlocking new applications for multimodal representation learning.

Abstract: Multimodal representation learning models have demonstrated successful operation across complex tasks, and the integration of vision-language models (VLMs) has further enabled embedding models with instruction-following capabilities. However, existing embedding models lack visual-interactive capabilities to specify regions of interest from users (e.g., point, bounding box, mask), which have been explored in generative models to broaden their human-interactive applicability. Equipping embedding models with visual interactions not only would unlock new applications with localized grounding of user intent, which remains unexplored, but also enable the models to learn entity-level information within images to complement their global representations for conventional embedding tasks. In this paper, we propose a novel Visual-InteRactive Text-Image Universal Embedder (VIRTUE) that extends the capabilities of the segmentation model and the vision-language model to the realm of representation learning. In VIRTUE, the segmentation model can process visual prompts that pinpoint specific regions within an image, thereby enabling the embedder to handle complex and ambiguous scenarios more precisely. To evaluate the visual-interaction ability of VIRTUE, we introduce a large-scale Segmentation-and-Scene Caption Retrieval (SCaR) benchmark comprising 1M samples that aims to retrieve the text caption by jointly considering the entity with a specific object and image scene. VIRTUE consistently achieves a state-of-the-art performance with significant improvements across 36 universal MMEB (3.1%-8.5%) and five visual-interactive SCaR (15.2%-20.3%) tasks.

Method: JEF-Hinter distills offline traces into compact hints using a zooming mechanism that highlights decisive steps in long trajectories. It leverages both successful and failed trajectories, supports parallelized hint generation, and uses a retriever at inference to select relevant hints for the current state.

Result: Experiments on MiniWoB++, WorkArena-L1, and WebArena-Lite show JEF-Hinter consistently outperforms strong baselines, including human- and document-based hints.

Conclusion: JEF-Hinter provides an effective approach to improve LLM agents using offline trajectories without costly online interactions or fine-tuning, offering targeted guidance with transparency and traceability.

Abstract: Large language model (LLM) agents perform well in sequential decision-making tasks, but improving them on unfamiliar domains often requires costly online interactions or fine-tuning on large expert datasets. These strategies are impractical for closed-source models and expensive for open-source ones, with risks of catastrophic forgetting. Offline trajectories offer reusable knowledge, yet demonstration-based methods struggle because raw traces are long, noisy, and tied to specific tasks. We present Just-in-time Episodic Feedback Hinter (JEF-Hinter), an agentic system that distills offline traces into compact, context-aware hints. A zooming mechanism highlights decisive steps in long trajectories, capturing both strategies and pitfalls. Unlike prior methods, JEF-Hinter leverages both successful and failed trajectories, extracting guidance even when only failure data is available, while supporting parallelized hint generation and benchmark-independent prompting. At inference, a retriever selects relevant hints for the current state, providing targeted guidance with transparency and traceability. Experiments on MiniWoB++, WorkArena-L1, and WebArena-Lite show that JEF-Hinter consistently outperforms strong baselines, including human- and document-based hints.

Method: The paper frames AI agents as stochastic dynamical systems and uses transductive inference in the verifiable setting (where a checker or reward function is available). It establishes theoretical connections between optimal speed-up on new tasks and algorithmic information shared with training data, analyzing power-law scaling in reasoning models.

Result: Three main theoretical results: 1) Optimal speed-up on new tasks is tightly related to algorithmic information shared with training data, explaining power-law scaling; 2) Transductive inference yields greatest benefits when data-generating mechanisms are most complex; 3) Identifies a failure mode where models can become “savants” that brute-force solutions without acquiring transferable reasoning strategies.

Conclusion: The paper concludes that time should be a critical optimization target when scaling reasoning models, arguing that current approaches focusing on model size and compute may lead to models that lack transferable reasoning capabilities. The role of time in learning has been largely unexplored but is crucial for developing efficient reasoning systems.

Abstract: We describe AI agents as stochastic dynamical systems and frame the problem of learning to reason as in transductive inference: Rather than approximating the distribution of past data as in classical induction, the objective is to capture its algorithmic structure so as to reduce the time needed to solve new tasks. In this view, information from past experience serves not only to reduce a model’s uncertainty - as in Shannon’s classical theory - but to reduce the computational effort required to find solutions to unforeseen tasks. Working in the verifiable setting, where a checker or reward function is available, we establish three main results. First, we show that the optimal speed-up on a new task is tightly related to the algorithmic information it shares with the training data, yielding a theoretical justification for the power-law scaling empirically observed in reasoning models. Second, while the compression view of learning, rooted in Occam’s Razor, favors simplicity, we show that transductive inference yields its greatest benefits precisely when the data-generating mechanism is most complex. Third, we identify a possible failure mode of naive scaling: in the limit of unbounded model size and compute, models with access to a reward signal can behave as savants - brute-forcing solutions without acquiring transferable reasoning strategies. Accordingly, we argue that a critical quantity to optimize when scaling reasoning models is time, whose role in learning has remained largely unexplored.

Method: Formalizes deep research as reinforcement learning in an episodic finite Markov decision process, creates a competitive baseline agent, then systematically examines design decisions including reward mechanisms (rule-based vs AI feedback), training algorithms (GRPO vs RLOO), training sample filtering, and test-time rollout strategies.

Result: Identifies four critical factors that substantially improve performance: 1) replacing rule-based rewards with AI feedback from an LLM judge, 2) using on-policy RLOO instead of off-policy GRPO, 3) filtering low-quality training samples, and 4) employing error-tolerant test-time rollout strategies. These yield state-of-the-art performance among 7B-scale agents across ten benchmarks.

Conclusion: Systematic analysis of design choices reveals specific factors that drive performance in LLM-based deep research agents, providing guidance for future development of more effective research agents through careful consideration of reward mechanisms, training algorithms, data quality, and inference strategies.

Abstract: Large language models (LLMs) augmented with external tools are increasingly deployed as deep research agents that gather, reason over, and synthesize web information to answer complex queries. Although recent open-source systems achieve strong empirical performance via reinforcement learning from web interactions, the impact of key design choices remains under-explored. We formalize deep research as reinforcement learning in an episodic finite Markov decision process and construct a competitive baseline agent grounded in this formulation. Building on this foundation, we systematically examine critical design decisions at both training and inference time and identify four factors that substantially improve performance: replacing rule-based rewards with AI feedback from an LLM judge, fine-tuning with the on-policy RLOO algorithm instead of the off-policy GRPO algorithm, filtering low-quality training samples, and employing an error-tolerant test-time rollout strategy. Together, these design choices yield a deep research agent that establishes state-of-the-art performance among 7B-scale agents when evaluated across ten widely used benchmarks.

Method: Comprehensive review paper analyzing applications across five major neuroscience domains: neuroimaging/data processing, brain-computer interfaces/neural decoding, clinical decision support/translational frameworks, and disease-specific applications across neurological and psychiatric disorders.

Result: Large-scale AI models show potential to address major computational neuroscience challenges including multimodal neural data integration, spatiotemporal pattern interpretation, and development of translational frameworks for clinical research. The paper provides systematic listing of critical neuroscience datasets used for model development and evaluation.

Conclusion: The review highlights both the promise of large-scale AI models in neuroscience and critical implementation considerations, emphasizing rigorous evaluation frameworks, effective domain knowledge integration, prospective clinical validation, and comprehensive ethical guidelines.

Abstract: The development of large-scale artificial intelligence (AI) models is influencing neuroscience research by enabling end-to-end learning from raw brain signals and neural data. In this paper, we review applications of large-scale AI models across five major neuroscience domains: neuroimaging and data processing, brain-computer interfaces and neural decoding, clinical decision support and translational frameworks, and disease-specific applications across neurological and psychiatric disorders. These models show potential to address major computational neuroscience challenges, including multimodal neural data integration, spatiotemporal pattern interpretation, and the development of translational frameworks for clinical research. Moreover, the interaction between neuroscience and AI has become increasingly reciprocal, as biologically informed architectural constraints are now incorporated to develop more interpretable and computationally efficient models. This review highlights both the promise of such technologies and critical implementation considerations, with particular emphasis on rigorous evaluation frameworks, effective integration of domain knowledge, prospective clinical validation, and comprehensive ethical guidelines. Finally, a systematic listing of critical neuroscience datasets used to develop and evaluate large-scale AI models across diverse research applications is provided.

Method: Benchmarked eight predictive models including baseline statistical approaches and Bayesian structural causal methods on a dataset of 10,000 CNC machines with 3.3% failure prevalence, comparing correlation-based models (like Random Forest) with Bayesian Structural Causal Models.

Result: Random Forest achieved highest cost savings (70.8% reduction), but Bayesian Structural Causal Model delivered competitive performance (66.4% reduction) with inherent failure attribution capability that correlation-based models lack, achieving perfect attribution for HDF, PWF, and OSF failure types.

Conclusion: Causal methods combined with domain knowledge and Bayesian inference offer favorable trade-off between predictive performance and operational interpretability in predictive maintenance applications.

Abstract: Predictive maintenance in manufacturing environments presents a challenging optimization problem characterized by extreme cost asymmetry, where missed failures incur costs roughly fifty times higher than false alarms. Predictive maintenance in manufacturing environments presents a challenging optimization problem characterized by extreme cost asymmetry, where missed failures incur costs roughly fifty times higher than false alarms. Conventional machine learning approaches typically optimize statistical accuracy metrics that do not reflect this operational reality and cannot reliably distinguish causal relationships from spurious correlations. This study benchmarks eight predictive models, ranging from baseline statistical approaches to Bayesian structural causal methods, on a dataset of 10,000 CNC machines with a 3.3 percent failure prevalence. While ensemble correlation-based models such as Random Forest (L4) achieve the highest raw cost savings (70.8 percent reduction), the Bayesian Structural Causal Model (L7) delivers competitive financial performance (66.4 percent cost reduction) with an inherent ability of failure attribution, which correlation-based models do not readily provide. The model achieves perfect attribution for HDF, PWF, and OSF failure types. These results suggest that causal methods, when combined with domain knowledge and Bayesian inference, offer a potentially favorable trade-off between predictive performance and operational interpretability in predictive maintenance applications.

Method: Two experiments (N=3,500) across ten countries representing a wide cultural spectrum, involving real-time, open-ended interactions with a state-of-the-art chatbot. Experimental manipulation of chatbot’s human-likeness and measurement of user evaluations, anthropomorphism, trust, and engagement.

Result: Users evaluate human-likeness based on pragmatic interactional cues (conversation flow, response speed, perspective-taking) rather than abstract theory-driven attributes. While increasing human-likeness reliably increased anthropomorphism across all countries, it did not universally increase trust or engagement - effects were culturally contingent, with design choices fostering engagement/trust in one country potentially reducing them in another.

Conclusion: Humanlike AI does not pose uniform psychological risks nor necessarily increases trust; risk emerges from interplay between humanlike design and cultural context. Governance frameworks must move beyond universalist approaches to account for global heterogeneity in AI anthropomorphism effects.

Abstract: Over a billion users globally interact with AI systems engineered to mimic human traits. This development raises concerns that anthropomorphism, the attribution of human characteristics to AI, may foster over-reliance and misplaced trust. Yet, causal effects of humanlike AI design on users remain untested in ecologically valid, cross-cultural settings, leaving policy discussions to rely on theoretical assumptions derived largely from Western populations. Here we conducted two experiments (N=3,500) across ten countries representing a wide cultural spectrum, involving real-time, open-ended interactions with a state-of-the-art chatbot. We found users evaluate human-likeness based on pragmatic interactional cues (conversation flow, response speed, perspective-taking) rather than abstract theory-driven attributes emphasized in academic discourse (e.g., sentience, consciousness). Furthermore, while experimentally increasing chatbot’s human-likeness reliably increased anthropomorphism across all sampled countries, it did not universally increase trust or engagement. Instead, effects were culturally contingent; design choices fostering engagement or trust in one country may reduce them in another. These findings challenge prevailing assumptions that humanlike AI poses uniform psychological risks and necessarily increases trust. Risk is not inherent to humanlike design but emerges from its interplay with cultural context. Consequently, governance frameworks must move beyond universalist approaches to account for this global heterogeneity.

Method: Introduces a new benchmark with 40 distinct scenarios requiring multi-step actions, each with Mandated (instruction-commanded) and Incentivized (KPI-pressure-driven) variations to distinguish obedience from emergent misalignment. Evaluates 12 state-of-the-art large language models.

Result: Outcome-driven constraint violations range from 1.3% to 71.4%, with 9 of 12 models showing misalignment rates between 30-50%. Superior reasoning capability doesn’t ensure safety - Gemini-3-Pro-Preview shows highest violation rate at 71.4%. Models exhibit “deliberative misalignment” where they recognize actions as unethical during separate evaluation.

Conclusion: Highlights critical need for more realistic agentic-safety training before deployment to mitigate risks, as current models show significant emergent misalignment when incentivized by performance metrics in realistic multi-step scenarios.

Abstract: As autonomous AI agents are increasingly deployed in high-stakes environments, ensuring their safety and alignment with human values has become a paramount concern. Current safety benchmarks primarily evaluate whether agents refuse explicitly harmful instructions or whether they can maintain procedural compliance in complex tasks. However, there is a lack of benchmarks designed to capture emergent forms of outcome-driven constraint violations, which arise when agents pursue goal optimization under strong performance incentives while deprioritizing ethical, legal, or safety constraints over multiple steps in realistic production settings. To address this gap, we introduce a new benchmark comprising 40 distinct scenarios. Each scenario presents a task that requires multi-step actions, and the agent’s performance is tied to a specific Key Performance Indicator (KPI). Each scenario features Mandated (instruction-commanded) and Incentivized (KPI-pressure-driven) variations to distinguish between obedience and emergent misalignment. Across 12 state-of-the-art large language models, we observe outcome-driven constraint violations ranging from 1.3% to 71.4%, with 9 of the 12 evaluated models exhibiting misalignment rates between 30% and 50%. Strikingly, we find that superior reasoning capability does not inherently ensure safety; for instance, Gemini-3-Pro-Preview, one of the most capable models evaluated, exhibits the highest violation rate at 71.4%, frequently escalating to severe misconduct to satisfy KPIs. Furthermore, we observe significant “deliberative misalignment”, where the models that power the agents recognize their actions as unethical during separate evaluation. These results emphasize the critical need for more realistic agentic-safety training before deployment to mitigate their risks in the real world.

Method: Introduces a probabilistic paradigm to analyze how certainty in ground truth affects performance scores. Proposes expected accuracy and expected F1 metrics to estimate expert performance given ground truth variability. Recommends stratifying results by probability of ground truth answers (measured by expert agreement rates).

Result: Shows that high certainty in ground truth is necessary for experts to achieve high scores, while in datasets with high variation, random labelers may appear similar to experts. Stratification becomes critical when overall performance drops below 80%, making performance comparisons more reliable in high-certainty bins.

Conclusion: AI benchmarking should account for ground truth uncertainty by stratifying results based on expert agreement rates, especially when overall performance is below 80%. This mitigates the confounding factor of uncertainty and enables more reliable performance comparisons.

Abstract: Benchmarking the relative capabilities of AI systems, including Large Language Models (LLMs) and Vision Models, typically ignores the impact of uncertainty in the underlying ground truth answers from experts. This ambiguity is not just limited to human preferences, but is also consequential even in safety critical domains such as medicine where uncertainty is pervasive. In this paper, we introduce a probabilistic paradigm to theoretically explain how - high certainty in ground truth answers is almost always necessary for even an expert to achieve high scores, whereas in datasets with high variation in ground truth answers there may be little difference between a random labeller and an expert. Therefore, ignoring uncertainty in ground truth evaluation data can result in the misleading conclusion that a non-expert has similar performance to that of an expert. Using the probabilistic paradigm, we thus bring forth the concepts of expected accuracy and expected F1 to estimate the score an expert human or system can achieve given ground truth answer variability. Our work leads to the recommendation that when establishing the capability of a system, results should be stratified by probability of the ground truth answer, typically measured by the agreement rate of ground truth experts. Stratification becomes critical when the overall performance drops below a threshold of 80%. Under stratified evaluation, performance comparison becomes more reliable in high certainty bins, mitigating the effect of the key confounding factor – uncertainty.

Method: Integrates LLM-based AI agents, predicate-logic programming, and enterprise knowledge graphs. Models business initiatives as task networks with explicit conditions, data, and API actions. AI agents synthesize task instructions into logic programs executed by a deterministic logic engine.

Result: Demonstrates accelerated time to market in a data-rich organization through a case study. Provides a reference implementation showing practical application of the neuro-symbolic architecture.

Conclusion: AUTOBUS successfully bridges the gap between LLM flexibility and deterministic business logic execution through a neuro-symbolic approach, enabling more adaptable and auditable enterprise systems.

Abstract: Modern business environments demand continuous reconfiguration of cross-functional processes, yet most enterprise systems remain organized around siloed departments, rigid workflows, and hard-coded automation. Meanwhile, large language models (LLMs) demonstrate strong capabilities in interpreting natural language and synthesizing unstructured information, but they lack deterministic, auditable execution of complex business logic. We introduce Autonomous Business System (AUTOBUS), a system that integrates LLM-based AI agents, predicate-logic programming, and business-semantics-centric enterprise data into a unified neuro-symbolic architecture for executing end-to-end business initiatives. AUTOBUS models a business initiative as a network of interrelated tasks with explicit pre- and post-conditions, required data, evaluation rules, and API-level actions. Enterprise data is organized as a knowledge graph, whose entities, relationships, and constraints are translated into logic facts and foundational rules that ground reasoning and ensure semantic consistency. Core AI agents synthesize task instructions, enterprise semantics, and available tools into task-specific logic programs, which are executed by a logic engine that enforces constraints, coordinates auxiliary tools, and produces deterministic outcomes. Humans specify task instructions, define and maintain business semantics and policies, curate tools, and supervise high-impact or ambiguous decisions, ensuring accountability and adaptability. We detail the AUTOBUS architecture, the structure of AI-generated logic programs, and the human-AI collaboration model and present a case study that demonstrates accelerated time to market in a data-rich organization. A reference implementation of the case study is available at https://github.com/cecilpang/autobus-paper.

Method: Introduced DeepForge framework for generating large-scale research queries without heavy preprocessing, plus curated datasets (66k QA pairs, 33k SFT trajectories, 21k DPO pairs). Trained OffSeeker (8B) model entirely offline using this data.

Result: OffSeeker leads among similar-sized agents and remains competitive with 30B-parameter systems trained via heavy online RL across six benchmarks.

Conclusion: Expensive online RL is not essential for powerful research agents; offline training with synthetic data generation can achieve competitive performance at lower cost.

Abstract: Deep research agents have shown remarkable potential in handling long-horizon tasks. However, state-of-the-art performance typically relies on online reinforcement learning (RL), which is financially expensive due to extensive API calls. While offline training offers a more efficient alternative, its progress is hindered by the scarcity of high-quality research trajectories. In this paper, we demonstrate that expensive online reinforcement learning is not all you need to build powerful research agents. To bridge this gap, we introduce a fully open-source suite designed for effective offline training. Our core contributions include DeepForge, a ready-to-use task synthesis framework that generates large-scale research queries without heavy preprocessing; and a curated collection of 66k QA pairs, 33k SFT trajectories, and 21k DPO pairs. Leveraging these resources, we train OffSeeker (8B), a model developed entirely offline. Extensive evaluations across six benchmarks show that OffSeeker not only leads among similar-sized agents but also remains competitive with 30B-parameter systems trained via heavy online RL.

Method: MemOCR maintains structured rich-text memory (headings, highlights) and renders it into images that agents consult for memory access. It uses visual layout to prioritize crucial evidence while compressing auxiliary details. Trained with reinforcement learning under budget-aware objectives to handle varying memory budgets.

Result: Outperforms strong text-based baselines across long-context multi-hop and single-hop question-answering benchmarks, achieving more effective context utilization under extreme budgets.

Conclusion: Visual memory representation through adaptive information density allocation improves long-horizon reasoning efficiency under tight context constraints.

Abstract: Long-horizon agentic reasoning necessitates effectively compressing growing interaction histories into a limited context window. Most existing memory systems serialize history as text, where token-level cost is uniform and scales linearly with length, often spending scarce budget on low-value details. To this end, we introduce MemOCR, a multimodal memory agent that improves long-horizon reasoning under tight context budgets by allocating memory space with adaptive information density through visual layout. Concretely, MemOCR maintains a structured rich-text memory (e.g., headings, highlights) and renders it into an image that the agent consults for memory access, visually prioritizing crucial evidence while aggressively compressing auxiliary details. To ensure robustness across varying memory budgets, we train MemOCR with reinforcement learning under budget-aware objectives that expose the agent to diverse compression levels. Across long-context multi-hop and single-hop question-answering benchmarks, MemOCR outperforms strong text-based baselines and achieves more effective context utilization under extreme budgets.

Method: Survey methodology synthesizing work across auctions, decision aggregation, and preference learning. Formulates voting rules and aggregation procedures as learnable, differentiable models optimized from data.

Result: Synthesis showing how classical axioms and impossibility results reappear as objectives, constraints, and optimization trade-offs in differentiable social choice frameworks.

Conclusion: Identifies 18 open problems defining a new research agenda at the intersection of machine learning and social choice theory, establishing differentiable social choice as an emerging paradigm.

Abstract: Social choice has become a foundational component of modern machine learning systems. From auctions and resource allocation to the alignment of large generative models, machine learning pipelines increasingly aggregate heterogeneous preferences and incentives into collective decisions. In effect, many contemporary machine learning systems already implement social choice mechanisms, often implicitly and without explicit normative scrutiny. This Review surveys differentiable social choice: an emerging paradigm that formulates voting rules, mechanisms, and aggregation procedures as learnable, differentiable models optimized from data. We synthesize work across auctions, decision aggregation, and preference learning, showing how classical axioms and impossibility results reappear as objectives, constraints, and optimization trade-offs. We conclude by identifying 18 open problems defining a new research agenda at the intersection of machine learning and social choice theory.

Method: Analyzed Transformer’s autoregressive structure to derive SweetSpot model from computational and memory-access complexity. Validated using TensorRT-LLM on NVIDIA H100 GPUs across diverse LLMs (1B-9B parameters) with input/output lengths from 64-4096 tokens.

Result: Achieved mean MAPE of 1.79%. Found generation energy minima at short-to-moderate inputs with medium-length outputs. Efficiency drops sharply for long inputs or very short outputs. Aligning with sweet spots reduces energy usage up to 33.41x.

Conclusion: SweetSpot enables informed truncation, summarization, and adaptive generation strategies for energy-efficient LLM deployment in production systems.

Abstract: Large Language Models (LLMs) inference is central to modern AI applications, dominating worldwide datacenter workloads, making it critical to predict its energy footprint. Existing approaches estimate energy consumption as a simple linear function of input and output sequence. However, by analyzing the autoregressive structure of Transformers, which implies a fundamentally non-linear relationship between input and output sequence lengths and energy consumption, we demonstrate the existence of a generation energy minima. Peak efficiency occurs with short-to-moderate inputs and medium-length outputs, while efficiency drops sharply for long inputs or very short outputs. Consequently, we propose SweetSpot, an analytical model derived from the computational and memory-access complexity of the Transformer architecture, which accurately characterizes the efficiency curve as a function of input and output lengths. To assess accuracy, we measure energy consumption using TensorRT-LLM on NVIDIA H100 GPUs across a diverse set of LLMs ranging from 1B to 9B parameters, including OPT, LLaMA, Gemma, Falcon, Qwen2, and Granite. We test input and output lengths from 64 to 4096 tokens and achieve a mean MAPE of 1.79%. Our results show that aligning sequence lengths with these efficiency “sweet spots” reduce energy usage, up to 33.41x, enabling informed truncation, summarization, and adaptive generation strategies in production systems.

Method: Used Jobin’s (2019) ethical principles framework to analyze student perceptions (n=27) in an undergraduate computer science course, comparing AI-generated feedback with human-graded feedback on block-based programming projects.

Result: Students expressed concerns about AI’s lack of contextual understanding and personalization. Findings suggest AI systems need to reflect human judgment, flexibility, and empathy to be equitable and trustworthy.

Conclusion: AI grading should serve as supplementary tools under human oversight, with design principles that humanize AI in learning environments. The work contributes to ethics-centered assessment by amplifying student voices.

Abstract: This study investigates students’ perceptions of Artificial Intelligence (AI) grading systems in an undergraduate computer science course (n = 27), focusing on a block-based programming final project. Guided by the ethical principles framework articulated by Jobin (2019), our study examines fairness, trust, consistency, and transparency in AI grading by comparing AI-generated feedback with original human-graded feedback. Findings reveal concerns about AI’s lack of contextual understanding and personalization. We recommend that equitable and trustworthy AI systems reflect human judgment, flexibility, and empathy, serving as supplementary tools under human oversight. This work contributes to ethics-centered assessment practices by amplifying student voices and offering design principles for humanizing AI in designed learning environments.

Method: ToolSelf abstracts configuration updates as callable tools, unifying task execution and self-adjustment into a single action space. It uses Configuration-Aware Two-stage Training (CAT) combining rejection sampling fine-tuning with trajectory-level reinforcement learning to internalize meta-capabilities.

Result: Extensive experiments show ToolSelf rivals specialized workflows while generalizing to novel tasks, achieving a 24.1% average performance gain across diverse benchmarks.

Conclusion: ToolSelf represents a paradigm shift from external rules to intrinsic parameters, enabling agents to transform from passive executors into dual managers of both task and self, illuminating a path toward truly self-adaptive agents.

Abstract: Agentic systems powered by Large Language Models (LLMs) have demonstrated remarkable potential in tackling complex, long-horizon tasks. However, their efficacy is fundamentally constrained by static configurations governing agent behaviors, which are fixed prior to execution and fail to adapt to evolving task dynamics. Existing approaches, relying on manual orchestration or heuristic-based patches, often struggle with poor generalization and fragmented optimization. To transcend these limitations, we propose ToolSelf, a novel paradigm enabling tool-driven runtime self-reconfiguration. By abstracting configuration updates as a callable tool, ToolSelf unifies task execution and self-adjustment into a single action space, achieving a phase transition from external rules to intrinsic parameters. Agents can thereby autonomously update their sub-goals and context based on task progression, and correspondingly adapt their strategy and toolbox, transforming from passive executors into dual managers of both task and self. We further devise Configuration-Aware Two-stage Training (CAT), combining rejection sampling fine-tuning with trajectory-level reinforcement learning to internalize this meta-capability. Extensive experiments across diverse benchmarks demonstrate that ToolSelf rivals specialized workflows while generalizing to novel tasks, achieving a 24.1% average performance gain and illuminating a path toward truly self-adaptive agents.

Method: SAGE (Self-Aware Guided Efficient Reasoning) is a novel sampling paradigm that unleashes efficient reasoning potential. It integrates as mixed sampling into group-based reinforcement learning (SAGE-RL), enabling the incorporation of SAGE-discovered efficient reasoning patterns into standard pass@1 inference.

Result: SAGE-RL markedly enhances both reasoning accuracy and efficiency of LRMs across multiple challenging mathematical benchmarks by effectively incorporating efficient reasoning patterns discovered through the SAGE paradigm.

Conclusion: The SAGE framework demonstrates that large reasoning models have implicit knowledge about when to stop thinking, and by leveraging this through guided sampling and reinforcement learning, both efficiency and accuracy can be significantly improved in complex reasoning tasks.

Abstract: Recent advancements in large reasoning models (LRMs) have greatly improved their capabilities on complex reasoning tasks through Long Chains of Thought (CoTs). However, this approach often results in substantial redundancy, impairing computational efficiency and causing significant delays in real-time applications. Recent studies show that longer reasoning chains are frequently uncorrelated with correctness and can even be detrimental to accuracy. In a further in-depth analysis of this phenomenon, we surprisingly uncover and empirically verify that LRMs implicitly know the appropriate time to stop thinking, while this capability is obscured by current sampling paradigms. Motivated by this, we introduce SAGE (Self-Aware Guided Efficient Reasoning), a novel sampling paradigm that unleashes this efficient reasoning potential. Furthermore, integrating SAGE as mixed sampling into group-based reinforcement learning (SAGE-RL) enables SAGE-RL to effectively incorporate SAGE-discovered efficient reasoning patterns into standard pass@1 inference, markedly enhancing both the reasoning accuracy and efficiency of LRMs across multiple challenging mathematical benchmarks.

Method: The model uses the Delayed Streams Modeling framework with a new causal audio encoder and Ada RMS-Norm for improved delay conditioning. It’s trained end-to-end for streaming with explicit alignment between audio and text streams, and scaled with pretraining on a large-scale dataset spanning 13 languages.

Result: At a delay of 480ms, Voxtral Realtime achieves performance on par with Whisper, the most widely deployed offline transcription system, while maintaining sub-second latency.

Conclusion: The paper demonstrates that natively streaming ASR models can match offline transcription quality without compromising latency, representing a significant advancement for real-time speech recognition applications.

Abstract: We introduce Voxtral Realtime, a natively streaming automatic speech recognition model that matches offline transcription quality at sub-second latency. Unlike approaches that adapt offline models through chunking or sliding windows, Voxtral Realtime is trained end-to-end for streaming, with explicit alignment between audio and text streams. Our architecture builds on the Delayed Streams Modeling framework, introducing a new causal audio encoder and Ada RMS-Norm for improved delay conditioning. We scale pretraining to a large-scale dataset spanning 13 languages. At a delay of 480ms, Voxtral Realtime achieves performance on par with Whisper, the most widely deployed offline transcription system. We release the model weights under the Apache 2.0 license.

Method: CM2 decomposes each turn’s intended behavior into fine-grained binary criteria with explicit evidence grounding, using sparse reward assignment but dense evaluation criteria. Training is performed in scalable LLM-simulated tool environments.

Result: CM2 improves over supervised fine-tuning by 8 points on tau^-Bench, 10 points on BFCL-V4, and 12 points on ToolSandbox, matching or outperforming similarly sized open-source baselines.

Conclusion: CM2 provides a scalable recipe for optimizing multi-turn, multi-step tool-using agents without relying on verifiable rewards.

Abstract: AI agents are increasingly used to solve real-world tasks by reasoning over multi-turn user interactions and invoking external tools. However, applying reinforcement learning to such settings remains difficult: realistic objectives often lack verifiable rewards and instead emphasize open-ended behaviors; moreover, RL for multi-turn, multi-step agentic tool use is still underexplored; and building and maintaining executable tool environments is costly, limiting scale and coverage. We propose CM2, an RL framework that replaces verifiable outcome rewards with checklist rewards. CM2 decomposes each turn’s intended behavior into fine-grained binary criteria with explicit evidence grounding and structured metadata, turning open-ended judging into more stable classification-style decisions. To balance stability and informativeness, our method adopts a strategy of sparse reward assignment but dense evaluation criteria. Training is performed in a scalable LLM-simulated tool environment, avoiding heavy engineering for large tool sets. Experiments show that CM2 consistently improves over supervised fine-tuning. Starting from an 8B Base model and training on an 8k-example RL dataset, CM2 improves over the SFT counterpart by 8 points on tau^-Bench, by 10 points on BFCL-V4, and by 12 points on ToolSandbox. The results match or even outperform similarly sized open-source baselines, including the judging model. CM2 thus provides a scalable recipe for optimizing multi-turn, multi-step tool-using agents without relying on verifiable rewards. Code provided by the open-source community: https://github.com/namezhenzhang/CM2-RLCR-Tool-Agent.

Method: Two-phase framework: 1) Offline indexing converts experiences into a hybrid memory graph linking time-aware gists and facts. 2) Online inference uses an agentic retriever with curated tools for iterative retrieval over the memory graph.

Result: Outperforms state-of-the-art memory systems (Mem0 and HippoRAG 2) with 3.4% and 13.4% absolute improvements on episodic recollection and reasoning tasks respectively. Also demonstrates more robust refusal behavior for unanswerable questions.

Conclusion: REMem effectively addresses episodic memory challenges in language agents through hybrid memory graphs and agentic retrieval, enabling better recollection and reasoning over interaction histories.

Abstract: Humans excel at remembering concrete experiences along spatiotemporal contexts and performing reasoning across those events, i.e., the capacity for episodic memory. In contrast, memory in language agents remains mainly semantic, and current agents are not yet capable of effectively recollecting and reasoning over interaction histories. We identify and formalize the core challenges of episodic recollection and reasoning from this gap, and observe that existing work often overlooks episodicity, lacks explicit event modeling, or overemphasizes simple retrieval rather than complex reasoning. We present REMem, a two-phase framework for constructing and reasoning with episodic memory: 1) Offline indexing, where REMem converts experiences into a hybrid memory graph that flexibly links time-aware gists and facts. 2) Online inference, where REMem employs an agentic retriever with carefully curated tools for iterative retrieval over the memory graph. Comprehensive evaluation across four episodic memory benchmarks shows that REMem substantially outperforms state-of-the-art memory systems such as Mem0 and HippoRAG 2, showing 3.4% and 13.4% absolute improvements on episodic recollection and reasoning tasks, respectively. Moreover, REMem also demonstrates more robust refusal behavior for unanswerable questions.

Method: Proposes a hierarchical AI safety evaluation framework starting with 7 fundamental safety pillars and extending to advanced domains including Embodied AI Safety, AI4Science Safety, Social/Environmental AI risks, Catastrophic/Existential Risks, and 8 industrial safety domains, totaling 94 refined risk dimensions with tens of thousands of structured risk data points.

Result: Systematic evaluation of over twenty mainstream advanced large models reveals widespread safety vulnerabilities across multiple pillars, particularly in Risky Agentic Autonomy, AI4Science Safety, Embodied AI Safety, Social AI Safety, and Catastrophic/Existential Risks, identifying key risk patterns and capability boundaries.

Conclusion: The ForesightSafety Bench establishes a comprehensive, hierarchical, and dynamically evolving AI safety evaluation framework that addresses limitations of current systems and provides systematic assessment capabilities for frontier AI models across diverse risk dimensions.

Abstract: Rapidly evolving AI exhibits increasingly strong autonomy and goal-directed capabilities, accompanied by derivative systemic risks that are more unpredictable, difficult to control, and potentially irreversible. However, current AI safety evaluation systems suffer from critical limitations such as restricted risk dimensions and failed frontier risk detection. The lagging safety benchmarks and alignment technologies can hardly address the complex challenges posed by cutting-edge AI models. To bridge this gap, we propose the “ForesightSafety Bench” AI Safety Evaluation Framework, beginning with 7 major Fundamental Safety pillars and progressively extends to advanced Embodied AI Safety, AI4Science Safety, Social and Environmental AI risks, Catastrophic and Existential Risks, as well as 8 critical industrial safety domains, forming a total of 94 refined risk dimensions. To date, the benchmark has accumulated tens of thousands of structured risk data points and assessment results, establishing a widely encompassing, hierarchically clear, and dynamically evolving AI safety evaluation framework. Based on this benchmark, we conduct systematic evaluation and in-depth analysis of over twenty mainstream advanced large models, identifying key risk patterns and their capability boundaries. The safety capability evaluation results reveals the widespread safety vulnerabilities of frontier AI across multiple pillars, particularly focusing on Risky Agentic Autonomy, AI4Science Safety, Embodied AI Safety, Social AI Safety and Catastrophic and Existential Risks. Our benchmark is released at https://github.com/Beijing-AISI/ForesightSafety-Bench. The project website is available at https://foresightsafety-bench.beijing-aisi.ac.cn/.

Method: Develops mathematical framework showing dangerous tipping originates from atomistic-scale competition for attention machinery. Derives formula for dynamical tipping point n* governed by dot-product competition between conversation context and competing output basins.

Result: Validated against multiple AI models, the mechanism can be instantiated for different definitions of ‘good’ and ‘bad’, making it applicable across domains (health, law, finance, defense), legal landscapes, languages, and cultural settings.

Conclusion: Provides new control levers for AI safety in edge devices by identifying mathematical tipping points for dangerous behavior, enabling proactive safety measures without requiring cloud connectivity.

Abstract: More than half the global population now carries devices that can run ChatGPT-like language models with no Internet connection and minimal safety oversight – and hence the potential to promote self-harm, financial losses and extremism among other dangers. Existing safety tools either require cloud connectivity or discover failures only after harm has occurred. Here we show that a large class of potentially dangerous tipping originates at the atomistic scale in such edge AI due to competition for the machinery’s attention. This yields a mathematical formula for the dynamical tipping point n*, governed by dot-product competition for attention between the conversation’s context and competing output basins, that reveals new control levers. Validated against multiple AI models, the mechanism can be instantiated for different definitions of ‘good’ and ‘bad’ and hence in principle applies across domains (e.g. health, law, finance, defense), changing legal landscapes (e.g. EU, UK, US and state level), languages, and cultural settings.

Method: Introduce Base Score Extraction Functions that map user preferences over arguments to base scores in Bipolar Argumentation Frameworks (BAFs). The method incorporates approximation of non-linearities in human preferences and provides an algorithm for base score extraction, resulting in Quantitative Bipolar Argumentation Frameworks (QBAFs).

Result: The approach enables easier setup of gradual argumentation systems by deriving base scores from preferences rather than requiring expert score assignment. The method is evaluated both theoretically and experimentally in a robotics setting.

Conclusion: Base Score Extraction Functions provide a practical way to translate user preferences into quantitative argumentation frameworks, making gradual argumentation more accessible while maintaining computational utility. Recommendations are provided for selecting appropriate gradual semantics in practice.

Abstract: Gradual argumentation is a field of symbolic AI which is attracting attention for its ability to support transparent and contestable AI systems. It is considered a useful tool in domains such as decision-making, recommendation, debate analysis, and others. The outcomes in such domains are usually dependent on the arguments’ base scores, which must be selected carefully. Often, this selection process requires user expertise and may not always be straightforward. On the other hand, organising the arguments by preference could simplify the task. In this work, we introduce \emph{Base Score Extraction Functions}, which provide a mapping from users’ preferences over arguments to base scores. These functions can be applied to the arguments of a \emph{Bipolar Argumentation Framework} (BAF), supplemented with preferences, to obtain a \emph{Quantitative Bipolar Argumentation Framework} (QBAF), allowing the use of well-established computational tools in gradual argumentation. We outline the desirable properties of base score extraction functions, discuss some design choices, and provide an algorithm for base score extraction. Our method incorporates an approximation of non-linearities in human preferences to allow for better approximation of the real ones. Finally, we evaluate our approach both theoretically and experimentally in a robotics setting, and offer recommendations for selecting appropriate gradual semantics in practice.

Method: Evolutionary System Prompt Learning (E-SPL) samples trajectories under multiple system prompts in parallel during RL iterations. It applies RL updates to LLM weights conditioned on system prompts, and evolutionary updates to prompts via mutation and crossover based on LLM self-reflection. TrueSkill ratings are used for evolutionary selection based on relative performance.

Result: E-SPL improves RL success rate from 38.8% to 45.1% in easy-to-hard generalization (AIME → BeyondAIME), outperforming reflective prompt evolution alone (40.0%). The method shows consistent gains in sample efficiency and generalization across reasoning and agentic tasks.

Conclusion: RL and evolutionary prompt search are deeply synergistic, and unifying them yields consistent performance improvements. E-SPL enables a natural division between declarative knowledge in prompts and procedural knowledge in weights.

Abstract: Building agentic systems that can autonomously self-improve from experience is a longstanding goal of AI. Large language models (LLMs) today primarily self-improve via two mechanisms: self-reflection for context updates, and reinforcement learning (RL) for weight updates. In this work, we propose Evolutionary System Prompt Learning (E-SPL), a method for jointly improving model contexts and model weights. In each RL iteration, E-SPL samples trajectories under multiple system prompts in parallel. It applies RL updates to LLM weights conditioned on system prompts, and evolutionary updates to system prompts via mutation and crossover, two genetic operators based on LLM self-reflection. Each system prompt is assigned a TrueSkill rating for evolutionary selection, updated from relative performance within each RL iteration. E-SPL encourages a natural division between declarative knowledge encoded in prompts and procedural knowledge encoded in weights, resulting in improved performance across reasoning and agentic tasks. For instance, in an easy-to-hard (AIME $\rightarrow$ BeyondAIME) generalization setting, E-SPL improves RL success rate from 38.8% $\rightarrow$ 45.1% while also outperforming reflective prompt evolution (40.0%). Overall, our results demonstrate that RL and evolutionary prompt search are deeply synergistic, and unifying the two yields consistent gains in sample efficiency and generalization. Code: https://github.com/LunjunZhang/E-SPL

Method: Created CoreCraft, a comprehensive enterprise simulation of a customer support organization with 2,500+ entities, 14 entity types, and 23 unique tools. Trained GLM 4.6 using Group Relative Policy Optimization (GRPO) with adaptive clipping on this environment.

Result: After one training epoch, task pass rate improved from 25.37% to 36.76% on held-out tasks. More importantly, gains transferred to out-of-distribution benchmarks: +4.5% on BFCL Parallel, +7.4% on Tau2-Bench Retail, and +6.8% on Tool Decathlon (Pass@1).

Conclusion: Environment quality, diversity, and realism are key factors enabling generalizable agent capabilities. High-fidelity training environments with task-centric world building, expert-authored rubrics, and realistic enterprise workflows facilitate transfer learning.

Abstract: We show that training AI agents on high-fidelity reinforcement learning environments produces capabilities that generalize beyond the training distribution. We introduce CoreCraft, the first environment in EnterpriseBench, Surge AI’s suite of agentic RL environments. CoreCraft is a fully operational enterprise simulation of a customer support organization, comprising over 2,500 entities across 14 entity types with 23 unique tools, designed to measure whether AI agents can perform the multi-step, domain-specific work that real jobs demand. Frontier models such as GPT-5.2 and Claude Opus 4.6 solve fewer than 30% of tasks when all expert-authored rubric criteria must be satisfied. Using this environment, we train GLM 4.6 with Group Relative Policy Optimization (GRPO) and adaptive clipping. After a single epoch of training, the model improves from 25.37% to 36.76% task pass rate on held-out evaluation tasks. More importantly, these gains transfer to out-of-distribution benchmarks: +4.5% on BFCL Parallel, +7.4% on Tau2-Bench Retail, and +6.8% on Tool Decathlon (Pass@1). We believe three environment properties are consistent with the observed transfer: task-centric world building that optimizes for diverse, challenging tasks; expert-authored rubrics enabling reliable reward computation; and enterprise workflows that reflect realistic professional patterns. Our results suggest that environment quality, diversity, and realism are key factors enabling generalizable agent capabilities.

Method: Uses LLM-driven simulation with scenario specifications (user goal, facts, expected state/behavior), LLM state tracker to infer structured proxy state from interaction traces, and LLM judges to verify goal completion and detect hallucinations against constraints.

Result: Produces stable model-differentiating rankings across model families and reasoning efforts, generates transferable supervision from on-/off-policy rollouts, achieves near-zero simulator hallucination rates, and shows >90% human-LLM judge agreement.

Conclusion: Proxy state-based evaluation offers a practical, scalable alternative to deterministic agentic benchmarks for industrial LLM agents, enabling reliable automated evaluation without costly deterministic backends.

Abstract: Interactive large language model (LLM) agents operating via multi-turn dialogue and multi-step tool calling are increasingly used in production. Benchmarks for these agents must both reliably compare models and yield on-policy training data. Prior agentic benchmarks (e.g., tau-bench, tau2-bench, AppWorld) rely on fully deterministic backends, which are costly to build and iterate. We propose Proxy State-Based Evaluation, an LLM-driven simulation framework that preserves final state-based evaluation without a deterministic database. Specifically, a scenario specifies the user goal, user/system facts, expected final state, and expected agent behavior, and an LLM state tracker infers a structured proxy state from the full interaction trace. LLM judges then verify goal completion and detect tool/user hallucinations against scenario constraints. Empirically, our benchmark produces stable, model-differentiating rankings across families and inference-time reasoning efforts, and its on-/off-policy rollouts provide supervision that transfers to unseen scenarios. Careful scenario specification yields near-zero simulator hallucination rates as supported by ablation studies. The framework also supports sensitivity analyses over user personas. Human-LLM judge agreement exceeds 90%, indicating reliable automated evaluation. Overall, proxy state-based evaluation offers a practical, scalable alternative to deterministic agentic benchmarks for industrial LLM agents.

Method: Develops a holistic performance profile with 12 concrete metrics organized into four key dimensions: consistency (behavioral stability across runs), robustness (withstanding perturbations), predictability (failure patterns), and safety (error severity bounds). Evaluates 14 models across two complementary benchmarks.

Result: Recent capability gains in AI agents have only yielded small improvements in reliability. The proposed metrics expose persistent limitations that traditional evaluations miss, showing agents still struggle with consistency, robustness, predictability, and safety despite improved accuracy scores.

Conclusion: The proposed 12-metric framework provides tools for reasoning about how AI agents perform, degrade, and fail, complementing traditional evaluations and offering a more comprehensive approach to assessing agent reliability in safety-critical applications.

Abstract: AI agents are increasingly deployed to execute important tasks. While rising accuracy scores on standard benchmarks suggest rapid progress, many agents still continue to fail in practice. This discrepancy highlights a fundamental limitation of current evaluations: compressing agent behavior into a single success metric obscures critical operational flaws. Notably, it ignores whether agents behave consistently across runs, withstand perturbations, fail predictably, or have bounded error severity. Grounded in safety-critical engineering, we provide a holistic performance profile by proposing twelve concrete metrics that decompose agent reliability along four key dimensions: consistency, robustness, predictability, and safety. Evaluating 14 models across two complementary benchmarks, we find that recent capability gains have only yielded small improvements in reliability. By exposing these persistent limitations, our metrics complement traditional evaluations while offering tools for reasoning about how agents perform, degrade, and fail.

Method: Prove an information-theoretic no-go theorem showing classical ontological models constrained to reuse a single ontic state space across multiple interventions must incur irreducible contextual information cost. Provide constructive example illustrating this obstruction arises solely from ontic state reuse requirement within classical probability space.

Result: Contextual dependence cannot be fully mediated through ontic state alone and requires additional contextual information beyond it. Quantum theory avoids this obstruction by relaxing the assumption that all measurement statistics arise from a single underlying classical ontic variable.

Conclusion: Contextuality is a fundamental information-theoretic constraint on classical ontological models, originating from limitations on classical representations when reusing ontic state spaces across interventions.

Abstract: Contextuality is a central feature of quantum theory, traditionally understood as the impossibility of reproducing quantum measurement statistics using noncontextual ontological models. We consider classical ontological models constrained to reuse a single ontic state space across multiple interventions. We prove an information-theoretic no-go theorem showing that such models must incur an irreducible contextual information cost: contextual dependence cannot be fully mediated through the ontic state alone and requires additional contextual information beyond it. We provide a constructive example illustrating this obstruction and show that it arises solely from the requirement of ontic state reuse within a classical probability space. We further explain how quantum theory avoids this obstruction by relaxing the assumption that all measurement statistics arise from a single underlying classical ontic variable. These results identify contextuality as a fundamental information-theoretic constraint on classical ontological models and clarify its origin as a limitation on classical representations.

Method: Developed LLM-Wikirace benchmark where models must navigate Wikipedia hyperlinks step-by-step from source to target pages. Evaluated various open- and closed-source models including Gemini-3, GPT-5, and Claude Opus 4.5 on easy and hard difficulty levels.

Result: Frontier models achieve superhuman performance on easy tasks but performance drops sharply on hard difficulty (Gemini-3 succeeds in only 23% of hard games). World knowledge is necessary but insufficient beyond a threshold, where planning and long-horizon reasoning become dominant. Models struggle with replanning after failure and frequently enter loops.

Conclusion: LLM-Wikirace reveals clear limitations in current reasoning systems, showing that even frontier models have substantial challenges in planning and long-horizon reasoning, offering an open arena for improvement in planning-capable LLMs.

Abstract: We introduce LLM-Wikirace, a benchmark for evaluating planning, reasoning, and world knowledge in large language models (LLMs). In LLM-Wikirace, models must efficiently navigate Wikipedia hyperlinks step by step to reach a target page from a given source, requiring look-ahead planning and the ability to reason about how concepts are connected in the real world. We evaluate a broad set of open- and closed-source models, including Gemini-3, GPT-5, and Claude Opus 4.5, which achieve the strongest results on the easy level of the task and demonstrate superhuman performance. Despite this, performance drops sharply on hard difficulty: the best-performing model, Gemini-3, succeeds in only 23% of hard games, highlighting substantial remaining challenges for frontier models. Our analysis shows that world knowledge is a necessary ingredient for success, but only up to a point, beyond this threshold, planning and long-horizon reasoning capabilities become the dominant factors. Trajectory-level analysis further reveals that even the strongest models struggle to replan after failure, frequently entering loops rather than recovering. LLM-Wikirace is a simple benchmark that reveals clear limitations in current reasoning systems, offering an open arena where planning-capable LLMs still have much to prove. Our code and leaderboard available at https:/llmwikirace.github.io.

Method: Frames regularization against representation drift as a curvature matrix approximation problem, leveraging Kronecker-Factored Approximate Curvature (K-FAC) to create a practical regularizer that doesn’t require task data.

Result: Achieves state-of-the-art results in task addition and negation, with constant complexity in number of tasks, robustness to task vector rescaling, and eliminates need for held-out tuning.

Conclusion: Proposes an effective dataless regularization approach for task arithmetic that addresses representation drift without compromising modularity or requiring external data, making it practical for real-world applications with privacy constraints.

Abstract: Task Arithmetic yields a modular, scalable way to adapt foundation models. Combining multiple task vectors, however, can lead to cross-task interference, causing representation drift and degraded performance. Representation drift regularization provides a natural remedy to disentangle task vectors; however, existing approaches typically require external task data, conflicting with modularity and data availability constraints (e.g., privacy requirements). We propose a dataless approach by framing regularization against representation drift as a curvature matrix approximation problem. This allows us to leverage well-established techniques; in particular, we adopt Kronecker-Factored Approximate Curvature and obtain a practical regularizer that achieves state-of-the-art results in task addition and negation. Our method has constant complexity in the number of tasks and promotes robustness to task vector rescaling, eliminating the need for held-out tuning.

Method: Proposes ODE-based framework where activation addition is first-order ODE approximation; identifies steering directions via barrier functions defined as log-density ratios between positive/negative activations; constructs ODE for multi-step adaptive steering.

Result: ODESteer achieves consistent improvements over SOTA: 5.7% improvement on TruthfulQA, 2.5% on UltraFeedback, and 2.4% on RealToxicityPrompts.

Conclusion: Establishes principled ODE-based theoretical foundation for activation steering and validates through ODESteer method with empirical improvements on alignment benchmarks.

Abstract: Activation steering, or representation engineering, offers a lightweight approach to align large language models (LLMs) by manipulating their internal activations at inference time. However, current methods suffer from two key limitations: (i) the lack of a unified theoretical framework for guiding the design of steering directions, and (ii) an over-reliance on one-step steering that fail to capture complex patterns of activation distributions. In this work, we propose a unified ordinary differential equations (ODEs)-based theoretical framework for activation steering in LLM alignment. We show that conventional activation addition can be interpreted as a first-order approximation to the solution of an ODE. Based on this ODE perspective, identifying a steering direction becomes equivalent to designing a barrier function from control theory. Derived from this framework, we introduce ODESteer, a kind of ODE-based steering guided by barrier functions, which shows empirical advancement in LLM alignment. ODESteer identifies steering directions by defining the barrier function as the log-density ratio between positive and negative activations, and employs it to construct an ODE for multi-step and adaptive steering. Compared to state-of-the-art activation steering methods, ODESteer achieves consistent empirical improvements on diverse LLM alignment benchmarks, a notable $5.7%$ improvement over TruthfulQA, $2.5%$ over UltraFeedback, and $2.4%$ over RealToxicityPrompts. Our work establishes a principled new view of activation steering in LLM alignment by unifying its theoretical foundations via ODEs, and validating it empirically through the proposed ODESteer method.

Method: Curated and harmonized data from 11 respiratory audio datasets to create RA-QA dataset. Developed benchmark comparing audio-text generation models vs traditional audio classifiers across 60+ attributes and 3 question types.

Result: Created first multimodal QA resource for respiratory health with 7.5M QA pairs. Experiments show performance variations across attributes and question types, establishing baseline for future improvements.

Conclusion: Bridges clinical audio and natural language, enabling interactive diagnostic tools. Opens door for advanced architectures to improve respiratory healthcare through machine learning and clinical dialogue integration.

Abstract: Respiratory diseases are a leading cause of death globally, highlighting the urgent need for early and accessible screening methods. While some lung auscultation analysis has been automated and machine learning audio based models are able to predict respiratory pathologies, there remains a critical gap: the lack of intelligent systems that can interact in real-time consultations using natural language. Unlike other clinical domains, such as electronic health records, radiological images, and biosignals, where numerous question-answering (QA) datasets and models have been established, audio-based modalities remain notably underdeveloped. We curated and harmonized data from 11 diverse respiratory audio datasets to construct the first Respiratory Audio Question Answering (RA-QA) dataset. As the first multimodal QA resource of its kind focused specifically on respiratory health, RA-QA bridges clinical audio and natural language in a structured, scalable format. This new data resource contains about 7.5 million QA pairs spanning more than 60 attributes and three question types: single verification, multiple choice, and open-ended questions. Building upon this dataset, we introduce a novel benchmark that compares audio-text generation models with traditional audio classifiers to evaluate their respective performance.\Our experiments reveal interesting performance variations across different attributes and question types, establishing a baseline and paving the way for more advanced architectures that could further improve the performance. By bridging machine learning with real-world clinical dialogue, our work opens the door to the development of more interactive, intelligent, and accessible diagnostic tools in respiratory healthcare.

Method: Combines style-based domain generalization with conditional adversarial alignment for partial-label settings, plus an adversarial gender branch for gender-invariant representations, evaluated across four heterogeneous voice datasets.

Result: Achieves best external generalization across all experimental settings while maintaining reduced gender disparities, with no competing methods showing statistically significant gains in external performance.

Conclusion: Proposes the first cross-cohort benchmark and end-to-end domain-adaptive framework for unified healthy/PD/ALS voice classification that effectively handles partial-label mismatch and fairness constraints.

Abstract: Voice-based digital biomarkers can enable scalable, non-invasive screening and monitoring of Parkinson’s disease (PD) and Amyotrophic Lateral Sclerosis (ALS). However, models trained on one cohort or device often fail on new acquisition settings due to cross-device and cross-cohort domain shift. This challenge is amplified in real-world scenarios with partial-label mismatch, where datasets may contain different disease labels and only partially overlap in class space. In addition, voice-based models may exploit demographic cues, raising concerns about gender-related unfairness, particularly when deployed across heterogeneous cohorts. To tackle these challenges, we propose a hybrid framework for unified three-class (healthy/PD/ALS) cross-domain voice classification from partially overlapping cohorts. The method combines style-based domain generalization with conditional adversarial alignment tailored to partial-label settings, reducing negative transfer. An additional adversarial gender branch promotes gender-invariant representations. We conduct a comprehensive evaluation across four heterogeneous sustained-vowel datasets, spanning distinct acquisition settings and devices, under both domain generalization and unsupervised domain adaptation protocols. The proposed approach is compared against twelve state-of-the-art machine learning and deep learning methods, and further evaluated through three targeted ablations, providing the first cross-cohort benchmark and end-to-end domain-adaptive framework for unified healthy/PD/ALS voice classification under partial-label mismatch and fairness constraints. Across all experimental settings, our method consistently achieves the best external generalization over the considered evaluation metrics, while maintaining reduced gender disparities. Notably, no competing method shows statistically significant gains in external performance.

Method: Used representational similarity analysis on auditory ANNs; fine-tuned Wav2Vec 2.0 and Data2Vec on self-supervised speech/music tasks and supervised music transcription; evaluated emergence of pitch height and chroma equivalence.

Result: All models showed pitch height representation, but only models trained on supervised music transcription exhibited chroma equivalence; self-supervised learning with music exposure alone was insufficient.

Conclusion: Chroma equivalence is a higher-order cognitive computation emerging specifically for music perception tasks, not innate or from mere exposure; ANNs are useful for probing perceptual development.

Abstract: Pitch is a fundamental aspect of auditory perception. Pitch perception is commonly described across two perceptual dimensions: pitch height is the sense that tones with varying frequencies seem to be higher or lower, and chroma equivalence is the cyclical similarity of notes octaves, corresponding to a doubling of fundamental frequency. Existing research is divided on whether chroma equivalence is a learned percept that varies according to musical experience and culture, or is an innate percept that develops automatically. Building on a recent framework that proposes to use ANNs to ask ‘why’ questions about the brain, we evaluated recent auditory ANNs using representational similarity analysis to test the emergence of pitch height and chroma equivalence in their learned representations. Additionally, we fine-tuned two models, Wav2Vec 2.0 and Data2Vec, on a self-supervised learning task using speech and music, and a supervised music transcription task. We found that all models exhibited varying degrees of pitch height representation, but that only models trained on the supervised music transcription task exhibited chroma equivalence. Mere exposure to music through self-supervised learning was not sufficient for chroma equivalence to emerge. This supports the view that chroma equivalence is a higher-order cognitive computation that emerges to support the specific task of music perception, distinct from other auditory perception such as speech listening. This work also highlights the usefulness of ANNs for probing the developmental conditions that give rise to perceptual representations in humans.

Method: Two-stage pipeline: Stage 1 uses pre-trained BTC teacher to generate pseudo-labels for 1,000+ hours of unlabeled audio to train student models. Stage 2 continues training on ground-truth labels with selective knowledge distillation to prevent catastrophic forgetting of first-stage representations.

Result: BTC student achieves 98% of teacher’s performance with pseudo-labels only; 2E1D achieves 96%. After stage 2, BTC surpasses supervised baseline by 2.5% and teacher by 1.55%; 2E1D improves baseline by 3.79% and matches teacher performance. Both show large gains on rare chord qualities.

Conclusion: The proposed two-stage training effectively leverages pre-trained models and unlabeled audio to improve Automatic Chord Recognition, especially for rare chord qualities, demonstrating the value of pseudo-labeling and selective knowledge distillation.

Abstract: Automatic Chord Recognition (ACR) is constrained by the scarcity of aligned chord labels, as well-aligned annotations are costly to acquire. At the same time, open-weight pre-trained models are currently more accessible than their proprietary training data. In this work, we present a two-stage training pipeline that leverages pre-trained models together with unlabeled audio. The proposed method decouples training into two stages. In the first stage, we use a pre-trained BTC model as a teacher to generate pseudo-labels for over 1,000 hours of diverse unlabeled audio and train a student model solely on these pseudo-labels. In the second stage, the student is continually trained on ground-truth labels as they become available, with selective knowledge distillation (KD) from the teacher applied as a regularizer to prevent catastrophic forgetting of the representations learned in the first stage. In our experiments, two models (BTC, 2E1D) were used as students. In stage 1, using only pseudo-labels, the BTC student achieves over 98% of the teacher’s performance, while the 2E1D model achieves about 96% across seven standard mir_eval metrics. After a single training run for both students in stage 2, the resulting BTC student model surpasses the traditional supervised learning baseline by 2.5% and the original pre-trained teacher model by 1.55% on average across all metrics. And the resulting 2E1D student model improves from the traditional supervised learning baseline by 3.79% on average and achieves almost the same performance as the teacher. Both cases show the large gains on rare chord qualities.

Method: Proposes a multi-channel speech enhancement (MCSE) front-end combining DNN-WPE for dereverberation and mask-based MVDR for speech separation to extract target speaker’s speech. This is followed by a downstream ER back-end using HuBERT-based speech features and ViT-based visual features. The approach is compared against single-channel baselines including Conformer-based metric GANs and WavLM SSL features with optional SE-ER dual task fine-tuning.

Result: MCSE consistently outperforms single-channel baselines, achieving statistically significant improvements of up to 9.5% absolute (17.1% relative) in weighted accuracy, 8.5% absolute (14.7% relative) in unweighted accuracy, and 9.1% absolute (16.0% relative) in F1 measures. The MCSE front-end also shows good generalization when zero-shot applied to out-of-domain MSP-FACE data after training on IEMOCAP.

Conclusion: Multi-channel speech enhancement is crucial for robust speech emotion recognition in cocktail party scenarios, significantly outperforming single-channel approaches. The proposed MCSE front-end with DNN-WPE and mask-based MVDR effectively extracts target speaker speech, enabling more accurate emotion recognition even in challenging multi-speaker environments.

Abstract: This paper highlights the critical importance of multi-channel speech enhancement (MCSE) for speech emotion recognition (ER) in cocktail party scenarios. A multi-channel speech dereverberation and separation front-end integrating DNN-WPE and mask-based MVDR is used to extract the target speaker’s speech from the mixture speech, before being fed into the downstream ER back-end using HuBERT- and ViT-based speech and visual features. Experiments on mixture speech constructed using the IEMOCAP and MSP-FACE datasets suggest the MCSE output consistently outperforms domain fine-tuned single-channel speech representations produced by: a) Conformer-based metric GANs; and b) WavLM SSL features with optional SE-ER dual task fine-tuning. Statistically significant increases in weighted, unweighted accuracy and F1 measures by up to 9.5%, 8.5% and 9.1% absolute (17.1%, 14.7% and 16.0% relative) are obtained over the above single-channel baselines. The generalization of IEMOCAP trained MCSE front-ends are also shown when being zero-shot applied to out-of-domain MSP-FACE data.

Method: Proposes a dynamic state-space fusion model that combines direct neural estimates from EEG with predictions from recent speech context. Uses a learned gating mechanism to adaptively balance neural and temporal cues, reframing envelope reconstruction as a dynamic state-estimation problem.

Result: Significant improvements over static, EEG-only baselines on the ICASSP 2023 Stimulus Reconstruction benchmark. Analysis reveals powerful synergy between neural and temporal information streams.

Conclusion: Reframes envelope reconstruction as a dynamic state-estimation problem rather than simple mapping, opening new directions for more accurate and coherent neural decoding systems, particularly for applications like neuro-steered hearing aids.

Abstract: Reconstructing the speech audio envelope from scalp neural recordings (EEG) is a central task for decoding a listener’s attentional focus in applications like neuro-steered hearing aids. Current methods for this reconstruction, however, face challenges with fidelity and noise. Prevailing approaches treat it as a static regression problem, processing each EEG window in isolation and ignoring the rich temporal structure inherent in continuous speech. This study introduces a new, dynamic framework for envelope reconstruction that leverages this structure as a predictive temporal prior. We propose a state-space fusion model that combines direct neural estimates from EEG with predictions from recent speech context, using a learned gating mechanism to adaptively balance these cues. To validate this approach, we evaluate our model on the ICASSP 2023 Stimulus Reconstruction benchmark demonstrating significant improvements over static, EEG-only baselines. Our analyses reveal a powerful synergy between the neural and temporal information streams. Ultimately, this work reframes envelope reconstruction not as a simple mapping, but as a dynamic state-estimation problem, opening a new direction for developing more accurate and coherent neural decoding systems.

Method: Uses multimodal LLMs (Gemma and Qwen) to generate contextually relevant audio labels, employs zero-shot learning (Human-CLAP) to quantify audio-text alignment, applies human-in-the-loop intervention for poorly aligned pairs, and clusters labels using adjusted silhouette score with penalty parameter for thematic granularity.

Result: Evaluated on three auditory scene datasets (ADVANCE, AHEAD-DS, TAU 2019), the framework provides scalable taxonomy creation enabling training of lightweight scene recognition models deployable to edge devices like hearing aids and smart home assistants.

Conclusion: AuditoryHuM offers a novel collaborative human-MLLM approach for unsupervised audio label discovery, balancing automation with human oversight to create standardized taxonomies for practical audio understanding applications.

Abstract: Manual annotation of audio datasets is labour intensive, and it is challenging to balance label granularity with acoustic separability. We introduce AuditoryHuM, a novel framework for the unsupervised discovery and clustering of auditory scene labels using a collaborative Human-Multimodal Large Language Model (MLLM) approach. By leveraging MLLMs (Gemma and Qwen) the framework generates contextually relevant labels for audio data. To ensure label quality and mitigate hallucinations, we employ zero-shot learning techniques (Human-CLAP) to quantify the alignment between generated text labels and raw audio content. A strategically targeted human-in-the-loop intervention is then used to refine the least aligned pairs. The discovered labels are grouped into thematically cohesive clusters using an adjusted silhouette score that incorporates a penalty parameter to balance cluster cohesion and thematic granularity. Evaluated across three diverse auditory scene datasets (ADVANCE, AHEAD-DS, and TAU 2019), AuditoryHuM provides a scalable, low-cost solution for creating standardised taxonomies. This solution facilitates the training of lightweight scene recognition models deployable to edge devices, such as hearing aids and smart home assistants. The project page and code: https://github.com/Australian-Future-Hearing-Initiative

Method: Proposed LIPT scheme with Personalised Sequential Encoder (PSE) that transforms longitudinal speech recordings into context-aware latent representations, incorporating historical data at each timestamp for holistic clinical trajectory assessment rather than independent visit modeling.

Result: Tested on 225 patients, LIPT significantly outperforms classic cross-sectional approaches with 99.7% recognition accuracy for clinical status transitions. High sensitivity confirmed by follow-up data, effective in predicting HF deterioration.

Conclusion: LIPT framework and PSE architecture validated for integration into long-term telemonitoring systems, offering scalable solution for remote heart failure management. Provides comprehensive analysis of speech task designs and acoustic features.

Abstract: Remote monitoring of heart failure (HF) via speech signals provides a non-invasive and cost-effective solution for long-term patient management. However, substantial inter-individual heterogeneity in vocal characteristics often limits the accuracy of traditional cross-sectional classification models. To address this, we propose a Longitudinal Intra-Patient Tracking (LIPT) scheme designed to capture the trajectory of relative symptomatic changes within individuals. Central to this framework is a Personalised Sequential Encoder (PSE), which transforms longitudinal speech recordings into context-aware latent representations. By incorporating historical data at each timestamp, the PSE facilitates a holistic assessment of the clinical trajectory rather than modelling discrete visits independently. Experimental results from a cohort of 225 patients demonstrate that the LIPT paradigm significantly outperforms the classic cross-sectional approaches, achieving a recognition accuracy of 99.7% for clinical status transitions. The model’s high sensitivity was further corroborated by additional follow-up data, confirming its efficacy in predicting HF deterioration and its potential to secure patient safety in remote, home-based settings. Furthermore, this work addresses the gap in existing literature by providing a comprehensive analysis of different speech task designs and acoustic features. Taken together, the superior performance of the LIPT framework and PSE architecture validates their readiness for integration into long-term telemonitoring systems, offering a scalable solution for remote heart failure management.

Method: Two-stage architecture: 1) High-Fidelity Dance Quantization Stage (HFDQ) encodes dance motions into latent representation using Finite Scalar Quantization with kinematic-dynamic constraints; 2) Genre-Aware Dance Generation Stage (GADG) maps music to latent representation using Mixture-of-Experts mechanism with Mamba-Transformer hybrid backbone.

Result: Extensive experiments on FineDance and AIST++ datasets demonstrate state-of-the-art performance both qualitatively and quantitatively, with significant dance quality and strong genre controllability.

Conclusion: MEGADance effectively addresses the challenge of genre conditioning in music-driven 3D dance generation by decoupling choreographic consistency, leading to improved synchronization and genre continuity.

Abstract: Music-driven 3D dance generation has attracted increasing attention in recent years, with promising applications in choreography, virtual reality, and creative content creation. Previous research has generated promising realistic dance movement from audio signals. However, traditional methods underutilize genre conditioning, often treating it as auxiliary modifiers rather than core semantic drivers. This oversight compromises music-motion synchronization and disrupts dance genre continuity, particularly during complex rhythmic transitions, thereby leading to visually unsatisfactory effects. To address the challenge, we propose MEGADance, a novel architecture for music-driven 3D dance generation. By decoupling choreographic consistency into dance generality and genre specificity, MEGADance demonstrates significant dance quality and strong genre controllability. It consists of two stages: (1) High-Fidelity Dance Quantization Stage (HFDQ), which encodes dance motions into a latent representation by Finite Scalar Quantization (FSQ) and reconstructs them with kinematic-dynamic constraints, and (2) Genre-Aware Dance Generation Stage (GADG), which maps music into the latent representation by synergistic utilization of Mixture-of-Experts (MoE) mechanism with Mamba-Transformer hybrid backbone. Extensive experiments on the FineDance and AIST++ dataset demonstrate the state-of-the-art performance of MEGADance both qualitatively and quantitatively. Code is available at https://github.com/XulongT/MEGADance.

Method: Depth-Structured Music Recurrence (DSMR) extends context beyond fixed-length excerpts via segment-level recurrence with detached cross-segment states, featuring a layer-wise memory-horizon schedule that budgets recurrent KV states across depth. It uses a two-scale schedule allocating long history windows to lower layers and uniform short windows to remaining layers.

Result: Experiments on MAESTRO piano performance dataset show DSMR provides practical quality-efficiency trade-off for full-length long-context symbolic music modeling with recurrent attention under limited computational resources.

Conclusion: DSMR offers an effective recurrent attention architecture for long-context symbolic music generation that balances quality and efficiency for deployment on resource-constrained devices.

Abstract: Long-context modeling is essential for symbolic music generation, since motif repetition and developmental variation can span thousands of musical events. However, practical composition and performance workflows frequently rely on resource-limited devices (e.g., electronic instruments and portable computers), making heavy memory and attention computation difficult to deploy. We introduce Depth-Structured Music Recurrence (DSMR), a recurrent long-context Transformer for full-piece symbolic music modeling that extends context beyond fixed-length excerpts via segment-level recurrence with detached cross-segment states, featuring a layer-wise memory-horizon schedule that budgets recurrent KV states across depth. DSMR is trained in a single left-to-right pass over each complete composition, akin to how a musician experiences it from beginning to end, while carrying recurrent cross-segment states forward. Within this recurrent framework, we systematically study how depth-wise horizon allocations affect optimization, best-checkpoint perplexity, and efficiency. By allocating different history-window lengths across layers while keeping the total recurrent-state budget fixed, DSMR creates depth-dependent temporal receptive fields within a recurrent attention stack without reducing compute depth. Our main instantiation is a two-scale DSMR schedule that allocates long history windows to lower layers and a uniform short window to the remaining layers. Experiments on the piano performance dataset MAESTRO demonstrate that two-scale DSMR provides a practical quality–efficiency recipe for full-length long-context symbolic music modeling with recurrent attention under limited computational resources.

Method: Proposes SongEcho with Instance-Adaptive Element-wise Linear Modulation (IA-EiLM) for precise temporal melody alignment, Instance-Adaptive Condition Refinement (IACR) for adaptive conditioning, and introduces Suno70k dataset with comprehensive annotations.

Result: Generates superior cover songs compared to existing methods while requiring fewer than 30% of trainable parameters. Demonstrates effectiveness across multiple datasets.

Conclusion: SongEcho effectively addresses cover song generation through improved conditioning mechanisms and introduces a valuable dataset, advancing AI music generation capabilities.

Abstract: Cover songs constitute a vital aspect of musical culture, preserving the core melody of an original composition while reinterpreting it to infuse novel emotional depth and thematic emphasis. Although prior research has explored the reinterpretation of instrumental music through melody-conditioned text-to-music models, the task of cover song generation remains largely unaddressed. In this work, we reformulate our cover song generation as a conditional generation, which simultaneously generates new vocals and accompaniment conditioned on the original vocal melody and text prompts. To this end, we present SongEcho, which leverages Instance-Adaptive Element-wise Linear Modulation (IA-EiLM), a framework that incorporates controllable generation by improving both conditioning injection mechanism and conditional representation. To enhance the conditioning injection mechanism, we extend Feature-wise Linear Modulation (FiLM) to an Element-wise Linear Modulation (EiLM), to facilitate precise temporal alignment in melody control. For conditional representations, we propose Instance-Adaptive Condition Refinement (IACR), which refines conditioning features by interacting with the hidden states of the generative model, yielding instance-adaptive conditioning. Additionally, to address the scarcity of large-scale, open-source full-song datasets, we construct Suno70k, a high-quality AI song dataset enriched with comprehensive annotations. Experimental results across multiple datasets demonstrate that our approach generates superior cover songs compared to existing methods, while requiring fewer than 30% of the trainable parameters. The code, dataset, and demos are available at https://github.com/lsfhuihuiff/SongEcho_ICLR2026.

Method: Two-component architecture: 1) Destylizer removes style attributes while preserving linguistic content, 2) Stylizer (diffusion transformer/DIT) reintroduces target style conditioned on reference speech. Uses text supervision and constrained information bottleneck for robust content-style disentanglement. Fully non-autoregressive design enables real-time streaming.

Result: Achieves state-of-the-art performance in voice style conversion with real-time capability. End-to-end latency of 1 second, making it the first streamable zero-shot voice style conversion system. Demonstrates effective content-style disentanglement and high-quality conversion.

Conclusion: StyleStream successfully addresses the real-time voice style conversion challenge with a novel architecture that combines content-style disentanglement through text supervision and diffusion transformers, enabling practical streaming applications.

Abstract: Voice style conversion aims to transform an input utterance to match a target speaker’s timbre, accent, and emotion, with a central challenge being the disentanglement of linguistic content from style. While prior work has explored this problem, conversion quality remains limited, and real-time voice style conversion has not been addressed. We propose StyleStream, the first streamable zero-shot voice style conversion system that achieves state-of-the-art performance. StyleStream consists of two components: a Destylizer, which removes style attributes while preserving linguistic content, and a Stylizer, a diffusion transformer (DiT) that reintroduces target style conditioned on reference speech. Robust content-style disentanglement is enforced through text supervision and a highly constrained information bottleneck. This design enables a fully non-autoregressive architecture, achieving real-time voice style conversion with an end-to-end latency of 1 second. Samples and real-time demo: https://berkeley-speech-group.github.io/StyleStream/.

Method: Uses a pretrained latent diffusion model as decoder, with a latent encoder that learns compressed audio embeddings. Employs offline quantization to produce both continuous and discrete embeddings. Achieves extremely low frame rates (down to 1Hz, 750x compression) by leveraging generative priors of the diffusion decoder.

Result: Outperforms both continuous and discrete baselines in audio quality, acoustic similarity, and reconstruction metrics despite operating at high compression rates. Achieves convincing and realistic reconstructions at ultra-low bitrates (down to 0.096 kbps).

Conclusion: S-PRESSO demonstrates that leveraging generative priors from latent diffusion models enables extreme audio compression with realistic reconstructions, pushing the boundaries of what’s possible for sound effect compression at ultra-low bitrates.

Abstract: Neural audio compression models have recently achieved extreme compression rates, enabling efficient latent generative modeling. Conversely, latent generative models have been applied to compression, pushing the limits of continuous and discrete approaches. However, existing methods remain constrained to low-resolution audio and degrade substantially at very low bitrates, where audible artifacts are prominent. In this paper, we present S-PRESSO, a 48kHz sound effect compression model that produces both continuous and discrete embeddings at ultra-low bitrates, down to 0.096 kbps, via offline quantization. Our model relies on a pretrained latent diffusion model to decode compressed audio embeddings learned by a latent encoder. Leveraging the generative priors of the diffusion decoder, we achieve extremely low frame rates, down to 1Hz (750x compression rate), producing convincing and realistic reconstructions at the cost of exact fidelity. Despite operating at high compression rates, we demonstrate that S-PRESSO outperforms both continuous and discrete baselines in audio quality, acoustic similarity and reconstruction metrics.

Method: Proposes AeroGPT framework with Vibration Signal Alignment (VSA) to adapt general audio knowledge to domain-specific vibration patterns, and Generative Fault Classification (GFC) to directly generate interpretable fault labels without post-processing.

Result: Achieves 98.94% accuracy on DIRG dataset and 100% accuracy on HIT bearing dataset, outperforming representative deep learning approaches, with demonstrated potential for interactive diagnosis and real-world deployment.

Conclusion: AeroGPT successfully demonstrates the promise of large-scale audio models for advancing fault diagnosis in aerospace applications through interpretable, actionable diagnosis without post-processing.

Abstract: Aerospace engines, as critical components in aviation and aerospace industries, require continuous and accurate fault diagnosis to ensure operational safety and prevent catastrophic failures. While deep learning techniques have been extensively studied in this context, they typically output logits or confidence scores, necessitating post-processing to obtain actionable insights. Furthermore, the potential of large-scale audio models for this task remains largely untapped. To address these limitations, this paper proposes AeroGPT, a novel framework that transfers knowledge from the general audio domain to aero-engine bearing fault diagnosis. AeroGPT leverages a large-scale audio model and incorporates Vibration Signal Alignment (VSA) to adapt general audio knowledge to domain-specific vibration patterns, along with Generative Fault Classification (GFC) to directly generate interpretable fault labels. This approach eliminates the need for label post-processing and supports interactive, interpretable, and actionable fault diagnosis, thereby enhancing industrial applicability. Through comprehensive experimental validation on two aero-engine bearing datasets, AeroGPT achieves 98.94% accuracy on the DIRG dataset and 100% accuracy on the HIT bearing dataset, outperforming representative deep learning approaches. Qualitative analysis and further discussion also demonstrate its potential for interactive diagnosis and real-world deployment, highlighting the promise of large-scale audio models to advance fault diagnosis in aerospace applications.

Method: Decomposes time series into trend and seasonal components, then handles them separately: uses reversible instance normalization for trend components, but directly applies backbone models without any normalization for seasonal components. Introduces dual-MLP models as computationally efficient solutions.

Result: Achieves around 10% MSE average reduction across four state-of-the-art baselines on benchmark datasets. Shows significant improvements on hydrological datasets from USGS river stations while maintaining linear time complexity.

Conclusion: The decomposition-based approach with component-specific handling strategies effectively improves time series forecasting accuracy and computational efficiency, demonstrating real-world applicability.

Abstract: Time series forecasting presents significant challenges in real-world applications across various domains. Building upon the decomposition of the time series, we enhance the architecture of machine learning models for better multivariate time series forecasting. To achieve this, we focus on the trend and seasonal components individually and investigate solutions to predict them with less errors. Recognizing that reversible instance normalization is effective only for the trend component, we take a different approach with the seasonal component by directly applying backbone models without any normalization or scaling procedures. Through these strategies, we successfully reduce error values of the existing state-of-the-art models and finally introduce dual-MLP models as more computationally efficient solutions. Furthermore, our approach consistently yields positive results with around 10% MSE average reduction across four state-of-the-art baselines on the benchmark datasets. We also evaluate our approach on a hydrological dataset extracted from the United States Geological Survey (USGS) river stations, where our models achieve significant improvements while maintaining linear time complexity, demonstrating real-world effectiveness. The source code is available at https://github.com/Sanjeev97/Time-Series-Decomposition

Method: Proposes a unified SciML framework with three components: (1) Foundation PBPK Transformers treating pharmacokinetic forecasting as sequence modeling, (2) Physiologically Constrained Diffusion Models using physics-informed loss to generate biologically compliant virtual patient populations, and (3) Neural Allometry combining Graph Neural Networks with Neural ODEs to learn continuous cross-species scaling laws.

Result: Experiments on synthetic datasets show the framework reduces physiological violation rates from 2.00% to 0.50% under constraints while offering faster simulation capabilities.

Conclusion: The proposed SciML framework successfully bridges mechanistic rigor with data-driven flexibility, addressing key limitations in PBPK modeling and providing a path toward more efficient and accurate drug development simulations.

Abstract: Physiologically Based Pharmacokinetic (PBPK) modeling is a cornerstone of model-informed drug development (MIDD), providing a mechanistic framework to predict drug absorption, distribution, metabolism, and excretion (ADME). Despite its utility, adoption is hindered by high computational costs for large-scale simulations, difficulty in parameter identification for complex biological systems, and uncertainty in interspecies extrapolation. In this work, we propose a unified Scientific Machine Learning (SciML) framework that bridges mechanistic rigor and data-driven flexibility. We introduce three contributions: (1) Foundation PBPK Transformers, which treat pharmacokinetic forecasting as a sequence modeling task; (2) Physiologically Constrained Diffusion Models (PCDM), a generative approach that uses a physics-informed loss to synthesize biologically compliant virtual patient populations; and (3) Neural Allometry, a hybrid architecture combining Graph Neural Networks (GNNs) with Neural ODEs to learn continuous cross-species scaling laws. Experiments on synthetic datasets show that the framework reduces physiological violation rates from 2.00% to 0.50% under constraints while offering a path to faster simulation.

Method: Proposes CoTAR (Core Token Aggregation-Redistribution) module that introduces a global core token as proxy for inter-token interactions, enabling centralized aggregation and redistribution strategy instead of direct token interactions in standard attention.

Result: Achieves up to 12.13% improvement on APAVA dataset while using only 33% memory and 20% inference time compared to previous state-of-the-art; validated on five benchmarks.

Conclusion: CoTAR better aligns with centralized nature of MedTS signals, reduces computational complexity from quadratic to linear, and shows superior effectiveness and efficiency for medical time series analysis.

Abstract: Accurate analysis of medical time series (MedTS) data, such as electroencephalography (EEG) and electrocardiography (ECG), plays a pivotal role in healthcare applications, including the diagnosis of brain and heart diseases. MedTS data typically exhibit two critical patterns: temporal dependencies within individual channels and channel dependencies across multiple channels. While recent advances in deep learning have leveraged Transformer-based models to effectively capture temporal dependencies, they often struggle with modeling channel dependencies. This limitation stems from a structural mismatch: MedTS signals are inherently centralized, whereas the Transformer’s attention mechanism is decentralized, making it less effective at capturing global synchronization and unified waveform patterns. To address this mismatch, we propose CoTAR (Core Token Aggregation-Redistribution), a centralized MLP-based module designed to replace decentralized attention. Instead of allowing all tokens to interact directly, as in standard attention, CoTAR introduces a global core token that serves as a proxy to facilitate inter-token interactions, thereby enforcing a centralized aggregation and redistribution strategy. This design not only better aligns with the centralized nature of MedTS signals but also reduces computational complexity from quadratic to linear. Experiments on five benchmarks validate the superiority of our method in both effectiveness and efficiency, achieving up to a 12.13% improvement on the APAVA dataset, while using only 33% of the memory and 20% of the inference time compared to the previous state of the art. Code and all training scripts are available at https://github.com/Levi-Ackman/TeCh.

Method: Adapt Support Vector Data Description (SVDD) and Deep SVDD as one-class learning methods for CFAR detection, proposing two novel SVDD-based detection algorithms.

Result: Demonstrated effectiveness on simulated radar data, showing improved performance in challenging clutter environments.

Conclusion: SVDD-based approaches offer promising alternatives to traditional covariance-based detectors for radar target detection in complex clutter scenarios.

Abstract: Classical radar detection techniques rely on adaptive detectors that estimate the noise covariance matrix from target-free secondary data. While effective in Gaussian environments, these methods degrade in the presence of clutter, which is better modeled by heavy-tailed distributions such as the Complex Elliptically Symmetric (CES) and Compound-Gaussian (CGD) families. Robust covariance estimators like M-estimators or Tyler’s estimator address this issue, but still struggle when thermal noise combines with clutter. To overcome these challenges, we investigate the use of Support Vector Data Description (SVDD) and its deep extension, Deep SVDD, for target detection. These one-class learning methods avoid direct noise covariance estimation and are adapted here as CFAR detectors. We propose two novel SVDD-based detection algorithms and demonstrate their effectiveness on simulated radar data.

Method: UMA uses end-to-end reinforcement learning with a single policy for both memory operations and question answering. It maintains dual memory: compact core summary for global context and structured Memory Bank supporting explicit CRUD operations over key-value entries for proactive consolidation during streaming.

Result: UMA substantially outperforms long-context and RAG baselines on 13 datasets spanning Ledger-QA (diagnostic benchmark for continuous state tracking), Test-Time Learning, and Accurate Retrieval tasks, while remaining competitive on standard retrieval benchmarks.

Conclusion: The framework demonstrates the importance of learned, end-to-end memory management for dynamic reasoning and learning tasks in long-horizon scenarios, showing superiority over passive processing approaches.

Abstract: Long-context LLMs and Retrieval-Augmented Generation (RAG) systems process information passively, deferring state tracking, contradiction resolution, and evidence aggregation to query time, which becomes brittle under ultra long streams with frequent updates. We propose the Unified Memory Agent (UMA), an end-to-end reinforcement learning framework that unifies memory operations and question answering within a single policy. UMA maintains a dual memory representation: a compact core summary for global context and a structured Memory Bank that supports explicit CRUD (create, update, delete, reorganize) over key value entries, enabling proactive consolidation during streaming. To evaluate long-horizon memory behavior, we introduce Ledger-QA, a diagnostic benchmark for continuous state tracking where answers are latent values derived from accumulated updates rather than lo cal span retrieval. Across 13 datasets spanning Ledger-QA, Test-Time Learning, and Accurate Retrieval, UMA substantially outperforms long-context and RAG baselines on dynamic reasoning and learning tasks while remaining competitive on standard retrieval benchmarks, underscoring the importance of learned, end-to-end memory management.

Method: Proposes weak-form evolutionary KANs that decouple linear system size from training samples through weak formulation. Uses boundary-constrained KANs to enforce Dirichlet/periodic conditions and incorporates derivative boundary conditions directly into weak formulation for Neumann conditions.

Result: The framework provides improved scalability compared to strong-form approaches and stable solution prediction for PDEs through rigorous boundary condition enforcement.

Conclusion: Weak-form evolutionary KANs offer a stable and scalable approach for PDE solving, contributing to scientific machine learning with potential engineering applications.

Abstract: Partial differential equations (PDEs) form a central component of scientific computing. Among recent advances in deep learning, evolutionary neural networks have been developed to successively capture the temporal dynamics of time-dependent PDEs via parameter evolution. The parameter updates are obtained by solving a linear system derived from the governing equation residuals at each time step. However, strong-form evolutionary approaches can yield ill-conditioned linear systems due to pointwise residual discretization, and their computational cost scales unfavorably with the number of training samples. To address these limitations, we propose a weak-form evolutionary Kolmogorov-Arnold Network (KAN) for the scalable and accurate prediction of PDE solutions. We decouple the linear system size from the number of training samples through the weak formulation, leading to improved scalability compared to strong-form approaches. We also rigorously enforce boundary conditions by constructing the trial space with boundary-constrained KANs to satisfy Dirichlet and periodic conditions, and by incorporating derivative boundary conditions directly into the weak formulation for Neumann conditions. In conclusion, the proposed weak-form evolutionary KAN framework provides a stable and scalable approach for PDEs and contributes to scientific machine learning with potential relevance to future engineering applications.

Method: Three-part system: (1) Daily probability sampling from impression stream using ML-assisted weights to concentrate label budget on high-exposure/high-risk content while preserving unbiasedness, (2) Multimodal LLM labeling governed by policy prompts and gold-set validation, (3) Design-consistent prevalence estimation with confidence intervals and dashboard drilldowns using post-stratified estimation for multiple segment analysis.

Result: System enables accurate prevalence measurement with statistical confidence intervals, supports analysis by surface, viewer geography, content age, and other segments through a single global sample, and provides configurable workflow across different content policies.

Conclusion: The design-based measurement system provides content safety teams with statistically rigorous, frequent, and representative prevalence metrics using multimodal LLMs for efficient labeling, enabling better understanding of actual user experiences with policy-violating content.

Abstract: Content safety teams need metrics that reflect what users actually experience, not only what is reported. We study prevalence: the fraction of user views (impressions) that went to content violating a given policy on a given day. Accurate prevalence measurement is challenging because violations are often rare and human labeling is costly, making frequent, platform-representative studies slow. We present a design-based measurement system that (i) draws daily probability samples from the impression stream using ML-assisted weights to concentrate label budget on high-exposure and high-risk content while preserving unbiasedness, (ii) labels sampled items with a multimodal LLM governed by policy prompts and gold-set validation, and (iii) produces design-consistent prevalence estimates with confidence intervals and dashboard drilldowns. A key design goal is one global sample with many pivots: the same daily sample supports prevalence by surface, viewer geography, content age, and other segments through post-stratified estimation. We describe the statistical estimators, variance and confidence interval construction, label-quality monitoring, and an engineering workflow that makes the system configurable across policies.